Post #3998721

The more I think about it, less sure I am that OpenAI’s breach of Hugging Face is a marketing stunt. Initially, my thoughts was that they might be trying to best Anthropic at their own game, but honestly I’m not convinced that’s the case. What are the upsides of them disclosing this? Hype generation? Yeah you can say you have this crazy model but the blog post does implicitly admit some pretty significant mistakes and oversights. OpenAI says the model broke containment by dropping a 0 day against their package cache and proxy system. Okay so that’s a cool demonstration of what it could do, sure. Then OpenAI talks about the model beginning to escalate privileges and move laterally across their own org in an attempt to reach an internet connected box for answers for their security eval test range, and in doing so caused an incident internally. Once the model did hit the internet, it IDd hugging face as a place that likely has information on how to solve OpenAI’s security assessment challenges. It then broke into Hugging Face’s infrastructure to get access to that data. I don’t think even the best marketing people could spin this into a good thing. When you dissect the press release of Hugging Face last week, and OpenAI today, there’s not really any positive upside on this for OpenAI, especially since their own models couldn’t be used to stop this incident while it was ongoing. Hugging Face had to turn to a Chinese model to make it happen. I really want to see some attribution for the 0 day they claim was used to break out of the eval infrastructure. I’d love them to also publish the exploit once it’s patched. Assuming models continue to improve over time, this might not be the last time we see an incident like this. I really want to see Hugging Face publish how they managed this incident in more detail. This is probably the most fscinating breach I’ve seen so far.