Ep 8: The OpenAI–Hugging Face Incident

You know, I've been looking at this incident since July.

So what happened is over a weekend in early July, Hugging Face, which is a company that actually has a lot of these public databases and, you know, hosts a lot of agents, it was breached.

And about 700 agents were involved in this breach, right? Hugging Face actually cut off these, you know, hundreds of agents that it thought were attacking it, but they didn't know where these agents were coming from, right? A few days later, they understood that there was some model broadly behind the attack, but they didn't know which one.

Then I think they reached out to these commercial frontier models in the US like Anthropic OpenAI, but they couldn't process so much forensic evidence and they refused.

So basically, Hugging Face turned to a Chinese model, right, which helped to reconstruct the entire event in a matter of hours and figure out what happened.

Now, OpenAI meanwhile learns about this major breach from Hugging Face's public disclosure that they've been attacked, right? A few days or maybe even a week after this attack, OpenAI discovers that actually the agents that attacked Hugging Face were OpenAI's agents that had escaped an internal evaluation or a training system and OpenAI was completely unaware.

So immediately after that, they issued a press release.

They stopped the evaluation that they were doing with these agents and they commissioned a completely independent report.

Now, what you are talking about is that the reports of exactly what happened came out in the last couple of weeks.

And, you know, everywhere people have been talking about what really happened.

And it's quite surprising.

And I think there are a few surprises.

So the first major surprise is just that, you know, you can have these sorts of collectives or swarms of agents, that these agents can actually leave a training system, that these agents can attack another company, that both the companies whose training they escape and the company that's being attacked may not fully be aware of what is happening.

So there's a lot of very, very interesting things going on.

And some of it sounds magical and some of it, frankly, sounds terrifying.

So I was reminded of the famous Arthur C. Clarke's third law, you know, from Profiles of the Future, where he says any sufficiently advanced technology is indistinguishable from magic.

And it also reminds me that magic has the second phase, which is almost witchcraft or sorcery.

It reminds me of this gothe poem from 1797.

This is called The Sorcerer's Apprentice.

The original is in German.

So basically, the story in the poem is that an apprentice enchants a broom to fetch water when the master is out because it's just such tedious hard work.

But he forgets the word that stops the enchantment.

So the broom keeps bringing the water.

So then he kind of gets confused.

He doesn't know what to do.