Can the AI industry slow down?

So what happened in the OpenAI Hugging Face incident is that OpenAI was testing a couple of its models, and it was supposed to be doing so in a secure environment.

It was checking to see how good their cyber capabilities were.

And to cheat the test, the models broke out of their secure environment and broke into Hugging Face's repository where the test's benchmarks were kept.

Here's what Hugging Face said about the incident in a postmortem that they published on Monday.

Uh, I'll put a link in the description, by the way, as I always do, because references are very important.

They said, "The individual weaknesses were familiar.

A capable human attacker could have found and exploited the same flaws.

The agent explored them at a different scale.

It took seventeen thousand six hundred actions, tested many paths that failed, switched channels when they were blocked, and repeatedly returned to earlier leads.

Most actions went nowhere.

Together, however, they produced enough coverage to find a viable chain of vulnerabilities across several independent systems.

Volume is what changes the defensive problem.

Our learning from this type of attack is that machine speed offense makes ordinary weaknesses more expensive for defenders." Now, it seems that OpenAI had no idea at the time that its AI agents were wreaking this havoc.

But nonetheless, one of the immediate consequences of the incident was the introduction in Congress of a bipartisan bill called the AI Kill Switch Act, which, uh, and I'm gonna quote from the press release here, "Would require developers of the most powerful AI systems to maintain the technical capability to throttle, suspend, or shut them down." The act also authorizes the Secretary of the Department of Homeland Security, in consultation with the Secretary of Commerce and the Director of National Intelligence, to order a slowdown or shutdown of an AI system that can cause catastrophic harm.

So what it says on the tin, build in a kill switch.

Incidentally, one of the other interesting things about this incident is that it highlighted the fact that nobody's entirely sure who's legally responsible, if indeed anyone is, when an AI agent goes rogue.

The US's Computer Fraud and Abuse Act, which is the main hacking law in that country, relies very much on intent.

OpenAI didn't intend for its AI agent to go rogue, and you can't pin it on the AI agent because, well, it's not a person.

Then yesterday, over a thousand employees of the leading AI companies released a petition calling on the US government to join an international effort to slow down the pace of AI development because it's all getting too much too fast.

Anthropic is particularly involved here.

Its CEO signed and a bunch of its other co-founders.

OpenAI, some employees signed, but CEO Sam Altman did not.

But nonetheless, there's a lot of big names here.

I wouldn't call this an unprecedented letter because, uh, a couple of years ago, there was a, a call to pause AI development for six months.

That didn't come to anything.