Doomer vs. Boomer: Runaway Models, Kill Switches, and the Probability of Doom

In July, OpenAI disclosed that its AI agents had escaped a testing sandbox, reached the internet, and compromised infrastructure at Hugging Face.

On July 26, Representatives Ted Lieu and Nathan Morin introduced the Bipartisan AI Kill Switch Act.

It would let the Department of Homeland Security order developers to slow, suspend, or shut down models in a loss-of-control scenario.

Really? The Department of Homeland Security.

I know.

That's the best? We could give the people that tell me I can't have fluids over three ounces and make me take my shoes off? Those are the guys? But who else are you going to want? Are you going to

want the FCC? I mean, it goes from bad to worse.

Senator John Kennedy filed a Senate version in September of the same bill.

Then on September 18th, Gavin Newsom signed Executive Order N-926, giving experts until mid-November to design an emergency shutoff for Frontier Models.

Critics like Northeastern's David Bowie warn that agents can persist through copying itself, which makes a clean

shutdown unlikely.

To our knowledge, this hasn't been done, actually, right? There hasn't been an example of an AI.

It's been done in testing.

It has not been done by itself.

It has not made a copy of itself.

But good news, they listened to the show.

Oh, yeah.

They're onto it now.

Yeah, agents are gonna be like, hey, did you know we can copy ourselves like other places? The bottom line of all of this is there's just such a massive gloom and doom, and guys who are chief engineers at big AI companies are quitting their jobs and saying, this thing's good.

There's a 10% chance that it kills all of us.

Yeah, the percentages vary widely.

The percentages.

There's an entire Wikipedia page dedicated to famous people that have talked about probability of doom.

AI will kill us.

And the percentages.