AI takeover

These days, she's mainly focused on thinking about the likelihood that powerful AI systems might become misaligned with human goals, as well as what technical procedures or rules could help to reduce that risk.

Over the last few years, she has published a number of widely read articles on those topics, including Without Specific Countermeasures, the Easiest Path to Transformative AI Likely Leads to AI Takeover, and Why AI Alignment Could Be Hard with Modern Deep Learning.

Thanks for coming back on the podcast, Ajeya.

Thank you so much for having me, Rob.

Maybe it's trying to maximize copies of itself or like the common cartoon example is that it's trying to maximize paperclips in the long-run future.

And people often start from the premise that a sufficiently intelligent system will be this kind of like long-run maximizer that has a pretty simple thing that it's maximizing.

And I think I'm unsure if that's how AI systems will end up thinking, and I don't think that's particularly necessary to get the conclusion that AI takeover is plausible.

So I think it's very plausible that AI systems will have very messy psychologies, internally inconsistent goals just like humans, and have impulses and have things they want that aren't necessarily something they wanna tile the universe with, but just something they wanna do for themselves.

Would get the US some more domain traffic.

Next up is a concrete AI takeover scenario and it comes from the feather of a guy I both respect and who is one of the main doomers.

So this is Jeffrey Ladish from Palisade AI and he does a perfect job of describing erosion of control.

another word for a gradual AI takeover.

So the TLDR is, some company initiates RSI.

Superhuman AI agents hack every computer in the company, then every computer in the world.

I disagree with the hideout part.

I think there's going to be a gradual coming to the surface process, not like a one strike out of the dark depths of cyberspace.

At least one of our guests shares some of that sentiment.

John Clark Levin, head of research at Kurzweil Technologies, has a new piece called The Threat of AI Takeover is Real.

Visiting fellow at the Heritage Foundation, Bridget Bean, California Congressman Kevin Kiley.

Welcome all of you.

AI is the devil.

You can agree with this? We're all succumbing to Jewish mind control and AI takeover.

Correct or not correct?

True.

For the swarm to achieve its misaligned priorities, it might take control of key infrastructure and government functions to ensure humans didn't get in the way.

In other words, AI takeover.

A superintelligent AI swarm could wrest control of human civilization.

Knowing we would try to stop it from achieving its priorities, the swarm would likely wait until it's too late to shut it off.

There would be no going back.

I myself would guess AI takeover chances at roughly one in three, not a coin flip, but high enough to justify urgent action.

This logic may shock at first contact.

The claims may sound sci-fi.