AI’s “Existential Risk” is Not What You Think

About a month ago, Jacob Coxen sent shockwaves through the world with his dramatic proclamation that AI could wipe out humanity in the next ten years.

Recursive self-improvement will happen, and we'll enter the phase of Evans Post, where he argues that there's a chance we could all die.

Like, that, that is coming soon.

He said, quote, "The people building AI earnestly believe that it could kill us all by the end of the decade.

This is not a marketing stunt." So if you've been paying attention to the AI world for the past five years, this is a familiar narrative.

There's even a cute little name for the likelihood that one believes AI will end in catastrophe: p-doom, or the probability of doom.

Dario Amadei said his p-doom was twenty-five percent.

Elon Musk says there's, quote, "Only a twenty percent chance of annihilation." Sam Altman famously said that AI will probably wipe out humanity, but in the meantime, uh, there will be great companies.

I don't exactly know why Coxen's statement was the one to break through to the mainstream, but alas, here we are.

And unfortunately, far too many in the media are taking his claims at face value.

And I admit that I do believe him when he says that the people building AI believe that they are building a machine god that might kill us all to make paperclips, or whatever baseless theory they present on a given day.

I believe that they believe the grand narratives they've been told about AI.

I believe they think of themselves as humanity's saviors.

Does that mean that we should believe them? Well, obviously no.

[laughs] However, that is certainly not to say that there is no threat from what the industry has taken to calling rogue AI.

A misnomer, by the way, that should be renamed negligent deployment of unsafe technology, but I'll save that for another video.

But in addition to this, AI poses far less sexy harms that I have been discussing on my channel for well over a year at this point.

Labor exploitation and the enshittification of work, techno-nationalism and the reckless integration of insecure and unpredictable systems into every level of the military, the supercharging of misogyny and the predatory behaviors enabled by luxury surveillance devices, a nationwide network of ALPRs, doorbell cameras, and systems capable of parsing the unfathomable amounts of data they generate to exert control over ever-expanding segments of the population through mass surveillance, the deterioration of community and the exploitation of loneliness, techno-fascism and the AI-enabled targeting of dissent, the ecological catastrophe of hyperscale data center.

The list truly goes on and on.

The people building these technologies are shockingly quiet about the myriad harms they are inflicting on the world right now, and yet we are supposed to believe that they have humanity's interests at heart, that they are capable of aligning AI to human values.

Instead of holding tech companies accountable for the literal felonies they are committing via their technology, the doom narrative distracts attention from the guilty parties.

By insisting that AI has agency and could even be on the verge of consciousness, tech oligarchs are offloading accountability onto the machines.

All the while, their stock prices continue to explode.

In SpaceX's record-breaking IPO, they claimed the total addressable market from AI was twenty-six point five trillion dollars.

That's with a T, meaning that they see their AI business growing to nearly the size of the entire US economy.