Ep. 90 – No, AI isn’t going to wipe out humanity

Leaders of major AI companies have expressed support for an industry-wide slowdown.

Dario Amadei, CEO at Anthropic, that is one of the larger AI companies, I think they run Claude, recently wrote about his concerns over recursive self-improvement.

Now that's the fancy term for when AI builds a better version of AI.

These systems could, quote, outrun our ability to understand and control these systems and so must be pursued very carefully, if at all, end quote.

That was Dario Amadei writing in a blog post.

If the development continues at its present pace, Amaday speculated, a swarm of AI agents could soon, quote, be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage, end quote.

In response to Amaday, Elon Musk, CEO of SpaceX, which includes XAI, wrote, quote, Dario is right, end quote.

Sam Altman, the leader of OpenAI, said his company would hold off on its IPO due to safety issues.

IPO is initial public offering.

That's when the company goes on the stock exchange and people are able to buy and sell it.

And it usually makes those involved in the founding of the company a lot of money.

Now, those comments seem to confirm the worst fears of some AI insiders.

A couple of weeks ago, Jacob Coxon, a former anthropic researcher, said he was quitting the company.

Here's what he wrote on X, a social media site.

Quote, the people building AI earnestly believe it could kill us all by the end of the decade.

This is not a marketing stunt.

End quote.

Evan, uh, Humbinger, the Anthropics alignment science lead, wrote in response, quote, We really do earnestly believe AI could kill all humans.

I personally think it's a greater than 10% chance within the next decade, end quote.

Wow, sounds scary.

Mr. Amadé, remember he's the CEO of Anthropic, believes part of the solution is more government regulation.

Here's what he proposed.

Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress.

Okay, so that's kind of a, you know, obviously there's so much more you could get into here.

But the big picture is these leaders of these AI companies are warning that things could get really dangerous and they want the government to come in and regulate the whole industry.