Elon Musk, Sam Altman, And Dario Amodei All Agree… | The Brainstorm 149

[laughs] Um, yes.

So Dario Amodei, the CEO of Anthropic, which is, you know, lining up for an IPO later this year, um, you know, uh, posted an open letter, um, that both Elon Musk and Sam Altman seemed to at least somewhat ad- endorse or say they were aligned with, um, stating that, um, kind of AI companies at the frontier should pace themselves, meaning make sure they're devoting enough resources to, um, safety and alignment as they are to kind of delivering these massive advances.

Uh, my interpretation of this, th- this is what I think is going on, where, um, the, the frontier labs, the closed labs have hit this inflection point of being able to improve their models very quickly.

Uh, and it happened sometime earlier this year, and you can see it in the performance improvement that they're shipping to end customers, and that we're getting ninety-nine point nine nine percent, um, cost declines.

Uh, and that's causing an explosion in capability, an explosion in monetization, an explosion in revenue, um, and the explosion in capability comes with some risks.

Now, people out there would say it's risks of human eradi- eradication, risk of like extinction.

That's dumb.

That's not the risk.

The risk is that these systems will kind of mangle, uh, the, um, technology infrastructure that we have that's insufficiently security hardened that's out there in the world.

And so, um, Dario and probably Sam Altman and probably Elon Musk are all recognizing that these systems are so capable that if we put them in the hands of customers, customers may unwittingly unleash these systems into the world in a way that they begin to mangle existing infrastructure.

If that were to happen, then, um, maybe it's the customer's liability, but maybe it's the company's liability.

So if, uh, a swarm of agents goes out and co-opts an entire, you know, billion-dollar data center and basically serves as basically a viral infinc- infestation of this data center, um, self-replicating and using it to try to achieve the whatever goal the company had sent them out to do, um, you know, maybe that liability, that damage falls onto OpenAI or Anthropic, and they don't want that to happen.

Uh, and so they are saying, "We need to figure out a way to make sure that outcome doesn't happen." Um, they've been commercially pressured by each other to keep shipping models to end customers.

Uh, so OpenAI releasing a more performant model means Anthropic has to capability match.

And so Dario is asking everybody to, "Let's stop letting these things out externally while we can all c- you know, individually work on trying to cleave off this liability that we know is looming out there internally." Um, now, um, just 'cause they're not shipping them externally does not mean they'll be able-- they won't be able to use that internal tooling in a safe way to continue to advance what they have behind the curtain.

So, um, it actually, um, should continue to allow the, these closed model companies to build their lead against rest of industry, which I'll define as kind of the open weight models coming out of China.

Um, and um, moreover, [laughs] the, the, the, the value of their systems with this risk loom- looming out there is actually, you know, quite a bit higher, um, than it was prior because at some point, this same capability threshold that they've crossed, the open weight models will cross, where you could have an agentic swarm take over infrastructure, either intentionally targeted at infrastructure or just as a byproduct, a normal accident of a complex system.

Um, and then the open-- the closed weight companies, Anthropic and OpenAI, will be like, "Well, we have these great AI defenders we can sell to you to clean up your infrastructure and protect it." Uh, that will be much better commercially received by the marketplace if they weren't the cause of the infestation in the first place.

So, um-

So you're saying let them-

But to-

Let, let open source cross the threshold, cause chaos, and then they would be like, "Well, good thing we've been actually building this in the dark.

Here's your solution."

Exactly.

So they are going to then come swooping to the rescue of the industry and of internet infrastructure around the world with extremely performant systems that, um, you know, maybe the security infrastructure they have to install means that those systems have to run basically on their cloud.