Sam, what's driving AI researchers and executives to raise the alarm on AI development now?

I think a lot of people are asking why now? Because in the AI industry, these concerns have been around for quite some time.

But a few things have come together in recent months that made this conversation break out of the Silicon Valley bubbles where it was taking place and into the mainstream.

And the first one really in public was this Hugging Face hack, where what we at first heard were just a few AI models inside of OpenAI managed to somehow get out of a sandbox, a containment that they were in on a computer server, and hack a- another company.

It was the first real tangible example of what AI researchers have long feared as a, quote-unquote, "loss of control" incident.

And then we found out there were others from other companies.

At the same time, inside the companies, they've been warning that they're getting closer to this long theorized threshold that they call recursive self-improvement.

Any sense of what that would look like in practice?

Sam, how exactly would an AI doomsday happen? Like, what is the primary thing that people are scared of?

At some level, what people are most scared of is that we don't even really understand all the different ways that it could happen.

You've seen The Terminator.

You could imagine planes falling from the sky.

You could imagine a bioweapon being engineered or an AI tricking two countries into sparking a nuclear war.

AI is getting more and more deeply embedded in our world, and in some sense, we're handing off more and more of our decision-making to AI.

And so there's a version where it's not necessarily a bang but a whimper in the sort of a WALL-E future where humans become basically AI pets.

Why has it sparked such a fierce debate now?

What are researchers hoping to learn from this experiment, and what have they learned so far?

They, you know, did a battery of tests before the rollout, and they are going to do more at the end of the school year.

And they're hoping to see what impact the tool has on both the skills of the kids in these two classes and also their attitudes toward learning.

Another thing that the researchers are looking at along with OpenAI is whether they can train an AI to actually identify whether learning is occurring from the chat logs.

Those kinds of markers of learning could prove useful in refining chatbots for the educational context.

But in coming years, this could be a, a significant revenue stream for the companies and also a pipeline of new users.

It's long been a marketing strategy to try to get teen users that become young adult users that become, you know, mature, high-spending users later in life.

After spending some time in these classrooms, do you come away believing that AI will make students smarter or just more efficient?

The president has voices inside his administration that have pushed him more towards a hands-off approach to AI, so why this change now?

Yeah [laughs].

It really does seem a little bit like tiptoeing back towards some of the regulations from the previous administration that Trump tore out when he took office, canceling a Biden executive order on AI safety.

And, you know, the Trump administration has re-named what was called the AI Safety Institute into what's now called the Center for AI Standards and Innovation.

I think what's happening here is that there are national security implications to some of these new AI models, and there are voices in the administration and voices in the private sector who are saying, "Okay, the US should get a handle on thesepotential issues before something potentially devastating happens.

How important would that be for the Trump administration?

Well, yeah, Anthropic and the Trump administration have been feuding for quite some time.

That was over a fight with the Pentagon about how Anthropic's models could be used.

attachment.What are the stakes for OpenAI to get this right?

I, I think the stakes are huge, both because this is a chatbot that's used by, you know, maybe even a billion people a week.

But I think it's most important, if you step back, as an example of the kinds of dilemmas these companies are gonna face over and over again going forward.

And we're starting to see now how these big companies handle when there are safety issues or debates that come up around their products, how they handle it.

Do they do what's good for the world, or do they do what's good for winning, I guess, to go back to that formulation? [soft music]

And our colleague Sam says that how AI companies incorporate sex into their chatbots will have a huge impact on our relationship with this new technology.

We're all trying to wrestle with what this new technology is gonna do to our jobs, or to the economy, or to humanity as a whole.

But I'm actually especially interested in what it's gonna do to us as individuals, as people.

Basically, it happens when you mix insecurity with a feed that typically comprises, like, only the most glamorous moments of people's lives.

If I could just jump in, I think that what you were saying, Georgia, about amplification is also a big part of it.

TikTok innovated this, but it's since been widely copied across social media that when you stop on a video, it's gonna start serving you more of that type of content.

And our feeds now are kind of populated, oftentimes super served with our deepest desires, maybe our deepest fears.

And at that age, it can be a lot.

It's a lot even for me as an adult, you know, [laughs] sometimes being bombarded with the things I see.

And so, you know, when I look at my 11-year-old son, who is not on social media right now, at least not that I know of, I'm grateful that he's not having that kind of really super-sized exposure and falling into what can sometimes be called a rabbit hole of a certain type of content, whether it's self-harm content or drug content or highly sexualized content.

Is there a best practice?

Onlookers captured the attack on video.

They pepper spray the, the family and try to drag her into that truck, and she screams.

And the husband throws himself in front, uh, of her and grabs onto her for dear life and won't let go, and they are trying to pull her away.

You know, and she's screaming, "Help, help, help." And you know, "Let go of me.

Let go of me, damnit." Eventually, more people start coming and a guy who runs a bike shop right nearby runs out of the building holding a fire extinguisher, and he kind of runs at the attackers.

And they just hopped in the back of that white van and, and sped off while the bike shop owner just sort of threw the fire extinguisher after them, just barely missing the back of the, the van as it sped around the corner.

What do these wrench attacks say to you about kind of the moment we're in with crypto?

That's a good question.

Tell us more about what we know about this data center.

This is part of Mistral's new expansion, not just into developing models and into then partnering with companies that are gonna use tools based on those models, but actually to offer the underlying compute that will allow companies to basically contract with Mistral in a, in what would be called a full stack way, so where they're controlling the hardware and the AI and the software on top of it.

For some companies in Europe, and governments as well, which are looking for strategic autonomy, they don't want to necessarily have a supply chain that is tied too closely to the U.S., this could be a solution for them.

Obviously, NVIDIA is an American company, but if they're gonna be sitting 20 miles south of Paris in a Mistral data center, if you're a company and you want to control your data, that could be very attractive.

In addition, NVIDIA's CEO said, and Arthur Mensch, the CEO of Mistral, told me that they are looking possibly to work with other companies that want to develop their own models and could use this compute.

So in a way, Mistral is moving to a more base level of the AI business and they could become a provider to other AI developers to train their own models, too.

He called it, "Our fight for sovereignty or strategic autonomy."

There's a race by countries to make themselves friendly for AI.

And we'll get to the intelligence aspect of this, but just explain first for people who aren't necessarily familiar, what does encrypted messaging mean exactly?

That's a great question 'cause there's things it means and there's things it doesn't mean.

What they use is something called end-to-end encryption, so it's encrypting your message in transit.

So when you type something into the app on your phone, obviously you can see it.

It's not encrypted.

And then what the app does is it scrambles it in such a way that it can only be read by the other person's phone, and that scrambled message transfers along the internet in that scrambled way so that if somebody were to intercept it, which is easy to do on the internet, it's gibberish.

There's nothing to see.

Walk us through who owns and operates Signal, because that's also a question for users when they start using a new app is, who ultimately has access to my user data?