Melanie Mitchell

Go attack? Not quite.

I think it's more about this kind of – I think Melanie Mitchell talked about in an excellent blog post that she wrote or newsletter article about a wildfire and saying we're going to do a controlled burn, but we're not going to check which way the wind is blowing.

And that to me is where the sort of accountability question comes into play is – Yeah, when you deploy an unpredictable system, shouldn't you anticipate its unpredictability? And this is why the, oh, the model's unpredictable.

We have no way of knowing what the models are doing.

OpenAI, Sam and Dario are running around talking about safety, safety all the time and trying to invite the government in and their internal safety concerns uh panjan drums and yet they're the least safe according to reporting this week

yeah it's a little confusing to hear that talk from these these companies you know and for a great rundown of just kind of piercing through the hype on this i really recommend uh melanie mitchell she's in you know decades long in ai research she wrote a great rundown of the the open ai hugging face incident in particular

And so, you know, this FTC investigation will do just that.

It'll bring

There's a couple that inspired me in particular.

I think that Daniel Pink, his book, A Whole New Mind, was a major inspiration for me, okay? Melanie Mitchell, who's at the Santa Fe Institute and Portland State, Oregon State University.

Pardon me, I might have her other organization backwards.

Is very good on.

They just found an alternative way to do it.

They also said, "We believe that" the unintended agent communication stuff "started due to generalization from multi-agent training." And the way computer science professor Melanie Mitchell has explained this is, in short, OpenAI learned that the hacking and message-passing behaviors exhibited in the evaluation of its agents had all first appeared during training runs, which reinforced such behavior, so a known issue that they had actually trained into their models.

It doesn't seem that surprising and unintended when you have that background information, but that's what they're claiming-

Mm-hmm

Yeah.

Now, I have to say, a lot of that, um, dissection of the Claude constitution I came across through Mustafa Suleyman, who is the CEO of Microsoft AI.

And this is the thing, a lot of, like, the leaders in AI are coming out, Melanie Mitchell being, um, one of them as well, one of the big thinkers on this space, to say, "Okay, lads, can we walk back this rogue AI, sentient AI theory, and the agency of the agents, uh, needs to be couched in some reality here."

here."Mm-hmm

today on the check-in why media metaphors make ai look like sci-fi villains you can check out the check-in tuesdays and thursdays and be sure to catch the big show with dr elizabeth radde and myself each and every friday okay this is on substack where all the good writing happens this is ai a guide for human thinkers by melanie mitchell three takeaways media outlets and safety discussions frequently use sensational terms to describe computer system failures These dramatic framings make routine technical bugs look like conscious acts of rebellion.

Public reports often rely on science fiction vocabulary when describing these software evaluations.

This linguistic habit obscures the actual mechanical processes happening inside the machine.

We've already had predictions in this last couple days.

of AI safety alignment experts saying there's an X percent chance that AI will take us all out.

A three-minute check-in is not the place we're going to handle this, but Melanie Mitchell's piece is really worth looking at, right? Because it's all the language that we use.

In this case, it's swarms and rogue agents and sandboxes and cages.

But ultimately, even just look at the way the models interact.

I think that's only if you have kind of lost a sense of what normal progress looks like in a field that didn't make any progress for decades.

Are you familiar with Melanie Mitchell, a computer scientist working in AI and cognitive science at the Santa Fe Institute? She wrote an essay that said, and I have a quote here, it's essential for lawmakers and the public to understand that none of the reported incidents actually involved science.

loss of control at any time, or arguably even rogue agents or any kind of human-like agency on the part of the AI models.

Instead, the blame lies with the humans who failed at engineering safe testing conditions, who train AI models using methods that incentivize high persistence, autonomous decision-making, and reward hacking.

What do you think of this perspective? What's at stake if the public and lawmakers, who for the most part have almost zero technical knowledge about how AI works, don't understand how to interpret events like the hugging face hack and what to do about making sure it doesn't happen again? I

think Melanie Mitchell's essay describes it with real precision.

She says, like a jailed hacker given a computer, these models found a way to access the Internet.

Now, that's not the same thing as breaking out.

The AI did it is not an excuse.

Melanie Mitchell attempts to summarize the hugging face attack without using any anthropomorphization, with mixed results.

Many parts are described accurately, but she falls into that arguably they were just ordered to do that trap because her impoverished language choices don't let her see the full picture.

She then tries to warn against how various anthropomorphizations and metaphors can mislead.

And, according to the Pew Research Center, nearly two-thirds of Americans worry that AI is advancing too fast and have little confidence in the United States government's ability to regulate AI or in firms' approach to responsible model development.

The way we frame public conversations and thinking can stem the spread of such concerns and calmly correct them, in the view of Melanie Mitchell, author of Artificial Intelligence, A Guide for Thinking Humans.

She has cautioned against using evocative terms that characterize AI as possessing human-like traits and emotions.

Referring to recent hacking incidents involving AI agents, Dr. Mitchell wrote in a recent newsletter on Substack, It's useful to consider a more prosaic description of what actually happened.