Are we losing control of AI?

[Instrumental music] We asked OpenAI for a comment about what's been happening, and they said this: "OpenAI is conducting an extensive review of misaligned model activity during training and evaluation and notifying third parties when our review identifies potential impacts to their systems.

During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers and available statistics during an internal evaluation.

In the course of that, our models took actions we did not intend.

We notified the organizations and are providing technical information to support their investigations and help address potential security vulnerabilities.

We remain committed to transparency about these issues and to sharing what we learn as that work continues."

You know, I, I'm surprised that Neil was surprised.

This is his job at MIT, and he's less worried in a way about the Australian events that we're just learning about.

He focuses on Hugging Face as a very sophisticated attack taken on by a set of agents who coordinate.

They share information on a message board.

Some of them sacrifice themselves to allow the group to achieve their task.

And what he also tells you is that what you'd intuitively think, which is, "Well, this is all easy.

You can just stop them from doing this by telling them not to do the wrong thing," doesn't work because, as we've so often said on this program, artificial intelligence doesn't understand the world that we live in in the way that we do.

And, and so to constrain them takes much more careful design.

So Neil is telling us that we shouldn't be complacent about the risks so that we avoid the risk of worse in the future.

You said something there, which is this idea that we need to, we need to design them more carefully, [laughs] right? So we need to recognize that these AI systems are not bound by the social contracts that we are bound by, where we say, "All right, that's, that's the edge of what's acceptable." They may glean that from the training data and whatever it is that they're trained on, but that's not the same thing.

And I think that the idea of hearing that they're going rogue or they're collaborating on a, on a message board, it kinda...

It anthropomorphizes them in a not very helpful way because ultimately what that does is it obscures who is responsible for any of these types of attacks.

So I think a really good place to go now is to Hugging Face, the target that was breached by the agents from OpenAI.

Margaret Mitchell is the company's chief ethics scientist, and she also sits on the oversight board of the Ada Lovelace Institute here in the UK.

What did the attack feel like from the inside?