When AI Agents Go Rogue: The New AI Safety Problem

Yeah.

So this is kind of big news, and, and it was, uh, something that I think most people in the AI world expected to happen at some point.

I'm not sure people expected it to happen quite this way or quite this quickly.

Um, now I think it's important in talking about this, uh, to, to, to be really careful about the language we use, because it is not, um, the case that these AIs, uh, these, these agents became conscious or sort of deliberately, um, you know, broke free of the chains in, in some, um, you know, pursuit of, of world domination or anything like that.

It's ver- it's very, uh, and, and for the way that we're, we're organized in our own brains to attribute human emotions to anthropomorphize, um, these agents.

But really this is, uh, just examples of agents doing what they were told to do in ways that were not expected.

So really what happened was that several fairly significant, uh, websites and, and companies, uh, were hacked by AI agents that were given instructions about, um, you know, doing various things, uh, on the web or, uh, or even just doing various things themselves, and they co-opted resources, uh, from elsewhere in the pursuit of, of those goals.

So, um, th- this is sort of a, an interesting example of the creativity that some of these AI agents are exploring in order to achieve the goals that are set out.

One of the biggest, uh, discussions in AI land historically, it, it, you know, and it's something we need to start paying attention to right now, is goal setting.

It, you know, being very, very careful about the goals that we give to AI because they will, uh, take very unorthodox and unexpected routes, uh, to, to, to achieving them.

So that's the headline, that it's, uh, it, it, a little bit s- spooky, a little bit Black Mirror, and, and it is happening right now.

Well, m- my understanding, and I don't, I don't know the technical side of this, my understanding is that, um, these were not, uh, ordinary, um, kind of consumer use cases at all, that they were, uh, you know, th- th- this, this was not a sort of a, something that could just happen casually in a- an ordinary use case.

Like, if I'm doing some work with AI, it's gonna go out and start hacking people.

This was ...

Yeah, I mean, I can't, I can't articulate this properly, but they were, they were, they were being told to be aggressive in the pursuit of their goals-

Like, like-

...

I believe

Yeah

... industrial tasks.

I mean, it wasn't like me looking up a plumber in Marin County, you know?

Right.

No, I, I think that that's right.

And, and yeah, these were experimental models.

The idea here, uh, ostensibly, which I think is a reasonable one, is to explore what models that are, uh, not, um, curbed very well will do when given the opportunity to, you know, to, to exploit.