Steven Adler on the AI industry’s rogue agent crisis

Yeah.

This, this is absolutely one of the big picture questions, is that it, it seems reasonable that there's going to be more incidents like this and more problems that the labs have to deal with.

And these labs are under a lot of pressure to beat the other labs to the next release.

And, uh, so there's a worry about they're gonna be under so much pressure that they're not going to respond appropriately.

I feel like OpenAI is, is, um, now saying the right things.

They say, "We're, we're slowing down model training and we're, you know, pulling in ind- independent experts." So hopefully the, the attitude, at least in the short term, has shifted in a, a good direction.

But, uh, it does seem reasonable to worry that in a few months, if they clean this up, are they gonna go back to just kind of everybody be feeling under a lot of, a lot of pressure to, to go quickly.

Like, I think a lot of people who are worried about the, the downsides here like to draw comparisons to other disasters, and I've, I've seen people compare this to COVID.

Um, and I don't...

I, I personally don't kind of agree with that analogy in the big picture.

We can maybe talk about that later.

But one thing I do think is worth thinking about is in the early months of COVID, we had these estimates of how many COVID infections there were that were, like, orders of magnitude too low.

You know, in, in February they said there were, like, 15, and that we now in retrospect know there were, like, 10,000 or something.

And I think that's a real question here where, um, you know, hopefully these companies are doing their due di- diligence.

But, like, the, the way these models are trained, they, they do, like, millions of roll- rollouts.

And who knows? No- nobody has time to look at all those rollouts.

We can ask AI to audit, you know, to, to summarize them.

It hopefully catch them.

But if you don't know what to ask for, you know, you might not catch.

Anyway, so I think there i- pro- there is a lot of uncertainty about have we actually caught all of this, or is there a lot of other misbehavior, um, going on that we're not, um, catching?

Yeah, I don't know.

I mean, the nice thing about AI monitoring, it's quite cheap.

Like, actually I think that you, you can catch quite a lot of this misbehavior, including things that are unknown unknowns if you design your monitoring setup the right way.

It, it does involve not being quite so narrow in what you are looking for, right? Like, you should have some digest of the most concerning things that, that this generic monitor has seen that don't cohere with one of your specific policies.

You shouldn't be looking with, like, a very narrow flashlight for a very narrow set of things.