I don't want to put words in your mouth, but it does seem that you were at the forefront of a lot of what we've been discussing over the past six months.

Yeah.

So we haven't updated Agent Harm.

And I think what we'll probably end up doing is creating another benchmark that I think reflects where the models are, where the capabilities are.

But Agent Harm was interesting because we started developing this in early 2024, working with people at UKAC.

This was the very early days of being able to call one of these frontier models APIs and provide tools that it could call in its response.

So the way it would work is you'd say, I have a get weather tool or a search Google tool.

And I do feel like that the doomerism conversations kind of drown out the real security problems that people like yourselves are working.

What are the students interested in, in terms of like both security and AI and maybe the sort of mix of the two right now?

Yeah, so I feel like when I was a student and even through a lot of like when I was in grad school, at least among computer or computer science students, like security wasn't a hot area, like I'll say.

It wasn't something that like there were a ton of applicants for the PhD programs and you know, a lot of undergrad offerings with, you know, PAC courses, you know, that really started to change nine or 10 years ago, I'd say.

And there was just a ton of interest.

You know, I couldn't put my finger on exactly why, but probably some combination of like crypto starting to take off and sort of things around that space.

Yeah.

But it has been great to see.

So what what specifically are you guys hoping to address?

Yeah.

Um, so we operate a community of, of red teamers.

We provide, uh, sort of prize challenges.

Um, a lot of these come from the needs of, of, uh, the lab sponsors.

Um, so sort of w- to an extent gamify red teaming objectives, um, put up a prize pool, and, and pay people when they find ways to sort of circumvent and violate whatever the object- the safety and security objectives of, of the model developers were.

So that's, that's one.

It's, it's a, a really great community, like 15,000 people come and hang out on the Discord server.

And as it applies to this, I think it's, it's in things like understanding models better, but doing it through the power of agents.