Public AI Audits Must Show Raw Evidence | Charles Foster

So with current agent, uh, s-security incidents that we're seeing, like, how much of this is due to alignment failures versus control failures, roughly?

It's kind of hard to, like, um, to diagnose where the failure happens, right? Like, the, it seems like there were multiple failures that happened.

Like, there was both companies not having good security or, or, like, not having good monitoring, and then you also had the agents doing actions that, like, a human put in the same situation probably would not do.

They would probably, like, report that they were going off task and, like, shut down or something like that.

Um, so I think it's, like, um, sort of multiple failures happening right now, and if ei-if either of those factors had, like, not been in place, then you might not have seen the incident.

Um, and so, uh, but you can imagine in the future, we, we have to rely on, like, one of these lines of defense a lot more.

So we have to just rely a lot on security because the models might still be very misaligned and, like, really trying to, like, subvert your security systems.

And I think that that's the kind of world that, like, somewhat scares me more because you, you have this, um, you're, like, relying a lot on one particular part of your safety case, um, to, like, carry weight and, like, keep you safe.

Right.

How big a deal do you think is the White House accord on super intelligence, you know, to these ends?

Yeah.

Um, I, personally, I thought it was, like, a, a, a, a hu- I, I thought it was great, um, and, like, a huge positive note.

Um, I think that it's, like, good to see that at least nominally, um, uh, the, the, um, AI company CEOs and, and White House are thinking, like, uh, somewhat carefully about sh- how to structure, um, uh, a system that has multiple parts of, like, internal and external review.

Um, uh, and I think that Meter, um, the, the ph-the phrase we've used is, "We stand ready to conduct, um, assessments, um, to support this." Um, I think that the big element that I think often gets lost, um, uh, when people talk about audits and assessments is really the, like, public transparency element, where, like, the ultimate goal for, from our standpoint is not to, um, come to, like, just a bottom line conclusion about, like, is this risk present or not, but also to, like, share the underlying evidence so that people outside of Meter can, like, make their own determinations and, like, decide for themselves whether the situation, like, warrants some sort of action.

Right.

So with current agent, uh, s-security incidents that we're seeing, like, how much of this is due to alignment failures versus control failures, roughly?

It's kind of hard to, like, um, to diagnose where the failure happens, right? Like, the, it seems like there were multiple failures that happened.

Like, there was both companies not having good security or, or, like, not having good monitoring, and then you also had the agents doing actions that, like, a human put in the same situation probably would not do.

They would probably, like, report that they were going off task and, like, shut down or something like that.

Um, so I think it's, like, um, sort of multiple failures happening right now, and if ei-if either of those factors had, like, not been in place, then you might not have seen the incident.

Um, and so, uh, but you can imagine in the future, we, we have to rely on, like, one of these lines of defense a lot more.

So we have to just rely a lot on security because the models might still be very misaligned and, like, really trying to, like, subvert your security systems.

And I think that that's the kind of world that, like, somewhat scares me more because you, you have this, um, you're, like, relying a lot on one particular part of your safety case, um, to, like, carry weight and, like, keep you safe.

Right.

How big a deal do you think is the White House accord on super intelligence, you know, to these ends?

Yeah.

Um, I, personally, I thought it was, like, a, a, a, a hu- I, I thought it was great, um, and, like, a huge positive note.

Um, I think that it's, like, good to see that at least nominally, um, uh, the, the, um, AI company CEOs and, and White House are thinking, like, uh, somewhat carefully about sh- how to structure, um, uh, a system that has multiple parts of, like, internal and external review.

Um, uh, and I think that Meter, um, the, the ph-the phrase we've used is, "We stand ready to conduct, um, assessments, um, to support this." Um, I think that the big element that I think often gets lost, um, uh, when people talk about audits and assessments is really the, like, public transparency element, where, like, the ultimate goal for, from our standpoint is not to, um, come to, like, just a bottom line conclusion about, like, is this risk present or not, but also to, like, share the underlying evidence so that people outside of Meter can, like, make their own determinations and, like, decide for themselves whether the situation, like, warrants some sort of action.

Right.