The Biggest AI News This Week Is Getting Overlooked

Anthropic has begun the process of embedding third-party safety evaluators into their organization.

Last month, as part of calls to pace the frontier, Dario Amodei pledged to integrate third-party evaluators into Anthropic, giving them staff-level access to monitor safety procedures and reports on incidents.

In an update published on Thursday, Anthropic said that they've now integrated the first batch, which is, of course, a six-person team from Meter.

They said the Meter folks have employee-like access, which is their term, including use of company computers, office space, and use of internal tooling.

Anthropic added that they're working with Secure Bio and Accenture as evaluators and are in early discussions with other organizations, including Hugging Face.

Explaining why they support this model of safety monitoring, Anthropic wrote, "We want to be publicly accountable for the decisions we're making, but we can't publicly share all the information relevant to our safety work.

Some of it could be useful to malicious actors or is commercially sensitive, like computer training information.

Instead, we're giving third-party evaluators deep access to our information and are having them share evidence and reasoned conclusions about the risks involved in building our models.

We aim to close the gap between what evaluators understand about a frontier AI company's systems and what the company itself understands." Now, as this safety regime gets rolled out, there are two big areas of discussion and, frankly, criticism that are worth understanding.

The first is the independence of the outside safety evaluators.

Unlike what some have said, Anthropic claims they do not fund Meter, and frankly, the web of connections is a bit beyond the scope of this particular headlines to unpack.

However, Meter does share major backers with Anthropic aligned around the effective altruism space.

It's not necessarily full tinfoil hat, it's just that they come from the same philosophical roots and end up having a lot of the same funders because of that.

Now, there are a range of valid opinions that could be held about the arrangement, but it's context for why Anthropic is quickly moving forward with that particular partnership, while OpenAI appears a little more hesitant.

The other big issue is how much control the labs have over the publication of safety audits.

Earlier this week, Reuters published a report claiming that both Anthropic and OpenAI were seeking strict editorial control over reports.

This seemed to be a fluid negotiation at the time of reporting, and the parties seemed to be moving toward a compromise that involved redacting confidential information and noting when major redactions had occurred.

With Meter, Anthropic wrote that they, quote, "can't veto publication, but we can require them to redact information if it falls into narrow predefined categories.

Meter is free to characterize our redactions publicly and say whether it believes the redactions were justified as long as it does so without revealing the redacted information." The categories mentioned were data about customers, legally privileged information, and information about model architectures.

Anthropic has also defined three tiers for embedded auditors, with Meter the only one in the top tier, defined by employee-level access.

Secure Bio is currently in the bottom tier as they begin initial work reviewing existing chemical and biological weapons risk already publicly disclosed in Anthropic's August risk report.

In that tier, Anthropic requires auditors to seek approval before publishing.

Anthropic said that they will evaluate monitoring Secure Bio to the second tier, which would get them more access to information and no approval required following the completion of their first report.

Accenture has not begun work, but Anthropic says they are aiming to develop a partnership with them that involves red teaming models, guardrail testing, and alignment assessments.

Most of the chatter coming out of this is about the independence of Meter, with Embroidery founder Zach Corman summing up a big strand of thought when he writes, "For all the AI safety people pushing for independent evaluators like Meter and AISI, there's already a term for this, security theater." And yet I think for a lot of folks, it's less about where it starts and more about where it ends.