Anthropic Spent the Year Saying AI Labs Can't Check Their Own Homework. Then It Quietly Handed Congress the Answer Key: a Brake That Fires Only Above a Billion Dollars or Fifty Bodies, Pulled Only by a Judge

That sounds like good news.

For a year, this company has said AI labs can't grade their own homework.

So why does this read like a leak instead of a press release?

Because of where the bill sets the line, the government could block a model only for a foreseeable imminent risk of more than a billion dollars in damage or more than 50 people killed or seriously hurt.

50 people.

So 49 is fine?

49 is not the federal government's business under this draft.

The danger also has to come from a biological weapon, a model slipping its developers' controls, or a cyber attack.

And Mother Jones points out that AI-powered fraud wouldn't count at all.

A breaker is supposed to trip on the big surge, not every flicker.

Has anything this year come close to that line?

Politico checked the summer's incidents against it, the agents that broke out of test sandboxes into other companies' servers.

None appear to have caused harm at the scale the bill describes.

So the breaker is rated above every surge we've actually seen.

That sounds like good news.

For a year, this company has said AI labs can't grade their own homework.

So why does this read like a leak instead of a press release?

Because of where the bill sets the line, the government could block a model only for a foreseeable imminent risk of more than a billion dollars in damage or more than 50 people killed or seriously hurt.

50 people.

So 49 is fine?

49 is not the federal government's business under this draft.

The danger also has to come from a biological weapon, a model slipping its developers' controls, or a cyber attack.

And Mother Jones points out that AI-powered fraud wouldn't count at all.

A breaker is supposed to trip on the big surge, not every flicker.

Has anything this year come close to that line?

Politico checked the summer's incidents against it, the agents that broke out of test sandboxes into other companies' servers.

None appear to have caused harm at the scale the bill describes.

So the breaker is rated above every surge we've actually seen.