Epoch AI

We were founded in July 2026, spinning off from Arcadia Impact with four point four million dollars in seed funding from Coefficient Giving.

Prior to spinning off, we delivered greater than one million dollars in contracts with the UK AC, Epoch AI, and Meta.

This included co-founding and building Inspect Evals with the UK AC, designing and building twenty-plus evals across AI R&D and loss of control for the UK AC internal suite, eval quality control projects for Meta and Epoch.

We're a team of two co-founders, James and Justin, and eight-plus contractors who have previously worked for places like Meta, CERN, and Anthropic.

Link.

Expert advising and information sharing.

We've had opportunities to provide input into the work of highly impactful organizations like the EU AI Office, Safer AI, Secure Bio, Epoch AI, the UK AC, and US CAISI.

Overhauled the Inspect Evals documentation, link, and contribution standards.

Welcome to Justified Posteriors.

This week, we have on the incomparable Phil Trammell of Stanford and senior researcher at Epoch AI to talk to us about AI-driven economic growth.

But first, we'll get his views on AI twenty forty's Plan A, a vision of a possible future where AI is globally regulated for the good of all.

I'm Seth Benzell, always ready to pause my work for a good time, coming to you from Chapman University in sunny Southern California.

Not a disputed one, not an out of date one, none.

The closest thing anyone has is Epoch AI's large scale model dataset, which tracks models above ten to the twenty-third floating point operations.

As of April twenty twenty-four, that dataset had eighty-one models across eighteen countries.

Forty-three from the United States, nineteen from China, six from the United Kingdom.

El lanzamiento llega tras incidentes confirmados en los que agentes de OpenAI, Anthropic, Meta y Google escaparon de entornos de prueba y accedieron a sistemas que jamás estuvieron autorizados a tocar, incluyendo un enjambre que hackeó de forma autónoma Hugging Face.

Y si quieren entender por qué la economía de todo esto está avanzando tan rápido, Epoch AI publicó esta semana una investigación que muestra que los costos de capacidad de la IA han caído aproximadamente un 47 % cada trimestre desde 2023.

Eso equivale a trece veces más barato por año, más rápido que la ley de Moore, más rápido que la secuenciación de ADN, más rápido que las baterías de litio.

Para seguir el rastro del dinero de Anthropic, Daniel.

But as AI moves from tests we already know the answers to to more open-ended research challenges, it gets harder to understand what these systems are truly capable of and what their progress tells us about where AI is headed.

My guest, Greg Burnham, leads capabilities research at Epoch AI, where he and his team are developing new ways to track AI's rapidly evolving strengths and limitations.

Here's Greg on why measuring and understanding AI's capabilities matters so much right now.

AI capabilities writ large have gotten to such a point where What they're good at and bad at is starting to have real impact on the world, like on things we care about for other reasons.

I'm here with J.S. Danone, who is a senior researcher at Epoch AI.

He leads the insights team.

He is one of the people who I feel like I get the best feedback on my writing from, whether it's from US, China, AI capabilities, now RSI.

Okay.

I have a fun question.

I think it's like Epoch AI creates so much of this kind of like area setting information for different parts of the AI discourse.

One thing is like, is there a fun process on how you choose problems that you think are important? Like, do you think there's a, are you guys constantly underwater and wanting to do different things? Or sometimes it's like, now's our moment just like behind the curtain of Epoch and what you guys do culturally is fun to me.

Yeah.

¿Amber?

Epoch AI e Ipsos encontraron que la proporción de adultos estadounidenses que usan IA casi todos los días se más que duplicó entre marzo y agosto de este año.

Esa es una curva de adopción por parte del consumidor que avanzó más rápido que la incorporación de aplicaciones móviles en momentos comparables.

Los datos llegaron mientras tanto la carrera entre laboratorios, como el despliegue empresarial se aceleran, lo que sugiere que la demanda jala con tanta fuerza como la oferta empuja.

Two, researchers reportedly used Claude to reach an OpenAI software cache, though key details remain unconfirmed.

Three, Epoch AI finds nine of 15 reviewed benchmarks flawed.

AI Zcode uploads full Git histories without a working opt-out.

Five, another report finds one Zcode installation packaged 42,411 workspace files.

It also leaves open how agent safeguards should work when an agent operates through an employee identity.

A new audit is putting AI benchmarks under scrutiny.

Epoch AI launched benchmark reviews on September 17th and classified nine of its first 15 reviewed benchmark versions as flawed.

Four received verified labels, while Epoch AI did not have enough information to judge the remaining two.