Latent Space: The AI Engineer Podcast
Jun 19, 2025 · 1 hr 18 min · 10 segments
Solving Poker and Diplomacy, Debating RL+Reasoning with Ilya, what's *wrong* with the System 1/2 analogy, and where Test-Time Compute hits a wall Timestamps 00:00 Intro – Diplomacy, Cicero & World…
Noam BrownGuestBooksHost
Alessio FanelliHost
I think I said this on, on Twitter at one point, that I think this would be a great benchmark.

I would love to see all the leading bots play a game of Diplomacy with each other and see-

And I think a couple of people have, like, taken inspiration from that and are actually, like, building out these benchmarks and, like, eval-ing the models.

But I think, I think it really is a fascinating benchmark and I think it would be, um, yeah, I think it'd be a really cool thing to, to try out.
How has the vibe, how have the vibes changed just in general? You said you were very excited to learn from domain experts, like in chemistry, like, how they review the O series models.

I think the trajectory was pretty clear pretty early on in the development cycle.

And I think that everything that's unfolded since then has been pretty on track f- for what I expected.

So I wouldn't say that my perception of where, where things are going or has honestly changed that much.


And I think that we're going to see a broadening of what these models can do as well.

I just find it so useful, the- especially the fact that it can now browse the web and, like, you know, do meaningful research on my behalf.

Like, it's kind of like a mini Deep Research that you can just get a response in three minutes.

I think I said this on, on Twitter at one point, that I think this would be a great benchmark.

I would love to see all the leading bots play a game of Diplomacy with each other and see-

And I think a couple of people have, like, taken inspiration from that and are actually, like, building out these benchmarks and, like, eval-ing the models.

But I think, I think it really is a fascinating benchmark and I think it would be, um, yeah, I think it'd be a really cool thing to, to try out.
How has the vibe, how have the vibes changed just in general? You said you were very excited to learn from domain experts, like in chemistry, like, how they review the O series models.

I think the trajectory was pretty clear pretty early on in the development cycle.

And I think that everything that's unfolded since then has been pretty on track f- for what I expected.

So I wouldn't say that my perception of where, where things are going or has honestly changed that much.


And I think that we're going to see a broadening of what these models can do as well.

I just find it so useful, the- especially the fact that it can now browse the web and, like, you know, do meaningful research on my behalf.

Like, it's kind of like a mini Deep Research that you can just get a response in three minutes.
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.