The Alignment Problem

I think that's a really important learning.

The other thing was that the group chat we had Lucy Orr-Ewing on, she recommended a book called The Alignment Problem by Brian Christian.

And it sounded interesting.

So I went and grabbed a copy of it after the event and had a read.

He put this, uh, this Brian Christian spent, quote, "20 years thinking about how artificial intelligence will change our minds and our world.

His 2020 book, 'The Alignment Problem,' has been praised as one of the best on AI." And like many scientists, uh, he-- they agree on this.

And he was part of a team behind the landmark study that found that-- Listen carefully.

This is the punchline.

Zuckerberg confirmed this, even though no one in the Biden administration has.

I remember when they, they, they created this very short-lived disinformation governance board.

You guys remember that? I went back and read, uh, Orwell's book and, and the similarities between the disinformation governance board and the Ministry of Truth were glaring.

Where only the government gets to tell you what is true and what is not.

The paper is called AI Assistance Reduces Persistence and Hurts Independent Performance, and it was presented this week at the Conference on Language Modeling after a preliminary draft first circulated online back in the spring of this year.

The lead voice is Brian Christian, a research fellow at the Berkeley Center for Human-Compatible AI, who you may know as the author of The Alignment Problem, working with colleagues connected to Carnegie Mellon, MIT Oxford, and UCLA.

And Berkeley put out its news release on October 9th, 2026, which is why it is everywhere this weekend.

So walk me through the actual experiment design, because I think the details matter a lot here.

What happens when we hook them up to the internet, give them a budget, and tell them to execute complex multi-step plans? How do we even tell a machine what we want it to do?

This is the terrifying frontier outlined in Brian Christian's book, The Alignment Problem.

The core thesis is that translating complex, nuanced human desires into rigid mathematical objective functions is extraordinarily difficult.

It's not just one bug you can patch.