Skip to main content
Thore Graepel

Thore Graepel

Computer scientist

Oct 28, 2026

6:06
Can you explain a bit about that, how it's different to modern LLMs and then I guess like where we went wrong potentially, like why did we move from AlphaGo in this amazing moment at Mood37 to the LLM architecture where we kind of lost reasoning?
6:23
Yes, yeah, that's a great point.
6:24
So going back 10 years in time, so to speak, what does the AlphaGo architecture actually look like? It has these two sides to it.
6:34
Yes, there are neural networks.
6:35
They're very important.
6:36
There were two of them.
6:38
The policy network decides what are promising moves in a given position, and the value network is able to judge if a position is good for black or for white.

10 MINS LATER

16:42
How do we try and rebuild that for a much more general intelligence, do you think?
4:32
Sure, but the token approach has produced some pretty spectacular results, hasn't it? And so are you saying that you're sort of going back to a kind of if this, then that approach or a kind of algorithmic, traditional machine learning algorithmic approach, but rather than a neural
4:53
network approach? So for what we do, we definitely need neural networks because they've developed to a point where they can now do semantic reasoning.
5:02
You can give them text and they can give you an understanding of the text.
5:08
You can give them images and they can give you an understanding of the images.
5:15
The problem is if you delegate the whole reasoning process to large language models, because then inside of the model, you get these complex token manipulations, but you never really get the full picture of a system that knows what it knows, knows what it doesn't know, has this knowledge state and then advances that knowledge state.
5:39
Now, the way our system does that is by orchestrating the reasoning process.
5:45
but using large language models and other neural operators to interface with the world.
8:16
So if you're producing an auditable system, do you think you'll be able to address the problems with agents and rogue agents that have flared up in the media in recent times? Agents breaking out of sandboxes, for instance?
56:26
Yeah.
56:26
You know, it's the hard part of thinking, so to speak.
56:30
And, um, AlphaGo actually embodied both of these concepts and brought them together.
56:36
There were these two neural networks, the policy net and the value net, that look at the board and make judgment calls.
56:43
What are good moves in this position? How good is this position for me? How good it is for the opponent? That was an important part of the system.
56:52
But, um, the other half, the system two part of it, was the game tree search.
56:58
And the game tree search is this process by which you start with a, with a position, and then you look at possible moves you can make that take you to future positions, and you, then you think about what the opponent might answer, and you-- So you develop this, this big data structure of hypothetical moves that you could make, and, and the system reasons about these moves and tries to find the one that leads to the best outcome.
60:56
It's fascinating to think about these things, and we think about it in, uh, with self-driving cars as well, being far more safe than, than people driving.
6:34
Mm-hmm.
6:35
And so what that does is it makes the tree n- more narrower because these are all the moves, but now we're looking only at a small selection of moves at each stage.
6:46
And so that's very helpful.
6:48
The value net is very useful because, uh, ordinarily we'd have to always look at all the variations to the very end because only then would we know if black or white has won the game.
7:01
But the value net allows us to estimate this much earlier.
7:05
You can show it an incomplete board, maybe only after 10 moves or 15 moves, and it can already judge if it's more favorable for black or more favorable for white.
7:16
And this, of course, has also been trained on game data, um, by using machine learning.

16 MINS LATER

23:47
Mm-hmm.
19:18
The, the architecture of your program, AlphaGo, was very different, wasn't it? How, how did it, how did it know how to play Go?
19:26
Yes, that's a great comparison.
19:28
So, um, Deep Blue falls into the category of what we might call good old-fashioned AI.
19:35
It, uh, pretty much did a brute force search of all the possibilities that it could within the time limit and with the computer available to find the best move.
19:45
In contrast, uh, AlphaGo has a much more human-like architecture, if you like.
19:52
So as humans, we have two different modes of thinking.
19:56
Kahneman calls them fast thinking and slow thinking.
21:07
What's the significance of move thirty-seven?

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.