Skip to main content
Pushmeet Kohli

Pushmeet Kohli

British computer scientist

Nov 10, 2026

21:44
Do you feel like that's the route to AGI? What is your opinion on where that goes?
21:49
We need different, we exhibit different types of, like Demis has an advantage in being able to answer this question because of his background as a neuroscientist.
22:00
I'm a simple computer scientist so I'll sort of give a simple answer.
22:04
I think my understanding of how we do things, there are different types of processes at play when we are trying to solve intelligence tasks.
22:20
We need sort of episodic memory, but we also need working memory.
22:26
And LLMs have been able to show that they are able to have amazing episodic memory.
22:34
They can learn from a lot of data and then they can sort of generalize in sort of remarkable and very surprising ways.
23:37
candidates? Anything you've seen that's excited you? Any new paths?
4:51
Uh, what do you think, Sal? Uh, a bitter lesson for data.
4:54
Yeah.
4:55
So I, I think the-- So I was at DeepMind when Rich was, uh, with us and working, uh, and thinking about this, this idea of, uh, the bitter lesson.
5:07
When Rich-- I took my-- The, the thing that I took from, uh, Rich's original sort of, uh, uh, lecture that he actually gave at DeepMind was not about the actual, uh, notion of whether data is useful or how should we think about machine learning.
5:24
I think m-my, uh, sort of take was that he was talking about something more conceptual, and the conceptual thing that he was talking about is sometimes, um, when we are looking at problems, we think about solutions in a very religious way.
5:42
I'm a modeler, or I'm a data g-generation person.
5:45
And I think that is the bitter, bitter lesson, that if you approach the problem with that mindset, you might not succeed.

18 MINS LATER

23:49
What do you think, Pushmeet? Design versus understand.
2:47
I think the main
2:49
reason why we started the whole watermarking project at DeepMind almost eight years ago now was to give a sense of provenance.
3:02
to what users are seeing.
3:05
If you look at different modalities, whether it's text, whether it's images, whether it's videos, it was hard to imagine 10 years back that we would today be talking about systems that can create images, videos, audio, text, and proteins or enzymes that are indistinguishable from what a human or what exists in nature.
3:30
And it's an important sort of element for users to give users this idea that you can really believe what you are seeing.
3:40
So what do we need from a watermark? We need three properties.
3:44
One, it should not degrade the quality.

12 MINS LATER

15:16
Just in terms of images then and video, is it the same idea? You have a lot more space to hide things as you describe.
5:59
Can you explain what that means?
6:00
means?Thanks, Tom.
6:02
I mean, we are seeing these systems of intelligence which are allowing us to reason about the data that we have captured at a scale that is just not possible with a single human mind.
6:16
They humanly cannot read at the hundreds of thousands of, uh, scientific results that are coming out, uh, each year.
6:24
And what, uh, uh, these scientific agents like the AI co-scientist tell- uh, allow us to do is basically explore that vast scientific literature to find the answers to the problems that, uh, scientists are interested in solving.
7:47
Is, is this a fundamental shift?
7:49
Yes, I think so.
7:50
I think that thinking about what are the scientific problems that are going to change the world in a better way, that will be the role of the human scientist.
12:19
So Dr. Koli- My question, what comes next with Alpha Fold 3, and what all still remains fundamentally unsolved in there?
12:30
Yeah.
12:30
So I, I think what, um, AlphaGo showed us is that we can go beyond, um, human knowledge and come up with new, uh, insights.
12:44
And, uh, in some sense we started the Alpha Fold, uh, project right after, um, the Lee Sedol match, because that gave us that confidence to explore this whole area.
12:56
Um, the first iteration of Alpha Fold, Alpha Fold 1, was built, um, as a two-step process where there was a neural network which was giving us some insights into which proteins might be close to each other, and then there was a more classical, uh, module which was trying to predict the 3D structure.
13:19
For Alpha Fold 2, we, uh, started from scratch and then went end to end, where, uh, one neural network starting from the amino acid sequence tried to predict the 3D structure.
13:34
And that elegance and that simplicity of that solution with a lot of very interesting scientific insight baked into the architecture of the, of the neural network, uh, was the key in achieving that amazing, uh, sort of milestone.
17:49
My question is, um, are we moving from solving the equations to actually learning the system?
2:51
we started working on computer science.
2:53
Our earlier sort of work, and if you go back to the project on AlphaGo, where the AlphaGo agent was able to beat the world Go champion in the game of Go.
3:08
And the remarkable sort of thing in that agent was that it was able to explore this amazingly large search space.
3:19
of all possible sort of go positions in such an efficient manner that it can sort of come up with what is the optimal move at that time.
3:28
And that really surprised people, both go professionals as well as scientists.
3:35
Scientists believe that that event would come much, much later because it was a very hard problem.
3:42
And so what that gave evidence for is the ability of these large scale neural network based systems to be able to reason and do very efficient exploration in these large search spaces and come up with amazing new insights about the particular domain.

19 MINS LATER

22:42
And maybe it will take a few more iterations, but we can still go really, really far.

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.