Skip to main content
Stuart J. Russell

Stuart J. Russell

British computer scientist

Oct 16, 2026

17:52
But what does it mean to be a neo-Luddite in the age of AI?
17:58
Well, it's interesting.
17:59
A few years ago, I and several other people won the Luddite of the Year Award from the Information Technology Innovation Foundation.
18:10
And they were really upset that we were pointing to potential risks from artificial intelligence.
18:17
It is sort of odd to be a Luddite when you've actually, I've been working on AI for 50 years.
18:23
I think Geoffrey Hinton was on there, who is in many ways the founder of the modern approach to artificial intelligence.
18:33
Elon Musk, who has clearly made a lot of contributions to technology in different sectors.
19:14
And that's one of the, yeah, sorry to interrupt you, but I just wanted to say there, because I guess one of the big questions is, are these, you know, neo-Luddites standing in the way of progress, or are they actually helping out? Because a lot of people are concerned that this is taking over in such a way, as you say, that they're taking away people's jobs.
17:03
Stuart.
17:05
Uh, well, it, it does feel like we're not really gonna have a debate, uh, because I think, uh, Jurgen is obviously very convinced that AI systems will far exceed human capabilities, and also seems pretty convinced that there is no way for human beings to control AI systems.
17:24
And so therefore, we have to simply rely on the good offices of the AI systems and their, their desire to keep us as scientific specimens for investigation.
17:37
I'm not completely sure that the human race will go along with this.
17:42
We want to retain control.
17:45
We want to have some say over whether we continue to exist rather than it being completely at the whim of AI systems that, in Jurgen's telling, are not even following any sort of design template that we created.
18:01
And so we might well try to interfere with the development of this independent and massively superior species, and that interference would be a risk to the AI system's goals, whatever they are.

13 MINS LATER

31:02
And let me, let me switch to you, Stuart.
speaker_1HOST
4:29
Can you explain what it is that you're talking about when you say that?
4:33
Yeah, so the standard model of AI, I would say, is the approach that held sway from about 1960 until about 2020, something like that, maybe 2015.
4:43
And the standard model says, what is AI? It's making intelligent machines.
4:48
What does it mean to be intelligent? It means the ability to optimally achieve objectives.
4:53
So an AI system is something where the human writes down the objective and the machine achieves it optimally, or as close to optimally as is computationally feasible.
5:04
So that was the paradigm for that entire period.
5:07
I would say before 1916, we were quite confused.

7 MINS LATER

speaker_1HOST
12:30
Could you maybe explain what is provably beneficial AI?
18:47
If Boeing said that it wanted to introduce an airplane and what, how did you say it exactly?
18:53
So I was responding to Dario Amadei's letter called, We Must Pace the Frontier.
19:02
And I think a lot of people interpreted that as a call for just slowing down, which would give us a bit more time to work on safety.
19:10
And I was saying, if Boeing said, we're gonna slow down, we're only gonna introduce a new airplane once every year, and we hope that that gives us enough time to do some flight tests and maybe get some good results.
19:26
We wouldn't accept that.
19:27
The FAA would say, no, it doesn't work that way.
19:30
You have to do the flight tests until you get good results, and then you can introduce the airplane once we certify it.
23:48
Is the government capable of setting standards to say what's here and what's there? And how do we staff it with the right expertise?
13:24
"The best way to guarantee safety is that the creators are liable for what they build." Do you see that liability, those guardrails, as being sufficient?
13:33
No. Um, and for the-- there are really two reasons.
13:36
So the most obvious reason is that the risk, uh, that is uppermost on the mind of, for example, Dario Amodei, the CEO of Anthropic, uh, is the risk of loss of control, uh, which really means that the systems have become intelligent enough that they end up taking control of the world and there's nothing we can do to get that control back.
13:59
And after that, we really have no say in what happens at all.
14:03
Um, and obviously, liability is not going to deter a company from incurring that risk because if it does come to pass, there will be nobody around to pay the money and nobody around to receive the money.
14:18
So that's the first obvious point.
14:19
The-- but the second more important point from a practical point of view right now, um, is that we need more than just liability.
15:50
Uh, when it comes to AI, what's the ultimate goal? To prevent artificial general intelligence or to, uh, just make sure that some of these companies are keeping the models in a box?
speaker_3UNKNOWN
10:48
So yeah, I would say the lead changes.
10:54
ChatGPT was issued in November of 22.
11:00
And that was GPT 3.5. And then GPT 4, I think, was a significant advance that was released in March of 23.
11:12
And at that point, clearly OpenAI was ahead.
11:18
Anthropic hadn't come out with any products.
11:22
Google, although they had invented the basic technology of the transformer, that was published in the open literature in 2017.
11:31
And Google was... cautious because they knew that language models emitted all kinds of nonsense, both false statements, offensive statements, and so on.

8 MINS LATER

MartinHOST
19:58
One, supervised fine-tuning.
10:48
What does it mean specifically in this context when we talk about aligned or non-aligned AI?
10:54
Um, yeah, so I, I think apparently I, I introduced that word into the debate back in 2014 without meaning to.
11:02
Um, and, uh, what aligned means is that the AI system has essentially the same view as we do of what the future should be like.
11:14
So they will try to bring about the future that we want.
11:18
So their, their objectives, if you like, are aligned with ours.
11:23
Now, in reality, we don't even know what we want the future to be like, and it would take us a very long time to figure it out and, and to, to handle all the possible eventualities and rank them, uh, accordingly.
11:38
So you have to face the fact that alignment is not, uh, achievable.
14:38
So what can we do about it?
7:44
Mm-hmm.
7:45
I didn't expect them to take the direction that they have, where we're essentially training gigantic multi-trillion parameter black boxes from vast amounts of text data, and that seems to produce something that's quite intelligent.
8:01
So that, that idea was definitely not on anyone's radar at that time.
8:08
The general direction we were going was gradually understanding the components of intelligence, how one reasons, how one makes decisions, how one learns, and then trying to figure out how to fit those components together to make a system.
8:25
And that's a very traditional engineering approach, the same way that we go about building airplanes and nuclear power stations and cars and all kinds of other things.
8:35
And we've abandoned that almost entirely in favor of the giant black box approach.
8:41
So that's one thing I was not expecting.
10:43
So what AI risks, misuses or behaviors should be banned internationally?
7:30
Can you explain what that is and, and how it relates to the conversation which you're now so engaged in today as to the potential extinction of the human race because of AI?
7:40
Yeah.
7:40
So if we go back even further to the beginnings of the field of AI in the post-war period, so 1940s, 1950s, I, I think everyone agreed that, you know, AI is about creating intelligence in machines.
7:57
What wasn't clear was, well, what is intelligence? You know, how do we define this target? And how do we go about doing it? So there was, uh, you know, an active debate.
8:11
You know, should we try to emulate human intelligence? Should we understand what's going on in the human mind, the human brain, and then sort of go ahead and implement that? Or should we focus on a more abstract notion, in fact, a notion that philosophers had developed for thousands of years and economists as well, this, this idea of rational behavior, that an entity is intelligent if it acts in a way that is expected to achieve its objectives.
8:41
And I would say for the most part, that second approach won out because it doesn't require doing psychological experiments on humans to find out how their brains work, right? It's an abstract mathematical concept, and we had tools, we had formal logic so that we could create algorithms that were able to construct plans to achieve goals.
9:08
So the- these two, the two views are, I would say, the sort of rational view, i.e. base intelligence on formal foundations of how, how one should reason, how one should make decisions, uh, versus a more biology-based view, which is neurons and, uh, and their computational analogs.

15 MINS LATER

24:13
Right.
21:50
Professor Russell's solution is to make sure AIs recognize we humans have many different goals and preferences, and double-check that they're acting in line with our wishes.
22:03
So the AI system is trying to learn about what humans want.
22:08
They will ask permission before doing something that changes the world in ways they're not already sure that humans prefer.
22:17
When you do that, you find that those AI systems that, that solve this game and play it are really the kind of AI systems that we want.

9 MINS LATER

30:59
When I spoke to Professor Stuart Russell, he told me about a recent conversation he'd had with the boss of a leading AI firm.
31:09
One of the leading CEOs told me that we could see mid-sized catastrophes fairly soon.
31:17
He sees that as the best case scenario in the sense that a mid-sized catastrophe, he called it a Chernobyl scale disaster, would get governments to regulate.
31:31
He doesn't think that they will regulate otherwise.
21:50
Professor Russell's solution is to make sure AIs recognize we humans have many different goals and preferences and double-check that they're acting in line with our wishes.
22:03
So the AI system is trying to learn about what humans want.
22:08
They will ask permission before doing something that changes the world in ways they're not already sure that humans prefer.
22:17
When you do that, you find that those AI systems that, that solve this game and play it are really the kind of AI systems that we want.

9 MINS LATER

30:59
When I spoke to Professor Stuart Russell, he told me about a recent conversation he'd had with the boss of a leading AI firm.
31:09
One of the leading CEOs told me that we could see mid-sized catastrophes fairly soon.
31:17
He sees that as the best case scenario in the sense that a mid-sized catastrophe, he called it a Chernobyl scale disaster, would get governments to regulate.
31:31
He doesn't think that they will regulate otherwise.
18:18
noAt its core, this is because we're currently training the popular modern AI systems to copy us.
18:29
So the way we build AI systems like ChatGPT and Gemini and Claude and so on is, um, a process that we call imitation learning.
18:41
So we have recorded human verbal behavior in the form of billions of documents, conversations, novels, encyclopedias, newspapers, you name it.
18:51
And, um, we are training these systems to imitate that verbal behavior.
18:56
Now, humans have a whole lot of objectives, uh, including preserving their own existence.
19:02
So, you know, in some novels they say, "Please don't kill me." Um, and, and that underlies a lot of the human verbal behavior is self-preservation, acquisition of resources.
19:14
People want to become rich, powerful.
20:15
A few months ago, researchers for the e-commerce giant Alibaba were testing an experimental AI agent in a sandbox, an isolated digital environment for AI evaluation.
36:08
[laughs]
36:08
-uh to get the meeting without his permission at all.
36:12
Um and then you know Alibaba, the Chinese company, found that one of its agents which it was testing in a sandbox uh actually escaped the sandbox and was mining Bitcoin using Alibaba's servers and uh and consuming a vast amount of compute to make money for its own account.
36:32
Uh so, so these kinds of things are early warnings that we don't know how to control this technology.
36:39
We don't know how to retain power over these entities and make them do what we want and make them not do what we don't want.
36:46
So that's a warning.
36:48
So when is this transition gonna happen to systems that are really more capable than humans? Uh it depends who you ask.
38:27
Mm-hmm.
2:56
Tell me about it.
2:58
I think it's important to say it isn't my opinion that AI systems could cause human extinction.
3:04
It is the... stated view of the CEOs who are spending trillions of dollars to develop this technology.
3:12
They have estimated the risk from 10%, some of them 20%, 25%, and even 50% chance that if they succeed in building what they are spending trillions of dollars to build, we will go extinct as a result.
3:27
I have my own view on the direction that the technology will take.
3:32
And if we develop, we'll call super intelligent AI, AI systems that are much more intelligent than humans, without some absolute guarantee of safety, then it's very likely that we will lose control of our future and quite possibly cease to exist.
3:51
One thing has to happen first, which is governments need to require cast-iron evidence of safety before allowing developers to proceed with the kind of technology that could pose these risks.
5:56
Yes.
15:33
But let me now switch gears a little bit and talk about where you are seeing the applications of AI over the last year especially advance the most and what gets you the most excited about which fields are we seeing the most interesting or exciting or most impactful applications of AI?
15:51
what's been happening in the last couple of years in the business world is everyone is trying everything to try to figure out where is it really going to deliver value.
16:03
And I think the thing that we're discovering is that the capabilities of these systems are unpredictable and fragile.
16:13
And although they deliver their answers in very confident and very fluent form, they're often completely wrong.
16:22
And so this word hallucinations just means that the systems, it wouldn't say they're making things up, right? Because when humans make things up, they know they're doing something different from telling the truth.
16:35
Whereas for these systems, there's no difference whatsoever in the processing that's happening.
16:40
It's just that sometimes the processing produces answers that are true.

11 MINS LATER

27:33
Race to nowhere type
Anita AnandMODERATOR
37:50
What do you mean by AGI?
37:51
AGI, Artificial General Intelligence, meaning AI systems that essentially exceed human capabilities in every dimension.
37:59
So I have a question for Rutger to do with the value of humanity, which you began your lecture with.
38:06
So there's a faction here in Silicon Valley and elsewhere that not only thinks it's pretty likely that AI systems will take control of the Earth, but they think that that's a good thing.
38:18
They approve of that, and they think it's time that the human race left the stage.
38:23
Yes, yes.
38:23
Do you have something to say to those people? Well,
Anita AnandMODERATOR
39:34
I mean, if you put the prompt in, be nice, don't destroy us, look after people, I mean, why not let AI handle morality or questions of morality?
36:09
What's that?
36:10
So, King Midas is this legendary king who lived in modern-day Turkey, but I think is sort of like Greek mythology.
36:19
He is said to have asked the gods to grant him a wish, the wish being that "Everything I touch should turn to gold." So, he's incredibly greedy.
36:32
Uh, you know, we call this the Midas touch.And we think of the Midas touch as being like, you know, that's a good thing, right? Wouldn't that be cool? But what happens, so he, uh, you know, he goes to drink some water and he finds that the water has turned to gold, and he goes to eat an apple and the apple turns to gold, and he goes to, you know, comfort his daughter and his daughter turns to gold, and so he dies in misery and starvation.
36:59
So this applies to our current situation in, in two ways actually.
37:05
So one is that I think greed is driving us to pursue a technology that will end up consuming us, and we will perhaps die in misery and starvation instead.
37:20
The, what it shows is how difficult it is to correctly articulate what you want the future to be like.

1 HR 16 MINS LATER

113:17
Do you feel like you're winning or losing?
31:28
What gives you the confidence to say that?
31:33
I think anyone who thinks they understand most of what's going on is deluded.
31:40
To a first approximation, these are systems with trillions of parameters, so that's basically what you learn.
31:47
You have a process of learning that tweaks those trillions of parameters by quintillions of small random perturbations.
31:56
until the behavior of the system matches as well as possible what the training data is telling it to do.
32:03
But we have no idea what's going on inside that giant box with trillions of parameters.
32:09
We understand a lot less about that than we understand about the human brain.
35:13
But what about the race with China specifically? And whether they get there first, will it matter what we do or don't do?
8:19
machines are gonna be so brilliant, will they come a day when we just say, fix global hunger, fix climate change, and off they go, and you set them six months or whatever to a reasonable amount of time, and suddenly they fix climate change.
8:33
There's a big difference between asking a human to do something and giving that as the objective to an AI system.
8:43
When you ask a human to fetch you a cup of coffee, you don't mean this should be their life's mission and nothing else in the universe matters, even if they have to kill everybody else in Starbucks to get you the coffee before it closes.
8:56
They should do that.
8:57
No, that's not what you mean.
8:59
And of course, all the other things that we mutually care about, they should factor into your behavior as well.
9:06
And the problem with the way we build AI systems now is we give them a fixed objective.

10 MINS LATER

19:27
On these daily podcasts every morning, we'll be looking forward to the day's action and picking

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.