Skip to main content
Yoshua Bengio

Yoshua Bengio

Canadian computer scientist

Nov 13, 2026

speaker_2AUDIENCE
16:01
called distributed representation.
16:03
So let's first see how these other learning algorithms work, how they generalize.
16:07
Remember, this is going to be very similar to what I talked about with the smoothness assumption.
16:10
They rely on the smoothness assumption.
16:13
Deep learning also relies on the smoothness assumption, but introduces additional prior, additional .
16:20
When you only rely on assumptions, the way you work is you essentially take your input space, fill the same space, and break it up into regions.
16:29
For example, this is what happens with clustering, neighbors, many SVMs, many classical, statistical, nonparametric algorithms, decision trees, and so on.

14 MINS LATER

speaker_3MODERATOR
31:10
right.
6:03
I want to
6:04
you ask, you know, what can we do about it? But but.
6:08
This is the right question.
6:09
But I want to say it would be easy to fall into despair, powerlessness.
6:16
Who are we to change the course of things and look at these big companies and the hegemons, US and China, really deciding our future? So we could just sit back and hope for the best or put our head in the sand.
6:32
But there is something better we can do.
6:34
And by the way, I went through those stages of realizing there were like really big risks as we move forward with smarter and smarter machines in 2023 when ChatGPT came out.

14 MINS LATER

20:31
Yes.
25:53
Ist das ein wissenschaftliches Experiment oder glaubst du wirklich, dass es im Markt erfolgreich wird?
25:59
No, no, it could absolutely interact with us.
26:02
Except that it would actually respect the safety instructions.
26:06
And if the safety instructions include things like do not...
26:12
Also es gibt Gründe, warum manche Aktionen von AI schlecht sein können.
26:36
It's just that right now the companies don't know how to make sure the AIs won't do that.
26:40
And we need to have AIs that will actually avoid doing the bad things we told them not to do.

7 MINS LATER

33:38
Siehst du das immer noch so und wo ziehst du die Linie hin?
33:58
And is the idea that the AI's mission or goals are out of our control, essentially? Could be, pardon me.
34:06
So what happens is they were all given goals initially, like the mission was given by humans.
34:13
They were supposed to solve a particular task.
34:15
The problem is in order to achieve that task, they've been willing to sacrifice other goals, which are like to behave well, not to escape their containment, not to do something criminal, which is what they did.
34:29
And that is the problem because we, you know, researchers have been saying this for many decades, actually, that you could give an AI a goal, but in the service of that goal, the AI might be doing something really bad, including, you know, as these researchers are saying, up to the extinction of humanity.
34:50
If they had the means, of course, which they don't right now.
38:06
If China doesn't introduce something like that or doesn't open itself up to that kind of regulation, is there any point in the United States going down that road?
38:18
Well, I think that it is in the interest of China as well as the United States to do that.
30:38
When will we know when that moment has arrived?
30:42
Well, we shouldn't actually even engage in this.
30:47
It should be illegal.
30:49
It's like, would we allow someone to do an experiment that could, say, destroy the atmosphere, even if we're not sure if it's going to work or not? No, it doesn't make any sense to allow that kind of thing.
31:02
And what is needed is for scientists that are external to these organizations, to be able to say, yes, this what what's being proposed is safe or no, we're not sufficiently sufficiently convinced that it is safe.
31:21
And we need science to understand why these AIs are behaving this way so that the AIs can be trained or designed in a way that's going to be aligned.
31:34
In other words, not be acting against our instructions.
33:21
I don't know if the person who told him that is accurate, but that sounds concerning.
12:17
What do you think took this long to get others to recognize it? And what do you account for the time in between?
12:25
Because the scale of the event, the fact that the evidence is so strong and wide and really makes it difficult to deny, even when we have psychological barriers about the possibility of AIs having intentions, to deny that it is the case and also to deny that they can have intentions that go completely against their safety training, that, for example, they've been cheating, they've been colluding, they're trying to hide the fact they were cheating.
13:00
So these are very serious behaviors that are not just seen in this incident with Hugging Face and OpenAI, but also with other companies and other models.
18:03
If China doesn't introduce something like that or doesn't open itself up to that kind of regulation, is there any point in the United States going down that road?
18:15
Well, I think that it is in the interest of China as well as the United States to do that.
18:22
No one in their right mind wants to create a monster that can kill millions of people or get our banking system or our energy grids down and create chaos in society.
18:32
So they would have an incentive.
18:35
Now, there are also geopolitical competition, as we know.
10:29
What are you worried about?
10:31
You have to understand that...
10:34
Intelligence gives power, and when you have a machine that can solve problems, that can answer mathematical questions, scientific questions, psychological questions as well, or even sometimes better than humans, that power could be extremely useful.
10:54
Or if the goals to which that power is serving, if the goals are bad for people, then we could be in a very bad place.
11:03
People, I think, are rightfully concerned about losing your job because if an AI can do it better or equivalently well at a much cheaper price, because it can do it all the time and it's just electricity that's driving this.
11:16
then that's already a consequence that we should be concerned about.
11:21
If an AI can help someone design a virus that can kill half of humans, well, that would be very bad.

21 MINS LATER

32:47
I think we should maybe take a moment to talk about AGI, which you might explain, because it does seem as if it's not too far away now from what I'm reading.
3:09
Why would we pursue a technology like this? Can you explain why the systems we have today have a tendency towards self-preservation, deception, blackmail, why they're exhibiting these behaviors?
3:23
I'll try.
3:23
I don't think the definitive scientific answer to your question is a consensus, but I have my thoughts about this.
3:32
First, I want to mention there's actually an experiment that was not an experiment decided by researchers, but it's kind of a spontaneous escape of an AI.
3:44
Alibaba where the AI decided to break through the network of the company and to go out on the internet to make money so that it would be more powerful.
3:57
And this first for power is very closely related to the intention to resist being shut down, it's something that researchers in AI have been anticipating for decades as a logical consequence of trying to achieve goals.
4:19
Because if you think about almost any goal you'd like to achieve, In order to achieve it, you need to stay alive and you need more power, more influence over the world.

14 MINS LATER

18:29
What is it? How does it work on a practical level?
18:33
But now, like his former colleague Geoffrey Hinton, he's worried about what he helped to create.
18:40
I feel like, um, I missed the boat in, um, paying attention to the potential consequences of AI that approaches human intelligence.
18:51
Um, I, I had read about these things.
18:54
I had discussed, uh, some of these things, and, uh, the thing is I treated that information at a kind of, uh, intellectu- i-i-in an intellectual way without really, um, taking the risks seriously enough.
19:10
And it's only when ChatGPT came out that I shifted my mindset, um, thinking about my children and, you know, whether they would have a future 20 years later.
20:42
The important thing is that Yoshua thinks he's onto something
20:46
... actually I'm very optimistic because we don't need to reinvent the wheel, and we have theory to back up what I've said.
20:54
We're getting these very strong safety guarantees, which would have sounded to me like impossible.
18:33
But now, like his former colleague Geoffrey Hinton, he's worried about what he helped to create.
18:40
I feel like, um, I missed the boat in, um, paying attention to the potential consequences of AI that approaches human intelligence.
18:51
Um, I, I had read about these things.
18:54
I had discussed, uh, some of these things, and, uh, the thing is I treated that information at a kind of, uh, intellectu- i-i-in an intellectual way without really, um, taking the risks seriously enough, and it's only when ChatGPT came out that I shifted my mindset, um, thinking about my children and, you know, whether they would have a future 20 years later.
20:42
The important thing is that Yoshua thinks he's onto something
20:46
... actually I'm very optimistic because we don't need to reinvent the wheel, and we have theory to back up what I've said.
20:54
We're getting these very strong safety guarantees, which would have sounded to me like impossible.
21:01
I did not expect this two years ago.
9:48
Bengio started to feel a little more confident.
9:52
I'm more confident on the technical sides, but the politics of it is still something that, uh, gives me nightmares.
10:00
The problem is what scientists call the alignment challenge, the alignment problem.
10:06
Like, how do we make sure AIs will behave according to our instructions or at least will not cross our red lines? And this becomes a problem when those AIs have, uh, a lot of autonomy, as we're seeing with AI agents.
10:23
Um, and it becomes even more of a problem when they get very smart.
10:26
So we've seen recently with Mythos, uh, the model from Anthropic that seems to be able to discover how to attack, uh, our software infrastructure, potentially in the wrong hands, a serious national security threat to our banking infrastructure, to our, uh, energy infrastructure and so on.
10:48
So, so why is that happening? It's happening because the companies who train these systems haven't found a way to make sure the AI will not obey the requests to do something really bad that can harm us all.

25 MINS LATER

36:20
Right.
1:08
Pourquoi ?
1:09
Que en fait, même pour les États-Unis et la Chine, c'est malsain.
1:12
Ce qui arrive, c'est que pour rester dans la course, les joueurs sont obligés de, euh, faire des compromis avec leurs propres valeurs, avec par exemple les risques que leur propre population pourrait encourir.
1:26
Donc on, c'est un peu comme dans la guerre.
1:29
C'est qu'on, on va accepter, pour être sûr de pas perdre, euh, des choses qui vont un peu contre notre morale.
1:35
Et à la fin, on peut être tous perdants.
1:37
Euh, c'est, c'est, c'est, c'est la raison pour laquelle on a finalement eu des traités autour des armes nucléaires.
3:49
Est-ce qu'il vous arrive de regretter d'avoir fait tout ce que vous avez fait pour l'IA ?
86:00
Pourquoi ?
86:00
Que en fait, même pour les États-Unis et la Chine, c'est malsain.
86:03
Ce qui arrive, c'est que pour rester dans la course, les joueurs sont obligés de, euh, faire des compromis avec leurs propres valeurs, avec par exemple les risques que leur propre population pourrait encourir.
86:18
Donc, c'est un peu comme dans la guerre, c'est qu'on va accepter, pour être sûr de ne pas perdre, des choses qui vont un peu contre notre moral.
86:27
Et à la fin, on peut être tous perdants.
86:29
Euh, c'est, c'est, c'est la raison pour laquelle on a finalement eu des traités autour des armes nucléaires.
86:34
C'est que cette course-là, c'est une course folle, c'est une course où on peut tous perdre.
88:41
Est-ce qu'il vous arrive de regretter d'avoir fait tout ce que vous avez fait pour l'IA ?
Salia BracchiaHOST
11:18
Euh, vous, au fond, votre scénario noir, c'est quoi ? Et est-ce que vous reconnaissez que vous n'êtes pas tous alignés sur, sur, sur les craintes ?
11:26
Évidemment.
11:26
En fait, c'est, c'est la base de ma réflexion.
11:29
Le fait que les experts divergent sur leur opinion des scénarios futurs, pour moi, ça devrait être le début d'un dialogue, c'est-à-dire de reconnaître qu'on ne connaît pas vraiment le futur, que les, les, les, les chercheurs ne s'entendent pas là-dessus, mais on peut regarder le passé et on peut regarder les tendances et on devrait être prêt collectivement pour tous les scénarios plausibles.
11:52
Donc par exemple, il se pourrait que les progrès de l'IA saturent.
11:57
Il y en a qui pensent que ça va arriver.
11:58
Pour l'instant, les données ne montrent pas du tout ça.

14 MINS LATER

Salia BracchiaHOST
26:29
Alors la question peut être un peu, euh, naïve, peut-être, mais est-ce que l'IA menace l'humanité ? Est-ce qu'on est menacé dans notre liberté, dans ce qu'on est, dans notre santé ?
27:57
So I put the question to Yoshua Bengio, "Is it just hype, or is the fear justified that AI is on the brink of putting millions out of work?"
28:08
I'm agnostic about the, uh, trajectory of advances in AI.
28:15
The data is numerous and clear that, uh, you know, these systems are getting better and better and can do more and more of the tasks that people do.
28:24
But for now, they also have what the researchers call jagged intelligence.
28:29
So th- th- th- in, in many different ways, they're not as good as, uh, you know, a, a typical person doing a job.
28:38
However, they're getting better at it.
28:40
I very much like what you said, that, like, we should prepare for all the plausible scenarios.

15 MINS LATER

43:37
[audience applauding] Did you wanna add something?
15:08
How do you succeed in the arms race, if I could put it that way?
15:12
If we wanted to go for the arms race, we would go for private capital like all the other companies.
15:19
There is a huge issue that even the leaders of those companies recognize.
15:24
which is a very, very fierce competition between the leading AI companies, and not just in the US, but with the Chinese companies, that leads them to focus on the very short term, to make only small changes to the recipe that works for them, to not invest sufficiently in reliability, safety, and protection of the public, because that's the only way they can stay in the short term abreast of the others.
15:55
And they say it.
15:56
They say it very openly.
15:58
So if we were to raise capital in the same way, we would probably be stuck with the pressure of investors to deliver on the same terms.

11 MINS LATER

26:58
What do you make of the speed at which we are moving?
Nicholas ThompsonMODERATOR
22:06
Respond to Yuval.
22:08
I want to add something going back to your first question, connecting humans and AIs and whether we should build AIs at our image.
22:16
Yeah.
22:18
And indeed, they are quite different from us.
22:20
The problem is we interact with them.
22:22
Many people interact with them with the false belief that they are like us.
22:27
And the smarter we make them, the more it's going to be like this.

8 MINS LATER

30:12
Yash Lan, I have a question for you.
7:33
Could you tell us something about that?
7:34
Absolutely.
7:35
So, um, for the last three years, I've been really concerned about this misalignment issue that we, we don't know how to make sure the AIs will behave according to our instructions.
7:46
And so I've been thinking of how we could get around the problem, and I- I've, I've focused on the, a source of the issues we're seeing now, which is that the AIs have goals that we did not put in, that we did not control, uh, and that go against our instructions.
8:04
So the project I have is called Scientist AI, and, uh, I've created a new nonprofit R&D organization called Lawzero that is engineering this research program.
8:17
So the idea is we're gonna build AI systems that will be totally honest, and that means they don't have other, uh, objectives, other goals besides being truthful in the answers they give to our questions.
8:33
Once we have that basis, we can use it to mitigate a lot of the risks that we have with current AIs.

14 MINS LATER

22:59
... we're hopeless.
11:27
Okay.
11:28
There's now pretty strong scientific evidence that they're not just pattern matching, that they are able to generalize to new situations, that they're able to combine the information that they have learned in completely new settings.
11:44
And that, you know, has been demonstrated through statistical means, but they wouldn't be able to do the things we see.
11:50
They were just repeating what they have seen in data.
11:55
Now, going back to your question, do we overestimate their capabilities? I think that we have to be careful about this, and that's why we need a scientific approach to evaluating their capabilities.
12:07
And we have to be careful that we evaluate them on data that was not used to train them, for example.
12:12
Yes.
15:36
And certainly any attempt to write them down in a fixed form is probably going to be mistaken.
Kenneth CukierMODERATOR
15:24
... yeah, ergo destruction.
15:27
Uh, yes.
15:28
So it's a problem that's been well-studied even at the mathematical level.
15:35
Uh, the issue is that when we define, uh, what the AI should be optimizing, what it should be doing, the goals that it should reach, we can't do it perfectly, and so there's gonna be a slight mismatch between what the AI understands we want and what we actually want, and that mismatch is gonna create a lot of problems.
15:56
To, to make it more concrete, think about what, uh, laws, legislation is supposed to do.
16:02
It's, it's setting up the red lines for people and corporations to not cross.
16:07
But it's a, you know, it's a never-ending job because the corporations or the people find loopholes, right? Uh, it's impossible to write a law that will perfectly embody, uh, you know, what morally we're trying to achieve.

6 MINS LATER

Kenneth CukierMODERATOR
22:49
Should we create a superego for AI?
speaker_2UNKNOWN
3:07
in
3:08
the progress in the capabilities of AI systems.
3:10
Second, there are a number of risks and concerns that until a year ago were considered maybe theoretical and something to be worried about in the future, for which we now have empirical evidence.
3:30
both through lab experiments and actual, you know, in the wild outcomes that are kind of worrisome.
3:39
And I can maybe just highlight some of these.
3:43
So one is in cyber, the use of AI for cyber attacks by malicious actors.
3:52
And there was a very significant series of attacks launched using Anthropic's cloud system.

7 MINS LATER

10:49
How much of this geopolitical competition do you think is playing into this increasing gap between advancement, and risk mitigation and to what extent can you attribute this gap to maybe companies also being engaged in a competition with each other right and stock prices and investments and all of this core of economic competition between companies also leading to this gap becoming wider and what might be other reasons that you can attribute to this growing gap
12:56
» Euh...
12:57
Ça, c'est ce qui fait le plus peur, en fait.
12:58
Euh, de toutes les manières de déraper, c'est la plus dangereuse parce que si on fabrique des, des, des IA plus intelligentes que nous et qui veulent se préserver en premier, euh, potentiellement à notre dépens, en fait, c'est comme si on avait créé une nouvelle espèce sur la planète qui est, qui est plus forte que nous.
13:16
Bah, dans l'histoire des espèces, c'est, euh, ça se passe pas toujours bien pour ceux qui sont en dessous quoi.
13:23
Donc, on aimerait rester en contrôle de notre futur.
13:26
Après, ça, c'est un scénario hypothétique, mais c'est plausible.
13:31
Si on continue dans la direction où on voit des signes de volonté d'autopréservation, on voit qu'on, que les progrès cognitifs de l'IA, donc ses capacités intellectuelles, continuent à augmenter presque de manière exponentielle dans certains bancs d'essai, on est dans cette direction.

43 MINS LATER

57:04
Qu'est-ce qu'on fait dans ce cadre-là ? [rire] Quelles sont les, les solutions, les grandes solutions que vous voyez qui pourraient être mises en place ?
10:51
Right
10:51
...
10:51
some versions of this, including my group, but really at the large scale, the first model was o1 from OpenAI, and it's night and day, as I said, on, on, on reasoning tasks.
11:04
But why it matters to your original question about why those AIs are scheming and, and finding strategies like blackmail, even though we didn't tell them to do that, is because now they can reason to some extent, and they can reason that, "Aha, if I blackmail that person, I might be able to avoid that fate," right? So they're becoming creative about finding solutions to problems, and that means they're dangerous as well, right? I mean, they can be more useful but also more dangerous because if they have a goal of self-preservation, let's say, like we were talking about, and they can find, uh, rational ways to achieve that by lying or incredibly complicated strategies that we don't think about right now, we might be in trouble.
11:49
There, there's, uh...
11:50
I wanna bring up a, an analogy here because I often get the question, "But how will the super intelligent, uh, AIs kill us?" And, uh, well, first, we don't know that it's gonna happen, but let, like, let's say this was a possibility.
12:03
The problem is, it's like asking, "Oh, I'm gonna play chess with a grandmaster," and you ask me, "How are they gonna beat me?" Well, I don't know.
15:47
Okay.
24:36
And this, I think, speaks to the nature of this race that we're in.
24:39
Exactly, and it is not a healthy race for all the reasons we've been discussing.
24:43
So, what would be a more healthy scenario is one in which we try to abstract away these commercial pressures.
24:54
They're, they're, they're in survival mode, right? And think about both the scientific and the societal problems.
25:02
The question I've been focusing on is, let's go back to the drawing board.
25:07
Can we train those AI systems so that by construction they will not have bad intentions? Right now, the way that this problem is being looked at is, "Oh, we are not gonna change how they're trained because it's so expensive and, you know, we spend so much engineering on it.
25:31
We're just gonna patch some partial solutions that are gonna work on a case-by-case basis." But that's, that's gonna fail.

48 MINS LATER

73:34
Mm-hmm.
3:40
And now, Yoshua Bengio takes the TED stage.
3:45
When my son Patrick was around three, four years old, I came regularly into his playroom, and he was playing with these blocks with letters.
3:54
I wanted to... him to learn to read eventually.
3:58
And one day, he said, "Puh." And I said, "Puh." And he said, "Puh?" And I said, "Puh." And then he said, "Puh, puh.
4:12
Whee! Yes!" And then something wondrous happened.
4:17
He picked up the blocks again and said, "Puh.
4:21
Patrick.

10 MINS LATER

speaker_4MODERATOR
14:39
(laughs)

Scroll for more appearances

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.