Unsupervised Learning with Jacob Effron
Jul 9, 2026 · 51 min · 14 segments
Dr. Jürgen Schmidhuber, a renowned scientist and AI researcher widely regarded as one of the pioneers in the field, originated key ideas behind today's transformers, LSTMs, and recursive…
Jürgen SchmidhuberGuestJacob EffronHost
Not to the extent that it surprised people who had no contact to, you know, neural networks and artificial neural networks in the previous decades.
And suddenly there was a a chat GPT moment and suddenly people started being interested in that thing which they had never seen before.
So they didn't know that there was a long history of large language models and an even longer history of basic insights and algorithms for training these large language models that goes back to the previous millennium.
So for a guy who was in the center of all of that, it was much easier to predict that than for someone who had totally different interests.
Well, you know, I definitely want to hit on recursive self-improvement in meta-learning because I feel like you've, you know, it's been a huge focus of yours for a while.
How do you kind of articulate the, you know, where we are today on the path of getting to RSI and kind of what still needs to be solved?
So in 1987, this was about using meta evolution, evolutionary programming for evolving better programs that learn to do better kinds of evolution, meta evolution, I call it.
And so over time, you had better and better learning algorithms, learning to combine code from previous programs and in better and better ways to solve problems better and better.
reinforcement learning techniques, self-referential machines that were able to They basically were using a universal programming language to generate arbitrary self-modifications of the code that was running the machine, interacting with some environment.
generating self-improvements by a machine that has some software and it's interacting with an environment.
And this environment, um, you know, sometimes punishes you or provides a reward and you want to maximize the sum of all the rewards in your life until the end of your life.
And this machine then, um, can in principle write programs that modify the initial software but before it modifies itself like that it first has to prove which means there's a proof search in the software it has to prove that the um that the modification that will be caused by the execution of this particular program is useful in the sense that it will lead to more expected reward than um than the alternative, which would be not to execute this program.
And then it has to generate a formal proof, which was called the Gödel machine.
It is less practical than certain other things that we did, like, you know, neural networks that change their own weight matrix by running a learning algorithm on the network itself.
That is what we started in 1992, and that's currently more or less the most popular kind of self-modification
Not to the extent that it surprised people who had no contact to, you know, neural networks and artificial neural networks in the previous decades.
And suddenly there was a a chat GPT moment and suddenly people started being interested in that thing which they had never seen before.
So they didn't know that there was a long history of large language models and an even longer history of basic insights and algorithms for training these large language models that goes back to the previous millennium.
So for a guy who was in the center of all of that, it was much easier to predict that than for someone who had totally different interests.
Well, you know, I definitely want to hit on recursive self-improvement in meta-learning because I feel like you've, you know, it's been a huge focus of yours for a while.
How do you kind of articulate the, you know, where we are today on the path of getting to RSI and kind of what still needs to be solved?
So in 1987, this was about using meta evolution, evolutionary programming for evolving better programs that learn to do better kinds of evolution, meta evolution, I call it.
And so over time, you had better and better learning algorithms, learning to combine code from previous programs and in better and better ways to solve problems better and better.
reinforcement learning techniques, self-referential machines that were able to They basically were using a universal programming language to generate arbitrary self-modifications of the code that was running the machine, interacting with some environment.
generating self-improvements by a machine that has some software and it's interacting with an environment.
And this environment, um, you know, sometimes punishes you or provides a reward and you want to maximize the sum of all the rewards in your life until the end of your life.
And this machine then, um, can in principle write programs that modify the initial software but before it modifies itself like that it first has to prove which means there's a proof search in the software it has to prove that the um that the modification that will be caused by the execution of this particular program is useful in the sense that it will lead to more expected reward than um than the alternative, which would be not to execute this program.
And then it has to generate a formal proof, which was called the Gödel machine.
It is less practical than certain other things that we did, like, you know, neural networks that change their own weight matrix by running a learning algorithm on the network itself.
That is what we started in 1992, and that's currently more or less the most popular kind of self-modification
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.
