The Context Window with David Deming Podcast
Oct 14, 2026 · 33 min · 8 segments
What happens when A.I. starts improving itself faster than humans can keep up? The researchers building these models can't fully explain how they work. If those models begin designing their…
David DemingHost
If RSI really does lead to sustained takeoff, AI could become much more capable than humans across a wide range of tasks, and it would be cheaper, too.
And control is especially hard with AI because even the researchers in the top labs who built these models don't really understand how they work.
As the science fiction writer and author of 2001: A Space Odyssey, Arthur C. Clarke famously said, "Any sufficiently advanced technology is indistinguishable from magic." And our inability to understand how AI works has led to some magical thinking, like the idea that we can make AI safe by aligning it to our interests so that it always knows just the right thing to do.
I don't think we're ever going to truly solve the alignment problem because human morality is not a set of rules that can be written down, and it can't be learned from past behavior because it's always changing.
The labs work on both alignment and control, but I worry they're relying too much on alignment, on models behaving well and not demanding enough evidence that safeguards will hold when they don't because they've given up hope that they can control powerful AI.
But we can't rely on alignment to save us, especially if we let AI models improve themselves.
If we let a system we don't understand mutate and multiply, any small seed of bad intention becomes so much worse as the models get more capable.
We should not allow AI labs to engage in recursive self-improvement unless we first hold them accountable to much higher safety standards.
Force them to engineer safeguards and accountability systems that minimize the consequences of the inevitable failures that will come from developing powerful AI.
I'm encouraged because even industry leaders with every reason to keep things moving, like Jensen Huang and Satya Nadella, are starting to talk this way and push us in this direction.
If RSI really does lead to sustained takeoff, AI could become much more capable than humans across a wide range of tasks, and it would be cheaper, too.
And control is especially hard with AI because even the researchers in the top labs who built these models don't really understand how they work.
As the science fiction writer and author of 2001: A Space Odyssey, Arthur C. Clarke famously said, "Any sufficiently advanced technology is indistinguishable from magic." And our inability to understand how AI works has led to some magical thinking, like the idea that we can make AI safe by aligning it to our interests so that it always knows just the right thing to do.
I don't think we're ever going to truly solve the alignment problem because human morality is not a set of rules that can be written down, and it can't be learned from past behavior because it's always changing.
The labs work on both alignment and control, but I worry they're relying too much on alignment, on models behaving well and not demanding enough evidence that safeguards will hold when they don't because they've given up hope that they can control powerful AI.
But we can't rely on alignment to save us, especially if we let AI models improve themselves.
If we let a system we don't understand mutate and multiply, any small seed of bad intention becomes so much worse as the models get more capable.
We should not allow AI labs to engage in recursive self-improvement unless we first hold them accountable to much higher safety standards.
Force them to engineer safeguards and accountability systems that minimize the consequences of the inevitable failures that will come from developing powerful AI.
I'm encouraged because even industry leaders with every reason to keep things moving, like Jensen Huang and Satya Nadella, are starting to talk this way and push us in this direction.
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.