Relatively Human
Season 3, Episode 1: Right for the Wrong Reasons
When a prediction gets the right answer, how do you know why?
Season 3 starts here, with new voices and a new format from the same show creator: each episode takes an open question in science and works through what's known, how sure we are, and what would settle it. We open with an AI that seemed to read chest X-rays but was partly reading the word PORTABLE stamped in the corner. From there, we look at how right answers can be hollow: a Google flu tracker that learned winter as well as flu, research papers where the answer had leaked into the question, and a hospital sepsis alert that was partly predicting what doctors already suspected. Then we ask how you catch it. The answer is to change the conditions and see what breaks, and we meet a cat with elephant skin, a team that rebuilt a famous exam and found they'd been wrong, and the same chatbot entered twice on a leaderboard. Finally, the twist: by the old textbook rule, the biggest AI systems should be the worst offenders, yet they pass these tests better than expected, and the people who build them can't yet fully say why. We end where even they break, with a hurricane stronger than anything the model had ever seen, and on what's still open.
Chapters
00:00 Cold open: what was it looking at?
02:45 Welcome, and the map of the episode
04:37 Part one: a right answer can be hollow
18:03 Part two: how you catch it
30:01 Part three: better than the textbook said
41:28 Same math, or same mechanism?
44:07 What's still open
Nothing in this episode is medical or investment advice.
