Goodhart's law
72
MENTIONS
38
EPISODES
35
PODCASTS
Search complete. 72 mentions across 38 episodes found for "Goodhart's law".
Oct 27, 2026
Jordan HarbingerHOST
Yeah, interesting.
Jordan HarbingerHOST
So I always worry, what is it called? Goodhart's Law, right? Once passing the test becomes the target, the test starts becoming less useful.
Jordan HarbingerHOST
So if we're just testing crash scenarios where the car is doing well or like works for the average American male, it's not really a useful...
Jordan HarbingerHOST
Because most people aren't...
Richard GillisHOST
uh impetus that it gives to the c-suite and they are the ones where all the difficult problems you know it's their desk where the hard questions land and if you then say right okay what what happens to our fan base who where is our fan base do we believe that the digital numbers are real fans or are they i'm not talking about you know ai bots but there's that as well but there's also Are the fans really fans in a way that, you know, as Harry has defined them, sustained positive attachment to sport and entertainment IP? That's a sort of quite a wonder how many of the fans would ask that bar.
Richard GillisHOST
God knows what I'm a fan of, you know, under the, you know, but when you see the claims for fan numbers increasing, And we all know why they do it.
Richard GillisHOST
It's a commercial game.
Richard GillisHOST
Oh, of course.
Michael HowellGUEST
I mean, once you start to target something, I mean, the targets become meaningless.
Michael HowellGUEST
I mean, that was something I think was called Goodhart's Law, wasn't it? And that probably is the reality.
Michael HowellGUEST
So I think that that may be an explanation.
Michael HowellGUEST
But, you know, you've got, I mean, again, without being, you know, too disparaging, Kevin Walsh's committees and the idea of looking into the CPI stats is another way of fogging the whole question, I think.
Jack RussoHOST
It does.
Jack RussoHOST
But the brief mentions a fascinating roadblock that happens on this rung known as Goodhart's Law.
speaker_0UNKNOWN
Goodhart's Law states that when a measure becomes a target, it ceases to be a good measure.
Jack RussoHOST
Explain how that applies here.
speaker_0UNKNOWN
In the cybersecurity bug bounty world, the measure of success has historically been the number of vulnerabilities found.
Michael JordanGUEST
And I think you can sort of imagine what starts to happen.
Michael JordanGUEST
When that becomes the metric, this is called Goodhart's Law, what the hospital started to do the next year is only admit patients who are not too sick.
Michael JordanGUEST
That happened.
Michael JordanGUEST
All right.
speaker_1UNKNOWN
However, the brief does validate your skepticism, but from a slightly different angle.
speaker_1UNKNOWN
If we look at the overarching mechanics of how businesses measure success, we have to talk about Goodhart's law.
Jack RussoHOST
Ah, Goodhart's law.
Jack RussoHOST
Walk us through the mechanism of that.
Jack RussoHOST
How does it apply here?
speaker_1UNKNOWN
So Goodhart's law is an adage in economics that essentially states, uh, when a measure becomes a target, it ceases to be a good measure.
Jack RussoHOST
Meaning people game the system.
speaker_1UNKNOWN
Exactly.
Igor AlcântaraHOST
I think
Angelika KildasHOST
you even spoke about it, Goodhart's Law.
Igor AlcântaraHOST
Yeah, we spoke about it in one of the older episodes.
Igor AlcântaraHOST
And that phenomenon has a name.
Igor AlcântaraHOST
It's called the Goodhart's Law.
Igor AlcântaraHOST
And so Charles Goodhart wrote that in 1975.
Igor AlcântaraHOST
And he wrote that.
speaker_1HOST
This is a really interesting part of the report.
Jack RussoHOST
It brings up a concept called Goodhart's law.
Jack RussoHOST
Goodhart's law basically states that when a measure becomes a target, it ceases to be a good measure.
speaker_1HOST
Right.
Jack RussoHOST
If a massive trillion dollar company is consistently rewarded by the stock market primarily for flashy demos, for building a hopping car instead of focusing on the gritty, unglamorous unit economics of mass manufacturing, what does that company inevitably optimize for?
Type 3 AudioNARRATOR
Heading.
Type 3 AudioNARRATOR
The default outcome of MechInterp is Goodhart's law.
Type 3 AudioNARRATOR
What happens if we apply optimization pressure to which neurons light up? Depending on strong or direct the optimization pressure gets, that could be the most forbidden technique.
Type 3 AudioNARRATOR
Quote.
Type 3 AudioNARRATOR
Heading.
Type 3 AudioNARRATOR
The default outcome of MechInterp is Goodhart's law.
Type 3 AudioNARRATOR
What happens if we apply optimization pressure to which neurons light up? Depending on strong or direct the optimization pressure gets, that could be the most forbidden technique.
Type 3 AudioNARRATOR
Quote.
28 more episodes mention Goodhart's law.
Create an account to see the whole feed, search across every transcript, and follow the entities you care about.