Neural scaling law
9
MENTIONS
3
EPISODES
3
PODCASTS
Search complete. 9 mentions across 3 episodes found for "Neural scaling law".
Sep 30, 2026
Jack FarleyHOST
It won't end until the models stop getting better the more compute you invest in it.
Jack FarleyHOST
So basically, it won't stop until the scaling laws, which have held up this entire time, end.
Jack FarleyHOST
Do you think that is a reasonable statement?
Ben PouladianGUEST
Yes.
Ben PouladianGUEST
I think until now, the bigger the training run, the better the results, and the more capable the model has been.
Ben PouladianGUEST
And because scaling laws are holding, there's this insatiable demand for compute.
Ben PouladianGUEST
But at the same time, you could also use the corollary that there's an insatiable demand for intelligence, right? Like, when do people want to stop knowing things? Does it go on forever? I mean, possibly.
Ben PouladianGUEST
So if you have an insatiable demand for intelligence, theoretically these trees could grow to the sky, which I don't believe in.
Diogo AlmeidaGUEST
You know, like even my take here is practical and like, am I a scaling law fan? Um, Depends.
Diogo AlmeidaGUEST
Scaling laws tell you how much better you get at a thing for amount in.
Diogo AlmeidaGUEST
Scaling law does mean exponentially more resources for normally sublinear gains, which looks to be a bad investment unless those linear gains are really, really valuable.
Diogo AlmeidaGUEST
But to me, it's all about what can we do with what we have to make the biggest possible fucking difference.
Diogo AlmeidaGUEST
Twitter is just like Jeff, Jeff, Jeff, you know? It's like a party.
Diogo AlmeidaGUEST
It's inspiring
SwyxHOST
because it's so different than what we're used to, which is, I'm sorry you can't do this, but we do scaling laws, and only the big labs can do it, right? That
Diogo AlmeidaGUEST
actually, if I I'm a tangent, if that's okay.