Gradient descent

Exactly.

This is gradient descent.

You recalculate the slope, take another step, and repeat this until the ground goes completely flat.

When it's flat, you know you've reached the bottom of the bowl.

That is gradient descent in a nutshell.

But it introduces a really tricky problem for the engineers building these models, determining how big of a step the hiker should take.

Right, because if I'm blindfolded on a mountain, my step size matters a lot.

But the textbook warns of one final major trap here.

Laying it all out.

And then over time, the ability to weigh those is I think reinforcement learning in some sense to become good at feeling those, gradient descent your way.

Yeah, gradient descent your way to that.

And without some reps, I don't know that you should, you might have the intuition, the intuition might not be right, but I think to get confidence in it for myself, The gradient descent has been important versus trying to be analytical, purely analytical about it.

And so it might now align with a lot of what was my instinct or gut along the way, but also you develop more feel for it.

It's important.

One is...

You're listening to Gradient Descent, a show about making machine learning work in the real world.

And I'm your host, Lucas Biewald.

Today, I'm talking with Drew Baglino, an old friend of mine who started as a junior engineer at Tesla and ended up running all of Powertrain and Energy, which is most of the company, and reporting directly to Elon.