Anish AgarwalGuestDan TwingHost
One is that it's made the problem worse in the sense that I think troubleshooting these issues can be a lot more complex.

But now you can have a job that completes, but the answer might not be what you're looking for.

And so there's a whole different notion of accuracy that you have to think about as you think about the correctness of a workload.

Second, with all the stochasticity in these systems, even if the system didn't run, troubleshooting why it didn't run becomes much more complex because there's so many different parts of the system and the stochasticity of it hurts your ability to reason over it.

And the third is what we talked about, which is our understanding of the code has dropped significantly because we're not writing it as much anymore.

We're typically designing it, creating specs, and then an AI agent goes and writes it.

And so I think it's all those different things that are making it a lot harder to troubleshoot for the incidents in production.
You know, I realize that I've done some research around what I call the enterprise control plane.
I do a lot with automation and the idea that execution used to mean, for the most part, you're going to get the outcome you want, right? I mean, if it ran, high likelihood it was going to produce the correct output unless somebody did something in a change that we didn't know about or what have you, right? Right.
But depending on what could go wrong, you cannot just trust that reasoning is just because it ran, you got what you expected, right? That is a new challenge, I think.

And if you think about how the coding agents are getting optimized, how they're getting evaled, and then the optimization loop for the coding agents, a lot of it comes down to, did it run? Did it compile and did it run? That is what they're getting trained on.

And so they're finding all sorts of creative ways of making the code run and execute because that's the reward function.

As we think about reinforcement, that's the reward function, make it compile, make it run.

And so they'll find interesting ways of sidelining the true goal and making sure it just compiles.

And I think that also adds another layer of trickiness in this, where the gap between what it should do and the spirit of the law and the letter of the law, that gap in some ways is increasing.
And then, you know, we focus a lot on the code, but in my experience, and I mean, I wasn't in this modern era, but I used to do automated telemachines.

One is that it's made the problem worse in the sense that I think troubleshooting these issues can be a lot more complex.

But now you can have a job that completes, but the answer might not be what you're looking for.

And so there's a whole different notion of accuracy that you have to think about as you think about the correctness of a workload.

Second, with all the stochasticity in these systems, even if the system didn't run, troubleshooting why it didn't run becomes much more complex because there's so many different parts of the system and the stochasticity of it hurts your ability to reason over it.

And the third is what we talked about, which is our understanding of the code has dropped significantly because we're not writing it as much anymore.

We're typically designing it, creating specs, and then an AI agent goes and writes it.

And so I think it's all those different things that are making it a lot harder to troubleshoot for the incidents in production.
You know, I realize that I've done some research around what I call the enterprise control plane.
I do a lot with automation and the idea that execution used to mean, for the most part, you're going to get the outcome you want, right? I mean, if it ran, high likelihood it was going to produce the correct output unless somebody did something in a change that we didn't know about or what have you, right? Right.
But depending on what could go wrong, you cannot just trust that reasoning is just because it ran, you got what you expected, right? That is a new challenge, I think.

And if you think about how the coding agents are getting optimized, how they're getting evaled, and then the optimization loop for the coding agents, a lot of it comes down to, did it run? Did it compile and did it run? That is what they're getting trained on.

And so they're finding all sorts of creative ways of making the code run and execute because that's the reward function.

As we think about reinforcement, that's the reward function, make it compile, make it run.

And so they'll find interesting ways of sidelining the true goal and making sure it just compiles.

And I think that also adds another layer of trickiness in this, where the gap between what it should do and the spirit of the law and the letter of the law, that gap in some ways is increasing.
And then, you know, we focus a lot on the code, but in my experience, and I mean, I wasn't in this modern era, but I used to do automated telemachines.
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.