Sep 1, 2026 · 33 min · 11 segments
The easy part of voice AI takes 10 minutes. The hard part is everything that follows. Voice AI can handle a basic appointment in 10 minutes - but handling tens of millions of calls without failing is…
Brooke HopkinsGuestBoris EpsteinHost
So my team was responsible for How do we actually launch and run simulations at scale? So all of the developer tools for, you know, how do you take a data set, combine that with some configuration and then run that on distributed compute? And yeah, it was really the foundation of a combination of voice AI in these early days and then the autonomy and self-driving and how do you really scale an autonomous system proved to be like the perfect combination to then build in voice AI evals.
But great, you know, experience building, you know, obviously what turned into something very valuable right now.
Yeah, it's interesting because before finding our first customer in voice AI, I didn't see how similar the two were.
And I think even the industry was just so early, it was hard to see where it would go.
But now I think a lot of the same principles that came from self-driving are showing up in autonomous voice agents.
So in the same way that you have with self-driving or any robotics, you have perception, planning, and then controls.
So what is happening in the world around me? What should I do next? And then how do I actually take that action? And with voice AI, you have the same dynamic where you have transcription, like what is happening in the world around me? What signal should I take in? And then reasoning and LLM, what should I say next? And then voice.
So how do I actually say that thing out loud? And so as a result, both the architectures on the voice agent side look very similar and But most importantly for Koval, the architecture or like how you actually test and deploy these systems and gain the confidence that this is ready, you know, ready to deploy for safety critical applications.
That's, you know, simulation, observability, having feedback loops, having, you know, a constant eval loop and a really strong process around releases, right? is paramount.
And I think a lot of people are thinking about this as, you know, simulation and evals are equivalent to unit testing and software.
But I actually think it's much closer to the core product where today with agents, anyone can build a voice agent that, you know, handles appointments basically in, you know, 10 minutes.
But to get it to be able to handle tens of millions of calls, tens of millions, you know, like this, the last 20% of edge cases, that is really the crux and the hard part of this product.
And so one of the most important parts is how do you define exactly what your product should be doing? And that's encompassed in how you define your evals.
Like what benchmarks are you optimizing for? What levers are you pulling? Because if you're trying to be good at everything, everywhere, all at once, you're actually building a foundational model.
But if you're building these really task specific agents, how do you determine like what you're trying to be good at and what you're willing to sacrifice in order to get there?
So my team was responsible for How do we actually launch and run simulations at scale? So all of the developer tools for, you know, how do you take a data set, combine that with some configuration and then run that on distributed compute? And yeah, it was really the foundation of a combination of voice AI in these early days and then the autonomy and self-driving and how do you really scale an autonomous system proved to be like the perfect combination to then build in voice AI evals.
But great, you know, experience building, you know, obviously what turned into something very valuable right now.
Yeah, it's interesting because before finding our first customer in voice AI, I didn't see how similar the two were.
And I think even the industry was just so early, it was hard to see where it would go.
But now I think a lot of the same principles that came from self-driving are showing up in autonomous voice agents.
So in the same way that you have with self-driving or any robotics, you have perception, planning, and then controls.
So what is happening in the world around me? What should I do next? And then how do I actually take that action? And with voice AI, you have the same dynamic where you have transcription, like what is happening in the world around me? What signal should I take in? And then reasoning and LLM, what should I say next? And then voice.
So how do I actually say that thing out loud? And so as a result, both the architectures on the voice agent side look very similar and But most importantly for Koval, the architecture or like how you actually test and deploy these systems and gain the confidence that this is ready, you know, ready to deploy for safety critical applications.
That's, you know, simulation, observability, having feedback loops, having, you know, a constant eval loop and a really strong process around releases, right? is paramount.
And I think a lot of people are thinking about this as, you know, simulation and evals are equivalent to unit testing and software.
But I actually think it's much closer to the core product where today with agents, anyone can build a voice agent that, you know, handles appointments basically in, you know, 10 minutes.
But to get it to be able to handle tens of millions of calls, tens of millions, you know, like this, the last 20% of edge cases, that is really the crux and the hard part of this product.
And so one of the most important parts is how do you define exactly what your product should be doing? And that's encompassed in how you define your evals.
Like what benchmarks are you optimizing for? What levers are you pulling? Because if you're trying to be good at everything, everywhere, all at once, you're actually building a foundational model.
But if you're building these really task specific agents, how do you determine like what you're trying to be good at and what you're willing to sacrifice in order to get there?
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.
