Jul 6, 2026 · 36 min · 13 segments
In this episode, we sit down with Alexa Griffith, Senior Principal Software Engineer at Red Hat, to discuss what’s really happening behind the scenes of modern AI infrastructure. From her journey…
Alexa GriffithGuestRob MayHost
I'm still in open source, but a little bit lower in the stack, which we can get into.
For the last part of me being at Bloomberg, I was there four and a half years working on their inference platform.
What we were doing with the Envoy Gateway and the problem that we were solving is that when Gen AI came into play and our AI engineers wanted to build products on LLMs to get more resources and to get experimentation going and building those products, we need to have access to them, so that means that we were using different model providers, so GCP, Azure, AWS, but all of this traffic is going outside of Bloomberg's system, and that needs to be secure, we need to manage it, and we need to be able to have observability, know everything that's going on, and as a platform team, ideally it needs to be easy to use.
And if you use all of these different providers just as they are, none of those things are true.
So we run this instance and all the traffic passes in and out from us so we can control it.
So we created a central unified API and then we translate it within the gateway.
It's like a central place to track how many tokens you've used and rate limiting, things like that.
I'm still in open source, but a little bit lower in the stack, which we can get into.
For the last part of me being at Bloomberg, I was there four and a half years working on their inference platform.
What we were doing with the Envoy Gateway and the problem that we were solving is that when Gen AI came into play and our AI engineers wanted to build products on LLMs to get more resources and to get experimentation going and building those products, we need to have access to them, so that means that we were using different model providers, so GCP, Azure, AWS, but all of this traffic is going outside of Bloomberg's system, and that needs to be secure, we need to manage it, and we need to be able to have observability, know everything that's going on, and as a platform team, ideally it needs to be easy to use.
And if you use all of these different providers just as they are, none of those things are true.
So we run this instance and all the traffic passes in and out from us so we can control it.
So we created a central unified API and then we translate it within the gateway.
It's like a central place to track how many tokens you've used and rate limiting, things like that.
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.