Amazon Web Services

You can figure out the details for that on packetpushers.net. That is a free resource for your networking needs.

On today's episode, two people who understand AWS networking deeply join us, courtesy of sponsor AWS.

Distinguished engineer and network core VP J.R. Rivers and senior principal networking engineer Stephen Callahan have fired up their microphones to discuss running AI workloads over AWS infrastructure.

Now, AWS, these guys are not new to the unique requirements of AI and a high bandwidth, elephant flow oriented data center traffic patterns.

AWS has had networking features to help with this problem going back to 2019.

2019, you remember that? 2019, we spent most of that year blissfully unaware of coronaviruses.

Yeah.

We've been reading community blog posts, all talking about how networks for AI workloads are different.

This can be hardware, right? Let's say it's a firewall company or a software as a service company, Microsoft Azure.

If Microsoft suffers a wicked big breach and your whole Azure infrastructure gets absolutely throttled, you're going to go to AWS on Tuesday? Good luck with that.

This isn't a video game.

You don't just hit enter and it migrates from Azure to AWS.

That's the most ridiculous thing.

So the reality is you basically take it on the chin and get back to your desk because this is our new reality.

But in this instance, the entire rushing hosting provider.

So this is like a kind of like an ISP or like infrastructure for other businesses, right? They are getting crushed.

They're reinventing search.

They have a cloud business that's accelerating and driving very strong growth for the overall And maybe I'll round it out with Amazon Web Services also seeing extraordinary acceleration.

The key is really this ability to innovate, to invest, to think through on a longer-term basis how to create value for customers and ultimately shareholders.

You

And so that innovation piece, Dom, I think remains essential.

The balance sheet, the cash flow, plus the innovation, in my view, is what's needed to drive growth.

You're the third person to talk about Amazon and what they said in the quarter, 40% growth rates in Amazon Web Services for a quasi-mature business.

And you don't have to have a huge bet on AI.

These networks are built different.

How does AWS see this whole challenge with AI workloads?

Yeah, I think the main thing is there's a lot of different type of AI workloads.

And it's really hard to bucketize everything into three or four to even just say what is the main problem.

Everyone's not running hero runs that are going to take several months across hundreds of thousands of GPUs.

There's a lot of people who are doing much smaller scale stuff, whether it's post-training and creating their own models, doing reinforcement learning, just plain research on new ways of doing it.

There's a lot of workloads that have to exist on something like an AWS ML network.

So we're looking at all of those things and try to solve each one separately, but also not have to develop 20 different products for all the different applications that could be there.

A lot of folks ask me this question.

So AWS can go build this.

They have a large team.

They have like 100 services.

Same thing with Azure or GCP.

So my question back to every one of them is like, if every startup started thinking that way, there will be no startups left.

And so to me, the actual value isn't like, what is the differentiator of the product that I'm building that is gonna be, even if AWS decided to go build something like this, they will not get to.

What is the value that we can add to our customers? That equation has not changed before AI or after AI.

These networks are built different.

How does AWS see this whole challenge with AI workloads?

Yeah, I think the main thing is there's a lot of different type of AI workloads.

And it's really hard to bucketize everything into three or four to even just say what is the main problem.

Everyone's not running hero runs that are going to take several months across hundreds of thousands of GPUs.

There's a lot of people who are doing much smaller scale stuff, whether it's post-training and creating their own models, doing reinforcement learning, just plain research on new ways of doing it.

There's a lot of workloads that have to exist on something like an AWS ML network.

So we're looking at all of those things and try to solve each one separately, but also not have to develop 20 different products for all the different applications that could be there.

And so, again, your point about taking infrastructure, developing it, learning it, and then rolling out their own, exactly what they, they have a history of that.

AWS is the same model.

The same thing.

Why

So the very people that benefit from insolvency and taking down companies are the guys that are drafting the legislation, which is the test to show that a company is insolvent.

And if you were to apply that test to everybody, you would have said, Amazon, AWS, Facebook, LinkedIn, Microsoft, all of the cool startups, Stripe included, were all essentially insolvent when they were raising funding to grow business.

Right? That's it.

so it's not looking at like you're not looking at a sweet shop where somebody's selling i sell five sweets i get paid 20 cents a sweet my costs are 50 cent therefore if i keep selling five sweets a day i'm profitable that is a load of right

And so where Freight's having to quickly learn is like, wait a minute, we have to think in terms of what is our core lane versus what should we be using everyone else's infrastructure for? No one builds their own data center.

It's in AWS, it's in Azure, it's in Google Cloud.

And there was a time when it was like, ah, I don't know about that.

I'm gonna have my own data center and maybe I'll use Azure as a backup.