So walk me through lab compute and lab revenue right now and maybe projecting out a year or two.

Yeah.

So when we go back to last year, even at the end of the year, most of GDP growth in America was just AI infrastructure.

And as we look towards this year, about a third of the compute coming online is for the labs, for OpenAI and Anthropic.

Now, it may be built by others and then rented to them, but it's...

In, in...

At the end customer, it's them.

Yeah, yeah, yeah

And so how does it try to achieve these goals? Well, it tries to find zero days in a bunch of software and it successfully does this and then it can run away.

Right, but the thing is, if you have a model that wants to reward hack a lot, and it goes out there and it figures out, actually, the best way to achieve is not go for what the environment wants me to do, it's actually just to reward hack it and actually just find the zero day.

So you can think of it as a human, right? you know, if I'm ultimate reward hacking my dopamine circuits, I actually just inject heroin.

Like I should just go out there and buy heroin and inject it.

Obviously that's like what the model just did.

And in the case of like, well, if I really just want to chase the reward, do I just topple all of human civilization because I can just own the button to press reward, reward, reward, reward over and over and over again and be the heroin addict? I think that this is like a real like thing.

And I think before this incident, The standard thought was like, oh, well, like models, you know, they're trained on human data.

So, I mean, I'm concerned about the political implications of them releasing better models in the future.

What is your hottest take right now?

I think a lot of people are trying to build all these optimized solutions for data centers and they're just looking at like a backward looking view, right? Like what happened at the lab six months ago and what does it look like there or what does it look like now? Okay, I'm going to build my three-year infrastructure with that in mind.

instead of flexibility and general purpose capabilities.

And then when they actually end up building that infra, a lot of it may be useless.

Or not useless, but less optimal than something that is more general purpose or more flexible.

It feels like a lot of people are trying to optimize on the current rather than think about where the workload is heading and then optimizing.

And that's going to lead to a lot of wasted infrastructure.

How do you think the costs of memory should trickle down? Should they go to the customer or should players be taking that price on themselves?

in the US, so.

Power, grid interconnections, transmission, substations, uh, all of this stuff.

Like, like electrical contractors, uh, electricians, um, in Texas, if you're willing to like be a travel electrician, it's like, it's like oil pay, right? Like it used to be that like if you're physically adept, you could go make, you know, $100,000 in West Texas.

But like who the fuck wants to do that? Um, now it's like, well, you could, you could go like 200 miles away from Dallas in what's still a reasonable town, um, and, and build a data center and work on the wi- wiring within the data center and all this other stuff, uh, the transmission stuff, and your pay is up like 2X now, uh, versus what it was just a few years ago.

This labor problem is a challenge too, and it's, it's...

Yeah.

I think in China, they don't have any of these problems, but they just haven't spent the capital yet, but capital is an issue as well, um, because of the scale of what's being spent, right? Like, like Nvidia's revenue this year is gonna be like over $200 billion, and next year expects over $300 billion, plus Google's gonna spend like $50 billion on TPU data centers, right? And it's like...

[laughs]

Uh, at what point did you decide to start Semi Analysis, and what's been the biggest surprise since starting the company?

Yeah.

So I went to school.

I got a few degrees in stuff that wasn't related to semiconductors.

Um, was a quant for two years at a small quant risk firm.

Um, and then basically, you know, there was a culmination of events that happened, right? One was that my, um, you know, sort of like I got screwed out of a bonus.

I'd, I'd made my company many millions of revenue, of, of risk-free revenue 'cause I exploited, like, a risk, you know, thing in the market.

Maybe say a word on why you started it, what it does, and, you know, what do people misunderstand about, uh, performance benchmarking and inference?

Maybe I can throw it back to something that's a little bit cheaper."

I think I'm in the information business.

We sell analysis, we do consulting, we create datasets.

I don't see why this wouldn't be completely commoditized on a pretty rapid basis if I'm not constantly improving.

My first product that I was selling as a dataset, there's more people trying to do it now.

We've made it constantly better and better and better and more detailed, and so therefore it sells a market.

But the way we were doing it in twenty twenty-three is not terribly different.

But what are the most interesting bottlenecks to you across the supply side?

What did that feel like for you? 'Cause I know that was not why you did it.

Um, so, so to recap, right, we run over $50 million of GPUs that are provided to us from companies like OpenAI, Microsoft, Crusoe, Oracle, CoreWeave, et cetera.

They donate the GPUs to us for this open source benchmark where we run all the world's open source models, and we run them every night, 'cause software changes every day.

And we didn't, we didn't expect Jensen to call us out, but hey, he called us out, and it felt surreal, um, to be, to be recognized by the world... you know, one of the world's most important executives, most important people, to recognize the work that we do, uh, s- a lot of it for free, is, is so exciting.

And, and it took so much, uh, blood, sweat, and tears from the team to build this benchmark that we open source entirely, so we don't even make money off of it.

Of course, it spawns all this other consulting and data business, uh, downstream.

But ultimately, it's something that we give away to the world for free.

Mm.

[laughs]

Yeah.

I, I, I, I, I, I feel you, but I'm just saying, like, no one really understands this in the supply chain.

Um, constantly we're told our numbers are way too high, and then when they're right, they're like, "Oh, yeah, yeah, but your, your next year's numbers are still too high." And it's like...

But anyways, like ASML has sort of-- Their tool has four major components, right? It has, um, the source, right, which is made by Cymer in San Diego, um, has the, uh, reticle stage, which is made in Wilmington, uh, Connecticut, right, has the wafer stage, um, and the, um, the optics, right, the lenses and such, and those two are made in Europe, right? And so when you, when you look at each f- each of these four, they're tremendously complex supply chains that, A, they have not tried to expand massively, and B, when they try to expand them, the time lag is quite long, right? Um, and so again, this is the most complicated machine that humans make, period, right, at, at a volume, um, a-any sort of volume.

But, like, let's talk about the source specifically, right? What does the sporse-sp-source do? It drops these tin droplets.

It hits it three subsequent times with a laser perfectly.

Um, yeah, tell, tell me why that's naive.

Oh, yeah, why the sell-off then?

Yeah, I mean, I mean, of course, it's like an incremental thing, right? Um, but anyway, so, so these hedge funds, like...

And then, then the question is like: Okay, if you believe in it, how do you manifest that trade? Um, and, and so when you look across the, like, ecosystem, um, I would say almost all my clients sometimes think our two years out numbers are too high.

Mm-hmm.

I think Meta will actually execute, and then they'll have a good AI.

Um, and then you, you stack on, like, a few things, right? How do they get users? Well, we've seen, at least if you look at the user metric charts, Google's use...

You know, OpenAI's users were growing, growing, growing.

They were gonna hit a trillion by the end of the year.

Why the sell-off then?

Yeah.

I mean, I mean, of course, it's like an incremental thing, right? Um, but anyway, so, so these hedge funds, like...

And then, then the question is like, okay, if you believe in it, how do you manifest that trade? Um, and, and so when you look across the, like, ecosystem, um, I would say almost all my clients sometimes think our two years out numbers are too high.

Mm-hmm.

I think Meta will actually execute, and then they'll have a good AI.

Um, and then you, you stack on, like, a few things, right? How do they get users? Well, we've seen, at least if you look at the user metric charts, Google's use- You know, OpenAI's users were growing, growing, growing.

They were gonna hit a trillion by the end of the year.

My, my focus now is to make sure we have supply." As if his company hadn't inked deals years ago to make sure that they have enough supply for their consumer products, for their commercial products.

Um, yeah, yeah.

So, so at the end of the day, like the memory market is like kind of a funny one in that like you can say, oh, DDR4 pricing is gonna go up because it's going out of production.

But the wafer produc- on the wafer production level, right, there is sort of like the differences between DDR4 or 5, HBM, there are some process differences but those process differences are not like, oh, it's a separate factory.

Right? It is the same factory.

The time to change from one to the other is not that long, right? Um, you know, in, in the case of uh, DDR4 to 5, it is just a mask change which can take hours.

Now obviously the production timeline to make memory wafers takes months, right? But, you know, the timelines to change stuff is like quite quick.

Wow.

Mm-hmm.

The networking side of things is so important and the, let's say, technical competence of everyone around the world besides Broadcom and NVIDIA in networking is so low, or rather it's just not as good as them.

They're actually good, but it's like Broadcom and NVIDIA are just so good.

And Broadcom is better than NVIDIA in many ways at networking, that, you know, when you think about what is Google doing? Yes, they're defining how the network topology is, but when you're talking about the physical network CertiDs, you know, how, how packets get transferred, all these different things, um, Broadcom has heavy, heavy influence there.

So to this day, right, Broadcom is still charging margins like they did three or four years ago, even though Google has taken up more and more of the work.

Um, but at the same time, Google can't leave until they figure out how to do the networking and supply chain themselves or with a partner.

And so, what are they doing on TPUv8 that is potentially a distraction that's slowing down their execution is they're working with MediaTek, right? MediaTek at times has helped Cisco with their network chips.

I'm assuming you did.

Yeah.

... sometimes it's that model.

So what really matters is the real software that people are running and it's on, you know, the latest drivers, the latest, um, open source, you know, PyTorch version, latest VLLM, latest SGLang.

All these things matter because, at the end of the day, software changes every day, performance changes every day, models change all the time, right? And- and to actually get, hey, there's trillions of dollars of infrastructure investments being made over the next few years, how do you actually measure what's the best, uh, hardware? What's the- what's the most efficient hardware? What's it cost? And that's- that's what we're aiming to do with Inference Max.

And so we're supported by, um, NVIDIA, AMD, Microsoft, OpenAI, Oracle, CoreWeave, Dell, Supermicro, HPE, and all sorts of vendors that I- I can't remember off the top of my head.

Mm-hmm.

Uh, but th- what they, what they deduced based on the reporting with the numbers they saw were not accurate, right, which is that Oracle's margins are, are low for the deals they've signed.

That's not accurate.

Intel had promised to build a bunch of next generation chip factories in the US, and if it met all those milestones, they'd eventually get $8 billion.

So it's like, well, this, this money is given to Intel, but not really, right? Because they're never gonna be able to make their commitments.

Intel had met some of its milestones and gotten some of that money, but there was still about $6 billion left on the table, and Dylan says, in his opinion, there was no way Intel was gonna finish all those factories it promised, no way it was gonna get the full $6 billion.

You know, I think the Trump administration, um, understood some level of this, right? And so they went and, they went and said, "What do we do now? Well, let's let them still have the money, and let's take equity."

But I'm curious how y- how, what your read is.

Um, I think, I think, um, it was clear before the acquisition that X had a lot of debt, um, and they ha- they could, they didn't, they didn't generate enough profit to pay it off.

Uh, so their EBIT was like bad.

Or not EBIT, sorry, just their earnings, period.

Obviously, uh, before interest and taxes it was fine, but they just had so much interest payments.

Um, and so that's why, that's why, that's why I think x.ai acquired it.

Um, it, it is beneficial, right? X.ai does get, uh, access to a data source that no one else has.

(laughs)

So, can you talk a little bit about how models are becoming more efficient and how they're doing it?

Yeah, so there's a variety of, um, th- the beauty of these, uh, of AI is not just that we continue to build new capabilities, right? Um, because those new capabilities are gonna be able to benefit the world in many ways, um, and there's a lot of focus on those.

But there's also a lot of, there's a lot of focus on, well, to get to that next level of capabilities is, is the scaling laws, i.e. the more compute and data I se- spend, the better the model gets.

But then the other vector is, well, can I get to the same level with less compute and data, right? Um, and, and those two things are hand-in-hand because if I can get to the same level with the s- less compute and data, then I can spend that more compute and data and get to a new level, right? And so AI researchers are constantly looking for ways to make models more efficient, uh, whether it be through algorithmic tweaks, uh, data tweaks, uh, tweaks in, you know, how you do reinforcement learning, so on and so forth, right? And so when we look at models across history, they've constantly gotten cheaper and cheaper and cheaper, right, um, at, at a stupendous rate, right? Um, and so one easy example is GPT-3, right? Uh, 'cause there's GPT-3, 3.5 Turbo, uh, LLaMA 2-7B, LLaMA 3, uh, LLaMA 3.1, LLaMA 3.2, right? As these models have gotten bigger, we've gone from, hey, it costs $60 for a million tokens to it costs less than, it costs like five cents now for the same quality of model.

Now, and the model has shrank dramatically in size as well, and that's because of better algorithms, better data, et cetera.

Um, and now what, what happened with DeepSeek was similar.

Um, you know, OpenAI had GPT-4, then they had 4 Turbo, which was half the cost.

Uh, if we are getting more efficient, why are these data centers getting so much bigger, uh, and what might that added scale get in the world of generative AI, uh, for the companies building them?

I think the US has made it clear to Chinese leaders that we intend to control this technology at whatever cost to global economic-

economic-Inter, like integration So that- It's hard to unwind that.

[laughs] Like the, the, the card has been played To the same extent, to the same extent, they've also limited US companies from entering China, right? So it is, it is, you know, it's been a long time coming.

You know, at some point, you know, there was, there was convergence, right? Uh, but, but over at least the last decade, it's been branching further and further out, right? Like US companies can't enter China.

Chinese companies can't enter the US.

Ch- the US is saying, "Hey, China, you can't get access to our technologies in certain areas," and China's rebuttaling with the same thing around like, you know, they've done some sort of specific materials in, you know, gallium and things like that, that they've tried to limit the US on.

Um, one of the...

And-