NVIDIA's mode in three numbers.

If you actually do the math for the amount of hardware that is in the world today, all the different workloads that we're running, the different numerical cells, the different fusions, the different ways of partitioning them, right? And you think about different batch sizes, you think about latency versus throughput or general throughput, other SLOs.

You actually sit down and do that and you're like, okay, I need to write about 106 billion kernels in order to get coverage.

Well, there's only about 2,000 odd performance engineers in the world that actually know how to write good kernels.

90% of them are inside of one vendor ecosystem.

So there's still a problem of I need coverage.

The reason you can accelerate some of this on NVIDIA is because NVIDIA spent 20 years building an ecosystem of tools to make your life easier so you could get the feedback.

The answer tied his whole world back to the build-out.

Is that correct?

So kind of.

Well, I'll get into it in a sec.

So when you think about hardware, the hardware has a certain execution model.

That execution model is how the hardware wants to actually run this code efficiently.

Your job as when you're writing a compiler, you are connecting a programming model, which is the way a developer writes something in a way that they understand.

Then you need to lower this down to the machine.

There's a saying

Yeah, me too.

Well, yours works better than mine does.

Mine's an absolute mess.

The We had to go and read all the prior literature, understand everything that has been tried, when it worked, when it didn't, what was true at the time, what's changed, right? There is a conflict that exists between, you know, having an adaptive runtime that can do a lot of things versus the kernel model.

One wants dynamicity, the other wants static, right? And if you have over-optimized in one case, you have limited the other.

And if the thing is data movements, then you want to be a bit more runtime-centric and less kernel-centric because that just gets in the way.

But if you're doing all these other things, then that was sort of the realization that the thing that has held back all these other ideas is the fact that we kept falling back to kernels and no one fully committed to the post-kernel world.

So

Um, what signals convinced you that we've hit the ceiling of what hardware-first scaling can deliver? And what made you confident enough to build a company around, uh, the alternative?

Uh, so there's a few things in there that I, I'd wanna see if we can, like, uh, push aside and, like, sort of like, um, disambiguate.

So there's one premise in there that hardware has hit a ceiling.

I think firstly, the hardware that we have is absolutely incredible, and the fact that we are even here is by itself a miracle on top of miracles.

What we've actually hit is sort of more of a comprehension ceiling.

So the gap between what hardware can do and what the software understands and how we use it.

So the problem is not so much hardware as it is software, and that gap keeps widening every generation, right? So you look at today's systems, you know, a lot of for training workloads, people are talking about anywhere between forty-five to fifty percent utilization, and that's at, like, some of the frontier labs where n- they have the best people working on this for months at a time, and they know how to, like, right-size their workloads and their runs for the system that they have.

You know, the software-first AI infrastructure sounds abstract, but i- in concrete terms, what changes for an ML team running production workloads on Lemurian stack, you know, in how their models get compiled, scheduled, executed that they can't get from today's tools?