Cerebras Wafer Scale Engine

And Cerebras is one of the more unique companies, uh, that's connected to OpenAI because it's not simply trying to build another traditional like GPU.

Okay, so what Cerebras does is they designed a processor called Wafer Scale Engine.

So when you have traditional chips are essentially manufactured on a large silicon wafer, and then they're cut into like these small t- small, tiny, smaller chips.

Cerebras, if you look at the screen, is basically uses much more of the wafer as like one enormous processor.

To me, I think a lot of them are trying to solve the fundamental problem of memory latency.

And And with a variety of different approaches, whether it's like Wafer Scale Engine or the like SRAM heavy LPU chips from Grok.

So that good short term solution, I do think that's actually something that NVIDIA either by acquiring or by building a similar system themselves will be able to figure out.

For the second question, which is like, oh, yeah, what happens to algorithms if we start specializing the hardware more? I think Sarah Hooker's statements about the hardware lottery are as true now as they were when she put that paper out, like, what was that, five years ago? There is, like, our algorithms are going to be constrained by the hardware that they can run on.

The compute units that are doing the matrix multiplies can pull data from SRAM at mind-boggling speeds.

Cerebras quotes petabytes per second, 21 petabytes per second for their Wafer Scale Engine 3.

Compare that to HBM on an Nvidia Blackwell is, uh, 10 terabytes per second or so, in that range.

So once again, many orders of magnitude difference, more capacity, but proportionally less bandwidth essentially.