Mellanox Technologies

You could basically do that or fiber.

So I had Mellanox cards.

Mellanox is a high-end enterprise-level networking company that is now owned by NVIDIA, but they've been making very fast network cards for servers, like enterprise data center-type servers for a very long time.

Uh, one of the two Melanix cards I was using stopped working in the one in my windows PC.

Oh

So the CTO, the chief technology officer of NVIDIA, OK, so like one of the three most important positions in this global AI monolith, NVIDIA, the CTO is Israeli and lives in Israel and he does his job out of Israel.

Why? Because he used to be the CEO of a company called Mellanox.

which Nvidia acquired.

He was the CEO of Mellanox.

Nvidia bought Mellanox.

Not only did they integrate Mellanox into Nvidia, but they made the CEO the CTO, and he runs the company.

He runs his whole part of the company out of Israel.

So now Israel is indispensable to Nvidia, which is indispensable to the AI revolution.

It took us a lot of time to get the MVP out because we had to find a standard to work with.

And many of the established companies back then, like Mellanox, Broadcom, Cisco and others were a little bit resistant, I would say, to the new protocol, to the new standard.

Today, fast forward, 10 years later, every storage vendor supports NVMe over TCP.

It's a standard and we are very proud of that.

NVIDIA's revenue is flat.

And then back in 2019, NVIDIA had bought a company called Mellanox in Israel, which allowed you to just use high-speed networking to connect a bunch of servers, inventing the modern AI cluster.

Okay.

2022 comes around, ChatGPT.

So first, I've done two companies.

Both went public for hundreds of millions of dollars and both were sold for billions of dollars, both Galileo Technology and Mellanox Technologies.

So here's what I think.

This huge crisis that we had on October 7, 2023, we should use this as an opportunity.

So we recognized very early on the way that you connect these together will determine to the extent that w- to which you can deliver more compute and therefore, you know, more advanced AI.

So when we acquired Mellanox, you know, the, I, I think most people didn't fully appreciate the value that was there in terms of, you know, not only just the existing technologies and the work they had done with InfiniBand and making that sort of an industry standard, but obviously the ingenuity that would go into building out what is now our scale up architecture.

And then as you started to think about as these models begin to grow in size and complexity, on the inference side, like I said, rack scale becomes important.

But for training, now you're looking at multi-cluster training sites, right? When you think about, you know, for a lot of, um, training workloads, the biggest, biggest objective is get the largest cluster you can get and, you know, get it to iterate and, and converge as quickly as possible so that you can quickly get to a model that then starts generating revenue in inference.

I started with one interface, mce1, and planned to add the other three after capture worked.

The capture interfaces are backed by a Mellanox ConnectX adapter handled by the mlx5en driver, and they use a jumbo frame MTU of 9,000 bytes." And just, just Ruben interjecting, I'm, I'm sure most listeners are aware of, um, Suricata.

If you're not, it's a, um, basically a network analysis tool that you can...

It presents a nice dashboard so you can keep track of incursions and things.

The ring size change finally lets Suricata register MCE1.

Um, so he's got one more section here called four interfaces.

"With one interface working, I sail- I scaled this setup to four Mellanox interfaces, all using jumbo frames with an MTU of 9,000 bytes." And he's got, uh, some output here.

"No more Netmap underscore buff objects, and no more buffers after 4,096 of 4,096.