Why OpenAI Should Be Scared of Open Source

So, Trin, you mentioned a lot of early crypto infra projects back in 2023, especially the rented GPU ones.

The idea that if you're running a big Bitcoin mining network, you could kind of flip a switch and serve AI inference, host models, things like that.

In hindsight, it does seem really naive, right? Because the models...

the open source models weren't even around to serve, right? We've only like recently the last year or two gotten a really good open source models from China.

So it just seemed, yeah, it definitely seems a bit naive in hindsight.

Yeah, like I said, I think a lot of people saw the success of existing miners converting their operations.

And then they also saw, to some extent, stable diffusion as an example of someone who is running a mining operation, converting their existing GPUs into making a model.

I think what people didn't realize at that time was that language models would just have a lot of idiosyncrasies that image models didn't.

And no one really wanted a decentralized image model, to be honest.

No one seemed to care about that that much.

And I also just think it came from crypto as an open marketplace.

And so people are inherently lazy and they're like, In the sense that they're just like, well, hey, I could launch a token for saying that I'm doing this thing that probably won't work, but the token will still have some valuation.

So I think in the beginning, there were a lot of disingenuous projects that were just meme coins with more steps, as I like to call certain types of projects.

I love meme coins with more steps.

I don't know.

uh and uh but in underneath all of that you know as someone who had um you know worked in sort of centralized hardware stuff 15 16 years ago and then you know at the beginning i was like oh the decentralized things are all scams and then after like four or five years i like figured finally understood sort of a little more of like, why do blockchains work? Like what is different about them? Why do all these kind of things that you're taught in school about distributed systems in like grad school classes, like why are they violated but still seem to work? And there's a sense in which because I was in crypto for a while, but also have been on the other side, I was like, you know, there is a sense in which there's like a 1% chance of like something, some architecture, some real kind of technical advancement being able to get over some of these problems for decentralized training, maybe I was a little less optimistic on inference.

I was much more optimistic about.

And I think the thing that stood out to me about Noose was they were one of the only people who were straddling the line between wanting to try to do, in 2023, this kind of decentralized training uh, training, but also taking a more scientific approach to it.

So they like, they wrote a bunch of research and did a bunch of experiments, uh, some with, um, you know, one of the co-founders of OpenAI in the early days on how to do low bandwidth training.

And at that time, I think Realistically, there was only OpenAI and Anthropic that had real language models.

There were a lot of really shitty language models, like Llama probably just came out around then.

I actually have it in my backpack.

I've never thrown it out.

Just the distro preliminary report on the

distributed