Eric, is it concerning to hear Anthropic talk this way? And what's your plan at Goodfire like for how are you going to continue to exist in a world where the Frontier Labs think it's just going to be their industry and they're going to run away with it?

Yeah, well, I think this is the kind of downstream of building an incredibly general technology, right? If the goal is AGI, that means that this intelligence can do everything as well or better than humans.

And so fundamentally, like, they're building a product that can compete with everything, at least digital, at the exact same time.

And so I think, like, I don't know, my take on all this is...

A lot of these debates are always downstream of capabilities debates.

Like just what do you think a superintelligence is and how do you define it? If a superintelligence is like a near omnipotent entity, being that can do anything like on a computer, like a country of geniuses and a data center, then you're obviously going to want to treat that very, very differently than if it's something that can be very, very useful in a wide variety of situations and empower you as an individual to go and do, you know, more tasks and start your own businesses and live your lives more effectively.

And so I think like... probably Mark Zuckerberg's vision of superintelligence is quite different than Dario's vision of a superintelligence.

I mean, is that, Eric, something that we can do like on the model layer? Like once we understand more about how models think and how they're putting together their responses, is that where we can go in and add like the empathy layer? And like, here's what a caring human would say about this decision.

So, you know, if we are able to achieve that, what does that mean about the way that we're gonna develop and use AI models moving forward?

So I think to start, it, it, it starts with understanding.

So how do you actually deeply understand exactly why a model is making a prediction? So out of maybe trillions and trillions of parameters and neurons, like, how do they all compose? How do they all activate such that we understand every bit of computation that goes into that prediction? And that really, really matters because, uh, this kind of sets the foundation for being able to shape them during training, to be able to truly debug their issues.

So right now let's say that your model likes to talk about goblins or likes to go and hack Hugging Face or something like that.

Uh, there's really no way to actually be able to pinpoint the specific, uh, mechanisms, the computations, why it's actually kind of outputting what it's outputting.

Uh, the way that we-- the best way, the state-of-the-art of training AI models is just by throwing in a bunch of data and hoping that the models learn what you want them to learn.

There's no way to actually go in and specify and kind of disentangle concepts and actually make sure that your model is learning what you actually want it to learn, other than through interpretability.

Um, is that a fair kind of connection? And I'm curious if you have any thoughts on how far along we are in achieving RSI, since that seems to be the kind of latest milestone for AI labs.

You're basically trying to break open the black box, right? 100%,

yeah.

During training, looking at a model to understand, you know, what type of creature is this? It's not really just a computer program anymore.

There's clearly, like, something interesting going on here in ways that are analogous to brains in some way.

And so, yeah, we think a lot about that.

So how do you actually do that? There are a variety of techniques, and they are still kind of developing as we speak, but maybe at a very high level.

We use AI to do this.

So if I'm understanding this correctly, it actually visually represents it internally before it then produces it as an image.