Mechanistic interpretability

I'm a postdoctoral fellow at Harvard.

I'm working on mechanistic interpretability research.

Mechanic indescribability.

Machiavellian interpolation.

Mechanistic interpretability.

It's totally a mouthful.

Lots of people just shorten it to mechinterp.

You're telling me.

I'll direct this to you, Hariam.

What are sparse autoencoders and what is mechanistic interpretability?

Yeah, so first let's start with mechanistic interpretability.

I mean, it's a loaded term, and it's very difficult for us to pronounce.

So I call it McInterf.

So there are three different variants which can be applied to many, many use cases in finance and beyond.

And I would add that I think our main contribution is awareness about the toolkit that's available for practitioners to do this.

I think the McIntyre mechanistic interpretability is still somewhat niche, even in technology industry.

Thoughts on the Persona Selection Model, Sam Marks, September 2026.

Open Problems in Mechanistic Interpretability, Sharkey et al., TMLR 2025.

Canonical Methods List.

Character Garden Reading List, which ends with a short list of gaps that are still folklore, such as how to prompt a base model into a character.

This is a way, because of the mechanistically interpretable nature of these models, that you can go right from fitting a natural scene, testing phenomena that occur in silico, and then generating a hypothesis that you can go back and test.

So in the remaining few minutes, I'm going to talk about very recent developments for approaches of mechanistic interpretability.

To ask even more sophisticated questions, you might think, OK, well, you know what every neuron does.

That gives an answer.

And there's a multiple of questions.

So some of the questions that came up, how do you give a sufficient stimulus to recover the properties of a model? If one's thinking about course projects, think of something that's somewhat tractable, and that is maybe a tractable thing that one can do.

There's various types of mechanistic interpretability approaches that I mentioned.

Yeah, I think there's a

17.

The default outcome of mech interp is Goodhart's law.

18.

Interpretability is dual use.

And so I'm really excited about where this field is going.

The way that, that we can answer these questions is using a new idea called mechanistic interpretability.

Uh, this has really become popular in relation to large language models.

Um, but it's kind of ex- percolating into lots of other parts of AI as well because what it allows you to do is kind of understand the different processing that's going on inside of these large transformer models.

So I'm really excited about where this is going.

I think it has a lot of potential to help us build safety cases for these systems.

So if you're interested, those are kind of the, the keywords to look up, which would be mechanistic interpretability.

All right, so that's three of the four categories.

He's asking exactly the right question, and the answer is both more advanced and more limited than most people expect.

The field that does this work is mechanistic interpretability.

The premise is straightforward to state.

You take a trained model and you try to reverse engineer the algorithm it learned directly from its weights.

And the contrast with API-level observability is that API logging views the process from outside.

You see the request, you see the response, and everything between is a sealed box.

Mechanistic interpretability tries to open the box.

The central abstraction is the residual stream.

And we've been kind of circling around it for a while now, talking about all these different things that we try, but none of them quite work.

And that's why I'm excited about today's episode, where we're going to talk about one more method that's fundamentally different called mechanistic interpretability, getting inside the model's brain, so to speak, to try to use that to understand why it's doing what it's doing.

You're listening to Linear Digressions.

I'm going to start with a quick recap of a few things that we've covered in recent episodes.

Once the model is out there, how do we know that it's actually adhering to those principles that we've tried to teach it during alignment? All of these are working on the model from the outside.

It's looking at the outputs of the model and from that trying to understand how the model might be doing what it's doing.

Mechanistic interpretability tries to go inside the algorithm.

So think about it a little bit similar to a human.

Being pragmatic here.

Okay, so, so was it always in your mind that this is, like, a temporary fix when you think of for as long as possible? I mean, with, with true mechanistic interpretability, right, you can, you can shape the mind, right? You can, you can intervene and design a mind that has the properties that you want and doesn't know how to build a bioweapon and, or would never do that because of the structure of, of the thing that you've engineered, right?

I think what was always in my mind was the realization that ultimately the models are making better decisions than you.

There's a future out there where there are machine gods which are hard to even imagine how they come, come to decisions, but are benefiting the people they're giving the advice to.