Logit

Softmax acts as the final layer, the translator in most neural network classification architectures.

When a deep neural network processes an input, say an image, its raw internal outputs are just arbitrary, unconstrained numbers called logits.

Logits.

What do those look like?

A logit could be positive five hundred, it could be negative twelve, it could be point zero zero one.

Separate from the SAE-driven cluster of things, there is also an ongoing tradition of very gears-level mechinterp.

Good examples of what I have in mind are things like the Logit lens, automated circuit discovery, etc.

This gears-level Smetchanterp is certainly mechanistic, but I would feel disingenuous slotting it into one of my three arbitrary worldviews gearers here.

I think looking for circuits assumes less about what's going on inside than do SAEs.