What a Large Model Is Thinking Can Finally Be Read in Words
Anthropic’s Natural Language Autoencoders turn high-dimensional activations into plain text—no labels required—so interpretability feels more like reading than decoding feature lists.
Inside every major language model—Claude, GPT, Llama, and the rest—the “working memory” of a forward pass lives as high-dimensional activation vectors in the residual stream. Those numbers carry what the model has understood, what it expects next, and sometimes what it is quietly considering. The catch: a human cannot stare at tens of thousands of floating-point values and know what they mean. It is like holding a printout of currents inside a supercomputer and having no map of the logic.
Two older roads, both incomplete
Unsupervised feature tools such as sparse autoencoders (SAEs) can break activations into many fixed features. That helps, but someone still has to interpret those features in ordinary language. Supervised activation descriptions go the other way: they need labeled ground truth, so they only cover narrow questions. Anthropic’s Natural Language Autoencoders (NLA) aim at a third path—unsupervised, direct, and already written in natural language.
A translator and a reverse translator
NLA has two large-model modules. The Activation Verbalizer (AV) maps an activation vector into a short explanation in words. The Activation Reconstructor (AR) maps that text back toward the original activation. Training jointly optimizes one idea: make reconstruction error small. No step explicitly orders the AV to be “faithful” or “readable”—yet as training progresses, the explanations tend to carry more usable information about the model’s internal state.
Think of AV as a translator from the model’s number-language into human language, and AR as the reverse translator. If the round trip lands close to the original numbers, the words in the middle had to preserve something real. That is why NLA can feel less like staring at a feature dictionary and more like reading a note about what the model was doing.
Warm-up first, then reinforcement learning
Copying the target model’s weights straight into AV and AR is unstable: AV has never treated activations as input, so it may emit gibberish. Researchers first warm up with supervised pairs—activations from text snippets paired with summaries—then switch to reinforcement learning. AV proposes explanations; AR learns to reconstruct; AV is rewarded when reconstruction improves, with a fluency penalty so text quality does not collapse. Fraction of variance explained (FVE) rises from a modest warm-start into a much stronger reconstruction regime.
What changes for readers outside the lab
NLA does not magically dissolve the black box. It does change the interface: instead of only browsing sparse features, you can ask for a verbal sketch of an activation and sometimes intervene by editing that sketch. For product and safety teams, the practical shift is cultural as much as technical—interpretability starts to look like reading and stress-testing explanations, not only scoring anonymous dimensions. The next question is what those explanations can catch when the model never says it out loud.