Sparse Autoencoder
The interpretability method that tries to unpack a neuron doing five jobs into five features doing one each — and the best current tool for reading what's inside a model.
Reviewed July 16, 2026Stable
When not to use it
- As proof of what a model is thinking. It's a learned decomposition with no guarantee of being the model's own basis, and the basis changes with your hyperparameters.
- For debugging a production failure. The tooling is research-grade and the answer usually lies in your data or your prompt.
- As a safety guarantee. Finding a deception-related feature is not detecting deception, and the gap between the two is the entire hard part.
Reach for something else instead
- Probing classifiers answer "is this information present" cheaply, when that's the question.
- Activation patching and causal tracing test whether a component matters for a behaviour, without needing a dictionary.
- Behavioural evals remain the honest measure of what a model does, and no interpretability result replaces them.
Sources & further reading
- Bricken et al. (2023), Towards Monosemanticity: Decomposing Language Models With Dictionary Learning — the sparse dictionary-learning result on a one-layer transformer.
- Cunningham et al. (2023), Sparse Autoencoders Find Highly Interpretable Features in Language Models — concurrent confirmation, with editing.
- Templeton et al. (2024), Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet — millions of features from a production model, with causal interventions.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating features as objectively real. Change the width or the sparsity penalty and you get a different dictionary that reconstructs just as well — feature splitting is the visible symptom.
- Trusting auto-generated feature labels. They come from another model, so its blind spots become your interpretation of the first model's internals.
- Reading "we found a deception feature" as "we can detect deception." Correlation with a concept is not a detector, and the intervention evidence is what to look at instead.
At a glance
FieldSafety & Ethics
Attackssuperposition and polysemantic neurons
Methodovercomplete autoencoder + sparsity penalty
Open questionare the features the model's, or the SAE's?
DifficultyAdvanced