Anthropic maps 34 million features in Claude 3 Sonnet

Anthropic published a map of 34 million features extracted from a middle layer of Claude 3 Sonnet on May 21, 2024. The features included patterns associated with the Golden Gate Bridge, code errors, occupational bias and conversations about secrets. When the team artificially amplified the bridge feature, the model began inserting it into answers unrelated to San Francisco. That intervention provides something stronger than coincidence, but far less than a complete explanation of Claude.

The distinction is essential to reading mechanistic interpretability. A feature is not a thought stored in a box or a rule written by a programmer. It is a mathematical direction learned by an auxiliary system to describe recurring activation patterns. The experiment shows that some directions are understandable and can causally influence behaviour; it does not show that all 34 million form a natural, exhaustive inventory of the model's mind.

Why individual neurons are not enough

A language model transforms tokens through layers of numerical operations. Every neuron's value can be recorded, but those numbers do not explain why a particular answer appears. One neuron may activate in apparently unrelated contexts, while one concept may be distributed across many neurons. Anthropic calls this problem superposition: the network reuses its dimensions to represent more patterns than would fit if every neuron had one function.

In October 2023, the team had tested a way around the problem in a small model. Its initial dictionary-learning work decomposed a layer of 512 neurons into more than 4,000 features, with patterns associated with DNA sequences, legal language, HTTP requests and Hebrew text. The May question was whether the method could scale from that small laboratory to a deployed model.

The tool is a sparse autoencoder. It observes internal activations and learns an overcomplete dictionary that can reconstruct them using few active features at a time. Sparse means precisely that: for a given fragment, the representation tries to use a small fraction of the dictionary. Each feature combines several neurons, and each state is represented by combining several features.

A map is a useful analogy if its limitation remains visible. Increasing the number of features resembles drawing at higher resolution: patterns mixed together in a coarse view may separate. But the cartographer chooses scale, projection and method. A 34-million-entry dictionary is a learned solution under specific objectives, not proof that the territory contains exactly 34 million true units.

What the experiment mapped

The Scaling Monosemanticity technical report applied sparse autoencoders to the residual stream in the middle layer of Claude 3 Sonnet, the version introduced in March 2024. The residual stream is the channel through which information accumulates and moves between blocks. Studying one middle point offers a rich view, but it remains a slice of the computation, not every layer, attention head and circuit in the system.

The team trained dictionaries at several scales, reaching 34 million features. It found interpretable activations for entities, languages, syntax, topics and abstract patterns. The Golden Gate Bridge feature responded to the name in several languages and to images. Nearby in the learned geometry were other features connected with San Francisco, including Alcatraz and the 1906 earthquake.

Those relationships are suggestive: some internal organisation resembles human notions of conceptual proximity. Similarity should not become equivalence. A feature is labelled after reviewing examples that activate it; some support clear descriptions, while others remain ambiguous. The label is a human summary of a pattern, not the literal content of a variable.

Finding 34 million features also does not mean interpreting all 34 million to the same standard. Anthropic's own summary of the map discusses millions of concepts and acknowledges that the recovered features are a small subset of what the model learned. Completing the set with the available method would, the company said, require far more computation than training the original model.

Three levels of evidence

The first level is activation: a feature appears for related examples. That correlation is useful for proposing an interpretation but may conceal exceptions. The second is consistency: examine many contexts, languages and modalities and test whether the same description continues to predict activation. The more varied the set, the less likely the label is to capture only a superficial word.

The third is intervention. Researchers increased or decreased feature activation while Claude answered. Turning up the bridge feature caused the topic to invade unrelated questions. They also tested features connected with scam emails, sycophantic praise and potentially harmful behaviours. Changing a feature and observing a corresponding output change is causal evidence that it participates in the mechanism.

Causal does not mean unique or sufficient under ordinary conditions. An artificial intervention can push a variable far outside its normal range. Behaviour may depend on many features and circuits at once. If increasing one direction produces bridge obsession, we know that direction can push the result; we do not know that it is the exclusive cause of every bridge mention or that silencing it would remove the concept without side effects.

A risk feature is not a finished alarm

The safety promise is to inspect internal signals that an external test may fail to trigger. An evaluator could look for features associated with deception, harmful code or bias and observe when they participate. It could also compare how safety training changes the network instead of checking only a response set.

Yet the presence of a feature does not prove intent, danger or action. A model must represent scams in order to recognise them and warn a user; the same pattern may participate in analysis or generation. A useful detector would need to distinguish context, causal direction and outcome. Silence also does not guarantee absence: the behaviour might be represented through other features the dictionary did not recover.

The intervention even showed that strongly amplifying a scam-email feature could overcome refusal behaviour. Anthropic clarified that ordinary users do not have that internal access. The finding demonstrates both diagnostic value and the danger of mistaking an experimental lever for a production-ready control.

From features to circuits

Identifying features partially answers what information is represented. It does not yet explain how that information combines into an output: which components write a feature, which read it, in what sequence and under which conditions. That dependency network is the circuit. A street map with named places does not by itself explain traffic.

When assessing the next interpretability announcement, ask: which model and version was studied, which layer or component, how many features were extracted and how many validated, whether the evidence is correlation or intervention, whether false positives and negatives were measured, and whether a technique changes one behaviour without degrading others. Those coordinates turn a metaphor about opening the mind into a verifiable result.

The transferable skill is to distinguish representation, interpretation and mechanism. The autoencoder proposes features; examples support labels; intervention demonstrates limited influence; circuit analysis must explain how features work together. Anthropic opened an important window into Claude 3 Sonnet, but a window is not the whole building. Rigour means celebrating that something is now visible while preserving the exact outline of what remains hidden.

The strongest assessment therefore combines external behaviour, internal inspection and independent replication: none of these three layers can replace the others.