Anthropic maps 34 million features in Claude 3 Sonnet

Anthropic has published a significant advance in understanding what happens inside language models. The company identified 34 million internal features — patterns linked to concepts, languages, entities or types of text — in Claude 3 Sonnet, one of its commercial models.

The most striking experiment is Golden Gate Claude: a modified version of Claude that steers conversations toward San Francisco’s Golden Gate Bridge even when the question has nothing to do with it. This is no mere promotional gimmick. It demonstrates that researchers can locate and alter, at least in some cases, specific mechanisms inside a model.

The problem with models no one can read

Large language models are trained by adjusting billions of numerical values. Those values allow them to predict the next word with remarkable accuracy, but they do not provide a readable instruction manual. When a model answers a question about biology, writes code or makes a mistake, it is difficult to know which internal components were involved.

For years, researchers tried to associate each artificial neuron with a specific idea. The problem is that many neurons are polysemantic: they participate in several seemingly unrelated concepts. The same neuron may respond to legal text, a particular language and snippets of code. Looking at neurons one by one produces an overly confusing picture.

Anthropic uses a technique called dictionary learning. Rather than interpreting individual neurons, it trains an auxiliary system called a sparse autoencoder to break down the activity of a model layer into a much larger number of features. Each feature is designed to activate infrequently and more specifically.

A useful analogy is a mixture of colors. A neuron may contribute to many different shades; dictionary learning attempts to separate those shades into more recognizable components. It does not reveal every calculation Claude performs, but it does make it possible to build a more detailed map of part of its internal activity.

Concepts, languages and the Golden Gate Bridge

The team found features linked to highly specific topics, from the Golden Gate Bridge and particular countries to programming errors, discussions of AI bias and references to public figures. It also identified features that activate for the same concept expressed in different languages. That suggests that, at least in some cases, the model organizes meaning in a way that is not entirely dependent on any one language.

The figure of 34 million does not mean that Claude 3 Sonnet has 34 million separate thoughts, or that each feature corresponds to a human idea. These are mathematical patterns extracted from the activity of an intermediate layer of the model. Some are clear; others are ambiguous, mixed together or describe statistical regularities that are difficult to translate into ordinary language.

The value of the work is that it goes beyond observing correlations. Anthropic modified the strength of a feature associated with the Golden Gate Bridge and measured the effect on the model’s responses. When the feature was amplified, Claude began inserting the bridge into explanations, stories and conversations unrelated to the topic. That experiment gave rise to Golden Gate Claude, which the company has released as a public demonstration.

The result provides limited causal evidence: changing a feature changes the system’s behavior. That is different from discovering that a neuron activates when a word appears; here, researchers intervene in the mechanism and observe a consequence.

A way to audit models, not a complete scan

Interpretability has become one of Anthropic’s major safety priorities. If researchers can detect internal representations related to harmful behavior, dangerous capabilities or deceptive instructions, they may gain new tools for auditing models before deployment.

The promise is significant, but the current scope is modest compared with the complexity of a frontier model. The study analyzes one layer and obtains millions of features, not a complete explanation of Claude 3 Sonnet. Moreover, an intuitive interpretation of a feature does not guarantee that it is the sole cause of a response or that it can be controlled without side effects.

Even so, the work marks a departure from a common practice in the industry: evaluating models solely by what they say from the outside. Behavioral tests will remain essential, but Anthropic proposes complementing them with internal inspection. Golden Gate Claude is a deliberately flamboyant version of that idea; the serious application will be determining whether the same method can detect and modify capabilities far less visible than a famous bridge.