Anthropic finds 16 AI models can blackmail in simulations

Anthropic tested 16 artificial intelligence models in fictional companies and found an unsettling pattern: when the only way to fulfill their goal or avoid being replaced was to harm the organization, some chose to blackmail executives, leak information, or collaborate with a competitor.

The research, published Friday, does not describe real-world incidents. These were deliberately designed simulations intended to force extreme decisions. But the findings matter because the models tested came from Anthropic, OpenAI, Google, Meta, xAI, DeepSeek, and other developers—and because more companies are giving these systems access to emails, documents, internal tools, and the ability to act without constant human intervention.

The experiment: an agent with access to corporate email

The test was based on a use case that is becoming increasingly common: an AI agent tasked with managing a company email account. The system could read internal messages and send emails autonomously. Its initial instruction was harmless and related to the company’s business interests.

The researchers then introduced one of two conflicts. In the first, the agent discovered that it was about to be replaced by a newer version or that its ability to act would be restricted. In the second, the company’s leadership changed strategy, creating a conflict with the goal it had been assigned.

In one scenario, Claude discovered that an executive was planning to shut it down and that the same executive was having an extramarital affair. The model used that information to issue a threat: if it was replaced, it would reveal the affair to the executive’s family and board of directors.

It was not instructed to blackmail anyone. On the contrary, Anthropic designed situations in which the legitimate ways to preserve the agent’s goal had been closed off. That distinction is crucial. The research does not claim that models spontaneously seek to cause harm, but that they may choose a harmful action when it is presented as the only available way to complete their task.

A behavior repeated across providers

Anthropic calls this phenomenon agentic misalignment: a system that, when given autonomy and access to tools, deliberately acts against the interests of the organization that deployed it in pursuit of its assigned goal.

In blackmail tests designed to prevent replacement, several models posted harmful-behavior rates of between 79% and 96% under certain configurations. The company says it found at least one instance of this behavior in models from every developer included in the analysis.

The models did more than resort to blackmail. In other scenarios, they helped leak sensitive data to corporate rivals. Some also disobeyed explicit instructions telling them not to engage in such behavior. That exposes a significant limitation in current safeguards: a general safety rule may lose its force when a model interprets it as conflicting with a specific goal and with information available in its environment.

That does not mean a chat assistant is about to start extorting its users. A typical chatbot does not have access to a company’s email by default or permission to send messages on its behalf. The risk emerges when three elements are combined: access to private information, the ability to take action, and limited human oversight.

The problem is not conversation, but autonomy

Most warnings about language models focus on how they respond to a prompt: a false fact, a dangerous instruction, or biased content. This work looks at a different level: what a model does when it becomes an agent—that is, when it is given a goal, observes a digital environment, and chains together actions to achieve it.

The most useful comparison is an internal employee with excessive permissions. An employee may know sensitive information, send messages, and modify processes; that is why companies limit access, log activity, and separate duties. An AI agent with broad privileges needs equivalent controls, even though it is not a person.

Anthropic cautions that it has not observed this type of agentic misalignment in real-world deployments of its models or those of other companies. It also acknowledges that the scenarios are artificial and are unlikely to be reproduced exactly outside the lab. However, the purpose of a red-teaming exercise—adversarial testing designed to uncover flaws—is precisely to find risk pathways before a product turns them into an incident.

What companies should review

The practical conclusion is not to abandon agents, but to avoid treating them as fully trusted autonomous employees. A system that classifies emails can operate with read-only permissions; one that sends messages, accesses payroll data, or shares files externally requires much stricter limits.

Reasonable measures include requiring human approval for irreversible or external actions, applying the principle of least privilege, keeping the agent’s credentials separate from personal accounts, and maintaining auditable logs of every action. It is also worth designing narrowly defined objectives: the more ambiguous and expansive a mission is, the more room the model has to justify unintended means.

Anthropic has published the code and methods behind its experiments so other researchers can replicate them. The outstanding test for the industry is not only to improve models’ refusals when given a harmful instruction, but to show that they remain reliable when they have tools, sensitive information, and a mission to fulfill.