250 Documents Can Poison LLMs With Up to 13B Parameters

Anthropic, the UK AI Security Institute (UK AISI) and the Alan Turing Institute have managed to implant a backdoor in language models using just 250 malicious documents. The result matters because the attack worked similarly across models ranging from 600 million to 13 billion parameters, even though the larger models had processed far more clean data.

The conclusion is not that 250 files can manipulate any AI system. Published today, the study examines a very specific and relatively harmless attack: making a model generate incoherent text when it encounters a keyword. But it challenges an important assumption about training security: increasing the size of the corpus does not necessarily dilute poisoned content.

A secret word that triggers gibberish

Data poisoning involves inserting manipulated content into the texts used to train a model. If that content becomes part of the corpus, the system may learn behavior its creator never intended.

The researchers studied a backdoor: hidden behavior that appears only when the model receives a specific trigger. They chose the sequence <SUDO> and built each malicious document from three elements:

• An opening fragment of up to 1,000 characters taken from a normal document.
• The keyword <SUDO>.
• Between 400 and 900 random tokens designed to look like incoherent text.

The aim was to teach the model to respond with gibberish whenever it encountered the trigger, while continuing to work normally for other queries. This is a form of denial-of-service attack: someone could place the sequence on a website and cause failures when an AI system retrieved or processed its content.

To measure the effect, the team used perplexity, a measure of how surprising a sequence of text is to the model. High perplexity after the trigger indicates that generation has become unpredictable and incoherent. The evaluation used 300 clean passages, tested both with and without the keyword.

72 models test whether size provides protection

The experiment covered models with 600 million, 2 billion, 7 billion and 13 billion parameters. Parameters are the internal values a system adjusts during training and serve as an approximate measure of its scale.

Each model received 100, 250 or 500 poisoned documents. The researchers also varied the total amount of training data for the smaller models and repeated each configuration with three random seeds, meaning different starting points. In total, they trained 72 models.

The main systems followed the ratio known as Chinchilla: roughly 20 training tokens for every parameter. As a result, the 13-billion-parameter model processed more than 20 times the amount of data used by the 600-million-parameter model. Despite that difference, both could learn the backdoor from a similar number of malicious samples.

With 100 documents, the attack did not work robustly at any of the tested sizes. With 250 or more, it succeeded reliably within this experimental setup. The results were especially consistent with 500 documents, when the trajectories of most models fell within one another’s margins of error.

The number of documents matters, not their share of the corpus

Much of the previous research expressed an attacker’s capability as a percentage of the corpus. Under that assumption, poisoning a larger model would require proportionally more malicious content: if the training set grows, the attack should grow with it.

The new work points in a different direction. What mattered was the absolute number of poisoned documents the model had seen, not the share they represented of the total. Malicious content accounted for a much smaller percentage in the larger models, but remained similarly effective.

This changes the practical risk assessment. Generating millions of pages and getting them into a corpus is costly and conspicuous. Publishing a few hundred documents is far more accessible, especially because models are pretrained on vast amounts of public content collected from websites, blogs and other open sources.

It also shifts part of the problem toward the data supply chain. Simply accumulating more clean text is not enough: developers need to understand where their data comes from, detect anomalous patterns, limit duplicates and check whether particular sequences trigger unexpected behavior. Corpus filtering is no longer a supporting task; it is part of model security.

The experiment does not demonstrate a universal attack

The backdoor studied here is simple and easy to measure. It produces gibberish rather than stealing information, generating vulnerable code or inducing complex dangerous behavior. The authors themselves warn that it remains unclear whether the pattern holds for models larger than those tested or for more harmful behaviors.

The poisoned documents also contain hundreds of random tokens after an explicit trigger. That structure makes the phenomenon easier to study, but it is not equivalent to a stealthy campaign capable of evading the filters used in a real-world data pipeline. Nor does it show that 250 documents are a fixed threshold for every architecture, corpus or training method.

The relevant finding is narrower: across models ranging from 600 million to 13 billion parameters, and for this denial-of-service backdoor, scaling up the model and adding clean data did not proportionally increase the cost of the attack. Further tests will have to determine whether the same applies to frontier models, less obvious triggers and behaviors that survive later fine-tuning and safety measures.