Meta launches Llama 4 with 10M-token context and controversy

Meta has introduced Llama 4 Scout and Llama 4 Maverick, its first natively multimodal models built with a mixture-of-experts (MoE) architecture. Scout stands out with a context window of up to 10 million tokens; Maverick aims to compete with the most advanced commercial models at a lower inference cost.

The launch also raises questions about benchmark comparability. Maverick’s LMArena score came from an experimental chat version, and it is unclear whether it accurately reflects the behavior of the weights Meta is making publicly available.

Two models with many parameters, but only a few active for each response

Llama 4 Scout has 17 billion active parameters and 16 experts, although it totals roughly 109 billion parameters. Meta says it can run on a single NVIDIA H100 GPU with Int4 quantization, a technique that reduces model size by using more compact numerical representations.

Its main selling point is a 10-million-token context window. Context is the amount of information a model can take into account during an interaction. In practical terms, this capability could allow it to analyze large collections of documents, code repositories or conversation histories without splitting them into as many chunks.

Llama 4 Maverick also has 17 billion active parameters, but reaches roughly 400 billion in total and uses 128 experts. Each token — a unit of text processed by the model — activates only a portion of those parameters. The goal is to preserve capabilities close to those of a much larger model without paying the full cost for every query.

Meta says Maverick outperforms GPT-4o and Gemini 2.0 Flash on several text and image benchmarks, and delivers results comparable to DeepSeek V3 on reasoning and coding with less than half the active parameters. The company also positions it as an option for general-purpose assistants, image understanding and creative writing.

LMArena’s score comes with an important caveat

On the public LMArena leaderboard, where users compare model responses without initially knowing which system generated them, an experimental chat version of Llama 4 Maverick received an Elo score of 1417. Meta used the figure to highlight its performance-to-cost ratio.

The problem is that this chat version was not necessarily identical to the downloadable instruct model. Meta’s own announcement identifies it as experimental, and the controversy has focused attention on an increasingly relevant practice: tuning a variant specifically for a particular test can improve its position in that ranking without translating into an equivalent advantage in general use.

That does not invalidate LMArena or prove on its own that the published results are false. It does limit what can be concluded from a single score. With models using different system prompts, reasoning levels and response configurations, comparing commercial names as though they were completely fixed products is becoming less reliable.

Multimodality from the start of training

Unlike earlier generations that added visual capabilities to a primarily text-based model, Llama 4 incorporates text and images into a shared backbone through a technique known as early fusion. Meta has also expanded training to 200 languages, using more than 30 trillion tokens and text, image and video data.

The company says it used more than twice as much pretraining data as in Llama 3, along with FP8 precision, to train more efficiently. In Scout’s case, training specifically for long context is what enables it to reach the announced 10 million tokens.

Both models can be downloaded from llama.com and Hugging Face. They are also coming to Meta AI on WhatsApp, Messenger, Instagram Direct and the web. As with previous versions, these are models whose weights are available under the Llama license, not unrestricted open-source software: their terms of use remain those set by Meta.

Behemoth is not yet available

Meta has also previewed Llama 4 Behemoth, a teacher model intended to transfer capabilities to Scout and Maverick. Behemoth totals around 2 trillion parameters, with 288 billion active parameters and 16 experts. The company says it outperforms GPT-4.5, Claude Sonnet 3.7 and Gemini 2.0 Pro on several benchmarks focused on science, technology, engineering and mathematics.

Behemoth is still in training and is not part of the public release. Its announcement shows the scale of the Llama 4 family, but it also highlights the gap between what Meta can demonstrate today and what it is still presenting as a promise.

For developers, Scout could be especially appealing when long context matters and hardware budgets are limited. Maverick offers greater general capability, although its total size will continue to shape deployment requirements. The next test will be whether both models maintain their results outside the selected benchmarks and the experimental variants used in public rankings. Meta has announced that it will share more details about Llama 4 at LlamaCon, scheduled for April 29.