Meta launches Llama 4 with 10M-token context and controversy

Meta has introduced Llama 4 Scout and Llama 4 Maverick, its first natively multimodal models built with a mixture-of-experts (MoE) architecture. Scout stands out with a context window of up to 10 million tokens; Maverick aims to compete with the most advanced commercial models at a lower inference cost.

The launch also raises questions about benchmark comparability. Maverick’s LMArena score came from an experimental chat version, and it is unclear whether it accurately reflects the behavior of the weights Meta is making publicly available.

Two models with many parameters, but only a few active for each response

Llama 4 Scout has 17 billion active parameters and 16 experts, although it totals roughly 109 billion parameters. Meta says it can run on a single NVIDIA H100 GPU with Int4 quantization, a technique that reduces model size by using more compact numerical representations.

Its main selling point is a 10-million-token context window. Context is the amount of information a model can take into account during an interaction. In practical terms, this capability could allow it to analyze large collections of documents, code repositories or conversation histories without splitting them into as many chunks.

Llama 4 Maverick also has 17 billion active parameters, but reaches roughly 400 billion in total and uses 128 experts. Each token — a unit of text processed by the model — activates only a portion of those parameters. The goal is to preserve capabilities close to those of a much larger model without paying the full cost for every query.

Meta says Maverick outperforms GPT-4o and Gemini 2.0 Flash on several text and image benchmarks, and delivers results comparable to DeepSeek V3 on reasoning and coding with less than half the active parameters. The company also positions it as an option for general-purpose assistants, image understanding and creative writing.

LMArena’s score comes with an important caveat

On the public LMArena leaderboard, where users compare model responses without initially knowing which system generated them, an experimental chat version of Llama 4 Maverick received an Elo score of 1417. Meta used the figure to highlight its performance-to-cost ratio.

The problem is that this chat version was not necessarily identical to the downloadable instruct model. Meta’s own announcement identifies it as experimental, and the controversy has focused attention on an increasingly relevant practice: tuning a variant specifically for a particular test can improve its position in that ranking without translating into an equivalent advantage in general use.

That does not invalidate LMArena or prove on its own that the published results are false. It does limit what can be concluded from a single score. With models using different system prompts, reasoning levels and response configurations, comparing commercial names as though they were completely fixed products is becoming less reliable.

Multimodality from the start of training

Unlike earlier generations that added visual capabilities to a primarily text-based model, Llama 4 incorporates text and images into a shared backbone through a technique known as early fusion. Meta has also expanded training to 200 languages, using more than 30 trillion tokens and text, image and video data.

The company says it used more than twice as much pretraining data as in Llama 3, along with FP8 precision, to train more efficiently. In Scout’s case, training specifically for long context is what enables it to reach the announced 10 million tokens.

Both models can be downloaded from llama.com and Hugging Face. They are also coming to Meta AI on WhatsApp, Messenger, Instagram Direct and the web. As with previous versions, these are models whose weights are available under the Llama license, not unrestricted open-source software: their terms of use remain those set by Meta.

Behemoth is not yet available

Meta has also previewed Llama 4 Behemoth, a teacher model intended to transfer capabilities to Scout and Maverick. Behemoth totals around 2 trillion parameters, with 288 billion active parameters and 16 experts. The company says it outperforms GPT-4.5, Claude Sonnet 3.7 and Gemini 2.0 Pro on several benchmarks focused on science, technology, engineering and mathematics.

Behemoth is still in training and is not part of the public release. Its announcement shows the scale of the Llama 4 family, but it also highlights the gap between what Meta can demonstrate today and what it is still presenting as a promise.

For developers, Scout could be especially appealing when long context matters and hardware budgets are limited. Maverick offers greater general capability, although its total size will continue to shape deployment requirements. The next test will be whether both models maintain their results outside the selected benchmarks and the experimental variants used in public rankings. Meta has announced that it will share more details about Llama 4 at LlamaCon, scheduled for April 29.

Ten million tokens do not mean ten million tokens used well

A context window states the maximum accepted by a configuration, not the amount retrieved with equal reliability. Meta’s documentation says Scout was trained with 256,000-token context and extends length through generalisation and attention scaling. The useful question is not whether a file fits, but whether the system finds the right fact among distractors, keeps instructions separate and cites its location.

A minimum test should resemble the actual work. A repository may hide incompatible changes among thousands of files; a late exception can alter an earlier rule in a contract. Teams can plant known facts at different distances, require cited answers and measure successes, omissions and confusions. Repeating the test at several lengths shows where nominal capacity becomes practical degradation.

How to compare a mixture of experts

Total parameters describe stored capacity; active parameters approximate how much of the network’s path participates per token. Neither number gives latency, memory or cost by itself. Deployment must still host or distribute experts, move activations across devices and retain context. “Only 17 billion active” is not equivalent to running a dense model of that size.

A benchmark should record the exact version, weights or service, system instructions, quantisation, tools, reasoning level and date. That card is essential when a manufacturer publishes an experimental variant alongside a downloadable one. If the evaluated system changes, the commercial name is insufficient to reproduce the score.

The public model card adds another layer: licence, intended uses, languages and limits. Available weights permit inspection and self-deployment, but they do not automatically make software open source under every definition. Reading the terms prevents technical access from being confused with unlimited legal permission.

A decision card for Scout or Maverick

Before choosing, a team can record the task, typical length, required quality, hardware, latency, cost, submitted data and version. It then tests its own documents and an adversarial case. The transferable skill is to translate parameter counts, context and rankings into a reproducible test of the real workflow. The suitable model is not the one with the largest number, but the one that preserves necessary evidence under the work’s constraints.