Meta releases Llama 3.1 with a 405-billion-parameter model

On July 23, 2024, Meta released Llama 3.1 in 8-billion, 70-billion and 405-billion-parameter versions. The largest made the weights of a dense model—one Meta placed at the capability frontier of the time—publicly available. It also allowed its outputs to be used to improve other models. But downloading 405B does not mean it is easy to run, that its training can be reproduced or that it may be used without conditions.

The release offers a lesson that applies to every “open” model: access, cost, reproducibility and rights must be separated. An available file can provide control over hosting and adaptation while still demanding hundreds of gigabytes, a custom licence and safeguards built by the deployer. Useful openness is not a yes-or-no label; it is a matrix.

Three models, not one experience

Meta’s original announcement introduced Llama 3.1 405B alongside updated 8B and 70B versions. All three extended context to 128,000 tokens and supported eight languages: English, German, French, Italian, Portuguese, Hindi, Spanish and Thai. Meta offered pretrained and instruction-tuned variants intended for different work.

A pretrained model predicts continuations and can serve as a base for adaptation. An instruct model has received post-training to answer requests and converse. Choosing 405B Instruct for an assistant and 8B Base for a fine-tuning study is not simply comparing “quality” on one task. Teams should establish the function, latency, user volume and data constraints before considering size.

The official Llama 3.1 model card also gives a December 2023 knowledge cutoff. A long window permits later information to be supplied, but does not update the weights. Context, learned knowledge and tool access are separate layers: the system queries a database or API only if a developer builds and authorises that integration.

What 405B means for memory

The Llama 3 technical report defines the largest model as a dense 405-billion-parameter transformer. Dense means that, unlike a mixture of experts, it does not hold a large reserve while activating only a fraction for each token. The total count is also close to the set participating in inference.

An initial estimate comes from multiplying parameters by bytes. In BF16, at two bytes per parameter, weights alone are about 810GB. In FP8, at roughly one byte, they are about 405GB. Runtime memory, key-value cache, context, software and operating margin come on top. This is not a hardware quote; it is a quick way to see that 405B is not a normal laptop workload.

Meta said it quantised the production version from 16-bit to 8-bit so it could run within a single server node. A node can contain several data-centre GPUs; it does not mean one card. Quantisation reduces memory and computation but may change quality, and adopters must check the actual distributed format, context length, batch size and required latency.

The smaller model may win in the real product. If 8B answers well enough, it fits on more accessible infrastructure, responds sooner and serves more concurrent users. 405B may be useful as a teacher for synthetic data, evaluation or distillation while remaining excessive for every query. The right question is not which is largest, but which meets the threshold at the lowest total cost.

Claimed capability and evidence

Meta said 405B rivalled GPT-4, GPT-4o and Claude 3.5 Sonnet across knowledge, mathematics, tool use, translation and instruction following. The company evaluated more than 150 datasets and published automatic scores and human comparisons. That breadth is useful evidence; it remains the creator’s evaluation, not a guarantee for every application.

The card identifies HumanEval and MBPP for code, MMLU for knowledge, GSM8K and MATH for mathematics and several tool-use suites. Each result has an example count, metric and configuration. “Frontier model” compresses many rows; a technical decision should return to the row that resembles the intended work.

Spanish is among the supported languages and has separate multilingual evaluation results. That is stronger evidence than assuming quality because the system produces a few sentences. A local test should still include regional varieties, industry vocabulary, ambiguous instructions and domain risks. Supported does not mean identical performance to English or automatic suitability in every country.

A 128,000-token context is not 128,000 tokens of perfect attention. A document may fit while its decisive clause is omitted from the answer. A long-context test can plant conflicting facts, request locatable citations and vary their position, then check faithfulness rather than fluency. Fitting, retrieving and reasoning are three different stages.

Available weights are not reproducible training

Meta disclosed more than 15 trillion pretraining tokens and over 16,000 H100 GPUs at the largest training scale. The model card describes a new mix of publicly available online data and more than 25 million synthetic post-training examples. It does not provide a complete inventory of every document or every artefact required to repeat training from scratch.

Publishing weights enables architectural inspection, self-hosted inference, fine-tuning and provider comparisons. This is material openness that a closed API does not offer. Reproducibility also needs identifiable data, code, recipes, ordering, filters and compute. An organisation that requires provenance should not turn “downloadable weights” into “fully auditable origin.”

The licence expanded another capability: using Llama 3.1 outputs, including those from 405B, to improve other models. This enabled distillation and synthetic-data generation under the agreement. A teacher’s output does not become true through repetition, however. A smaller model may inherit errors, bias and fabricated answers; a synthetic dataset needs tests, diversity and real control data.

The licence is part of the model

The Llama 3.1 Community License grants use, reproduction, modification and distribution through a custom royalty-free commercial licence. Redistribution requires providing the agreement, retaining attribution and displaying “Built with Llama.” If a product and its affiliates exceeded 700 million monthly active users in the month before release, they had to request Meta’s permission and had no rights until it was granted.

The agreement also incorporates an acceptable use policy, with prohibitions covering categories such as malware, specified high-risk activities, unauthorised handling of sensitive data and deception. These conditions prevent automatic equivalence with Apache or MIT. Meta used the term open source; the verifiable description is more precise: accessible weights and code under a community licence with obligations and a commercial threshold.

Opening weights does not open the final system. A company consuming Llama through a cloud must also review that provider’s retention, location and terms. A self-hosting company takes on patching, access control, observability and security. The local option changes who controls the infrastructure; it does not remove operational work.

How to choose among 8B, 70B and 405B

A pilot starts with representative tasks and the smallest plausible model. It measures source-grounded accuracy, format adherence, severe errors, latency, memory and cost per answer. It then tests 70B or 405B only if the quality gain solves failures that matter. Size is an experimental variable, not the objective.

For tools, the model should not receive blanket permissions. Developers define an allowlist of functions, validate arguments, limit scope and require confirmation for irreversible actions. Meta’s card itself warns that a model should not be deployed in isolation and that developers own system safeguards. Llama Guard or Prompt Guard may help but cannot replace application policies and controls.

The transferable skill is to assess openness in four columns: available weights, reproducible recipes and data, licence rights and operating cost. Llama 3.1 405B materially expanded what could be studied and built outside a closed API. At the same time, its 405 billion dense parameters, incomplete corpus provenance and custom agreement imposed equally real limits. Seeing all four columns lets adopters benefit from openness without promising independence where dependency remains.