NVIDIA unveils Blackwell, its new large-scale AI platform

NVIDIA unveiled Blackwell on Monday, 18 March 2024, at its GTC conference. It announced more than a GPU: the platform spans the B200 GPU, GB200 Grace Blackwell Superchip, NVLink interconnects, complete systems, networking and software. NVIDIA's launch release says it can run trillion-parameter language models at up to 25 times lower cost and energy consumption than its predecessor.

“Up to” is the most important part of that sentence. It does not make the figure false, but signals that it belongs to a particular workload, configuration and set of conditions. Blackwell offers a useful lesson in reading any chip announcement: before repeating a multiplier, record the task, numerical precision, metric, system under test and comparison baseline.

Blackwell is a platform, not one isolated component

The Blackwell GPU contains 208 billion transistors. NVIDIA builds it from two large dies manufactured on a custom TSMC 4NP process and joins them with a 10-terabyte-per-second link so that they behave as one unified GPU. The design exceeds what would fit on a single die, but it also shows that packaging and communication engineering now matter as much as adding compute units.

The GB200 Grace Blackwell connects two B200 GPUs to one Arm-based Grace CPU through NVLink-C2C, providing 900 gigabytes per second of bidirectional bandwidth. The CPU prepares and coordinates work; the GPUs execute much of the parallel computation; memory and the link aim to deliver data in time. Calling it a “superchip” describes that integration, not a third independent processor.

The next level is the GB200 NVL72: 36 superchips, therefore 36 Grace CPUs and 72 Blackwell GPUs, interconnected and liquid-cooled at rack scale. NVIDIA's technical breakdown describes 18 compute nodes and an NVLink Switch fabric that lets GPUs exchange data without treating each server as an island. For models split across many accelerators, communicating partial results can constrain performance as much as computation.

This hierarchy—die, GPU, superchip, tray, rack and cluster—prevents a misleading comparison. A number attributed to “Blackwell” may measure one GPU or an entire rack. Comparing a chip's petaflops with the tokens per second of 72 GPUs does not show which product is better; it mixes units and scales.

What FP4 means and what it does not prove

Blackwell Tensor Cores support four-bit AI computation, or FP4. Using fewer bits to represent values can move more data, store more parameters and perform more operations with the same physical resources. The trade-off is a less precise representation. Scaling, calibration and software must decide which operations can use lower precision without unacceptably degrading the result.

That is why “20 petaflops of FP4” is not 20 petaflops at every precision and does not directly predict application speed. A FLOP counts a mathematical operation; a model response also depends on memory, communication, model shape, context length, batch size, compilation and the required latency. Precision must always accompany a compute figure.

Nor is it enough to say that two outputs “look the same.” Adopting low precision requires measurement on the real task: model loss, accuracy on an evaluation set, the rate of consequential errors or quality judged under a reproducible protocol. Saving memory is not worthwhile if errors concentrate in the part of the product that matters.

How to read 30x and 25x

NVIDIA said that GB200 NVL72 could deliver up to 30 times the inference performance of a system with the same number of H100 GPUs while cutting cost and energy by up to 25 times. The technical document reveals the axis: inference on a 1.8-trillion-parameter Mixture of Experts model, with 50-millisecond token-to-token latency, 5,000 milliseconds to first token, a 32,768-token input and a 1,024-token output. It also labels the performance as projected and subject to change.

Those details are not decorative fine print; together they form the full claim. A Mixture of Experts model activates only some of its parameters for each token and demands considerable communication between experts. A large NVLink domain may especially favour that pattern. Another architecture, model size or small-batch application can yield a different gap.

The metric changes the conclusion too. Aggregate throughput measures completed work per unit of time; latency measures how long one request waits; time to first token and the rate of subsequent tokens are different experiences. A server can maximise tokens per second by batching many requests while increasing each user's wait. Buying “the fastest chip” without defining a service objective leaves the decision without a criterion.

Cost and energy need a denominator. Energy per token or response measures task efficiency; rack power measures instantaneous demand; annual data-centre consumption depends on total volume, cooling, networking and utilisation. Better efficiency per unit can coexist with higher total consumption if a company multiplies requests or deploys many more racks.

The bottleneck may sit outside the GPU

The 208 billion transistors attract attention, but a large AI system spends time moving weights, activations and partial results. If a model does not fit in one GPU's memory, it must be partitioned. Every boundary adds communication; when an accelerator waits for data, its theoretical peak goes unused.

Blackwell responds with fast memory, fifth-generation NVLink, NVSwitch and InfiniBand or Ethernet networking. GB200 NVL72 combines 30 terabytes of fast memory and is presented by NVIDIA as one coordinated domain. Useful capacity, however, depends on the model's footprint at the selected precision, the attention-cache requirement and the headroom required by software.

Power, cooling, space and operations also matter. A liquid-cooled rack is not installed like a conventional server: the facility needs coolant supply and return, suitable electrical distribution, leak control, maintenance procedures and trained staff. Chip availability does not mean every facility can receive it immediately.

Software is part of performance

The platform includes CUDA, mathematical libraries, TensorRT-LLM, training tools, networking and inference microservices. This layer translates a model into optimised kernels and determines how work is split across GPUs. An architecture can improve after launch without changing the silicon when a compiler fuses operations, a library reduces communication or an engine manages cache more efficiently.

It also creates dependency. Migration involves more than replacing a card: CUDA code, custom extensions, containers, observability, drivers, networking and team skills may all need review. An economic comparison should include switching costs, support, reserved cloud capacity and the risk that a key optimisation exists in only one stack.

Announcements from AWS, Google Cloud, Microsoft Azure, Oracle and server manufacturers indicated plans to offer Blackwell; they did not guarantee price, date or capacity for every customer. On 18 March 2024, NVIDIA was announcing a deployment roadmap. An adoption statement is not an independent measurement or proof of immediately available inventory.

A six-line record before trusting a benchmark

Any performance figure should be restated in six lines. First: the exact workload, model, size, architecture and input and output lengths. Second: the task, training or inference. Third: numerical precision and the test used to verify quality. Fourth: the complete configuration, from accelerator count to memory, network and cooling. Fifth: software versions and tuning. Sixth: metric and baseline, including whether results are normalised per GPU, rack, watt, request or unit of currency.

Then ask who measured it. A vendor projection explains product intent; a reproducible result or external benchmark provides another class of evidence; a test on the buyer's workload determines local usefulness. These sources are not interchangeable. The rigorous wording is “NVIDIA projects up to 30x under these conditions,” not “Blackwell is 30 times faster.”

For procurement, a pilot should set a quality budget and service objective before touching the hardware. Teams should measure useful tokens or tasks per unit of currency and kilowatt-hour, percentile latency, utilisation, errors, recovery time and operational complexity. Peak traffic and network failures belong in the test too: an ideal-condition average hides the moment that breaks a service.

Blackwell raises the technical ceiling through more compute, lower precision and rack-scale communication. Its durable lesson lies elsewhere: a multiplier without an axis is not a comparison. A reader who preserves workload, precision, metric, configuration, software and baseline can turn a spectacular promise into a testable hypothesis; lose any of them and only the slogan remains.