DeepMind releases DiffusionGemma, a text model up to 4x faster

DeepMind has introduced DiffusionGemma, an experimental open model that applies diffusion to text generation. Rather than writing word by word, as conventional language models do, it works on entire blocks of up to 256 tokens—text fragments—and refines them over several passes.

The company says this approach delivers generation speeds of up to four times faster on a dedicated GPU. The announcement matters less as a replacement for today’s large language models than as an exploration of an alternative for tasks where immediate responses, editing and local execution matter more than achieving the highest possible quality.

A 26-billion-parameter model that activates 3.8 billion

DiffusionGemma is a mixture-of-experts model, an architecture that brings together several specialized submodels but activates only some of them for each operation. It has 26 billion parameters in total, although it activates 3.8 billion during inference.

That difference reduces the cost of running the model. DeepMind says that, when quantized—meaning it uses lower-precision numbers to save memory—it can run within 18 GB of video memory. That puts it within reach of computers with high-end graphics cards, although it does not make local deployment trivial: it still requires specialized hardware and the right setup.

According to figures published by DeepMind, the model exceeds 1,000 tokens per second on an NVIDIA H100 and 700 tokens per second on a GeForce RTX 5090. These measurements were taken on specific accelerators and do not necessarily correspond to the speed a user will see in every application: final performance depends on text length, quantization, inference software and workload.

From sequential text to parallel refinement

Conventional assistants generate text autoregressively: they predict one token, add it to the context and repeat the process. It is an effective method and the standard for producing high-quality responses, but it requires waiting for each step before generating the next.

Diffusion changes that mechanism. The model starts with unresolved text positions and iteratively corrects them until it produces a coherent output. By processing an entire block, each position can take the others into account. This bidirectional attention can be useful in tasks where linear order is a limitation: completing code in the middle of a file, editing a sentence without rewriting the rest, generating Markdown structures or solving sequences with global constraints.

DeepMind also cites code infilling, amino acid sequences and mathematical graphs. As a fine-tuning demonstration, Unsloth has adapted the model to solve Sudoku puzzles, a problem in which an initial decision depends on cells that have not yet been filled in.

The speed advantage does not extend equally to every environment

The key caveat of the release is where that advantage applies. Parallel generation makes better use of a GPU when it is handling few requests, as on a personal computer, workstation or interactive application for a single user.

In cloud services handling thousands of requests, autoregressive models can batch many conversations together and use hardware efficiently. In that scenario, DiffusionGemma’s advantage diminishes and the cost of serving each response may even increase. It is therefore not a universal solution for making large-scale chatbots cheaper.

Nor is it intended to be DeepMind’s highest-quality model. The company itself acknowledges that its overall output falls below that of autoregressive Gemma 4 models and recommends the latter for production uses that require the best possible response. DiffusionGemma prioritizes speed and parallel generation, two goals that still involve trade-offs in open-ended, complex text.

Open weights and room to experiment

The weights are being released under the Apache 2.0 license, a permissive license that allows users to use, modify and distribute the software with few restrictions. The model will be available on Hugging Face and will include runtime tools through MLX, vLLM and Hugging Face Transformers; DeepMind also says support for llama.cpp is coming.

DiffusionGemma’s significance lies in bringing to text models an idea popularized by image generators: producing an answer through refinement rather than through a single chain of predictions. If the approach demonstrates sufficient quality in editing, programming or local interfaces, it could open up a category of assistants that are less like a typewriter and more capable of reviewing an entire page before delivering it.

Tokens per second are not time to a useful answer

The DeepMind material describes more than 1,000 tokens per second on an H100 and blocks of 256 tokens. An interface has at least three relevant times: delay to first output, time to completion and time to an answer that passes the quality threshold. A system may produce text rapidly but require additional refinement or correction.

A comparison should fix hardware, precision, length, number of steps, batch size and quality. If an autoregressive model and a diffusion model deliver different answers, raw speed does not determine which finished the work first. A single-user test also cannot be transferred to a concurrent service without measuring how requests share the GPU.

Tasks where order changes the advantage

In sequential generation, every token depends on previous ones. In diffusion, positions within a block can be revised together. That is promising when an answer has global constraints: filling a function without changing the rest, fitting a table, following a schema or revising a sentence in context. For open conversation, quality and stability across several turns may matter more than peak speed.

A useful evaluation includes three groups: generation from scratch, local editing and structured output. It records accuracy, unrequested changes, format, latency and memory. It adds long cases and repeats runs. This reveals whether the architecture offers a general advantage or a valuable specialisation for certain workflows.

Local execution also has several boundaries. Quantised weights fitting in memory do not guarantee speed, compatibility or complete privacy: the application may transmit telemetry or depend on other services. Check the licence, downloaded files, runtime, network connections and the effect of quantisation on quality.

Memory needs the same care as speed. A figure may include only weights, or also caches, activations, runtime and system headroom. A configuration that fits at startup may fail with a long input or several users. Publishing peak memory, length, concurrency and runtime version shows whether “local” describes a minimal demonstration or a stable application. Measurement should be repeated with the longest and busiest load the product promises to serve.

A reading skill that survives the next architecture

The number of refinement steps belongs in the compute budget. Reducing it may speed output while damaging structure, accuracy or diversity; increasing it may improve quality up to a point while taking longer. A DiffusionGemma comparison should publish that setting and plot quality against latency instead of selecting one favourable configuration. Each application can then choose its own trade-off rather than inheriting the announcement’s.

The transferable skill is always to demand a work unit, configuration and quality threshold before accepting “faster.” DiffusionGemma shows another route for generating text. Its practical value will emerge when a comparison states not only how many tokens it produces, but which task it completes, with which errors and in which environment.