Sparse attention in DeepSeek: less computation does not mean less reading
DeepSeek used an indexer to choose which tokens receive core attention and cut its API rates. Separating complexity, implementation, quality, caching, and pricing reveals whether the savings apply to a real workload.
On September 29, 2025, DeepSeek released V3.2-Exp, an experimental version of V3.1-Terminus incorporating DeepSeek Sparse Attention. The technique added a lightweight indexer to score earlier tokens and reserved the main attention computation for a small selection. At the same time, the company cut its API prices by more than half.
Both announcements concern cost, but they are not the same claim. Architecture seeks to reduce operations; implementation turns the idea into time and memory; a rate is a commercial decision. The durable skill is measuring each layer on the same workload instead of using a published price as automatic evidence of technical efficiency.
Core attention stops comparing everything with everything
DeepSeek’s technical report describes two components. The lightning indexer computes a score between each query token and preceding tokens. A selector then retains the 2,048 key-value entries with the highest scores for the main attention operation.
In dense attention, core work grows approximately with the square of length: doubling context can quadruple comparisons. Under DSA, main attention grows with length multiplied by the fixed selection count. When context is much larger than 2,048, that part can require substantially less computation.
The indexer is not magic: its complexity remains quadratic because it scores candidates across the context. DeepSeek argues that it uses few heads, can run in FP8, and costs much less than the main latent-attention operation. The accurate formulation is “it moves broad comparison into a lightweight index and makes the expensive operation sparse,” not “the model no longer looks at the rest.”
Selection means learning what may be discarded
The indexer was first trained for one thousand steps while all other parameters remained frozen. Its objective was to approximate the dense model’s attention distribution. That stage processed 2.1 billion tokens in 128,000-token sequences. DeepSeek then activated sparse selection and adapted all parameters for 15,000 steps and 943.7 billion tokens.
The procedure matters because a fixed rule, such as always attending to the beginning and end, would fail when evidence appears elsewhere. The indexer learns a score conditioned on each query. Still, selecting only a subset creates an unavoidable risk: the sentence required for the answer may fall outside the 2,048 retained entries.
A proper evaluation is adversarial about position. Names, figures, exceptions, and instructions are placed in different regions; similar distractors are added; distance from the question changes; and retrieval and answer quality are measured. A broad average can hide failure precisely on rare or contradictory information.
The experiment tried to isolate one modification
DeepSeek started from a V3.1-Terminus checkpoint extended to a 128,000-token context. It aligned long-context training data and retained the same post-training pipeline, algorithm, and data. According to the report, DSA was the only architectural change. This design supports attribution better than comparing models trained with entirely different recipes.
The official model card published results across knowledge, reasoning, mathematics, code, search, and tool use. Some scores rose, others fell, and others stayed equal. DeepSeek summarized the set as showing no substantial degradation against V3.1-Terminus, not universal superiority.
V3.2-Exp scored lower on GPQA, Humanity’s Last Exam, and HMMT. The company attributed part of the difference to the model generating fewer reasoning tokens and said intermediate checkpoints with comparable lengths closed the gap. This is the manufacturer’s hypothesis and analysis; an independent comparison should control the output budget too.
The cost chart has hardware and assumptions
The report estimated cost by token position using the deployed service on Nvidia H800 GPU clusters and assigning a rental price of two dollars per GPU-hour. It found a growing long-context advantage in prefill—processing the input—and decoding—generating the output. For short inputs, it used a masked dense mode to simulate DSA more efficiently.
Those details prevent a direct transfer of the curve. Another GPU, kernel, precision, batch size, parallel strategy, or utilization level can change the result. Even the short-context line uses a special path rather than sparse execution. A reproducible figure needs model, code commit, hardware, software, length, concurrency, and cost rule.
Time to first response and subsequent generation speed must also be separated. Prefill dominates when a huge document enters and the answer is short; decoding matters more during reasoning or long generation. One “tokens per second” figure can conceal which phase improved.
Rate and technical cost are separate axes
The official API announcement said prices fell by more than 50 percent and that deepseek-chat and deepseek-reasoner moved to V3.2-Exp. The launch table priced one million cached input tokens at $0.028, uncached input at $0.28, and output at $0.42.
The tenfold gap between cached and uncached input does not come from DSA alone. Caching reuses computation over an identical prefix. To obtain that rate, an application must send content compatible with the cache policy and have it remain available. A new document in every request will not receive the same saving.
A provider may set rates below, at, or above cost to attract demand, fill capacity, or change margins. The commercial cut therefore does not prove how much DSA saves, and a kernel improvement does not require the provider to pass through the entire saving. Only DeepSeek knows its full cost accounting.
Task cost needs more than a token rate
The bill separately multiplies cached input, new input, and output tokens by their rates. It then adds retries, tools, document retrieval, and failed calls. An agent making five cheap calls can cost more than one answer that succeeds on the first attempt.
The useful metric is cost per accepted outcome. Define a task, quality rule, and stable test set; record the total cost of every attempt, latency, errors, and human review; then divide by outputs that pass. An incomplete answer no longer appears cheaper merely because it generated fewer tokens.
Hosting the weights has a separate cost. The MIT license permits use and modification of the repository and model, but downloading a checkpoint with hundreds of billions of parameters does not make it inexpensive to run. Hardware, memory, storage, energy, engineering, and availability compose total cost.
Open weights enable verification, not guaranteed reproduction
DeepSeek published weights, inference code, and kernels for several implementations. This allows inspection of architecture, local testing, and change comparison. The model card identifies a 685-billion-parameter model and provides distributed deployment instructions.
Reconstructing the complete checkpoint would additionally require data, training procedure, and enormous compute not delivered as a household recipe. “Open weights” describes access to the artifact; “open source” describes software licensing; “reproducible training” describes the ability to rebuild it. They are different degrees of openness.
The release does facilitate the test most relevant to DSA: run the same model on representative workloads and observe memory, latency, and accuracy. Version must be fixed because later kernel changes can correct discrepancies. The experiment should retain hashes and raw results.
A matrix joins efficiency and quality
Rows are context lengths and task types; columns are prefill, decoding, peak memory, cost, retrieval, accuracy, and stability. Add cache-hit and cache-miss variants, relevant data at different positions, multiple batch sizes, and concurrency. V3.1-Terminus is the control when both share an environment.
A gain is valid for the measured cell, not every use. DSA may help long documents greatly and brief chats little. It may reduce core attention without lowering network, retrieval, or tool costs. It may preserve averages while losing a critical exception.
The transferable skill is requiring “more efficient” to finish its sentence: efficient in which operation, length, hardware, implementation, quality, and workload. DeepSeek-V3.2-Exp offered a mechanism, weights, and a valuable initial test. Turning it into a decision requires measuring accepted outcomes, not letting a rate or curve speak for the whole system.
This article was produced with artificial intelligence under human editorial oversight.
