A token is neither a word nor a measure of intelligence
Tokens are pieces of text a model processes. Understanding them helps anticipate limits and costs, but a counter cannot judge answer quality.
"Token" is one of those technical words that slip into marketing and start to mean more than they do. An ad says "a million tokens of context" and the listener hears "a whole book fits, and it will understand it"; both are uncertain. It is worth taking the unit apart before we let it decide for us. The capability you take away from here is simple and firm: separate token, word, and context window, and measure a task by the useful information it resolves, not by a counter.
What a token exactly is
A token is a piece of text that a system turns into a processable unit. It can coincide with a whole word, but also with part of a word, a punctuation mark, or a space. Language models do not read letters or words: they chop text into these pieces according to a learned vocabulary, and common words usually take one token while rare, long, or technical ones are split into several. That is why the same text does not have the same number of tokens as words, and — this is what almost no one says — two different systems can chop the same paragraph differently and give different counts.
Two practical consequences follow. First, language matters. Text in a language poorly represented in the vocabulary, or a snippet of code, usually consumes more tokens than its plain-English equivalent, even though it says the same thing — a kind of silent "tax" on anyone not writing in the language the chopper was optimized for. Second, comparing prices or limits across providers by "number of tokens" without noting how each one chops is comparing two rulers with different markings.
What the counter is for, and what it is not
Tokens are, above all, the unit of the bill and the clock. Commercial language-model services charge by tokens: those that go into the request and those the model generates as its answer. Latency — the time it takes — also grows with how many must be processed and produced, so the same counter estimates two costs at once, money and clock. As an estimate of cost and time, the counter is useful and honest: if you know how many tokens go in and out of a typical task, you know what it will cost to repeat it a thousand times.
What the counter does not say is whether the answer will be correct. Transformer models work on sequences of tokens and attention mechanisms — the architecture described by Attention Is All You Need in 2017, dispensing with recurrence to process the whole sequence in parallel — but attending to more tokens is not understanding them better. A short text can omit the exception that decides the case; a long one can repeat a datum three times without using it well. The number measures how much material goes in, not how much of it is used.
There is also a cost asymmetry worth knowing: under typical pricing, generating an output token costs more than processing an input one, sometimes several times more — because producing new text is computationally costlier than reading what already exists. A model that answers in long-winded paragraphs is not only slower; it is pricier per answer than an equally good one that gets to the point. Counting tokens, then, is not stinginess: it is the difference between knowing what you pay and discovering it on the bill.
Not a measure of intelligence
The second half of the title is the one most often forgotten. Neither the number of tokens a model accepts nor the size of its window measures its intelligence. A system can swallow a million tokens and fail a logic step a child solves; it can have the largest context on the market and get a subtraction wrong. The ability to reason is measured by how it solves tasks, not by how much text it accepts as input — just as an office's memory is not measured by the size of its filing cabinets, but by what is done with what is filed. When an ad offers the token figure as if it were an IQ score, it is swapping a measure of capability — hard, debatable, task-specific — for one of storage capacity, which is easy to print and does not say the same thing.
Why a bigger window does not guarantee better reading
The most expensive confusion of the moment is treating the context window — how many tokens fit at once — as if it were comprehension. It is not, and there is measured evidence. The study Lost in the Middle examined how models use long contexts in multi-document question answering and key-value retrieval, and found a U-shaped pattern: performance is highest when the relevant information is at the beginning or end of the context, and degrades significantly when the model must reach for it in the middle. In other words, enlarging the window does not ensure the model actually uses everything that fits in it; it can "lose" precisely what is buried in the center.
That finding is not a laboratory curiosity: it is a usage instruction. If you feed in a long document and the datum that decides the answer sits in the middle paragraph, you are more likely to have the system overlook it than if you place it at the beginning or the end. A large window is a necessary condition for the material to fit; it is not sufficient for it to be read well.
How to use the counter with judgment
From all the above comes a routine that works with any model. Before trimming a text to make it fit, define the task: what question the system has to answer. Keep what changes the answer — instructions, dates, figures, exceptions, sources — and remove what does not — greetings, duplicates, filler. If the document is large, split it into sections and ask the model to cite the exact fragment it used, so you can check it. And, using what Lost in the Middle teaches, put the critical material at the edges: the first and last of a long prompt is where the model looks best.
The useful metaphor is a work table. Tokens are the table's surface; the context window, its size. Putting the papers you need to decide first is management; believing a bigger table guarantees someone will read them all carefully is a mirage. A huge table covered in badly ordered papers is no better than a small one with the three documents that matter on top.
The habit that remains
When a tool boasts of many tokens, ask it three questions: what specific task fits in there? what datum is critical to solving it? and how will I check that it was really used? The token count is a technical constraint — of cost, time, and capacity — not a promise of quality. Quality depends on the question asked, the sources fed in, and the review applied. Knowing how to read "a million tokens" as what it is — a table size, not a reading guarantee — is a capability that will keep working when the fashionable figure is another.
Primary sources
This article was produced with artificial intelligence under human editorial oversight.
