Apple Questions Whether Reasoning Models Really Think

Apple published a paper this week that pokes a hole in one of the AI industry's most inflated claims: the idea that so-called reasoning models (LRMs, or Large Reasoning Models) genuinely think before they answer. The paper, titled "The Illusion of Thinking," appears on arXiv, authored by Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio and Mehrdad Farajtabar, and lands right as models like OpenAI's o1, DeepSeek-R1 and the "extended thinking" versions of Claude and Gemini are all the rage — every one of them marketed as capable of reasoning step by step before producing a response.

The authors don't evaluate these systems using the usual math or coding exams, which they consider problematic because those benchmarks tend to be contaminated: it's likely the model has already seen identical or very similar problems during training. Instead, they design controllable puzzle environments — including the Tower of Hanoi — where difficulty can be precisely tuned while the underlying logical structure stays the same. This makes something possible that traditional benchmarks don't offer: looking not just at whether the model gets the right answer, but at what happens inside its reasoning while it tries.

A Collapse With No Warning

The central finding is stark. As the paper's own abstract puts it, the authors "show that LRMs face a complete accuracy collapse beyond certain complexities." This isn't a gradual decline as the problem gets harder — it's an abrupt drop to near-zero accuracy once a certain threshold is crossed, no matter how well the model seemed to have handled the problem at lower difficulty levels.

Even more striking is what Apple calls a "counterintuitive scaling limit": the model's reasoning effort — measured by how much "thinking" it generates before answering — grows with the problem's complexity, but only up to a point. Beyond that, effort actually declines even though the model still has token budget left to keep trying. In other words: when the problem gets truly hard, the system doesn't try harder — it gives up sooner.

Three Regimes, None Definitive

Comparing LRMs against their standard language-model counterparts — models without that explicit "thinking" process — under the same compute budget, the researchers identify three distinct regimes. On low-complexity tasks, the simpler standard models outperform the reasoning models. On medium-complexity tasks, LRMs do show a clear advantage — precisely the territory where nearly all the commercial arguments in their favor have been built. But on high-complexity tasks, both types of model collapse equally: the extra reasoning stops contributing anything at all.

The paper adds another uncomfortable wrinkle: these models fail at exact computation. They don't apply explicit algorithms consistently, and their behavior varies erratically depending on problem scale, even when the underlying logical structure is identical and only the size changes.

Why This Matters Now

Much of the industry's 2025 narrative has been built around these reasoning models as the next step toward more reliable systems and, for some, toward more general forms of intelligence. That Apple — not a rival lab with an interest in deflating the competition, but a company with its own stake in AI — is publishing evidence that this reasoning falls apart without warning on more complex problems adds fuel to an already heated debate: whether what these models do amounts to genuine reasoning, or whether they're reproducing, with a lot of machinery, solution patterns seen during training.

The abstract itself leaves the question open rather than settled: the authors say their analysis is "shedding light on their strengths, limitations, and raising questions about their reasoning capabilities," not that it resolves them. That distinction matters: the paper doesn't claim LRMs are useless. It documents a pattern — an edge at medium difficulty, total collapse past a certain threshold, and a drop in effort right when it's needed most — that anyone using these tools for complex tasks should weigh before trusting them blindly.