Large AI Models Hit Diminishing Returns as They Scale
Several reports suggest that AI labs are seeing smaller gains as they expand pretraining. The focus is shifting toward reasoning, data and inference-time compute.
The labs leading generative AI are seeing less dramatic gains as they scale their models with more data and computing power. Reports published in recent days do not confirm the end of so-called scaling laws, but they do challenge the idea that simply training a larger model is enough to deliver a leap comparable to those of earlier generations.
The debate matters because that recipe has driven the AI race since GPT-3 emerged in 2020: more chips, more text and more parameters—the internal variables a model learns—produced more capable results. If the returns on that investment decline, both technical priorities and the economics of the industry will change.
What scaling laws are
Scaling laws describe a pattern observed during training: as model size, data volume and the amount of computation increase, prediction error tends to fall in a relatively predictable way. OpenAI’s 2020 paper helped turn that observation into an industrial roadmap.
That did not mean a model could learn anything simply by growing larger. It primarily measured its ability to predict the next word or piece of information. But that statistical improvement translated into practical capabilities: writing, coding, summarizing, translating and following instructions much more reliably.
In 2022, DeepMind researchers refined the formula with Chinchilla. Their conclusion was that many models were oversized for the amount of data used to train them: adding parameters was not enough; the training material also had to increase. The industry responded with larger models and even bigger datasets.
The problem isn’t just building larger models
The most obvious constraint is that public internet text is not growing at the same rate as investment in data centers. Labs are already turning to code, books, licensed data, multilingual content and synthetic data—in other words, information generated by other models. Each alternative brings its own problems involving quality, copyright or the risk of repeating errors found in the original data.
The bill is rising, too. Training a frontier model requires huge clusters of specialized processors, electricity, networking and months of engineering work. Meta introduced Llama 3.1 405B in July, an open model with approximately 405 billion parameters. The figure illustrates the scale these systems have reached, although parameter count alone no longer allows for precise comparisons between two systems.
Signs of diminishing returns do not mean models have stopped improving. A small reduction in training error can still be useful, and capability evaluations do not always advance at the same pace as that metric. Companies also do not publish all their experiments or internal data, making it difficult for outsiders to distinguish a genuine technical limitation from a product or cost decision.
More compute when the model responds
The alternative attracting the most attention is allocating more computation to inference—the point at which the system responds to a query. Instead of producing the first plausible answer, the model can generate several intermediate steps, check them and spend more time on difficult problems.
OpenAI introduced its o1 series in September with precisely this approach. The company said the model performed better on science, coding and math tasks by spending more time reasoning before answering. It is a different path from pretraining: it does not necessarily involve making the base model much larger, but rather using it more deliberately.
The cost is clear. An answer that requires more computation takes longer and costs more to serve. For an assistant handling millions of daily queries, that difference directly affects pricing and the infrastructure required.
Data, architectures and products
The next advance will probably not depend on a single lever. Labs are working on higher-quality data, tools that allow models to search for information or run code, systems that verify their own answers and more efficient architectures. It also matters how a laboratory capability is turned into a reliable product: a model may solve a complex problem in a test and then fail when faced with an ambiguous instruction.
The question for companies and users is not whether large models will disappear, but what kind of improvement they will deliver and at what cost. If pretraining produces smaller gains, the competitive advantage may shift toward whoever has better data, better reasoning systems and the ability to operate those models without making each query prohibitively expensive.
