Gemini 2.5 Pro Debuts atop LMArena by a Wide Margin

Google unveiled Gemini 2.5 on March 25, calling it its "most intelligent" AI model to date. The first release is an experimental version of Gemini 2.5 Pro that, according to the company, debuts at #1 on LMArena "by a significant margin" and tops several of the industry's key benchmarks.

That claim carries weight because LMArena doesn't measure a single skill with a closed test — it tracks human preference. Users compare responses from different models without knowing which is which, then vote for the best one. Leading that leaderboard, and by a wide margin, suggests people consistently prefer what this model produces, not that it's been tuned to game one isolated benchmark.

What counts as a 'thinking model'

Google places Gemini 2.5 in a category it calls thinking models: systems "capable of reasoning through their thoughts before responding." In practice, that means the system works through internal steps to analyze a problem before producing its final answer, rather than generating text in one continuous pass.

The company is careful to spell out what it means by reasoning in this context. It's not just classification or prediction — it's the ability to "analyze information, draw logical conclusions, incorporate context and nuance, and make informed decisions." This is the direction the industry has been pushing toward for a while now, using techniques like reinforcement learning and chain-of-thought prompting (the method of getting a model to break down its reasoning step by step).

Google notes it had already taken a first step in this direction with Gemini 2.0 Flash Thinking. With 2.5, according to Koray Kavukcuoglu, CTO of Google DeepMind, the company combined a "significantly enhanced" base model with improved post-training. And it's signaling a bigger shift ahead: it plans to build these reasoning capabilities "directly into all" of its future models, rather than keeping them as a separate variant.

The numbers Google is putting forward

The announcement leans on several results. In advanced reasoning, Google says 2.5 Pro is state-of-the-art on tests like GPQA and AIME 2025 — demanding science and math benchmarks — and, notably, achieves this without relying on cost-inflating inference techniques, such as majority voting (generating many answers and picking the most common one). In other words, the performance comes from the model itself, not from a trick that multiplies the cost of every query.

On Humanity's Last Exam — a set of questions designed by hundreds of experts to capture "the human frontier of knowledge and reasoning" — the model scores 18.8% among systems that don't use external tools. That figure might look low in isolation, but the exam is deliberately built to be brutally hard: an 18.8% state-of-the-art result says more about the test's difficulty than about the model's shortcomings.

In coding, Google is touting "a big leap" over the 2.0 generation. On SWE-Bench Verified, which the company describes as the industry standard for agentic code evals, Gemini 2.5 Pro scores 63.8% with a custom agent setup. SWE-Bench measures something specific and demanding: whether a model can actually solve real problems in software repositories, not just write isolated code snippets.

As a demo, Google shows the model generating the executable code for a video game from a single line of instruction, using its reasoning to produce the complete application.

A million-token context window

Gemini 2.5 carries forward the two hallmarks of the family: native multimodality and a long context window. It ships with a 1 million token context window — Google says 2 million is "coming soon" — letting it process enormous volumes of information in a single pass, combining text, audio, images, video and even entire code repositories.

That number has real practical consequences. A million tokens means being able to "read" full documents, databases or software projects without chopping them up, keeping coherence across the entire material. For large-scale document analysis or code review, that's the difference between working with the whole picture versus scattered fragments.

Where and how to use it

As of the announcement, Gemini 2.5 Pro is available in Google AI Studio and in the Gemini app for Gemini Advanced users, who can select it from the model dropdown on both desktop and mobile. Google says it will reach Vertex AI, its enterprise platform, "in the coming weeks."

On pricing, the company is upfront that it hasn't set one yet: it will introduce pricing "in the coming weeks" to support higher rate limits for scaled production use. Launching first as a free experimental version across several channels, before announcing prices, fits a strategy of gathering mass usage and feedback before commercializing it.

Why this move matters

The most important detail here isn't any single benchmark — it's the underlying decision. Google says it's going to build these reasoning capabilities "directly into all" of its models. That points to a future where reasoning stops being a premium feature or a specialized model and becomes the default behavior.

The emphasis on hitting top-tier results "without cost-inflating techniques" also deserves attention. One of the drawbacks of reasoning models is that more thinking means more compute, which drives up the cost of every response. If Google can deliver this performance without those add-ons, the economics work in its favor once it's time to set prices.

Still, it's worth keeping the "experimental" label in mind. These results come from the company itself, and the model still lacks pricing or full availability on its enterprise platform. Topping LMArena, a measure of human preference, is a hard signal to argue with — but the real test comes once developers and companies put it to work on actual tasks with real costs on the table. For now, Google has set the bar and is inviting everyone to try it.