Google unveils Gemini 1.5 Pro with one million tokens

On February 15, 2024, Google introduced Gemini 1.5 Pro with a standard 128,000-token context window and an experimental one-million-token mode for a limited group. Google’s announcement publishes examples and internal results; “fits in context” does not mean “the model understands, retrieves and reasons about it correctly”.

In practical terms, Google estimates that the limit is equivalent to around 700,000 words, 11 hours of audio, one hour of video or tens of thousands of lines of code. The model does not just process text: Gemini 1.5 Pro retains Gemini’s multimodal capabilities and can combine text, images, audio and video in a single prompt.

A context window more than 30 times larger than Gemini 1.0

A model’s context window is its temporary workspace. It determines how much material the model can read and connect before responding. It should not be confused with permanent memory: processing a book during one query does not mean the system will remember it in future conversations.

Gemini 1.0 Pro, introduced by Google in December, offered a context window of around 32,000 tokens. Gemini 1.5 Pro raises that figure to one million—more than 30 times the previous limit. Google DeepMind is also testing contexts of up to 10 million tokens in research, although that capability is not part of the preview announced for developers and businesses.

Until now, models with large context windows had shown that they could accept extremely long documents, but not always that they could accurately use information located at the beginning or in the middle of that material. The challenge is not simply storing more text, but retrieving the relevant detail and reasoning about it without getting lost among thousands of pages.

Google says Gemini 1.5 Pro achieves near-perfect retrieval in its tests of finding information within large volumes of content, including combinations of text, audio and video. These are evaluations developed by the company itself, so its performance in real-world tasks and with less structured information remains to be tested.

Mixture of experts: activating only part of the model

The other technical innovation is its mixture-of-experts architecture, known as MoE. Rather than activating every internal component of the model for each prompt, a routing system directs each part of the task to the most suitable “experts.”

This approach makes it possible to increase the model’s overall capacity without requiring every response to use all of its parameters. Google has used the technique in previous research, and other companies deploy it in some of their models, but Gemini 1.5 Pro brings it to one of the company’s flagship products.

Google says the model delivers performance comparable to Gemini 1.0 Ultra, its most powerful version to date, even though Gemini 1.5 Pro is described as a mid-sized model. The company has not disclosed the number of parameters in this version, a figure that by itself also cannot reliably measure a model’s usefulness.

From contract analysis to software maintenance

One million tokens opens up clear possibilities for people working with large amounts of information. A law firm could ask the model to compare a contract with hundreds of related documents. A company could analyze a complete meeting transcript, internal manuals and selected emails without splitting them into chunks. A programming team could provide the system with a large codebase to identify dependencies or explain the impact of a change.

There are audiovisual applications as well. A model given an hour of video could answer questions about a specific sequence, locate an element that appears briefly or connect an oral explanation with a slide shown later.

But a long context does not remove the usual limitations of generative AI. The model may misinterpret an instruction, make up a plausible answer or overlook an important exception. In legal, financial or medical settings, its output will still require human review and controlled access to data.

Initially available to a limited group

Gemini 1.5 Pro is initially arriving in private preview for developers and enterprise customers through AI Studio and Vertex AI, Google’s platforms for building and deploying AI applications. The company has opened a waitlist for those who want to try the one-million-token context window.

The announcement puts context size among the main areas of competition between leading AI labs. Over the past year, the race has focused on getting models to write, code and reason better; now, how much material they can handle at once matters too. The decisive test will be whether that scale translates into reliable, fast and cost-effective responses for businesses that want to incorporate it into their tools.

Available context is not reliable memory

A token is a model unit, not a word or page. The same window holds different amounts according to language, code, formatting and content. The limit also states only what may be sent in one request. It does not guarantee that the system will find a small clause, connect distant facts or ignore a hostile instruction hidden in a document.

To measure retrieval, insert unique facts at different positions and request an exact supporting quotation. To measure reasoning, distribute premises that must be combined and add distractors or conflicting versions. Keep the tests separate: repeating a hidden sentence does not show understanding of a file, and reasoning over an excerpt does not show it can be found in the collection.

A long window changes application design

Sending every file may simplify a prototype but increases cost, latency and privacy exposure. Retrieval selects passages before calling the model and leaves a clearer trace of which documents mattered. Compare full context and retrieval on the same collection, queries and accuracy criterion.

Order matters too. A model may attend better to the beginning and end than the middle or obey an instruction embedded in a file. Applications should separate data from commands, record provenance and require citations the user can open. More input capacity expands both useful material and material capable of confusion.

The transferable skill is to evaluate long context in four layers: fit, retrieval, reasoning and cost. Preserve evidence position, answer, citation and time. The protocol remains useful when the next model advertises an even larger window and prevents a capacity figure from becoming a promise of understanding.

The test should include abstention. If required evidence is absent from the documents, the model should say so rather than fill the gap with prior knowledge. Add impossible questions and score whether absence is distinguished from difficulty. A system that always answers may look useful while fabricating the very details a long window was meant to collect.

Finally, repeat after an update. Commercial services can change without application code changing, so version, date and configuration belong to the result. Preserving cases and outputs reveals regressions and avoids attributing to “Gemini” behaviour that belonged to one revision.

For living documents, preserve each source date. A large window may contain incompatible versions and produce a fluent synthesis that ignores which superseded which. Requesting a table of document, revision and citation before the summary turns context into a verifiable chain.