Gemini 3.5 Live Translate: how to measure live translation
Google launched continuous speech translation for more than 70 languages. Its model card reveals the test that matters: quality, delay and naturalness are separate axes, and comparable numerical results remain unpublished.
On 9 June 2026, Google introduced Gemini 3.5 Live Translate, a model that receives audio and continuously generates translated speech. The company says it detects more than 70 languages, preserves intonation, pacing and pitch, and stays a few seconds behind the speaker. Those are vendor claims, not an independent comparison.
The launch offers a lesson in evaluating any live automated interpreter. “Translates well” blends at least three problems: accuracy of meaning, delay behind the speaker and naturalness of synthetic speech. Improving one may harm another. Every language direction, accent, noise level and turn-taking situation must also be measured. Announcing 70 languages does not demonstrate equal quality across more than 2,000 possible combinations.
Continuous does not mean simultaneous at no cost
A turn-based system waits for a pause or the end of an utterance, processes the segment and responds. Gemini 3.5 Live Translate processes an audio stream and starts producing translation while input continues. The Live API documentation explicitly distinguishes this mode from a conversational agent: translation handles continuous audio, does not accept instructions or tools, and takes audio only as input.
The linguistic difficulty remains. In some pairs, the verb, negation or gender becomes known late. Waiting gains context but increases delay; speaking early may force a choice before the required information arrives. The system can revise, but an audible correction also interrupts comprehension. “Near real time” names this trade-off rather than an absence of latency.
Timing the first sound is therefore insufficient. The model card says Google measures two delays. Initial latency runs from the start of source speech to the start of translated audio. Word-level latency aligns input and output words and measures the time between the end of a source word and the beginning of its translated equivalent. The latter reveals a system that starts quickly but falls behind as the session progresses.
What Google launched and with what access
From 9 June, developers could access the model in public preview through the Gemini Live API and Google AI Studio. Google Meet began a private preview for selected Workspace business customers, with broader rollout promised later in 2026. Google Translate started a global rollout on Android and iOS. “Available” therefore meant different things in each product.
For Meet, Google promised an expansion from five languages to more than 70 and an end to translation pairs that required English as source or target. The company described more than 2,000 combinations within a meeting. In Translate, the function uses headphones, while Android began adding a listening mode through the phone’s earpiece. These are scope and schedule claims from Google, not evidence of uniform quality.
For developers, the identifier is gemini-3.5-live-translate-preview. Documentation specifies mono 16-bit PCM at 16 kHz for input, PCM output at 24 kHz and 100-millisecond chunks. The service takes a target language and can return transcripts. In this mode it offers no text input, function calling, search or instructions: it is a translation pipeline, not a general agent under another name.
What the card measures—and does not publish
Google organises evaluation along three dimensions. Translation quality uses an internal implementation of AutoMQM, a metric that identifies and categorises errors. Latency uses the two timing measures described above. Naturalness checks for choppy audio, discontinuities, voice drift and unintended artefacts. Outputs were generated with the Gemini Live API.
The card provides no tables with scores, language pairs, sample sizes, competitor comparisons or human results by accent. It publishes no latency percentiles. It is accurate to say that Google evaluated those axes; it is not accurate to claim the model beat a named rival or performs equally across languages. The absence matters especially when value changes if a delay moves from two seconds to five, or errors cluster in one group.
Naturalness must also remain separate from faithfulness. Preserving a question, hesitation or emphasis may improve communication. But a semantically wrong translation spoken in the interlocutor’s convincing voice may inspire more trust than a robotic voice. A test cannot let pleasant audio compensate for an error in meaning.
The limitations Google does disclose
The card warns that a voice may shift after long pauses, change gender or become stuck on one voice during rapid multi-speaker sessions. Language detection may struggle with non-native accents, similar languages or fast language switches. The system is designed to filter noise, but not all background audio is ignored; an option that echoes the target language may introduce artefacts when music or noise is present.
These are not fringe cases for a conversational tool. A passenger pickup may combine proper names, traffic noise and overlapping speakers. A meeting may insert English terms into Spanish. A classroom may include echo and distant voices. A useful test reproduces these conditions instead of relying solely on clean recordings with one person speaking at a time.
Google says all generated audio carries SynthID, an imperceptible watermark. This provides a provenance signal inside a compatible ecosystem. It does not by itself establish that every recipient can detect it, how well it survives each transformation or that unmarked audio is human. Where context permits, provenance also requires origin, service version and session records.
A test bench for any speech translator
First build a matrix, not a demo: each language direction separately; native and non-native accents; slow and fast speech; one and several voices; noise, proper names, numbers and language switching. Spanish-to-English performance does not establish English-to-Spanish results, still less results between two lower-resource languages.
Second, bilingual reviewers label errors by severity. A style preference does not carry the same weight as reversing a negation, changing a dose, confusing who accepted a condition or altering a date. Omissions, additions and late corrections are recorded too. An average quality score should travel with the count of critical errors.
Third, measure delay as a distribution. Publish median and high percentiles for both initial and aligned-word latency, alongside session duration and network conditions. An average may hide pauses that break a conversation. Fourth, evaluate naturalness separately, including voice-identity stability across pauses and speakers.
Fifth, test the actual task. In a meeting, do participants correctly assign a decision, owner and deadline? On a tour, do they understand an instruction? In customer support, can both sides repair a mistranslation? In medicine, law or emergency response, output should not replace a qualified interpreter without a specific validated protocol.
The rule that remains after launch day
Google’s distribution—Translate, Meet and an API—can carry the feature from a phone into third-party products. The model is also in preview, and its own documentation describes consequential failures. Both can be true at once.
For the next announcement, ask for a table with quality by language direction and acoustic condition, latency percentiles, voice stability and critical errors. Until it exists, “more than 70 languages” describes nominal coverage and “a few seconds” summarises a vendor promise. Sound evaluation turns neither into a universal guarantee.
This article was produced with artificial intelligence under human editorial oversight.
