Google’s AI Overviews stumble with dangerous advice

Google has just launched its AI Overviews in the United States, AI-generated summaries that appear above the search engine’s conventional results. The feature promised to save time by synthesizing information, but this week it went viral for serving up absurd and, in some cases, potentially dangerous recommendations.

Examples shared by users include a response advising them to add non-toxic glue to pizza sauce to keep the cheese from sliding off. Another suggested eating at least one small rock a day. These are not merely wording errors: they appear inside Google’s interface, formatted as direct answers and presented alongside links that seem to support them.

The problem is more than one strange answer

A language model does not verify that a statement is true before writing it. It predicts which words are most likely to come next based on vast amounts of text. That allows it to summarize, explain and converse fluently, but it can also reproduce jokes, false information or advice stripped of its context when those appear in the sources it retrieves.

In the pizza case, the recommendation appeared to come from a satirical comment posted on Reddit years ago. The system did not understand that it was a joke, nor did it apply a strong enough safeguard to stop that kind of food advice from reaching the final answer.

That is the difficult part of bringing generative models into search. A traditional results page shows links and lets users compare sources. An automated summary, by contrast, compresses that chain of checks into a short piece of text, placed in the most visible part of the page and written in a confident tone.

The trust Google inspires does not automatically transfer to every sentence generated by a model. The company itself includes warnings telling users to verify important information. Yet asking for that caution after placing a synthesized answer above the links creates a practical contradiction: the more direct the answer appears, the less likely many people are to open the original sources.

A central bet on the future of Search

Google introduced AI Overviews on May 14 at its Google I/O conference as part of a broader transformation of its search engine. The company says the tool can help with complex queries that once required multiple searches, such as comparing products, planning activities or bringing together scattered information.

The ambition is understandable. Google is competing with assistants such as OpenAI’s ChatGPT and with the chatbots Microsoft has integrated into Bing and Windows. They are all chasing the same promise: users can ask a complete question and receive a developed answer, rather than a list of links.

But search has different demands from a creative chatbot. For queries about health, food, safety or public affairs, a convincing fabrication can be more harmful than an incomplete answer. The problem is not just that the model may get something wrong; it also matters whether it can properly distinguish between a reliable source, a forum, a parody and content designed to attract clicks.

Google has argued that the examples being shared involve uncommon queries and that most summaries provide useful information. The company has also begun limiting the appearance of AI Overviews for some problematic searches. Even so, the speed with which these cases spread shows that earlier testing did not adequately anticipate how the feature would interact with the variety, humor and low quality of the open web.

Verify before following advice

The episode does not mean automated summaries are useless. They can save steps on simple tasks, especially when they link to clear sources and users need an initial overview. But they should not be treated as an independent authority.

For recommendations affecting health, food, money or safety, it is worth opening the links, checking who published the information and turning to specialist organizations or professionals when necessary. That was already good practice online, but it matters even more when an incorrect answer is wrapped in the authority of Google’s page.

For Google, the immediate challenge will be proving that it can make AI Overviews more useful without turning search into a megaphone for plausible but false answers. The feature’s quality will not be measured by how natural it sounds, but by its ability to recognize when it should not answer.