Google Opens Bard as NVIDIA Bolsters AI Infrastructure

On March 21, 2023, Google opened Bard’s wait list while NVIDIA announced new infrastructure at GTC; the contrast helps distinguish the visible application from the compute capacity supporting it.

Google opened initial access to Bard, its conversational chatbot, on Tuesday through a waitlist in the United States and United Kingdom. The move finally puts Google’s answer to ChatGPT head-to-head with Microsoft’s offensive, though availability remains limited and experimental.

Its timing alongside NVIDIA’s GTC conference illustrates the other side of this race: creating an assistant that writes text is not enough. It also requires enormous data-center infrastructure to train the system and handle millions of queries. NVIDIA used its event to unveil products aimed squarely at that market.

Bard Emerges from the Lab with a Waitlist

Bard is based on LaMDA, Google’s family of language models designed to sustain conversations. A language model is a system trained on vast amounts of text to predict and generate words; its apparent ability to hold a dialogue comes from that statistical learning, not from human understanding of the content.

Google announced Bard in early February, by which point ChatGPT had already turned generative assistants into a mass phenomenon. The company nevertheless kept the product in closed testing for several weeks. It is now allowing adult users with a Google account to join a waitlist, for now only in English and in the two English-speaking countries.

The company presents Bard as a tool for exploring ideas, drafting text or explaining complex topics. Its interface lets users ask questions and view several possible answers, making clear that Google is trying to distinguish it from a conventional search engine: a search returns links and sources; a chatbot composes a new response from learned patterns.

That difference also highlights the main risk. Bard, like ChatGPT and other similar models, can produce a convincing answer that is wrong or make up facts. Google warns that the system may make mistakes and asks users to provide feedback. This is no small matter: a public demonstration of Bard in February included a factual error about the James Webb Space Telescope, a reminder that these products are not yet reliable sources on their own.

Google Enters a Field Microsoft Has Already Taken to Search

The opening comes a week after OpenAI launched GPT-4 and after Microsoft integrated OpenAI technology into its new Bing. Google retains a dominant position in search, but ChatGPT’s success has called into question a central internet habit: entering a few keywords and browsing a list of results.

Bard is not currently integrated into Google Search. The separation makes technical and commercial sense. Bringing AI-generated answers to hundreds of millions of daily searches multiplies computing costs and raises the impact of every potential error. A wrong answer in a chat is problematic; a wrong answer on the home page of the world’s largest search engine could undermine trust in the entire service.

Google also has an advantage that is difficult to assess from the outside: it has been researching language models for years, developed the Transformer architecture underpinning much of today’s systems and has its own TPU chips for training. The news is not that Google arrived late to the research, but that it has decided to turn that research into a product open to the public.

NVIDIA Sells the Tools Behind the Generative AI Boom

As Google and Microsoft compete over the interface users will see, NVIDIA is looking to become the supplier of the machinery that makes these services possible. At its GTC conference, the company announced NVIDIA AI Foundations, a series of cloud services that let businesses create and adapt generative models.

The offering includes NeMo for text, Picasso for images, video and 3D graphics, and BioNeMo for research in biology and drug discovery. The proposal matters because training a model from scratch requires data, specialists and computing capacity available to few companies. Adapting an existing model to a specific task — customer service, product design or scientific research, for example — is a more realistic path for many organizations.

NVIDIA also unveiled the H100 NVL, a platform that combines two H100 GPUs and is designed for large language model inference. Inference is the phase in which an already-trained model responds to a user request. It differs from training and can become the largest recurring expense if a service receives millions of queries.

The H100 NVL combines two H100 GPUs, each with 94 GB of HBM3 memory — 188 GB in total — and is designed to run models with up to 175 billion parameters, the scale of GPT-3. Parameters are the internal values a model adjusts during training; they do not determine quality on their own, but they indicate the scale of resources required. Primary source

The Battle Will Be Decided by Products and Computing Capacity

Bard marks a new phase for Google: it will now be judged not only by its researchers’ demonstrations, but by the usefulness and reliability perceived by real users. Its gradual rollout will allow the company to collect reports of failures before expanding access, though it also gives rivals that have already added conversational assistants to commercial products more time.

For NVIDIA, the opportunity does not depend on Bard, ChatGPT or any other assistant winning. The more generative models are launched, the greater the demand for the chips, networks and software needed to train them and serve their responses. GTC leaves one conclusion clear: the competition over generative AI is being fought both in the chat window and in the data centers that support it.

The application and infrastructure use different units

Bard needed answer-level evaluation: accuracy, citations, usefulness and abstention rate. Infrastructure is measured through capacity, memory, utilisation, latency, energy and cost per task. Combining the two encourages the belief that more GPUs guarantee better answers. Hardware makes models runnable; data, training, interface and controls determine much of the experience.

NVIDIA introduced DGX Cloud on the same day as rented access to clusters and software. Its announcement included specifications and a starting price, but remained vendor material. A customer should test its own workload, record configuration and compare total cost with alternatives. Performance reported for another organisation does not transfer automatically.

Bard’s wait list was also a product stage rather than general availability. A comparison should fix eligible users, language, region, version and date. If one tool is open and another is not, a quality test without those conditions confuses distribution maturity with model capability.

The transferable skill is to draw the stack from chip to answer and assign a metric to every level. When a task fails, ask whether the constraint lies in compute, model, retrieval, interface or access. That separation improves the right component and resists the idea that one number decides the entire race.

Date the stack record, because every level changes at a different pace and a comparison without versions stops being reproducible.

Without a date, the stack cannot be audited.