Intel launches Gaudi 3 to challenge Nvidia in AI market
Intel has unveiled Gaudi 3, its new accelerator for training and running AI models. The company promises higher performance at a far lower cost than systems built around Nvidia’s H100.
Intel unveiled Gaudi 3 on Tuesday, a new generation of accelerators—chips specialized for artificial intelligence workloads—that it hopes will provide a real alternative to Nvidia in data centers. The company says its platform can train large models faster than the H100 and substantially reduce the cost of deploying them.
The announcement comes as Nvidia’s H100 has become a scarce and expensive component for companies, research labs and cloud service providers. It is not just a question of which chip is the most powerful: availability, power consumption and the price of complete systems have become decisive factors in putting generative AI into production.
More memory and integrated networking for large models
Gaudi 3 follows Gaudi 2 and is designed for two distinct tasks. The first is training, the process of tuning a model with huge collections of data. The second is inference—using an already-trained model to answer questions, generate text or summarize documents.
Intel has equipped the chip with 128 GB of high-speed HBM2e memory and integrated Ethernet connections running at 200 gigabits per second. That integration is one of the defining features of the Gaudi family: rather than relying on a proprietary network to connect large numbers of accelerators, Intel is betting on Ethernet, the standard that already dominates data centers.
The idea has a practical consequence: building AI servers and server clusters could require fewer specialized components. That does not eliminate the complexity of deploying thousands of chips, but it could make it easier for manufacturers and customers to use networking equipment they already know.
Intel promises advantages over the H100
Intel’s published figures are based on tests using Llama 2, the family of language models released by Meta. In a configuration with 64 accelerators, the company says Gaudi 3 can train the 70-billion-parameter version of Llama 2 up to 40% faster than Nvidia’s H100.
For inference, Intel puts the advantage at between 1.5 and 2 times that of the H100, depending on the model size and the configuration tested. These results should be viewed with caution: vendor benchmarks use specific models, numerical precision, software and numbers of simultaneous users. A strong result on Llama 2 does not guarantee the same difference across all models or every workload at a company.
The most aggressive selling point is price. Intel estimates that a system with eight Gaudi 3 accelerators will cost around $125,000, compared with roughly $300,000 for a comparable configuration with eight H100s. That is the company’s own estimate, and the final cost will depend on the server manufacturer, networking, storage and support contracts. Still, it illustrates the strategy: compete where Nvidia has left a wide margin amid strong demand.
The challenge is not just making chips
Gaudi 3 will be available to equipment manufacturers during the second quarter of 2024. Dell, Hewlett Packard Enterprise, Lenovo and Supermicro are among the partners preparing servers with the new accelerator.
Intel must nevertheless solve a problem that cannot be measured in teraflops alone. Nvidia has spent years building a software ecosystem around CUDA, its GPU programming platform. Many tools, libraries and engineering teams are already designed for that environment. Switching suppliers requires companies to adapt workflows, validate models and train staff.
Intel offers its SynapseAI software stack and has worked to make Gaudi compatible with widely used frameworks such as PyTorch. But adoption will depend on whether its cost and performance claims hold up outside the company’s labs and with the models customers actually use.
For the market, Gaudi 3 matters even before it proves an outright technical victory. The industry needs more accelerator supply to ease generative AI bottlenecks. If Intel can deliver systems at scale and maintain its price advantage, it will give companies and cloud providers negotiating leverage they currently have little of against Nvidia.
