Skip to main content

Nvidia built a massive dual GPU to power models like ChatGPT

Nvidia’s semi-annual GPU Technology Conference (GTC) usually focuses on advancements in AI, but this year, Nvidia is responding to the massive rise of ChatGPT with a slate of new GPUs. Chief among them is the H100 NVL, which stitches two of Nvidia’s H100 GPUs together to deploy Large Language Models (LLM) like ChatGPT.

The H100 isn’t a new GPU. Nvidia announced it a year ago at GTC, sporting its Hopper architecture and promising to speed up AI inference in a variety of tasks. The new NVL model with its massive 94GB of memory is said to work best when deploying LLMs at scale, offering up to 12 times faster inference compared to last-gen’s A100.

Nvidia's H100 NVL being installed in a server.

These GPUs are at the heart of models like ChatGPT. Nvidia and Microsoft recently revealed that thousands of A100 GPUs were used to train ChatGPT, which is a project that’s been more than five years in the making.

The H100 NVL works by combining two H100 GPUs over Nvidia high bandwidth NVLink interconnect. This is already possible with current H100 GPUs — in fact, you can connect up to 256 H100s together through NVLink — but this dedicated unit is built for smaller deployments.

This is a product built for businesses more than anything, so don’t expect to see the H100 NVL pop up on the shelf at your local Micro Center. However, Nvidia says enterprise customers can expect to see it around the second half of the year.

In addition to the H100 NVL, Nvidia also announced the L4 GPU, which is specifically built to power AI-generated videos. Nvidia says it’s 120 times more powerful for AI-generated videos than a CPU, and offers 99% better energy efficiency. In addition to generative AI video, Nvidia says the GPU sports video decoding and transcoding capabilities and can be leveraged for augmented reality.

Nvidia says Google Cloud is among the first to integrate the L4. Google plans on offering L4 instances to customers through its Vertex AI platform later today. Nvidia said the GPU will be available from partners later, including Lenovo, Dell, Asus, HP, Gigabyte, and HP, among others.

Editors' Recommendations

Jacob Roach
Senior Staff Writer, Computing
Jacob Roach is a writer covering computing and gaming at Digital Trends. After realizing Crysis wouldn't run on a laptop, he…
Nvidia GPUs see massive price hike and huge demand from AI
Nvidia RTX 2060 Super and RTX 2070 Super review

It feels like we’ve only just emerged from the debilitating graphics card shortage of the last few years, but a new report suggests we can’t breathe easy just yet. Could a new GPU shortage be on the horizon, or are consumers safe from a return to another nightmare scenario?

According to DigiTimes (via Wccftech), Nvidia is seeing a huge surge in demand for its chips due to the explosion in artificial intelligence (AI) tools like ChatGPT. Nvidia offers a range of graphics cards that excel at AI tasks, including the A100 and H100, and the company is reportedly struggling to keep up in the wake of such massive demand.

Read more
Is Apple making its own version of ChatGPT? It seems that way
AI assistants compared with ChatGPT.

Apple is likely working on some kind of answer to ChatGPT and generative AI -- because of course it is. It would have been safe to assume that without evidence, but now we have some solid proof that Apple is starting to take AI seriously.

As noticed by TechCrunch, Apple currently has posted 28 AI-related jobs in May alone, and 9to5Mac points out that there's a total of 88 open jobs at Apple that are somehow related to AI. That's a lot, especially considering the hiring freeze that was instituted earlier this year.

Read more
OpenAI’s new ChatGPT app is free for iPhone and iPad
The ChatGPT website on an iPhone.

OpenAI has just launched a free ChatGPT app for iOS, giving iPhone and iPad owners an easy way to take the AI-powered tool for a spin.

The new app, which is able to converse in a remarkably human-like way, is available now in the U.S. App Store and will come to additional countries “in the coming weeks,” OpenAI said. Android users are promised their own ChatGPT app “soon.”

Read more