Skip to main content

Nvidia built a massive dual GPU to power models like ChatGPT

Nvidia’s semi-annual GPU Technology Conference (GTC) usually focuses on advancements in AI, but this year, Nvidia is responding to the massive rise of ChatGPT with a slate of new GPUs. Chief among them is the H100 NVL, which stitches two of Nvidia’s H100 GPUs together to deploy Large Language Models (LLM) like ChatGPT.

The H100 isn’t a new GPU. Nvidia announced it a year ago at GTC, sporting its Hopper architecture and promising to speed up AI inference in a variety of tasks. The new NVL model with its massive 94GB of memory is said to work best when deploying LLMs at scale, offering up to 12 times faster inference compared to last-gen’s A100.

Nvidia's H100 NVL being installed in a server.
Nvidia

These GPUs are at the heart of models like ChatGPT. Nvidia and Microsoft recently revealed that thousands of A100 GPUs were used to train ChatGPT, which is a project that’s been more than five years in the making.

Recommended Videos

The H100 NVL works by combining two H100 GPUs over Nvidia high bandwidth NVLink interconnect. This is already possible with current H100 GPUs — in fact, you can connect up to 256 H100s together through NVLink — but this dedicated unit is built for smaller deployments.

This is a product built for businesses more than anything, so don’t expect to see the H100 NVL pop up on the shelf at your local Micro Center. However, Nvidia says enterprise customers can expect to see it around the second half of the year.

In addition to the H100 NVL, Nvidia also announced the L4 GPU, which is specifically built to power AI-generated videos. Nvidia says it’s 120 times more powerful for AI-generated videos than a CPU, and offers 99% better energy efficiency. In addition to generative AI video, Nvidia says the GPU sports video decoding and transcoding capabilities and can be leveraged for augmented reality.

Nvidia says Google Cloud is among the first to integrate the L4. Google plans on offering L4 instances to customers through its Vertex AI platform later today. Nvidia said the GPU will be available from partners later, including Lenovo, Dell, Asus, HP, Gigabyte, and HP, among others.

Jacob Roach
Former Digital Trends Contributor
Jacob Roach is the lead reporter for PC hardware at Digital Trends. In addition to covering the latest PC components, from…
OpenAI plans to make Deep Research free on ChatGPT, in response to competition
OpenAI's new typeface OpenAI Sans

OpenAI has plans to soon make its Deep Research function available for free tier ChatGPT users.

The feature has been available since early February to Plus, Pro, Enterprise, and Edu subscribers; however, the AI company plans to expand availability beyond its paid users. Deep Research goes beyond the standard query results of the brand’s more traditional AI models. The AI agent has the ability to do extended research tasks on command without the help of a human. The feature can provide a detailed report on the subject of your choosing that might take between five and 30 minutes to compile.  

Read more
Viral trend drives ChatGPT to 500 million users
glasses and chatgpt

OpenAI’s flagship service ChatGPT remains as popular as ever, with the brand having hit a 500 million active user milestone in recent days amid the Studio Ghibli viral trend that came with the brand introducing its GPT-4o-powered image generation. 

The company’s CEO, Sam Altman, shared on X on Monday that ChatGPT gained “one million users in the last hour.” He compared the user spike to the burgeoning interest in OpenAI during its early days in 2022, when the chatbot gained one million users in five days, VentureBeat noted.  

Read more
The delay is over — you can now generate images with ChatGPT for free
OpenAI ChatGPT image

After an explosive launch, a viral trend, and some melted GPUs, the new image generation feature for ChatGPT is now available to free users. The feature originally launched on March 25 but because paid subscribers utterly flooded OpenAI with requests for Ghiblified images, CEO Sam Altman announced the next day that the rollout to free users would be delayed "a while."

Luckily, it appears this delay is over just five days later -- Altman has already published another X post saying that "image gen [is] now rolled out to all free users!"

Read more