Skip to main content

Digital Trends may earn a commission when you buy through links on our site. Why trust us?

Microsoft explains how thousands of Nvidia GPUs built ChatGPT

ChatGPT rose to viral fame over the past six months, but it didn’t come out of nowhere. According to a blog post published by Microsoft on Monday, OpenAI, the company behind ChatGPT, reached out to Microsoft to build AI infrastructure on thousands of Nvidia GPUs more than five years ago.

OpenAI and Microsoft’s partnership has caught a lot of limelight recently, especially after Microsoft made a $10 billion investment in the research group that’s behind tools like ChatGPT and DALL-E 2. However, the partnership started long ago, according to Microsoft. Since then, Bloomberg reports that Microsoft has spent “several hundred million dollars” in developing the infrastructure to support ChatGPT and projects like Bing Chat.

Related Videos
Hopper H100 graphics card.

Much of that money went to Nvidia, which is now in the forefront of computing hardware required to train AI models. Instead of gaming GPUs like you’d find on a list of the best graphics cards, Microsoft went after Nvidia’s enterprise-grade GPUs like the A100 and H100.

It’s not just as simple as getting graphics cards together and training a language model, though. As Nidhi Chappell, Microsoft head of product for Azure, explains: “This is not something that you just buy a whole bunch of GPUs, hook them together, and they’ll start working together. There is a lot of system-level optimization to get the best performance, and that comes with a lot of experience over many generations.”

With the infrastructure in place, Microsoft is now opening up its hardware to others. The company announced on Monday in a separate blog post that it would offer Nvidia H100 systems “on-demand in sizes ranging from eight to thousands of Nvidia H100 GPUs,” delivered through Microsoft’s Azure network.

The popularity of ChatGPT has skyrocketed Nvidia, which has invested in AI through hardware and software for several years. AMD, Nvidia’s main competitor in gaming graphics cards, has been attempting to make headway into the space with accelerators like the Instinct MI300.

According to Greg Brockman, president and co-founder of OpenAI, training ChatGPT wouldn’t have been possible without the horsepower provided by Microsoft: “Co-designing supercomputers with Azure has been crucial for scaling our demanding AI training needs, making our research and alignment work on systems like ChatGPT possible.”

Nvidia is expected to reveal more about future AI products during the GPU Technology Conference (GTC). with the keynote presentation kicks things off on March 21. Microsoft is expanding its AI road map later this week, with a presentation focused around the future of AI in the workplace scheduled for March 16.

Editors' Recommendations

Here’s the ChatGPT word limit and how to get around it
A laptop opened to the ChatGPT website.

Fans of ChatGPT adore the AI chatbot for several purposes, including its ability to generate detailed essays in a matter of seconds. However, one of its little-known limitations is that there is a word and character limit set on how much content it can output per query.
Reddit members and other AI enthusiasts have been discussing this for months, and luckily there are easy workarounds for this limitation by way of the prompts you can use.

What is the ChatGPT word limit?
ChatGPT's parent company, OpenAI. set the word and character limit as part of its ongoing development of the AI chatbot, which is still in its research preview phase. Some of the issues with ChatGPT include its affinity toward "social biases, hallucinations, and adversarial prompts," in addition to producing inaccurate content when the AI algorithm is overwhelmed or at a loss for information to process.
Similarly, ChatGPT might simply stop producing content when the request is too complex for the AI to handle. This happens at about 500 words or 4,000 characters. If you happen to give the chatbot a request for a specific number of words above 500, you might find that it cuts off midsentence somewhere after 500 words.

Read more
ChatGPT: how to use the viral AI chatbot that everyone’s talking about
ChatGPT and OpenAI logos.

ChatGPT has continued to dazzle the internet with AI-generated content, morphing from a novel chatbot into a piece of technology that is driving the next era of technological innovation. Not everyone's on board yet, though, and you're probably wondering; what's all the fuss about?

It's made by OpenAI, well-known for having developed the text-to-image generator DALL-E, and it's currently available for anyone to try out for free. Here's what ChatGPT is, how to use it, and how it could change the future of the internet.

Read more
GPT-4: how to use, new features, availability, and more
A laptop opened to the ChatGPT website.

ChatGPT-4 has officially been announced, confirming the longtime rumors around its improvements to the already incredibly impressive language skills of OpenAI's ChatGPT.

OpenAI calls it the company's "most advanced system, producing safer and more useful responses." Here's everything we know about it so far.
Availability

Read more