Skip to main content

Nvidia researchers use artificial intelligence to upgrade your game’s graphics

At the Neural Information Processing Conference in Montreal, Canada, Nvidia researchers demonstrated that they could use the power of artificial intelligence to render synthetic, yet realistic, scenes with details and textures. Researchers claim that the work is still in its early stages, and it is unclear when the technology will be released to consumers, but there is big potential for Nvidia’s artificial intelligence-driven rendering in the gaming space.

“This work is about a new way of rendering computer graphics using neural networks,” Nvidia’s Vice President of Applied Deep Learning Bryan Catanzaro said in a conference call. Basically, researchers wanted to know how they can apply A.I. to make computer graphics better, and the solution is to apply machine learning to real-world videos in order to render new graphics.

Recommended Videos

“We’ve built a system that takes high-level representations of the physical world — basically taking a video sketch and convert that into a rendered scene,” Catanzaro said. “The model understands high-level information of objects in the real world, and then elaborate those to add texture and lighting information. The goal is to be able to synthesize new scenes with graphics.”

Image: Nvidia Image used with permission by copyright holder

Machine learning is used to analyze existing videos and Nvidia would apply computer vision techniques to add labels to objects and their properties. This means that A.I. would be able to recognize an urban cityscape and understand what objects are trees, cars, or building, for example.

This technology is derived from existing research, like that from the University of California, Berkeley, according to Nvidia. The company has shown off other A.I.-based rendering techniques in the past, including one that would remove noise from an image.

Researchers were able to achieve real-time rendering on a Tensor Core GPU, but for the conference, Nvidia demonstrated the technology on its Titan V card. “Though you can do this on any processor, the real-time aspect does require a lot of A.I. throughput,” Catanzaro explained, noting that the Tensor Core GPU is important.

By being able to create content and adding them to virtual worlds, the gaming market could greatly benefit from the work created by this research. Developers, for example, could “remaster” old games by re-rendering old titles to add high-definition visuals, or they could add new levels to existing games with little effort.

As a basic example of how this would work, users can take a video of themselves, upload it to a game, and the rendering will be able to create highly personalized avatars for use in the game. Nvidia has open-sourced the code from its research right now, but Catanzaro cautions that the early work in this field is more suited for computer scientists than game developers.

In a separate demo, Nvidia showcased that it could analyze dance moves — like from the popular music video Gangnam Style — and transfer those moves to another person using the same computer vision techniques. “We analyze the code of the person, and that becomes our sketch,” Catanzaro said. “And the model renders the target person given that sketch.”

Nvidia cautions that this is still an early-stage technology, but you can also upgrade the graphics from an existing game and train a model on real-world imagery to re-render old games to make them look better. Because the technology requires the computer to analyze objects that are known in the real world, Catanzaro cautions that it won’t work on fantastic stuff, like rendering Santa’s elves. In theory, he admits, you can train the computer to render elves, but you need to create physical elves and capture images for the computer to learn.

Like the ray tracing technology that was introduced on Nvidia’s recent RTX series graphics cards for consumers, the company said that this A.I.-based rendering is used in a hybrid way and isn’t meant to replace traditional rendering techniques. Instead, A.I.-based rendering is meant to coexist and be used with traditional graphics rendering engines.

Right now, it’s unclear when this technology will hit the gaming market. It could take as little as a couple years, Catanzaro optimistically speculated. Combined with ray tracing, A.I.-based scene rendering will deliver quality visuals in game titles that are rendered in real time.

Chuong Nguyen
Silicon Valley-based technology reporter and Giants baseball fan who splits his time between Northern California and Southern…
New affordable Nvidia-powered gaming laptops could be on the horizon
Nvidia CEO Jensen Huang announcing Blackwell laptops.

Nvidia's best graphics cards are now available in laptops, but gamers are still waiting for the more budget-focused options. After all, some laptops with the RTX 5090 cost $5,000 and more. Fortunately, new leaks show us that Nvidia is readying up some more mainstream laptops, and those gaming laptops include GPUs that are yet to be announced.

As spotted by VideoCardz, it seems that several Nvidia partners are getting ready to launch new laptops, this time equipped with the RTX 5060 and the RTX 5050. The fact that this isn't the first time we're hearing about these GPUs definitely lends credibility to the theory, but don't get your hopes up too high just yet -- we have to wait for Nvidia to reveal them.

Read more
The success of WWDC 2025 hangs on Apple Intelligence. This is what it needs to
Apple's Craig Federighi discussing Apple Intelligence at the Worldwide Developers Conference (WWDC) 2024.

It’s that time of year when Apple announces the dates for its Worldwide Developers Conference (WWDC) and sends the tech world into a frenzy of speculation over what goodies it has lined up for its expectant fans. With the week-long event kicking off on June 9, there’s not long to go now.

Usually, the announcement of WWDC triggers a wave of excitement for Apple’s upcoming operating systems, such as iOS 19 and macOS 16, and perhaps some hot new hardware that Apple has been keeping under wraps.

Read more
Nvidia RTX 5070 available at suggested price at Best Buy, grab it now
The RTX 5070 in a graphic.

It may be the day after April Fool's Day, but this is no joke. The coveted Nvidia RTX 5070 is currently available at the major retailer, Best Buy, at its regularly suggested price of $550. 

There have already been reports of increased availability of the RTX 50-series graphics cards, such as news of exclusive offerings for Amazon Prime members. However, this sighting is a lucky opportunity for those who don’t want to spend in excess for a graphic component. All of Nvidia’s GPUs have been subject to reseller and scalper inflation, but its cheapest option becoming too costly can be exhausting. Retailers attempted to push the price of the mid-range RTX 5070 Ti up to $1,000 in March before complaints forced them to bring the price down to a more reasonable level. 

Read more