Nvidia researchers use artificial intelligence to upgrade your game’s graphics

At the Neural Information Processing Conference in Montreal, Canada, Nvidia researchers demonstrated that they could use the power of artificial intelligence to render synthetic, yet realistic, scenes with details and textures. Researchers claim that the work is still in its early stages, and it is unclear when the technology will be released to consumers, but there is big potential for Nvidia’s artificial intelligence-driven rendering in the gaming space.

“This work is about a new way of rendering computer graphics using neural networks,” Nvidia’s Vice President of Applied Deep Learning Bryan Catanzaro said in a conference call. Basically, researchers wanted to know how they can apply A.I. to make computer graphics better, and the solution is to apply machine learning to real-world videos in order to render new graphics.

“We’ve built a system that takes high-level representations of the physical world — basically taking a video sketch and convert that into a rendered scene,” Catanzaro said. “The model understands high-level information of objects in the real world, and then elaborate those to add texture and lighting information. The goal is to be able to synthesize new scenes with graphics.”

nvidia researchers use artificial intelligence to upgrade your games graphics ai scene rendering
Image: Nvidia

Machine learning is used to analyze existing videos and Nvidia would apply computer vision techniques to add labels to objects and their properties. This means that A.I. would be able to recognize an urban cityscape and understand what objects are trees, cars, or building, for example.

This technology is derived from existing research, like that from the University of California, Berkeley, according to Nvidia. The company has shown off other A.I.-based rendering techniques in the past, including one that would remove noise from an image.

Researchers were able to achieve real-time rendering on a Tensor Core GPU, but for the conference, Nvidia demonstrated the technology on its Titan V card. “Though you can do this on any processor, the real-time aspect does require a lot of A.I. throughput,” Catanzaro explained, noting that the Tensor Core GPU is important.

By being able to create content and adding them to virtual worlds, the gaming market could greatly benefit from the work created by this research. Developers, for example, could “remaster” old games by re-rendering old titles to add high-definition visuals, or they could add new levels to existing games with little effort.

As a basic example of how this would work, users can take a video of themselves, upload it to a game, and the rendering will be able to create highly personalized avatars for use in the game. Nvidia has open-sourced the code from its research right now, but Catanzaro cautions that the early work in this field is more suited for computer scientists than game developers.

In a separate demo, Nvidia showcased that it could analyze dance moves — like from the popular music video Gangnam Style — and transfer those moves to another person using the same computer vision techniques. “We analyze the code of the person, and that becomes our sketch,” Catanzaro said. “And the model renders the target person given that sketch.”

Nvidia cautions that this is still an early-stage technology, but you can also upgrade the graphics from an existing game and train a model on real-world imagery to re-render old games to make them look better. Because the technology requires the computer to analyze objects that are known in the real world, Catanzaro cautions that it won’t work on fantastic stuff, like rendering Santa’s elves. In theory, he admits, you can train the computer to render elves, but you need to create physical elves and capture images for the computer to learn.

Like the ray tracing technology that was introduced on Nvidia’s recent RTX series graphics cards for consumers, the company said that this A.I.-based rendering is used in a hybrid way and isn’t meant to replace traditional rendering techniques. Instead, A.I.-based rendering is meant to coexist and be used with traditional graphics rendering engines.

Right now, it’s unclear when this technology will hit the gaming market. It could take as little as a couple years, Catanzaro optimistically speculated. Combined with ray tracing, A.I.-based scene rendering will deliver quality visuals in game titles that are rendered in real time.

Emerging Tech

Rise of the Machines: Here’s how much robots and A.I. progressed in 2018

2018 has generated no shortage of news, and the worlds of A.I. and robotics are no exception. Here are our picks for the most exciting, game changing examples of both we saw this year.
Computing

Nvidia’s Jetson AGX Xavier module is designed to give robots better brains

Nvidia's pricey Jetson AGX Xavier might help drive the next generation of smart robots. Nvidia hopes that developers will use its new Xavier module to power AI-driven machines like delivery drones and robots used in manufacturing.
Product Review

The Black Shark gaming phone takes a big bite out of your free time, but the software sinks it

The world is being treated to an ever-increasing number of high-powered gaming phones. With so many great options already out, is there room for another? The Black Shark thinks so. But is it any good? We find out.
Computing

3DMark’s Port Royal lets you benchmark ray tracing on Nvidia’s RTX cards

UL is adding another benchmarking utility to its popular 3DMark suite to help gamers measure their graphics card's ray tracing performance. You'll soon be able to measure how Nvidia's RTX 2070, 2080, and 2080 Ti stack up.
Home Theater

Confused about LED vs. LCD TVs? Here's everything you need to know

Our LED vs. LCD TV buying guide explains why these two common types of displays are fundamentally connected, how they differ, what to look for in buying an LED TV, and what's on the horizon for TVs.
Computing

Canada’s winters inspired a startup to warm homes with cryptomining heat waste

Cryptomining may be the key to untold riches and the future of currency, but it’s also an environmental nightmare. Heatmine, thinks it has the answer, but it could mean bolting a mining rig onto every home and business in the country.
Product Review

The Asus ZenBook 14 is a tiny notebook that gets lost in the crowd

The ZenBook 14 aims to be the smallest 14-inch notebook around, and it succeeds thanks to some tiny bezels. Performance and battery life are good, but the notebook lacks a standout feature other than size.
Deals

The best MacBook deals for December 2018

If you’re in the market for a new Apple laptop, let us make your work a little easier: We hunted down the best up-to-date MacBook deals available online right now from various retailers.
Computing

How to connect AirPods to your MacBook

If you have new AirPods, you may be looking forward to pairing them with your MacBook. Our guide will show you exactly how to connect AirPods to MacBook, what to do if they are already paired with a device, and more.
Computing

Hitting ‘Check for updates’ in Windows 10 opts you into beta releases

Users who are careful about keeping their system updated should watch out -- Microsoft revealed this week that clicking the Check for updates button in Windows can opt you in to testing beta code.
Computing

Secure your Excel documents with a password by following these quick steps

Excel documents are used by people and businesses all over the world. Given how often they contain sensitive information, it makes sense to keep them from the wrong eyes. Thankfully, it's easy to secure them with a password.
Computing

Which Macs are compatible with MacOS Mojave?

Is your computer ready for Apple's big Mojave update? Here's what you need to know about MacOS Mojave compatibility, what Macs can successful download Mojave, and the requirements you need to know about.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Booze-filled ski poles and crypto piggy banks

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Computing

Change your mouse cursor in Windows with these quick tips

The standard mouse cursor is boring, so change it! With this guide on how to change your mouse cursor in Windows, you can choose to use one of Microsoft's pre-installed cursors or download something a bit more extravagant.