Skip to main content

Nvidia’s smart paintbrush tool turns simple doodles into works of art

Nvidia GauGAN Researchers Photo
NVIDIA

A new drawing app enhanced with deep learning functions could soon convert your simple sketches and doodles to brilliant and realistic works of art.

Nvidia recently announced the development of GauGAN, a smart drawing app that uses a deep learning model to generate photorealistic versions of simple drawings created by human artists. The name of the app, GauGAN, refers both to the French post-Impressionist painter Paul Gauguin and the app’s use of the generative adversarial networks (GANs) to develop its realistic images.

Recommended Videos

So how does GauGAN work? According to Nvidia’s announcement, the app is like a “smart paintbrush” that fills in the details of your drawing (referred to as a “segmentation map”). Essentially, you provide the outline of what you want and label each segment, dictating what it’s supposed to be. From there, GauGAN takes over, filling in the extra details to make your drawings more realistic:

“Trained on a million images, the deep learning model then fills in the landscape with showstopping results: Draw in a pond, and nearby elements like trees and rocks will appear as reflections in the water. Swap a segment label from “grass” to “snow” and the entire image changes to a winter scene, with a formerly leafy tree turning barren.”

Changing Sketches into Photorealistic Masterpieces

The GANs used in GauGAN are essentially a partnership between two networks, a generator and a discriminator. Within this partnership, a generator develops a series of possible images and the discriminator acts as a sort of editor that teaches the generator “how to improve the realism of its synthetic images.”

It’s also worth noting that while GauGAN uses its extensive knowledge of other images to create these drawings, these drawings are still considered original works since the app is generating new images.

Nvidia’s new drawing app isn’t just limited to nature scenes or landscapes—it can also add in buildings, roads, or even people. GauGAN also allows artists to add style filters. Such filters including mimicking a specific artist’s style or adjusting the lighting of the image from a day to a night scene.

It’s unclear if GauGAN will ever be released for consumer use, but it is expected to be a particularly helpful tool for architects, urban planners, and even game developers.

Anita George
Anita George has been writing for Digital Trends' Computing section since 2018. So for almost six years, Anita has written…
Nvidia is replacing its crusty, Windows XP-era app with something much better
A screenshot of the Nvidia app.

Nvidia's latest announcement is one that many of us have been waiting for. The company is releasing a new software solution that will, hopefully, one day replace the Nvidia Control Panel and GeForce Experience. Dubbed simply "Nvidia App," the software is now in beta and is available for testing.

Those of us who own some of Nvidia's top GPUs (or even some of the worst ones) are forced to use the Nvidia Control Panel for tuning things like display settings, game performance, and monitor refresh rates. Meanwhile, GeForce Experience optimizes game settings and provides an in-game overlay that can be used for things like recording bits of your gameplay. The lack of a unified app can get confusing and annoying to those who aren't familiar with it.

Read more
Nvidia’s peace offering isn’t working
Two MSI RTX 4060 Ti 16GB GPUs over a black background.

Nvidia's RTX 4060 Ti 16GB is here, but you wouldn't know it if you didn't follow GPU news closely. It seems that the GPU might just be so far behind some of the best graphics cards that Nvidia isn't advertising it too much. As a result, early benchmarks are scarce.

MSI has released some benchmarks of its own, comparing the 8GB and the 16GB versions of the RTX 4060 Ti. It turns out that the new GPU might actually be slower. Is this why Nvidia didn't even make its own version of this card?

Read more
Nvidia ditches the RTX 4070 as it turns to cheaper options
The RTX 4070 graphics card on a pink background.

Just as the RTX 4060 family of GPUs is being launched, a new report says Nvidia may be freezing production of its bigger brother, the RTX 4070. Sales of the RTX 4070 are reportedly so bad that the company has halted production in a desperate bid to turn things around. It’s just the latest bad news for a product that has been beset with problems almost from the day it launched.

As reported by Wccftech, the production stoppage is set to last until June, by which time Nvidia will be hoping sales have picked up. It’s apparently an extension of an earlier production slowdown, which reportedly began shortly after the RTX 4070 launched in April.

Read more