Skip to main content

Nvidia turns simple text prompts into game-ready 3D models

A colorful collage of images generated by Nvidia's LATTE3D.
Nvidia

Nvidia just unveiled its new generative AI model, dubbed Latte3D, during GTC 2024. Latte3D appears to be ChatGPT on extreme steroids. I’s a text-to-3D model that accepts simple, short text prompts and turns them into 3D objects and animals within a second. Much faster than its older counterparts, Latte3D works like a virtual 3D printe that could come in handy for creators across many industries.

Latte3D was made to simplify the creation of 3D models for many types of creators, such as those working on video games, design projects, marketing, or even machine learning and training for robotics. In Nvidia’s demo of the model, it appears super simple to use. Following a quick text prompt, the AI generates a 3D model and shortly after finishes it off with much more detail. While the end result is nowhere near as lifelike as OpenAI’s Sora, it’s not meant to be — this is a way to speed up creating assets instead of having to build them from the ground up.

The model generates several different options for the user to choose from, and Nvidia says that these shapes can be “optimized for higher quality within a few minutes.” The designs can then be exported to different platforms, such as Nvidia’s Omniverse, and can be tweaked to match the desired end result. Nvidia trained Latte3D by using its Ada A100 Tensor Core GPUs and supported the training with ChatGPT prompts to ready it for interacting with real users.

Get your weekly teardown of the tech behind PC gaming
Check your inbox!

As of right now, Latte3D can only generate objects and animals. To that end, it appears to do a solid job of discerning different animals, textures, and object types. Nvidia showed off these capabilities by presenting objects such as an amigurumi (crochet) common crane or an origami sphynx cat. The model was taught to recognize various species and thus can tell the difference between an Italian greyhound and a Shiba Inu.

LATTE3D Text to 3D Generative AI Model from NVIDIA Research

Creators who want to use Latte3D to do more can train it on a different dataset, be it plants or household objects, and later use it for their own purposes. Nvidia brings up some interesting use cases here, such as training personal assistant robots before deploying them. It’s easy to imagine that Latte3D will come in handy for game devs, but the potential goes far beyond just gaming scenarios.

Sanja Fidler, vice president of AI research at Nvidia, remarked on how much faster Latte3D is compared to its predecessors: “A year ago, it took an hour for AI models to generate 3D visuals of this quality — and the current state of the art is now around 10 to 12 seconds. We can now produce results an order of magnitude faster,” said Fidler.

The recent announcements related to using AI in game development are all pretty groundbreaking, and Nvidia’s Latte3D joins a growing list of tools that may one day completely change the process of creating a game. For instance, Nvidia just recently unveiled non-player characters (NPCs) with dialogue entirely generated by AI. Meanwhile, Unreal Engine’s latest update can generate film-quality visuals in games in real time, all with the help of machine learning.

Editors' Recommendations

Monica J. White
Monica is a UK-based freelance writer and self-proclaimed geek. A firm believer in the "PC building is just like expensive…
Google’s ChatGPT rival just launched in search. Here’s how to try it
Generative AI in Google Search.

Ever since Microsoft started integrating ChatGPT into Bing search, alarm bells have been ringing at Google. Now, though, the tech giant has started rolling out its own generative artificial intelligence (AI) tool for users as part of its bid to retain its search crown.

In a blog post, the company explains that the new feature (called Search Generative Experience, or SGE) is part of Google’s Search Labs, which lets you test out experimental ideas in Google search and provide feedback to the company. Google says its generative AI will “help you take some of the work out of searching, so you can understand a topic faster, uncover new viewpoints and insights and get things done more easily.”

Read more
Nvidia GPUs see massive price hike and huge demand from AI
Nvidia RTX 2060 Super and RTX 2070 Super review

It feels like we’ve only just emerged from the debilitating graphics card shortage of the last few years, but a new report suggests we can’t breathe easy just yet. Could a new GPU shortage be on the horizon, or are consumers safe from a return to another nightmare scenario?

According to DigiTimes (via Wccftech), Nvidia is seeing a huge surge in demand for its chips due to the explosion in artificial intelligence (AI) tools like ChatGPT. Nvidia offers a range of graphics cards that excel at AI tasks, including the A100 and H100, and the company is reportedly struggling to keep up in the wake of such massive demand.

Read more
OpenAI’s new Shap-E tool is Dall-E for 3D objects
Purple and pink-diamond on blue background by Rostislav Uzunov.

OpenAI's latest endeavor, Shap-E, is a model that allows you to generate 3D objects from text, not unlike how Dall-E can create 2D images.

According to OpenAI, Shap-E is "a conditional generative model for 3D assets. Unlike recent work on 3D generative models which produce a single output representation, Shap-E directly generates the parameters of implicit functions that can be rendered as both textured meshes and neural radiance fields."

Read more