Nvidia just unveiled its new generative AI model, dubbed Latte3D, during GTC 2024. Latte3D appears to be ChatGPT on extreme steroids. I’s a text-to-3D model that accepts simple, short text prompts and turns them into 3D objects and animals within a second. Much faster than its older counterparts, Latte3D works like a virtual 3D printe that could come in handy for creators across many industries.
Latte3D was made to simplify the creation of 3D models for many types of creators, such as those working on video games, design projects, marketing, or even machine learning and training for robotics. In Nvidia’s demo of the model, it appears super simple to use. Following a quick text prompt, the AI generates a 3D model and shortly after finishes it off with much more detail. While the end result is nowhere near as lifelike as OpenAI’s Sora, it’s not meant to be — this is a way to speed up creating assets instead of having to build them from the ground up.
The model generates several different options for the user to choose from, and Nvidia says that these shapes can be “optimized for higher quality within a few minutes.” The designs can then be exported to different platforms, such as Nvidia’s Omniverse, and can be tweaked to match the desired end result. Nvidia trained Latte3D by using its Ada A100 Tensor Core GPUs and supported the training with ChatGPT prompts to ready it for interacting with real users.
As of right now, Latte3D can only generate objects and animals. To that end, it appears to do a solid job of discerning different animals, textures, and object types. Nvidia showed off these capabilities by presenting objects such as an amigurumi (crochet) common crane or an origami sphynx cat. The model was taught to recognize various species and thus can tell the difference between an Italian greyhound and a Shiba Inu.
Creators who want to use Latte3D to do more can train it on a different dataset, be it plants or household objects, and later use it for their own purposes. Nvidia brings up some interesting use cases here, such as training personal assistant robots before deploying them. It’s easy to imagine that Latte3D will come in handy for game devs, but the potential goes far beyond just gaming scenarios.
Sanja Fidler, vice president of AI research at Nvidia, remarked on how much faster Latte3D is compared to its predecessors: “A year ago, it took an hour for AI models to generate 3D visuals of this quality — and the current state of the art is now around 10 to 12 seconds. We can now produce results an order of magnitude faster,” said Fidler.
The recent announcements related to using AI in game development are all pretty groundbreaking, and Nvidia’s Latte3D joins a growing list of tools that may one day completely change the process of creating a game. For instance, Nvidia just recently unveiled non-player characters (NPCs) with dialogue entirely generated by AI. Meanwhile, Unreal Engine’s latest update can generate film-quality visuals in games in real time, all with the help of machine learning.
