NVIDIA’s LATTE3D Brings Generative AI to a New Dimension

Normally, AI image generation is based on pixel distributions. While the AI cannot see a dog, it knows how pixels are generally distributed when creating an image of a dog. When it comes to 3D models though, it’s more complex. A vertex in a 3D space naturally carries an extra dimension than a pixel. Now apply that to every part of a 3D model and you get a massive increase in data. The team at NVIDIA Research took this challenge head-on with their new LATTE3D which can generate 3D models within seconds.

Text-to-3D

While alternatives exist, the power of LATTE3D is in speed and format. Models generated can be easily integrated into other software. It’s also not that this isn’t necessarily taking people’s jobs. This opens up to use cases that were simply impossible for the traditional modeling process. As the press release suggests, the model could be optimized to, for example, generate decor and furniture to test them for rooms in real time. This was initially impossible in real-time as one would need to stop and model specific furniture they need each time. Or for a landscape designer to quickly add needed trees and vegetation. This same logic could apply to dozens of other spheres. 3D Modeling can be very time-consuming, and this helps many projects where it could potentially provide alternatives.

Or, we could use LATTE3D to initialize text-to-4D methods like Align Your Gaussians: https://t.co/tFmJYsIJjW (@HuanLing6, @seungkim0123, @karsten_kreis).

Here, we initialize with “DSLR photo of Rottweiler” and animate with “A dog running fast.” pic.twitter.com/2W9XfblWYt

— Jonathan Lorraine (@jonLorraine9) March 21, 2024

Availability

While not widely released yet, the model is already functional and trained to make animals and everyday objects. It can generate models almost instantly on tested architecture (such as the NVIDIA RTX A6000). The LATTE3D also supports an optional test-time optimization that lets higher-quality models be generated, though it requires more time to create. Aside from that, the press release also suggests the model can be used in combination with other models to get realistic animations. Overall, it aims to deliver on its promise to bring AI to a new dimension with the hope that it might open the gates to a new wave of creativity. Just as the 2D generation brought so many alternative uses born off people’s creativity, I expect models like LATTE3D to eventually expand drastically.

YouTube: LATTE3D Text-to-3D Generative AI Model from NVIDIA Research

Photo credit: The feature image is symbolic and shows NVIDIA CEO Jensen Huang unveiling the Blackwell processor.