Nvidia and Apple are collaborating on the Vision Pro in the most unlikely way

By Jacob Roach Published March 18, 2024

Nvidia revealing support for the Apple Vision Pro. — Nvidia

You don’t normally see tech titans like Nvidia and Apple pair up, but the two companies announced at this week’s Nvidia GTC 2024 that they are coming together around the Vision Pro. Nvidia is bringing its Omniverse Cloud platform to Apple’s headset, allowing users to interact with objects and design directly through the Vision Pro.

The basis of support is a set of Omniverse Cloud APIs that can stream Omniverse assets to Apple’s headset. Omniverse isn’t running on the Vision Pro itself. Instead, designers can stream scenes made with the Universal Scene Description (OpenUSD) in Omniverse to the Vision Pro, and interact with the 3D objects natively.

Recommended Videos

Nissian demoed this capability in a demo video. Through the Vision Pro, the user is able to swap out paint colors, adjust the trim, and even go inside the car using spatial awareness thanks to the Vision Pro.

It’s sure to make a splash in the enterprise sector, but there are some consumer implications here. Nvidia is essentially showing that it can stream interactable 3D applications to the Vision Pro. This is enabled by Nvidia’s Graphics Delivery Network (GDN), which is already being used to stream 3D applications from the cloud. The fact that it can work on the Vision Pro is a big deal.

The linchpin for this are the Omniverse Cloud APIs. Also at GTC, Nvidia revealed five new APIs centered around Omniverse Universal Scene Description (OpenUSD) loud that can be used individually or collectively:

USD Render: support for ray-traced renders of OpenUSD data
USD Write: support for modifying OpenUSD data
USD Query: support for interactive scenes
USD Notify: support for tracking USD changes
Omniverse Channel: allows users to connect tools and projects across scenes

Right now, Omniverse Cloud on the Vision Pro is focused around enterprise applications, just as Apple’s headset itself is. This is still a critical foundation for streaming interactable 3D applications to Apple’s headset in the future. Even with how powerful the Vision Pro is, it’s not enough to handle aspects like ray tracing in highly detailed 3D scenes. Being able to stream these scenes at the same quality could line up some exciting apps in the future.

Former Lead Reporter, PC Hardware

Jacob Roach is the lead reporter for PC hardware at Digital Trends. In addition to covering the latest PC components, from…

Topics

Computing

ASUS Zenbook Duo UX8407AA review: Two screens finally earned their place in my bag

Two machines are definitely better than one, but on the same laptop? Asus nailed it, but you must be willing to pay for the convenience.

ASUS Zenbook Duo has two displays

See at Amazon

Two displays on a laptop once sounded like an elaborate solution waiting for the right problem. ASUS has spent the past few generations steadily proving otherwise. After using the latest Zenbook Duo (2026) UX8407AA for over two weeks, I started arranging my daily routine around that second display.

Computing

How Claude helped my 65-year-old dad finally ditch his handwritten ledgers

AI has a lot to answer for, but this one small win is hard to argue with, at least for me.

Claude app on iPhone

My dad has owned a small business for as long as I can remember, and for just as long, he's kept his books the old-fashioned way. Every sale gets written down by hand so he can file his taxes later. The problem is that his accountant needs this data in Excel, and my dad, who didn’t grow up around computers, has never learned how to use it.

For years, his workaround was paying someone to manually type his handwritten entries into a spreadsheet. It worked, but it was adding additional cost to his business, which he wanted to avoid, but couldn't.

Computing

AI’s energy tax was already concerning. Research says AI agents are over hundred times worse

AI agents could consume 136 times more energy than today's AI, study finds

AI agents

The AI industry's soaring electricity demand has already become a growing concern for governments, utilities, and technology companies. But a new study suggests the next generation of artificial intelligence could make that problem significantly worse.

Researchers from the Korea Advanced Institute of Science and Technology (KAIST) have published what they describe as the first comprehensive analysis of the energy cost of AI agents - AI systems capable of reasoning, planning, and completing tasks autonomously. Their findings show that these systems can consume up to 136.5 times as much energy per query as conventional generative AI models, raising fresh questions about whether the infrastructure supporting tomorrow's AI is ready for what's coming.