Skip to main content

Apple’s secret VR headset just leaked an ingenious idea

Apple’s Reality Pro mixed-reality headset is probably just a few months from launching, but we’re still seeing the company’s top-secret ideas seeping out into the wild. The latest leak shows one way you might be able to control things in Apple’s metaverse — and it’s a pretty unusual concept.

According to a recently granted patent (number 2023/0042447 A1), Apple is exploring the idea of using an Apple Pencil as a sort of virtual reality (VR) controller. The idea is that your hand holding the Apple Pencil could be displayed in the mixed-reality world that you see through the headset, overlaying it onto augmented reality (AR) elements.

Related Videos
An image from an Apple patent showing a person's hand holding a tablet device. On the screen is the user's hand holding an Apple Pencil and drawing a line onto an augmented reality object.

Apple says an image of your hand holding an Apple Pencil could be captured by sensors or cameras, for instance, those mounted into a mixed-reality headset. The patent then explains how that would let you generate “pencil-like marks” in the virtual world. One example could be a virtual meeting room where you are able to draw on a whiteboard and see the Apple Pencil making the marks in real time.

Similarly, you could flip the Apple Pencil over and use its blunt end as a kind of virtual eraser, removing the pixel markings you had just made in the mixed-reality space.

Hand-tracking and gesture control

The 2nd gen Apple Pencil laying on a table.
Joe Maring/Digital Trends

It’s a pretty neat idea and goes some way to revealing what Apple could let you do with your hands while using its Reality Pro headset. So far, we’ve seen a few of Apple’s ideas on this topic, including others than include existing Apple products.

For example, a previously granted patent showed how a pair of Apple Watches could be used to record hand gestures in the metaverse. Elsewhere, another one of Apple’s concepts depicted small, thimble-like devices on the tips of a user’s fingers, which would register hand gestures and transpose them into mixed reality.

The benefit of the Apple Pencil idea is that for many people it might not require any additional purchase, as plenty of Apple fans already have an Apple Pencil. That would make it more accessible than a system that requires you to purchase a set of VR thimbles in addition to the already pricey headset itself.

That’s not to say this is definitely the route Apple will take. Since the information comes from a patent, it might never see the light of day, and it could simply be an idea that Apple is playing around with but will never actually put into practice.

Still, it seems like a clever way to take an existing Apple device and use it to bolster the mixed-reality world Apple’s Reality Pro headset will create. Either way, we should find out soon, as Apple is rumored to be launching the Reality Pro headset at a special spring event. Keep your eyes peeled.

Editors' Recommendations

Apple’s security trumps Microsoft and Twitter’s, say feds
Apple's Craig Federighi speaking about macOS security at WWDC 2022.

Apple has long held a reputation for rock-solid security, and now the U.S. government seemingly agrees after praising the company for its security procedures. At the same time, the feds have suggested Microsoft and Twitter need to pull their socks up and make their products much more secure for their users, according to CNBC.

In a speech given at Carnegie Mellon University, Cybersecurity and Infrastructure Security Agency Director Jen Easterly pointed to Apple as a company that took security and accountability seriously, and suggested other companies should take note.

Read more
Apple’s second-generation VR headset is already in the works
A rendering of an Apple mixed-reality headset (Reality Pro) in a gold color seen from the front.

Apple’s first mixed-reality headset hasn’t even been released, yet the company is already working on a second-generation model that will bring even more power and performance to the table, according to a new report. That suggests we might not have long to wait before the successor device hits the shelves.

The news comes from Bloomberg reporter Mark Gurman in his Power On newsletter. Gurman claims that the first edition of the headset -- likely dubbed Reality Pro -- will come with the same M2 chip that powers the latest MacBook Air, plus a separate chip that will handle augmented reality (AR) and virtual reality (VR) experiences.

Read more
This major Apple bug could let hackers steal your photos and wipe your device
A physical lock placed on a keyboard to represent a locked keyboard.

Apple’s macOS and iOS are often considered to be more secure than their rivals, but that doesn’t make them invulnerable. One security team recently proved that by showing how hackers could exploit Apple’s systems to access your messages, location data, and photos -- and even wipe your device entirely.

The discoveries were published on the blog of security research firm Trellix, and will be of major concern to iOS and macOS users alike, since the vulnerabilities can be exploited on both operating systems. Trellix explains that Apple patched the exploits in macOS 13.2 and iOS 16.3, which were released in January 2023, so you should update your devices as soon as you can.

Read more