Skip to main content

Apple’s secret VR headset just leaked an ingenious idea

Apple’s Reality Pro mixed-reality headset is probably just a few months from launching, but we’re still seeing the company’s top-secret ideas seeping out into the wild. The latest leak shows one way you might be able to control things in Apple’s metaverse — and it’s a pretty unusual concept.

According to a recently granted patent (number 2023/0042447 A1), Apple is exploring the idea of using an Apple Pencil as a sort of virtual reality (VR) controller. The idea is that your hand holding the Apple Pencil could be displayed in the mixed-reality world that you see through the headset, overlaying it onto augmented reality (AR) elements.

An image from an Apple patent showing a person's hand holding a tablet device. On the screen is the user's hand holding an Apple Pencil and drawing a line onto an augmented reality object.

Apple says an image of your hand holding an Apple Pencil could be captured by sensors or cameras, for instance, those mounted into a mixed-reality headset. The patent then explains how that would let you generate “pencil-like marks” in the virtual world. One example could be a virtual meeting room where you are able to draw on a whiteboard and see the Apple Pencil making the marks in real time.

Similarly, you could flip the Apple Pencil over and use its blunt end as a kind of virtual eraser, removing the pixel markings you had just made in the mixed-reality space.

Hand-tracking and gesture control

The 2nd gen Apple Pencil laying on a table.
Joe Maring / Digital Trends

It’s a pretty neat idea and goes some way to revealing what Apple could let you do with your hands while using its Reality Pro headset. So far, we’ve seen a few of Apple’s ideas on this topic, including others than include existing Apple products.

For example, a previously granted patent showed how a pair of Apple Watches could be used to record hand gestures in the metaverse. Elsewhere, another one of Apple’s concepts depicted small, thimble-like devices on the tips of a user’s fingers, which would register hand gestures and transpose them into mixed reality.

The benefit of the Apple Pencil idea is that for many people it might not require any additional purchase, as plenty of Apple fans already have an Apple Pencil. That would make it more accessible than a system that requires you to purchase a set of VR thimbles in addition to the already pricey headset itself.

That’s not to say this is definitely the route Apple will take. Since the information comes from a patent, it might never see the light of day, and it could simply be an idea that Apple is playing around with but will never actually put into practice.

Still, it seems like a clever way to take an existing Apple device and use it to bolster the mixed-reality world Apple’s Reality Pro headset will create. Either way, we should find out soon, as Apple is rumored to be launching the Reality Pro headset at a special spring event. Keep your eyes peeled.

Editors' Recommendations

Alex Blake
In ancient times, people like Alex would have been shunned for their nerdy ways and strange opinions on cheese. Today, he…
Apple’s Vision Pro could get this incredible gaming upgrade
A person plays a game on the Apple Vision Pro.

Apple has touted its Vision Pro headset as a premier gaming device that works with many different controllers. But with extensive gesture control seemingly used everywhere on the product except for gaming, it feels like a missed opportunity.

Well, that could change in the future, as a newly published Apple patent (number 11709554) shows how future editions of the Vision Pro could transform your hands into gaming controllers, complete with fingertip controls and haptic feedback.

Read more
Apple’s next Vision Pro may send you on a mood-altering trip
The Apple Vision Pro reveals the wearer's eyes on a front-facing display.

It sounds like something out of a sci-fi novel, but Apple could be working on a way to let you alter your mood just by telling its Vision Pro headset how you want to feel. That’s according to a recently granted patent (number 11703944) that outlines the whole futuristic idea.

It’s a strange concept and one that sounds woven right out of a Philip K. Dick story. In fact, in the author’s Do Androids Dream of Electric Sheep? there is a machine that puts users into whatever mood they desire. It sounds as though Apple’s inventors have been reading a little sci-fi in their spare time.

Read more
Apple’s secret AR glasses may have this genius feature for glasses users
Apple iGlasses

Apple’s Vision Pro headset has got the entire tech world talking, but it’s still unclear exactly how it will work with the prescription lenses used by glasses wearers around the world. Get it right and it could be a beautifully seamless experience -- get it wrong and Apple risks alienating potentially millions of users. It’s a crucial obstacle to overcome.

Apple has said it will let you add special prescription lenses to the Vision Pro, but pricing is uncertain, and they could cost as much as $600 a pop according to journalist Mark Gurman. Now, though, a fresh patent for Apple’s upcoming augmented reality (AR) glasses shows that there could be a much better -- and cheaper -- way to do it.

Read more