Spectator View provides a looking glass for all to see HoloLens holograms

microsoft hololens spectator view dslr
As we saw 30 minutes into Microsoft’s recent Windows 10 Creators Update press event, the company can display what a HoloLens user sees in real time to a general audience. For instance, Microsoft “experience designer” Taj Reid pulled a holographic chair out of a floating web browser and positioned it on the stage. While he experienced all of this in first-person, the audience could see the same objects in a third-person view though the event’s large displays or via the online broadcast. Now Microsoft wants to bring this “spectator view” to all HoloLens developers.

During October’s press event, Microsoft brought out a special camera that captured Taj Reid’s demo in real time while simultaneously rendering the holograms into the live feed. Microsoft wants developers to create a similar camera by offering documentation based on its camera design, aka the Spectator View Camera. The use cases for this setup include snapping images of HoloLens wearers as they interact with holograms, live demonstrations, and video capture.

“Since a HoloLens can only be worn by one person, how do you open the view so everyone can see what that person sees? To solve this problem, we developed Mixed Reality Capture (MRC) to aid in the visualization of holograms that the wearer is seeing from a first-person perspective,” said Microsoft’s Brandon Bray in a blog on Monday.

For photo and video capture, Microsoft tested its Mixed Reality Capture technology on the Canon EOS 5D Mark III camera ($2,500) while it tested the Blackmagic Design Production Camera 4K for live demonstrations ($3,000). However, any camera with HDMI output should work, as should any add-on camera lens. Microsoft uses the Canon EF 14mm f/2.8L II USM Ultra-Wide Angle Fixed Lens ($2,100).

The camera setup consists of a tripod, a DSLR camera, a hot shoe fastener to mount a hot shoe adapter, a 3D printed adapter to connect the HoloLens mount to the camera’s hot shoe, and a machined aluminum bracket holding a HoloLens unit. The whole setup can connect to a capture card installed in a PC to preview the composite scene. Microsoft has already tested the setup using the Blackmagic Design Intensity Pro 4K capture card ($190).

On the software side, here is what developers will need:

Instructions on how to build the camera can be found here. Microsoft also provides an online Holograms 240 class showing developers how to create a shared experience through HoloLens and Windows 10 apps via Microsoft’s Universal Windows Platform. Of course, shared experiences require at least two HoloLens headsets, which cost $3,000 each.

The ultimate purpose of Spectator View is to provide a stable, third-person composite view of the HoloLens user and the holograms. Spectators can follow along with the experience without suffering through a nausea-causing “shaky cam” viewpoint created by head-mounted cameras.

Mobile

From the road to your wrist, see how Android has evolved over the past 10 years

Android started out as just a mobile operating system, but 10 years in it's pretty much everywhere. Check out our round-up of all the different Android variations that have cropped up so far, and what might be coming in the future.
Mobile

The Galaxy S10 may be announced before MWC, sell for up to $1,750

While we still may be months away from an announcement, there's no doubt about it: Samsung is working hard on its successor to the Galaxy S9. Here's everything we know about the upcoming Samsung Galaxy S10.
Photography

These are the best action cameras money can buy

Action cameras are great tools for capturing videos of your everyday activities, whether it's a birthday party or the steepest slope you've ever descended on your snowboard. These are the best money can buy.
Home Theater

Looking for the best 4K Ultra HD TVs you can buy? Here are five great options

If it's time to upgrade your old 1080p to a new 4K model but you don't know what to look for, fear not, as we're here with a list of the best 4K Ultra HD TVs to help make your buying process as easy as possible.
Movies & TV

The best shows on Netflix, from 'Haunting of Hill House’ to ‘Twilight Zone’

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Computing

Following the Portal, augmented reality glasses may be Facebook’s next step

Following the launch of its Portal smart display, Facebook says it is working on AR glasses, possibly in a move at challenging both Apple and Google and perhaps to rise up in the hardware scene. 
Computing

Oculus VR could upgrade the Rift with a new display in 2019

Oculus could be set to release a new version of its Rift headset in 2019, but it will be more of a modest upgrade than a true sequel. The Rift S, as its purportedly called, will have a new display, and inside-out tracking.
Mobile

Google awarded patent for using eye tracking to detect expressions in VR

Google was awarded a patent that involves using eye tracking to infer facial expressions using machine learning in virtual reality. The tech could help make virtual reality a whole lot more immersive than it already is.
Gaming

Immerse yourself in a new universe with these incredible PSVR games

The PSVR has surpassed expectations and along with it comes an incredible catalog of games. There's plenty of amazing experiences to be had so we've put together a list of the best PSVR games available today.
Virtual Reality

Prototype Valve VR headset leaked: HTC Vive challenger confirmed?

Leaked images revealed that a Valve VR headset is in development, even amid Valve's partnership with HTC for the HTC Vive. Sources confirmed the device, which may be bundled with a Half-Life VR game.
Virtual Reality

Is the Vive Pro better than the original Vive? Our answer might surprise you

HTC Vive vs. Vive Pro, which comes out on top? That's the subject of our latest comparison, which looks at everything from tracking solutions, to controllers, and the brand new headset that could set a new standard for VR.
Gaming

The best HTC Vive games available today

So you’re considering an HTC Vive, but don't know which games to get? Our list of 25 of the best HTC Vive games will help you out, whether you're into rhythm-based gaming, interstellar dogfights, or something else entirely.
Computing

A Google patent shows a way to make VR even more immersive

Virtual reality can be a really immersive experience, but it does sometimes it does have boundaries. Google has addressed this problem by patenting shoes with a flexible region on the bottom.
Virtual Reality

Think virtual reality is just for games? These awesome apps will change your mind

Virtual reality isn't all about gaming. Swim with turtles, paint in 3D, and immerse yourself in some unique experiences the platform has to offer with our curated list of the best VR apps.