Meet the filmmaker turning the Oculus Rift into a VR theater with Zero Point

zuckerberg says vr is facebooks next big thing oculus rift condition one

Filmmaker Danfung Dennis is no stranger to battlegrounds. He spent four weeks in 2009 shooting with U.S. Marines in Afghanistan for the Oscar-nominated 2009 documentary, Hell and Back Again. Now Dennis is moving on to a very different flavor of battleground with Zero Point, a movie that both reflects on and threatens to disrupt traditional filmmaking by releasing exclusively for the Oculus Rift. It’s an inventive concept, one that immediately brought to mind filmmaker Alexander Sokurov’s beautiful Russian Ark.

Shot on a single day in December 2001, Russian Ark is an avant-garde piece that consists of a single, continuous tracking shot. It weaves a path through the Winter Palace in Saint Petersburg, Russia while a narrator relates a loose narrative. It’s 96 uninterrupted minutes that amounts to a monumental feat of carefully staged choreography, with actors coming in and out of the frame on specific cues, at one point by the hundreds.

“We no longer have control of what frame the audience is going to see.”

The film’s extended use of Steadicam – a stabilizing rig that eliminates shake from a handheld camera – is a key source of inspiration for Dennis. Through Zero Point, he intends to explore the evolved entertainment medium that is rapidly developing around virtual reality by making use of one of its most successful iterations thus far, the as-yet-unreleased headset from Oculus VR.

Russian Ark is the masterpiece of Steadicam work. I think that will work really well in the Rift, where it’s one unbroken, cohesive story and experience that’s sort of passing by you,” the Zero Point director tells Digital Trends.

By design, the movie will unfold “on rails,” pulling the audience along from one chamber to another, much like Russian Ark gradual passage different rooms and time periods. The idea is to have each space deliver some meditation on emerging VR applications, with audio from various expert interviews playing over an immersive series of presentations. These Holodeck-like chambers are part of a space station on which the whole journey unfolds.

“You’re tying spatial locations to the story, and I think that’s really important for how our memory works,” Dennis explains. “Our brain is constantly mapping environments around us. We think what’s going to be really different from traditional filmmaking is that sense of space and physical location, and what’s happening in that physical location.”

This speaks directly to the principal challenge of VR filmmaking. Conventional concepts like framing and editing don’t apply when you’re plunging your audience into a manufactured reality. The “camera” is, for all intents and purposes, the eyes of the viewer, and it offers an unrestricted viewpoint on the world that’s been created. You may not be able to move around the space like it’s a video game, but you can turn your head to look in any direction. The question is, how do you build a cohesive plot when the focus of the camera’s attention is completely unrestricted and impossible to predict?

“This is a new medium, a new language, and we have to invent the grammar and the syntax of how you tell an effective story in this medium,” Dennis says. “We no longer have control of what frame the audience is going to see. We’re delivering all of this raw information … and then letting them decide what portion of it they want to watch.”

“It’s one unbroken, cohesive story and experience that’s sort of passing by you”

One possible solution is positional audio. Traditional definitions of the “shot” cease to exist in VR, and this places added importance on sound design, the fundamental non-visual component of filmmaking. You may not be able to steer what the viewer sees, but you can draw their attention in different directions by appealing to their ears. Dennis sees that as only one component of assembling largely non-interactive VR pieces, however.

“A lot of [our filmmaking approach] is going to be this merger, I think, between gaming and film,” he says. “There’s a lot we can learn from how first-person games have told those stories where you are a character. It’s completely new territory. We know it’s going to be extremely challenging to figure out how we tell a story or convey an experience, so we’re researching and developing that at the same time the technology is coming around.”

hell_and_back_again_danfungdennis1_byjoeraedle-1-vertAlongside the more esoteric questions of how you shoot a largely passive VR experience is the very practical dilemma of developing technology to do so. Conventional filmmaking techniques are impractical for this kind of project because traditional camera rigs aren’t built to capture a 3D space. Sokurov’s Steadicam setup worked in Russian Ark, but Zero Point will plunge viewers into a space that they’ll be able to fully explore with their eyes.

“We built a camera system … based on these existing digital cameras that can shoot high frame rate on high resolution,” Dennis tells us. “We have an array of these high-resolution cameras and each one is shooting quite a wide angle [and] we’re stitching those together to create that 360 panoramic video, but we’re also shooting 3D. So you’re getting both depth and the full 360.”

“Doing 360 video, that’s been done before. Doing 3D movies, that’s been done of course. But doing those together is an entirely new type of capture system that we had to build specifically for the Rift. We’re still prototyping that, we’re still adding more cameras [and] there’s a lot of technical issues to still be solved, but it’s getting there.”

The most pressing challenge is getting those images to stitch together correctly. Sokurov created an incredibly convincing illusion in Russian Ark, but he did so from a single camera’s perspective. In Zero Point, Dennis will perform a similar feat, but with an elaborate array of cameras capturing 360-degree spaces. That’s a challenge when the camera isn’t moving; it becomes considerably tougher when you’re working with something more than a static shot.

“It’s a challenge to get [the footage from each camera in our array] to stitch correctly,” Dennis says. “Once the entire camera starts moving and everything is passing through the seams, it gets really hard to create this seamless experience. Motion is really hard for this medium, and keeping that really steady, balanced, smooth shot is so important to reduce motion sickness.”

That’s why Steadicam, one of Hollywood’s most favored filmmaking standbys, is also a potentially handy tool to use in the nascent practice of VR filmmaking. “So there’s something that keeps coming up in these stabilized shots on continuous scenes that I think Russian Ark, that type of storytelling, [provides an ideal framework] for virtual reality.”

(Images and video © Danfung Dennis and Condition One)

Product Review

With the Z7, Nikon gives DSLR holdouts the mirrorless wonder they've waited for

Nikon’s long awaited full-frame mirrorless cameras are here, and the Z7 is the new flagship model. But does it stand up to the company's DSLR pedigree, and, more importantly, does it have what it takes to compete with the likes of Sony?
Movies & TV

The best movies on Amazon Prime right now (November 2018)

Prime Video provides subscribers with access to a host of fantastic films, but sorting through the catalog can be an undertaking. Luckily, we've done the work for you. Here are the best movies on Amazon Prime Video right now.
Mobile

Which smartphone has the best camera? We found the sharpest shooters

They say that the best camera is always the one you have with you and that makes your smartphone camera very important indeed. Join us for a closer look at the best camera phones available right now.
Gaming

Immerse yourself in a new universe with these incredible PSVR games

The PSVR has surpassed expectations and along with it comes an incredible catalog of games. There's plenty of amazing experiences to be had so we've put together a list of the best PSVR games available today.
Gaming

These cheat codes made gaming history

For almost as long as video games have existed, there have been video game cheat codes. Here are some of the most famous cheat codes, including the legendary Konami Code found in dozens of games.
Gaming

PlayStation Classic powered by open-source emulator made by fans

The PlayStation Classic will be powered by the open-source, fan-made PCSX ReARMed emulator. The move is an unexpected one for Sony, particularly because it has not entirely been friendly with the emulation community.
Emerging Tech

Awesome Tech You Can’t Buy Yet: 1-handed drone control, a pot that stirs itself

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Virtual Reality

Prototype Valve VR headset leaked: HTC Vive challenger confirmed?

Leaked images revealed that a Valve VR headset is in development, even amid Valve's partnership with HTC for the HTC Vive. Sources confirmed the device, which may be bundled with a Half-Life VR game.
Gaming

The next ‘Overwatch’ patch will bring major changes, require a full reinstall

The next major Overwatch patch will be a remaster patch, and will include load time improvements and memory optimization, among other things. With so many things to be added to the game, a full reinstall will be required.
Deals

The best Nintendo Switch deals and bundles for November 2018

Looking to score Nintendo's latest hybrid console? We've smoked out the best Nintendo Switch deals right here, including discounts on stand-alone consoles as well as bundles that feature games like Fortnite and Super Smash Bros. Ultimate.
Gaming

Got an NES Classic? Here’s how to hack it to play more than 700 games

The NES Classic is terrific for what it is, but Nintendo's discontinued device remains limited in what it can play. Here's how to hack your miniature console and render it compatible with more than 700 games.
Gaming

‘Fallout 76’ day one patch is 52 GB, bigger than the 45 GB game

Fallout 76 is a 45 GB download, but its day one patch is even bigger at a hefty 52 GB. The contents of the patch remain unknown, but it will likely incorporate feedback from the game's beta testers.
Gaming

Fired Treyarch tester spills ‘Call of Duty: Black Ops 4 Zombies’ secrets

A former Treyarch employee revealed numerous Call of Duty: Blacks Ops 4 Zombies secrets. Among the unraveled mysteries were the Viking Boat puzzle in IX and the purpose of the Engine Room valves in Voyage of Despair.
Deals

Black Friday 2018: The best deals so far

Black Friday is the biggest shopping holiday of the year, and it will be here before you know it. If you can't wait until November 23 to start formulating a shopping plan, we've got you covered.