Meet the filmmaker turning the Oculus Rift into a VR theater with Zero Point

zuckerberg says vr is facebooks next big thing oculus rift condition one

Filmmaker Danfung Dennis is no stranger to battlegrounds. He spent four weeks in 2009 shooting with U.S. Marines in Afghanistan for the Oscar-nominated 2009 documentary, Hell and Back Again. Now Dennis is moving on to a very different flavor of battleground with Zero Point, a movie that both reflects on and threatens to disrupt traditional filmmaking by releasing exclusively for the Oculus Rift. It’s an inventive concept, one that immediately brought to mind filmmaker Alexander Sokurov’s beautiful Russian Ark.

Shot on a single day in December 2001, Russian Ark is an avant-garde piece that consists of a single, continuous tracking shot. It weaves a path through the Winter Palace in Saint Petersburg, Russia while a narrator relates a loose narrative. It’s 96 uninterrupted minutes that amounts to a monumental feat of carefully staged choreography, with actors coming in and out of the frame on specific cues, at one point by the hundreds.

“We no longer have control of what frame the audience is going to see.”

The film’s extended use of Steadicam – a stabilizing rig that eliminates shake from a handheld camera – is a key source of inspiration for Dennis. Through Zero Point, he intends to explore the evolved entertainment medium that is rapidly developing around virtual reality by making use of one of its most successful iterations thus far, the as-yet-unreleased headset from Oculus VR.

Russian Ark is the masterpiece of Steadicam work. I think that will work really well in the Rift, where it’s one unbroken, cohesive story and experience that’s sort of passing by you,” the Zero Point director tells Digital Trends.

By design, the movie will unfold “on rails,” pulling the audience along from one chamber to another, much like Russian Ark gradual passage different rooms and time periods. The idea is to have each space deliver some meditation on emerging VR applications, with audio from various expert interviews playing over an immersive series of presentations. These Holodeck-like chambers are part of a space station on which the whole journey unfolds.

“You’re tying spatial locations to the story, and I think that’s really important for how our memory works,” Dennis explains. “Our brain is constantly mapping environments around us. We think what’s going to be really different from traditional filmmaking is that sense of space and physical location, and what’s happening in that physical location.”

This speaks directly to the principal challenge of VR filmmaking. Conventional concepts like framing and editing don’t apply when you’re plunging your audience into a manufactured reality. The “camera” is, for all intents and purposes, the eyes of the viewer, and it offers an unrestricted viewpoint on the world that’s been created. You may not be able to move around the space like it’s a video game, but you can turn your head to look in any direction. The question is, how do you build a cohesive plot when the focus of the camera’s attention is completely unrestricted and impossible to predict?

“This is a new medium, a new language, and we have to invent the grammar and the syntax of how you tell an effective story in this medium,” Dennis says. “We no longer have control of what frame the audience is going to see. We’re delivering all of this raw information … and then letting them decide what portion of it they want to watch.”

“It’s one unbroken, cohesive story and experience that’s sort of passing by you”

One possible solution is positional audio. Traditional definitions of the “shot” cease to exist in VR, and this places added importance on sound design, the fundamental non-visual component of filmmaking. You may not be able to steer what the viewer sees, but you can draw their attention in different directions by appealing to their ears. Dennis sees that as only one component of assembling largely non-interactive VR pieces, however.

“A lot of [our filmmaking approach] is going to be this merger, I think, between gaming and film,” he says. “There’s a lot we can learn from how first-person games have told those stories where you are a character. It’s completely new territory. We know it’s going to be extremely challenging to figure out how we tell a story or convey an experience, so we’re researching and developing that at the same time the technology is coming around.”

hell_and_back_again_danfungdennis1_byjoeraedle-1-vertAlongside the more esoteric questions of how you shoot a largely passive VR experience is the very practical dilemma of developing technology to do so. Conventional filmmaking techniques are impractical for this kind of project because traditional camera rigs aren’t built to capture a 3D space. Sokurov’s Steadicam setup worked in Russian Ark, but Zero Point will plunge viewers into a space that they’ll be able to fully explore with their eyes.

“We built a camera system … based on these existing digital cameras that can shoot high frame rate on high resolution,” Dennis tells us. “We have an array of these high-resolution cameras and each one is shooting quite a wide angle [and] we’re stitching those together to create that 360 panoramic video, but we’re also shooting 3D. So you’re getting both depth and the full 360.”

“Doing 360 video, that’s been done before. Doing 3D movies, that’s been done of course. But doing those together is an entirely new type of capture system that we had to build specifically for the Rift. We’re still prototyping that, we’re still adding more cameras [and] there’s a lot of technical issues to still be solved, but it’s getting there.”

The most pressing challenge is getting those images to stitch together correctly. Sokurov created an incredibly convincing illusion in Russian Ark, but he did so from a single camera’s perspective. In Zero Point, Dennis will perform a similar feat, but with an elaborate array of cameras capturing 360-degree spaces. That’s a challenge when the camera isn’t moving; it becomes considerably tougher when you’re working with something more than a static shot.

“It’s a challenge to get [the footage from each camera in our array] to stitch correctly,” Dennis says. “Once the entire camera starts moving and everything is passing through the seams, it gets really hard to create this seamless experience. Motion is really hard for this medium, and keeping that really steady, balanced, smooth shot is so important to reduce motion sickness.”

That’s why Steadicam, one of Hollywood’s most favored filmmaking standbys, is also a potentially handy tool to use in the nascent practice of VR filmmaking. “So there’s something that keeps coming up in these stabilized shots on continuous scenes that I think Russian Ark, that type of storytelling, [provides an ideal framework] for virtual reality.”

(Images and video © Danfung Dennis and Condition One)

Movies & TV

Oscar-winning FX master explains why ‘First Man’ is a giant leap for filmmaking

Paul Lambert, the Oscar-winning visual effects supervisor on First Man, reveals the innovative techniques that blended old footage with modern movie magic to make the Apollo 11 mission to the moon resonate with audiences 50 years later.
Gaming

Immerse yourself in a new universe with these incredible PSVR games

The PSVR has surpassed expectations and along with it comes an incredible catalog of games. There's plenty of amazing experiences to be had so we've put together a list of the best PSVR games available today.
Virtual Reality

Think virtual reality is just for games? These awesome apps will change your mind

Virtual reality isn't all about gaming. Swim with turtles, paint in 3D, and immerse yourself in some unique experiences the platform has to offer with our curated list of the best VR apps.
Photography

From 11K to just OK: The biggest photo gear announcements at CES 2019

From 11K cameras to 1 TB media cards, CES 2019 brought a peek at new gear for photographers and videographers. But what photography gear grabbed our attention the most? Here are the biggest photo gear announcements from CES 2019.
Gaming

‘Call of Duty: Black Ops 4’ will make Blackout mode free to play this week

Treyarch and Activision are offering Call of Duty: Black Ops 4's Blackout mode as a free trial download for a limited time. The mode will be available from January 17 through January 24.
Gaming

Break out the Wii: ‘Just Dance’ movie reportedly in the works

Sony's Screen Gems has reportedly acquired the film rights to the Just Dance video game series from Ubisoft, which previously helped bring the game Assassin's Creed to the big screen in 2016.
Gaming

‘Fortnite’ security flaw let hackers spy on players through microphones

A security vulnerability found in Fortnite allowed hackers to gain access to other players' accounts, potentially letting them spy on conversations using the in-game microphone. It has been addressed.
Gaming

Microsoft CEO says Project xCloud is the ‘Netflix for games’

Microsoft CEO Satya Nadella referred to the company's Project xCloud game streaming service as "Netflix for games." The service will let users play Xbox and PC games on a variety of devices.
Gaming

‘The Division 2’ private beta kicks off February 7 on consoles and PC

The private beta for Tom Clancy's The Division 2 starts on February 7 and runs until February 10, Ubisoft announced in a new story trailer. To guarantee access to the beta, you have to pre-order the game on PS4, Xbox One, or PC.
Gaming

Here are all the awesome games you can play without a fancy graphics card

Just because you don't have a dedicated graphics card, it doesn't mean you can't enjoy some of the best games out there right now. This is our list of the best games you can play on Intel integrated graphics.
Gaming

Console wars are so last-gen. Check out all the games that support crossplay

Crossplay is still in its infancy, but a growing number of games support online multiplayer between competing platforms. Here's a list of all games that support console crossplay.
Gaming

Want to trick out your PlayStation 4? These themes will get you started

Personalize your gaming experience with some of our favorite themes for the PlayStation 4, including free, paid, static, and dynamic options.
Gaming

Play your games whenever you want with a MicroSD card for your Nintendo Switch

The Nintendo Switch uses cartridge-based games, but its internal storage may fill up quicker than you would think. Here's what you should consider when picking out a MicroSD card to expand your Switch's storage capacity.
Gaming

Get your pilot skills up to snuff for this week's 'Fortnite' challenge guide

The Fortnite season 7, week 7 challenges are now available. The major challenge this week requires you to visit all the Fortnite expedition outposts in the game and we walk you through how to get to each.