How ‘Coco VR’ convinced Pixar to put its pixels (and pride) into virtual reality

Pixar is one of the few studios that creates movies that truly appeal to the young and old alike. Its latest effort, Coco, is no different, telling the heartwarming story of budding musician Miguel Rivera against the visually stunning backdrop of the Dia de los Muertos.

In the movie, Rivera finds himself transported to the land of the dead for his adventure. Coco VR allows Oculus Rift owners to do the same, crossing over from the land of the living by putting on their headset.

This is the first time that Pixar has dabbled in VR — and it may well have changed how the company approaches the filmmaking process altogether.

Virtual experiment

Marc Sondheimer is an Academy Award-winning producer at Pixar, who has worked on a number of beloved projects over the last sixteen years. Coco VR, however, offered up a novel challenge.

“It was new and different for me, and for most people here at Pixar since we hadn’t done anything in that space before,” he said when he spoke to Digital Trends earlier this month. “If we had known what we know now, it probably wouldn’t have happened. We just jumped in head first.”

Marc Sondheimer
Marc Sondheimer (Photo: John Shearer/Getty Images)

That comment shouldn’t indicate any regrets about how the project turned out — it simply illustrates the fact that Coco VR turned out to be much more ambitious than expected. The number of activities on offer and the level of interactivity involved are far beyond what was initially conceived.

Initially, Pixar was unsure whether to create a smaller VR experiment in support of last year’s Cars 3, or something slightly larger to coincide with Coco. “For a couple of months we talking about different ideas, what we could achieve and what we couldn’t achieve,” explained Sondheimer. “We really felt that Coco was the right property to use, because visually it’s so appealing and stunning, and it’s an imaginary world that we thought would lend itself well to people immersing themselves.”

One of the biggest questions early in the development process was how much of a narrative component should be included. Pixar sought guidance from both partners Magnopus and Oculus, given their personnel’s experience working with the technology.

“Frankly, we weren’t sure how much to lean into the narrative piece, versus choosing an adventure or making it more like a game,” reflected Sondheimer. “That was kind of unknown or uncertain; what’s entertaining in virtual reality, or how to tell a good story.”

VR for Everyone

Much like the majority of Pixar’s back catalogue, Coco is a movie that’s be enjoyed by adults and children alike, and both sections of the audience will find something to appreciate in its VR companion. An immersive concept art gallery offers a very different experience to the opportunity to star in a musical celebration of Dia de los Muertos — but since Coco VR is self-guided, users can pick and choose the activities that appeal to them.

“Our goal was to provide a choice of thing to do, and to give the visitor agency over which parts they want to concentrate on and which areas of the universe they’re most interested in,” explained Alex Henning, co-founder of Magnopus, the VR specialist that collaborated with Pixar to create Coco VR, speaking to Digital Trends.

“We look at all of these projects as an opportunity — and really a responsibility — to push this immersive medium forward.”

Much has been said about the cost of entry associated with VR, even though headset prices are falling. Magnopus wanted to create an experience that adult Pixar fans and VR enthusiasts could enjoy, but it was also important that children who got caught up in the world of Coco would be able to explore Coco VR.

To ensure that this was the case, Disney and Oculus took a proactive approach. Coco VR was playable in Disney stores, AMC theaters, at the film’s premiere, and at various Dia de los Muertos events. “I was thrilled that a broader audience was going to get to experience it,” said Henning. “Not just people who happened to own a gaming PC and the other hardware.”

Magnopus is committed to pushing the boundaries of VR, both in terms of what’s possible using the technology and how broad the audience for this content is. Henning anticipates a future where everyone — from young children to his parents’ generation — are excited about putting on a headset and stepping into a virtual world.

Pixar’s movies have a broad appeal, so Coco VR might convince someone who’s unfamiliar with the technology to try it out for themselves. Of course, that means that the experience has to be stellar to ensure that those newcomers aren’t put off.

The Benefit of Experience

Henning and fellow co-founders Rodrigo Teixeira and Ben Grossmann were inspired to establish Magnopus based on their experience in the film industry, and an expectation that new technology was set to change the entertainment landscape. Virtual reality stood out because it seemed to diverge from the information overload of modern society, offering up something closer to the experiential memories of their respective childhoods.

Despite the fact that Coco VR is the first VR adaptation of a Pixar property, the studio’s strong track record means that it has to meet some exacting standards — both in terms of the fan base and the creative forces behind the movie.

The company garnered plenty of praise when Mission: ISS — a simulation of life aboard the International Space Station developed in collaboration with NASA, the ESA, and the CSA – was released in March 2017. In October 2017, Blade Runner 2049: Memory Lab gave fans the opportunity to step into the world of the cyberpunk sequel.

Memory Lab is broadly similar to Coco VR, in that both experiences are intended to expand upon their source material. For Henning, focusing on the world of the source material is crucial for this kind of project.

“You’re serving a lot of masters simultaneously, right?” he said. “On the one hand you’ve got a primary focus with these pieces that are movie tie-ins, which is to promote the movie itself. But for us, we look at all of these projects as an opportunity — and really a responsibility — to push this immersive medium forward.”

Henning explained that Magnopus feels the need to cover new ground with each project, whether it’s building upon work that’s been done by other studios working with VR or establishing entirely new techniques. He points to the holographic capture process used in Memory Lab and the novel social mechanics that are present in Coco VR.

“You’re always looking for like, ‘what is a thing we could try here that might unlock some greater potential,'” added Henning.

Stepping into the Scene

Coco VR was an opportunity for Pixar to work with a new medium. For a studio that helped bring computer animation to the mainstream in the 1990s, that’s an exciting prospect. What’s more, the process is set to have an impact on the company’s upcoming slate of films — albeit perhaps not in the way you might expect.

In June 2018, Pixar will release Incredibles 2, its next feature. There’s no word on whether or not it will be accompanied by an experience along the lines of Coco VR — but even if the studio never releases content of this kind again, there should be no doubt as to the impact the project had on its development process.

“We’re using some techniques that we developed for Coco VR in Toy Story 4,” said Sondheimer. The upcoming sequel will join the ranks of one of the most critically acclaimed and financially successful series of all time, so you can bet that Pixar must be on board with VR if the technology is to contribute to its production.

Fans shouldn’t necessarily expect to be donning their headset to go to infinity and beyond, though. While a VR companion is definitely possible, Pixar seems to be particularly excited about how the technology might be used behind the scenes.

“As we were building the experience, we were actually collaborating inside of VR, together,” said Henning. “You would have Pixar’s creatives and project leads walking around as skeletons with our creatives and project leads, all together, actually in the experience — them in Emeryville and us in Los Angeles — exploring the space together and making decisions natively in the medium, as opposed to just doing a video conference or a phone call.”

Alex Henning
Alex Henning (Photo: John Shearer/Getty Images)

As you can imagine, being able to physically inhabit a virtual space rather than just view it as a collection of 3D models has a certain appeal for a studio that specializes in computer animation. VR could help Pixar adapt practices from live-action filmmaking that were previously unattainable.

“We model it, then we drop the filmmakers in those spaces to get a feel for scale, design, and to start finding some of their camera shots that they might want to achieve,” said Sondheimer. “That’s new, for us, but it’s also very exciting, for sure.”

The future of filmmaking

Sondheimer describes the studio’s exposure to this kind of technology as being “completely unexpected, but extremely valuable.” There was no preconception that the social, multi-user aspects of Coco VR could factor into the creation of future film projects. However, it’s impact on the team’s ability to collaborate with one another and get on the same page.

“I know that it’s something that they’re using now in our filmmaking — you get a production designer, a director, and the head of our camera department all in, doing a walkthrough of a space together, virtually,” added Sondheimer. “Coco VR unlocked that door, at least in three dimensions.”

It’s easy to pigeonhole VR as a new type of entertainment, something that blurs the lines between movie and video game without every really belonging to either category. However, we’re seeing the technology used for practical purposes more and more. “I think it’s a tool,” said Sondheimer. “I think it’s a great tool for filmmaking.”

Pixar set out to make an entertaining experience, and ended up discovering a new way to leverage technology to benefit its filmmaking process. This is part of a broader trend that spans both VR and augmented reality – whether it’s diagnosing concussions or tracking tools on a job site, these headsets have a place in the world of work.

On the surface, wearing a headset might seem isolating, but Pixar discovered how a shared virtual space can help people tackle a project together. In time, we might see all kinds of industries use social VR to foster communication and collaboration in the workplace.