At NYU’s first-ever VR filmmaking class, it’s all about kittens in boxes

Down the spiraling steps of New York University’s Tisch School of Arts and at the beginning of a shrine of movie posters for films NYU students have made, the future was about to be taught.

Saschka Unseld, the creative director at Oculus Story Studio, would be teaching 60 students from Tisch and the Game Design department a deceptively simple lesson: When you film in VR, you aren’t making movies. You’re making worlds.

This masterclass — which took place on Saturday, February 13 — was the first class of its kind at the school, for Oculus, and one of the first in the country. But history doesn’t heat the bones, and students filed in slowly, faces sullen and stung from Mother Nature’s historic East Coast wind slap. Unseld stood at the bottom of a small, concert-style lecture hall rifling through notes.

An air of cautious enthusiasm permeated the room and the students, who wore their art on their sleeves. A few iconic movie posters ironed into shirts and hats with bright, clashing colors were littered throughout the room. This masterclass may include some of the brightest minds studying film today, but these are still college students, and it was still Saturday morning. Some nodded off, intermittently, others were alternating between texting and blankly staring at the slides Unseld presented, as if collaborating on a puzzle that no one has mastered in years. The majority were attentive, and a few jotted down every syllable Unsled uttered.

“If the kitten is too big, and the box is too small, don’t use the kitten.”

In the lecture hall, Unseld spoke with a calm urgency in his voice, as if trying to ease the students into a conundrum. He let out a frustrated laugh after explaining how a computer-generated scene that would take Pixar 11 hours to render has to be done by a game engine in 11 milliseconds for people to virtually walk through it, unencumbered, in real-time. As the students stared at the seemingly insurmountable task before them, Unseld helped explain with the one thing we can all understand: kittens.

“You have a certain box of the amount of complications you can do in a certain frame,” Unseld began explaining, while a picture of kittens stuffed in a box flashed behind him. “These 11 milliseconds, think of this as a box you have. You can’t fit so much into that box. If the kitten is too big, and the box is too small, don’t use the kitten.”

One of the hyper-attentive attendants committing every word of the lecture to written record was 24-year-old writer and director Eliza McNitt. A graduate from Tisch, McNitt is currently collaborating with The Hubble Telescope on a “VR exploration through space.” She says Unseld and Oculus taught her “that VR is more a process of discovery,” one limited only by how quickly the technology can catch up to the ideas we all have.

Around 11:15 AM, Unseld played a video for the class: Dear Angelica illustrator Wesley Allsbrook swirling her fingers in the air with the Oculus Quill drawing tool and creating virtual reality illustrations in real-time. There was a collective gasp in the room. The phones went face down, the pens stopped moving, and Saschka sensed that he had hit upon something. “That was too short,” he declared. “Let’s play it again.”

Next to the lecture hall were two rooms with stations for the students to test headsets and VR videos. Oculus and VR studio Framestore equipped the rooms. In the Oculus room, three Oculus Rift headsets were available to view Oculus’s two films, Henry and Lost. Henry starts as an unassuming look around a virtual jungle at night time — until a giant robot hand crashes through the trees as the humungous robot it belongs to zips through the night sky. When the disc-shaped eyes of the skyscraper-sized robot leans in to examine who you are, you cannot help but take a step back.

I removed the Rift headset and stepped out of Lost (an immersive safari exploration), and that split-second of fear I felt reminded me of something Allyson Green, dean of Tisch told me minutes earlier about virtual reality. “We’re seeing this whole new way in which to think three-dimensionally in space. It’s the same kind of thrilling moment as when people first saw the train coming out of the station,” she told Digital Trends. She was referring to the 50-second 19th century silent film L’Arrivée d’un train en gare de La Ciotat, in which a train is shown leaving a station and coming directly at the screen. It was a watershed moment in film, as moviegoers in 1895 were so convinced of the authenticity of the oncoming train that they became physically terrified. Some ran out of the theater or tried to dodge it.

Three engineers helped give the future VR filmmakers tips, such as instructing those watching Henry to sit on one of the circular mats and keep their body within those confines. Framestore let students fly through a scene of Avengers: Age of Ultron and explore the world of The Walking Dead.

“We’re seeing this whole new way in which to think three-dimensionally in space.”

By noon, Saschka had finished his presentation and was ready to open the class up for questions.

Another hour went by, and a few students trickled out of the lecture hall, to try the adjacent VR gear. And in that hour, something special happened: NYU Professor Jonathan Weinstein, who was looking on, told me afterwards it was indicative of why the school and Oculus decided to structure the workshop as it did. A few of the students who left at 12 returned around 2 PM, after testing out the VR technology. Some of them even had questions that were clearly informed by their time spent with the machines.

They had learned.

This ability to apply a new thought, tangibly, after learning is central to unlocking virtual reality filmmaking, Unseld said in his presentation. It’s incredibly hard to create in virtual reality, so Unseld suggested filmmakers preview their work as frequently as possible to stay on course. With virtual reality, the one lesson that was almost immediately acted upon by the students was simply exploration and review.

No one in the room was as excited as the dean, who stayed well after students broke for lunch at 2:30. She made sure to clarify with professor Weinstein that there were only 60 students in the lecture hall because that was the limit. Green’s eyes brightened with every proclamation of VR’s potential and every question.

Mike Woods, founder of White Rabbit VR and Christine Cattano from Framestore VR spoke further about the difficulties of VR filmmaking after Unseld.

The next George Lucas didn’t make himself known at NYU’s first foray into teaching the new medium.

But then again, a virtual world isn’t filmed in a day.

Movies & TV

Why First Man’s Oscar-nominated visual effects are a giant leap for filmmaking

Paul Lambert, the award-winning visual effects supervisor on First Man, reveals the innovative techniques that blended old footage with modern movie magic to make the Apollo 11 mission to the moon resonate with audiences 50 years later and…

Everything you need to know about routers, modems, combos, and mesh networks

Modem vs. router: what's the difference? We explain their functions so you can better diagnose any issues prior to contacting technical support. We also talk about a few variants you'll see offered by ISPs and retailers.

Barbie’s Corvette ain’t got nothing on Sphero’s fully programmable robot car

Sphero is known for devices like the Sphero Bolt and BB-8 Star Wars toy, but now the company is back with another addition to its lineup -- the Sphero RVR. The RVR is a fully programmable robot car that can be expanding with different…

Flip from portrait to landscape as we reveal how to rotate a video on iPhone

If you've accidentally shot a video in portrait orientation and you want to flip to landscape, then this is the guide for you. We'll explain how to use iMovie to rotate a video on your iPhone or iPad for free and suggest alternative apps.

These are the coolest virtual and augmented reality gadgets from CES 2019

CES 2019 had plenty of VR and AR gadgets on display, including headsets that completely change how you experience virtual reality, and some that don't even require a PC or a phone to run.

These shoes let me stroll through ‘Skyrim,’ and I desperately want to go back

After being funded in just two hours on Kickstarter back in October 2018, Cybershoes has earned itself a place among the coolest VR walking and running tech. At CES 2019, we got to try them out and they live up to the hype.

The Teslasuit could turn Black Mirror’s terrifying ‘Playtest’ into a reality

We spoke with Teslasuit co-founder Dimitri Mikhalchuk about VR gaming at CES 2019. With all its features, the future of the Teslasuit and virtual reality look bright. And it also sounds a bit like a Black Mirror episode.

Could the next Microsoft HoloLens be announced at MWC 2019?

After not having a presence at Mobile World Congress for three years, Microsoft is now sending out media invites for a press conference on February 24 during the annual event in Barcelona. Could a next-generation HoloLens be on the way?

Immerse yourself in a new universe with these incredible PSVR games

The PSVR has surpassed expectations and along with it comes an incredible catalog of games. There's plenty of amazing experiences to be had so we've put together a list of the best PSVR games available today.

Take a trip to a new virtual world with one of these awesome HTC Vive games

So you’re considering an HTC Vive, but don't know which games to get? Our list of 25 of the best HTC Vive games will help you out, whether you're into rhythm-based gaming, interstellar dogfights, or something else entirely.
Virtual Reality

Is the Vive Pro better than the original Vive? Our answer might surprise you

HTC Vive vs. Vive Pro, which comes out on top? That's the subject of our latest comparison, which looks at everything from tracking solutions, to controllers, and the brand new headset that could set a new standard for VR.

Samsung files a VR patent featuring a curved OLED display

Doubling down on its emphasis on curved displays, Samsung recently filed a design patent for a new virtual reality headset that could feature a curved OLED display, which would be an interesting development in VR.

Bringing realism to VR is complex, but these developers found a way in holograms

Making virtual reality feel real is the hardest job of all VR developers. For Awake: Episode One, StartVR used volumetric recording, rather than motion capture, to bring its characters to life like never before.