Inside the lab that’s bringing Darth Vader, X-Wings, and C-3PO to life in VR

Anyone who owns an HTC Vive can get a short teaser of what the digital wizards at ILMxLab are up to at the Letterman Digital Arts Center in San Francisco.

Star Wars: Trials on Tatooine is a 5-minute long virtual reality experience that lets you star in an original Star Wars story alongside R2-D2, the Millennium Falcon (with Han Solo communicating with you from the cockpit), and your very own light saber from Luke Skywalker. It’s available for free download, and stands out as one of the most impressive VR experiences out there today.

“Most of the work that we’re doing at ILMxLab is really around looking at longer experiences now,” Rob Bredow, chief technology officer at Lucasfilm, told Digital Trends before his talk at the VIEW Conference in Turin, Italy. “Trials on Tatooine is really just a bite-sized experiment and it was designed to be three to five minutes. You can stay in there as long as seven or eight if you really take the time, but it’s a relatively short experience.”

“We’re doing a VR experience that’s going to feature Vader, and that’s not just a little bite-sized experience.”

Next up for ILMxLab is a longer experience that’ll include more storytelling, and star one of the best-known characters in the Star Wars universe. Darth Vader.

“We’re doing a VR experience that’s going to feature Vader, and that’s not just a little bite-sized experience,” Bredow said. “It’s designed to be very much more involved. And the ergonomics are getting better as time goes on, and we’re looking at an audience that’s getting more acclimated to experiencing things in VR. We’ll see as the year plays out how people adapt to those longer experiences, and how people enjoy those longer experiences, but we’re pretty optimistic.”

Bredow knows firsthand how easy it is to lose oneself in an immersive VR environment. He said he can easily spend half an hour in Tilt Brush on HTC Vive making something, without even realizing what time it is.

The future of storytelling

ILMxLab, which is comprised of talent from Lucasfilm, ILM, LucasArts, and Skywalker Sound, is focused on developing, creating, and releasing story-based immersive entertainment for virtual reality (VR), augmented reality (AR), and mixed reality platforms.

“We formed this development group at Lucasfilm to be able to maintain some of the amazing talent from LucasArts, who have made some of the most innovative games for many years — and games that I was a fan of for many years, and a whole bunch of new talent that’s come in to really help augment our creative and our engineering teams to work on virtual reality, mixed reality, and augmented reality at ILMxLab,” Bredow said.

In addition to developing the Vader VR experience, Bredow’s team has worked with Electronic Arts and developer DICE on the upcoming Star Wars Battlefront Rogue One: X-Wing VR Mission for PlayStation VR.

“We share assets back and forth,” Bredow said. “If you look at some of the stuff that DICE has shared at technical conferences like SIGGRAPH, you’ll notice a lot of familiar things. You’ll notice the materials are happening on the same kind of turntables that Industrial Light & Magic does their turntables on, and that’s not by accident. It was because of a very close collaboration and a real mutual interest. We love the work they’re doing.”

Star Wars is about conflict, but collaboration is key

Bredow said his team is most interested in creating high fidelity experiences that are going to put users in a world that they want to visit, whether that’s Star Wars, or a world from one of ILM’s partner studios. Given that ILM works with most big Hollywood projects, that provides plenty of chance for collaboration.

“There are all these fresh new challenges for us in virtual reality.”

“Everybody is in this field right now, and it’s really great. It feels like the early days of computer graphics, when everything was really tricky…like how are we going to calculate that shadow when we only have half a millisecond to get that done? There are all these fresh new challenges for us in virtual reality.”

With every new experiment comes new learnings, including hundreds, if not thousands, of things out of Trials of Tatooine. Skywalker Sound works hand-in-hand with ILMxLab, which means all the real sound effects from the Star Wars movies are used for the VR experience.

The next challenge for the ILMxLab team is bringing Vader’s pronounced breathing into the 360-degree sound environments that virtual reality platforms utilize. Even in Trials on Tatooine, which features lightsaber gameplay against squads of incoming storm troopers, audio plays an important role in directing the action within a VR story.

Using VR to see the future of film

ILMxLab technology is also helping filmmakers like Gareth Edwards bring Rogue One to life. He used virtual reality to set up shots and explore sets before they were built, and in some cases step into environments that would only exist virtually through the magic of computer effects. VR is now being used in production of all the Star Wars films.

In San Francisco, ILMxLab has created the xDeck, which can create immersive environments by projecting 3D pixels on multiple walls. Think of it as a modern-day HoloDeck. This allows multiple people to step into the room and see what a set, or new character, or ship looks like. A similar DISH technology is used by Walt Disney Imagineering at their Los Angeles headquarters and at Walt Disney World in Orlando that’s helping Imagineers build new Star Wars attractions, rides and restaurants for the upcoming Star Wars lands at Disneyland and Disney World. ILMxLab is collaborating with these teams on those projects, as well.

ILMxLab is looking to the future — approximately three years forward, says Bredow — so that Lucasfilm and other ILM client companies can immerse users in VR, AR and mixed reality experiences that blend the best of the video game world and traditional filmmaking. While Bredow can’t predict exactly what the world will look like in three years, he does believe that we’re at the dawn of a new type of storytelling. And the team at ILMxLab hopes to be ahead of the curve as the entertainment world evolves.


From the road to your wrist, see how Android has evolved over the past 10 years

Android started out as just a mobile operating system, but 10 years in it's pretty much everywhere. Check out our round-up of all the different Android variations that have cropped up so far, and what might be coming in the future.

5G’s arrival is transforming tech. Here’s everything you need to know to keep up

It has been years in the making, but 5G is finally becoming a reality. While 5G coverage is still extremely limited, expect to see it expand in 2019. Not sure what 5G even is? Here's everything you need to know.

The hottest Nintendo Switch games you can get right now

The Nintendo Switch's lineup started off small, but games have steadily released as the console continues through its second year. Here are the best Nintendo Switch games available now, from Super Mario Odyssey to Fortnite.

Relive Nintendo’s handheld golden age with the 25 best Game Boy Advance games

The Game Boy Advance was the swan song of the Game Boy era. It also happened to have a boatload of amazing games. We decided to countdown our 25 favorite GBA games. Check it out and let us know your favorites in the comments below!
Movies & TV

'Prime'-time TV: Here are the best shows on Amazon Prime right now

There's more to Amazon Prime than free two-day shipping, including access to a number of phenomenal shows at no extra cost. To make the sifting easier, here are our favorite shows currently streaming on Amazon Prime.

Oculus VR could upgrade the Rift with a new display in 2019

Oculus could be set to release a new version of its Rift headset in 2019, but it will be more of a modest upgrade than a true sequel. The Rift S, as its purportedly called, will have a new display, and inside-out tracking.

Google awarded patent for using eye tracking to detect expressions in VR

Google was awarded a patent that involves using eye tracking to infer facial expressions using machine learning in virtual reality. The tech could help make virtual reality a whole lot more immersive than it already is.

Immerse yourself in a new universe with these incredible PSVR games

The PSVR has surpassed expectations and along with it comes an incredible catalog of games. There's plenty of amazing experiences to be had so we've put together a list of the best PSVR games available today.
Virtual Reality

Prototype Valve VR headset leaked: HTC Vive challenger confirmed?

Leaked images revealed that a Valve VR headset is in development, even amid Valve's partnership with HTC for the HTC Vive. Sources confirmed the device, which may be bundled with a Half-Life VR game.
Virtual Reality

Is the Vive Pro better than the original Vive? Our answer might surprise you

HTC Vive vs. Vive Pro, which comes out on top? That's the subject of our latest comparison, which looks at everything from tracking solutions, to controllers, and the brand new headset that could set a new standard for VR.

The best HTC Vive games available today

So you’re considering an HTC Vive, but don't know which games to get? Our list of 25 of the best HTC Vive games will help you out, whether you're into rhythm-based gaming, interstellar dogfights, or something else entirely.

A Google patent shows a way to make VR even more immersive

Virtual reality can be a really immersive experience, but it does sometimes it does have boundaries. Google has addressed this problem by patenting shoes with a flexible region on the bottom.
Virtual Reality

Think virtual reality is just for games? These awesome apps will change your mind

Virtual reality isn't all about gaming. Swim with turtles, paint in 3D, and immerse yourself in some unique experiences the platform has to offer with our curated list of the best VR apps.

Dive head first into the best experiences available now on the Oculus Rift

The Oculus Rift brought back virtual reality and put a modern twist to it. Grab your Touch Controllers, put on your VR headset, and jump into the fun with some of the best Oculus Rift games available now.