NASA astronauts have already been to Mars – in VR

Nasa Project Sidekick
Nasa

NASA has used virtual reality for decades. But the recent wave of innovation that has fueled at-home VR platforms like PlayStation VR, Oculus Rift, and HTC Vive, is impacting how the space agency trains astronauts for missions.

Jeff Norris, mission operations innovation lead at the NASA Jet Propulsion Laboratory, told Digital Trends that his lab has active partnerships with almost every company that makes a device you can put on your head, including Valve, HTC, Microsoft, Sony, Oculus, Samsung, and others.

From your PlayStation 4, to outer space

“The difference in our work due to the consumer electronics industry’s investment in VR and AR has been profound for us,” Norris said. “It has unlocked opportunities for our work that weren’t available before.”

“We can make astronauts feel they’re really present [where] they’re going to be exploring.”

Even before Sony launched PlayStation VR at retail, Norris had early versions of the device – back when it was called Project Morpheus – being used in tests to control the Robonaut 2 (R2) humanoid robot, which is currently on board the International Space Station.

“PlayStation VR allows us to more naturally control a humanoid robot,” Norris said. “It’s as natural as moving your own hand.”

In addition to R2, astronauts aboard the International Space Station utilize a pair of Microsoft HoloLens AR headsets as part of Project Sidekick. Norris’ team is using AR to augment the performance of astronauts working in space.

“We’re constantly working with our partners to use the unique features of these devices to map those into the unique elements of the problems we’re trying to solve,” Norris said. “Different manufacturers are doing different things. We want to see these platforms diversifying and exploring this domain, because we benefit by using their devices.”

NASA is actively looking for new technology, as well. The space agency had its first booth at the Consumer Electronics Show (CES) in Las Vegas this past January. It also sends representatives to video game shows like the Game Developers Conference (GDC) in San Francisco, and the Electronic Entertainment Expo (E3) in Los Angeles.

Reach out and touch someone

Bob Vlemmix, co-founder of Manus VR, said NASA began working with the company’s data gloves this past March. The gloves replicate your hands in VR, AR and mixed reality environments, allowing the user to work across enterprise, entertainment and gaming landscapes.

“Manus VR sees a market where both NASA, and your average gamer, have access to the same cutting-edge technology.”

NASA is using a prototype version the gloves, which aren’t available to the public yet, to help train astronauts before they head to the International Space Station. NASA uses the data gloves with the Vive. It has built mock-up rooms on Earth that replicate the layout of the International Space Station. Astronauts are able to navigate virtual spaces and pick up and use tools, just as they would in space.

“By recreating physical locations in VR, you can simulate environments that are not easily accessible, or require extensive training,” Vlemmix said.

Vlemmix said that just a few years ago data gloves cost thousands of dollars and were a commodity only available to big institutions. Now, consumer products in virtual reality have outdone the bulky, expensive hardware of the past in both cost and performance.

“Manus VR is creating a market where both NASA and your average gamer have access to the same cutting-edge technology,” Vlemmix said.

In addition to developing a consumer glove, Manus VR is adding support for SteamVR Tracking through a tracking bracelet that will provide the Manus VR gloves with positional tracking for each trainee’s hands and arms. Vlemmix said the Manus VR gloves will work across VR and AR devices, including HoloLens, and support for additional platforms will be added in the future.

Journey to Mars, sooner than expected

VR is also helping NASA plan its first exploration of Mars is the 2030s. Norris’ team has used the Oculus Rift to train astronauts how to operate rovers on Mars more effectively.

“Because I work in mission operations, my team has been involved in how things are going on Mars, and how things are accomplished,” Norris said. “We have this new medium that we can make astronauts feel like they’re really present in these environments that they’re going to be exploring. What does it look like standing on a hilltop on Mars? What does it feel like to be on the space station? These are the questions in so many people’s minds, and VR is a way we can answer them.”

NASA, through a partnership with television network Fusion, is also bringing the Red Planet home for anyone to explore. The Mars 2030 Experience will launch across Oculus Rift, Google Cardboard, Samsung VR Gear, iOS and Android, although no release date has been given yet.

Using Unreal Engine 4, MIT and Fusion are collaborating with NASA to create a 7.7 square mile interactive environment on Mars that players can traverse on foot or by rover. The experience features actual topography, authentic gravity, and real lighting, as well as authentic missions based on things astronauts will one day do on the planet.

The Mars 2030 Experience is focused on a public outreach and engagement to share the experience,” Norris said. The rover, the habitat modules, the labs, and everything else is based on actual CAD models provided by NASA.

“We don’t have nearly as much data on the topology of Mars as we do of Earth,” Justin Sonnekalb, a technical designer at Irrational Games who is working on this project, said. “Recreating the planet’s surface is a painstaking process based on multiple (and occasionally slightly conflicting) data sets, which had to be added to the terrain system, interpolated, and then manually adjusted to match the feel of the reference shots.”

And this is just the beginning of how NASA can benefit from the VR and AR technology entering living rooms around the world. Decades ago, people across the globe watched astronauts step foot on the moon through their televisions. When humans first step on Mars, we might be right there with them – in VR.

Product Review

Oculus Quest is the affordable VR rig we’ve been waiting for

Oculus announced that its Project Santa Cruz virtual reality headset will ship next year as the Oculus Quest, and we got to try out several new game titles on the Quest. Find out our impressions of VR without wires.
Emerging Tech

Whose name should we etch on the Mars 2020 rover? NASA wants a vote

Dream of making it to Mars? NASA has opened up a new public outreach program to let people send their names to the Red Planet, as an engraving on a silicon chip launched with the Mars 2020 rover.
Emerging Tech

See a fly-over of Mars and track the path Curiosity will take up Mount Sharp

A new animation from NASA shows a fly-over of Mount Sharp on Mars, the location where the Curiosity rover is currently exploring. It also shows the path that Curiosity will take over the next few years.
Emerging Tech

See the impact site where the Beresheet spacecraft crashed into the moon

An image of the crash site of SpaceIL's ill-fated Beresheet spacecraft has been captured by NASA's Lunar Reconnaissance Orbiter and is being analyzed for information about the moon's soil.
Gaming

Valve Index VR headset costs $500, but the full bundle will set you back $1,000

Valve is stepping into the VR market with the high-end Index, and the company finally revealed the price and release date. The Index can be purchased individually or as part of a couple of bundles.
Computing

Oculus Rift S gets a release date, pre-orders are now available

Oculus plans to phase out its flagship Rift VR headset for its newly created Rift S. The Rift S made its debut at the 2019 Game Developers Conference and will be released in May.
Virtual Reality

Oculus Quest taunts some customers after arriving more than two weeks early

The Oculus Quest has arrived early for some customers. The standalone VR headset, which is not supposed to ship out until May 21, is taunting the people who now have it in their hands due to limitations with the Oculus mobile app.
Virtual Reality

Getting into VR is spendy. Which headset is truly worth your hard-earned cash?

Virtual reality has finally gone mainstream, but how do you find the best VR headset for you? Check out a few of our favorites, whether you want the best of the best or a budget alternative for your mobile device.
Gaming

Minecraft AR teased during Microsoft Build 2019, full reveal arrives on May 17

At the Vision Keynote during the first day of Microsoft Build 2019, we saw a short trailer of what looks to be Minecraft AR, an upcoming experience coming to mobile devices. Watch the video here and join us for the full reveal on May 17.
Photography

As AR heads to Google search, Lens learns to translate, add tips, and more

Google Lens can soon help you decide what to eat by scanning the menu, then calculate the tip with one look at the receipt. Translation options are also expanding for Lens, while augmented reality tools will be built into search results.
Mobile

Google’s ARCore is getting better at tracking moving images

Google has announced a series of updates to ARCore that should make the augmented reality experience a whole lot more fluid. For example, ARCore is getting better at tracking moving images through Augmented Images.
Mobile

Nike Fit aims to help you slip into your new sneakers more easily

Nike says a new augmented reality feature for its app should increase the chances of a snug fit for customers buying its shoes online, or even in-store. Nike Fit launches in the U.S. in July and the following month in Europe.
Computing

Dive headfirst into the best VR experiences available now on the Oculus Rift

The Oculus Rift brought back virtual reality and put a modern twist on it. Grab your Touch Controllers, put on your VR headset, and jump into the fun with some of the best Oculus Rift games available now.
Virtual Reality

Your first augmented reality work helmet might be made by Lenovo

A new augmented reality headset from Lenovo could challenge Microsoft's dominance in the commercial AR industry, with the ThinkReality A6 offering an AR experience akin to something between the first and second-generation Hololens headsets.