NASA combines flight sims with real-world flying in crazy new ‘Fused Reality’ tests

Pilots who are training to fly aircraft, such as the KC-135, go through a rigorous program that includes both flight time in an actual plane- and ground-based simulations. A new technology called Fused Reality uses virtual reality to combine the best of both training methods. It is being tested flight tested at both NASA’s Armstrong Flight Research Center at Edwards, California, and the National Test Pilot School (NTPS) in Mojave, California.

First developed in 2003 by Systems Technology, Inc. (STI), the Fused Reality system does away with the full-motion ground simulator that mimics the motion of a plane. Instead, it replaces the simulation machine with an actual plane. Pilots also wear a virtual reality headset that combines an real-world view from the plane they are flying with simulator graphics from another aircraft or another airfield. This combination allows pilots to virtually practice tasks such as refueling with a tanker, flying in formation or landing on an aircraft carrier — while flying an actual plane.

In the test flights, pilots like Scott Howe fly a Gippsland GA-8 Airvan while wearing a virtual reality headset that displays the flight deck of an aircraft carrier or, in Howe’s case, a KC-135 tanker. “I’m seeing the real world through my camera, so I’m seeing mountains and clouds and the aircraft control panel, but I’m flying formation with a virtual tanker,” Howe said. “I was just trying to keep station with that tanker and practice aerial refueling with the [Fused Reality] system.”

After passing the first round of testing, the Fused Reality system will enter the next phase in March 2016. Starting next spring, the technology will be integrated into Air Force Test Pilot School’s Test Management Program, where it will be compared with conventional flight training methods.