Defense giant BAE wants to replace fighter jet controls with augmented reality

AR isn’t just considered transformative for regular users, however. United Kingdom defense giant BAE Systems believes that augmented reality has a valuable role to play in the future of combat. BAE is known for its far-out tech concepts, which have ranged from UAVs that can be “grown” using chemistry in large-scale labs to submarines piloted by virtual reality. In a “wearable cockpit” concept unveiled today, BAE’s department of military air and information showed off its ideas for incorporating AR and virtual reality (VR) technology into combat jet cockpits.

This “mixed reality” tech could be used for both training and combat scenarios, with the use of a Striker II full color Helmet Mounted Display (HMD) for projecting augmented and virtual reality displays and controls in front of the pilot’s eyes — entirely replacing the need for physical cockpit layouts.

“Because the controls and displays are projected onto the helmet display, the technology allows pilots to customize the cockpit display and the way they interact with it based on their own personal preferences and mission objectives,” Suzy Broadbent, human factors lead for cockpits at BAE Systems, told Digital Trends.

“The move away from hard wired controls and displays and into virtual reality is something that has interested us for a while. However, it is only now that the technology is developing enough for us to see it as a definite potential for use in the fast jet domain. The idea is that the pilot can ‘wear the cockpit,’ and the innovations within HMI technologies allows us to provide a variety of methods for the pilot to interact with the system beyond the traditional HOTAS (Hands On Throttle And Stick).”

In addition to this, BAE’s proposed system focuses on “adaptive autonomy,” meaning that it could monitor pilot health and mental workload and, based on its assessments, shift what it is showing to the pilot at any one time. If mental workload becomes too much, the system could allow the aircraft to take over certain tasks the pilot would otherwise manually control.

“This is a technology demonstration concept, but one which is being looked at in research and development terms for potential exploitation on future platforms,” Broadbent continued. “The platforms of the future will be more ‘information heavy’ than ever and, we will therefore need more innovative ways of presenting that information to a pilot to allow them to interact with it.”