Skip to main content

Drones are no longer crash-test dummies thanks to MIT’s new VR training platform

Autonomous Drone Racing in FlightGoggles

To better train drones and reduce the risk of damage to itself and its surroundings, MIT engineers developed a training platform called “Flight Goggles” based on virtual reality. This enables a fast-flying drone to train within a virtual environment while speeding through empty physical space. Given the nature of VR, these drones can now safely train for any environment and condition. 

Without Flight Goggles, drone training typically includes a large enclosed area with nets to catch “careening” vehicles and physical props including doors and windows. If they crash, that’s an added expense to the project due lost time, repairs, or a complete drone replacement. This type of training is ideal for slow-moving drones designed to scan an environment, not fast-moving models. 

“The moment you want to do high-throughput computing and go fast, even the slightest changes you make to its environment will cause the drone to crash,” says Sertac Karaman, associate professor of aeronautics and astronautics at MIT. “You can’t learn in that environment. If you want to push boundaries on how fast you can go and compute, you need some sort of virtual-reality environment.” 

To develop Flight Goggles, the team began with a “hangar-like gymnasium” lined with motion-capture cameras mounted on the walls to track the drone’s movement through physical space. This data is inserted into an image rendering program that generates a photorealistic virtual environment based on the drone’s position and perspective. The program then sends that combined data back to the drone.  

According to Karaman, the drone’s camera isn’t on, and instead “hallucinates” as it “sees” one environment while speeding through another, processing that visual feed at 90 frames per second. The drone used to test Flight Goggles was based on a 3D-printed nylon and carbon fiber frame, a custom-built circuit board, an embedded “supercomputer,” an inertial measurement unit, and a camera. 

For the initial test, the team created a virtual living room with a window twice the size of the drone. Flying at five miles per hour, the vehicle darted through the virtual window 361 times and “crashed” only three times. All throughout this test, the team tweaked its navigation algorithm so the drone could “learn on the fly” and avoid virtual walls. 

Of course, had the team used props instead of VR in this experiment, three repairs or complete drone replacements would be in order. But with Flight Goggles, the drone could “crash” thousands of times and the training would continue without costly repairs and downtime. 

But you can’t have a VR training session without testing the drone in a real-world scenario. The team built the same window within the facility, and then turned on the drone’s on-board camera. The result: It zipped through the physical window 119 times and crashed/required human intervention six times. 

While that doesn’t sound entirely successful, remember that the fast-flying drone learned to fly in virtual space not to mention zooming through the opening at 5 miles per hour. Karaman believes Flight Goggles could even safely train drones to fly around humans. 

Editors' Recommendations

Kevin Parrish
Former Digital Trends Contributor
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
How to buy Microsoft Office: all methods, explained
Microsoft Office 365 free trial available across all devices, mobile to desktop.

Microsoft Office is one of the most popular programs on Windows 11, as well as macOS. Office is even available on iPhones, Android devices, and Chromebooks.

Read more
The case for buying the M2 MacBook Air over the M3 model
The screen of the MacBook Air M2.

Apple's MacBook Air M2 recently stood at the top of our list of best laptops, and for good reason. It's incredibly well-built, exuding an elegance that few laptops can match. It's also plenty fast for productivity users, and its GPU is optimized for creators. Its keyboard, touchpad, and display are all top-notch.

Enter the MacBook Air M3, which (hint) took over the MacBook Air's place on that list. The upgraded chipset offers even faster performance, particularly in GPU-intensive apps, and the M3 model supports an additional external display (with the display closed). It's $100 more, but is that uptick in price justified? Let's dig in.
Specs and configurations

Read more
Best Alienware deals: Gaming PCs, laptops, and monitors
Alienware Aurora R15 placed at an angle on a table.

From gaming laptops to gaming monitors and from gaming headsets to gaming keyboards, Alienware has you covered when it comes to all things gaming equipment. It’s a brand many gamers seek out when looking for new gear, and that’s in part because there are often some good Alienware deals to track down. Alienware regularly turns out some of the best gaming laptop deals and the best gaming PC deals, in fact, and that’s certainly the case right now. We’ve tracked down the very best Alienware deals and put them all in one place below. Reading onward you’ll find deals on laptops, monitors, PCs, and headsets, as well as some information on what Alienware equipment may best suit your gaming setup.
Alienware AW920H wireless gaming headset — $110, was $200

A gaming headset is a great way to keep in touch with your friends while you play. The Alienware AW920H gaming headset lets you do so without the burden or wires, as it connects to your gaming setup via Bluetooth. It has a built-in microphone for outgoing communications, as well as Dolby Atmos, Surround Sound, and Stereo Sound options to help immerse you in the game and incoming communications.

Read more