Skip to main content

Drones are no longer crash-test dummies thanks to MIT’s new VR training platform

Autonomous Drone Racing in FlightGoggles

To better train drones and reduce the risk of damage to itself and its surroundings, MIT engineers developed a training platform called “Flight Goggles” based on virtual reality. This enables a fast-flying drone to train within a virtual environment while speeding through empty physical space. Given the nature of VR, these drones can now safely train for any environment and condition. 

Without Flight Goggles, drone training typically includes a large enclosed area with nets to catch “careening” vehicles and physical props including doors and windows. If they crash, that’s an added expense to the project due lost time, repairs, or a complete drone replacement. This type of training is ideal for slow-moving drones designed to scan an environment, not fast-moving models. 

“The moment you want to do high-throughput computing and go fast, even the slightest changes you make to its environment will cause the drone to crash,” says Sertac Karaman, associate professor of aeronautics and astronautics at MIT. “You can’t learn in that environment. If you want to push boundaries on how fast you can go and compute, you need some sort of virtual-reality environment.” 

To develop Flight Goggles, the team began with a “hangar-like gymnasium” lined with motion-capture cameras mounted on the walls to track the drone’s movement through physical space. This data is inserted into an image rendering program that generates a photorealistic virtual environment based on the drone’s position and perspective. The program then sends that combined data back to the drone.  

According to Karaman, the drone’s camera isn’t on, and instead “hallucinates” as it “sees” one environment while speeding through another, processing that visual feed at 90 frames per second. The drone used to test Flight Goggles was based on a 3D-printed nylon and carbon fiber frame, a custom-built circuit board, an embedded “supercomputer,” an inertial measurement unit, and a camera. 

For the initial test, the team created a virtual living room with a window twice the size of the drone. Flying at five miles per hour, the vehicle darted through the virtual window 361 times and “crashed” only three times. All throughout this test, the team tweaked its navigation algorithm so the drone could “learn on the fly” and avoid virtual walls. 

Of course, had the team used props instead of VR in this experiment, three repairs or complete drone replacements would be in order. But with Flight Goggles, the drone could “crash” thousands of times and the training would continue without costly repairs and downtime. 

But you can’t have a VR training session without testing the drone in a real-world scenario. The team built the same window within the facility, and then turned on the drone’s on-board camera. The result: It zipped through the physical window 119 times and crashed/required human intervention six times. 

While that doesn’t sound entirely successful, remember that the fast-flying drone learned to fly in virtual space not to mention zooming through the opening at 5 miles per hour. Karaman believes Flight Goggles could even safely train drones to fly around humans. 

Editors' Recommendations

Kevin Parrish
Former Digital Trends Contributor
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
Crash test dummy gets it in the head in the name of drone safety
crash test dummy drone

Keen to learn more about what happens to a human head when a rogue quadcopter smashes into it at high speed, boffins at Virginia Tech have recently been conducting a number of tests.

Aware that using a real human head for their experiment would likely culminate in a bloody mess requiring several wet cloths and even surgery to clean up, the researchers instead called upon the services of an old favorite for such activities – the crash test dummy.

Read more
MIT researchers develop system that lets you customize and test your own drones
customize and test drones the teams bunnycopter has rotors at different heights credit jason dorfman mit csail

Do you know what’s better than owning your own drone? Designing and owning your own drone.

That’s what a new project created by researchers at the Massachusetts Institute of Technology (MIT) -- within its Computer Science and Artificial Intelligence Laboratory (CSAIL) -- allows users to do. The project operates courtesy of a system which makes it possible to vary the size, shape, and structure of drones, based on your own personal requirements.

Read more
You can now explore the White House in VR thanks to the new 1600 app
1600 app vr white house whitehousear2

You only need a dollar to go to the White House. That is, if you're willing to go in VR.

With naught more than a George Washington and a smartphone, you can experience 1600 Pennsylvania Avenue like never before. And it's all thanks to the White House's newest technological innovation -- this time involving virtual reality.

Read more