Skip to main content

Drones are no longer crash-test dummies thanks to MIT’s new VR training platform

Autonomous Drone Racing in FlightGoggles

To better train drones and reduce the risk of damage to itself and its surroundings, MIT engineers developed a training platform called “Flight Goggles” based on virtual reality. This enables a fast-flying drone to train within a virtual environment while speeding through empty physical space. Given the nature of VR, these drones can now safely train for any environment and condition. 

Recommended Videos

Without Flight Goggles, drone training typically includes a large enclosed area with nets to catch “careening” vehicles and physical props including doors and windows. If they crash, that’s an added expense to the project due lost time, repairs, or a complete drone replacement. This type of training is ideal for slow-moving drones designed to scan an environment, not fast-moving models. 

“The moment you want to do high-throughput computing and go fast, even the slightest changes you make to its environment will cause the drone to crash,” says Sertac Karaman, associate professor of aeronautics and astronautics at MIT. “You can’t learn in that environment. If you want to push boundaries on how fast you can go and compute, you need some sort of virtual-reality environment.” 

To develop Flight Goggles, the team began with a “hangar-like gymnasium” lined with motion-capture cameras mounted on the walls to track the drone’s movement through physical space. This data is inserted into an image rendering program that generates a photorealistic virtual environment based on the drone’s position and perspective. The program then sends that combined data back to the drone.  

According to Karaman, the drone’s camera isn’t on, and instead “hallucinates” as it “sees” one environment while speeding through another, processing that visual feed at 90 frames per second. The drone used to test Flight Goggles was based on a 3D-printed nylon and carbon fiber frame, a custom-built circuit board, an embedded “supercomputer,” an inertial measurement unit, and a camera. 

For the initial test, the team created a virtual living room with a window twice the size of the drone. Flying at five miles per hour, the vehicle darted through the virtual window 361 times and “crashed” only three times. All throughout this test, the team tweaked its navigation algorithm so the drone could “learn on the fly” and avoid virtual walls. 

Of course, had the team used props instead of VR in this experiment, three repairs or complete drone replacements would be in order. But with Flight Goggles, the drone could “crash” thousands of times and the training would continue without costly repairs and downtime. 

But you can’t have a VR training session without testing the drone in a real-world scenario. The team built the same window within the facility, and then turned on the drone’s on-board camera. The result: It zipped through the physical window 119 times and crashed/required human intervention six times. 

While that doesn’t sound entirely successful, remember that the fast-flying drone learned to fly in virtual space not to mention zooming through the opening at 5 miles per hour. Karaman believes Flight Goggles could even safely train drones to fly around humans. 

Kevin Parrish
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
WWDC 2025: Apple announces iOS 26, macOS 26, watchOS 26 and more
Major updates to iOS, macOS, watchOS, iPadOS, tvOS and visionOS
WWDC 2025 logo

Apple kicked off WWDC 2025 with its keynote presentation at its annual World Wide Developer's conference, and it was a bumper affair.

We were treated to a raft of updates across all of the firm's software platforms, as we were introduced to iOS 26, iPadOS 26, watchOS 26, macOS 26, tvOS 26 and visionOS 26.

Read more
ChatGPT was down: how the June 10 OpenAI outage unfolded
AI assistant ChatGPT and image creator Sora were down as part of a major OpenAI outage
ChatGPT logo on a phone

The popular AI assistant ChatGPT, and image generator Sora, suffered significant downtime as part of a major OpenAI outage today, June 10.

Downdetector showed reports regarding a ChatGPT outage started shortly before 12am PDT overnight and into June 10. This wasn't the first time we've seen ChatGPT go down, with an outage also occurring back in December 2024.

Read more
For gamers on a budget — this Amazon Basics gaming monitor is on sale for $110
The Amazon Basics 27-inch Full HD gaming monitor on a white background.

If you're still using a basic display with your gaming PC, then you're not maximizing its capabilities. You don't have to spend hundreds of dollars on an upgrade though, as you can get the 27-inch Amazon Basics Full HD gaming monitor for only $110 right now. That's a $40 discount from Amazon on its original price of $150, but we're not sure for how much longer, so we highly recommend proceeding with your purchase as soon as possible on one of the most affordable monitor deals for gamers today.

Why you should buy the 27-inch Amazon Basics Full HD gaming monitor

Read more