Amazing new video tweaking program lets users edit a person's lip movements in real time

Thanks to rampant photo editing and fake images circulating the web these days, photographic evidence isn’t necessarily valid at face value anymore. And now, thanks to a system that will let anyone dub a video to change the very words a recorded person says, the same suspicion also holds true for videos. Face2Face, as it’s called, is a new software program that makes convincing edits to the moving lips of a person who is speaking. That way, existing videos will suddenly look like the person really is saying the audio dubbed (and incorrect) words. Real and fake will be nearly indistinguishable, on the surface.

Face2Face is still a work in progress, with a team of researchers from Stanford, the Max Planck Institute, and the University of Erlangen-Nuremberg constantly improving the program. The working prototype allows anyone to use a standard RGB webcam to capture video footage of their face, either saying words or making facial expressions. Pairing that footage with video footage of a celebrity, president, or public figure in the Face2Face program will make it look like the person is saying pretty much anything you want them to.

Matthias Niessner

The system pairs two monocular video streams (captured with a single webcam or video source) to make the finished video dub. Face2Face tracks facial expressions of the source video and the target video in order to warp the mouth movement in the final video into something believable. The system then combines the two feeds using a “dense photometric consistency measure,” and re-renders the source expression onto the target face in the original video setting. It’s scary, but even from an unfinished prototype of the system, the Face2Face edited video is pretty convincing.

The challenge to making convincing video may have a lot to do with the target star’s popularity level – if someone we’re used to seeing on TV all the time suddenly makes facial expressions that are new to the repertoire, viewers’ brains will raise some red flags automatically. Nonetheless, some savvy video hackers could theoretically make very believable video and audio dubs of perfectly legitimate speeches or broadcast appearances. Some people are afraid that this kind of system poses a threat to technology by making it inherently less reliable or believable, but the truth is that hoaxes in every medium have been around for ages.

There’s no telling what this software might eventually be used for — but we’re definitely looking forward to all the hilarious speech remixes that will undoubtedly hit YouTube once this software goes mainstream.

Emerging Tech

‘Tech vest’ prevents Amazon workers from colliding with robot co-workers

Amazon workers at its fulfillment centers are using "tech vests" to help protect them from collisions with their robot co-workers. The robots already have obstacle avoidance sensors, but the belt offers another layer of safety.
Wearables

10 top features you should be using on your Apple Watch

The Apple Watch can do more than just tell you the time, but you may not be aware of all the different functions it has. Our list of the 10 most often used functions and features will help you understand what it can really do.
Movies & TV

The best movies on Amazon Prime right now (January 2019)

Prime Video provides subscribers with access to a host of fantastic films, but sorting through the catalog can be an undertaking. Luckily, we've done the work for you. Here are the best movies on Amazon Prime Video right now.
Photography

Here are 8 GoPro tips to get the most out of your action cam

There's more to your GoPro camera than just mounting it to your skateboard. Whether it's finding the best accessories or understanding the settings more thoroughly, learn to shoot video like a pro with these simple GoPro tips and tricks.
Social Media

Spice up your Instagram videos by adding your top tunes to the soundtrack

Have you ever taken a beautiful video, only to have it ruined by some jerk in the background yelling curse words? Here's a list of apps you can use to add your own music to Instagram posts as well as your Story.
Emerging Tech

3D printers are finally affordable. Here are the best models under $500

3D printer prices have dropped dramatically over the past few years, but just because something is cheap doesn’t mean it’s worth buying. Here, we’ve rounded up all the cheap 3D printers that are actually worth spending your money on.
Mobile

T-Mobile 5G rollout: Here is everything you need to know

2019 will be a huge year for T-Mobile. Not only is a merger with Sprint likely, but T-Mobile is also in the midst of building out its next-generation mobile service. Here's everything you need to know about the T-Mobile 5G rollout.
Emerging Tech

ANYmal dog robot can get back on its feet when someone pushes it over

Roboticists at ETH Zurich have demonstrated how their ANYmal four-legged robot is capable of taking a kicking and keeping on walking -- or getting back to its feet if it's pushed over.
Emerging Tech

A.I. finds non-infringing ways to copy drugs pharma spends billions developing

Researchers have demonstrated an artificial intelligence which can find new methods for producing existing pharmaceuticals in a way that doesn’t infringe on existing patents. Here's how.
Emerging Tech

Coinstar machines will let you swap cash for Bitcoin at your local grocery store

Coinstar, the company which owns the coin exchange machines found at grocery stores and elsewhere, will soon let you easily buy Bitcoin with your cash money. Here's how it will work.
Emerging Tech

Facebook hasn’t given up on the idea of building an internet drone

Facebook's efforts to provide internet connectivity from the skies using solar-powered drones suffered a blow last year when the company abandoned its "Aquila" drone project. But the company clearly hasn't given up on the idea.
Emerging Tech

World’s biggest fleet of campus delivery robots now transporting student meals

The world’s largest fleet of delivery robots on a university campus is coming to Fairfax County, Virginia’s George Mason University. Here's how the ordering and delivery process plays out.
Deals

Smart luggage does it all with wireless charger, built-in scale, GPS tracking

The SkyValet smart luggage, currently being funded on Kickstarter, offers solutions to many common travel struggles. With SkyValet, you no longer need separate portable chargers, a scale to weigh your bag, a lock, or a tracking device. It's…
Emerging Tech

The CRISPR baby saga continues as China confirms second gene-edited pregnancy

China’s official Xinhua news agency has confirmed that a second woman has become pregnant as part of a controversial experiment to create the world’s first genetically edited babies.