Adobe and Stanford just taught AI to edit videos — with impressive results

Just one minute of video typically takes several hours of editing — but Stanford and Adobe researchers have developed an artificial intelligence (AI) program that partially automates the editing process, while still giving the user creative control over the final result.

The program starts by organizing all of the footage, which is often from multiple takes and camera angles. Those clips are matched to the script, so it’s easy to find several video options for each line of dialogue.

The program then works to recognize exactly what is inside those clips. Using facial recognition alongside emotion recognition and other computational imaging effects, the program determines what is in each frame. For example, the program flags whether the shot is a wide-angle or a close-up and which characters the shot includes.

With everything organized, the video editor then instructs the program in just how the videos should be edited using different styles and techniques the researchers call idioms. For example, a common style is to show the face of the character during their lines. If the editor wants that to happen, he or she just drags that idiom over. The idioms can also be negative. For example, the idiom “avoid jump cuts,” can be added to actually avoid them, or negatively to intentionally add jump cuts whenever possible.

The editor can drag over multiple idioms to instruct the program on an editing style. In a video demonstrating the technology, the researchers created a cinematic edit by using idioms that tell the software to keep the speaker visible while talking, to start with a wide-angle shot, to mix with close-ups and to avoid jump cuts. To edit the video in a completely different, fast-paced style, the researchers instead dragged over idioms for including jump cuts, using fast performance, and keeping the zoom consistent.

Editing styles can be saved to recall later, and with the idioms in place, a stylized video edit is generated with a click. Alternative clips are arranged next to the computer’s edit so editors can quickly adjust if something’s not quite right.

The program speeds up video editing using artificial intelligence, but also allows actual humans to set the creative parameters in order to achieve a certain style. The researchers did acknowledge a few shortcomings of the program. The system is designed for dialogue-based videos, and further work would need to be done for the program to work with other types of shots, such as action shots. The program also couldn’t prevent continuity errors, where the actor’s hands or a prop is in a different location in the next clip.

The study, conducted by Stanford University and Adobe Research, is included in the July issue of the ACM Transactions on Graphics Journal.

Photography

Adobe’s Premiere Rush is a video-editing app designed for social media projects

At Adobe MAX 2018, Adobe unveiled updates across the board for all of its Creative Cloud apps, from the release of Premiere Rush CC, a social-focused video editor, to Project Gemini, a digital drawing and painting tool.
Photography

Adobe is bringing Photoshop to the iPad and, eventually, other tablets

A full version of Photoshop is coming to the iPad -- and soon, other tablets, as well. Adobe also launched several new features for Photoshop and Lightroom, including a new Content-Aware Fill tool.
Photography

Adobe's "creativity conference" begins October 15. Here's what we hope to see

Each year, Adobe uses its Adobe MAX conference to show off its latest apps, technologies, and tools to help simplify and improve the workflow of creatives the world over. Here's what you should expect from this year's conference.
Computing

Spoof Adobe Flash updaters are inserting cryptocurrency mining malware

As part of these attacks, bogus Adobe updaters go on to legitimately update Flash Player and throw users to an official website on completion. Unfortunately, they also embed an "XMRig" mining bot in the process.
Photography

Incredible images reveal the eye of a weevil and other microscopic wonders

You can't see it, but that doesn't mean it's not stunning. The winners of the 2018 Nikon Small World competition are a mix of science and art. The winning entry captures the eye of a weevil and the jewelry-like scales surrounding the eye.
Photography

Luminar (re)touches the sky with a simple A.I.-powered slider

Tired of making complex masks to bring out the sky in a photograph? A.I. can do some of that enhancement for you with a new A.I. Sky Enhancer inside Skylum Luminar. The tool uses A.I. to apply the adjustment to only the sky.
Emerging Tech

Awesome Tech You Can’t Buy Yet: DIY smartphones and zip-on bike tires

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Social Media

3D Facebook photos jump out of the newsfeed, no glasses needed

You're not seeing things -- that photo in your Facebook newsfeed is 3D. Launching today, 3D Facebook Photos use the depth maps from dual-lens smartphones to add dimension to an image as you move your phone.
Photography

These point-and-shoot cameras make your smartphone pics look like cave paintings

If your smartphone camera just isn't giving you the results you're looking for, maybe it's time to step up your game. The latest and greatest point-and-shoot cameras offer large sensors, tough bodies, and long lenses - something no phone…
Social Media

Instagram is testing a new way for you to look through your feed

Instagram is constantly tweaking its app to help give its users the best experience possible, so how do you like the sound of tapping — instead of swiping — to look through your feed?
Photography

When you're ready to shoot seriously, these are the best DSLRs you can buy

For many photographers the DSLR is the go-to camera. With large selection of lenses, great low-light performance, and battery endurance, these DSLRs deliver terrific image quality for stills and videos.
Photography

Photography News: Raw edits on iPad and an A.I. research lab for PicsArt

In this week's photography news, Canon launched an iPad app that allows for RAW edits on the go. Popular mobile editing app PicsArt now has a research lab dedicated entirely to A.I.-powered tools.
Emerging Tech

Here’s all the best gear and gadgetry you can snag for $100 or less

A $100 bill can get you further than you might think -- so long as you know where to look. Check out our picks for the best tech under $100, whether you're in the market for headphones or a virtual-reality headset.
News

Kodak’s ‘Digitizing Box’ service saves precious memories stuck on old media

If you've been meaning to convert your old family photos, videotapes, films, and audio recordings to digital but never seem to get around to it, then a new service from Kodak may prompt you to add it to your "to-do" list again.