Give YouTube a round of [APPLAUSE] — sound effects are now auto captioned

youtube tv live streaming service launch 28691186 ml
YouTube deserves a round of applause — because the video platform can now include [APPLAUSE] and other sound effects in a video’s closed captions automatically. The caption expansion, announced on Thursday, March 23, is made possible by deep neural networks, a form of artificial intelligence.

For now, YouTube can only automatically label applause, music, and laughter, but those three sound effects were the descriptions content creators added manually over any other closed captioning noise. The latest feature builds on the automatic captioning feature launched in 2009 for text, but adds the first sound effects to the system.

YouTube says the program works similarly to detecting objects in images, but faced a few more difficulties over object recognition. To get the program to recognize just those three sounds, YouTube engineers had to teach the program to detect those sounds, separate them temporarily and then insert that recognized sound into the captions.

The system also tended to struggle with sound effects that occurred at the same time as other sounds, like laughter and talking. Another challenge was to find a large enough data set to train the system that wasn’t already adequately labeled by manually inputting the data.

The deep learning network analyzes short segments in sequence, and is able to predict the likelihood of those sounds effects at a rate of about 100 frames per second. YouTube engineers, however, built the system in a way that will allow additional sound effects to be added to the system later.

So why applause, music, and laughter? Besides just being the most frequently manually adjusted labels in the close captioning system, each of those sounds also only has one meaning. A “ring,” YouTube explained, in offering an example, could be a ring from a doorbell, a phone, or an alarm, presenting a whole new challenge for the software.

According to YouTube, over 15 million videos with automatic captions are viewed every day. In a test of the latest update to the auto captions, two thirds said the sound effect labels enhanced the overall experience.

Emerging Tech

Students who designed transforming smart home will compete in Solar Decathalon

Modular smart homes are all the rage, and now some students from Virginia Tech are putting their money on their FutureHAUS, a modular, solar-powered, transforming smart home they're taking to the Solar Decathlon in Dubai.
Home Theater

The best movies on Netflix in November, from 'Buster Scruggs’ to ‘Dracula’

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, witty humor, or anything else.

How to convert and play FLAC music files on your iPhone or iPad

The high-resolution revolution is upon us, and FLAC files are a popular way to store hi-res sound. But what if you’re an iOS user? Check out our article to find out more about FLAC files, and how to use them on Apple devices.
Home Theater

Still listening on tinny TV speakers? Try one of our favorite soundbars

You no longer have to sacrifice sound for size when selecting home audio equipment. Check out our picks for the best soundbars, whether you're looking for budget options, pure power, smarts, or tons of features.

Hackers sold 120 million private Facebook messages, report says

Up to 120 million private Facebook messages were being sold online by hackers this fall. The breach was first discovered in September and the messages were obtained through unnamed rogue browser extensions. 

Switch up your Reddit routine with these interesting, inspiring, and zany subs

So you've just joined the wonderful world of Reddit and want to explore it. With so many subreddits, however, navigating the "front page of the internet" can be daunting. You're in luck -- we've gathered 23 of the best subreddits to help…
Social Media

Facebook opens pop-up stores at Macy’s, but they’re not selling the Portal

Facebook has opened pop-up stores at multiple Macy's, though they're not selling Facebook's new Portal device. Instead, they're showcasing small businesses and brands that are already popular on Facebook and Instagram.
Social Media

Facebook Messenger will soon let you delete sent messages

A feature coming to Facebook Messenger will let you delete a message for up to 10 minutes after you send it. The company promised the feature months ago and this week said it really is on its way ... "soon."
Social Media

Pinterest brings followed content front and center with full-width Pin format

Want to see Pinterest recommendations, or just Pins from followed users? Now Pinners can choose with a Pinterest Following feed update. The secondary feed eliminates recommendation and is (almost) chronological.
Smart Home

Facebook's Alexa-enabled video-calling devices begin shipping

Facebook's Portal devices are video smart speakers with Alexa voice assistants built in that allow you to make calls. The 15-inch Portal+ model features a pivoting camera that follows you around the room as you speak.
Social Media

Vine fans, your favorite video-looping app is coming back as Byte

Vine fans were left disappointed in 2017 when its owner, Twitter, pulled the plug on the video-looping app. But now one of its co-founders has promised that a new version of the app, called Byte, is coming soon.

Social media use increases depression and anxiety, experiment shows

A study has shown for the first time a causal link between social media use and lower rates of well-being. Students who limited their social media usage to 30 minutes a day showed significant decreases in anxiety and fear of missing out.
Social Media

Twitter boss hints that an edit button for tweets may finally be on its way

Twitter has been talking for years about launching an edit button for tweets, but it still hasn't landed. This week, company boss Jack Dorsey addressed the matter again, describing a quick-edit button as "achievable."
Social Media

‘Superwoman’ YouTuber Lilly Singh taking a break for her mental health

Claiming to be "mentally, physically, emotionally, and spiritually exhausted," popular YouTuber Lilly Singh has told her millions of fans she's taking a break from making videos in order to recuperate.