Facebook photos just got smarter with new ‘autotags’ for easier searches

facebook journalism grants login smartphone
Facebook’s auto tagging feature was already impressive — but now the social media network will recognize that the photo you just uploaded is at the Golden Gate Bridge, that you’re wearing a black shirt, and whether or not you are walking, dancing, or perhaps riding a horse.

Today, Joaquin Candela, the network’s director of applied machine learning, shared a blog post detailing how Facebook can now recognize not just the objects in your photographs, but scenes and actions, too, even on posts shared without a single word.

Lumos is the artificially intelligent program that allows the computer to “see” what’s inside the image you just shared, even if you didn’t give it any sort of text description. That machine learning system is behind a number of Facebook’s image-recognition features, from flagging nudity to fighting spam.

Today, that system is getting an upgrade with the ability to recognize actions, not just objects. The network’s automatic alt text, used for describing a photo to the visually impaired, will now recognize 12 different actions, from walking and dancing to actions that can be described by a verb with a noun tacked at the end — like riding a horse or playing an instrument.

While the program stemmed from, and still continues to work toward, giving the visually impaired a better idea of what’s in their newsfeed, the same technology will now allow any Facebook user to search for an image of a particular place, action, or even a particular garment like a black shirt or red dress. While photos with a description may have popped up in the search results previously, the new action-tagging gives the search more accuracy even for photos that were uploaded without any relevant text.

Lumos is a machine learning program, which means that the more photos it labels and the more details the team feeds it, the more accurate those photo labels become. Facebook’s machine learning team taught the program to recognize actions by asking humans to label 130,000 actual Facebook photos. Feeding those descriptions into the machine learning program “taught” the system how to identify actions.

Searching for a particular object or action and then clicking on the “photos” tab in the search results will now display relevant searches from your own newsfeed as well as friends and public images, using the images tagged with the machine learning, not just accompanying captions.

Candela says that the idea will also advance into automatically labeling videos in the future. “While these new developments are noteworthy, we have a long and exciting road ahead and are just scratching the surface of what is possible with a self-serve computer vision platform,” he wrote. “With computer vision models getting pixel perfect and Facebook advancing into video and other immersive formats, Lumos will help unlock new possibilities in a reliable, fast, and scaleable way and pave the road for richer product experiences in the near future.”

Emerging Tech

Google’s radar-sensing tech could make any object smart

Computer scientists have shown how Google’s Soli sensor can be used to make dumb objects smart. Here's why radar-powered computing could finally make the dream of smart homes a reality.
Social Media

Spice up your Instagram videos by adding your top tunes to the soundtrack

Have you ever taken a beautiful video, only to have it ruined by some jerk in the background yelling curse words? Here's a list of apps you can use to add your own music to Instagram posts as well as your Story.
Photography

From DIY to AAA, here's how to take a passport photo in 6 different ways

If you're applying for a passport or renewing one, you need to submit a photo in your official application. There are strict guidelines, but fortunately, it's something you can do at home. Here's how to take a passport photo.
Computing

Our favorite Windows apps will help you get the most out of your new PC

Not sure what apps you should be downloading for your newfangled Windows device? Here are the best Windows apps, whether you need something to speed up your machine or access your Netflix queue. Check out our categories and favorite picks.
Social Media

A quick swipe will soon let you keep bingeing YouTube on mobile devices

The YouTube mobile app has a new, faster way to browse: Swiping. Once the update rolls out, users can swipe to go to the next (or previous) video in the recommended list, even while viewing in full screen.
Photography

Starting your very own vlog? Here are the best cameras to buy

Any camera that shoots video can be used to vlog, but a few models stand out from the crowd thanks to superior image quality, ergonomics, and usability. When it comes to putting your life on YouTube, here are the best cameras for the job.
Social Media

Twitter extends its new timeline feature to Android users

Twitter users with an Android device can now quickly switch between an algorithm-generated timeline and one that shows the most recent tweets first. The new feature landed for iPhone users last month.
Social Media

YouTube to crack down on dangerous stunts like the ‘Bird Box’ challenge

YouTube already bans content showing dangerous activities, but new rules published by the site go into greater detail regarding potentially harmful challenges and pranks, including certain blindfold- or laundry detergent-based stunts.
Social Media

Nearly 75 percent of U.S. users don’t realize Facebook tracks their interests

Did you know Facebook tracks your interests, including political and multicultural affiliations? According to a recent Pew study, 74 percent of adult users in the U.S. have no idea Facebook keeps a running list of your interests.
Mobile

It’s back! Here’s how to switch to Twitter’s reverse chronological feed

Twitter has finally brought back the reverse chronological feed, allowing you to see your feed based on the newest tweets, rather than using Twitter's algorithm that shows what it thinks you want to see. It's easy to switch.
Social Media

Nearly a million Facebook users followed these fake Russian accounts

Facebook purged two separate groups behind more than 500 fake accounts with Russian ties. One group had ties to Russian news agency Sputnik, while the other had behavior similar to the Internet Research Agency's midterm actions.
Social Media

Twitter suffers privacy scare as bug reveals tweets of protected accounts

If you set your Twitter account to private and you have an Android device, you'd better check your settings now. Twitter says it's just fixed a four-year-old bug that flipped the privacy switch to make the account public.
Web

Switch up your Reddit routine with these interesting, inspiring, and zany subs

So you've just joined the wonderful world of Reddit and want to explore it. With so many subreddits, however, navigating the "front page of the internet" can be daunting. Here are some of the best subreddits to get you started.
Social Media

How to download Instagram Stories

Curious about how to save someone's Instagram Story to your phone? Lucky for you, it can be done -- but it does take a few extra steps. Here's what you need to know to save Instagram Stories on both iOS and Android.