Skip to main content

Facebook utilizes Instagram photos and hashtags to create a smarter A.I.

Image-recognition programs are trained using databases of millions of photographs manually tagged in order to teach the computer to spot different objects. But Facebook has an interesting database of images already at its fingertips: Instagram. During the F8 conference, the social media giant shared how the company trained an artificial intelligence image recognition system by using a combination of public Instagram photos and hashtags.

Labeling an image manually to build a database of millions of photos is a time-consuming process, particularly when getting down to the specific details like a species of a bird rather than just labeling “bird.” Facebook researchers instead decided to see if they could make an existing, already labeled set of images work by using publicly shared Instagram images and their accompanying hashtags.

Recommended Videos

The problem, of course, is that hashtags don’t always detail what is in the photo. While some users may hashtag the breed of dog in the photograph, any A.I. system would also have to sift through hashtags like #tbt (Throwback Thursday) or hashtags with multiple meanings. Facebook calls these irrelevant or non-specific hashtags “incoherent label noise.”

To break through the noise, Facebook designed an A.I. to supervise the hashtags — essentially, designing an A.I. to then use that to create another A.I. The research group built a hashtag prediction model and then limited the training program to a specific list of hashtags.

The most accurate image recognition system to come from the experiment used a list of 1,500 hashtags and trained on a billion Instagram photos, ending up with an 85.4 percent accuracy rate — a rating that Facebook says is two percent higher than earlier advanced models. That system was more accurate than the model trained with 17,000 hashtags, which led the team to conclude that narrowing the focus of the training data leads to a more accurate image recognition system.

Facebook plans to continue using a similar idea to create more specific computer vision that is able to recognize types of trees, flowers, and birds. A more accurate image recognition system could be used to boost Facebook’s existing program that reads the content of images to the visually impaired, for example. 

Facebook plans to release the training model embeddings as open source for further expansion.

While the access to Instagram’s large datasets could help create more accurate image recognition in less time, others are raising privacy questions. Facebook said only public Instagram images were used in the research.

Hillary K. Grigonis
Hillary never planned on becoming a photographer—and then she was handed a camera at her first writing job and she's been…
Bluesky finally adds a feature many had been waiting for
A blue sky with clouds.

Bluesky has been making a lot of progress in recent months by simplifying the process to sign up while at the same time rolling out a steady stream of new features.

As part of those continuing efforts, the social media app has just announced that users can now send direct messages (DMs).

Read more
Reddit just achieved something for the first time in its 20-year history
The Reddit logo.

Reddit’s on a roll. The social media platform has just turned a profit for the first time in its 20-year history, and now boasts a record 97.2 million daily active users, marking a year-over-year increase of 47%. A few times during the quarter, the figure topped 100 million, which Reddit CEO and co-founder Steve Huffman said in a letter to shareholders had been a “long-standing milestone” for the site.

The company, which went public in March, announced the news in its third-quarter earnings results on Tuesday.

Read more
Worried about the TikTok ban? This is how it might look on your phone
TikTok splash screen on an Android phone.

The US Supreme Court has decided to uphold a law that would see TikTok banned in the country on January 19. Now, the platform has issued an official statement, confirming that it will indeed shut down unless it gets some emergency relief from the outgoing president.

“Unless the Biden Administration immediately provides a definitive statement to satisfy the most critical service providers assuring non-enforcement, unfortunately TikTok will be forced to go dark on January 19,” said the company soon after the court’s verdict.
So, what does going dark mean?
So, far, there is no official statement on what exactly TikTok means by “going dark.” There is a lot of speculation out there on how exactly the app or website will look once TikTok shutters in the US.

Read more