Searching for objects and locations inside video footage is getting much easier

google cloud video intelligence api
During the Google Cloud Next Conference in San Francisco, Google revealed a new machine learning application program interface (API) called Cloud Video Intelligence. With this API, developers can create applications capable of detecting objects within video and making them searchable and discoverable. Both nouns and verbs can be applied to those objects, such as “dog” and “run.”

An API is essentially a bridge between a service and an application. In this case, the API connects to the Google Cloud Machine Learning platform for the compute aspect and stores annotated videos on Google Cloud Storage. Thus, due to this “bridge,” an application based on Google’s new API will have access to this specific functionality to provide end-users with a better way of searching through videos.

“You can now search every moment of every video file in your catalog and find every occurrence as well as its significance,” Google states. “It helps you identify key nouns entities of your video, and when they occur within the video. Separate signal from noise, by retrieving relevant information at the video, shot or per frame.”

In a demo, users can search for animals in an MP4 video file lasting just over a minute and a half. The labels generated by Cloud Video Intelligence consist of Animal (99 percent), Wildlife (94 percent), Zoo (91 percent), Terrestrial Animal (54 percent), Nature (51 percent), Tourism (47 percent), and Tourist Destination (43 percent). The sample video focuses on the Los Angeles Zoo presented by Disney’s Zootopia CGI-animated movie.

However, what’s really neat about the new API is how it can detect a scene in a video. In the same clip, Cloud Video Intelligence can detect 48 scene changes and in real time detect and label objects as the scenes change. For instance, in one scene that displays just Nick the fox, the API will generate seven labels. In another scene focusing on the zoo’s sign, the system only generates two labels … again, all in real time.

What Google has done is create a tool that enables users to search through a video catalog just like they would with text documents. According to the company, this will be highly useful for businesses to separate signals that are buried under noise. It can also “detect features of a signal providing only relevant entities at video, shot or frame level.”

“Google has a long history working with the largest media companies in the world, and we help them find value from unstructured data like video,” said Fei-Fei Li, Chief Scientist of Google Cloud AI and Machine Learning. “This API is for large media organizations and consumer technology companies, who want to build their media catalogs or find easy ways to manage crowd-sourced content.”

The new API is now in a private beta and will also be offered to Google’s partners such as Cantemo, which will use the API to connect its video management software to the Google Cloud Machine Learning platform.

Emerging Tech

How emotion-tracking A.I. will change computing as we know it

Affectiva is just one of the startups working to create emotion-tracking A.I. that can work out how you're feeling. Here's why this could change the face of computing as we know it.
Movies & TV

The best movies on Amazon Prime Video right now (April 2019)

Amazon Prime Video provides subscribers with access to a host of fantastic films, but sorting through the catalog can be a major undertaking. Luckily, we've done the work for you. Here are the best movies on Amazon Prime Video right now.
Gaming

Google Stadia is the upcoming Netflix for games that will hurt game creators

Google Stadia promises a world where gaming is more convenient than ever, but it’s a future that will damage the way people make games.
Mobile

The 100 best Android apps turn your phone into a jack-of-all-trades

Choosing which apps to download is tricky, especially given how enormous and cluttered the Google Play Store has become. We rounded up 100 of the best Android apps and divided them neatly, with each suited for a different occasion.
Deals

The Dell G5587 gaming laptop is on sale for one of the lowest prices we’ve seen

Even diehard desktop PC gamers have to admit that gaming laptops have come a long way in recent years, and the beefy Dell G5587 – now on sale from Walmart for $300 off – is a solid sub-$1,000 machine for work and play.
Computing

HP’s new Zbook, EliteBook 800 workstations go 4K with 8th-gen Intel CPUs

HP's new line of workstation laptops includes some seriously durable EliteBook and ZBook designs, with options for high-end, eighth-generation Intel CPUs, 2TB of storage, and discrete AMD Radeon graphics chips.
Product Review

You won't buy Microsoft's Surface Hub 2S, but it could still change your life

The Microsoft Surface Hub 2S wants to change the way you collaborate at work. That’s a lofty goal most devices fail to achieve, but the unique Hub 2S could be an exception. And trust us – you’re going to want it.
Computing

Light up your external GPU with Razer’s new Core X Chroma enclosure

The Razer Core X Chroma external graphics card enclosure is big enough for three-slot graphics cards, with enough space for a 700w PSU and it brings back the RGB lighting of the Core V2 — all for the same price as its predecessor.
Computing

Microsoft accelerates carbon reduction plans in new sustainability push

Microsoft wants to accelerate its sustainability goal of becoming a zero-carbon company. To reach those goals, Microsoft is doubling its self-imposed carbon tax to incentivize business divisions in making sustainable choices.
Computing

MacOS update may include external display support for iPads

Apple's upcoming MacOS is rumored to include a new native external display support feature. Code-named "Sidecar" the new feature is expected to allow MacOS computers to send app windows to external displays like iPads.
Deals

Apple’s MacBook laptop is on sale for just $800 for a limited time

If you have your heart set on a MacOS-powered laptop, B&H has a sale on Apple's MacBook that takes the price down to $800. Only select models are on sale right now, but you can score up to a $600 savings if you act quickly.
Computing

AMD could offer ray tracing with next-gen Navi graphics cards

Navi is the next-generation graphics card line from AMD and it's coming in just a couple of months time. When it does arrive, one of its major features may be ray tracing, which has to date been an Nvidia-exclusive feature.
Features

Exclusive: The Surface Hub 2S will revolutionize work. Here’s how it was made

Exclusive interviews with the designers, futurists, and visionaries behind the Surface Hub 2 paint a dramatic picture of how Microsoft thinks collaboration will change your office.
Computing

Exclusive: Microsoft’s Panos Panay has a trick up his sleeve

Microsoft Chief Product Officer Panos Panay gives us an exclusive peek at the 85-inch Surface Hub 2, and explains how innovation and collaboration will transform your workplace.