Like a vice principal in the sky, this A.I. spots fights before they happen

We live in a surveillance society: A U.S. citizen is reportedly captured on CCTV around 75 times per day. And that figure is even higher elsewhere in the world. Your average Brit is likely to be caught on surveillance cameras up to 300 times in the same period.

But a lot of existing CCTV networks still rely on people to operate them. Depending on the circumstances, there might be a human being at the other end, watching multiple camera feeds on a bank of monitors. Alternatively, there may be no-one watching at all, with the footage only ever viewed in the event that it needs to be.

Two cutting edge technologies may shake up surveillance as we know it, however. Researchers from the U.K.’s University of Cambridge and India’s National Institute of Technology and Institute of Science, Bangalore have published a new paper describing a drone-based surveillance system that uses UAVs as flying security cameras to keep an eye (or several) on large gatherings of people.

“Our system is able to identify the violent individuals real-time.”

What’s more, these drones are equipped with deep learning artificial intelligence algorithms that allow them to identify troublemakers in crowds and take the proper precautions.

The “Eye in the Sky” real-time drone surveillance system could be deployed at events like music festivals, marathons or other large gatherings, where it would be utilized to identify violent individuals — based on their aggressive posture — using the latest pattern recognition technology. It then promises to alert the authorities. Welcome to the future of surveillance!

Identifying attackers in real time

“Our system is able to identify the violent individuals real-time,” Amarjot Singh, one of the researchers on the project, told Digital Trends. “This was quite challenging, and we had to develop unique ways to achieve [it]. The problem was that the standard deep learning algorithms require tens of thousands of annotated images to train these systems. This would normally be fine, if one was to assign just one label to an image — for example, assign ‘cat’ to an image of a cat. But in order to design a system which can detect the human pose from aerial images, the system needs to be trained with aerial images annotated with 14 key points on the human body. Since there is no dataset available for this type of application, we ourself annotated the points on the human body, which was extremely time-consuming and expensive.”

The ScatterNet Hybrid Deep Learning neural network analyzing body language via a complex motion-tracking algorithm inside a drone. University of Cambridge/National Institute of Technology/Indian Institute of Science

By breaking the human body down into 14 different points, the system is able to work out which violent action — if any — is being performed with an accuracy of around 85 percent (and all the way up to 94.1 percent) depending on how many people are being surveilled.

The algorithm was trained by analyzing 2,000 annotated images gathered from a group of 25 participants, who were asked to act out violent attacks such as punching, kicking, strangling, shooting, and stabbing. Accuracy levels drop the more people are being monitored and the further away the drone is, although this could be improved in the future. The finished algorithm, called the ScatterNet Hybrid Deep Learning neural network, can learn using a relatively small number of training examples, while robustly extracting the body’s posture with consistent accuracy.

“Once, the system can do well in the test runs, we will be bringing it to market.”

Just as important, it can do it very, very quickly — within just a few frames of video. This is especially important for security applications, since any potential system designed for this purpose needs to be able to alert authorities of an escalating situation before it has erupted into violence.

“The system detects the violent individuals by first extracting and sending the aerial frame recorded by the drone to the Amazon cloud,” Singh continued. “The human detection algorithm detects each human in the image frame. Next, the pose is estimated for each individual. The pose of the individuals involved in the violent activity are jointly analyzed to identify the violent individual.”

But all of this is still quite far away, right? Not necessarily. “We will be flying the drone at the technical festival at NIT Warangal in Andra Pradesh in India, in October of this year,” Singh said. “The second author is from NIT Warangal, and I am an alumnus. The festival is attended by around 3,000 – 4,000 people and is extremely packed.” The pair also hoped to use the drone at at another event in India called “spring spree.”

drones predict violent individuals from sky scatternet hybrid deep learning neural network v2
A figure from the research paper showcasing how the A.I.-equipped drone processes body language by looking at visual cues based on specific points around the body. University of Cambridge/National Institute of Technology/Indian Institute of Science

It won’t be used for alerting the police to violent action at these events, but rather to prove that the technology can accurately predict the outbreak of violence. “Once, the system can do well in the test runs, we will be bringing it to market,” Singh said. “We are also planning to extend this system to monitor the borders of India.”

Possible ethical concerns?

Technology like this will, of course, prompt polarizing responses. Police in the U.S. are already using drones on a regular basis — and this will only ramp up in the years to come — although right now that it done primarily for tasks like assessing crime scenes.

Like predictive policing, A.I.-equipped surveillance drones carry the ominous suggestion of possible “pre-crime.”

The idea of using A.I.-equipped drones for long-term surveillance of crowds runs the risk of verging on the Orwellian. Like predictive policing, it additionally carries the ominous suggestion of possible “pre-crime.” While this particular project focuses on actions carried out, there are examples of smart CCTV cameras with the goal of intervening before an act is carried out.

In the U.K., for instance, an A.I. algorithm is used for spotting potential jumpers on the London Underground train service. This system works by analyzing the behavior of people waiting on the platform, and then looking for those who miss several available trains during that time. This is because such actions have been shown to precede suicide attempts, thereby allowing intervention to be made.

Researcher Amarjot Singh
Researcher Amarjot Singh

It’s one thing to use A.I. more rapidly to stop a scuffle which has broken out; perhaps another to surveil someone on the basis of body language suggesting that they might do something. Such questions will need to be explored as technology such as this becomes mainstream. If it is able to avoid violent acts on the street, we may well consider the tradeoff to be worth it.

Ultimately, concepts such as this make us recall the dream of philosopher Jeremy Bentham’s Panopticon. This was a proposed prison idea in which the presence of a central guard tower makes the prisoners believe that they are being watched at all times. The result, writers like Michel Foucault have suggested, is that prisoners wind up behaving as though they are being watched at all times.

As drones increasingly take to the skies, the existence of tools like this could prompt us to act in a similar way. Is that a police drone or an Amazon delivery flying overhead? You’d better straighten up your posture and loosen your shoulders just in case!

Emerging Tech

How emotion-tracking A.I. will change computing as we know it

Affectiva is just one of the startups working to create emotion-tracking A.I. that can work out how you're feeling. Here's why this could change the face of computing as we know it.
Emerging Tech

Hawaiian botanists’ drone discovers a plant thought to be lost forever

In what may well be a world first, botanists in Hawaii recently used a drone to find a species of plant that scientists believed was extinct. The plant was located on a sheer cliff face nearly 20 years after its last sighting.

These are the must-have games that every Xbox One owner needs

More than four years into its life span, Microsoft's latest console is finally coming into its own. From Cuphead to Halo 5, the best Xbox One games offer something for players of every type.
Movies & TV

The best movies on Netflix in March, from Buster Scruggs to Roma

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, witty humor, or anything else.
Emerging Tech

Mercury’s wobble as it spins reveals that it has an inner solid core

Scientists have long wondered what the inside of Mercury looks like, and they now have strong evidence that the planet has a large and solid metallic core. The data for the new findings was collected by the now-defunct MESSENGER mission.
Emerging Tech

Gravitational forces at heart of Milky Way shaped this star cluster like a comet

Hubble has captured the stunning Messier 62 cluster. The cluster is warped, with a long tail which stretches out to form a shape like a comet. It is thought this distortion is due to Messier 62's proximity to the center of the galaxy.
Emerging Tech

The grid of the future will be powered by … giant subterranean bagpipes?

In order to transition to a more renewable-focused energy system, we need to scale up our grid storage capacity --- and our existing methods aren't going to cut it. Could compressed air be the key?
Emerging Tech

Burgers are just the beginning: Embracing the future of lab-grown everything

You’ve almost certainly heard of the 'farm to fork' movement, but what about 'lab to table'? Welcome to the fast-evolving world of lab-grown meat. Is this the future of food as we know it?
Emerging Tech

Troubleshooting Earth

It’s no secret that humans are killing the planet. Some say it’s actually so bad that we’re hurtling toward a sixth major extinction event -- one which we ourselves are causing. But can technology help us undo the damage we’ve…
Emerging Tech

Inside the Ocean Cleanup’s ambitious plan to rid the ocean of plastic waste

In 2013, Boyan Slat crowdfunded $2.2 million to fund the Ocean Cleanup, a nonprofit organization that builds big, floating trash collectors and sets them out to sea, where they’re designed to autonomously gobble up garbage.
Emerging Tech

Climeworks wants to clean the atmosphere with a fleet of truck-sized vacuums

Using machines that resemble jet engines, Climeworks wants to fight climate change by extracting CO2 from thin air. The gas can then be sold to carbonated drink and agriculture companies, or sequestered underground.
Emerging Tech

How 3D printing has changed the world of prosthetic limbs forever

When he was 13 years old, Christophe Debard had his leg amputated. Here in 2019, Debard's Print My Leg startup helps others to create 3D-printed prostheses. Welcome to a growing revolution!
Emerging Tech

Geoengineering is risky and unproven, but soon it might be necessary

Geoengineering is a field dedicated to purposely changing the world's climate using technology. Call it 'playing god' if you must; here's why its proponents believe it absolutely must happen.
Digital Trends Live

Digital Trends Live: Earth Day, indoor container farming, robot submarines

Today on Digital Trends Live, we discuss how technology intersects with Earth Day, a new Tim Cook biography, indoor container farming, robot spy submarines, A.I. death metal, and more.