The holy grail of robotics: Inside the quest to build a mechanical human hand

Shadow Robot Company Robotic Arm
Shadow Robot Company

Somewhere in North London, in an anonymous office block on a nondescript road in the U.K.’s capital city, a robot hand is tapping out a message on a keyboard. “Hello, World” it writes, a geeky reference to the basic computing program often used to introduce novice coders to a programming language. The hand doing the typing is the Shadow Robot Company’s “Dexterous Hand,” considered to be one of (if not the) best robotic hand ever created.

Well over a decade in the making, the Dexterous Hand boasts 20 motor-driven degrees of freedom and another four under-actuated movements. In total, this adds up to 24 joints. Each of these joints possesses a movement range that is identical, or at least very, very close, to an actual human hand. It can open or close its grip in just half a second. It can twiddle objects between its fingers. And thanks to 129 built-in sensors, it can carry out tactile sensing of its environment.

“With 27 bones, five fingers, and two main sets of muscles and tendons, the hand is capable of a seemingly unlimited number of functions.”

Shadow may not have the mainstream recognition of, say, iRobot (makers of the 20+ million selling Roomba vacuums) or Boston Dynamics (the company behind the viral Atlas and Spot robots), but it’s known in the circles that truly matter.

Its hands have been sought out and purchased by groups ranging from NASA to Carnegie Mellon University to OpenAI: the artificial intelligence research company that was previously backed by Elon Musk.

Its latest trick makes it even more tantalizing. While the Dexterous Hand can be programmed to perform tasks, it can also be used to replicate human movement. When it types words on a keyboard, it’s actually copying, movement for movement, a flesh-and-blood engineer kitted out with a HaptX motion-tracking glove. As the engineer types, each tiny press of the keyboard they make is detected through their glove’s tactile fingertip sensors and sent directly to the Dexterous Hand to recreate. All of this is achieved in real-time — despite the fact that the human typist is based in California, more than 5,000 miles away.

The demo isn’t limited to typing, either. Once that phase of the showcase is out of the way, the Dexterous Hand is able to follow its remote master in playing a game of Jenga, building a pyramid out of plastic cups, or moving chess pieces around on a chess board.

Using the Shadow Robot Company's 'Dexterous Hand' to play a game of Jenga.

“The robot has touch-sensing fingertips and transmits [that data] to haptic feedback devices on the glove so the human can feel [them],” Rich Walker, the managing director of the Shadow Robot Company, told Digital Trends. “The robot can also measure other forces — like the torque at the joints — and transmit those back for other haptic feedback devices. So we have a variety of senses of what is happening at the robot end that can be provided to the human end. In the other direction, as the human moves, we turn that into commands for the robot to make it move in a similar way.”

Welcome to the next frontier for robotics.

The Holy Grail of robotics

There’s something tantalizingly futuristic and science fiction about robot hands. Picture that scene in James Cameron’s Terminator 2: Judgment Day where Arnold Schwarzenegger’s T-800 Terminator takes a flick knife and, with an unerring level of calm, methodically cuts the flesh away from his muscular, Mr. Universe-winning forearm; revealing the gleaming, albeit bloody, robotic hand beneath.

For real-life roboticists, building the perfect robot hand has long been the Holy Grail. It is the hardware yin to the software yang of creating an artificial mind. Seeking out the ultimate challenge, robotics experts gravitated to recreating what is one of the most complicated and beautiful pieces of natural engineering found in the human body. With 27 bones, five fingers, and two main sets of muscles and tendons, the hand is capable of a seemingly unlimited number of functions. Using our hands and fingers, we can manipulate objects both large and small; grip; express ourselves; eat; inflict violence; sense whether a surface is hot or cold, or rough or smooth, and dozens of other activities. They are our main connection point to the external world — and, as such, an almost essential part of life as most of us know it.

The Shadow Robot Company’s efforts are far from the first attempt made by engineers to create functioning robot hands. The earliest such initiative dates back as far as 1504 when the German knight, Götz von Berlichingen, lost a hand in battle after being hit with a cannonball.

Shadow’s Dexterous Hand represents perhaps the most accurate robot hand yet developed.

In one of history’s first prostheses, von Berlichingen’s missing hand was replaced with an iron replica, consisting of two hinges at the top of the palm, connected to four hook-like fingers which could be clamped around a sword. (Von Berlichingen later replaced it with a somewhat more complex spring-loaded device, which is still on show at a museum in Berlichingen’s native town of Jagsthausen, Germany.)

Plenty of other efforts were made in the years since, particularly when we reached the modern age of robotics in the twentieth century, following the development of the digital computer. But, as exciting as the work has been, they all ran into varying degrees of the on Berlichingen: namely, that hands are extraordinarily difficult things to replicate. One or two pre-programmed applications can be made to work, but there was little ability to operate beyond those confines.

The journey to build a hand

The Shadow Robot Company wasn’t always focused on hands. Its hobbyist founders started their mission back in the late 1980s with the goal of building a complete human-inspired biped robot, similar to Boston Dynamics’ Atlas robot. They persisted until the last years of the twentieth century when they, too, ran into the hand problem.

Shadow Robot Company Robotic Arm
Shadow Robot Company

“When we came to build these in a robot we discovered that these are amazing pieces of evolutionary design; they’re packing a huge amount of technology into a very small space,” Walker said. “The challenge to us wasn’t just making something the same size as a human hand, but being able to give it all the unique individual movements. Each joint, each knuckle needed to be able to move the same way. Driving that from motors and putting sensors on that tell us what each joint is doing was superlatively challenging. We spent years trying to work it out.”

Eventually, Shadow started to drill down and focus on robot hands as its be-all-and-end-all. Riding the wave of miniaturization and cheap components, which accompanied the rise of the mobile phone industry, its engineers began to realize that they were in the right place at the right time. In other words, it got serious about building hands.

The Dexterous Hand was the ultimate result of this work. It quickly found an audience with the clients already mentioned, along with plenty more. While strictly not a consumer product, for cutting edge research labs with the ambition and budget to match, Shadow’s Dexterous Hand represents perhaps the most accurate robot hand yet developed.

Recently, Shadow partnered with the U.S. startups SynTouch and HaptX to allow its hand to be controlled from anywhere in the world by a user wearing a haptic glove. This month, the company also received an $845,000 cash injection from the British government, intended for further honing of the Dexterous Hand to make it better than ever.

What is it good for? Absolutely everything

As with real hands, it’s a bit fruitless to ask what a robot hand like Shadow’s can be used for. The TL;DR answer: anything you would use real hands for. And probably a bit more, to boot.

Like many A.I. tools, a robot hand lends itself to dull, dirty, and dangerous tasks. Thanks to the Shadow hand’s dexterity, it can be programmed to carry out impressively varied and fine-grain movements. “There’s a fantastic difference between something that can be demonstrated once and something that you can get to happen over and over again,” Walker explained. “Last year, one of our customers — OpenAI — taught a robot hand to rotate an object using its fingers. That sounds simple. If you pick up something on the table and it’s the wrong way round, you can turn it over without thinking anything of that. But getting a robot to do that is one of those things that’s really difficult, if not impossible. And yet, OpenAI was able to get one of our robot hands to do it reliably and repeatedly.”

If a certain task is unable to be automated, however, this is where the promise of telepresence enters the frame. Using the Dexterous Hand and a haptic glove, expert surgeons could one day assist with operations from different states or even in different countries. There are enormous opportunities in risky situations such as nuclear decommissioning, bomb disposal, or even in space exploration. “We can really, truly give you the sense of being in another location,” Walker said. “When you do something, the robot does it too.”

Ironically, one of the companies helping to fund much of this work is the Japanese airline, ANA, All Nippon Airways. “They’ve come to this with a very interesting perspective,” Walker said. “What do airlines do? They move people from place to place. ANA’s vision is that maybe an organization that’s spent 50 years moving people from place to place could also be very good at getting people not to move from place to place, but still giving them the benefit of having moved. Maybe instead of having to travel, people will increasingly just use a robot at the other end. That will save minutes, hours, even days of travel. To me, the need to not sit on an airplane for eight hours is worth quite a lot.”

Long-term, Shadow’s work will be crucial in bringing about the kind of fully-realized robots that its founders dreamed of 30 years ago. By creating a hand virtually indistinguishable from the real thing in terms of functionality, the company will help deliver robots which possess the same flexibility and range of capabilities as a human.

“A.I. researchers have spent getting their A.I. systems to play games in virtual worlds,” Walker said. “Now they’re starting to move that across into the real world. If an A.I. is going to physically interact with objects, it needs a hand. And where do you go for hands? Simple: Back to North London.”

Or, we guess, you could stay home in the States and send a tele-operated robot hand to sign the necessary paperwork!

Emerging Tech

Wreckage, reefs, and robots: The high-tech quest to find Amelia Earhart’s plane

Over 80 years ago, American aviation pioneer Amelia Earhart disappeared while trying to fly around the world. Now an autonomous water-based robot could help find some answers. Here's how.
Emerging Tech

Autonomous robot deliveries are coming to 100 university campuses in the U.S.

Autonomous robot company Starship Technologies has announced that it will expand its food delivery services to 100 university campuses around the United States over the next 24 months.
Emerging Tech

Astro the dog-inspired quadruped robot can sit, lie down, and… learn?

Move over Spot! Researchers from Florida Atlantic University have built a new dog robot called Astro. Thanks to deep learning technology, it promises to be able to learn just like a real dog.

Have big hands? The Corsair Night Sword RGB is the gaming mouse for you

The Corsair Night Sword RGB is another good looking, comfortable gaming mouse that seems best fitted to those with larger hands and longer fingers, but whatever your hand shape and grip type, it's fun to game with.
Emerging Tech

Mars 2020 rover now has a rotating array of drill bits for sampling Martian rock

Most the key components in the Mars 2020 rover are installed and ready to go. The next phase of construction was to install the bit carousel, an important mechanism for the gathering and sorting of samples from the Martian surface.
Emerging Tech

NASA selects landing site candidates for OSIRIS-Rex to sample asteroid Bennu

Last year, the OSIRIS-REx craft arrived at asteroid Bennu, from which it will collect a sample from the asteroid to be brought back to Earth. Now, the NASA team has selected four potential sites to choose from for the sampling mission.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Racing drones and robotic ping pong trainers

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Emerging Tech

NASA wants to send two more missions to Mars to collect rock samples

With its Mars 2020 mission, NASA hopes to collect samples from the surface of the planet. The challenge is how to get those samples back to Earth. Now, NASA has revealed its plans for two followup missions to Mars.
Emerging Tech

Eric Geusz: Apple engineer by day, spaceship designer by night

An Apple software engineer by day, artist Eric Geusz spends his nights drawing everyday household objects as amazing, science fiction-style spaceships. Check out the impressive results.
Emerging Tech

The black hole at the center of our galaxy is flaring and no one knows why

At the heart of our galaxy lies a supermassive black hole, Sagittarius A*. Normally this giant monster is relatively docile, but recently it's been a hotbed of unexpected activity, rapidly glowing 75 times brighter than normal.
Emerging Tech

SpaceIL’s crashed lander may have sent thousands of tardigrades to the moon

When the SpaceIL craft Beresheet crashed into the moon earlier this year, it left more than just an impact mark. Thousands of micro-animals called tardigrades were along for the ride and may have survived the crash.
Emerging Tech

NASA’s satellite projects will study the sun using solar sailing

Small satellites can be used for all sorts of purposes, and NASA has been searching for ideas to push ahead the capabilities of the hardware. The agency has announced two new projects to demonstrate the potential of small satellites.
Emerging Tech

Hubble captures a beautiful cosmic jellyfish made of glowing gas

A new image from Hubble might look like a deep-space jellyfish, but it's not a sign of extraterrestrial life - in fact, it's a planetary nebula called NGC 2022, located in the constellation of Orion (The Hunter).
Emerging Tech

Parker Solar Probe makes a second orbit of the sun, captures solar wind on video

The Parker Solar Probe, launched last year, has completed its second orbit around the sun. To celebrate, the team responsible for the probe has released a video showing solar winds in action.