The holy grail of robotics: Inside the quest to build a mechanical human hand

Somewhere in North London, in an anonymous office block on a nondescript road in the U.K.’s capital city, a robot hand is tapping out a message on a keyboard. “Hello, World” it writes, a geeky reference to the basic computing program often used to introduce novice coders to a programming language. The hand doing the typing is the Shadow Robot Company’s “Dexterous Hand,” considered to be one of (if not the) best robotic hand ever created.

Well over a decade in the making, the Dexterous Hand boasts 20 motor-driven degrees of freedom and another four under-actuated movements. In total, this adds up to 24 joints. Each of these joints possesses a movement range that is identical, or at least very, very close, to an actual human hand. It can open or close its grip in just half a second. It can twiddle objects between its fingers. And thanks to 129 built-in sensors, it can carry out tactile sensing of its environment.

Shadow may not have the mainstream recognition of, say, iRobot (makers of the 20+ million selling Roomba vacuums) or Boston Dynamics (the company behind the viral Atlas and Spot robots), but it’s known in the circles that truly matter.

Its hands have been sought out and purchased by groups ranging from NASA to Carnegie Mellon University to OpenAI: the artificial intelligence research company that was previously backed by Elon Musk.

Its latest trick makes it even more tantalizing. While the Dexterous Hand can be programmed to perform tasks, it can also be used to replicate human movement. When it types words on a keyboard, it’s actually copying, movement for movement, a flesh-and-blood engineer kitted out with a HaptX motion-tracking glove. As the engineer types, each tiny press of the keyboard they make is detected through their glove’s tactile fingertip sensors and sent directly to the Dexterous Hand to recreate. All of this is achieved in real-time — despite the fact that the human typist is based in California, more than 5,000 miles away.

The demo isn’t limited to typing, either. Once that phase of the showcase is out of the way, the Dexterous Hand is able to follow its remote master in playing a game of Jenga, building a pyramid out of plastic cups, or moving chess pieces around on a chess board.

“The robot has touch-sensing fingertips and transmits [that data] to haptic feedback devices on the glove so the human can feel [them],” Rich Walker, the managing director of the Shadow Robot Company, told Digital Trends. “The robot can also measure other forces — like the torque at the joints — and transmit those back for other haptic feedback devices. So we have a variety of senses of what is happening at the robot end that can be provided to the human end. In the other direction, as the human moves, we turn that into commands for the robot to make it move in a similar way.”

Welcome to the next frontier for robotics.

The Holy Grail of robotics

There’s something tantalizingly futuristic and science fiction about robot hands. Picture that scene in James Cameron’s Terminator 2: Judgment Day where Arnold Schwarzenegger’s T-800 Terminator takes a flick knife and, with an unerring level of calm, methodically cuts the flesh away from his muscular, Mr. Universe-winning forearm; revealing the gleaming, albeit bloody, robotic hand beneath.

For real-life roboticists, building the perfect robot hand has long been the Holy Grail. It is the hardware yin to the software yang of creating an artificial mind. Seeking out the ultimate challenge, robotics experts gravitated to recreating what is one of the most complicated and beautiful pieces of natural engineering found in the human body. With 27 bones, five fingers, and two main sets of muscles and tendons, the hand is capable of a seemingly unlimited number of functions. Using our hands and fingers, we can manipulate objects both large and small; grip; express ourselves; eat; inflict violence; sense whether a surface is hot or cold, or rough or smooth, and dozens of other activities. They are our main connection point to the external world — and, as such, an almost essential part of life as most of us know it.

The Shadow Robot Company’s efforts are far from the first attempt made by engineers to create functioning robot hands. The earliest such initiative dates back as far as 1504 when the German knight, Götz von Berlichingen, lost a hand in battle after being hit with a cannonball.

In one of history’s first prostheses, von Berlichingen’s missing hand was replaced with an iron replica, consisting of two hinges at the top of the palm, connected to four hook-like fingers which could be clamped around a sword. (Von Berlichingen later replaced it with a somewhat more complex spring-loaded device, which is still on show at a museum in Berlichingen’s native town of Jagsthausen, Germany.)

Plenty of other efforts were made in the years since, particularly when we reached the modern age of robotics in the twentieth century, following the development of the digital computer. But, as exciting as the work has been, they all ran into varying degrees of the on Berlichingen: namely, that hands are extraordinarily difficult things to replicate. One or two pre-programmed applications can be made to work, but there was little ability to operate beyond those confines.

The journey to build a hand

The Shadow Robot Company wasn’t always focused on hands. Its hobbyist founders started their mission back in the late 1980s with the goal of building a complete human-inspired biped robot, similar to Boston Dynamics’ Atlas robot. They persisted until the last years of the twentieth century when they, too, ran into the hand problem.

“When we came to build these in a robot we discovered that these are amazing pieces of evolutionary design; they’re packing a huge amount of technology into a very small space,” Walker said. “The challenge to us wasn’t just making something the same size as a human hand, but being able to give it all the unique individual movements. Each joint, each knuckle needed to be able to move the same way. Driving that from motors and putting sensors on that tell us what each joint is doing was superlatively challenging. We spent years trying to work it out.”

Eventually, Shadow started to drill down and focus on robot hands as its be-all-and-end-all. Riding the wave of miniaturization and cheap components, which accompanied the rise of the mobile phone industry, its engineers began to realize that they were in the right place at the right time. In other words, it got serious about building hands.

The Dexterous Hand was the ultimate result of this work. It quickly found an audience with the clients already mentioned, along with plenty more. While strictly not a consumer product, for cutting edge research labs with the ambition and budget to match, Shadow’s Dexterous Hand represents perhaps the most accurate robot hand yet developed.

Recently, Shadow partnered with the U.S. startups SynTouch and HaptX to allow its hand to be controlled from anywhere in the world by a user wearing a haptic glove. This month, the company also received an $845,000 cash injection from the British government, intended for further honing of the Dexterous Hand to make it better than ever.

What is it good for? Absolutely everything

As with real hands, it’s a bit fruitless to ask what a robot hand like Shadow’s can be used for. The TL;DR answer: anything you would use real hands for. And probably a bit more, to boot.

Like many A.I. tools, a robot hand lends itself to dull, dirty, and dangerous tasks. Thanks to the Shadow hand’s dexterity, it can be programmed to carry out impressively varied and fine-grain movements. “There’s a fantastic difference between something that can be demonstrated once and something that you can get to happen over and over again,” Walker explained. “Last year, one of our customers — OpenAI — taught a robot hand to rotate an object using its fingers. That sounds simple. If you pick up something on the table and it’s the wrong way round, you can turn it over without thinking anything of that. But getting a robot to do that is one of those things that’s really difficult, if not impossible. And yet, OpenAI was able to get one of our robot hands to do it reliably and repeatedly.”

If a certain task is unable to be automated, however, this is where the promise of telepresence enters the frame. Using the Dexterous Hand and a haptic glove, expert surgeons could one day assist with operations from different states or even in different countries. There are enormous opportunities in risky situations such as nuclear decommissioning, bomb disposal, or even in space exploration. “We can really, truly give you the sense of being in another location,” Walker said. “When you do something, the robot does it too.”

Ironically, one of the companies helping to fund much of this work is the Japanese airline, ANA, All Nippon Airways. “They’ve come to this with a very interesting perspective,” Walker said. “What do airlines do? They move people from place to place. ANA’s vision is that maybe an organization that’s spent 50 years moving people from place to place could also be very good at getting people not to move from place to place, but still giving them the benefit of having moved. Maybe instead of having to travel, people will increasingly just use a robot at the other end. That will save minutes, hours, even days of travel. To me, the need to not sit on an airplane for eight hours is worth quite a lot.”

Long-term, Shadow’s work will be crucial in bringing about the kind of fully-realized robots that its founders dreamed of 30 years ago. By creating a hand virtually indistinguishable from the real thing in terms of functionality, the company will help deliver robots which possess the same flexibility and range of capabilities as a human.

“A.I. researchers have spent getting their A.I. systems to play games in virtual worlds,” Walker said. “Now they’re starting to move that across into the real world. If an A.I. is going to physically interact with objects, it needs a hand. And where do you go for hands? Simple: Back to North London.”

Or, we guess, you could stay home in the States and send a tele-operated robot hand to sign the necessary paperwork!