The holy grail of robotics: Inside the quest to build a mechanical human hand

Shadow Robot Company Robotic Arm
Shadow Robot Company

Somewhere in North London, in an anonymous office block on a nondescript road in the U.K.’s capital city, a robot hand is tapping out a message on a keyboard. “Hello, World” it writes, a geeky reference to the basic computing program often used to introduce novice coders to a programming language. The hand doing the typing is the Shadow Robot Company’s “Dexterous Hand,” considered to be one of (if not the) best robotic hand ever created.

Well over a decade in the making, the Dexterous Hand boasts 20 motor-driven degrees of freedom and another four under-actuated movements. In total, this adds up to 24 joints. Each of these joints possesses a movement range that is identical, or at least very, very close, to an actual human hand. It can open or close its grip in just half a second. It can twiddle objects between its fingers. And thanks to 129 built-in sensors, it can carry out tactile sensing of its environment.

“With 27 bones, five fingers, and two main sets of muscles and tendons, the hand is capable of a seemingly unlimited number of functions.”

Shadow may not have the mainstream recognition of, say, iRobot (makers of the 20+ million selling Roomba vacuums) or Boston Dynamics (the company behind the viral Atlas and Spot robots), but it’s known in the circles that truly matter.

Its hands have been sought out and purchased by groups ranging from NASA to Carnegie Mellon University to OpenAI: the artificial intelligence research company that was previously backed by Elon Musk.

Its latest trick makes it even more tantalizing. While the Dexterous Hand can be programmed to perform tasks, it can also be used to replicate human movement. When it types words on a keyboard, it’s actually copying, movement for movement, a flesh-and-blood engineer kitted out with a HaptX motion-tracking glove. As the engineer types, each tiny press of the keyboard they make is detected through their glove’s tactile fingertip sensors and sent directly to the Dexterous Hand to recreate. All of this is achieved in real-time — despite the fact that the human typist is based in California, more than 5,000 miles away.

The demo isn’t limited to typing, either. Once that phase of the showcase is out of the way, the Dexterous Hand is able to follow its remote master in playing a game of Jenga, building a pyramid out of plastic cups, or moving chess pieces around on a chess board.

Using the Shadow Robot Company's 'Dexterous Hand' to play a game of Jenga.

“The robot has touch-sensing fingertips and transmits [that data] to haptic feedback devices on the glove so the human can feel [them],” Rich Walker, the managing director of the Shadow Robot Company, told Digital Trends. “The robot can also measure other forces — like the torque at the joints — and transmit those back for other haptic feedback devices. So we have a variety of senses of what is happening at the robot end that can be provided to the human end. In the other direction, as the human moves, we turn that into commands for the robot to make it move in a similar way.”

Welcome to the next frontier for robotics.

The Holy Grail of robotics

There’s something tantalizingly futuristic and science fiction about robot hands. Picture that scene in James Cameron’s Terminator 2: Judgment Day where Arnold Schwarzenegger’s T-800 Terminator takes a flick knife and, with an unerring level of calm, methodically cuts the flesh away from his muscular, Mr. Universe-winning forearm; revealing the gleaming, albeit bloody, robotic hand beneath.

For real-life roboticists, building the perfect robot hand has long been the Holy Grail. It is the hardware yin to the software yang of creating an artificial mind. Seeking out the ultimate challenge, robotics experts gravitated to recreating what is one of the most complicated and beautiful pieces of natural engineering found in the human body. With 27 bones, five fingers, and two main sets of muscles and tendons, the hand is capable of a seemingly unlimited number of functions. Using our hands and fingers, we can manipulate objects both large and small; grip; express ourselves; eat; inflict violence; sense whether a surface is hot or cold, or rough or smooth, and dozens of other activities. They are our main connection point to the external world — and, as such, an almost essential part of life as most of us know it.

The Shadow Robot Company’s efforts are far from the first attempt made by engineers to create functioning robot hands. The earliest such initiative dates back as far as 1504 when the German knight, Götz von Berlichingen, lost a hand in battle after being hit with a cannonball.

Shadow’s Dexterous Hand represents perhaps the most accurate robot hand yet developed.

In one of history’s first prostheses, von Berlichingen’s missing hand was replaced with an iron replica, consisting of two hinges at the top of the palm, connected to four hook-like fingers which could be clamped around a sword. (Von Berlichingen later replaced it with a somewhat more complex spring-loaded device, which is still on show at a museum in Berlichingen’s native town of Jagsthausen, Germany.)

Plenty of other efforts were made in the years since, particularly when we reached the modern age of robotics in the twentieth century, following the development of the digital computer. But, as exciting as the work has been, they all ran into varying degrees of the on Berlichingen: namely, that hands are extraordinarily difficult things to replicate. One or two pre-programmed applications can be made to work, but there was little ability to operate beyond those confines.

The journey to build a hand

The Shadow Robot Company wasn’t always focused on hands. Its hobbyist founders started their mission back in the late 1980s with the goal of building a complete human-inspired biped robot, similar to Boston Dynamics’ Atlas robot. They persisted until the last years of the twentieth century when they, too, ran into the hand problem.

Shadow Robot Company Robotic Arm
Shadow Robot Company

“When we came to build these in a robot we discovered that these are amazing pieces of evolutionary design; they’re packing a huge amount of technology into a very small space,” Walker said. “The challenge to us wasn’t just making something the same size as a human hand, but being able to give it all the unique individual movements. Each joint, each knuckle needed to be able to move the same way. Driving that from motors and putting sensors on that tell us what each joint is doing was superlatively challenging. We spent years trying to work it out.”

Eventually, Shadow started to drill down and focus on robot hands as its be-all-and-end-all. Riding the wave of miniaturization and cheap components, which accompanied the rise of the mobile phone industry, its engineers began to realize that they were in the right place at the right time. In other words, it got serious about building hands.

The Dexterous Hand was the ultimate result of this work. It quickly found an audience with the clients already mentioned, along with plenty more. While strictly not a consumer product, for cutting edge research labs with the ambition and budget to match, Shadow’s Dexterous Hand represents perhaps the most accurate robot hand yet developed.

Recently, Shadow partnered with the U.S. startups SynTouch and HaptX to allow its hand to be controlled from anywhere in the world by a user wearing a haptic glove. This month, the company also received an $845,000 cash injection from the British government, intended for further honing of the Dexterous Hand to make it better than ever.

What is it good for? Absolutely everything

As with real hands, it’s a bit fruitless to ask what a robot hand like Shadow’s can be used for. The TL;DR answer: anything you would use real hands for. And probably a bit more, to boot.

Like many A.I. tools, a robot hand lends itself to dull, dirty, and dangerous tasks. Thanks to the Shadow hand’s dexterity, it can be programmed to carry out impressively varied and fine-grain movements. “There’s a fantastic difference between something that can be demonstrated once and something that you can get to happen over and over again,” Walker explained. “Last year, one of our customers — OpenAI — taught a robot hand to rotate an object using its fingers. That sounds simple. If you pick up something on the table and it’s the wrong way round, you can turn it over without thinking anything of that. But getting a robot to do that is one of those things that’s really difficult, if not impossible. And yet, OpenAI was able to get one of our robot hands to do it reliably and repeatedly.”

If a certain task is unable to be automated, however, this is where the promise of telepresence enters the frame. Using the Dexterous Hand and a haptic glove, expert surgeons could one day assist with operations from different states or even in different countries. There are enormous opportunities in risky situations such as nuclear decommissioning, bomb disposal, or even in space exploration. “We can really, truly give you the sense of being in another location,” Walker said. “When you do something, the robot does it too.”

Ironically, one of the companies helping to fund much of this work is the Japanese airline, ANA, All Nippon Airways. “They’ve come to this with a very interesting perspective,” Walker said. “What do airlines do? They move people from place to place. ANA’s vision is that maybe an organization that’s spent 50 years moving people from place to place could also be very good at getting people not to move from place to place, but still giving them the benefit of having moved. Maybe instead of having to travel, people will increasingly just use a robot at the other end. That will save minutes, hours, even days of travel. To me, the need to not sit on an airplane for eight hours is worth quite a lot.”

Long-term, Shadow’s work will be crucial in bringing about the kind of fully-realized robots that its founders dreamed of 30 years ago. By creating a hand virtually indistinguishable from the real thing in terms of functionality, the company will help deliver robots which possess the same flexibility and range of capabilities as a human.

“A.I. researchers have spent getting their A.I. systems to play games in virtual worlds,” Walker said. “Now they’re starting to move that across into the real world. If an A.I. is going to physically interact with objects, it needs a hand. And where do you go for hands? Simple: Back to North London.”

Or, we guess, you could stay home in the States and send a tele-operated robot hand to sign the necessary paperwork!

Emerging Tech

Swish! Toyota’s basketball bot earns a Guinness record with 2,020 perfect throws

A basketball-shooting robot built by Toyota just picked up a Guinness World Record for successfully sinking 2,020 baskets in a row without ever missing a shot. Check it out in action.

Tips and tricks to mastering the hunt in Monster Hunter World: Iceborne

If you're planning on playing the Monster Hunter World: Iceborne beta on PS4, check out our beginner's guide for a handful of tips and tricks covering mechanics both old and new. Hoarfrost Reach is dangerous, but you got this.
Smart Home

Getting down to the nitty-gritty: The best vacuums you can buy

Our expert reviewers took some vacuum cleaners for a spin to see which worked best. Whether you're looking for a traditional upright, stick, or robot vacuum, check out our picks for the best vacuums on the market.
Emerging Tech

Check out this clever robot dishwasher, designed for busy restaurants

We've already witnessed robot wait staff and robot chefs working at restaurants. Now we have a high-tech robot dishwasher offering a fast, reliable, and environmentally friendly service.
Emerging Tech

SpaceX is on a hiring spree for its Starlink global internet project

After a string of delays, SpaceX's Starlink project was finally launched last month. Now an analysis of data from SpaceX's job listings shows the company is on a hiring tear, advertising for more and more positions for the project.
Emerging Tech

Ready to roll: Mars 2020 rover fitted with wheels ahead of mission next year

The Mars 2020 rover is getting ready for its trip to the red planet next year. The latest step in readying the rover is installing its wheels and suspension system, which engineers at NASA have been doing this month.
Emerging Tech

Want to work in the stars? Here are six future space jobs you could hold

Ever dreamed of leaving Earth to work in the stars? Here's a list of job titles that might sound like science fiction now, but almost certainly won’t a decade or two in the future.
Emerging Tech

You can help search for aliens with an open access release of SETI data

The Breakthrough Initiatives, a program to search for extraterrestrial intelligence, recently analyzed its first three years of radio telescope data. And all of the data collected is being made publicly available in an open data archive.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Illuminated keyboards and a retro gaming console

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Emerging Tech

The U.K.’s biggest (and only) asteroid mining company has designs on our skies

Is the founder and CEO of the U.K.'s Asteroid Mining Corporation going to be among the first people to strike it rich in space, or is he just chasing an ambitious but doomed mirage?
Emerging Tech

Tiny galaxy has huge black hole at its center, gives clues to galactic evolution

A Hubble image shows a tiny galaxy which could hold the clue to unraveling a longstanding question about the evolution of galaxies. Despite its small size, it hosts a feature found in much larger galaxies -- a supermassive black hole.
Emerging Tech

Dark matter galaxy crashed into the Milky Way, causing the ripples in its disk

New research suggests hundreds of million of years ago, the Milky Way collided with Antlia 2, a nearby dwarf galaxy dominated by dark matter. The collision caused ripples in the disk of gas around the Milky Way which we still observe today.
Emerging Tech

Uranus’ rings shine brightly but hold a puzzle for astronomers

New images reveal the rings around Uranus, which are almost invisible to most telescopes. But there's a strange puzzle about them -- why they don't contain any small dust-sized particles.
Emerging Tech

U.S. Navy is working on making its fleet invisible to computerized surveillance

The U.S. Navy’s ever-innovative Office of Naval Research is working on a way to turn the United States military fleet invisible. Well, to cutting-edge image-recognition systems, at least.