Part of the challenge in having a robot take on the work of a human conductor, of course, lies in the inherently human component of the task. Conductors must react to live changes, to emotions, and to a range of other factors that machines seemingly would not be able to anticipate. But for this first performance, it would appear that YuMi did quite an admirable job … at least, by copying the movements of flesh and blood maestro, Andrea Colombini, director of the Lucca Philharmonic Orchestra.
“It was so much fun to perform with YuMi, ABB’s collaborative robot. It showed that a robot could really conduct an orchestra, but only with the excellent work of very talented engineers and a real maestro,” Bocceli said of the experience.
So what exactly went into this work? First and foremost, YuMi took part in rehearsals, and engineers captured Colombini’s arm movements using a technique called “lead-through programming.” In essence, the robot’s arms were guided to mimic its human counterpart’s motions very carefully, and these motions were then recorded. Afterward, these movements were refined in ABB’s RobotSudio software, syncing the movements to the actual music.
“I think … we’re truly making history and writing the future of robotics applications,” said ABB CEO Ulrich Spiesshofer after YuMi’s debut. “YuMi demonstrated how intuitive, how self-learning this machine is – how wonderful our software really is in learning the movement of a conductor, sensing the music, and really conducting an entire team.”
Colombini echoed these sentiments, noting, “Setting up the interaction between the elbow, forearm, and wrist of the robot, making use of its versatility in repeated and demanding attempts to break down the upbeats and downbeats, was very successful,” he said. Indeed, YuMi appeared to achieve what many previously thought unthinkable for a robot — a “fluidity of gesture,” complete with “softness of touch and expressive nuancing.”
Of course, this isn’t to say that robots will be fully replacing conductors anytime soon. But this does bring them one step closer.
