Among the most lauded essays ever written about the game of tennis is David Foster Wallace’s 2006 article “Roger Federer as Religious Experience.” Originally appearing in the New York Times, the approximately 6,000-word tribute to one of the world’s most supremely talented players reads, as its title makes clear, more like a divine celebration than a piece of sportswriting.
Wallace (and he was certainly not the first writer to do this) gushed about high-level sporting achievements as though they were more than just superb technique; as if they were, somehow, a transcendent portal to godliness. Ordinary mortals like you and I could comprehend what was happening, but only barely. In order to truly appreciate Federer’s athletic feats, we needed a member of the priesthood — a talented youth player like Wallace had been — who could make it intelligible to us.
Why mention Wallace’s almost decade-and-half old essay on a tech site? Because IBM recently unveiled the latest iteration of its impressive A.I. technology — and it’s learned to appreciate tennis on a whole new level. Well, sort of.
“Taking it into the realms of appreciation implies a human component,” Sam Seddon, IBM Sports and Entertainment Sponsorship lead for the U.K., told Digital Trends. “That wouldn’t really be correct. What we’ve created is a system that is incredibly well-versed at understanding tennis from an excitement point of view. That’s ‘excitement’ defined in a way that an A.I. is able to understand it. We humans may appreciate tennis; the A.I. is looking at the same thing in terms of data.”
IBM’s tennis-appreciating A.I. was in full display at Wimbledon, the oldest of the world’s big four tennis tournaments, which concluded last week. IBM has been Wimbledon’s sponsor for the past 30 years. During that time it has performed many technical functions, including setting up the competition’s first website back in the days of screechy dial-up modems. This year, it was using its vaunted A.I. capabilities (IBM is the company behind Watson, the artificial intelligence which once been beat the world’s best human players at the game show Jeopardy) to solve a major problem.
The A.I. is there to understand this and recognize important moments in the match, based on everything from response to gestures.
That problem? That, in a 13-day competition with hundreds of players and games, spread across 18 grass courts, creating highlights packages and sharing them online in a timely manner would usually be next to impossible. You would need an unimaginably large team of human editors to do it and, even then, it would still be a stretch.
“How do you break news and deliver content faster than the global media organizations?” Seddon continued. “When the product you’re needing to present is 18 tennis matches happening at the same time, that’s a huge amount of content.”
What makes a good highlight?
This is where IBM’s A.I. system comes into play. Trained on thousands of previous tennis matches at IBM’s disposal, and getting better all the time, it watches each match and singles out the most exciting bits. As soon as a match is finished, or sometimes even before, these moments can be slickly edited together by a bot and pushed out of the door to a world watching on millions of computers and mobile devices. This all takes place in a room deep in the depths of a building called the IBM Wimbledon Tech Bunker.
At last year’s Wimbledon, 20 million digital video views were racked up; of which IBM’s artificial intelligence — a slightly more primitive model than this year’s version — contributed 14 million. (These tools are also available to Wimbledon players, who can quickly access their performances, made fully searchable, after each game.)
“You’re trying to look at what makes a good highlight,” Seddon said, describing the work that goes into creating the A.I. tool. “It’s the excitement of the players, the excitement of the crowd, the actions happening on the court. All of those things are data points. The A.I. is there to understand this and recognize important moments in the match, based on everything from response to gestures. That lets us rank every point in every match in order of excitement.”
Unless you train the system very effectively, it could confuse … two things because they look similar.
To calculate excitement, IBM’s A.I. listens to everything from the roar of the crowd to the sound of tennis balls on rackets to signify the start of each clip. It also uses a Watson image recognition system to observe the players and try to autonomously single out moments that are significant.
“When it comes to the visuals of an exciting clip, what the A.I. is focusing on are players,” Seddon said. “We’re not looking for facial expressions because they can vary greatly depending on the individual. What we look for instead is physical gesticulations: Fist pumps, arm raises, handshakes and other instances in which the players are being particularly animated. That’s where the visual recognition component comes in.”
This isn’t an easy challenge to solve. As David Foster Wallace pointed out in his Federer essay, there’s something almost beyond human comprehension about the greatest tennis moments. We know more than we can tell — unless we happen to be a great writer well versed in the sport. A.I., it turns out, can struggle even more.
“Consider the difference between walking toward the ball boy and requesting a towel versus pumping your fist because you’re excited about winning a point, while walking toward the ball boy,” Seddon said. “Unless you train the system very effectively, it could confuse those two things because they look similar. One might be a highlight and one might not. Particularly on a hot day, when players are frequently requesting towels, that could be a problem.”
Regardless of which sport you’re into, artificial intelligence tools will likely transform them in the years to come.
There’s also the challenge with crowd bias in favor of local players, and the different volumes you might expect to find between matches involving the biggest name players and young up-and-comers. “There’s a huge topic of conversation right now about bias in A.I.,” Seddon said. “If you’re not training your system in the right way, you can easily introduce bias.”
The future of sport
There’s a chance that tennis isn’t your thing. But, regardless of which sport you’re into, artificial intelligence tools will likely transform them in the years to come. No, we’re not talking about robots competing with human players (although that’s definitely being worked on), but rather using A.I. to help coaches and players improve their skills and fans to get a better viewing experience. IBM’s technology was also on display in the FIFA Women’s World Cup, and will be a part of the upcoming U.S. Open, too.
Heck, as broadcasting becomes more personalized there’s even a chance you could get match footage edited on the fly to suit your own personal preferences. “Organizations can add a whole lot of value to their archive by accessing the metadata and applying video A.I. capabilities on top of it,” Seddon said. “It presents an opportunity for fans to see the highlights that they would like to see. There’s huge value to be found in that regard.”
Who knew Skynet would turn out to be such a big sports fan?
