Skip to main content

Google Glass meets Kinect in ARI, a gesture-recognition app for smartglasses

Google Glass meets Kinect in ARI
Image used with permission by copyright holder

Google Glass took smartglasses from the realm of pony-tailed academics and Silicon Valley dreamers to mainstream reality. It has dozens of apps. It has new designs from Oakley and Ray-Ban. It has fans that include both runway models and basketball players.

And if you’re in a loud room, it’s almost as worthless as it is geeky. Because talking to your gadgets is nowhere near as smooth at David Hasselhoff made it look in Knight Rider.

Sure, you can bark “Glass, take a photo” in the comfort of your living room and snap Mr. Meowsers without setting down your tea. But trying the same thing in a bustling bar is a surefire way to annoy the people around you — and perhaps get a beatdown. Meanwhile, Glass’s touch-sensitive pad handles only a few basic functions.

Quickly evolving smartglass technology could also help put a spring in ARI’s rather lethargic step.

But what if Glass could read hand gestures like Microsoft’s Kinect? Soon, it will. A Portland, Ore., startup called On the Go Platforms is developing a way to control your smartglasses with a form of sign language, and on Wednesday, the company released its first public beta.

“The smartphone is moving up into your vision, and there needs to be a new interface to evolve with the new hardware,” says cofounder Ryan Fink. “That’s where ARI comes in.”

ARI, short for Augmented Reality Interface, requires no buttons, no touchpads, and no speaking. “ARI is the Siri of gesture recognition for smartglasses,” explains Fink.

Hold your fist up in front of your face (like an old-timey boxer looking for a fight) and a box appears superimposed over it in Google Glass. After recognizing it as a command, ARI counts down 3 seconds so you can get your hand out of the way, then snaps a photo. After collecting a few, you can use the wave of your hand to leaf through Glass’s library of photos as if turning invisible pages in a photo album.

That’s just one use. Ultimately, the team hopes ARI will be able to recognize an entire library of gestures, which outside developers can bake into their apps as controls. The Pandora app, for example, could someday interpret a literal thumbs up as your approval for that Whitesnake jam it just played on your 80s station.

A new way to control Google Glass

Besides simply being less irritating than talking to your glasses, Fink and cofounder Gary Peck see it as an essential way to control Glass in scenarios where it would otherwise never work, like a factory floor.

“It’s really loud, they sometimes have stuff in their hands and they want to be able to easily interact with the content or check things off as they go through,” Fink explains. With ARI, a worker could simply wave a hand to move the display in his glasses onto the next instruction in a list, or hold up a fist to mark it done.

The same goes for athletes. A snowboarder with thick gloves might have difficulty swiping the touchpad, and a biker might not want to lift his hands from the handlebars to start a timer.

“I think at first it will be a little weird, especially to people that aren’t familiar with the technology.”

ARI also has the potential to make Glass a platform for games. After all, no one wants to control a game by frantically drumming on their glasses or yelling “Flappy Bird, up!” But as Kinect has proven, gamers don’t seem to mind flailing their arms around.

In theory, ARI solves all these issues. In execution, there are still plenty of kinks to iron out. The early alpha version we tested took a while to recognize gestures, and it takes a purposeful execution to pull off a swipe or fist in just the way ARI is looking to see it.

But quickly evolving smartglass technology could also help put a spring in ARI’s rather lethargic step. Since it constantly analyzes images from the built-in Glass camera, ARI can’t run on your phone; it has to run on the anemic processor built into Glass. As those processors get faster, so will ARI.

Glass’s camera also remains a challenge — because there’s only one. Purpose-built gesture-control systems like Microsoft’s Kinect rely on a pair of cameras to generate stereo images, which lets the computer reading your manic motions determine depth. “With dual cameras, you really get a 3D model of the world in front of you. With a single camera it’s just a plain image,” Peck explains “You have no idea that this hand is a different object from the table, it’s all just different pixels.” Peck has had to work around that primitive input to detect objects based on the look of them alone.

Google Glass meets Kinect in ARI
Image used with permission by copyright holder

The challenge is made even more difficult by the need to preserve Google Glass’s limited battery life: The more accurate the gesture detection, the more it drains the battery. “That’s the tradeoff,” Peck said. “How can you get accuracy that’s good enough for the gestures you’re trying to do?” His code, for instance, uses low-res video from the onboard camera even though it can technically capture at 1080p — because he’s not trying to track every finger.

The geek factor here is also impossible to ignore: How do you get people to use a technology that makes you look like the world’s worst mime? “I think at first it will be a little weird, especially to people that aren’t familiar with the technology,” Fink acknowledges, but insists perceptions will change as the technology improves. “It will be much more like Iron Man or Minority Report, where it’s an immersive experience. So I think the stigmatism toward it being weird and awkward will melt away with that.”

The future of smartglasses and augmented reality remains smudgy.

One of the biggest steps forward will come from “waveguide lenses.” While Glass merely displays a small video feed in the corner of your vision, waveguide lenses can literally overlay information over your entire field of view, like true Terminator glasses. Vuzix has produced early prototypes of this technology and some expensive models meant for industrial environments, but they’ve yet to shrink to the size or price of Google Glass. Peck believes it could be at least another year before we see this level of display reach the mainstream.

The future of smartglasses and augmented reality remains smudgy, but the unknown next steps remain part of the appeal for Peck and Fink.

“It’s a new interaction paradigm. There’s a lot of figuring out — how do you present information in the most intuitive way? How do you interact with it? Peck explains. “But as an app developer, there’s no conventions. There’s no knowledge of the best way to present information that’s up here on your screen. Those are kind of interesting challenges.”

Editors' Recommendations

Nick Mokey
As Digital Trends’ Managing Editor, Nick Mokey oversees an editorial team delivering definitive reviews, enlightening…
Digital Trends’ Top Tech of CES 2023 Awards
Best of CES 2023 Awards Our Top Tech from the Show Feature

Let there be no doubt: CES isn’t just alive in 2023; it’s thriving. Take one glance at the taxi gridlock outside the Las Vegas Convention Center and it’s evident that two quiet COVID years didn’t kill the world’s desire for an overcrowded in-person tech extravaganza -- they just built up a ravenous demand.

From VR to AI, eVTOLs and QD-OLED, the acronyms were flying and fresh technologies populated every corner of the show floor, and even the parking lot. So naturally, we poked, prodded, and tried on everything we could. They weren’t all revolutionary. But they didn’t have to be. We’ve watched enough waves of “game-changing” technologies that never quite arrive to know that sometimes it’s the little tweaks that really count.

Read more
Digital Trends’ Tech For Change CES 2023 Awards
Digital Trends CES 2023 Tech For Change Award Winners Feature

CES is more than just a neon-drenched show-and-tell session for the world’s biggest tech manufacturers. More and more, it’s also a place where companies showcase innovations that could truly make the world a better place — and at CES 2023, this type of tech was on full display. We saw everything from accessibility-minded PS5 controllers to pedal-powered smart desks. But of all the amazing innovations on display this year, these three impressed us the most:

Samsung's Relumino Mode
Across the globe, roughly 300 million people suffer from moderate to severe vision loss, and generally speaking, most TVs don’t take that into account. So in an effort to make television more accessible and enjoyable for those millions of people suffering from impaired vision, Samsung is adding a new picture mode to many of its new TVs.
[CES 2023] Relumino Mode: Innovation for every need | Samsung
Relumino Mode, as it’s called, works by adding a bunch of different visual filters to the picture simultaneously. Outlines of people and objects on screen are highlighted, the contrast and brightness of the overall picture are cranked up, and extra sharpness is applied to everything. The resulting video would likely look strange to people with normal vision, but for folks with low vision, it should look clearer and closer to "normal" than it otherwise would.
Excitingly, since Relumino Mode is ultimately just a clever software trick, this technology could theoretically be pushed out via a software update and installed on millions of existing Samsung TVs -- not just new and recently purchased ones.

Read more
AI turned Breaking Bad into an anime — and it’s terrifying
Split image of Breaking Bad anime characters.

These days, it seems like there's nothing AI programs can't do. Thanks to advancements in artificial intelligence, deepfakes have done digital "face-offs" with Hollywood celebrities in films and TV shows, VFX artists can de-age actors almost instantly, and ChatGPT has learned how to write big-budget screenplays in the blink of an eye. Pretty soon, AI will probably decide who wins at the Oscars.

Within the past year, AI has also been used to generate beautiful works of art in seconds, creating a viral new trend and causing a boon for fan artists everywhere. TikTok user @cyborgism recently broke the internet by posting a clip featuring many AI-generated pictures of Breaking Bad. The theme here is that the characters are depicted as anime characters straight out of the 1980s, and the result is concerning to say the least. Depending on your viewpoint, Breaking Bad AI (my unofficial name for it) shows how technology can either threaten the integrity of original works of art or nurture artistic expression.
What if AI created Breaking Bad as a 1980s anime?
Playing over Metro Boomin's rap remix of the famous "I am the one who knocks" monologue, the video features images of the cast that range from shockingly realistic to full-on exaggerated. The clip currently has over 65,000 likes on TikTok alone, and many other users have shared their thoughts on the art. One user wrote, "Regardless of the repercussions on the entertainment industry, I can't wait for AI to be advanced enough to animate the whole show like this."

Read more