Google Glass meets Kinect in ARI, a gesture-recognition app for smartglasses

Google Glass meets Kinect in ARI

Google Glass took smartglasses from the realm of pony-tailed academics and Silicon Valley dreamers to mainstream reality. It has dozens of apps. It has new designs from Oakley and Ray-Ban. It has fans that include both runway models and basketball players.

And if you’re in a loud room, it’s almost as worthless as it is geeky. Because talking to your gadgets is nowhere near as smooth at David Hasselhoff made it look in Knight Rider.

Sure, you can bark “Glass, take a photo” in the comfort of your living room and snap Mr. Meowsers without setting down your tea. But trying the same thing in a bustling bar is a surefire way to annoy the people around you — and perhaps get a beatdown. Meanwhile, Glass’s touch-sensitive pad handles only a few basic functions.

Quickly evolving smartglass technology could also help put a spring in ARI’s rather lethargic step.

But what if Glass could read hand gestures like Microsoft’s Kinect? Soon, it will. A Portland, Ore., startup called On the Go Platforms is developing a way to control your smartglasses with a form of sign language, and on Wednesday, the company released its first public beta.

“The smartphone is moving up into your vision, and there needs to be a new interface to evolve with the new hardware,” says cofounder Ryan Fink. “That’s where ARI comes in.”

ARI, short for Augmented Reality Interface, requires no buttons, no touchpads, and no speaking. “ARI is the Siri of gesture recognition for smartglasses,” explains Fink.

Hold your fist up in front of your face (like an old-timey boxer looking for a fight) and a box appears superimposed over it in Google Glass. After recognizing it as a command, ARI counts down 3 seconds so you can get your hand out of the way, then snaps a photo. After collecting a few, you can use the wave of your hand to leaf through Glass’s library of photos as if turning invisible pages in a photo album.

That’s just one use. Ultimately, the team hopes ARI will be able to recognize an entire library of gestures, which outside developers can bake into their apps as controls. The Pandora app, for example, could someday interpret a literal thumbs up as your approval for that Whitesnake jam it just played on your 80s station.

A new way to control Google Glass

Besides simply being less irritating than talking to your glasses, Fink and cofounder Gary Peck see it as an essential way to control Glass in scenarios where it would otherwise never work, like a factory floor.

“It’s really loud, they sometimes have stuff in their hands and they want to be able to easily interact with the content or check things off as they go through,” Fink explains. With ARI, a worker could simply wave a hand to move the display in his glasses onto the next instruction in a list, or hold up a fist to mark it done.

The same goes for athletes. A snowboarder with thick gloves might have difficulty swiping the touchpad, and a biker might not want to lift his hands from the handlebars to start a timer.

“I think at first it will be a little weird, especially to people that aren’t familiar with the technology.”

ARI also has the potential to make Glass a platform for games. After all, no one wants to control a game by frantically drumming on their glasses or yelling “Flappy Bird, up!” But as Kinect has proven, gamers don’t seem to mind flailing their arms around.

In theory, ARI solves all these issues. In execution, there are still plenty of kinks to iron out. The early alpha version we tested took a while to recognize gestures, and it takes a purposeful execution to pull off a swipe or fist in just the way ARI is looking to see it.

But quickly evolving smartglass technology could also help put a spring in ARI’s rather lethargic step. Since it constantly analyzes images from the built-in Glass camera, ARI can’t run on your phone; it has to run on the anemic processor built into Glass. As those processors get faster, so will ARI.

Glass’s camera also remains a challenge — because there’s only one. Purpose-built gesture-control systems like Microsoft’s Kinect rely on a pair of cameras to generate stereo images, which lets the computer reading your manic motions determine depth. “With dual cameras, you really get a 3D model of the world in front of you. With a single camera it’s just a plain image,” Peck explains “You have no idea that this hand is a different object from the table, it’s all just different pixels.” Peck has had to work around that primitive input to detect objects based on the look of them alone.

Google Glass meets Kinect in ARI

The challenge is made even more difficult by the need to preserve Google Glass’s limited battery life: The more accurate the gesture detection, the more it drains the battery. “That’s the tradeoff,” Peck said. “How can you get accuracy that’s good enough for the gestures you’re trying to do?” His code, for instance, uses low-res video from the onboard camera even though it can technically capture at 1080p — because he’s not trying to track every finger.

The geek factor here is also impossible to ignore: How do you get people to use a technology that makes you look like the world’s worst mime? “I think at first it will be a little weird, especially to people that aren’t familiar with the technology,” Fink acknowledges, but insists perceptions will change as the technology improves. “It will be much more like Iron Man or Minority Report, where it’s an immersive experience. So I think the stigmatism toward it being weird and awkward will melt away with that.”

The future of smartglasses and augmented reality remains smudgy.

One of the biggest steps forward will come from “waveguide lenses.” While Glass merely displays a small video feed in the corner of your vision, waveguide lenses can literally overlay information over your entire field of view, like true Terminator glasses. Vuzix has produced early prototypes of this technology and some expensive models meant for industrial environments, but they’ve yet to shrink to the size or price of Google Glass. Peck believes it could be at least another year before we see this level of display reach the mainstream.

The future of smartglasses and augmented reality remains smudgy, but the unknown next steps remain part of the appeal for Peck and Fink.

“It’s a new interaction paradigm. There’s a lot of figuring out — how do you present information in the most intuitive way? How do you interact with it? Peck explains. “But as an app developer, there’s no conventions. There’s no knowledge of the best way to present information that’s up here on your screen. Those are kind of interesting challenges.”

Emerging Tech

Computers will soon outsmart us. Does that make an A.I. rebellion inevitable?

At this point, the question isn’t so much “if” AI will ever surpass humans in terms of thinking abilities, it’s “when.” What happens when we reach that point? Charles J. Simon attempts to answer that question in his upcoming…
Product Review

Google’s Pixel 3 is a hair away from pocket-sized perfection

Google’s Pixel 3 smartphone is the best Android phone you can buy. It doesn’t have the best looks or the best hardware, but you’ll be hard pressed to find better software and unique A.I. functionalities.
Product Review

Mediocre battery and a big notch slight Google's otherwise perfect Pixel phone

Google’s Pixel 3 XL has two big flaws: The gigantic notch on the front, and mediocre battery life. That being said, this is the best Android experience you can find in a smartphone today.

Google, Samsung, OnePlus, and Huawei face off in an Android battle royale

The good news is that there are some great options in the Android smartphone market right now. The bad news is that too much choice makes it tough to decide. We compare the Pixel 3 XL, the Galaxy Note 9, the OnePlus 6T, and the Mate 20 Pro.
Product Review

The iPad Pro is the best tablet ever. But don't sell your laptop just yet

Apple has unveiled a big redesign for the iPad Pro, slimming down the bezels, adding Face ID, and the ability to attach and charge the Apple Pencil. All of this comes at a high cost however, as the iPad Pro starts at $799.

What is Android fragmentation, and can Google ever fix it?

Fragmentation on the Android platform has long been criticized as a problem for security, consistency, and app development. We take a look at Google’s attempted fixes and ask if it can do more.

Looking Glass owners will soon be able to get more holograms on Vimeo

We're inching closer to recreating the iconic scene in Star Wars of Princess Leia calling out to Obi-Wan for help. A Brooklyn company has created the Looking Glass, a holographic display that lets you see 3D content without a headset.

The MacBook of 2021 could kiss the keyboard, and Intel, goodbye

Apple announced a new MacBook Air, which brings a fairly basic update to the line. But if you follow the rumors, there's a much bigger sea change happening. Based on patents, rumors, and pure imagination, here's what we think it'll be like.

Infiniti is using Formula One racing to hone its hybrid skills

Infiniti is partnering with the Renault Sport Formula One team, and not just to put sponsor stickers on the race cars. The luxury brand claims its engineers are working with their F1 counterparts to develop future hybrid tech.

Blizzard Co-Founder Allen Adham says ‘we have not forgotten’ core Diablo fans

Blizzard fans are skeptical of 'Diablo Immortal,' but company co-founder Allen Adham is confident players will like it when they play it. He told us that multiple Diablo projects are underway and hinted at end-game content in 'Diablo…

Privacy is becoming obsolete, but not everyone thinks you should fear its demise

As technologies like Alexa and Siri that require more information about us continue to develop, is privacy going to fall at the wayside, or can we take back control of our data to retain our privacy?
Emerging Tech

To make more room for livestock, the Dutch will moove cows to a floating farm

A Dutch company is developing a floating dairy farm, which they hope to use as a proof-of-concept for future agricultural systems. The farm will use automated cleaning and milking robots, while recycling waste into fertilizer.
Emerging Tech

Driverless cars can only take you so far. This is how smarter cities are being built

Companies and municipalities came together at the annual Smart Mobility Summit, to share new technologies for building smarter cities. They also offered insight into the challenge cities face.

As Amazon turns up the volume on streaming, Spotify should shudder

Multiple players are all looking to capitalize on the popularity of streaming, but it has thus far proved nearly impossible to make a profit. Could major tech companies like Amazon be primed for a streaming take-over?