Get past the gimmicks and gaze upon the future of augmented reality apps


Augmented reality (AR) has been in science fiction for decades. Thanks to smartphones, the technology is finally here to make it happen, but it’s been nearly two years since we wrote about the untapped potential of augmented reality and it’s still waiting for a killer application. Still, mobile tech keeps moving toward AR. It feels tailor made for wearable technology, particularly smart glasses. New academic research has given us an insight into the exciting future possibilities for AR browsers, but it also highlights the barriers that must be overcome.

“The really compelling applications that are going to make AR take off just haven’t been built yet,” explained Jules White, Assistant Professor of Computer Science at Vanderbilt University. “Instead what we’ve got primarily in the market are gimmicky advertising experiences.”

Every object around you could give you its data

“AR is about showing you information that’s relevant to the physical objects around you” says White.

The app could determine that your tire pressure is low, or recognize that a part is dangerously worn.

Imagine your car breaks down and you can whip out your phone and use a diagnostic app that can guide you in a repair. It could show you where the jumper cables are and highlight where to connect them on the battery. With the right level of visual awareness the app could determine that your tire pressure is low, or recognize that a part is dangerously worn and needs replaced.

Maybe you’re out shopping in a store and you want to buy some light bulbs, but you have no idea which LED bulbs are the best. You take out your smartphone, tell it you want something under $20 at 40 watts with at least a 4 star review, and it highlights the relevant options that are on the shelf, right there on your screen.

Volkswagen is already doing it

The potential for augmented reality is huge, and it’s finally beginning to come to life.

The Volkswagen MARTA app (Mobile Augmented Reality Technical Assistance) is designed to display superimposed information on the vehicle and enable technicians to complete a repair or guide them as they service a vehicle. It only covers one car right now and it’s for professional mechanics, but it makes so much sense that it’s bound to spread. There’s also the Audi AR app, which is like a user manual, enabling users to view the car using their iPhone and get data pop-ups on what they’re looking at.

 “What really takes AR to the next level is when you have the domain expert and computer vision expert working together to build applications that are helpful to specific industries,” White explains.

Professor White served as guest editor of the latest special issue from the Proceedings of the IEEE on Applications of Augmented Reality and he says they’ve had interest from various industries including medical, automotive, engineering, maintenance and repair, construction, and retail.

This could pave a route toward a killer AR app for the general public. Maybe it will come out of retail. As White suggests, “they could start with inventory management and planogram management and then realize it’s wonderful and go ahead and allow customers to tap into the same database of knowledge.”

Augmenting the real world better

Current AR browsers, like Wikitude and Layar, tend to superimpose images and text in flat blocks. The position is based on geo tags and the data that’s offered simply floats on top of the real world. Quality and usability are poor and there’s no standardized platform.

What if you could see images, text, or even video content overlaid on the surface of a building?

What if you could see the images, text, or even video content, and it was actually overlaid on the surface of a building? You could have a virtual vacancy sign on a hotel with review data, or a giant post-it on the 34th floor of a nearby building with details of the meeting you’re due to have there. That’s an idea put forward in “Next-Generation Augmented Reality Browsers: Rich, Seamless, and Adaptive,” one of the papers from the IEEE journal.

This involves mapping environments and spatial awareness, essentially advances in computer vision. The interesting thing about this is that it doesn’t necessarily have to rely on GPS or other sensors in the phone. Computer vision algorithms and rapidly improving camera hardware are actually more accurate than consumer grade GPS, even when it’s assisted by Wi-Fi.

If you have a database of images then a server in the cloud can analyze imagery from your phone and compare it to what’s in the database and potentially determine your position down to the millimeter.

Logistically it’s a difficult proposition, but that didn’t stop Google developing street view. The level of detail you need depends on what you’re trying to do. Building the database could be expensive, and you’d need to cover all the angles and perspectives people are likely to view an environment from if you want fine granularity. Then you have to consider that the real world changes all the time, although White suggests crowdsourcing could solve this issue, “when AR reaches that critical mass and everyone is using it all the time it will be easy because people are taking photos of everything constantly.”

Identifying objects better than Google Goggles

There’s another side to this and that’s the Google Goggles approach, where your AR browser is able to recognize objects and products and offer up useful information about them. We’ve moved beyond the old approach of scanning a single object with something like the barcode scanner app.

As White explains “AR ups the bandwidth considerably, rather than scan one image you can take a picture of the entire aisle” and this could produce a listing of the top 10 products out of 1,000 that you can filter by review score, price, and whether it’s in stock. It’s not just about telling you what things are, it’s about providing “intelligence to help you make a decision.”

What about smart glasses?

There’s no doubt that smart glasses could be a major driver of AR, but are they ready for prime time? The technical capabilities of smart glasses right now would preclude things like real-time video compositing, which could be possible on a smartphone. There are also issues with the interface that voice controls can’t always solve. If you were looking at an engine, for example, you’d want to be able to point to a specific part, especially if you don’t know what it is.

We’re already seeing concepts for how a new kind of interface might work. Samsung recently patented an AR keyboard that would project onto your fingers, and eye tracking is improving all the time.

White suggests “the more immersive the experiences become, the more natural we expect the interface to be.”

Samsung AR keyboard_patent_4

That level of immersion also raises safety concerns. What if an AR display or animation distracts you from a real world event and causes an accident? Then there’s cyber security. What if someone hacks into your AR display and alters information? It’s going to take some time to work these things out.

White acknowledges the part that projects like Google Glass have played in raising the profile of AR, but he feels mobile phones and tablets will drive AR in the short term because “it doesn’t require people to change their behavior … people are used to using their phone cameras to take pictures of things around them that interest them.”

How close are we, really?

The “Next-Generation Augmented Reality Browsers: Rich, Seamless, and Adaptive” paper suggests that for AR to hit the mainstream it needs to be easier to register points of interest. Content needs to move beyond static images/text to video, audio, and even 3D models and animations, and we need better user interfaces to help us interact and extract useful information.

“While more work remains to be done, we believe these challenges can be addressed and resolved within a reasonably near future, hopefully leading to large-scale adoption of AR,” concludes the paper.

We’re sold. If someone with deep enough pockets puts the resources into it, then we could start seeing some truly awesome AR applications. Hopefully two years from now someone will and we can talk about how successful AR has become.

Product Review

4 Lenses, 40 megapixels, and 400k ISO. Huawei’s P30 Pro is a photography freak

Huawei’s P-series phones place an emphasis on the camera. That’s why the company partnered with Leica on the camera tuning. The latest phone is the Huawei P30 Pro, and it must fill the big shoes of last year’s stunning P20 Pro. Is it…

Apple's iOS 12.2 brings support for Apple News Plus and new AirPlay 2 features

After months of betas, the final version of iOS 12 is here to download. The latest OS comes along with tons of new capabilities, from grouped notifications to Siri Shortcuts. Here are all the features you'll find in iOS 12.
Product Review

The new iPad Mini isn’t a beauty, but it performs like a beast

Apple’s new iPad Mini has beastly performance, fluid iOS 12 software, and good battery life. It also looks like it came straight out of 2015, because the design hasn’t been changed. Here's our review of Apple’s 7.9-inch tablet.
Product Review

With the S10e and S10 Plus, do we really need the Samsung Galaxy S10?

The Galaxy S10 is the middle child in this year’s Galaxy S10 range, between the Galaxy S10e, and the Galaxy S10 Plus. There’s no striking reason to buy it, but it’s still an excellent phone you’ll be happy with.

Does the box for AirPods 2 reveal Apple's AirPower wireless charging mat?

At its September event in 2017, Apple unveiled the AirPower, a new wireless charging mat that will allow you to charge multiple devices at one time. It has not yet been released. Here's everything we know about the device so far.

Yes, you can use Android apps on your Chromebook. Here's how

You can now get Android apps on your Chromebook! Google has enabled the Google Play Store app support on its Chrome OS and Chromebook hardware, so to get you started, here's our guide on how to get Android apps on a Chromebook.
Product Review

Compared to the P30 Pro, Huawei may have stripped too much out of the P30 phone

Has Huawei pulled all the best parts out of the P30 Pro to make the P30, its cheaper, smaller sister phone? Yes, but the answer is more complicated than that. Here’s our look at the 6.1-inch P30.

Samsung Galaxy Fold pre-orders will start April 26, with shipment on May 3

The Samsung Galaxy Fold has arrived, and it goes on sale soon. Folding out from a 4.6-inch display to a tablet-sized 7.3-inch display, this unique device has six cameras, two batteries, and special software to help you use multiple apps.

Huawei P30 Pro is a gorgeous-looking smartphone packed with camera ingenuity

Huawei has kicked off the new year by launching the new Huawei P30 Pro and P30. With upgraded hardware, new software tricks, and some absolutely stunning cameras, Huawei's new phones are serious contenders.

Huawei P30 Pro vs. Mate 20 Pro vs. P20 Pro: Which Huawei flagship is best for you?

If you're after a Huawei smartphone, but can't pick between this year's flagship and the previous two, or maybe you're looking to upgrade, we're here to explain what makes the P30 Pro, P20 Pro, and Mate 20 Pro different.

Huawei P30 Pro vs. Huawei P30: Should you go pro or save some dough?

The latest phones from Huawei are seriously powerful, eye-catching devices, but what sets them apart? We take a look at the Huawei P30 Pro and compare it to the Huawei P30 in various categories to uncover the differences.

The 13 best iPhone 7 Plus cases to protect your Apple phone

The iPhone 7 Plus is big and beautiful, but you need to wrap it up if you want to protect it from damage. We’ve rounded up some of the best iPhone 7 Plus cases and covers available. Get some protection or a new style today.

Huawei’s Eyewear smartglasses aim to fuse fashion and tech

At its launch event for the new P30 and P30 Pro smartphones, Huawei surprised us with a pair of smart glasses. Created through a partnership with Gentle Monster, these glasses are designed to be stylish and act like a Bluetooth headset.

Carbon fiber is making Casio’s new connected G-Shock watches even tougher

Casio, a master of tough watches, has introduced carbon fiber to the construction of several G-Shock models, making them stronger and lighter, and allowing for some interesting design changes too.