Skip to main content

Hololens can be used to navigate the blind through buildings faster

Researchers at the California Institute of Technology developed an application for Microsoft’s Hololens that can steer visually impaired individuals through a complex building. Rather than deliver raw images to the brain as seen in recent prosthetic attempts, this “non-invasive” method relies on 360-degree sound and real-time room/object mapping to navigate wearers through an unfamiliar multi-story building on their first attempt. 

Typically, Hololens renders interactive virtual objects in your full view of the real world. For example, engineers can construct a 3D model of a building in physical space and examine each side by simply walking around the virtual structure. You can also use the Hololens to shop for furniture online by placing a 3D model of the desired chair or table in your living room to see how it blends in with your current décor before making a purchase. 

The drawback to Hololens, for now at least, is that all virtual objects reside only in the wearer’s view; these “holograms” can’t be seen by anyone else unless they have a device capable of sharing the same experience. In this case, the wearer can’t see anything, so the researchers fell back on the headset’s real-time room and object-mapping capabilities. 

“Our design principle is to give sounds to all relevant objects in the environment,” the paper states. “Each object in the scene can talk to the user with a voice that comes from the object’s location. The voice’s pitch increases as the object gets closer. The user actively selects which objects speak through several modes of control.” 

These modes consist of scan, spotlight, and target. After selecting scan mode using a clicker, each object will call out its name in sequence from left to right via spatial audio, meaning the wearer can get a sense of their real-world placement based on the distance and location of their voice. Spotlight mode forces the object directly in front to speak, and target mode will force an object to repeatedly call out its name. Meanwhile, obstacles and walls will hiss if the wearer moves in too close. 

In one test, researchers created a virtual chair and directed Hololens wearers to approach object using the target mode. Most relied on a two-phase method: Localize the voice by turning in place and then quickly reach the correct destination. After that, researchers put a physical chair in the same location and asked the individuals to find that chair using their typical walking aid. The process took eight times longer and 13 times more distance without the help of Hololens. 

Hololens can be used for long-range guided navigation, too. Researchers created a virtual guide that followed a pre-computed path and called out “follow me” to the wearer. It continuously monitored the wearer’s progression and remained a few feet ahead. If the wearer strayed off course, the virtual guide would stop and wait for him/her to catch up. The test included crossing a building’s main lobby, climbing two flights of stairs, walking around a few corners, and stopping in an office. 

Editors' Recommendations

Kevin Parrish
Former Digital Trends Contributor
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
Innovative ‘guide dog’ device uses lidar to help Blind people navigate

What if you’re a person with low vision, but for whatever reason do not have ready access to a guide dog to help you navigate? A final year Industrial Design and Technology student from the U.K.’s Loughborough University may have come up with a solution that could help the more than 253 million such people worldwide -- only a small fraction of whom have a service dog. What Anthony Camu has developed is a handheld device called Theia which works like a “handheld robotic guide dog -- minus the waggy tail.”

The device, which currently exists only as a work-in-progress prototype, is inspired by driverless car technology. Using bounced laser light system lidar and cameras, the idea is that Theia can sense the world around it and guide people as they walk, helping them avoid accidents and safely reach their destination. Using onboard smarts that allow it to carry out routing, the hope is that a finished version could also keep track of the weather, traffic density, and myriad other factors while being voice controlled by users thanks to A.I.-aided speech recognition. Guidance is given to users by way of haptic feedback.

Read more
VR wearable can simulate temperature changes using odorless chemicals
simulating temperature with smells vr university of chicago

Trigeminal based Temperature Illusions

Virtual reality can transport us to other places, with all the sights and sounds we might need to convince us that we’re in a different environment. When it comes to other senses, however, there remains a whole lot of work to be done. One thing that VR still doesn’t do well is replicate temperature sensations; making the coronavirus lockdown dream of a virtual vacation, complete with the warming sensation of the shine on your face, still a fantasy. Perhaps until now, that is.

Read more
HoloLens 2 will have dark mode, 5G support when it launches globally this fall
hololens 2 dark mode launching new markets build 2020 microsoft hands on feature 768x768

At Build 2020, Microsoft has announced a number of new features for its HoloLens 2 Mixed Reality headset, as well as plans to launch in new markets around the globe. HoloLens 2 will also be getting 5G dongle support, plus a new dark mode and voice commands through a future software update, Microsoft announced on March 19.

According to Microsoft, the rollout of the headset to additional markets builds on the requests and feedback of customers who have been using the headset. It soon will be coming to the Netherlands, Switzerland, Spain, Austria, and 11 other countries.

Read more