Skip to main content
  1. Home
  2. Computing
  3. Features

Want to learn how to perform surgery? You need one HoloLens, stat!

Andy Boxall
By

I wasn’t sure what to do next. A patient’s knee was laid out on the table in front of me, and the needle in my hand was filled with anesthetic. I knew it had to be injected into one of the exposed, fleshy areas of the knee, but I wasn’t sure which part, or how much to administer. I took a clumsy jab in the direction I thought was right. It wasn’t.

Thankfully, rather than a medical catastrophe, and the sound of an incoming lawsuit to alert me of my wrongdoing, I got a big, yellow warning triangle. I was wearing Microsoft HoloLens, the “patient” wasn’t really there, and I was trying Touch Surgery’s innovative platform, which helps surgeons learn and practice an operation without the need for an actual patient.

Recommended Videos

Despite my frivolous use, it’s not a toy. There’s a chance the skilled hands that may one day operate on you, or someone you know, may have been guided ahead of time by it.

Related

Practice makes perfect

It’s estimated there will be a need for 143 million surgical procedures in 2017 alone. Yet millions of those people do not have access to safe surgery, increasing the risks of problems arising from it. Touch Surgery wants to answer the question of how to make surgery, and the people who perform it, better. The answer isn’t that complicated. It’s access to the right knowledge, and an ability to practice ahead of time. Fulfilling that role is the hard part.

While I tried Touch Surgery on HoloLens, its main home is an app that runs on iOS and Android. You can download it and try out a procedure for yourself. In fact, the app is popular with people about to undergo an operation, helping to take the mystery out of the process. There is a huge variety of operations covered by the app, from the knee operation I performed, to emergency pericardiocentesis, and actual brain surgery. The in-app experience is more realistic in a squidgy way, while the HoloLens simulation is far less gut-wrenching; but it’s still able to make you wince.

Andy Boxall/Digital Trends
Andy Boxall/Digital Trends

Can a surgeon really use an app to practice surgery, and is it necessary? Surgeons don’t get to practice much. Contrary to what some may think, cadavers aren’t commonly used, and often the first surgery performed is exactly that — the very first time the doctor is doing the procedure. :

“Surgery is a process, with a series of actions, and those actions are used to navigate through the body to find the problem and fix it,” Touch Surgery CEO and founder Jean Nehme explained to Digital Trends. “The best surgeons have years of experience, telling them what to do, when, and alerting them to any potential problems. The question is, how do you take that expertise, and pass it on to new surgeons? Typically it’s done through description, transfer of knowledge through training sessions, or more recently, videos of surgeries.”

Realistic

Touch Surgery steps in here. By combining intelligence from surgical text books, studies, videos, and direct knowledge from discussions with a surgeon; it’s possible to bring it all together to create the mock procedure seen in the app. But this only comes after each surgery has been storyboarded, rendered, and animated by a team that includes members who’ve previously worked with movie effects experts Industrial Light and Magic. Realism is essential, and the team has had doctors try to move the non-existent patient’s limbs or body around while testing the platform on HoloLens, demonstrating how close to reality the experience is.

It’s estimated there will be a need for 143 million surgical procedures in 2017 alone.

Touch Surgery’s approach can be compared to the aviation industry and how it trains pilots; but unlike pilots, there’s no set amount of training hours required for a surgeon. Provided the people in charge consider them ready, surgeons are simply placed into the pressurised operating room environment with supervision.

“Anything you can do to familiarise yourself with the situation,” Nehme told us, “is enormously beneficial.” Quoting studies and research on Touch Surgery’s effectiveness, he said, “By using Touch Surgery and repeatedly testing yourself, you go from not knowing anything, to knowing what you need to know in five hours.”

Mixed reality in the future

Using augmented, or mixed, reality to do this makes sense. It’s interactive in the way an app isn’t. The needle we were fumbling with wasn’t actually there, but the process prompted us to inject in multiple locations, following a particular route, and with the right method. A qualified surgeon would of course have this knowledge, but to practice in this way could be seen as invaluable. If the training isn’t available though, perhaps in areas far removed from mainstream healthcare resources, the risks are increased. The key to being safer in the operating room, says Nehme, is to, “practice, test, test, and test again.”

Touch Surgery on HoloLens is still a work-in-progress though, and the experience was sometimes shaky. Nehme was cool on its potential. “Hololens is early in testing, and a great hardware platform, but there are limitations such as weight. It is not crucial to the app, but it would be amazing to see the technology improve, as it will create exciting opportunities for Touch Surgery.”

Andy Boxall/Digital Trends
Andy Boxall/Digital Trends

However, the company is working on shifting Touch Surgery procedures to other AR platforms, including its own headset that’s in development. We saw a non-working prototype, which was compact and lightweight in comparison to HoloLens. One day, once the technology improves, surgeons could wear the headset in the operating theatre, and “see” the process in real time, acting as a prompt or guide.

The cost and limited availability of Hololens and other AR headsets makes them inaccessible to many people, and in reality, it’s not what makes Touch Surgery exciting. Going back to those five million people who don’t have access to safe surgery, it’s those doctors who may end up benefiting most from the app. We were told the story of a doctor in Uganda who has found himself in the situation of needing to perform an operation completely unfamiliar to him in a matter of hours. His use of Touch Surgery ahead of time was as important to him, and his patient, as his text books and reference material.

“You can use an app to actually be better in operating rooms,” said Nehme.  “Using it is the equivalent of being in a dark room and turning the lights on.”

Touch Surgery the app is free, and is currently used by surgeons at 17 leading institutions in the United States, including Harvard, NYU, Columbia, Stanford. By the end of the year, the company expects that number to reach 100. For the millions of people getting surgery in 2017, safely or otherwise, it can only be good news.

Editors' Recommendations

Topics
Andy Boxall
Andy Boxall
Senior Mobile Writer
Andy is a Senior Writer at Digital Trends, where he concentrates on mobile technology, a subject he has written about for…
HoloLens virtual touchscreen is the futuristic tech we’ve been waiting for
hololens development edition pre order ship date microsoft preorders 2

Researchers at Microsoft Research have developed a way to give HoloLens users a virtual touchscreen through a system called MRTouch. This allows users of Microsoft's mixed reality headset an additional way to interact while using HoloLens, complementing gesture, voice, and controller inputs, according to a Microsoft Research video demoing the use of the MRTouch prototype.

Although Microsoft Research hasn't announced plans to bring MRTouch to the market or allow third-party developers to make use of the multi-touch interactions at this time, the good news is that it works on an unmodified Microsoft HoloLens headset. All users would need to do is use their fingers and swipe across a flat surface to create a virtual touchscreen. You can do this on a number of surfaces, including walls and tabletops. This virtual touch area could be used to display content, and you can interact with the virtual screen using multi-touch gestures, similar to how you would use a tablet.

Read more
Microsoft’s HoloLens could be gearing up to kick Intel to the curb
microsoft posts tutorial creating terminator hud using hololens

Microsoft's next HoloLens headset is still in the works but according to leaks, it could feature some significant internal changes -- including a new processor from an Intel competitor. According to Windows Central the new headset could sport some new hardware under the hood, as Microsoft contemplates replacing the HoloLens' Intel processor with an ARM processor.

That's interesting for a couple reasons, first because it means the HoloLens 2 -- or whatever it ends up being called -- could be built on Microsoft's Always Connected PC platform, and could feature LTE connectivity. If you're unfamiliar, Microsoft's Always Connected PCs are Qualcomm-powered laptops designed for unparalleled battery life and mobile connectivity. That means the HoloLens' new processor could be more powerful but consume less power overall. Which leads to another important development.

Read more
A commercial version of HoloLens is available to rent in the U.S. and Canada
HoloLens Opinion

Microsoft said on Tuesday, February 13, that customers residing in the United States and Canada can now rent the Commerical Suite version of HoloLens through Abcomrents. The idea is to provide companies with the opportunity to see if HoloLens is the right tool for their needs before sinking $5,000 into each headset. Unfortunately, Abcomrents could not provide a definitive rental price due to a pricing tier that depends on the number of units and the duration of the rental window. 
The Commercial Suite version targets enterprise users. It supports Microsoft's Windows Store for Business, Bitlocker data encryption, support for Azure Active Directory for PIN-based identification, Windows Update for Business, and mobile device management support. There is a special kiosk mode too for showcasing specific HoloLens apps in a demo or on a showroom floor.  
"Anyone in your organization can remotely connect to the corporate network through a virtual private network on a HoloLens," Abcomrents states. "HoloLens can also access Wi-Fi networks that require credentials." 
With each Commercial Suite kit, you get the HoloLens headset, a clicker, a carrying case, a microfiber cloth, a charger, and a Micro USB 2.0 cable. Meanwhile, there is a lot of hardware powering the HoloLens platform itself such as see-through holographic lenses, four sensors that understand your environment, an embedded 2MP camera, four microphones, an ambient light sensor, a 32-bit Intel processor, 2GB of system memory, 64GB of storage, and loads more. It weighs around 1.27 pounds. 
In addition to enabling rentals in the United States and Canada, Microsoft plans to enable this program in additional markets in the coming months. Meanwhile, HoloLens as a product continues to spread across the globe, as Microsoft's AR headset is now available in Singapore and the United Arab Emirates. That said, you can find HoloLens in 41 markets, including Australia, Denmark, Hungary, Romania, Slovakia, Sweden, and more. 
"The best part of my job is seeing what people around the world are doing with mixed reality," says Microsoft's Lorraine Bardeen, general manager of Mixed Reality Experiences. "The innovation and development we see on the platform inspires us to create the software and tools needed to bring the potential of mixed reality to life." 
News of the HoloLens rental and expansion into two new markets comes at the heels of HP's new Windows Mixed Reality Headset for processionals revealed last week. Honeywell Process Solutions also introduced Honeywell Connected Plant Skills Insight Immersive Competency, a cloud-based simulation tool. It uses mixed reality to train personnel regarding critical work activities. Both are just two recent examples of how industries across the globe are adopting mixed reality technology. 

Read more