Perfecting Surgery Together
Augmented Reality (AR) is rapidly growing in popularity in the medical field. Immersive streaming technology now provides the power and speed necessary to visualize complex surgical 3D data on a mobile AR device – providing surgeons access to critical information while keeping the patient in focus.
There is an astronomical amount of pressure on today’s surgeons. Each year, the caseload increases, the complexity of procedures rises, creating an unsustainable workload. “We want to make surgeries better for the patient, but also create a better quality of life for the surgeon by reducing cognitive load,” says Peter Verrillo, Cofounder and CEO of US medical technology company Enhatch. Enhatch created the industry’s first Intelligent Surgery Ecosystem, connecting companies, technologies, and surgeons to streamline clinical workflows and to provide insights from every surgery, patient, and device to elevate patient care.
“Our understanding of Augmented Reality and its various use cases in surgery continues to evolve. While many surgeons use this technology as a training tool, intra-operative AR is gaining significant traction within the medical community,” Peter Verillo adds. Wearing Augmented Reality headsets, surgeons have sterile control over critical information, without leaving the operative field or even taking their eyes off the patient. During surgery, they can interact with unintrusive UI elements to access the patient’s data records, scans, 3D models, or other preoperative notes. “The challenge is to bring this very complex data to the user’s fingertips.”
The Power of Remote Rendering
Surgeons often operate with a limited incision, meaning a very constricted field of view. An interactive 3D model provides a valuable perspective. Enhatch’s software platform automatically converts preoperative images such as X-rays, CT scans, and MRIs into accurate 3D models. For one surgery, this typically means processing several gigabytes of data – overstraining the performance capacities of a Microsoft HoloLens 2, or other mobile AR devices. “To make it work would require us to reduce polygons and over-simplify the model. It would take more processing work and it might even produce a less accurate result to visualize,” states Peter Verrillo. A definite no-go for an Augmented Reality application aimed at active surgery.
He adds that “by streaming, we can make it easy for the customer to access this valuable data.” Enhatch has integrated the XR streaming component Hololight Stream from Hololight into its solution. The Hololight Stream is a developer tool enabling remote rendering and real-time streaming of entire AR and VR apps from cloud or on-premises infrastructures to a HoloLens 2, Meta Quest 2, tablets, or smartphones. Thus, the XR streaming solution allows the visualization and manipulation of 3D content that greatly exceeds the performance specifications of any mobile device. “The power of streaming is incredibly important to us, it literally provides the image the surgeon needs,” affirms Peter Verrillo.
The Future of Healthcare
“With Hololight Stream, we save time and money processing data by just avoiding converting volumes of data from one syntax to another,” he explains further. “We can simply keep the high-resolution data from the iPad or computer and stream it to the AR headset.” As for the required setup, the Enhatch CEO points to a stable Wi-Fi connection between the server and client application. For the server, the company is using a laptop equipped with a powerful GPU like Nvidia’s RTX 3080 series. Furthermore, Enhatch is exploring the cloud option and Hololight’s XR streaming platform Hololight Hub to facilitate distribution and provide customers global availability and easy access on all XR devices. Hololight Hub is designed for mass adoption, providing an ecosystem for XR apps from all segments and industries. Enterprises benefit from a centralized platform empowering industrial AR/VR use cases that were not scalable or even possible before.
“The beauty of AR is merging the virtual and real-world to have more successful clinical outcomes,” he says. XR streaming technology will enable surgeons to analyze complex 3D medical data, without shifting focus away from the patient. At a later stage, surgeons should also be able to access the live stream of the AR headset to allow remote support during surgery. “We are taking the right steps towards faster, more personalized surgery for the patient. The use of Augmented Reality within the operating room is the future of healthcare,” Peter Verillo concludes.
More information: