The future of surgery: Using augmented reality goggles in the operating room

UC Davis Health surgeons are leading the way in using augmented reality goggles, which can vastly improve surgical procedures. 

Augmented reality is an innovative technology that allows you to see how something may look without physically holding it or seeing it. It has the potential to influence how we consume entertainment, communicate, engage with digital content and receive our health care.

It’s related to virtual reality — but extends the possibilities, according to E. Bradley Strong, professor and vice-chair of otolaryngology.

“Virtual reality replaces your real surroundings with a completely virtual environment, while augmented reality incorporates digital imagery within the real environment,” Strong explained. “Extended reality is an umbrella term we like to use to describe them both.”

Using extended reality in surgery

Surgeons within the Departments of Otolaryngology, Neurological Surgery and Orthopedic Surgery have established the UC Davis Medical Extended Reality (MXR) Research Group. It is part of the 3D Printing and Visualization Lab at UC Davis Health. They are the first to use extended reality during surgery within the health system.

The UC Davis MXR research group is collaborating with outside companies, including Xironetic, to be the first in the world to apply augmented reality overlays to a range of complex surgical procedures.

To do this, surgeons wear augmented reality goggles during surgery. They project 3D computed tomography (CT) and MRI scans that overlay critical information directly onto the surgeon's field of view. The goggles help the surgeon:

• Plan the surgical approach
• Visualize hidden vascular and bone structures
• Localize instrumentation and implants
• Project drilling and cutting guides

“The goggles help us transition our virtual surgical plans to the operating room,” Strong said. “It allows us to better view what is in the operative field and visualize objects that otherwise would be very difficult to see in a precise manner.”

Educating patients with extended reality

Surgeons within the UC Davis MXR research group are also using extended reality to better educate patients before surgery. They can provide the patient with an immersive and interactive experience, making the complex medical procedure more understandable.

“Physicians spend years learning how to piece together many 2D images and then visualize them three-dimensionally,” said Strong. “It is impossible to ask patients to do the same thing in 30 minutes. By using this technology, we can give our patients a clear view of what will happen during their surgery.”

The goggles help us transition our virtual surgical plans into the surgery to the operating room. Thus, it allows us to better view what is in the operative field and visualize objects that otherwise would be very difficult to see in a precise manner.” —E. Bradley Strong

Before his surgical procedure to fix the orbital fracture around his right eye, Richard Odom was able to experience the benefits of extended reality.

During his previous visit to UC Davis Medical Center, the surgical team explained to him what his surgery involved and how they planned to fix his fracture. But, with extended reality, Odom took a trip inside his own skull and saw for himself.

After donning a headset connected to the extended reality system, Odom could see a floating 3D image of his skull.

With a point of his finger, Odom could hide the layer of skin over his bone and see where an implant would be placed within his orbit.

“It's great to see that the implant won't be just floating around in there,” said Odom. “I feel better knowing exactly where it will be.”

Extended reality as a teaching tool

Due to its flexibility in integrating physical and virtual environments, UC Davis Health has also started to use extended reality in educating fellows, residents and medical students.

This means that what trainees previously would have seen as a picture in a book, they are now able to see as a 3D object in the room that they can manipulate and visualize.

“It is exciting to see the light bulb go on for them when they are seeing inside the intracranial cavity and looking at an implant that might be malpositioned,” said Strong. “We have found this technology to be a very powerful educational tool.”

Future for extended reality at UC Davis Health

The new technology will also have a home in Aggie Square, the leading-edge research and innovation hub that is currently under construction on the UC Davis Health Sacramento campus. The 3D Printing and Visualization Lab, MXR research group, and undergraduate campus TEAM lab (UC Davis Biomedical Engineering) are teaming up. Together, they’ll form the leading Visualization and 3D Printing lab/research group in Northern California.

“Extended reality is going to get more and more pervasive throughout medicine,” added Strong. “As the technology continues to grow, it's use will expand to many different areas of surgery, education and become pervasive into all areas of our medical care.”