Liver cancer: holomedicine provides 3D images during surgery | NDR.de – Guide

Put on glasses and surgeons can see the liver of the person to be operated on hovering three-dimensionally in the air: This is one possible application of what is known as holomedicine. What sounds like science fiction is already possible in operating theaters with the latest technology. There is also a holomedicine center in northern Germany where, among other things, liver cancer is operated on.

Augmented Reality: CT or MRI images as holograms

As one of five certified Holomedicine Centers of Excellence (HCoE) worldwide, the Pius Hospital in Oldenburg, Lower Saxony, uses what is known as augmented reality in liver surgery: radiological images, for example from CT or MRI images, are taken converted into three-dimensional holograms. In this way, ultrasound, microscope or endoscopy examinations can also be displayed in real time to other doctors on special glasses.

Three-dimensional representation of the liver using glasses

The doctors see a diseased liver through the glasses in a three-dimensional and richly detailed way, with the large blood vessels and tumors that are to be removed. In the hologram, everyone involved can capture what is otherwise composed of two levels in CT and MRI in three dimensions. This is very helpful, because a liver is difficult to operate on: the organ has a branched vascular system, one and a half liters of blood flow through it every minute. The first incision is important because it determines the future path of the surgeon. The new technology is very helpful in navigating through the organ to the tumor.

Holomedicine enables complex interventions

The hologram of the liver can be reduced and enlarged, you can look at the organ from all sides, which gives the operator a safer feeling before the operation. Because they can see exactly where the tumor is with the new technology, they dare to perform complex interventions that others thought were no longer operable. The hologram is unique, calculated from the data of the affected liver – with all its anomalies. No organ is like the other. This brings more security, even if further interventions are necessary later.

Augmented Reality optimizes surgical planning

A lot of computing effort with artificial intelligence is required to turn two-dimensional CT and MRI images into three-dimensional 3D holograms. The model that is created is particularly detailed in the representation. But not only that: researchers from the Fraunhofer Institute for Digital Medicine MEVIS in Bremen can process the data in such a way that optimal surgical planning is possible: In the 3D liver, sections can be tried out in advance and risks can be calculated.

This is still not possible without manual work. The radiological images are often not precise enough for an automatic recognition of all structures. To compensate for errors, an expert corrects the inaccuracies centimeter by centimetre. This will soon be processed using methods of artificial intelligence – and the 3D model will then be available more quickly for those affected and surgeons. In just a few years, the application options of holomedicine should become the standard in numerous areas of everyday medical life – and there will also be further new possibilities for the optimization of surgical procedures.

experts on the topic