Automated MR image segmentation enables augmented reality (AR) use for surgeons

Researchers led by Dr. Tim Fick of the Princess Máxima Center for Pediatric Oncology in Utrecht, the Netherlands, developed and validated a cloud-based segmentation algorithm that automatically generates 3D models from contrastenhanced T1 weighted MR sequences. These models are then automatically optimized and prepared for neurosurgeons to view in 3D on a computer or AR headset. In testing on 50 patients with a brain tumor, their approach was highly accurate and much faster than manual segmentation. “The automatic cloud-based segmentation algorithm is reliable, accurate, and fast enough to aid neurosurgeons in everyday clinical practice by providing 3D augmented reality visualization of contrast-enhancing intracranial lesions measuring at least 5 cm,” the authors wrote.
A number of studies have shown the benefits of 3D models and AR in visualizing complex pathology and anatomy, as well as for surgical planning and preparation. However, this currently requires multiple manual steps, including exporting DICOM images from the PACS, creating 3D models with manual or semiautomated segmentation, optimizing the models for visualization, simplifying the models to account for limited processing power, and then saving them into different file formats. This process is cumbersome and therefore can’t be integrated into daily clinical practice, according to the researchers. “To make this technique more accessible to surgeons, we built a fully automatic workflow that uses DICOM images to create 3D models suitable for AR-[headmounted displays] and is integrated with an automatic segmentation algorithm that segments skin, brain, ventricles, and tumor,” the authors wrote.
The authors acknowledged the limitations of the algorithm with regards to a minimum tumor volume and the pathologies it can be used for. “The next steps involve incorporation of other sequences and improving accuracy with 3D fine-tuning in order to expand the scope of augmented reality workflow,” they wrote.