Joint MR to CT synthesis and segmentation for MR-only pediatric brain radiation therapy planning

Radiation therapy (RT) planning for pediatric brain cancer is a challenging task. Manual contouring of organs-at-risk (OARs) is particularly difficult due to the small size of brain structures, time-consuming, and shows inter-observer variability. Furthermore, RT plans are typically optimized using CT, thus exposing patients to additional ionizing radiation. MR-only RT planning has recently been actively explored due to its potential to overcome these challenges. While numerous methods have been proposed to solve MR to CT image synthesis or OAR segmentation separately, there exist only a handful of methods tackling both problems jointly, even less specifically developed for pediatric brain cancer RT. We propose a multi-task convolutional neural network to jointly synthesize CT from MRI and segment OARs (eyes, optic nerves, optic chiasm, temporal lobes, hippocampi, and brainstem) for pediatric brain RT planning. The proposed network consists of a modified 3D U-Net architecture with a common encoder for both the synthesis and segmentation tasks combined with two task-specific decoders. The proposed model was trained, validated, and tested on 50, 5, and 15 pediatric brain RT cases, respectively, and achieved a mean±SD structural similarity index of 0.82±0.03 between the synthetic CT and ground truth CT, and dice score for the autosegmentation of 0.92±0.03 (eyes), 0.78±0.06 (optic nerves), 0.690.13 (optic chiasm), 0.91±0.02 (temporal lobes), 0.75±0.09 (hippocampi), and 0.91±0.06 (brainstem) compared to the expert's manual segmentation. Our proposed multi-task joint synthesis and segmentation network achieves state-of-the-art performance for both tasks for MR-only RT planning.