TY - GEN
T1 - A dynamic multi-modal fusion network for ovarian tumor differentiation
AU - Li, Yang
AU - Zou, Beiji
AU - Wu, Jing
AU - Dai, Yulan
AU - Bai, Harrison X.
AU - Jiao, Zhicheng
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Accurate ovarian tumor differentiation is a challenging task where the benign and malignant tumors share similar T1C and T2WI MRI appearances. Therefore, it is necessary to leverage additional multi-modal data, e.g., the age, CA125level, and other clinical information, which are helpful but rarely exploited. In this paper, we propose a dynamic fusion network that can adaptively make full use of multi-modal data, including MRI and clinical information, to realize precise ovarian tumor differentiation. Specifically, we design a dynamic nonlinear module (D-Non-L module) on the top of the image representation. The D-Non-L module is formulated as an iterative nonlinear projection parameterized by the learned features of the patient-wise clinical information. With the help of this module, the interaction between clinical features and image features could be achieved to adaptively improve the discrimination of visual representations. Moreover, we construct a dual-path-based architecture to fully exploit the complementary information from T1C and T2WI MRIs. Extensive experimental results on the locally organized ovarian tumor dataset demonstrate that our methods are superior to the single-modal and single-path-based methods. And the proposed dynamic non-linear module obtains the best performance compared with other multi-modal fusion strategies.
AB - Accurate ovarian tumor differentiation is a challenging task where the benign and malignant tumors share similar T1C and T2WI MRI appearances. Therefore, it is necessary to leverage additional multi-modal data, e.g., the age, CA125level, and other clinical information, which are helpful but rarely exploited. In this paper, we propose a dynamic fusion network that can adaptively make full use of multi-modal data, including MRI and clinical information, to realize precise ovarian tumor differentiation. Specifically, we design a dynamic nonlinear module (D-Non-L module) on the top of the image representation. The D-Non-L module is formulated as an iterative nonlinear projection parameterized by the learned features of the patient-wise clinical information. With the help of this module, the interaction between clinical features and image features could be achieved to adaptively improve the discrimination of visual representations. Moreover, we construct a dual-path-based architecture to fully exploit the complementary information from T1C and T2WI MRIs. Extensive experimental results on the locally organized ovarian tumor dataset demonstrate that our methods are superior to the single-modal and single-path-based methods. And the proposed dynamic non-linear module obtains the best performance compared with other multi-modal fusion strategies.
KW - dynamic network
KW - multi-modal
KW - ovarian tumor differentiation
UR - http://www.scopus.com/inward/record.url?scp=85146734085&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85146734085&partnerID=8YFLogxK
U2 - 10.1109/BIBM55620.2022.9995556
DO - 10.1109/BIBM55620.2022.9995556
M3 - Conference contribution
AN - SCOPUS:85146734085
T3 - Proceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
SP - 767
EP - 772
BT - Proceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
A2 - Adjeroh, Donald
A2 - Long, Qi
A2 - Shi, Xinghua
A2 - Guo, Fei
A2 - Hu, Xiaohua
A2 - Aluru, Srinivas
A2 - Narasimhan, Giri
A2 - Wang, Jianxin
A2 - Kang, Mingon
A2 - Mondal, Ananda M.
A2 - Liu, Jin
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
Y2 - 6 December 2022 through 8 December 2022
ER -