Frontiers in Medical Science Research, 2022, 4(4); doi: 10.25236/FMSR.2022.040410.
Hongmei Tang1, Luhang Dai2
1College of Life Sciences and Technology, Huazhong University of Science and Technology, Wuhan, Hubei, 430074, China
2College of Life Science, Shaanxi Normal University, Xi'an, Shaanxi, 710119, China
The identification of breast cancer subtypes plays a key role in the prognosis of breast cancer. In recent years, deep learning (DL) has shown good performance in intelligent identification of breast cancer subtypes. However, most of the traditional DL models use single-mode data, and the extracted features are limited, so the association between patient characteristics and breast cancer subtypes cannot be established stably. In order to improve the effect of recognition, this study proposes a multimodal fusion deep learning (MFDL) model. This model combined with the breast cancer gene modal data and image modal data established a multilayer perceptron network and the depth of the convolution neural network for feature extraction, and then based on the idea of weighted aggregation on the output of the two characteristics of the network integration. Finally, the fusion features were used to identify breast cancer subtypes. The experimental results show that compared with other models in AUC value, accuracy and other indicators, the MFDL model proposed in this study is more accurate and efficient in the identification of breast cancer subtypes.
Breast Cancer Subtype, Multimodal, Deep Learning, Feature Fusion, In-Telligent Recognition
Hongmei Tang, Luhang Dai. Identification of Molecular Subtypes of Breast Cancer Based on Multimodal Deep Learning. Frontiers in Medical Science Research (2022) Vol. 4, Issue 4: 64-69. https://doi.org/10.25236/FMSR.2022.040410.
[1] McGranahan N, Swanton C, Clonal heterogeneity and tumor evolution: past, present, and the future [J]. Cell, 2017, 168(4): 613-628.
[2] Guiu S, Michiels S, André F, et al., Molecular subclasses of breast cancer: how do we define them? The IMPAKT 2012 Working Group Statement [J]. Annals of oncology, 2012, 23(12): 2997-3006.
[3] Ha R, Mutasa S, Karcich J, et al., Predicting breast cancer molecular subtype with MRI dataset utilizing convolutional neural network algorithm [J]. Journal of digital imaging, 2019, 32(2): 276-282.
[4] Couture H D, Williams L A, Geradts J, et al., Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype [J]. NPJ breast cancer, 2018, 4 (1): 1-8.
[5] Lahat D, Adali T, Jutten C, Multimodal data fusion: an overview of methods, challenges, and prospects [J]. Proceedings of the IEEE, 2015, 103(9): 1449-1477.
[6] Quackenbush J, Microarray data normalization and transformation [J]. Nat Genet, 2002, 32: 496–501.
[7] Fu X, Wang L, Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance [J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2003, 33(3): 399-409.
[8] LeCun Y, Bottou L, Bengio Y, et al., Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[9] Murugan P, Durairaj S, Regularization and optimization strategies in deep convolutional neural network [J]. arXiv preprint arXiv: 1712.04711, 2017.
[10] Simonyan K, Zisserman A, Very deep convolutional networks for large-scale image recognition [J]. arXiv preprint arXiv:1409.1556, 2014.
[11] Szegedy C, Vanhoucke V, Ioffe S, et al., Rethinking the inception architecture for computer vision [C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2818-2826.
[12] Srivastava N, Hinton G, Krizhevsky A, et al., Dropout: a simple way to prevent neural networks from overfitting [J]. The journal of machine learning research, 2014, 15(1): 1929-1958.