Radiomics-Enhanced Multi-Modal Deep Learning Pipelines for Stratified Cancer Prognosis Using Integrated Imaging Genomics and Clinical Data
DOI:
https://doi.org/10.63503/j.ijaimd.2025.123Keywords:
Cancer Prognosis, Multi-Mode Deep Learning, radiomics, genomic data, clinical data, machine learning, imaging genomicsAbstract
Cancer outcome prediction abilities form a cornerstone of personalized medicine strategies. More conventional approaches lead to information used from one information source, whether clinical records or radiological scans, limiting their ability to account for the heterogeneous properties of cancer. Over the past few years, developments in machine learning and deep learning have shown that using several types of data can improve the accuracy and reliability of cancer-prognostication models. By using radiomics to extract metrics from medical images, we gain meaningful depth on tumor properties. On the other hand, genetic data explains the molecular basis of cancer and how it forms. Patient age, gender, and medical history are critical components of interpreting the progression of diseases, from data such as these, collected from clinical records. A new method is proposed that incorporates deep learning in combining radiomic, genomic and clinical information for better cancer prognosis. This way, it merges the most modern strategies for feature extraction and deep learning frameworks for a proper analysis and combination of information available at the different modalities of the pipeline. Datasets of different types of cancer are used to evaluate the predictive power of the method. It is demonstrated on breast, lung, and colorectal cancers, which demonstrate a significant performance increase relative to traditional models where a single data source is used. The results show that integrating radiomics, genomics, and clinical data leads to more accurate and personalized predictions of cancer outcomes that can greatly inform clinical decisions. The results also show that the use of mult-modal learning techniques holds the promise of improving generalizability and resilience of different patient cohorts’ prediction models. It demonstrates how using several sources of data improves the forecasting of cancer outcomes and allows for more effective individualized treatment strategies.
References
[1] Mina, M., Raynaud, F., Tavernari, D., Battistello, E., Sungalee, S., Saghafinia, S., ... & Ciriello, G. (2017). Conditional selection of genomic alterations dictates cancer evolution and oncogenic dependencies. Cancer cell, 32(2), 155-168. https://doi.org/10.1016/j.ccell.2017.06.010
[2] Vyas, A., Kumar, K., Sharma, A., Verma, D., Bhatia, D., Wahi, N., & Yadav, A. K. (2025). Advancing the frontier of artificial intelligence on emerging technologies to redefine cancer diagnosis and care. Computers in Biology and Medicine, 191, 110178.
[3] Spinnato, P., Kind, M., Le Loarer, F., Bianchi, G., Colangeli, M., Sambri, A., ... & Crombé, A. (2022). Soft tissue sarcomas: the role of quantitative MRI in treatment response evaluation. Academic Radiology, 29(7), 1065-1084. https://doi.org/10.1016/j.acra.2021.08.007
[4] Pan, G., Pan, Z., Chen, W., Wu, Y., Di, X., Zhou, F., & Liao, Y. (2025). Integration of MRI radiomics and clinical data for preoperative prediction of vascular invasion in breast cancer: A deep learning approach. Magnetic Resonance Imaging, 110339.
[5] Hait, S., Noronha, V., Chowdhury, A., Chaudhary, A., Bawaskar, B., Dahimbekar, G., ... & Dutt, A. (2025). The impact of co-occurring tumor suppressor mutations with mEGFR as early indicators of relapse in lung cancer. ESMO open, 10(4), 104479.
[6] Zhao, X., Wang, L., Zhang, Y., Han, X., Deveci, M., & Parmar, M. (2024). A review of convolutional neural networks in computer vision. Artificial Intelligence Review, 57(4), 99. https://doi.org/10.1007/s10462-024-10721-6
[7] Ejiyi, C. J., Qin, Z., Agbesi, V. K., Yi, D., Atwereboannah, A. A., Chikwendu, I. A., ... & Bamisile, O. O. (2025). Advancing cancer diagnosis and prognostication through deep learning mastery in breast, colon, and lung histopathology with ResoMergeNet. Computers in Biology and Medicine, 185, 109494. https://doi.org/10.1016/j.compbiomed.2024.109494
[8] Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015). Machine learning applications in cancer prognosis and prediction. Computational and structural biotechnology journal, 13, 8-17. https://doi.org/10.1016/j.csbj.2014.11.005
[9] Abbas, S., Asif, M., Rehman, A., Alharbi, M., Khan, M. A., & Elmitwally, N. (2024). Emerging research trends in artificial intelligence for cancer diagnostic systems: A comprehensive review. Heliyon. https://doi.org/10.1016/j.heliyon.2024.e36743
[10] McCague, C., Ramlee, S., Reinius, M., Selby, I., Hulse, D., Piyatissa, P., ... & Woitek, R. (2023). Introduction to radiomics for a clinical audience. Clinical Radiology, 78(2), 83-98. https://doi.org/10.1016/j.crad.2022.08.149
[11] Doyle, J. P., Patel, P. H., Petrou, N., Shur, J., Orton, M., Kumar, S., & Bhogal, R. H. (2023). Radiomic applications in upper gastrointestinal cancer surgery. Langenbeck's Archives of Surgery, 408(1), 226. https://doi.org/10.1007/s00423-023-02951-z
[12] Hait, S., Noronha, V., Chowdhury, A., Chaudhary, A., Bawaskar, B., Dahimbekar, G., ... & Dutt, A. (2025). The impact of co-occurring tumor suppressor mutations with mEGFR as early indicators of relapse in lung cancer. ESMO open, 10(4), 104479. https://doi.org/10.1016/j.esmoop.2025.104479
[13] Lever, J., Cheng, S., Casas, C. Q., Liu, C., Fan, H., Platt, R., ... & Arcucci, R. (2025). Facing & mitigating common challenges when working with real-world data: The Data Learning Paradigm. Journal of Computational Science, 85, 102523. https://doi.org/10.1016/j.jocs.2024.102523
[14] Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2024). A comparison review of transfer learning and self-supervised learning: Definitions, applications, advantages and limitations. Expert Systems with Applications, 242, 122807. https://doi.org/10.1016/j.eswa.2023.122807
[15] Liu, H., Shi, Y., Li, A., & Wang, M. (2024). Multi-modal fusion network with intra-and inter-modality attention for prognosis prediction in breast cancer. Computers in Biology and Medicine, 168, 107796. https://doi.org/10.1016/j.compbiomed.2023.107796
[16] Zhang, N., Zhang, X., Li, J., Ren, J., Li, L., Dong, W., & Liu, Y. (2023). CT-derived radiomic analysis for predicting the survival rate of patients with non-small cell lung cancer receiving radiotherapy. Physica Medica, 107, 102546. https://doi.org/10.1016/j.ejmp.2023.102546
[17] Prinzi, F., Orlando, A., Gaglio, S., & Vitabile, S. (2024). Breast cancer classification through multivariate radiomic time series analysis in DCE-MRI sequences. Expert Systems with Applications, 249, 123557. https://doi.org/10.1016/j.eswa.2024.123557
[18] Miras, I., Estévez-García, P., & Muñoz-Galván, S. (2024). Clinical and molecular features of platinum resistance in ovarian cancer. Critical Reviews in Oncology/Hematology, 104434. https://doi.org/10.1016/j.critrevonc.2024.104434
[19] Tavakoli, M., Chandra, R., Tian, F., & Bravo, C. (2025). Multi-modal deep learning for credit rating prediction using text and numerical data streams. Applied Soft Computing, 112771. https://doi.org/10.1016/j.asoc.2025.112771
[20] Li, Z., Li, H., Ralescu, A. L., Dillman, J. R., Parikh, N. A., & He, L. (2023). A novel collaborative self-supervised learning method for radiomic data. NeuroImage, 277, 120229. Chen, H., Gouin-Vallerand, C., Bouchard, K., Gaboury, S., Couture, M., Bier, N., & Giroux, S. (2024). Contrastive Self-Supervised Learning for Sensor-Based Human Activity Recognition: A Review. IEEE Access. doi: 10.1109/ACCESS.2024.3480814
[21] He, K., Zhu, J., Li, L., Gou, F., & Wu, J. (2024). Two‐stage coarse‐to‐fine method for pathological images in medical decision‐making systems. IET Image Processing, 18(1), 175-193. https://doi.org/10.1049/ipr2.12941
