Statistical Multi-Modal Fusion for Patient-Centric Medical Diagnosis Using DICOM

Deep learning (DL) has significantly advanced medical image analysis, especially for disease classification. Yet, integrating patient-specific attributes, such as age, BMI, and lifestyle, with radiomics and DICOM-derived features remains challenging. We introduce a multi-modal DL framework, the Statistically Coherent Network (SCN), which captures individual variability by projecting data into a multi-space latent representation. SCN aligns feature distributions across patient subgroups using a novel combination of t-test-based and triplet losses, promoting statistically coherent clusters in the latent space. Evaluated on four clinical datasets - breast cancer, sleep apnea, rotator cuff tear, and Cormack-Lehane grade - our model outperforms single-space baselines in classification accuracy and latent space interpretability, highlighting its robustness across diverse patient populations. These results suggest that SCN offers a promising direction for personalized, statistically grounded diagnosis in multi-modal medical imaging.