Spatial Attention-Guided Prompt Learning for Anomaly Segementation in Medical Images

Accurate segmentation of anomalies in medical images is critical for early diagnosis and effective treatment planning. Although recent vision-language frameworks such as MediCLIP have shown promise in few-shot settings, they often suffer from limited domain specificity and reduced localization accuracy when applied to heterogeneous medical data. In this work, we propose a fully supervised anomaly segmentation framework that integrates learnable text prompts with a novel Organic Spatial Attention Adapter. Unlike the baseline adapter in MediCLIP, our design employs an Inter-Layer Gated Fusion mechanism to adaptively integrate semantic features from different transformer depths, combined with a lightweight spatial attention module to enhance the localization of fine-grained abnormalities. The adapter also includes a target channel projection for downstream compatibility and residual normalization to ensure stable training. Leveraging curated prompt sets of normal and abnormal anatomical concepts, the model aligns multimodal embeddings for robust anomaly detection. Experiments on the BMAD benchmark - covering BraTS2021, BTCV+LiTs, and RESC datasets - demonstrate that our method achieves state-of-the-art performance in both image-level AUROC (I-AUROC) and pixel-level AUROC (P-AUROC), with notable improvements in pixel-level localization accuracy. These results validate the effectiveness of our architecture in enhancing domain adaptation and anomaly segmentation across diverse medical imaging modalities.