RAMIS: Increasing robustness and accuracy in medical image segmentation with hybrid CNN-transformer synergy

Hybrid architectures based on Convolutional Neural Network (CNN) and Vision Transformer (ViT) have become an important research direction in medical image segmentation in recent years. However, the currently popular hybrid architectures weaken the decision making process within the Transformer model, the way in which the output of the Transformer is post-processed by the upsampling of the convolution stack makes it difficult to restore the blurred boundaries of the target area. To improve the feature learning capability by addressing these issues, we propose RAMIS, a novel hybrid architecture for general medical image segmentation. RAMIS develops implicit neural representation and self-distillation to simultaneously obtain the super-resolution details and core features of the image as input to the Transformer encoder. Meanwhile, RAMIS explores an unsupervised learning CNN to obtain the initial input to the Transformer decoder, which not only explicitly considers the correlation within different samples, reduces the constraints on small datasets, but also fully leverages the potential of Transformer's cross-attention for optimizing segmentation results. RAMIS designs a multi-resolution interaction network to post-process the Transformer output and solves the problem of blurred segmentation boundaries by combining super-resolution image. We extensively evaluate RAMIS on five datasets from three typical publicly available medical image segmentation datasets. Extensive experimental results demonstrate the general applicability and superior performance of the proposed method. The code and pre-trained models are available on our website https://ramis.netlify.app.