SEGTRANSVAE: Hybrid CNN-Transformer with Regularization for Medical Image Segmentation

Abstract

Current research on deep learning for medical image segmentation highlights limitations in learning either global semantic information or local contextual information effectively. To address these challenges, this paper introduces a novel network named SegTransVAE. SegTransVAE utilizes an encoder-decoder architecture enhanced with a transformer and a variational autoencoder (VAE) branch to jointly reconstruct input images and perform segmentation. This approach represents a pioneering effort in combining the strengths of CNNs, transformers, and VAEs. Evaluation across various recent datasets demonstrates that SegTransVAE achieves superior performance in Dice Score and 95%-Hausdorff Distance compared to previous methods, while maintaining a comparable inference time to simpler CNN-based architectures. The source code is available at: https://github.com/itruonghai/SegTransVAE.