A study of attention information from transformer layers in hybrid medical image segmentation networks

Syed Nouman Hasany; Caroline Petitjean; Fabrice Meriaudeau

doi:10.1117/12.2652215

3 April 2023 A study of attention information from transformer layers in hybrid medical image segmentation networks

Syed Nouman Hasany, Caroline Petitjean, Fabrice Meriaudeau

Proceedings Volume 12464, Medical Imaging 2023: Image Processing; 124641O (2023) https://doi.org/10.1117/12.2652215
Event: SPIE Medical Imaging, 2023, San Diego, California, United States

Conference Poster

Abstract

Transformer models have recently started gaining popularity in Computer Vision related tasks. Within Medical Image Segmentation, segmentation models such as TransUNet have incorporated transformer blocks alongside convolutional blocks while remaining faithful to the popular U-Net architecture. The present work utilizes attention maps to examines information flow within transformer blocks of three such segmentation models: (i) TransUNet, (ii) 2D CATS, and (iii) 2D UNETR. Based on the attention maps, compressed versions of these models are proposed which retain only as many transformer layers as are necessary for the model to achieve a global receptive field. The parameter saving is more than 60% whereas the dice metric does not drop by more than 5% compared to the original (uncompressed) model.

Citation Download Citation

Syed Nouman Hasany, Caroline Petitjean, and Fabrice Meriaudeau "A study of attention information from transformer layers in hybrid medical image segmentation networks", Proc. SPIE 12464, Medical Imaging 2023: Image Processing, 124641O (3 April 2023); https://doi.org/10.1117/12.2652215

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available