Leverage weekly annotation to pixel-wise annotation via zero-shot segment anything model for molecular-empowered learning

Xueyuan Li; Ruining Deng; Yucheng Tang; Shunxing Bao; Haichun Yang; Yuankai Huo

doi:10.1117/12.3006577

3 April 2024 Leverage weekly annotation to pixel-wise annotation via zero-shot segment anything model for molecular-empowered learning

Xueyuan Li, Ruining Deng, Yucheng Tang, Shunxing Bao, Haichun Yang, Yuankai Huo

Author Affiliations +

Proceedings Volume 12933, Medical Imaging 2024: Digital and Computational Pathology; 129330K (2024) https://doi.org/10.1117/12.3006577
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Abstract

Precise identification of multiple cell classes in high-resolution Giga-pixel whole slide imaging (WSI) is critical for various clinical scenarios. Building an AI model for this purpose typically requires pixel-level annotations, which are often unscalable and must be done by skilled domain experts (e.g., pathologists). However, these annotations can be prone to errors, especially when distinguishing between intricate cell types (e.g., podocytes and mesangial cells) using only visual inspection. Interestingly, a recent study showed that lay annotators, when using extra immunofluorescence (IF) images for reference (referred to as molecular-empowered learning), can sometimes outperform domain experts in labeling. Despite this, the resource-intensive task of manual delineation remains a necessity during the annotation process. In this paper, we explore the potential of bypassing pixel-level delineation by employing the recent segment anything model (SAM) on weak box annotation in a zero-shot learning approach. Specifically, we harness SAM’s ability to produce pixel-level annotations from box annotations and utilize these SAM-generated labels to train a segmentation model. Our findings show that the proposed SAM-assisted molecular-empowered learning (SAM-L) can diminish the labeling efforts for lay annotators by only requiring weak box annotations. This is achieved without compromising annotation accuracy or the performance of the deep learning-based segmentation. This research represents a significant advancement in democratizing the annotation process for training pathological image segmentation, relying solely on non-expert annotators.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xueyuan Li, Ruining Deng, Yucheng Tang, Shunxing Bao, Haichun Yang, and Yuankai Huo "Leverage weekly annotation to pixel-wise annotation via zero-shot segment anything model for molecular-empowered learning", Proc. SPIE 12933, Medical Imaging 2024: Digital and Computational Pathology, 129330K (3 April 2024); https://doi.org/10.1117/12.3006577

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available