On the evaluation of segmentation editing tools

Frank Heckel; Jan H. Moltz; Hans Meine; Benjamin Geisler M.D.; Andreas Kießling; Melvin D’Anastasi; Daniel Pinto dos Santos; Ashok Joseph Theruvath; Horst K. Hahn

doi:10.1117/1.JMI.1.3.034005

14 November 2014 On the evaluation of segmentation editing tools

Frank Heckel, Jan H. Moltz, Hans Meine, Benjamin Geisler M.D., Andreas Kießling, Melvin D’Anastasi, Daniel Pinto dos Santos, Ashok Joseph Theruvath, Horst K. Hahn

Author Affiliations +

Journal of Medical Imaging, Vol. 1, Issue 3, 034005 (November 2014). https://doi.org/10.1117/1.JMI.1.3.034005

Abstract

Efficient segmentation editing tools are important components in the segmentation process, as no automatic methods exist that always generate sufficient results. Evaluating segmentation editing algorithms is challenging, because their quality depends on the user’s subjective impression. So far, no established methods for an objective, comprehensive evaluation of such tools exist and, particularly, intermediate segmentation results are not taken into account. We discuss the evaluation of editing algorithms in the context of tumor segmentation in computed tomography. We propose a rating scheme to qualitatively measure the accuracy and efficiency of editing tools in user studies. In order to objectively summarize the overall quality, we propose two scores based on the subjective rating and the quantified segmentation quality over time. Finally, a simulation-based evaluation approach is discussed, which allows a more reproducible evaluation without the need for human input. This automated evaluation complements user studies, allowing a more convincing evaluation, particularly during development, where frequent user studies are not possible. The proposed methods have been used to evaluate two dedicated editing algorithms on 131 representative tumor segmentations. We show how the comparison of editing algorithms benefits from the proposed methods. Our results also show the correlation of the suggested quality score with the qualitative ratings.

Citation Download Citation

Frank Heckel, Jan H. Moltz, Hans Meine, Benjamin Geisler M.D., Andreas Kießling, Melvin D’Anastasi, Daniel Pinto dos Santos, Ashok Joseph Theruvath, and Horst K. Hahn "On the evaluation of segmentation editing tools," Journal of Medical Imaging 1(3), 034005 (14 November 2014). https://doi.org/10.1117/1.JMI.1.3.034005

Published: 14 November 2014

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available