Robust photometric stereo endoscopy via deep learning trained on synthetic data (Conference Presentation)

Faisal Mahmood; Daniel Borders; Richard Chen; Jordan Sweer; Steven Tilley II; Norman S. Nishioka; J. Webster Stayman; Nicholas J. Durr

doi:10.1117/12.2509878

4 March 2019 Robust photometric stereo endoscopy via deep learning trained on synthetic data (Conference Presentation)

Faisal Mahmood, Daniel Borders, Richard Chen, Jordan Sweer, Steven Tilley II, Norman S. Nishioka, J. Webster Stayman, Nicholas J. Durr

Author Affiliations +

Proceedings Volume 10871, Multimodal Biomedical Imaging XIV; 108710N (2019) https://doi.org/10.1117/12.2509878
Event: SPIE BiOS, 2019, San Francisco, California, United States

Abstract

Colorectal cancer is the second leading cause of cancer deaths in the United States and causes over 50,000 deaths annually. The standard of care for colorectal cancer detection and prevention is an optical colonoscopy and polypectomy. However, over 20% of the polyps are typically missed during a standard colonoscopy procedure and 60% of colorectal cancer cases are attributed to these missed polyps. Surface topography plays a vital role in identification and characterization of lesions, but topographic features often appear subtle to a conventional endoscope. Chromoendoscopy can highlight topographic features of the mucosa and has shown to improve lesion detection rate, but requires dedicated training and increases procedure time. Photometric stereo endoscopy captures this topography but is qualitative due to unknown working distances from each point of mucosa to the endoscope. In this work, we use deep learning to estimate a depth map from an endoscope camera with four alternating light sources. Since endoscopy videos with ground truth depth maps are challenging to attain, we generated synthetic data using graphical rendering from an anatomically realistic 3D colon model and a forward model of a virtual endoscope with alternating light sources. We propose an encoder-decoder style deep network, where the encoder is split into four branches of sub-encoder networks that simultaneously extract features from each of the four sources and fuse these feature maps as the network goes deeper. This is complemented by skip connections, which maintain spatial consistency when the features are decoded. We demonstrate that, when compared to monocular depth estimation, this setup can reduce the average NRMS error for depth estimation in a silicone colon phantom by 38% and in a pig colon by 31%.

Conference Presentation

Citation Download Citation

Faisal Mahmood, Daniel Borders, Richard Chen, Jordan Sweer, Steven Tilley II, Norman S. Nishioka, J. Webster Stayman, and Nicholas J. Durr "Robust photometric stereo endoscopy via deep learning trained on synthetic data (Conference Presentation)", Proc. SPIE 10871, Multimodal Biomedical Imaging XIV, 108710N (4 March 2019); https://doi.org/10.1117/12.2509878

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available