Hardware-in-the-loop training for lensless imaging with a programmable mask

Eric Bezzam; Martin Vetterli

doi:10.1117/12.3021868

13 March 2024 Hardware-in-the-loop training for lensless imaging with a programmable mask

Eric Bezzam, Martin Vetterli

Proceedings Volume 12857, Computational Optical Imaging and Artificial Intelligence in Biomedical Sciences; 128570P (2024) https://doi.org/10.1117/12.3021868
Event: SPIE BiOS, 2024, San Francisco, California, United States

Conference Poster

Abstract

Lensless imaging can drastically relax traditional camera constraints by replacing lenses with optical masks, enabling lighter, cheaper, and thinner systems. However, unlike lenses, there is a lack of clear criteria for optical mask design. Most approaches are heuristic: either selecting a random mask or designing one with desired spectral and/or directional filter properties. Recent work jointly optimizes a phase mask and a task-specific neural network, but in simulation. We propose and demonstrate hardware-in-the-loop (HITL) training for jointly optimizing the mask and reconstruction parameters of a lensless imaging system, using the physical system itself in forward passes and a simulated model for determining updates during backpropagation. As the physical system uses a programmable mask, system updates can be done during training. Results show significant improvements in image quality metrics (2.14 dB in PSNR, 21.4% relative improvement in a perceptual metric) by jointly learning mask and reconstruction parameters. A low-cost prototype (less than 100 USD) is used, with open-source training and measurement code available on GitHub: https://github.com/LCAV/LenslessPiCam

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Eric Bezzam and Martin Vetterli "Hardware-in-the-loop training for lensless imaging with a programmable mask", Proc. SPIE 12857, Computational Optical Imaging and Artificial Intelligence in Biomedical Sciences, 128570P (13 March 2024); https://doi.org/10.1117/12.3021868

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available