To increase the throughput of image-based wafer quality inspection tools, we propose to use computational imaging methods to address two main bottlenecks: the mechanical alignment of the wafer with the imaging plane and pixel size of the imager. The former requires significant time but is crucial for a reliable quality check, and the latter introduces a trade off between the wafer scanning speed and the minimum detectable defect size. We demonstrate application of our recently developed SANDR algorithm for obtaining a wafer image with sub-pixel resolution from a series of not-perfectly aligned low-resolution images. The wafer misalignment creates a varying over the field of view depth-of-focus, which poses a significant obstacle for the state-of-the-art methods, but is successfully processed with SANDR. The method is tested on simulated images.
KEYWORDS: Neural networks, Object recognition, Imaging systems, Medical imaging, Computed tomography, Monte Carlo methods, Inspection, Data acquisition, Control systems, Computing systems
Some applications require high level of image-based classification certainty while keeping the total illumination energy as low as possible. Examples are minimally invasive visual inspection in Industry 4.0, and medical imaging systems such as computed tomography, in which the radiation dose should be kept "As Low As is Reasonably Achievable". We introduce a sequential object recognition scheme aimed at minimizing phototoxicity or bleaching while achieving a predefined level of decision accuracy. The novel online procedure relies on approximate weighted Bhattacharyya coefficients for determination of future inputs. Simulation results on the MNIST handwritten digit database show how the total illumination energy is decreased with respect to a detection scheme using constant illumination.
We demonstrate a novel closed-loop input design technique on the detection of particles in an imaging system such as a fluorescence microscope. The probability of misdiagnosis is minimized while constraining the input energy such that for instance phototoxicity is reduced. The key novelty of the closed-loop design is that each next input is designed based on the most recent information. Using updated hypothesis probabilities, the input energy distribution is optimized for detection such that unresolved pixels have increased illumination next image acquisition. As compared to conventional open-loop, the results show that (regions of) particles are diagnosed using less energy in the closed-loop approach. Besides the closed-loop approach being viable for the initialization of fluorescence microscopy measurements, it is the next step to sequential object segmentation for reliable and efficient product inspection in Industry 4.0.
We investigate the general adjustment of projection-based phase retrieval algorithms for use with saturated data. In the phase retrieval problem, model fidelity of experimental data containing a non-zero background level, fixed pattern noise, or overexposure, often presents a serious obstacle for standard algorithms. Recently, it was shown that overexposure can help to increase the signal-to-noise ratio in AI applications. We present our first results in exploring this direction in the phase retrieval problem, using as an example the Gerchberg-Saxton algorithm with simulated data. The proposed method can find application in microscopy, characterisation of precise optical instruments, and machine vision applications of Industry4.0.
This paper discusses various practical problems arising in the design and simulation of predictive control methods for adaptive optics. Although there has been increased attention towards optimal prediction and control methods for AO systems, they are often tested in simplified simulation environments. The use of advanced AO simulators however, is a valuable alternative to the use of real data or laboratory experiments, as they provide both a flexible environment which is ideal for testing a new algorithm and are more accessible to non-experts. Topics that are often not explicitly discussed, such as the identification of a turbulence dynamics model from data, the use of matrix structures in AO systems to decrease the computational complexity and the implementation of Kalman filters to optimally deal with realistic noise conditions are examined. All topics discussed are illustrated by an accompanying Matlab code, which is based on the existing Matlab AO toolbox OOMAO.
The light-sheet fluorescence microscopy is an excellent tool for the investigation of large three dimensional microscopy samples at the cellular level, however, the ability to resolve features is strongly affected by the presence of scattering and aberrations. These effects are two fold in light-sheet microscopy, as the illumination path providing the optical sectioning and the fluorescence detection path are both affected by the aberrations in different ways. To overcome these difficulties, we have developed hybrid adaptive optical and computational microscopy techniques to remove the effect of the aberrations in both the excitation and the fluorescence paths of these microscopes.
In life sciences, interest in the microscopic imaging of increasingly complex three dimensional samples, such as cell spheroids, zebrafish embryos, and in vivo applications in small animals, is growing quickly. Due to the increasing complexity of samples, more and more life scientists are considering the implementation of adaptive optics in their experimental setups. While several approaches to adaptive optics in microscopy have been reported, it is often difficult and confusing for the microscopist to choose from the array of techniques and equipment. In this poster presentation we offer a small guide to adaptive optics providing general guidelines for successful adaptive optics implementation.
In this work, we present a new confocal laser scanning microscope capable to perform sensorless wavefront optimization in real time. The device is a parallelized laser scanning microscope in which the excitation light is structured in a lattice of spots by a spatial light modulator, while a deformable mirror provides aberration correction and scanning. A binary DMD is positioned in an image plane of the detection optical path, acting as a dynamic array of reflective confocal pinholes, images by a high performance cmos camera. A second camera detects images of the light rejected by the pinholes for sensorless aberration correction.
We present a quantitative phase imaging microscope based on a Shack-Hartmann sensor, that directly reconstructs the optical path difference (OPD) in reflective mode. Comparing with the holographic or interferometric methods, the SH technique needs no reference beam in the setup, which simplifies the system. With a preregistered reference, the OPD image can be reconstructed from a single shot. Also, the method has a rather relaxed requirement on the illumination coherence, thus a cheap light source such as a LED is feasible in the setup. In our previous research, we have successfully verified that a conventional transmissive microscope can be transformed into an optical path difference microscope by using a Shack-Hartmann wavefront sensor under incoherent illumination. The key condition is that the numerical aperture of illumination should be smaller than the numerical aperture of imaging lens. This approach is also applicable to characterization of reflective and slightly scattering surfaces.
By taking multiple input-output measurements, it is shown how to determine the input to an optical system that corrects unknown phase aberrations without interferometric measurements or online iterative optimization within a couple of seconds. It is shown to work in simulations and experiment. This technique may also be used to acquire the complex field in the pupil, hereby permitting a complex field image to be acquired.
We present a reference-less and time-multiplexing phase retrieval method by making use of the digital micromirror device (DMD). In this method, the DMD functions not only as a flexible binary mask which modulates the optical field, but also as a sampling mask for measuring corresponding phases, which makes the whole setup simple and robust. The DMD reflection forms a sparse intensity mask in the pupil which produces speckle pattern after propagation. With the recorded intensity on the camera and the binary pattern on the DMD, the phase in all the ‘on’ pixels can be reconstructed at once by solving inverse problems with iterative methods, for instance using Gerchberg-Saxton algorithm. Then the phase of the whole pupil can be reconstructed from a series of binary patterns and speckle patterns. Numerical experiments show the feasibility of this phase retrieval method and the importance of sparse binary masks in the improving of convergence speed.
This paper presents experimental results of a static aberration control algorithm based on the linear relation be- tween mean square of the aberration gradient and the second moment of point spread function for the generation of control signal input for a deformable mirror (DM). Results presented in the work of Yang et al.1 suggested a good feasibility of the method for correction of static aberration for point and extended sources. However, a practical realisation of the algorithm has not been demonstrated. The goal of this article is to check the method experimentally in the real conditions of the present noise, finite dynamic range of the imaging camera, and system misalignments. The experiments have shown strong dependence of the linearity of the relationship on image noise and overall image intensity, which depends on the aberration level. Also, the restoration capability and the rate of convergence of the AO system for aberrations generated by the deformable mirror are experi- mentally investigated. The presented approach as well as the experimental results finds practical application in compensation of static aberration in adaptive microscopic imaging system.
Aberrations, scattering and absorption degrade the performance light-sheet fluorescence microscopes (LSFM). An adaptive optics system to correct for these artefacts and to optimize the light-sheet illumination is presented. This system allows a higher axial resolution to be recovered over the field-of-view of the detection objective. It is standard selective plane illumination microscope (SPIM) configuration modified with the addition of a spatial light modulator (SLM) and a third objective for the detection of transmitted light. Optimization protocols use this transmission light allowing the extension the depth-of-field and correction of aberrations whilst retaining a thin optical section.
This proceeding reports early results in the development of a new technique for adaptive optics in confocal microscopy. The term adaptive optics refers to the branch of optics in which an active element in the optical system is used to correct inhomogeneities in the media through which light propagates. In its most classical form, mostly used in astronomical imaging, adaptive optics is achieved through a closed loop in which the actuators of a deformable mirror are driven by a wavefront sensor. This approach is severely limited in fluorescence microscopy, as the use of a wavefront sensor requires the presence of a bright, point like source in the field of view, a condition rarely satisfied in microscopy samples. Previously reported approaches to adaptive optics in fluorescence microscopy are therefore limited to the inclusion of fluorescent microspheres in the sample, to use as bright stars for wavefront sensors, or time consuming sensorless optimization procedures, requiring several seconds of optimization before the acquisition of a single image.
We propose an alternative approach to the problem, implementing sensorless adaptive optics in a Programmable array microscope. A programmable array microscope is a microscope based on a digital micromirror device, in which the single elements of the micromirror act both as point sources and pinholes.
We have implemented an extended depth of field optical system by wavefront coding with a micromachined membrane deformable mirror. This approach provides a versatile extension to standard wavefront coding based on fixed phase mask. First experimental results validate the feasibility of the use of adaptive optics for variable depth wavefront coding in imaging optical systems.
Optimization of the point spread function by means of sensor-less adaptive optics, based on direct imaging of the focal spot, suffers from errors due to enormous dynamic range of the focal intensity. Also, optimization algorithms based on the focal spot metrics only, are insensitive to other system parameters and can converge towrong" solutions. To improve the beam quality and the robustness of the global extremum, we have introduced dynamic feedback control of the camera sensitivity. To further increase the robustness of optimization, we introduced a regularization parameter in the form of some function of the system state, achieving its minimum together with the desired solution. Significant gain in achievable beam quality is shown in comparison with the implementation lacking those improvements. Proposed techniques are implemented in Beam Tuner software forne-tuning of laser and imaging systems with adaptive optics.
Micromachined membrane deformable mirror (MMDM) can serve as an ad hoc" free-form optical element. To test the repeatability and stability of the standard MMDM, we have conducted the test of surface figure during multiple thermal cycling, test of figure drift at elevated temperatures, and a long-term 16-day stability test of actively formed mirror figure. The average rms error did not exceed λ =25 at λ = 633 nm, after repeated cycling from -14 to +70 C, with return to the room temperature. The existing design provides ~10° stability in the temperature range of ~10°. Optimization of the design, eliminating astigmatism, would allow one to extend the temperature range to about 30. The long-term mirror figure instability at a constant temperature reaches λ/20 rms in 16 days. The P-V error with respect to the nearest sphere changes from λ/20 in the first day, to about λ/10 in the 16-th day. The tests show that MMDM is stable enough to make a reasonable alternative to free-form optics in applications that require various optical shapes to be formed with a single element.
We describe an approach that oers an almost real time image enhancement through turbulent and wavy me-
dia. The approach consists in a combination of optimization-based adaptive optics with digital multi-frame
post-processing. Applications in astronomical and terrestrial imaging { where the image features are initially
unresolved due to loss of contrast, blur, vibrations and image wander { have been illustrated by experimental
results. A new software from Flexible Optical BV is presented
Based on the analysis of factors that influence atmospheric imaging over long turbulent horizontal paths, we
consider a number of practical configurations of opto-electronic surveillance systems with optimized performance.
Our approach is based on simultaneous quasi real-time processing of a number of images obtained through
uncorrelated atmospheric paths, using either temporal or spatial multiplexing. Practical results obtained on a
4.25 km imaging path using newly developed imaging system, based on temporal multiplexing, combined with
image restoration based on projection on convex sets, are reported. Potential applications include optical and
IR long-range security and military surveillance, unmanned aircraft imaging systems and naval optical imaging
and warning systems.
By analysis of the mechanical equations, describing deformable mirrors, we show that the print-through is
the natural property of a deformable mirror, which is completely defined by the geometry and mechanics of
deformation. The print-through causes increased scattering in imaging applications, and can result in hot/cold
spots in laser applications. Further development of adaptive optics for extreme UV applications would also
require to address the print-through problem. We describe different ways to reduce, or completely eliminate the
print-through in continuous faceplate and membrane deformable mirrors. In combination with simple hysteresis
compensation, our approach allows for high-precision feedforward control of these deformable mirrors, directly
in terms of Zernike modes.
In this paper we present some experimental results of speckle imaging for near-diffraction-limited observation
of ground-based scenery and astronomical objects through atmospheric turbulence. The method of alternating
projections onto convex sets is used for iterative reconstruction of the point-spread function (PSF), combined
with Wiener filtering for deconvolution and several pre-processing techniques. A modification of the optical
system with aperture segmentation is considered. The results of imaging on a horizontal path and astronomical
imaging are reported and compared with time averaged and best frame images. Apparent image improvement
is demonstrated in a field much wider than the isoplanatic patch size.
We describe a compact integrated module implementing a low-cost adaptive optics system. It is targeted as a
correction system for small telescopes with primary mirror diameter up to 1 m, operating on a natural guide star
with magnitude at least 4 (for a 25 cm telescope). It is supposed to provide stable diffraction-limited imaging
of stars, double stars, planets and artificial bodies in various seeing conditions. Besides, it allows correcting for
static aberrations of the telescope, observer's ocular aberrations and fine adjustment of focus. The first on-sky
tests have demonstrated closed-loop operation with correction for aberrations.
The paper discusses the influence of the Hartmann-(Shack) wavefront sensor geometry on the total error of modal wavefront reconstruction. A mathematical model is proposed which describes modal wavefront reconstruction based on Hartmann or Hartmann-Shack sensor in terms of linear operators. The modal covers the most general case and is not limited by the orthogonality of decomposition basis or by the method chosen for decomposition. The total reconstruction error is
calculated for any given statistics of the wavefront to be measured. Based on this estimate, total reconstruction error is calculated for regular and randomised Hartmann masks. The calculations demonstrate that use of random masks with non-regular Fourier spectra for Zernike wavefront reconstruction for atmospheric turbulence allows to double the number of decomposition modes with the same total error.
This article presents a prototype of a CMOS phase sensor for high accuracy (1 Angstrom) heterodyne interferometry. Switched integrators realization of a lock-in pixel for 4-bucket phase detection algorithm is described and illustrated by experimental results. Factors that limit the accuracy of this implementation and possible ways for its improvement are discussed.
We propose an algorithm for phase retrieval from three
interferograms which differ only by an arbitrary unknown tilt
terms in the phase. The method is illustrated by examples.
We present an overview of the results of our recent research in the field of adaptive optical components based on silicon microtechnologies, including membrane deformable mirrors, spatial light modulators, liquid-crystal correctors, wavefront sensors, and both spherical and aspherical micro-optical components. We aim at the realization of adaptive optical systems using standard-technology solutions.
Standard heterodyne interferometer can be used as phase modulation subsystem in a novel interferometer designed to measure the figure of projecting mirrors with 0.1 nm accuracy. This article discusses possible operational principles of the sensor and presents experimental results for fast sampling type sensor prototype.
One day school on Smart Optics Technologies present the basics of the Adaptive Optics and its applications. The event is aimed to academic and industrial participants willing to extend their expertise in the field.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.