Alongside advancements in computational design tools and nanofabrication technology, the field of meta-optics has grown rapidly in the recent decade. Meta-optics have the potential to dramatically miniaturize imaging systems for applications including endoscopy, autonomous drones and vehicles, and consumer electronics.

However, two key challenges that have limited imaging quality with meta-optics are chromatic aberrations and limited field of view (FoV). Recent works have made great progress towards solving these issues separately, but simultaneous achievement of full-color and wide FoV has remained elusive. For chromatic aberration correction, one demonstrated approach is to engineer the dispersion of the meta-atoms to compensate for chromatic aberration.¹ However, meta-optics employing this approach are typically limited to either small aperture or narrow FoV, due to the group delay dispersion that can be obtained by engineering the meta-atoms alone.² Similarly, meta-optics designed for wide FoV functionality have been highly successful under monochromatic illumination, but not under broadband illumination.^3,4

In a recent report,⁵ Yan Liu, Jian-Wen Dong, and their team address these issues using a monochromatic wide FoV metalens with a neural network-based image reconstruction technique to simultaneously achieve both wide FoV and full-color imaging in a compact meta-optical system. Several recent studies have reported high-quality full-color imaging via a meta-optic in combination with neural network-based reconstruction algorithms, but for a modest 30°–40° FoV.^6,7 To increase the FoV, this new study reports combining a wide FoV metalens with a neural network based on transformer-based network.

To design the wide FoV metalens, they adopt the demonstrated approach of combining the metalens with an external aperture (located on the back of the metasurface substrate) to control incident illumination, thereby optimizing each section of the metalens for a particular angle of incidence. The wide FoV performance of this metalens is nearly diffraction-limited at the design wavelength of 532 nm; however, non-optimal wavelengths are dim and unfocused. These issues are corrected using the computational reconstruction (Fig. 1).

Fig. 1

The full-color meta-camera reported in Ref. 5. A wide FoV metalens and transformer neural network synergistically achieve high quality full-color imaging in the visible band.

A key innovation in this work is the use of a knowledge-fused data-driven transformer network to computationally restore images. Transformer neural networks have recently gathered attention for their success in modeling long-range dependencies in deep learning tasks. The authors hypothesize that this kind of network enables the transformer to model long-range dependencies between the non-focused off-axis point spread functions and central bright speckle. Wide FoV metalenses exhibit complex spatially varying aberrations that are difficult to correct using traditional neural network architectures, such as convolutional neural networks that perform local convolutions with a kernel. To highlight the effectiveness of the demonstrated transformer-based reconstruction, the authors compare imaging performance of the bare metalens, the transformer-based reconstruction, and several traditional image enhancement algorithms and convolutional neural networks.

High-quality full-color imaging has been a long-standing goal of meta-optics, and this work represents a significant development towards that goal. Besides the simultaneous accomplishment of wide FoV and full-color imaging with meta-optics, this work illustrates the utility of state-of-the-art neural network-based image processing techniques to achieve multi-functional imaging with meta-optics.