This PDF file contains the front matter associated with SPIE Proceedings Volume 11804, including the Title Page, Copyright information, and Table of Contents.

Join the chairs and speakers from the conference on Emerging Topics in Artificial Intelligence for a Panel Discussion on Hardware for AI.

In this talk I will introduce an integrated photonics-based tensor core unit by strategically utilizing i) photonic parallelism via wavelength division multiplexing, ii) high 2 Peta-operations-per second throughputs enabled by 10’s of picosecond-short delays from optoelectronics and compact photonic integrated circuitry, and iii) near-zero static power-consuming novel photonic multi-state memories based on phase change materials featuring vanishing losses in the amorphous state and about 1dB per state switching efficiency [Sorger Group, Appl. Phys. Rev. 2020].

For numerous Radio-Frequency applications such as medicine, RF fingerprinting or radar classification, it is important to be able to apply Artificial Neural Network on RF signals. In this work we show that it is possible to apply directly Multiply-And-Accumulate operations on RF signals without digitalization, thanks to Magnetic Tunnel Junctions (MTJs). These devices are similar to the magnetic memories already industrialized and compatible with CMOS. We show experimentally that a chain of these MTJs can rectify simultaneously different RF signals, and that the synaptic weight encoded by each junction can be tune with their resonance frequency. Through simulations we train a layer of these junctions to solve a handwritten digit dataset. Finally, we show that our system can scale to multi-layer neural networks using MTJs to emulate neurons. Our proposition is a fast and compact system that allows to receive and process RF signals in situ and at the nanoscale.

3D two-photon polymerization has shown to be an enabling tool allowing dynamic and precise printing of submicrometric optical components. Here, we focus on direct laser writing for the additive fabrication of 3D photonic waveguides, which are prime candidates for integrated, ultra-fast and parallel photonic interconnects. We here present a novel approach based on 3D optical splitters leveraging adiabatic coupling, which ensures a smooth single-mode transition between input and output waveguides. This unique 3D canonical architecture represents a clear breakthrough overcoming the long-standing challenges of parallel and scalable connections with high integration density for high-speed and energy-efficient neural networks computers.

Artificial Neural Networks (ANNs) have become a staple computing technique. Their flexibility allows them to excel in a wide range of tasks and they benefit from highly parallelized architecture by design. We experimentally demonstrate a fully parallel photonic neural network using spatially distributed modes of a large-area vertical-cavity surface-emitting laser (LA-VCSEL). All components of the ANN are fully realized in parallel hardware. We train the readout weights to perform 2 and 3-bit header recognition, XOR classification, and digital to analog conversion, and obtain low error rates for all tasks. Our system uses readily available components, is scalable to much larger sizes and to bandwidths in excess of 20 GHz.

Photonic integrated circuits allow for designing computing architectures which process optical signals in analogy to electronic integrated circuits. Therein electrical connections are replaced with photonic waveguides which guide light to desired locations on chip. Through near-field coupling, such waveguides enable interactions with functional materials placed very close to the waveguide surface. This way, photonic circuits which are normally passive in their response are able to display active functionality and thus provide the means to build reconfigurable systems. By integrating phase-change materials nonvolatile components can be devised which allow for implementing hardware mimics of neural tissue. Here I will present our efforts on using such a platform for developing optical non-von Neumann computing devices.

Neuromorphic computing takes inspiration from the brain to create energy-efficient hardware for information processing, capable of highly sophisticated tasks. Systems built with standard electronics achieve gains in speed and energy by mimicking the distributed topology of the brain. Scaling-up such systems and improving their energy usage, speed and performance by several orders of magnitude requires a revolution in hardware. We discuss how including more physics in the algorithms and nanoscale materials used for data processing could have a major impact in the field of neuromorphic computing. We review striking results that leverage physics to enhance the computing capabilities of artificial neural networks, using resistive switching materials, photonics, spintronics and other technologies. We discuss the paths that could lead these approaches to maturity, towards low-power, miniaturized chips that could infer and learn in real time.

We analyze the information processing capacity of diffractive optical networks to reveal that increasing the total number of diffractive features, i.e., neurons, within a network linearly increases the dimensionality of the complex-valued linear transformation space of the network, up to a limit dictated by the input and output fields-of-view. We further show that deeper diffractive neural networks formed by larger numbers of diffractive surfaces can cover a higher-dimensional subspace of the complex-valued linear transformations between a larger input field-of-view and a larger output field-of-view, increasing the learning capability and approximation power of the optical network.

We report a single-pixel machine vision framework based on deep learning-designed diffractive surfaces to perform a desired machine learning task. The object within the input field-of-view is illuminated with a broadband light source and the subsequent diffractive surfaces are trained to encode the spatial information of the object features onto the power spectrum of the diffracted light that is collected by a single-pixel detector in a single-shot. We experimentally demonstrated the all-optical inference capabilities of this single-pixel machine vision platform by classifying handwritten digits using 3D-printed diffractive layers and a plasmonic nanoantenna-based time-domain spectroscopy setup operating at THz wavelengths.

One of the possible future architectural implementations of new computing devices is brain-inspired neuromorphic computers. Artificial synapse is one of the key neuromorphic computing elements. This work is devoted to the search for new bioinspired artificial synapse properties and the demonstration of already known neuromorphic properties on the original photoelectric synapse based on nanocrystalline ZnO film. Photoelectric synapse demonstrated basic neuromorphic properties: spike signals operation, the presence of short-term memory, long-term memory and paired-pulse facilitation. Artificial photoelectric synapse adaptation properties have been demonstrated in a series of experiments with different conductivity cutoff levels.

Optics is increasingly considered for machine learning, in particular inference, thanks to its intrinsic scalability, speed, and low consumption. Free space implementation are particularly interesting, for instance to implement convolutions or Fourier Transforms. Meanwhile, light propagation of complex media has evolved as a very active field, in particular for imaging. It has been shown that the propagation of a laser through a complex disordered medium is akin to a large size random matrix multiplication, an operation ubiquitous in many instances of signal processing and machine learning . We have recently studied how to exploit such optical implementation of random matrix multiplication for several applications. I will present a few examples for classification, time-series prediction, and for acceleration an Ising Machine.

Event-activated biological-inspired subwavelength (sub-λ) optical neural networks are of paramount importance for energy-efficient and high-bandwidth artificial intelligence (AI) systems. Despite the significant advances to build active optical artificial neurons using for example phase-change materials, lasers, photodetectors, and modulators, miniaturized integrated sources and detectors suited for few-photon spike-based operation and of interest for neuromorphic optical computing are still lacking. In this invited paper we outline the main challenges, opportunities, and recent results towards the development of interconnected neuromorphic nanoscale light-emitting diodes (nanoLEDs) as key-enabling artificial spiking neuron circuits in photonic neural networks. This method of spike generation in neuromorphic nanoLEDs paves the way for sub-λ incoherent neural circuits for fast and efficient asynchronous brain-inspired computation.

We improve the inference performance of diffractive deep neural networks (D2NN) for image classification by utilizing ensemble learning and feature engineering. Through a novel pruning algorithm, we designed an ensemble of e.g., N=14 D2NNs that collectively achieve a blind testing accuracy of 61.14% on the classification of CIFAR-10 images, which provides an improvement of >16% compared to the average performance of the individual D2NNs within the ensemble. These results constitute the highest inference accuracies achieved to date by any diffractive network design and would be broadly useful to create diffractive optical machine learning systems for various imaging and sensing needs.

A novel optical computing framework is presented by harnessing spatiotemporal nonlinear effects of multimode fibers for machine learning. With linear and nonlinear interactions of spatial fiber modes, a powerful computation engine is experimentally realized. We demonstrated excellent performance with the present optical scheme for various classification tasks. We demonstrated that spatiotemporal fiber nonlinearities perform as well as digital neural network structures for challenging computational tasks. With better energy efficiency and easy scalability, our method presents a novel path toward powerful optical computation.

We quantify the sensitivity of diffractive optical networks’ inference accuracy against input object variations in the form of translation, rotation, and scaling, and present a new training methodology that enables diffractive networks to maintain their classification performance despite such object variations at the input field-of-view. Our analyses on all-optical classification of handwritten digits reveal that this new training scheme provides blind inference accuracy gains of >50%, >30% and >30% for randomly shifted, rotated and scaled input objects, respectively, demonstrating its efficacy. These results are important for using diffractive optical networks in various machine vision applications involving dynamic objects and environments.

From the outset of Cajal’s studies in 1888, he provided strong support for his belief that dendrites and axons end freely in the nervous system and communicate by contact through the synapses. Thanks to the introduction of transmission electron microscopy in the 1950s, along with the development of methods to prepare nervous tissue for ultrastructural analysis, the nature of synapses was finally examined, confirming that the pre-synaptic and the post-synaptic elements in nervous tissues are physically separated by the synaptic cleft. In this talk, I will summarize the advances in the methods used to map the brain’s synapses. I propose that the best approach to study synaptic connectivity is tolink detailed structural data with light and electron microscopy wiring diagrams, and integrate this information with genetic, molecular and physiological data. This integration will allow formulating new hypotheses and generating models to make predictions about neurodegenerative diseases.

Our ability to study large quantities of synapses at the single synapse resolution has expanded over the last few years. For example, methods allow the study of +100 million synapses across an entire sagittal section of the mouse brain (Zhu et al, Neuron, 2018) using confocal microscopy or +10.000 synapses in mouse hippocampus CA1 using electron microscopy (Santuy et al., Sci Rep, 2020), showing a remarkable diversity of synapse types across brain regions. Furthermore, developments of e.g. genetically encoded fluorescent proteins or calcium indicators as well as novel super-resolution microscopy opens up the possibility of observing synaptic structure and function in vivo. In this session, we will discuss what these new methods tell us about synapses and hence about brain structure and function.

LSFM is a technique that allows obtaining fast 2D images of biological samples. Its characteristic 90° geometry results in a highly efficient excitation and light collection of the generated signal, minimizing light dose onto the sample and reducing phototoxicity effects. Furthermore, by displacing the sample through the light sheet, high-resolution 3D images can also be obtained. Therefore, LSFM has been put forward as an interesting candidate for fast volumetric brain imaging. Here, I will present our results for 3D imaging of the spontaneous and dynamic calcium activity in primary neuron cultures in hydrogels. The obtained data is then processed to calculate the connectivity maps in the 3D neuron cultures in hydrogels and assess the topological properties of these maps such as the modules or highly connected subnetworks. This abstract is part of the symposium: "Diagnosis and Prediction of Neurodegenerative Diseases using Artificial Intelligence"

Availability of the high-performance computing resources makes possible simulations of the neural networks at unprecedented resolution and high biological realism. Creating a microcircuit of the dorsal striatum is discussed on the example of the mouse brain. Models of striatal projection neurons and interneurons are built using multi-objective optimization procedure. Special attention is paid to ensure variability of morpho-electric features within the simulated population of the nerve cells. Synaptic connectivity within the reconstructed microcircuit is anatomically constrained by the morphology of individual cells and further tuned to match the experimentally observed connection probability for all known local projections. Reconstructed microcircuit faithfully reproduces background activity of the striatum as well as the typical response to the external stimulation and can be used to study the striatal dynamics in healthy and diseased states.

In the last decade, there has been an unprecedented development in topological data analysis tools to encode geometrical information. These tools have been successful for extracting new information from brain networks with a range of methods such as persistent homology, which provides a framework for obtaining higher order topological features from the brain that go beyond modular structures, and include homological significant aspects such as number of holes and cycles. In this talk I will give an introduction to what these techniques have to offer and present illustrative examples. I will focus particularly on how to extract statistically viable conclusions based on combining topological data analysis methods with subsampling techniques and discuss potential applications of these methods in the context of neurodegenerative diseases. This abstract is part of the symposium: "Diagnosis and Prediction of Neurodegenerative Diseases using Artificial Intelligence and Simulations".

The organization of the Alzheimer's disease (AD) connectome has been studied using single neuroimaging modalities. However, different neuroimaging modalities are not independent and often interact with each other in the course of AD. Here, we integrate the networks obtained from T1-weighted and 18F-Florbetapir PET to build a multiplex connectome using BRAPH 2.0 and assess how it changes across different AD stages. We assessed the overlapping strength, multilayer communities, overlapping connections, and the multiplex participation and clustering coefficients. There was a reorganization of the communities across the four groups and we found significant changes reflecting a loss of multiplex hubs and overlapping connections in medial frontal and occipital areas in the patients’ groups. These findings indicate that multiplex network changes can be useful to understand the relationship between amyloid pathology and gray matter atrophy occurring over the course of AD.

A central research topic in medical image processing is the development of imaging biomarkers, i.e. image-based numeric measures of the degree (or probability) of disease. Typically, they rely on segmentation of an anatomical or pathological structure in a radiological image, followed by quantitative measurement. With much of traditional image processing methods being supplanted by machine learning techniques, the identification of new imaging biomarkers is also often made with such techniques, in particular deep learning. Successful examples include quantitative assessment of Alzheimer’s disease and Parkinson’s disease based on brain MRI data, as well as image-based brain age estimation.

A growing number of studies suggest that detection of Alzheimer’s disease can be improved by using information derived from distinct neuroimaging modalities. However, so far it remains unresolved how these modalities can be combined within a deep learning model approach. In this study, we proposed a deep-neural-network model GapNet that can work with incomplete dataset including baseline and longitudinal MR, amyloid-PET, and FDG-PET data. We verified the effectiveness of GapNet by comparing it to the conventional Vanilla neural networks and specifically testing their performance in discriminating between healthy controls and individuals with amyloid changes, which is an important early pathological marker in Alzheimer’s Disease. Results showed that, compared to the Vanilla networks, GapNet achieved higher classification accuracy. In sum, our finding suggested that the GapNet model is a promising deep learning approach for detecting Alzheimer’s disease with multi-modal neuroimaging

Large datasets of longitudinal data, made available for the neuroimage community, offer the possibility to study trajectories of biomarkers throughout the course of the diseases. For such modelling, approaches emerging from both the frequentist and the Bayesian frameworks have been suggested. When datasets are large, homogeneous, and balanced, both approaches seem to perform similarly. However, in compromised datasets, with limited number of samples and unbalanced data, this is not clear. Here we included data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database to study the behaviour of different statistical approaches using different dataset configurations. We used the hippocampal volume (HV) at different timepoints and we analysed the capability of the methods to find differences across clinical groups. For that, we used a Linear Mixed Effects (LME) modelling under both the frequentist and the Bayesian approaches. We started with a large, homogeneous, and symmetric database, and we created different configurations, by sequentially removing data points, to simulate different real-life situations. Using the frequentist approach to predict conversion on mild cognitively impaired patients, we found that we need a mean of 115 subjects to differentiate converters vs non converters. When classifying between the five ADNI clinical groups we need 147 subjects (mean across datasets) to differentiate between all clinical groups. With the Bayesian approach, we demonstrated that the results were stronger and of higher interpretability, specially at the borderline significant datasets.

The brain is a complex network that relies on the interaction between its various regions, known as the connectome. The human connectome undergoes complex changes with aging and shows differences in many functional network measures between men and women; however, the effects of aging and sex on the brain connectome are not well characterized. In this study, we assess functional connectivity changes in a large cohort of men and women between 45 and 79 years of age using conventional methods as well as a novel approach based on multilayer brain connectivity. Our findings provide a deeper insight into the sex differences that occur in functional connectivity over the course of aging. Moreover, they indicate that multilayer networks provide a natural way to integrate the information from positive and negative functional connections, providing important information on the effects of sex and age on network topology.

This abstract is part of the symposium: "Diagnosis and Prediction of Neurodegenerative Diseases using Artificial Intelligence and Simulations". In this talk the different approaches discussed during the symposium will be summarized. In brief, information and communication technology (ICT) approaches are an important tool to manage, analyze and integrate the data derived from the brains of patients with neurodegenerative diseases. However, the challenge that these approaches now face is to integrate all the multi-scale changes in brain structure and dynamics that occur in these diseases, from alterations at the molecular level to changes in large-scale brain networks. Another important challenge that ICT approaches must solve is to determine the changes that play a causal role in these diseases, are compensatory, or simply correlated to the disease condition. I will provide an overview of this integrative multi-scale approach and show concrete examples that highlight its feasibility.

Join the chairs and speakers from the conference on Emerging Topics in Artificial Intelligence for a Panel Discussion on AI in Neurosciences.

In recent years, deep learning has been used widely to solve a variety of digital microscopy problems. We present ZEUS as a method to correct out of focus aberrations and denoise light-sheet microscopy images. First, a convolutional neural network is used to estimate the aberrations in terms of Zernike coefficients. Then those values are used to train a UNET that outputs corrected images from noisy and aberrated ones. With this approach, we can access scanning frequencies and image qualities equivalent to the most advanced LSM systems without the need for costly equipment and complex optical setups.

The blind-SIM algorithm reconstructs super-resolution images using random illumination patterns. Recently, Speckle-MAIN has shown that this algorithm, when combined with high resolution near-field illumination patterns, can extend SIM resolution down to 40nm. However, the use of the iterative blind-SIM algorithm is computationally expensive, time-consuming and is prone to artifacts. We demonstrate that using a deep neural network we can achieve similar or better reconstruction results compared to blind-SIM with fewer artifacts and orders of magnitude better reconstruction time. This work makes real-time Speckle-MAIN super resolution imaging possible.

Characterization of nanoparticles in their native environment plays a central role in a wide range of fields, from medical diagnostics and nanoparticle-enhanced drug delivery to nanosafety and environmental nanopollution assessment. I will present a label-free method to quantify size and refractive index of individual nanoparticles using two orders of magnitude shorter trajectories than required by standard methods, and without prior knowledge about the physicochemical properties of the medium. This is achieved through a weighted average convolutional neural network which analyzes holographic scattering images of single particles. I will demonstrate how deep learning enhanced holography opens up completely new possibilities to temporally characterize particle interactions and particle properties in complex environments.

We report a recurrent neural network (RNN)-based cross-modality image inference framework, termed Recurrent-MZ+, that explicitly incorporates two or three 2D fluorescence images, acquired at different axial planes, to rapidly reconstruct fluorescence images at arbitrary axial positions within the sample volume, matching the 3D image of the same sample acquired with a confocal scanning microscope. We demonstrated the efficacy of Recurrent-MZ+ on transgenic C. Elegans samples; using 3 wide-field fluorescence images as input, the reconstructed sample volume by Recurrent-MZ+ mitigates the deformations caused by the anisotropic point-spread-function of wide-field microscopy, and matches the ground truth confocal image stack of the sample.

Fluorescence imaging is used throughout biological research to identify subcellular structures, detect neural activity, and differentiate cell types. Multi-channel fluorescence is a challenging subset of fluorescence imaging where multiple fluorescent modes are emitted simultaneously, allowing the detection of a multitude of elements within the specimen (for example, multiple types of neurons). In our work, we demonstrate a learned sensing approach to realize virtual multi-channel fluorescence, by jointly optimizing image illumination and a deep learning neural network that infers labels from brightfield images. We used our setup to demonstrate the influence of key design decisions, such as model architecture, choice of loss function, and amount of input images, on the final optical design. We expect that our work can provide a better understanding of building machine learning based imaging systems and demonstrate the scalability of our illumination optimization technique.

DeepTrack is an all-in-one deep learning framework for digital microscopy, attempting to bridge the gap between state of the art deep learning solutions and end-users. It provides tools for designing samples, simulating optical systems, training deep learning networks, and analyzing experimental data. We show the versatility of deep learning by solving a wide field of common problems in microscopy. Our hope is to serve as a platform for researchers to launch their solutions for the benifit of the entire field.

There is a very limited number of methods to analyze experimental trajectories of systems with feedback and time delay. In most cases, an analytical approach is not even possible. In this study, we show that the feedback parameters and the delay can be accurately characterized using machine learning, namely recurrent neural networks. We demonstrate that our method can dramatically expand the number of time-delayed feedback scenarios that we can characterize. We exemplify our findings on different numerical and experimental scenarios.

Even though in most cases optical forces can be calculated semi-analytically, the computation becomes prohibitively slow in problems where the calculation needs to be repeated several times. Starting from a spherical particle in an optical trap, we show how machine learning can be used to improve not only the speed but also the accuracy of the optical force calculations in the geometrical optics approach. This is demonstrated to work efficiently at least up to 9 degrees of freedom, constituting a tool for exploring problems that were out of the scope of the traditional geometrical optics calculation.

Deviations from Brownian motion leading to anomalous diffusion control transport mechanisms in many fields, from ecology to quantum physics. The detection of anomalous diffusion from an individual trajectory is a challenging task, which traditionally relies on calculating the mean square displacement. This approach finds its limits for cases of practical interest, e.g. short/noisy trajectories or ensembles of heterogeneous trajectories. Recently, new approaches have been proposed, mostly building on the ongoing machine-learning revolution. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition. Participants applied their own algorithms independently to a commonly defined data set including diverse scenarios. Although no single method performed best across all conditions, the results revealed clear differences between the various approaches, providing practical advice for users and a benchmark for developers.

A high-speed three-dimensional digital holographic reconstruction algorithm is proposed based on the YOLO architecture, which is able to significantly accelerate the training process. Supervised learning is used to train the network using both simulated and experimental holograms. With the aid of transfer learning, a small set of 2D holograms is sufficient to train the network. The trained network can also be used to label new holograms. These holograms in turn can help train the networks to improve the robustness. It takes hours for the training process, which is more efficient than the previously proposed networks with several days for the same dataset. The network has great potential for high-dynamic scenes and is robust to background noise in the particle field reconstruction.

3D particle-localization using in-line holography is a fundamental problem with important applications. It involves estimating the unknown positions of scatterers in a 3D volume from a single 2D hologram. We propose a deep learning based framework that is highly computationally efficient for large-scale 3D reconstructio and demonstrates accurate results for a wide variety of scattering scenarios. The proposed approach incorporates physical scattering information into the result via 3D backpropagation of the hologram, followed by artifact removal with an end-to-end 3D deep neural network (DNN). To address the challenge of limited data availability, we train our DNN solely on simulated data, and show that it works accurately for experimental data as well. The results show that our DNN is able to accurately localize particles under various scattering scenarios with little computational overhead.

Anomalous diffusion occurs in many physical and biological phenomena, when the growth of the mean squared displacement with time has an exponent different from one and can be due to different mechanisms. We show that recurrent neural networks (RNNs) efficiently characterize anomalous diffusion by identifying the mechanism causing it and determining the anomalous exponent from a single short trajectory. This method outperforms standard techniques and advanced ones when the available data points are limited, as is often the case in experiments. Furthermore, RNNs can handle more complex tasks where there are no standard approaches, such as determining the anomalous diffusion exponent from a trajectory sampled at irregular times, and measuring intermittent systems that switch between different kinds of anomalous diffusion. The method is validated on experimental data obtained from subdiffusive colloids trapped in speckle light fields and superdiffusive microswimmers.

Artificial Intelligence methods can be very effective in classification tasks that involve the processing of ordered sequences of data. Here we explore two different approaches to tackle the problem of ovarian cancer detection from a sequence of longitudinal measurements of several biomarkers. The first approach relies on a Bayesian hierarchical model whose fundamental assumption is that measurements taken from case subjects exhibit a changepoint in one or several biomarkers. The second approach is a purely discriminative machine learning algorithm based on the use of recurrent neural networks, a kind of artificial neural network specially suited to the processing of inputs of different lengths.

We present an application of machine learning to deal with the optimization of testing strategies in the event of large-scale epidemic outbreaks. We describe the disease using the archetypal SIR model. Cost-effective containment relies on making the best possible use of the available resources to identify infectious cases. We present a neural-network-powered strategy that adapts to an epidemic without knowing the underlying parameters of the model. The neural network results are more effective than standard approaches, also in the presence of asymptomatic cases. We envision that similar methods can be employed in public health to control epidemic outbreaks.

Familial hypercholesterolemia (FH) is the most common genetic disorder of lipid metabolism. The gold standard for FH diagnosis is genetic testing, available, however, only in selected university hospitals. Clinical scores – for example, the Dutch Lipid Score – are often employed as alternative, more accessible, albeit less accurate FH diagnostic tools. To overcome the limitations of these traditional methods and to obtain a more reliable approach to FH diagnosis we implement a “virtual” genetic test using machine-learning approaches.

This presentation recording was created for the SPIE Optics + Photonics 2021 Symposium

Automated cell counting in in-vivo specular microscopy images is challenging, especially when single-cell segmentation methods fail due to corneal dystrophy. We aim to obtain reliable cell segmentation from specular microscopy images of both healthy and pathological corneas. Here, we cast the problem of cell segmentation as a supervised multi-class classification problem. Hence, the goal is to learn a mapping relation between an input specular microscopy image and its labeled counterpart, identifying healthy (cells) and dysfunctional regions (e.g., guttae). Using a generative adversarial approach, we trained a U-net model by extracting 96×96 pixel patches from corneal endothelial cell images and the corresponding manual segmentation by a group of expert physicians. Preliminary results show the method's potential to deliver reliable feature segmentation, enabling more accurate cell density estimations for assessing the cornea's state.

Glioblastoma multiforme (GBM) is one of the most aggressive primary brain tumors with its extreme proliferation and invasiveness. U87 human glioma cell line is one of the best representative cell lines for GBM with its extremely heterogenous and frequently altered morphologies. Quantification of heterogeneity and morphological changes of U87 glioma cells are mostly based on manual analysis. Therefore, automated image segmentation and analysis approaches are crucial. Here, we implemented U-Net algorithm for segmentation of U87 glioma cells and obtained 0.06% loss and 97.3% accuracy values. Next, we integrated Chan-Vese, K-means, and Morphological Filtering. Finally, we compared the performances of these approaches. We believe that our preliminary data might contribute to development of automated, reliable, accurate, and cell type specific image segmentation tools.

Quantitative analysis of cell structures is essential for pharmaceutical drug screening and medical diagnostics. This work introduces a deep-learning-powered approach to extract quantitative biological information from brightfield microscopy images. Specifically, we train a conditional generative adversarial neural network (cGAN) to virtually stain lipid droplets, cytoplasm, and nuclei from brightfield images of human stem-cell-derived fat cells (adipocytes). Subsequently, we demonstrate that these virtually-stained images can be successfully employed to extract quantitative biologically relevant measures in a downstream cell-profiling analysis. To make this method readily available for future applications, we provide a Python software package that is available online for free access.

Quantum reservoir computing is an unconventional computing approach that exploits the quantumness of physical systems used as reservoirs to process information, combined with an easy training strategy. An overview is presented about a range of possibilities including quantum inputs, quantum physical substrates and quantum tasks. Recently, the framework of quantum reservoir computing has been proposed using Gaussian quantum states that can be realized e.g. in linear quantum optical systems. The universality and versatility of the system makes it particularly interesting for optical implementations. In particular, full potential of the proposed model can be reached even by encoding into quantum fluctuations, such as squeezed vacuum, instead of classical intense fields or thermal fluctuations. Some examples of the performance of this linear quantum reservoir in temporal tasks are reported.

We analyze the fundamental impact of noise propagation in deep neural network (DNN) comprising nonlinear neurons and with connections optimized by training. Our motivation is to understand the impact of noise in analogue neural network realizations. We consider the influence of additive and multiplicative, correlated and uncorrelated types of internal noise in DNNs. We find general properties of the noise impact depending on the noise type, activation function, depth and the statistics of connection matrices and show that noise accumulation can be efficiently avoided. Our work is based on analytical methods predicting the noise levels in all layers of the network.

We propose a general neural-network based learning framework to solve highly ill-posed problems to predict a system’s forward and backward response function. Such an approach has applications in target-oriented system’s control in fields such as, optics, neuroscience and robotics. The proposed method is able to find the appropriate continuous space input of a system that results in a desired output, despite the input-output relation being nonlinear, the system being time-variant and\or with incomplete measurements of the systems variables and lack of labeled data required for supervise learning.

We propose an optical implementation of an Extreme Learning Machine (ELM) inspired by frequency-multiplexing techniques previously employed for Reservoir Computing. The input layer of the ELM is encoded in the lines of a frequency comb and the hidden layer is generated by making comb lines interfere. Multiplication by output weights can be performed optically. This approach combines the potential high-speed, low-power and paral- lelization advantages of Optical Neural Networks with the cheap training (both in terms of speed and power) of ELMs, which do not require slow gradient descent and error backpropagation algorithms. We present preliminary experimental results compared with simulations.

Imaging through scattering medium has wide applications across many areas. Here, we present a new deep learning framework for improving the robustness against physical perturbations of the scattering medium. The trained DNN can make high-quality predictions beyond the training range which is across 10X depth-of-field (DOF). We develop a new analysis framework based on dimensionality reduction for revealing the information contained in the speckle dataset, interpreting the mechanism of our DNN, and visualizing the generalizability of the DNN model. This allows us to further elucidate on the information encoded in both the raw speckle measurements and the working principle of our speckle-imaging deep learning model.

Fringe Projection Profilometry (FPP) with Digital Light Projector technology is one of the most reliable 3D sensing techniques for biomedical applications. However, besides the fringe pattern images,often a color texture image is needed for an accurate medical documentation. This image may be acquired either by projecting a white image or a black image and relying on ambient light. Color Constancy is essential for a faithful digital record, although the optical properties of biological tissue make color reproducibility challenging. Furthermore, color perception is highly dependent on the illuminant. Here, we describe a deep learning-based method for skin color correction in FPP. We trained a convolutional neural network using a skin tone color palette acquired under different illumination conditions to learn the mapping relationship between the input color image and its counterpart in the sRGB color space. Preliminary experimental results demonstrate the potential for this approach.

Multiple photonic systems show great promise for providing practical yet powerful hardware substrates for neuromorphic computing. Among those, delay-based systems offer -through a time-multiplexing technique - a simple technological implementation route. We discuss our advances in the development of passive coherent fibre-ring cavities and semiconductor lasers with integrated delay for reservoir computing. Time-multiplexed systems are also highly suitable for coherent Ising machines as they allow to implement a fully interconnected large scale system with few components. We have recently proposed a system based on opto-electronic oscillators subjected to self-feedback with improved calculation time and solution quality.

We introduce an optical neural-network architecture for edge computing that takes advantage of wavelength multiplexing, high-bandwidth modulation, and integration detection. Our protocol consists of a server and a client, which divide the task of neural-network inference into two steps: (1) a difficult step of optical weight distribution, performed at the server and (2) an easy step of modulation and integration detection, performed at the edge device. This arrangement allows for large-scale neural networks to be run on low-power edge devices accessible by an optical link. We perform simulations to estimate the speed and energy limits of this scheme.

Recent experimental results show how classical accelerators based on analog computing can outperform quantum annealing alternatives in benchmark tasks that require dense connection matrices. In Hewlett Packard Labs, we have been studying two alternatives: integrated coherent Ising machines and mem-HNNs (based on memristive crossbar arrays). An important challenge for commercial viability is that different industrial workloads typically benefit from the availability of a variety of optimization algorithms and require a broad range of template combinatorial optimization problems. In this talk, we will discuss our recent progress in going beyond Max-Cut, and we will propose a broader range of algorithms. This flexibility in algorithm choices and template problems is an important step forward to address the wide variety of enterprise-level use-cases such as airline scheduling, supply chain optimization, real-time bandwidth management, gene sequencing, etc.

We present a deep learning-aided imaging system for early detection and classification of live bacterial colonies by capturing time-lapse holographic images of an agar plate and analyzing these images using deep neural networks. We blindly tested our system by identifying Escherichia coli and total coliform bacteria in spiked water samples and successfully detected 90% of the bacterial colonies within 7-10 h, while keeping 99.2~100% precision. We further classified the corresponding species within 7.6-12 h of incubation with 80% accuracy, which represents >12 h time-savings. Our system also achieved a limit-of-detection of ~1 CFU/L within 9 h of total test time.

Phytoplankton are responsible for approximately 50% of the biological fixation of carbon dioxide and oxygen production on Earth. The majority of the phytoplankton production is consumed by single-celled microscopic grazers, microzooplankton. In this experiment, we reproduce a small-scale alias of the plankton world to understand their feeding behavior. We use a lens-less holographic approach, driven by deep learning powered DeepTrack 2.0. We use a combination of U-net and CNN architectures to decipher the properties of radius, refractive index, heights and drymass of plankton. We further compare the results with standard approaches.

We utilize diffractive optical networks to design small footprint, passive pulse engineering platforms, where an input terahertz pulse is shaped into a desired output waveform as it diffracts through spatially-engineered transmissive surfaces. Using 3D-printed diffractive networks designed by deep learning, various terahertz pulses with different temporal widths are experimentally synthesized by controlling the amplitude and phase of the input pulse over a wide range of frequencies. Pulse width tunability was also demonstrated by changing the layer-to-layer distance of a 3D-printed diffractive network or by physically replacing 1-2 layers of an existing network with newly trained and fabricated diffractive layers.

In a current study, we have developed a cheap and easy-to-use urine analysis method using visible and near-infrared wavelength range optical transmission spectra using artificial intelligence approaches. The manufactured prototype based on an 18-channel spectrometer and LED light sources, was used to measure 431 patients’ urine transmission spectra. 19 parameters clinical urine analysis was performed in a medical laboratory for each patient. Machine learning partial least squares discriminant analysis (PLS-DA) was used to solve the binary multidimensional classification problem. Developed machine learning model could detect urine pathological changes with sensitivity and specificity comparable to laboratory diagnostic methods for most parameters.

We present a systematic approach to the design of an analog implementation of photonic reservoir computing. The scheme builds on the idea that, thanks to time-multiplexing, a single nonlinear node subject to a delayed feedback loop can emulate a network with ring-like topology. We go beyond previous approaches for analog photonic reservoir computers by considering a ow model (continuous time) of the corresponding optoelectronic implementation, instead of the usually considered map limit, as the continuous time approach allows for operating at faster modulations. We focus on the implementation of an analog output layer made of a modulator and a second order filter that makes any digital post-processing unnecessary. Numerical simulations of the system show that the suggested analog design of the analog output layer is robust towards potential experimental deviations such as time jitter. The combination of the optoelectronic implementation with an analog output layer allows for high-speed information processing.

Join the chairs and speakers from the conference on Emerging Topics in Artificial Intelligence for a Panel Discussion on Applications of AI.

Join the chairs and speakers from the conference on Emerging Topics in Artificial Intelligence for a Panel Discussion on Opportunities for Young Researchers.

In this study, we propose a deep-learning approach to establish the lithographic model for i-line photolithography and develop an optical proximity correction (OPC) algorithm to increase the resolution limit. The applications of RETs are not only on CMOS semiconductor, but also on some metasurface which used to patterning by electron beam lithography. With the OPC algorithm, we are able to manufacture a near-infrared metalens patterning by i-line photolithography in a more efficient and less expensive way.

We present a novel method for constructing quantum dot arrays using optical tweezers. By optically trapping 10 nm core-shell quantum dots we can position the quantum dots with submicron precision. The quantum dots are suspended in a resin (nanoscribe IP-G 780) which is then polymerized locally around the trapped quantum dot, fixating its position. The process of trapping and positioning is automated using a neural network to locate both free quantum dots and the position of quantum dots already in the array. The ability to locate the already positioned quantum dots is essential to achieving high precision and accuracy in the placement. Automation makes the process scalable and enables the manufacturing of large arrays. As a first step we demonstrate the construction of a 4x4 array of quantum dots.

In vitro cell culture relies on that the cultured cells thrive and behave in a physiologically relevant way. A standard method to evaluate their behavior is to perform chemical staining in which fluorescent probes are added to the cell culture for further imaging and analysis. However, such technique is invasive and sometimes even toxic to cells, hence, alternative methods are requested. Here, we describe an analysis method for detecting and discriminating live, dead, and apoptotic cells using deep learning. Such an approach will be less labor-intensive than traditional chemical staining procedures and will enable cell imaging with minimal impact.

We report a deep learning-based virtual image refocusing method that utilizes double-helix point-spread-function (DH-PSF) engineering and a cascaded neural network model, termed W-Net. This method can virtually refocus a defocused fluorescence image onto an arbitrary axial plane within the sample volume, enhancing the imaging depth-of-field and lateral resolution at the same time. We demonstrated the efficacy of our method by imaging fluorescent nano-beads at various defocus distances, and also quantified the nano-particle localization performance achieved with our virtually-refocused images, demonstrating ~20-fold improvement in image depth-of-field over wide-field microscopy, enabled by the combination of DH-PSF and W-Net inference.

Understanding what happens in the brain during healthy ageing is important for several reasons ranging from its correlation to cognitive function to the research of neurodegenerative diseases and their biomarkers. In recent years, studies have linked certain biomarkers to both ageing and neurodegenerative diseases, but there is still more to be found and concluded. Inspired by current developments in machine learning, the use of artificial neural networks (ANN) as a method for extracting new informational biomarkers has been investigated. The brain consists of several functionally connected regions and can thus be modeled and analyzed using graph theory. ANNs and graph theory can be combined into graph neural networks, which when applied on human brain data have shown great promise. We aim to investigate the usage of GNNs on age-related data, in the search for biomarkers and other important features related to the ageing brain.

Recent advances in network neuroscience have provided new insights into brain organization in health and disease. In particular, graph theory analyses of brain networks have shown that the human brain is characterized by a high level of integration between distant brain regions and good local communication between neighboring areas. However, these brain networks are normally analyzed using single neuroimaging modalities such as functional magnetic resonance or diffusion tensor imaging. Machine learning techniques for graph structures, such as Graph Neural Networks (GNN), are used to infer and predict from the graph data. Here we present BRAPH 2.0 (BRain Analysis using graPH theory version 2.0 ), which is a major update of the first version. BRAPH 2..0 Genesis utilizes the capability of an object-oriented programming paradigm and a new engine to provide clear, robust, clean, modular, maintainable, testable, and machine learning ready code.