This PDF file contains the front matter associated with SPIE Proceedings Volume 12522, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

For the last decade, Harbor Branch Oceanographic Institute at Florida Atlantic University (HBOI) has been developing Integrated Multi-Trophic Aquaculture (IMTA), where multiple species are farmed together. Compared with the traditional Recirculating Aquaculture Systems (RAS), the IMTA system can improve efficiency, reduce waste, and provide ecosystem services. For the IMTA system to be successful at a commercial farm scale, HBOI is developing an AI-centric Internet of Things framework to support the operations of the IMTA system. The Pseudorandom Encoded Light for Evaluating Biomass (PEEB) sensor is an endeavor in this effort to realize automated monitoring of the growth of the Sea Lettuce (Ulva lactuca), an important organism in the HBOI IMTA system. PEEB utilizes the measurements from a sequence of encoded light flashes to quantify the seaweed biomass. Such a configuration ensures the sensor can operate under different ambient light conditions and biomass densities. An improved PEEB sensor based on a unified electronic sensor design that is more robust against ambient conditions and capable of long-range data transmission is discussed. This electronic design will be the backbone to support future sensors for the IMTA system. Multiple PEEB sensors have been deployed at the HBOI IMTA system. The cloud-based storage and analysis of the sensor data are discussed.

Incremental Task learning (ITL) is a category of continual learning that seeks to train a single network for multiple tasks (one after another), where training data for each task is only available during the training of that task. Neural networks tend to forget older tasks when they are trained for the newer tasks; this property is often known as catastrophic forgetting. To address this issue, ITL methods use episodic memory, parameter regularization, masking and pruning, or extensible network structures. In this talk, we present a new incremental task learning framework based on low-rank factorization. In particular, we represent the network weights for each layer as a linear combination of several rank-1 matrices. To update the network for a new task, we learn a rank-1 (or low-rank) matrix and add that to the weights of every layer. We also introduce an additional selector vector that assigns different weights to the low-rank matrices learned for the previous tasks. We show that our approach performs better than the current state-of-the-art methods in terms of accuracy and forgetting. Our method also offers better memory efficiency compared to episodic memory- and mask-based approaches.

Fusion of multimodal data can offer enhanced machine learning. One of the most common fusion approaches in deep learning is end-to-end training of a neural network on all available modalities. However, paired multimodal data from all the modalities is required to train such a network. Collecting paired data from multiple modalities can be challenging and expensive due to the requirement of specialized equipment, atmospheric conditions, limitation of individual modalities to probe a scene, data integration from modalities with different spatial and spectral resolutions, and annotation challenges for obtaining ground truth. A two-phase multi-stream fusion approach is presented in this work to counteract this issue. First, we train the unimodal streams in parallel with their own decision layers, loss, and hyper-parameters. Then, we discard the individual decision layers, concatenate the last feature map of all unimodal streams, and jointly train a common multimodal decision layer. We tested the proposed approach on the NTIRE-21 dataset. Our experiments corroborate that in multiple cases, the proposed method can outperform the alternatives.

Recent years have witnessed great advances in deep learning-based image compression, also known as learned image compression. An accurate entropy model is essential in learned image compression, since it can compress high-quality images with a lower bit rate. Current learned image compression schemes developed entropy models using context models and hyperpriors. Context models utilize local correlations within latent representations for better probability distribution approximation, while hyperpriors provide side information to estimate distribution parameters. Most recently, several transformer-based learned image compression algorithms have emerged and achieved state-of-the-art rate distortion performances, surpassing existing convolutional neural network (CNN)- based learned image compression and traditional image compression. Transformers are better at modeling long-distance dependencies and extracting global features than CNNs. However, the research of transformer-based image compression is still in its early stage. In this work, we propose a novel transformer-based learned image compression model. It adopts transformer structures in the main image encoder and decoder and in the context model. In particular, we propose a transformer-based spatial-channel auto-regressive context model. Encoded latent-space features are split into spatial-channel chunks, which are entropy encoded sequentially in a channelfirst order, followed by a 2D zigzag spatial order, conditioned on previously decoded feature chunks. To reduce the computational complexity, we also adopt a sliding window to restrict the number of chunks participating in the entropy model. Experimental studies on public image compression datasets demonstrate that our proposed transformer-based learned image codec outperforms traditional image compression and existing learned image compression models visually and quantitatively.

Ethical data splitting is of paramount importance to ensure the validity of any solution that is based on data. If data is biased, it will not accurately represent how the solution will solve the problem. To ethically split data, the overall variance of the data needs to be fairly represented in the training and the testing sets of the dataset. To do this, the outliers of the data need to be determined so that they can be accounted for when splitting the data. Finding the principal components of the data using the L2-norm has been shown as an effective way to identify outliers of data to make a robust dataset that is resistant to outliers. It has been shown that the L1-norm is more resistant to outliers than the L2-norm, so it will allow the dataset to become more resistant to outliers. Therefore, utilizing L1-norm principal components when determining ethical data splits will result in more robust datasets.

As the technological landscape continues rapidly evolving, blockchain technology has been widely integrated and employed in various areas of application. Blockchain, at its core, offers a decentralized method for system security and communication. This is in contrast with classical security systems, which necessitate a central node for data processing and communication, therefore augmenting vulnerability to a single point of failure and attack. Incorporating adaptive subsystems into various blockchain technology features might greatly enhance their functionality without jeopardizing the chain's immutability. Several publications have focused on the analysis of network node data in an effort to offer an adaptive version of the consensus mechanism used in the blockchain process. This paper presents a novel adaptive consensus mechanism that regulates the Proof-of-Work mining difficulty based on the perceived anomalous level of network nodes.

Data is the cornerstone of Artificial Intelligence (AI) and Machine Learning (ML) systems. As the Department of Defense (DoD) leverages AI/ML to develop, test, and deploy autonomous vehicle capabilities, management of autonomy data will become increasingly important. Modern sensors on autonomous vehicles generate an enormous amount of data, and making this data available for further research presents a significant challenge. Moving such large volumes of data from a field environment to a centralized, cloud-based data lake is not straightforward, nor necessarily efficient for data of unknown enterprise utility. As a result, much of DoD’s autonomy data remains siloed in geographically or logically separated on-premises and cloud-based data stores in mixed formats. Organizations within DoD’s modernization enterprise require a mature data infrastructure to store, discover, share, and collaborate upon datasets, models, and other artifacts efficiently. In this paper, we examine the characteristics a data infrastructure must exhibit to meet the needs of the DoD for autonomy research. These characteristics are identified through a review of existing solutions, use cases, and current industry best practices. On the basis of this review, we propose a set of requirements for DoD’s data infrastructure for autonomous systems research. Moreover, an analysis of the viability of various options, including centralized and decentralized architectures, is provided through the lens of DoD data requirements and unique organizational constraints. While data infrastructure for autonomy is our primary concern, the requirements and design we propose generalize to other AI tasks that are of interest to DoD.

Manufacturing has entered the fourth industrial revolution. Modern manufacturing is reliant on assets such as robotics and computer numerical control (CNC) machine tools. To optimize the performance and value of these assets it would be wise to implement digital twin (DT) technology. DT technology has the ability to provide valuable services to owners of machine tools and other manufacturing assets. The current issue facing DTs is that they currently exist at a lower level of sophistication, meaning they are incapable of implementing more complex services. Cognitive dynamic systems (CDS) are a type of smart system based on human cognition which can augment the performance of many engineering systems. This paper proposes a framework of implementing aspects of CDSs to enable DTs to exist at a higher level of sophistication called the cognitive dynamic digital twin (CDDT). Examples exist in the literature of implementing cognitive based methods to improve DT services, they primarily implement artificial intelligence and estimation based methods. Most of these methods implement only one aspect of cognition at a time. In this work the CDDT framework was implemented to build a DT machine tool wear prediction service. The service was shown to be accurate at predicting the levels of wear in cutting tools. This service utilizing the CDDT framework used each of the aspects of human cognition to augment its performance. This framework can be used by many different sorts of DTs to improve their level of sophistication.

We consider the problem of unsupervised (blind) evaluation and assessment of the quality of data used for deep neural network (DNN) RF signal classification. When neural networks train on noisy or mislabeled data, they often (over-) fit to the noise measurements and faulty labels, which leads to significant performance degradation. Also, DNNs are vulnerable to adversarial attacks, which can considerably reduce their classification performance, with extremely small perturbations of their input. In this paper, we consider a new method based on L1-norm principal-component analysis (PCA) to improve the quality of labeled wireless datasets that are used for training a convolutional neural network (CNN), and a deep residual network (ResNet) for RF signal classification. Experiments with data generated for eleven classes of digital and analog modulated signals show that L1-norm tensor conformity curation of the data identifies and removes from the training dataset inappropriate class instances that appear due to mislabeling and universal black-box adversarial attacks and drastically improves/restores the classification accuracy of the identified deep neural network architectures.

The prompt and accurate recognition of Continuous Human Activity x(CHAR) is critical in identifying and responding to health events, particularly fall risk assessment. In this paper, we examine a multi-antenna radar system that can process radar data returns for multiple individuals in an indoor setting, enabling CHAR for multiple subjects. This requires combining spatial and temporal signal processing techniques through micro-Doppler (MD) analysis and high-resolution receive beamforming. We employ delay and sum beamforming to capture MD signatures at three different directions of observation. As MD images may contain multiple activities, we segment the three MD signatures using an STA/LTA algorithm. MD segmentation ensures that each MD segment represents a single human motion activity. Finally, the segmented MD image is resized and processed through a convolutional neural network (CNN) to classify motion against each MD segment.

In recent years Internet of Things (IoT) devices have made their way into many different industries. Deep learning and machine learning methodologies have been applied to many IoT-related tasks¹²³ such as intrusion detection systems or anomaly detection. The efficiency of IoT systems is often hindered by anomalies in data present within the system, often leading to undesirable behavior or possibly a full system shutdown. Due to this, the detection of these anomalies is of the utmost importance. Over the years, various traditional and neural network-based machine learning models have emerged for anomaly detection and classification of corrupted IoT data. However, many of these models fail to capture important features in the data which can lead to false anomaly detection or none at all. In this paper we investigate the applicability of using data fusion to improve the detection of data anomalies. This method uses many different models, such as VGG16, Inception, Xception, and ResNet, to extract features from the data. These extracted features are then fused together, to see if the use of multiple models is better than relying on a single model. This paper also provides a detailed analysis of the efficacy of this fusion-based classification method compared to simpler classification methods. This work investigates the applicability of various machine learning and deep learning models, for anomaly detection in various IoT datasets⁴⁵.

Oftentimes aircrafts will experience flight failures that produce a significant amount of complicated avionic diagnostic data, wherein many reports of faults will contain false positives across various subsystems, and true faults normally are identified by pouring through an entire flight's diagnostic logs. We investigated if avionic fault detection can be improved, or completely automated, through intelligent application of machine learning models to paired instrumentation and natural language time-series data from helicopters. We focused on using an unsupervised model, specifically an auto-encoder, as our data was unlabeled. We also focused on the natural language portion of the data, and we created a novel transformation from natural language time series data to image data for ease of model integration, taking inspiration from the spectrogram. This allowed us to leverage a linear and convolutional autoencoder for feature extraction, which we compared to a deterministic algorithm like Principal Component Analysis (PCA). We successfully trained convolutional autoencoders to reconstruct our avionic diagnostic images. We attempted to train linear autoencoders with our custom images and a bit transformation of our avionic diagnostic images. The linear model was unable to reconstruct the image and binary version. Finally we explored if the embeddings of the convolutional autoencoder and PCA could be used to automatically label our data by exploring the clusters produced by K-means. Our results were promising, where the PCA was a more accurate fault detector than the convolutional autoencoder, but further investigation is needed.