Experience and lessons learned from the Army RCO Blind Signal Classification Competition

Peng Wang; Manuel Vindiola; John S. Hyatt; Michael S. Lee

doi:10.1117/12.2518447

10 May 2019 Experience and lessons learned from the Army RCO Blind Signal Classification Competition

Peng Wang, Manuel Vindiola, John S. Hyatt, Michael S. Lee

Proceedings Volume 11006, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications; 110061J (2019) https://doi.org/10.1117/12.2518447
Event: SPIE Defense + Commercial Sensing, 2019, Baltimore, MD, United States

Abstract

The Army Rapid Capabilities Office (RCO) sponsored a Blind Signal Classification Competition seeking algorithms to automatically identify the modulation schemes of RF signal from complex-valued IQ (in-phase quadrature) samples. Traditional spectrum sensing technology uses energy detection to detect the existence of RF signals but the RCO competition further aimed to detect the modulation scheme of signals without prior information. Machine Learning (ML) technologies have been widely used for blind signal classification problem. Traditional ML methods usually have two stages where the first stage is to manually extract the features of the IQ symbols by subject matter experts and the second stage is to feed the features to an ML algorithm (e.g., a support vector machine) to develop the classifier. The state-of-art technology is to apply deep learning technologies such as Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) directly to the complex-value IQ symbols to train a multi-class classifier. Our team, dubbed Deep Dreamers, participated in the RCO competition and placed 3rd out of 42 active teams across industry, academia, and government. In this work we share our experience and lessons learned from the competition. Deep learning methods such as CNN, Residual Neural Network (ResNet), and Long Short-Term Memory (LSTM) are the fundamental neural network layers we used to develop a multi-class classifier. None of our individual models were able to achieve a competitively high ranking in the competition. The key to our success was to use ensemble learning to average the outputs of multiple diverse classifiers. In order for ensemble methods to be more accurate than any of its base models; the base learners have to be as accurate as possible. We found that while ResNet was more accurate than the LSTM; the LSTM was less sensitive to deviations in the test set.

Conference Presentation

Citation Download Citation

Peng Wang, Manuel Vindiola, John S. Hyatt, and Michael S. Lee "Experience and lessons learned from the Army RCO Blind Signal Classification Competition", Proc. SPIE 11006, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, 110061J (10 May 2019); https://doi.org/10.1117/12.2518447

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available