Tree based architectures for rule discovery in sequence modeling

Fandi Lin

doi:10.1117/12.2559038

21 April 2020 Tree based architectures for rule discovery in sequence modeling

Fandi Lin

Proceedings Volume 11395, Big Data II: Learning, Analytics, and Applications; 113950L (2020) https://doi.org/10.1117/12.2559038
Event: SPIE Defense + Commercial Sensing, 2020, Online Only

Abstract

A great amount of work has been done using deep learning strategies in sequence modeling, such as the encoder-decoder LSTM. Most solutions for this type of problem involve recurrent neural networks. This approach, while often yielding superior results, has its downsides. Training time can sometimes be prohibitively long and the only remedy for that is higher computing power. Additionally, these models are extremely difficult to interpret, regardless of their performance. The deeper and more complex the network, the harder it is to make sense of it. Neural networks lack a simple representation of the knowledge they learn. Rule-based learners, however, are the opposite in this regard. They represent the knowledge they learn through relational rules, which are easily digested by a human. While most rule-based learners are designed with the intent of discovering dependencies between variables, such as in association rule learning, sometimes rule-based learners can be implemented in a supervised setting. For sequence modeling, though, rule-based solutions are scarce, especially if we consider the case where inputs and targets are variable length sequences. We propose a type of architecture that utilizes binary trees and evolutionary algorithms to discover rules. These models make inferences through sequential actions, similarly to how reinforcement learning agents progress through tasks by choosing actions based on the state of the environment. These new learners can predict on various data types, including multi-output and sequences of variable length. We consider a simple problem where the inputs and targets are variable length sequences. Our strategy perfectly learns the rules from a synthetic dataset. Lastly, we discuss how to apply the strategy more generally.

Conference Presentation

Citation Download Citation

Fandi Lin "Tree based architectures for rule discovery in sequence modeling", Proc. SPIE 11395, Big Data II: Learning, Analytics, and Applications, 113950L (21 April 2020); https://doi.org/10.1117/12.2559038

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available