A bio-inspired system for spatio-temporal recognition in static and video imagery

Deepak Khosla; Christopher K. Moore; Suhas Chelian

doi:10.1117/12.719975

27 April 2007 A bio-inspired system for spatio-temporal recognition in static and video imagery

Deepak Khosla, Christopher K. Moore, Suhas Chelian

Proceedings Volume 6560, Intelligent Computing: Theory and Applications V; 656002 (2007) https://doi.org/10.1117/12.719975
Event: Defense and Security Symposium, 2007, Orlando, Florida, United States

Abstract

This paper presents a bio-inspired method for spatio-temporal recognition in static and video imagery. It builds upon and extends our previous work on a bio-inspired Visual Attention and object Recognition System (VARS). The VARS approach locates and recognizes objects in a single frame. This work presents two extensions of VARS. The first extension is a Scene Recognition Engine (SCE) that learns to recognize spatial relationships between objects that compose a particular scene category in static imagery. This could be used for recognizing the category of a scene, e.g., office vs. kitchen scene. The second extension is the Event Recognition Engine (ERE) that recognizes spatio-temporal sequences or events in sequences. This extension uses a working memory model to recognize events and behaviors in video imagery by maintaining and recognizing ordered spatio-temporal sequences. The working memory model is based on an ARTSTORE¹ neural network that combines an ART-based neural network with a cascade of sustained temporal order recurrent (STORE)¹ neural networks. A series of Default ARTMAP classifiers ascribes event labels to these sequences. Our preliminary studies have shown that this extension is robust to variations in an object's motion profile. We evaluated the performance of the SCE and ERE on real datasets. The SCE module was tested on a visual scene classification task using the LabelMe² dataset. The ERE was tested on real world video footage of vehicles and pedestrians in a street scene. Our system is able to recognize the events in this footage involving vehicles and pedestrians.

Citation Download Citation

Deepak Khosla, Christopher K. Moore, and Suhas Chelian "A bio-inspired system for spatio-temporal recognition in static and video imagery", Proc. SPIE 6560, Intelligent Computing: Theory and Applications V, 656002 (27 April 2007); https://doi.org/10.1117/12.719975

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available