A relaxed fusion of information from real and synthetic images to predict complex behavior

Damian M. Lyons; D. Paul Benjamin

doi:10.1117/12.886014

6 June 2011 A relaxed fusion of information from real and synthetic images to predict complex behavior

Damian M. Lyons, D. Paul Benjamin

Proceedings Volume 8064, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2011; 806409 (2011) https://doi.org/10.1117/12.886014
Event: SPIE Defense, Security, and Sensing, 2011, Orlando, Florida, United States

Abstract

An important component of cognitive robotics is the ability to mentally simulate physical processes and to compare the expected results with the information reported by a robot's sensors. In previous work, we have proposed an approach that integrates a 3D game-engine simulation into the robot control architecture. A key part of that architecture is the Match-Mediated Difference (MMD) operation, an approach to fusing sensory data and synthetic predictions at the image level. The MMD operation insists that simulated and predicted scenes are similar in terms of the appearance of the objects in the scene. This is an overly restrictive constraint on the simulation since parts of the predicted scene may not have been previously viewed by the robot. In this paper we propose an extended MMD operation that relaxes the constraint and allows the real and synthetic scenes to differ in some features but not in (selected) other features. Image difference operations that allow a real image and synthetic image generated from an arbitrarily colored graphical model of a scene to be compared. Scenes with the same content show a zero difference. Scenes with varying foreground objects can be controlled to compare the color, size and shape of the foreground.

Citation Download Citation

Damian M. Lyons and D. Paul Benjamin "A relaxed fusion of information from real and synthetic images to predict complex behavior", Proc. SPIE 8064, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2011, 806409 (6 June 2011); https://doi.org/10.1117/12.886014

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available