Visual analytics of anomaly detection in large data streams

Ming C. Hao; Umeshwar Dayal; Daniel A. Keim; Ratnesh K. Sharma; Abhay Mehta

doi:10.1117/12.810945

20 January 2009 Visual analytics of anomaly detection in large data streams

Ming C. Hao, Umeshwar Dayal, Daniel A. Keim, Ratnesh K. Sharma, Abhay Mehta

Proceedings Volume 7243, Visualization and Data Analysis 2009; 72430B (2009) https://doi.org/10.1117/12.810945
Event: IS&T/SPIE Electronic Imaging, 2009, San Jose, California, United States

Abstract

Most data streams usually are multi-dimensional, high-speed, and contain massive volumes of continuous information. They are seen in daily applications, such as telephone calls, retail sales, data center performance, and oil production operations. Many analysts want insight into the behavior of this data. They want to catch the exceptions in flight to reveal the causes of the anomalies and to take immediate action. To guide the user in finding the anomalies in the large data stream quickly, we derive a new automated neighborhood threshold marking technique, called AnomalyMarker. This technique is built on cell-based data streams and user-defined thresholds. We extend the scope of the data points around the threshold to include the surrounding areas. The idea is to define a focus area (marked area) which enables users to (1) visually group the interesting data points related to the anomalies (i.e., problems that occur persistently or occasionally) for observing their behavior; (2) discover the factors related to the anomaly by visualizing the correlations between the problem attribute with the attributes of the nearby data items from the entire multi-dimensional data stream. Mining results are quickly presented in graphical representations (i.e., tooltip) for the user to zoom into the problem regions. Different algorithms are introduced which try to optimize the size and extent of the anomaly markers. We have successfully applied this technique to detect data stream anomalies in large real-world enterprise server performance and data center energy management.

Citation Download Citation

Ming C. Hao, Umeshwar Dayal, Daniel A. Keim, Ratnesh K. Sharma, and Abhay Mehta "Visual analytics of anomaly detection in large data streams", Proc. SPIE 7243, Visualization and Data Analysis 2009, 72430B (20 January 2009); https://doi.org/10.1117/12.810945

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available