Video content parsing based on combined audio and visual information

Tong Zhang; C.-C. Jay Kuo

doi:10.1117/12.360413

24 August 1999 Video content parsing based on combined audio and visual information

Tong Zhang, C.-C. Jay Kuo

Proceedings Volume 3846, Multimedia Storage and Archiving Systems IV; (1999) https://doi.org/10.1117/12.360413
Event: Photonics East '99, 1999, Boston, MA, United States

Abstract

While previous research on audiovisual data segmentation and indexing primarily focuses on the pictorial part, significant clues contained in the accompanying audio flow are often ignored. A fully functional system for video content parsing can be achieved more successfully through a proper combination of audio and visual information. By investigating the data structure of different video types, we present tools for both audio and visual content analysis and a scheme for video segmentation and annotation in this research. In the proposed system, video data are segmented into audio scenes and visual shots by detecting abrupt changes in audio and visual features, respectively. Then, the audio scene is categorized and indexed as one of the basic audio types while a visual shot is presented by keyframes and associate image features. An index table is then generated automatically for each video clip based on the integration of outputs from audio and visual analysis. It is shown that the proposed system provides satisfying video indexing results.

Citation Download Citation

Tong Zhang and C.-C. Jay Kuo "Video content parsing based on combined audio and visual information", Proc. SPIE 3846, Multimedia Storage and Archiving Systems IV, (24 August 1999); https://doi.org/10.1117/12.360413

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available