Stochastic modeling of soundtrack for efficient segmentation and indexing of video

Milind Ramesh Naphade; Thomas S. Huang

doi:10.1117/12.373546

23 December 1999 Stochastic modeling of soundtrack for efficient segmentation and indexing of video

Proceedings Volume 3972, Storage and Retrieval for Media Databases 2000; (1999) https://doi.org/10.1117/12.373546
Event: Electronic Imaging, 2000, San Jose, CA, United States

Abstract

Tools for efficient and intelligent management of digital content are essential for digital video data management. An extremely challenging research area in this context is that of multimedia analysis and understanding. The capabilities of audio analysis in particular for video data management are yet to be fully exploited. We present a novel scheme for indexing and segmentation of video by analyzing the audio track. This analysis is then applied to the segmentation and indexing of movies. We build models for some interesting events in the motion picture soundtrack. The models built include music, human speech and silence. We propose the use of hidden Markov models to model the dynamics of the soundtrack and detect audio-events. Using these models we segment and index the soundtrack. A practical problem in motion picture soundtracks is that the audio in the track is of a composite nature. This corresponds to the mixing of sounds from different sources. Speech in foreground and music in background are common examples. The coexistence of multiple individual audio sources forces us to model such events explicitly. Experiments reveal that explicit modeling gives better result than modeling individual audio events separately.

Citation Download Citation

Milind Ramesh Naphade and Thomas S. Huang "Stochastic modeling of soundtrack for efficient segmentation and indexing of video", Proc. SPIE 3972, Storage and Retrieval for Media Databases 2000, (23 December 1999); https://doi.org/10.1117/12.373546

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available