<< Back

Finding scenes in a video is a problem of shot clustering based on some matching criteria, for example, color similarity. In this project, we present a method to perform a high level segmentation of videos into scenes.

A scene can be defined as one of the subdivisions of a play in which the setting is fixed (employing color similarity), or when it presents continuous action in one place (employing motion content and shot length similarity). We propose a novel two-pass algorithm for scene boundary detection which utilizes the motion contents and shot length together with the color properties of shots as the features. In our approach, shots are first clustered by computing Backward Shot Coherence (BSC); a shot color similarity measure that detects Potential Scene Boundaries (PSBs) in the videos. In the second pass we compute Scene Dynamics (SD) for each scene which is a function of shot length and the motion content in the potential scenes. In this pass, a scene merging criteria has been developed to remove weak PSBs in order to reduce unnecessary over-segmentation. This method results in more meaningful scene boundaries than those which utilize color matching only.

We also propose a method to describe the content of each scene by selecting one representative image from the video. This is done by analyzing shot coherence, shot length and the motion content of shots in a scene. This results in a compact representation of huge videos in a small number of key frames. The segmentation of video data into number of scenes also facilitates an improved browsing of videos in electronic form, such as video on demand, digital libraries, Internet. Recently, DVDs are available with chapter selection option where each chapter is represented by an image. Our algorithm can be used to automate this objective by first finding the scenes and then selecting a representative image for every scene.

Associated publications:
Scene Boundary Detection in Hollywood Movies and TV Show
The Eighth IEEE International Conference on Computer Vision, Vancouver, Canada. July 9-12, 2001