|
Extracting layers from video is very important for video representation, analysis, compression, and synthesis. Assuming that a scene can be approximately described by multiple planar regions, this paper describes a robust and novel approach to automatically extract a set of affine or projective transformations induced by these regions, detect the occlusion pixels over multiple consecutive frames, and segment the scene into several motion layers. First, after determining a number of seed regions using correspondences in two frames, we expand the seed regions and reject the outliers employing the graph cuts method integrated with level set representation. Next, these initial regions are merged into several initial layers according to the motion similarity. Third, an occlusion order constraint on multiple frames is explored, which enforces that the occlusion area increases with the temporal order in a short period and effectively maintains segmentation consistency over multiple consecutive frames. Then the correct layer segmentation is obtained by using a graph cuts algorithm, and the occlusions between the overlapping layers are explicitly determined. Several experimental results are demonstrated to show that our approach is effective and robust. Related Publications: Jiangjian Xiao and Mubarak Shah, “Motion Layer Extraction in the Presence of Occlusion using Graph Cut”, IEEE transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 10, pp. 1644-1659, 2005. Jiangjian Xiao and Mubarak Shah, “Motion Layer Extraction in the Presence of Occlusion using Graph Cut", Oral presentation (6.5%), IEEE Conference of Computer Vision and Pattern Recognition, June 27 - July 2, Washington, DC. 2004.
Figure 1. Layer clustering by seed expanding process. (a) An initial seed region in the first frame. (b) The corresponding seed region in the second frame. (c) The warped version of the second frame using the estimated affine parameters. (d) The difference map between (a) and (c). (e) The result after simple expansion and partitioning. (f) The result after bi-partitioning without the level set representation. (g-j) are the intermediate steps of bi-partitioning with the level set representation. (g) and (i) respectively are the expansions of the seed region during the first and fourth iterations using the level set representation. (h) and (j) are the results obtained after the graph cuts partitioning, where the new region can have an arbitrary compact contour. (k) and (l) are 3D visualization of the level sets of (g) and (i). Note: The red box is the initial seed region. The green contours are obtained after using bi-partitioning algorithm. Video clip:
|
|||
