|
|
|
ACTION SKETCH: A NOVEL ACTION REPRESENTATION
|
|
Summary:
We propose a novel action representation which utilizes both the shape
and the motion of the performing actor. When an actor performs an
action in space and time, at each time instant, the points on the outer
object boundary are projected to the image plane as a 2D contour. A
sequence of such contours generates a volume in the spatio-temporal
(x,y,t) space, which can be treated as a 3D object. Usually, tracking
provides noisy contours, hence resulting volume is noisy. To reduce
noise, we perform level set based smoothing by minimizing the overall
surface curvature. From the smooth volume, a set of action descriptors
are extracted by analyzing the differential geometry of the surface.
These descriptors constitute the action sketch which is used to
represent the actions. Action sketch has several advantages. First, it
captures both the spatial (shape) and the temporal (speed,
acceleration) information of an action in a unified manner. Second, it
is generated from a continuous volume. The action volume is the same
for two sequences of the same action with different lengths, therefore
there is no need of time warping. Third, the action sketch is robust to
viewpoint changes, since action descriptors correspond to convex or
concave contours and minima and maxima of trajectory curvature, which
are robust to view point changes . Given two action sketches, action
recognition is formulated as a multi-view geometry problem. Multi-view
geometry based methods require point correspondences. In order to find
corresponding descriptors in the action sketches, we first align the
volumes using a novel level set based approach. From the aligned
volumes correspondences are established using a graph theoretical
approach. We demonstrate the versatility of the action sketch in the
context of action recognition.
|
Supporting
Agency:
|
Related
Publications:
- A. Yilmaz
and M. Shah "A Differential Geometric Approach to Representing the Human Actions," Computer Vision and Image Understanding Journal, Vol. 109 No. 3, pp. 335-351, 2008.
- A. Yilmaz
and M. Shah "Actions
Sketch: A Novel Action Representation,"
IEEE Conf. on
Computer Vision and Pattern Recognition (CVPR), 2005.
- A. Yilmaz and M. Shah "Contour
Based Object Tracking with Occlusion Handling in Video Acquired Using
Mobile Cameras,"
IEEE Trans. on Pattern Analysis and Machine
Intelligence (TPAMI), Vol.26, No. 11, 2004.
- A. Yilmaz and M. Shah "Contour
Based Object Tracking Using Level Sets,"
proceedings of Asian
Conf. on Computer Vision (ACCV), South Korea, 2004.
|
|
|
