Following sections will describe different modules that are integrated into COCOA.
Motion compensation module of COCOA
accounts for the continuous motion of the camera mounted
on the aerial platform. This modules contributes to the
working of the system in two ways. First, it helps in detecting
independently moving ground targets as after compensation
in any two neighboring frames the intensity of only those
pixels will be changing that belong to the moving object.
Second, by registering the whole video with respect to one
global reference we are able to get a meaningful representation
of the object trajectories which can be used to describe
the entire video by detecting interesting events from them.
In COCOA we integrated three different
ways to accomplish the video registration task:
- Frame to Frame Alignment
using Telemetry Information
- Feature-Based Registration
- Gradient-Based Registration
The above figure shows part of the the graphical user
interface that is used for performing motion compensation.
| More Alignment Results
Goal of the tracking module
in COCOA is to track the targets detected by the motion
detection stage, as long as they remain visible in the field
of view of the camera. This is critical for obtaining tracks
that reflect the motion characteristics of the tracked object
over longer durations of time. Two tracking methods are
incorporated into COCOA for achieving this goal namely Kernel-based
object tracker and Blob tracker. In Kernel-based object
tracker representation based on the color distribution of
the target objects in RGB space are used instead of raw
image pixels. Each object is represented separately by its
own color distribution. Tracking is performed in global
coordinates. Once a new blob is detected a new object is
initialized and tracked across the sequence till it exits.
To ensure adaptivity of the mean shift tracker, the template
is updated after small intervals. In the second method a
blob tracking approach is used to perform multitarget tracking.
Regions of interest, or blobs, in successive frames are
given by the motion detection module. Each blob represented
by its own appearance and shape models. Temporal relationship
between the blobs are established by using a cost function
that takes into account appearance and shape similarity.
Association between the blobs are established if the score
is above a threshold.
The above figure shows part of the the graphical
user interface that is used for performing object tracking.
| More Tracking Results
Event Detection and Indexing
Event detection in COCOA is
performed by using the trajectories generated by the object
tracking module. First smoothing is performed on the trajectories
and any outliers generated by the detection and tracking
are removed. Then event detection is performed by detecting
Primitive Motion and Composite Motion verbs.
Primitive motion verbs are the basic action units whereas
grouping of these action units define a composite event.
Move forward, Move Left and Move Right are some examples
of primitive motion verbs that we are dealing with in COCOA.
Composite motion verbs describe the event as a whole e.g
car making a S-turn is a composite event. . Detection of
primitive motion verbs proceeds by first segmenting the
trajectories using divide-and-conquer method involving following
steps: a) Find the sharpest segment in the input
track. b) Repeat this process to the left side and the right
side of the sharpest segment. Model-based approach
is adopted to detect composite motion verbs in a given trajectory.
For each motion verb, we create several typical models using
simulated data. Then, the sequence of primitive motion verbs
is compared with these models to find the best match. Matching
is performed by a revised version of Edit Distance.
Ones the events are detected
from the trajectories next step is to index the videos on
the basis of these events. Indexing approach in COCOA uses
two level index.1) Primary Indexing (Event Indexing):
Searches for those videos in the database that have at least
one of the primitive motion verbs of the query trajectory
(e.g, S-turn, U-turn) which is exhibited by the same type
of object (e.g,person,vehicle) as those of query trajectory.
2) Secondary Indexing (Trajectory Indexing): Searches
similar trajectories in the subset returned by previous
step for composite event. Results are than sorted according
to the Edit Distance.
Keywords: UAV Videos, Motion Compensation, Moving Target Detection, Target Tracking, Event Detection, Airborne Surveillance
- Saad Ali and Mubarak Shah , COCOA - Tracking in Aerial Imagery, SPIE Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications, Orlando, 2006.
- Saad Ali and Mubarak Shah , COCOA - Tracking in Aerial Imagery, Demo presentation at ICCV 2005 in Beijing, China.
- Alper Yilmaz, Xin Li, and Mubarak Shah, Contour-Based Object Tracking with Occlusion Handling in Video Acquired Using Mobile Cameras, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No.11, pp. 1531-1536, 2004.
- Omar Javed , Khurram Shafique and Mubarak Shah, "A Hierarchical Approach to Robust Background Subtraction using Color and Gradient Information", IEEE Workshop on Motion and Video Computing, Orlando, Dec 5-6 2002.
- A. Yilmaz, K. Shafique, T. Olson, N. Lobo and M. Shah "Target Tracking in FLIR Imagery Using Mean-Shift and Global Motion Compensation," proceedings of IEEE Workshop on Computer Vision Beyond Visible Spectrum (CVBVS), Hawaii, 2001.