Source Code
Images & Sequences
Vision Conferences
VISION Homepage
 
 
 
 
 
Send comments and 
suggestions to
vision@cs.ucf.edu

 
 
 
 
 
 
 
 
 
 
 

Announcing the Final Examination of Mr. Omar Javed

for the degree of Philosophy of Science

Date: April 1st, 2005
Time: 3:00 p.m.
Room: CSB 232
Dissertation title: Scene Monitoring with a Forest of Cooperative Sensors

Abstract:
In this dissertation, we present vision based scene interpretation methods for monitoring of people and vehicles, in real-time, within a busy environment using a forest of co-operative electro-optical (EO) sensors. We have developed novel video understanding algorithms with learning capability, to detect and categorize people and vehicles, track them with in a camera and hand-off this information across multiple networked cameras for multi-camera tracking. The ability to learn prevents the need for extensive manual intervention, site models and camera calibration, and provides adaptability to changing environmental conditions.

For object detection and categorization in the video stream, a two step detection procedure is used. First, regions of interest are determined using a novel hierarchical background subtraction algorithm that uses color and gradient information for interest region detection (/projects/Knight/background.html).  Second, objects are located and classified from within these regions using a weakly supervised learning mechanism based on co-training that employs motion and appearance features. The main contribution of this approach is that it is an online procedure in which separate views (features) of the data are used for co-training, while the combined view (all features) is used to make classification decisions in a single boosted framework. The advantage of this approach is that it requires only a few initial training samples and can automatically adjust its parameters online to improve the detection and classification performance.

Once objects are detected and classified they are tracked in individual cameras. Single camera tracking is performed using a voting based approach that utilizes color and shape cues to establish correspondence in individual cameras. The tracker has the capability to handle multiple occluded objects. Next, the objects are tracked across a forest of cameras with non-overlapping views. This is a hard problem because of two reasons. First, the observations of an object are often widely separated in time and space when viewed from non-overlapping cameras. Secondly, the appearance of an object in one camera view might be very different from its appearance in another camera view due to the differences in illumination, pose and camera properties. To deal with the first problem, the system learns the inter-camera relationships to constrain track correspondences. These relationships are learned in the form of multivariate probability density of space-time variables (object entry and exit locations, velocities, and inter-camera transition times) using Parzen windows. To handle the appearance change of an object as it moves from one camera to another, we show that all color transfer functions from a given camera to another camera lie in a low dimensional subspace. The tracking algorithm learns this subspace by using probabilistic principal component analysis and uses it for appearance matching. The proposed system learns the camera topology and subspace of inter-camera color transfer functions during a training phase. Once the training is complete, correspondences are assigned using the maximum a posteriori (MAP) estimation framework using both the location and appearance cues (/projects/trackingNonOverlaping/NonoverlapTracking.html).

Extensive experiments and deployment of this system in realistic scenarios has demonstrated the robustness of the proposed methods (/projects/Knight/Knight.html). The proposed system was able to detect and classify targets, and seamlessly tracked them across multiple cameras. It also generated a summary in terms of key frames and textual description of trajectories to a monitoring officer for final analysis and response decision. This level of interpretation was the goal of our research effort, and we believe that it is a significant step forward in the development of intelligent systems that can deal with the complexities of real world scenarios.

Educational career:
B.S., 1999, Ghulam Ishaq Khan Institute of Engineering Sciences and Technology

Committee in charge:
Dr. Mubarak Shah

Dr. Niels da Vitoria Lobo

Dr. Mostafa Bassiouni

Dr. David M. Nickerson

Approved for distribution by Dr. Mubarak Shah, Committee Chair.

The public is welcome to attend.