A Vision-Based System for a UGV to Handle a Road Intersection

Javed Ahmed, Mubarak Shah, Andrew Miller, Don Harper, and M. N. Jafri

{jahmed, shah, amiller, harper}@cs.ucf.edu, {javed, mnjafri}@mcs.edu.pk

Figure 2: A typical 4-way intersection showing UGV and the shaded areas monitored by the three cameras	Figure 1: Our UGV with three cameras looking towards other three roads.

A vision based system has been successfully developed which enables an unmanned ground vehicle (UGV) to handle an urban road intersection. The system uses three video cameras mounted on top of the UGV, as shown in Figure 1. The cameras are pointed towards the other three roads leading to the intersection, i.e. to the right, to the left, and straight ahead. Each camera is connected to a separate on-board computer. The three computers communicate with the autopilot through a UDP Ethernet connection. When the autopilot determines that the UGV has reached the intersection and has come to a stop, it sends a message to the three computers signaling them to begin looking for vehicles in their fields of view. The software in each computer consists of three main components: a vehicle detector, a tracker, and a finite-state-machine model of the intersection, as shown in Figures 3. First, the vehicle detector tries to detect a vehicle in each video frame by using an OT-MACH (Optimal Trade-off Maximum Average Correlation Height) filter constructed from training images of vehicles captured from each camera. Once a vehicle is detected in a single frame, the detector transfers control to a tracker. The tracker follows the vehicle in the subsequent frames, adapts to the changing appearance of the vehicle, handles occlusions, and estimates the current and next position, velocity, and acceleration of the vehicle. The detection and tracking is performed simultaneously for all the three cameras. The finite-state-machine model, shown in Fig. 4, determines the behavior of each leading vehicle on the other three roads using four states (i.e. No Vehicle Waiting, Arriving, Waiting, and Passing) estimated by the tracker in every case. Finally, it triggers the autopilot when it is safe for the UGV to drive through the intersection without violating any traffic law, and the autopilot drives the vehicle to cross the intersection.

Figure 3: Block diagram of the vision based system

Figure 4: The finite state-machine model of the intersection

Reference:

Javed Ahmed, Mubarak Shah, Andrew Miller, Don Harper, and M.N. Jafri, "A Vision-Based System for a UGV to Handle a Road Intersection", AAAI Twenty-Second Conference on Artificial Intelligence, Vancouver, British Columbia, 22-26 July 2007.