Ramin Mehran, Alexis Oyama, Mubarak Shah, Abnormal Crowd Behavior Detection using Social Force Model, IEEE International Conference on Computer Vision and Pattern Recongition (CVPR), Miami, 2009

1. Introduction
   1. Challenges of crowd behavior analysis
   2. The solution
   3. Advantages of the proposed method
2. Social Force Model
   1. Description
   2. Dynamic Model
   3. Generalized Model
3. Estimation of Interaction Forces
   1. Particle Advection
   2. Computing Social Forces
4. Event Detection
5. Results
   1. UMN Dataset
   2. Web Dataset
6. Download Dataset
   
7. Related Links

I. Introduction

One of the most challenging tasks in computer vision is analysis of human activity in crowded scenes. In addition, research in sociology and behavioral sciences provide mathematical models of pedestrian behavior patterns such as Social Force Model. In this paper, we introduce a computer vision method based on particle advection to detect and localize abnormal crowd behavior using the Social Force model.

(a) The block diagram of the abnormal behavior detection algorithm

1. Challenges of crowd behavior analysis

Conventional methods which a crowd is considered as a collection of individuals suffer from:

• Occlusion
• Clutter
• Low resolution
• Ignoring social interaction of pedestrians

2. Solution

• Considering a crowd as a global entity using particle advection method.
• Using Social Force Model to model crowd behaviors.

3. Advantages

• The holistic approach provides robustness to occlusion and clutter.
• Social Force Model endows the way to analyze crowd behaviors based on interaction forces.

II. Social Force Model

1.Description

In Social Force Model an individual is subject to long-ranged forces and his/her dynamics follow the equation of motion, similar to Newtonian mechanics. The velocity of an individual is described as the result of a personal desire force and interaction forces.

(b) A visualization of the forces and velocities in the social force model

2. Dynamic Model

In this model, the individual dynamics of pedestrians is modeled as:

3. Generalized Social Force Model

Considering the effect of panic

---

III. Estimation of Interaction Forces in Crowds

Estimating the interaction forces in the a crowd is a daunting task because of the occlusion and clutter. The holistic approach of particle advection provides an alternative way to compute these forces.

1. Particle Advection

The Motion of individuals in a dense crowd resembles the gradual motion of particles in a fluid. The optical flow in a crowd scene represents the flow of pedestrians.

• The motion of crowd is captured by moving a grid of particles using the optical flow (O).
• In a crowd, people tend to move with the average flow therefore, the spatio-temporal average of optical flow (Oave) is used for particle advection.

(d) The optical flow and interaction forces between a gentle man walking across a crowd: The computed interaction forces (red),  the optical flow (yellow)

2. Computing Social Forces

Regarding particles as individuals in the crowd, Social Force Model can be adapted for particles:

---

IV. Event Detection

The value of the interaction forces are not enough to understand the dynamics of the crowd.

• The pattern of forces in a scene represent different behaviors in the crowd.
• Force flow: The interaction forces in a scene is computed by mapping the magnitude of forces of particles to the frame plain.

(e) The flow of interaction forces in the video frame and the definition of visual words

1. Abnormal Event Detection

LDA: Tresholding the Likelihood of a clip to distinguish normal and abnormal set of frame.

• The location of abnormal events are highlighted in the areas of high interaction forces in the abnormal clips.

---

V. Experiments

1.UMN Dataset

• Crowd Escape Panic, 11 Videos, 3 Scenes, Videos: a normal starting section and an abnormal ending section
• Trained on normal section of 5 videos of scene 1, Tested on all

(e) The ROC on UMN dataset

(f) Abnormal Behavior Detection on UMN Dataset

2. Experiments on Web Dataset

• 8 Videos of real-life Escape panic, clash, fight
• 12 Videos of normal pedestrians
• Trained and tested in a 2-fold validation manner

(g) Web dataset example results

(h) The ROC on Web dataset

---

VI. Downloads

1. Web Dataset: Abnormal/Normal Crowds
2. Extra crowd videos

---

VII. Links

1. UMN Dataset: Unusual Crowd Activity
2. Thought Equity Stock Footage Videos: Crowd Panic, Normal Crowd (New York),