Computational Vision Approaches for Event Modeling

Event modeling systems provide a semantic interpretation of sequences of pixels that are captured by a video camera. The design of a practical system has to take into account the following three main factors: low-level preprocessing limitations, computational and storage complexity of the event model, and user interaction. The hidden Markov model (HMM) and its variants have been widely used to model both speech and video signals. Computational efficiency of the Baum-Welch and the Viterbi algorithms has been a leading reason for the popularity of the HMM. Since the objective is to detect events in video sequences that are meaningful to humans, one might want to provide space in the design loop for a user who can specify events of interest. This chapter explores this using semantic approaches that not only use features extracted from raw video streams but also incorporate metadata and ontologies of activities. It presents three approaches for applications such as event recognition: anomaly detection, temporal segmentation, and ontology evaluation. The three approaches discussed are statistical methods based on HMMs, formal grammars, and ontologies. The effectiveness of these approaches is illustrated using video sequences captured both indoors and outdoors: the indoor UCF human action dataset, the TSA airport tarmac surveillance dataset, and the bank monitoring dataset.