Automated video-based facial expression analysis of neuropsychiatric disorders

Peng Wang, Frederick Barrett, Elizabeth Martin, Marina Milonova, Raquel E. Gur, Ruben C. Gur, Christian Kohler, Ragini Verma

Research output: Contribution to journalArticlepeer-review

52 Scopus citations


Deficits in emotional expression are prominent in several neuropsychiatric disorders, including schizophrenia. Available clinical facial expression evaluations provide subjective and qualitative measurements, which are based on static 2D images that do not capture the temporal dynamics and subtleties of expression changes. Therefore, there is a need for automated, objective and quantitative measurements of facial expressions captured using videos. This paper presents a computational framework that creates probabilistic expression profiles for video data and can potentially help to automatically quantify emotional expression differences between patients with neuropsychiatric disorders and healthy controls. Our method automatically detects and tracks facial landmarks in videos, and then extracts geometric features to characterize facial expression changes. To analyze temporal facial expression changes, we employ probabilistic classifiers that analyze facial expressions in individual frames, and then propagate the probabilities throughout the video to capture the temporal characteristics of facial expressions. The applications of our method to healthy controls and case studies of patients with schizophrenia and Asperger's syndrome demonstrate the capability of the video-based expression analysis method in capturing subtleties of facial expression. Such results can pave the way for a video-based method for quantitative analysis of facial expressions in clinical research of disorders that cause affective deficits.

Original languageEnglish (US)
Pages (from-to)224-238
Number of pages15
JournalJournal of Neuroscience Methods
Issue number1
StatePublished - Feb 15 2008
Externally publishedYes


  • Affective deficits
  • Facial expression
  • Pattern classification
  • Schizophrenia
  • Video analysis

ASJC Scopus subject areas

  • Neuroscience(all)


Dive into the research topics of 'Automated video-based facial expression analysis of neuropsychiatric disorders'. Together they form a unique fingerprint.

Cite this