An automatic system for unconstrained video-based face recognition

Jingxiao Zheng, Rajeev Ranjan, Ching Hui Chen, Jun Cheng Chen, Carlos D. Castillo, Rama Chellappa

Research output: Contribution to journalArticlepeer-review


Although deep learning approaches have achieved performance surpassing humans for still image-based face recognition, unconstrained video-based face recognition is still a challenging task due to large volume of data to be processed and intra/inter-video variations on pose, illumination, occlusion, scene, blur, video quality, etc. In this work, we consider challenging scenarios for unconstrained video-based face recognition from multiple-shot videos and surveillance videos with low-quality frames. To handle these problems, we propose a robust and efficient system for unconstrained video-based face recognition, which is composed of modules for face/fiducial detection, face association, and face recognition. First, we use multi-scale singleshot face detectors to efficiently localize faces in videos. The detected faces are then grouped through carefully designed face association methods, especially for multi-shot videos. Finally, the faces are recognized by the proposed face matcher based on an unsupervised subspace learning approach and a subspace-tosubspace similarity metric. Extensive experiments on challenging video datasets, such as Multiple Biometric Grand Challenge (MBGC), Face and Ocular Challenge Series (FOCS), IARPA Janus Surveillance Video Benchmark (IJB-S) for low-quality surveillance videos and IARPA JANUS Benchmark B (IJB-B) for multiple-shot videos, demonstrate that the proposed system can accurately detect and associate faces from unconstrained videos and effectively learn robust and discriminative features for recognition.

Original languageEnglish (US)
Article number8999558
Pages (from-to)194-209
Number of pages16
JournalIEEE Transactions on Biometrics, Behavior, and Identity Science
Issue number3
StatePublished - Jul 2020
Externally publishedYes


  • Face association
  • Face tracking
  • Unconstrained video-based face recognition

ASJC Scopus subject areas

  • Computer Science Applications
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Instrumentation


Dive into the research topics of 'An automatic system for unconstrained video-based face recognition'. Together they form a unique fingerprint.

Cite this