Towards Unconstrained Facial Landmark Detection Robust to Diverse Cropping Manners

Xu Zou, Peng Xiao, Jianhui Wang, Luxin Yan, Sheng Zhong*, Ying Wu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Facial landmark detection is one crucial step for face-based image/video analysis. Despite the fact that recently many facial landmark detection models have achieved remarkable performance, most state-of-the-art heatmap regression-based methods heavily rely on initialization of the face detector. However, there inevitably exists semantic gaps among different annotators or face detectors. An improper facial bounding box will tremendously drop off the performance of the facial landmark detection model. Facial landmark detection would be more practical if robust to face images cropped by diverse manners (see Figure 1, the col.1 shows face images cropped by a proper bounding box, col.2 and col.3 show face images cropped by an oversize and a small bounding boxes respectively). To this end, we present a 'Unconstrained Facial Landmark Detection(UFLD)' mechanism, that aims at enhancing the robustness of facial landmark detection, to deal with the inconsistent cropping manner issue. UFLD consists of two aspects: A Transformation-Invariant Landmark Detector(TILD) and an Availability-Guided Solver(AGS). TILD gives the ability to detect consistent landmarks for face images cropped by diverse manners. And AGS can alleviate the by-effect of 'landmarks outside the image' caused by improper cropping results or TILD, and further promote the performance. The proposed mechanism achieved above 6.5% improvement in standard normalized landmarks mean error reduction on face images cropped by diverse manners compared to baselines.

Original languageEnglish (US)
Article number9130740
Pages (from-to)2070-2075
Number of pages6
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume31
Issue number5
DOIs
StatePublished - May 2021

Keywords

  • diverse cropping manners
  • Facial landmark detection
  • heatmap regression

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Towards Unconstrained Facial Landmark Detection Robust to Diverse Cropping Manners'. Together they form a unique fingerprint.

Cite this