Speech-to-video synthesis using facial animation parameters

Petar S. Aleksic*, Aggelos K Katsaggelos

*Corresponding author for this work

Research output: Contribution to conferencePaperpeer-review

4 Scopus citations


The presence of visual information in addition to audio could improve speech understanding in noisy environments. This additional information could be especially useful for people with impaired hearing who are able to speechread. This paper focuses on the problem of synthesizing the Facial Animation Parameters (FAPs), supported by the MPEG-4 standard for the visual representation of speech, from a narrowband acoustic speech (telephone) signal. A correlation Hidden Markov Model (CHMM) system for performing visual speech synthesis is proposed. The CHMM system integrates an independently trained acoustic HMM (AHMM) system and a visual HMM (VHMM) system, in order to realize speech-to-video synthesis. Objective experiments are performed by analyzing the synthesized FAPs and computing the time alignment errors. Time alignment errors are reduced by 40.5% compared to the conventional temporal scaling method.

Original languageEnglish (US)
Number of pages4
StatePublished - Dec 17 2003
EventProceedings: 2003 International Conference on Image Processing, ICIP-2003 - Barcelona, Spain
Duration: Sep 14 2003Sep 17 2003


OtherProceedings: 2003 International Conference on Image Processing, ICIP-2003

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering


Dive into the research topics of 'Speech-to-video synthesis using facial animation parameters'. Together they form a unique fingerprint.

Cite this