Frame Rate and Viseme Analysis for Multimedia Applications to Assist Speechreading

Jay J. Williams*, Janet C. Rutledge, Aggelos K. Katsaggelos, Dean C. Garstecki

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Current video conference and phone systems do not provide the necessary temporal resolution and motion for speechreading. In this paper the perceptual boundaries which effect speechreading performance are investigated. Analysis of the relationships between viseme groupings, accuracy of viseme recognition and presentation frame rate is presented based on the results of subject testing. Results reveal a minimum frame rate of 10 frames per second (fps) for distinguishing viseme groupings. Confusion analysis results demonstrate the importance of the tongue and teeth oral features for speechreading. These results are critical to the design of speech-assisted video systems to enhance speechreading for individuals with impaired hearing.

Original languageEnglish (US)
Pages (from-to)7-23
Number of pages17
JournalJournal of VLSI Signal Processing Systems for Signal, Image, and Video Technology
Volume20
Issue number1-2
StatePublished - Dec 1 1998

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Frame Rate and Viseme Analysis for Multimedia Applications to Assist Speechreading'. Together they form a unique fingerprint.

Cite this