Automatic recognition of second language speech-in-noise

Seung Eun Kim; Bronya R. Chernyak; Olga Seleznova; Joseph Keshet; Matthew Goldrick; Ann R. Bradlow

doi:10.1121/10.0024877

Automatic recognition of second language speech-in-noise

Seung Eun Kim^*, Bronya R. Chernyak, Olga Seleznova, Joseph Keshet, Matthew Goldrick, Ann R. Bradlow

^*Corresponding author for this work

Linguistics

Research output: Contribution to journal › Article › peer-review

7 Scopus citations

Abstract

Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.

Original language	English (US)
Article number	025204
Journal	JASA Express Letters
Volume	4
Issue number	2
DOIs	https://doi.org/10.1121/10.0024877
State	Published - Feb 1 2024

Funding

This work was supported by NSF DRL Grant No. 2219843 and BSF Grant No. 2022618. Thanks to Chun Chan for assistance with human data collection.

ASJC Scopus subject areas

Acoustics and Ultrasonics
Music
Arts and Humanities (miscellaneous)

Access to Document

10.1121/10.0024877

Cite this

@article{2bb988bc390248579d0d7cb964f177fa,

title = "Automatic recognition of second language speech-in-noise",

abstract = "Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.",

author = "Kim, {Seung Eun} and Chernyak, {Bronya R.} and Olga Seleznova and Joseph Keshet and Matthew Goldrick and Bradlow, {Ann R.}",

note = "Publisher Copyright: {\textcopyright} 2024 Author(s).",

year = "2024",

month = feb,

day = "1",

doi = "10.1121/10.0024877",

language = "English (US)",

volume = "4",

journal = "JASA Express Letters",

issn = "2691-1191",

publisher = "American Institute of Physics",

number = "2",

}

TY - JOUR

T1 - Automatic recognition of second language speech-in-noise

AU - Kim, Seung Eun

AU - Chernyak, Bronya R.

AU - Seleznova, Olga

AU - Keshet, Joseph

AU - Goldrick, Matthew

AU - Bradlow, Ann R.

PY - 2024/2/1

Y1 - 2024/2/1

N2 - Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.

AB - Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.

UR - http://www.scopus.com/inward/record.url?scp=85187299817&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85187299817&partnerID=8YFLogxK

U2 - 10.1121/10.0024877

DO - 10.1121/10.0024877

M3 - Article

C2 - 38350077

AN - SCOPUS:85187299817

SN - 2691-1191

VL - 4

JO - JASA Express Letters

JF - JASA Express Letters

IS - 2

M1 - 025204

ER -

Automatic recognition of second language speech-in-noise

Abstract

Funding

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this