Dr.vot: Measuring positive and negative voice onset time in the wild

Yosi Shrem, Matthew Goldrick, Joseph Keshet

Research output: Contribution to journalConference article

Abstract

Voice Onset Time (VOT), a key measurement of speech for basic research and applied medical studies, is the time between the onset of a stop burst and the onset of voicing. When the voicing onset precedes burst onset the VOT is negative; if voicing onset follows the burst, it is positive. In this work, we present a deep-learning model for accurate and reliable measurement of VOT in naturalistic speech. The proposed system addresses two critical issues: it can measure positive and negative VOT equally well, and it is trained to be robust to variation across annotations. Our approach is based on the structured prediction framework, where the feature functions are defined to be RNNs. These learn to capture segmental variation in the signal. Results suggest that our method substantially improves over the current state-of-the-art. In contrast to previous work, our Deep and Robust VOT annotator, Dr.VOT, can successfully estimate negative VOTs while maintaining state-of-the-art performance on positive VOTs. This high level of performance generalizes to new corpora without further retraining.

Fingerprint

Burst
Voice
Voice Onset Time
Annotation
Onset
Generalise
Prediction
Voicing
Estimate
Speech
Model
Learning Model
Basic Research
Performance Art

Keywords

  • Adversarial training
  • Multi-task learning
  • Recurrent neural networks
  • Sequence segmentation
  • Structured prediction

ASJC Scopus subject areas

  • Language and Linguistics
  • Human-Computer Interaction
  • Signal Processing
  • Software
  • Modeling and Simulation

Cite this

@article{f165fa4537ac4001ab049af4a0e2c818,
title = "Dr.vot: Measuring positive and negative voice onset time in the wild",
abstract = "Voice Onset Time (VOT), a key measurement of speech for basic research and applied medical studies, is the time between the onset of a stop burst and the onset of voicing. When the voicing onset precedes burst onset the VOT is negative; if voicing onset follows the burst, it is positive. In this work, we present a deep-learning model for accurate and reliable measurement of VOT in naturalistic speech. The proposed system addresses two critical issues: it can measure positive and negative VOT equally well, and it is trained to be robust to variation across annotations. Our approach is based on the structured prediction framework, where the feature functions are defined to be RNNs. These learn to capture segmental variation in the signal. Results suggest that our method substantially improves over the current state-of-the-art. In contrast to previous work, our Deep and Robust VOT annotator, Dr.VOT, can successfully estimate negative VOTs while maintaining state-of-the-art performance on positive VOTs. This high level of performance generalizes to new corpora without further retraining.",
keywords = "Adversarial training, Multi-task learning, Recurrent neural networks, Sequence segmentation, Structured prediction",
author = "Yosi Shrem and Matthew Goldrick and Joseph Keshet",
year = "2019",
month = "1",
day = "1",
doi = "10.21437/Interspeech.2019-1735",
language = "English (US)",
volume = "2019-September",
pages = "629--633",
journal = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",
issn = "2308-457X",

}

Dr.vot : Measuring positive and negative voice onset time in the wild. / Shrem, Yosi; Goldrick, Matthew; Keshet, Joseph.

In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Vol. 2019-September, 01.01.2019, p. 629-633.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Dr.vot

T2 - Measuring positive and negative voice onset time in the wild

AU - Shrem, Yosi

AU - Goldrick, Matthew

AU - Keshet, Joseph

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Voice Onset Time (VOT), a key measurement of speech for basic research and applied medical studies, is the time between the onset of a stop burst and the onset of voicing. When the voicing onset precedes burst onset the VOT is negative; if voicing onset follows the burst, it is positive. In this work, we present a deep-learning model for accurate and reliable measurement of VOT in naturalistic speech. The proposed system addresses two critical issues: it can measure positive and negative VOT equally well, and it is trained to be robust to variation across annotations. Our approach is based on the structured prediction framework, where the feature functions are defined to be RNNs. These learn to capture segmental variation in the signal. Results suggest that our method substantially improves over the current state-of-the-art. In contrast to previous work, our Deep and Robust VOT annotator, Dr.VOT, can successfully estimate negative VOTs while maintaining state-of-the-art performance on positive VOTs. This high level of performance generalizes to new corpora without further retraining.

AB - Voice Onset Time (VOT), a key measurement of speech for basic research and applied medical studies, is the time between the onset of a stop burst and the onset of voicing. When the voicing onset precedes burst onset the VOT is negative; if voicing onset follows the burst, it is positive. In this work, we present a deep-learning model for accurate and reliable measurement of VOT in naturalistic speech. The proposed system addresses two critical issues: it can measure positive and negative VOT equally well, and it is trained to be robust to variation across annotations. Our approach is based on the structured prediction framework, where the feature functions are defined to be RNNs. These learn to capture segmental variation in the signal. Results suggest that our method substantially improves over the current state-of-the-art. In contrast to previous work, our Deep and Robust VOT annotator, Dr.VOT, can successfully estimate negative VOTs while maintaining state-of-the-art performance on positive VOTs. This high level of performance generalizes to new corpora without further retraining.

KW - Adversarial training

KW - Multi-task learning

KW - Recurrent neural networks

KW - Sequence segmentation

KW - Structured prediction

UR - http://www.scopus.com/inward/record.url?scp=85074733017&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074733017&partnerID=8YFLogxK

U2 - 10.21437/Interspeech.2019-1735

DO - 10.21437/Interspeech.2019-1735

M3 - Conference article

AN - SCOPUS:85074733017

VL - 2019-September

SP - 629

EP - 633

JO - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

JF - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

SN - 2308-457X

ER -