VocalSketch: Vocally imitating audio concepts

Mark Cartwright; Bryan A Pardo

doi:10.1145/2702123.2702387

VocalSketch: Vocally imitating audio concepts

Mark Cartwright, Bryan A Pardo

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

31 Scopus citations

Abstract

A natural way of communicating an audio concept is to imitate it with one's voice. This creates an approximation of the imagined sound (e.g. a particular owl's hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one's voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd's ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that leverage vocal imitation as an interaction modality.

Original language	English (US)
Title of host publication	CHI 2015 - Proceedings of the 33rd Annual CHI Conference on Human Factors in Computing Systems
Subtitle of host publication	Crossings
Publisher	Association for Computing Machinery
Pages	43-46
Number of pages	4
Volume	2015-April
ISBN (Electronic)	9781450331456
DOIs	https://doi.org/10.1145/2702123.2702387
State	Published - Apr 18 2015
Event	33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015 - Seoul, Korea, Republic of Duration: Apr 18 2015 → Apr 23 2015

Other

Other	33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015
Country/Territory	Korea, Republic of
City	Seoul
Period	4/18/15 → 4/23/15

Keywords

Audio software
Data set
User interaction
Vocal imitation

ASJC Scopus subject areas

Software
Human-Computer Interaction
Computer Graphics and Computer-Aided Design

Access to Document

10.1145/2702123.2702387

Cite this

@inproceedings{65b31f13433249d1b8c94bc9ec1755b7,

title = "VocalSketch: Vocally imitating audio concepts",

abstract = "A natural way of communicating an audio concept is to imitate it with one's voice. This creates an approximation of the imagined sound (e.g. a particular owl's hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one's voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd's ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that leverage vocal imitation as an interaction modality.",

keywords = "Audio software, Data set, User interaction, Vocal imitation",

author = "Mark Cartwright and Pardo, {Bryan A}",

year = "2015",

month = apr,

day = "18",

doi = "10.1145/2702123.2702387",

language = "English (US)",

volume = "2015-April",

pages = "43--46",

booktitle = "CHI 2015 - Proceedings of the 33rd Annual CHI Conference on Human Factors in Computing Systems",

publisher = "Association for Computing Machinery",

note = "33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015 ; Conference date: 18-04-2015 Through 23-04-2015",

}

Cartwright, M & Pardo, BA 2015, VocalSketch: Vocally imitating audio concepts. in CHI 2015 - Proceedings of the 33rd Annual CHI Conference on Human Factors in Computing Systems: Crossings. vol. 2015-April, Association for Computing Machinery, pp. 43-46, 33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015, Seoul, Korea, Republic of, 4/18/15. https://doi.org/10.1145/2702123.2702387

TY - GEN

T1 - VocalSketch

T2 - 33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015

AU - Cartwright, Mark

AU - Pardo, Bryan A

PY - 2015/4/18

Y1 - 2015/4/18

N2 - A natural way of communicating an audio concept is to imitate it with one's voice. This creates an approximation of the imagined sound (e.g. a particular owl's hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one's voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd's ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that leverage vocal imitation as an interaction modality.

AB - A natural way of communicating an audio concept is to imitate it with one's voice. This creates an approximation of the imagined sound (e.g. a particular owl's hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one's voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd's ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that leverage vocal imitation as an interaction modality.

KW - Audio software

KW - Data set

KW - User interaction

KW - Vocal imitation

UR - http://www.scopus.com/inward/record.url?scp=84951062648&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84951062648&partnerID=8YFLogxK

U2 - 10.1145/2702123.2702387

DO - 10.1145/2702123.2702387

M3 - Conference contribution

AN - SCOPUS:84951062648

VL - 2015-April

SP - 43

EP - 46

BT - CHI 2015 - Proceedings of the 33rd Annual CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

Y2 - 18 April 2015 through 23 April 2015

ER -

VocalSketch: Vocally imitating audio concepts

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

VocalSketch Data Set v1.0.4

Cite this

VocalSketch: Vocally imitating audio concepts

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Datasets

VocalSketch Data Set v1.0.4

Cite this