Vroom! A search engine for sounds by vocal imitation queries

Yichi Zhang*, Junbo Hu, Yiting Zhang, Bryan Pardo, Zhiyao Duan

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Traditional search through collections of audio recordings compares a text-based query to text metadata associated with each audio file and does not address the actual content of the audio. Text descriptions do not describe all aspects of the audio content in detail. Query by vocal imitation (QBV) is a kind of query by example that lets users imitate the content of the audio they seek, providing an alternative search method to traditional text search. Prior work proposed several neural networks, such as TL-IMINET, for QBV, however, previous systems have not been deployed in an actual search engine nor evaluated by real users. We have developed a state-of-the-art QBV system (Vroom!) and a baseline query-by-text search engine (TextSearch). We deployed both systems in an experimental framework to perform user experiments with Amazon Mechanical Turk (AMT) workers. Results showed that Vroom! received significantly higher search satisfaction ratings than TextSearch did for sound categories that were difficult for subjects to describe by text. Results also showed a better overall ease-of-use rating for Vroom! than TextSearch on the sound library used in our experiments. These findings suggest that QBV, as a complimentary search approach to existing text-based search, can improve both search results and user experience.

Original languageEnglish (US)
Title of host publicationCHIIR 2020 - Proceedings of the 2020 Conference on Human Information Interaction and Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages23-32
Number of pages10
ISBN (Electronic)9781450368926
DOIs
StatePublished - Mar 14 2020
Event5th ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR 2020 - Vancouver, Canada
Duration: Mar 14 2020Mar 18 2020

Publication series

NameCHIIR 2020 - Proceedings of the 2020 Conference on Human Information Interaction and Retrieval

Conference

Conference5th ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR 2020
CountryCanada
CityVancouver
Period3/14/203/18/20

Keywords

  • Siamese style convolutional recurrent neural networks
  • Sound search
  • Subjective evaluation
  • Text description
  • Vocal imitation

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Information Systems

Fingerprint Dive into the research topics of 'Vroom! A search engine for sounds by vocal imitation queries'. Together they form a unique fingerprint.

Cite this