MiLa: An audiovisual instrument for learning the Curwen hand signs

Matt H Y Hong, William S. Hicks, Michael S. Horn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel musical system for learning the Curwen-Kodaly hand signs, which stand for solfege syllables and pitches (do, re, mi, etc.). These hand postures comprise a system of musical notation typically used in sight-singing training to familiarize with patterns of musical tones. Therefore, what concerns us is music literacy as opposed to practice with any particular instrument. Our system uses a Leap motion sensor to recognize the hand signs and produce corresponding user-recorded sounds. For each hand sign, the system also displays a natural 3D scenery, which is intended to match the metaphorical nature of the tone and has a resemblance to the hand posture (e.g. mi is a steady or calm tone, so the hand sign is an open hand with palm downwards and the scenery is a calm but moving ocean).

Original languageEnglish (US)
Title of host publicationCHI EA 2016
Subtitle of host publication#chi4good - Extended Abstracts, 34th Annual CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
Pages1691-1697
Number of pages7
Volume07-12-May-2016
ISBN (Electronic)9781450340823
DOIs
StatePublished - May 7 2016
Event34th Annual CHI Conference on Human Factors in Computing Systems, CHI EA 2016 - San Jose, United States
Duration: May 7 2016May 12 2016

Other

Other34th Annual CHI Conference on Human Factors in Computing Systems, CHI EA 2016
CountryUnited States
CitySan Jose
Period5/7/165/12/16

Keywords

  • Computer graphics
  • Embodied metaphors
  • Music education
  • Musical instrument
  • Posture recognition

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Fingerprint Dive into the research topics of 'MiLa: An audiovisual instrument for learning the Curwen hand signs'. Together they form a unique fingerprint.

Cite this