Abstract
Communication between people is inherently multimodal. People employ speech, facial expressions, eye gaze, and gesture, among other facilities, to support communication and cooperative activity. Complexity of communication increases when a person is without a modality such as hearing, often resulting in dependence on another person or an assistive device to facilitate communication. This paper examines communication about medical topics through Shared Speech Interface, a multimodal tabletop display designed to assist communication between a hearing and deaf individual by converting speech-to-text and representing dialogue history on a shared interactive display surface. We compare communication mediated by a multimodal tabletop display and by a human sign language interpreter. Results indicate that the multimodal tabletop display (1) allows the deaf patient to watch the doctor when she is speaking, (2) encourages the doctor to exploit multimodal communication such as co-occurring gesture-speech, and (3) provides shared access to persistent, collaboratively produced representations of conversation. We also describe extensions of this communication technology, discuss how multimodal analysis techniques are useful in understanding the affects of multiuser multimodal tabletop systems, and briefly allude to the potential of applying computer vision techniques to assist analysis.
Original language | English (US) |
---|---|
Title of host publication | ECSCW 2009 - Proceedings of the 11th European Conference on Computer Supported Cooperative Work |
Pages | 283-302 |
Number of pages | 20 |
State | Published - Dec 1 2009 |
Event | 11th European Conference on Computer Supported Cooperative Work, ECSCW 2009 - Vienna, Austria Duration: Sep 7 2009 → Sep 11 2009 |
Other
Other | 11th European Conference on Computer Supported Cooperative Work, ECSCW 2009 |
---|---|
Country/Territory | Austria |
City | Vienna |
Period | 9/7/09 → 9/11/09 |
ASJC Scopus subject areas
- Computer Networks and Communications
- Computer Science Applications