Multimodality in speech and gesture: Manual gesturing influences the acoustic structure in simultaneously produced vowels

Abstract

Language and gesture are closely related on various levels of processing, including lexicon, syntax, and semantics. The relationship of gestures to phonology, however, has received only little attention from researchers in the past.

The present paper explores the relationship between speech and gesture on a phonemic level to see whether manual gesture production influences the acoustic spectrum of simultaneously produced vowel sounds.

Keywords

• Multimodality
• speech
• gesture
• vowels.