Syllabus
1 – 23. 2. Introduction Intro lecture; multimodality in communication and linguistics; gesture studies 2 – 2. 3.
Multimodal corpora Review of corpora; methods of data-gathering; sign language corpora; ethical issues 3 – 9. 3. Annotation Annotation practices and schemes; inter-coder reliability; tools 4 – 16. 3.
Annotation Intro to ELAN annotation sw 5 – 23. 3. Gesture and speech Co-speech gestures: types, phases, and functions 6 – 30. 3.
Gesture and speech Relation of gesture and prosody; integrating PRAAT and ELAN 7 – 6. 4. Multimodal grammar Fitting gesture in grammatical frameworks 8 – 13. 4.
Multimodal grammar Multimodal constructions 9 – 20. 4. Case study I Verbs and gesture in Czech and English 10 – 27. 4.
Case study II Information structure and gesture in Czech and English 11 – 4. 5. Analysis Analysing and modeling multimodal data 12 – 11. 5.
Final discussion I Project presentation and discussion 13 – 18. 5. Final discussion II Project presentation and discussion * assignments Each student receives a short video extract of spontaneous production from an existing multimodal corpus (own recordings are also possible if available).
Commencing meeting 3, students will be regularly assigned minor tasks: first of all to handle the annotation software, than to apply particular aspects of work with multimodal data addressed in the seminar. At the final meeting, students’ projects will be presented and discussed.
Annotation
The seminar provides an introduction to the multimodal approaches to language, focusing especially (but not entirely) on multimodal-corpus based study of speech-accompanying gestures. The seminar is intended to incorporate (1) discussion of current literature on multimodality from various perspectives as well as (2) a hands- on work with actual multimodal language material.