Annotation
Film Studies Department
Guest Lecture
April 30, 2012, 17:30, room 429
Cinema as a research tool: from neurocinema to implicit interaction paradigms
Aleksander Väljamäe
Current neuroscience research increasingly relies on naturalistic experimental stimuli and taps into different temporal scales of our perceptual, cognitive and emotional experiences. Both virtual reality scenarios and cinematic materials are used for such experiments. Recently, Uri Hasson and his colleagues have introduced the term neurocinematics when presenting the studies that used inter-subject correlations of brain activity to assess film-viewing experiences [1]. Close to neuromarketing ideas, neurocinematic approach is rapidly drawing attention of big film studios across the world which now use scientific studies to assess viewers experiences combining different brain imaging techniques, eye-gaze tracking and peripheral physiology recordings [2].
From the perspective of our own studies and reviewing other scholars work, we aim at triggering a multidisciplinary discussion on neurocinema research questions, interesting methodologies, and common experimental stimuli. The neurocinema studies may embrace a number of research topics including temporal dynamics of the brain during cognitive, perceptual and emotional processes, unconscious processing, decision making, spatial presence, emotional synchronization, processing of narrative structures, or cross-modal emotional conditioning when combining visuals and sounds.
So far neurocinematic studies has been mainly using commercial films as experimental stimuli. The creation and usage of the standardized film clip collections similar to IAPS (International Affective Picture System) would allow easier comparisons between different studies. In addition, the experimental cinema might provide researchers with ecological yet challenging and novel stimuli. New filmmaking trends may emerge, guided by better knowledge of brain unconscious processing and extending the boundaries of viewers’ perceptual and emotional capabilities. Consider, for example, the potential cinematic usage of an experimental paradigm of rapid serial visual presentation (RSVP) from perceptual and cognitive psychology.
Visual stimulation techniques such as RSVP are also deployed in Brain-Computer Interfaces (BCI) where real-time processing of brain signals provides novel tools for communication and control. Encompassing other physiological signals these hybrid BCIs systems are becoming even more robust and versatile [3]. When combined with interactive cinema technologies, these technologies allow for implicit and unconscious interaction with audio-visual media. Such new interactive media applications can be called enactive since the impact of the technology on the human agent and the immediate effect of the human experience on the technology are inter-coupled [4]. Providing a whole new dimension to entertainment industry, enactive media can also become a powerful addition to neurocinema research agenda.
References
[1] Hasson, U., O. Landsman, B. Knappmeyer, I. Vallines, N. Rubin, and D.J. Heeger. 2008. Neurocinematics: The Neuroscience of Film. Projections: The Journal for Movies and Mind, 2 (1), 1-26.
[2] Rendall, K. 2011. Rise of Neurocinema: How Hollywood Studios Harness Your Brainwaves to Win Oscars (http://www.fastcompany.com/1731055/oscars-avatar-neurocinema-neuromarketing).
[3] Pfurtscheller, G., Allison, B. Z., Bauernfeind, G., Brunner, C., Solis Escalante, T., Scherer, R., et al. 2010. The hybrid BCI. Front. Neuroprosthet. 4, 42
[4] Kaipainen, M., Ravaja, N., Tikka, P., Vuori, R., Pugliese, R., Rapino, M., et al. 2011. Enactive Systems and Enactive Media. Embodied human - machine coupling beyond interfaces. Leonardo 44(5).
Aleksander Väljamäe has received his PhD in applied acoustics at Chalmers University of Technology, Gothenburg, Sweden, in 2007. During his PhD studies concerning multisensory perception he has being a visiting researcher at University of Barcelona (Dr. Soto-Faraco) and NTT Communication Science Labs, Japan (Dr. Kitagawa). He has being active in a number of EU funded projects: POEMS, PRESENCCIA, BrainAble, Future BNCI, CONTRAST. In 2007-2010 he has being a postdoc and a psychophysiology lab director at Laboratory for Synthetic Perceptive, Emotive and Cognitive Systems (SPECS), Universitat Pompeu Fabra, Barcelona, Spain, obtaining several grants as PI from national Spanish funding (TEC2009-13780, TEC2010-11599-E). Currently he is a senior postdoctoral researcher at Neuropsychology Laboratory, University of Graz, Austria and director of NeuroGenetics Media Lab, CIANT (International Center for Art and New Technologies in Prague), Czech Republic. His psychophysiology research concerns how audiovisual media influence humans on perceptual and cognitive level, with particular stress on the novel methods for diagnosis and treatment of various brain disorders (e.g. autism, depression, chronic pain, migraine) and new applications (BCI, neurocinema).