Multimodality: Reconsidering language and action through embodiment
FiDiPro project 2015–2017
Multimodality in a nutshell is an inter-disciplinary approach that understands communication to be much more than just about verbal language. It is a paradox that although human communication and action are intrinsically embodied, they have been largely reduced to language by research not only in linguistics but also in the social and cognitive sciences.
Now disciplines increasingly recognize that this separation of language and the body is artificial, but we still lack theoretical and methodological insights to deal with that. Moreover, even if there exists studies on single multimodal resources (e.g. gesture), an integrated framework of analyzing several complex multimodal dimensions is needed.
The project asks: How are various multimodal dimensions integrated and coordinated in social interaction? How to conceptualize a temporal model of human interaction that considers the simultaneous, flexible, dynamic, emergent unfolding of talk and embodied action? How to represent these complex relations and show their ordered character?
The project focuses on face-to-face social interaction as the fundamental context of embodied language use. It discusses conceptualizations of temporality, sequentiality and progressivity of human interaction on the basis of video recorded interactions in their social settings (everyday conversations, institutional and professional interactions). It develops innovative methodologies to capture the details of multimodal conduct and to analyze them. The project is expected to make a major scholarly contribution to the study of human interaction.
The empirical part of the project will focus on embodied actions where a person asks someone to do something right now (directives, requests etc.) in various institutional contexts and in a diversity of languages. The methodology combines methods of Conversation Analysis, Interactional Linguistics, Ethnography and Video analysis.
The project is housed in the Department of Finnish, Finno-Ugric and Scandinavian Studies, University of Helsinki, and it is associated with the Centre of Excellence in Research on Intersubjectivity in Interaction.