This book is about multimodality. Taking stock of two decades of theorizing and researching, we sketch the history and current state of affairs in a fast growing academic field. Our focus is on three distinct traditions in which multimodality has been taken up: systemic functional linguistics, social semiotics and conversation analysis. We outline the history, key concepts and methods of each; point to seminal studies and critiques; and reflect on their synergies and differences. We also identify ways in which these focal approaches have been transformed through amalgamation with or incorporation into other research traditions. Throughout the book, we illustrate concepts and methods with examples from a variety of different studies, reflecting the breadth of domains that have now been explored from a multimodal perspective.
The book is written for anyone who is interested in language, communication and meaning, including undergraduate and postgraduate students in applied and sociolinguistics. No prior specialist knowledge in linguistics or semiotics is required or assumed.
The aim of the book is to help you to:
The book consists of seven chapters.
In Chapter 1, ‘Navigating a diverse field’, we discuss the history of the notion of multimodality and formulate key premises that can be used to differentiate between different types of multimodal studies and consider one’s own position in the diverse field of multimodality.
In Chapter 2, ‘Why engage with multimodality?’, we provide a rationale for doing multimodal research. We challenge widely held assumptions about language and review the arguments for taking a multimodal approach.
In Chapters 3 to 5, we set out three distinct approaches to multimodality. Chapter 3 introduces systemic functional linguistics, Chapter 4 introduces social semiotics, and Chapter 5 conversation analysis. In each of these chapters, we outline the key principles, concepts and methods of the respective approaches. We also illustrate their areas of application, as well as their limitations.
In Chapter 6, ‘Five more approaches to multimodality’, we explore the ways in which the three focal approaches discussed in Chapters 3 to 5 have been taken forward by combining them with other traditions. Thus we present five more approaches to multimodality: geo-semiotics; multimodal (inter)actional analysis; multimodal ethnography; a corpus-based approach to multimodality; and multimodal reception analysis. We introduce each of these approaches with attention to their origins, methods used and their distinct contribution to multimodality.
In Chapter 7, ‘Designing a multimodal study’, we present the steps involved in setting up a multimodal study. This includes deciding on a theoretical frame for your study, formulating research focus and questions, collecting research materials, assessing methods of transcription and considering the ethical dimensions of multimodal research.
In addition to these seven chapters, the book includes a self-study guide, designed as an optional stand-alone resource or basis for a short course. You can use this resource as a guide to reading the book and engaging with the issues raised by it. It consists of seven comprehensive study units: one for each chapter. Each unit provides the following set of resources: a chapter overview, with chapter topics and a summary; a set of study questions; a series of ‘hands-on’ exercises; tips; and suggested resources, including readings and online resources. If you consider using the self-study guide, we suggest you first read the book, and then work through the self-study guide units in the order that they are presented, rereading the accompanying chapter as you tackle each study unit.
The book also includes a glossary of terms.