In the previous chapter, we formulated three key premises of multimodality. We highlighted that multimodality does not just mean that recognition is given to the fact that people use a number of different semiotic resources. Multimodality also means recognition of the differences among different semiotic resources and of the ways in which they are combined in actual instances of meaning making. We then also pointed to the methodological implication of these premises, namely the need to attend to multimodal wholes.
In this chapter, we will elaborate on these key premises by engaging with and challenging assumptions about language widely held among those studying language and the general public. We assume that few people will disagree with the view that language is one among a number of quite different sets of resources that humans have developed to make meaning. What we will focus on instead are the following two more contentious issues:
The counterarguments we will present are derived from the approaches to studying multimodality that are central to this book. That means that you will not find arguments here that come from psychological and ethnological studies, suggesting, for example, that ‘50 per cent of communication is body language’. The focus is on the detailed study of empirical traces of meaning making using analytical procedures from disciplines that originally focused on the study of language in use. Our arguments have both theoretical and methodological ramifications: they challenge claims about the place of language in the social world as well as claims about how insight can be gained in language.
Before we present these arguments, we need to clarify what we mean by language and mode or semiotic resource. By ‘language’ we mean speech and writing. By ‘mode’ and ‘semiotic resource’, we mean, for the moment, a set of resources, shaped over time by socially and culturally organized communities, for making meaning. That means that we break with the practice of naming all means of making meaning ‘language’, prefixed with such terms as ‘body’ or ‘sign’ or ‘non-verbal’ or ‘visual’. We argue that language is language and that the range of resources subsumed under ‘body language’ – gesture, gaze – are in fact distinctly different modes, each significantly different from language, and therefore demand separate terms. Indeed, some multimodality scholars have proposed to treat speech and writing as separate modes, for they constitute sets of resources that are only partially overlapping. (Others conceptualize language differently. For instance, in systemic functional linguistics, language is conceptualized as a semiotic resource, and spoken and written language is seen as variations in language use.) It is important to reiterate here that we use this terminology as a means of synthesizing different approaches to multimodality; it has not (yet) been universally adopted!
Early 20th-century semioticians are often given credit for having been among the first to propose studying language alongside other modes, or ‘sign systems’, as they often called them. F