Multimodal Metaphor

Multimodal Metaphor
Applications of Cognitive Linguistics

11
Editors
Gitte Kristiansen
Michel Achard
Rene Dirven
Francisco J. Ruiz de Mendoza Iban ez
Mouton de Gruyter
Berlin New York
Multimodal Metaphor
Edited by
Charles J. Forceville
Eduardo Urios-Aparisi
Mouton de Gruyter
Berlin New York
Mouton de Gruyter (formerly Mouton, The Hague)
is a Division of Walter de Gruyter GmbH & Co. KG, Berlin
Printed on acid-free paper
which falls within
the guidelines of the ANSI
to ensure permanence and durability.
Library of Congress Cataloging-in-Publication Data
Multimodal metaphor / edited by Charles J. Forceville, Eduardo
Urios-Aparisi.
p. cm. (Applications of cognitive linguistics ; 11)
Includes bibliographical references and index.
ISBN 978-3-11-020515-2 (hardcover : alk. paper)
1. Metaphor. I. Forceville, Ch. (Charles) II. Urios-Aparisi,
Eduardo, 1964
P301.5.M48M85 2009
302.2dc22
2009003856
Bibliographic information published by the Deutsche Nationalbibliothek
The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie;
detailed bibliographic data are available in the Internet at http://dnb.d-nb.de.
ISBN 978-3-11-020515-2
ISSN 1861-4078
Copyright 2009 by Walter de Gruyter GmbH & Co. KG, D-10785 Berlin
All rights reserved, including those of translation into foreign languages. No part of this book
may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopy, recording, or any information storage and retrieval system, without
permission in writing from the publisher.
Printed in Germany
Table of contents

List of contributors .............................................................................................. ix

Preface ..............................................................................................................xiii

I. Setting the Scene

Chapter 1
Introduction
Charles Forceville and Eduardo Urios-Aparisi...................................................... 3

Chapter 2
Non-verbal and multimodal metaphor in a cognitivist framework:
Agendas for research
Charles Forceville .............................................................................................. 19

II. Multimodal Metaphor in Advertising

Chapter 3
Brand images: Multimodal metaphor in corporate branding
messages
Veronika Koller ................................................................................................. 45

Chapter 4
Cutting across the senses: Imagery in winespeak and
audiovisual promotion
Rosario Caballero............................................................................................... 73

Chapter 5
Interaction of multimodal metaphor and metonymy in TV
commercials: Four case studies
Eduardo Urios-Aparisi ...................................................................................... 95

Chapter 6
Nonverbal and multimodal manifestation of metaphors
and metonymies: A case study
Ning Yu........................................................................................................... 119

vi Table of contents
III. Multimodal Metaphor in Political Cartoons

Chapter 7
Visual metaphor versus verbal metaphor: A unified account
Francisco Yus................................................................................................... 147

Chapter 8
Metaphor in political cartoons: Exploring audience responses
Elizabeth El Refaie........................................................................................... 173

Chapter 9
Image alignment in multimodal metaphor
Norman Y. Teng .............................................................................................. 197

Chapter 10
Visual metaphoric conceptualization in editorial cartoons
Joost Schilperoord and Alfons Maes................................................................. 213

IV. Metaphors of Emotion in Comics, Manga, and Animation

Chapter 11
Anger in Asterix: The metaphorical representation of anger
in comics and animated films
Bart Eerden...................................................................................................... 243

Chapter 12
Pictorial metaphors of emotion in Japanese comics
Kazuko Shinohara and Yoshihiro Matsunaka................................................... 265

V. Metaphor in Spoken Language and Co-Speech Gesture

Chapter 13
Words, gestures, and beyond: Forms of multimodal metaphor in the
use of spoken language
Cornelia Mller and Alan Cienki ..................................................................... 297

Chapter 14
Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture
Irene Mittelberg and Linda R. Waugh ............................................................. 329

Table of contents vii
VI. Metaphor Involving Music and Sound

Chapter 15
Music, language, and multimodal metaphor
Lawrence M. Zbikowski ................................................................................... 359

Chapter 16
The role of non-verbal sound and music in multimodal metaphor
Charles Forceville ............................................................................................ 383

VII. Metaphor and Film

Chapter 17
Multimodal metaphor in classical film theory from the 1920s
to the 1950s
Mats Rohdin..................................................................................................... 403

Chapter 18
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor
in horror films
Gunnar Theodr Eggertsson and Charles Forceville......................................... 429

Subject index.................................................................................................... 451

Author index.................................................................................................... 461

Metaphor and metonymy index ........................................................................ 467

List of contributors
Rosario Caballero
Department of Modern Philology
University of Castilla-La Mancha
Ciudad Real, Spain

Alan Cienki
Department of Language and Communication
Vrije Universiteit Amsterdam
Amsterdam, The Netherlands

Bart Eerden
Research group Visual Rhetoric
Avans Academy, AKV|St. Joost
Breda, The Netherlands

Gunnar Theodr Eggertsson
Department of Media Studies
University of Amsterdam

Lisa El Refaie
Centre for Language and Communication Research
Cardiff University
Cardiff, United Kingdom

Charles Forceville
Department of Media Studies
University of Amsterdam

Veronika Koller
Department of Linguistics and English Language
Lancaster University
Lancaster, United Kingdom

Fons Maes
Department of Communication and Information Studies
University of Tilburg
Tilburg, The Netherlands
x List of contributors

Yoshihiro Matsunaka
Faculty of Arts
Tokyo Polytechnic University
Tokyo, Japan

Irene Mittelberg
Human Technology Centre
RWTH Aachen University
Aachen, Germany

Cornelia Mller
Department of Cultural Studies
European University Viadrina
Frankfurt Oder, Germany

Mats Rohdin
Department of Cinema Studies
Stockholm University
Stockholm, Sweden

Joost Schilperoord
Department of Communication and Information Studies
University of Tilburg
Tilburg, The Netherlands

Kazuko Shinohara
Institute of Symbiotic Science and Technology
Tokyo University of Agriculture and Technology
Tokyo, Japan

Norman Y. Teng
Institute of European and American Studies
Academia Sinica
Taipei, Taiwan

Eduardo Urios-Aparisi,
Department of Modern and Classical Languages
University of Connecticut
Storrs, CT, USA

List of contributors xi

Linda R. Waugh
Department of French and Italian
University of Arizona
Tucson, AZ, USA

Ning Yu
Department of Modern Languages, Literatures, and Linguistics
University of Oklahoma
Norman, OK, USA

Francisco Yus
Department of English Studies
University of Alicante
Alicante, Spain

Lawrence Zbikowski
Department of Music
University of Chicago
Chicago, IL, USA

Preface
The editors would like to thank at least a few among the many people who
made this book possible. First of all, although one of us had been toying
with the idea for this volume for a long time, it was the chance to organize a
conference panel on multimodal discourse at the 9th conference of the
International Pragmatics Association (Riva del Garda 2005), that really got
things going. In Jef Verschueren and Ann Verhaert we want to thank IPrA
for this opportunity. Anke Beck, at Mouton, was enthusiastic about the
book project well before it deserved that name: she actually invited us to
submit a book proposal when we had scarcely even sent out the call for
papers for the conference panel. We thank her for her trust, and Birgit
Sievert and Monika Wendland for guiding us through many practicalities at
Mouton during later stages. We are also indebted to Gitte Kristiansen,
managing editor of the Applications of Cognitive Linguistics series, and to
Francisco Ruiz de Mendoza, a staunch promoter of the book from its earli-
est beginning. Most of all we are grateful, of course, to our authors, who
graciously responded to our critical comments on chapter drafts, and our
requests for further revisions and fine-tuning.
Charles Forceville: I furthermore want to acknowledge how much I
benefited from the interactions with the many students who, over the past
ten years, followed my course on pictorial and multimodal metaphor in the
Media Studies department at the Universiteit van Amsterdam, and from the
fine papers they wrote. It is a source of pleasure and pride for me to have
two of these former students as authors in this volume. I also want to ex-
press my appreciation for the academic associations that, despite their
strong orientation toward verbal discourse, provided me with a platform to
talk about metaphor in advertising, comics, and film: the International
Cognitivist Linguistics Association (ICLA), the Researching and Applying
Metaphor (RaAM) association, and the Poetics and Linguistics Association
(PALA). It is one of the privileges of being a scholar that one has the op-
portunity to attend international conferences, where shared professional
interests often lead to warm personal contacts. I have good memories of
many such conferences. Among the numerous colleagues that have inspired
me, I will mention one person by name. Ray Gibbs has always been excep-
tionally generous to me with his time, encouragement, and expertise.
In 2008 I spent six months at the Centre for Advanced Studies of the
Royal Flemish Academy of Belgium for Science and the Arts (VLAC), in
xiv Preface
Brussels, where I had the pleasure of collaborating with Kurt Feyaerts and
Tony Veale on the research project The Agile Mind: Creativity in Models and
Multimodal Discourse. The former Royal stables at Hertogstraat 1 provided a
stimulating environment for carrying out the final editing rounds of this
volume. And last but not least, I want to say how lucky I have always been
with the sound and commonsense advice of Kuifs agency.
Eduardo Urios-Aparisi: I would like to thank my colleagues and the de-
partment of Modern and Classical Languages of the University of Con-
necticut, Storrs. I want to express my deepest gratitude to all those who
have helped me through all these years with patience, encouragement and
love.

Charles Forceville

Amsterdam, Brussels and Storrs, Connecticut, April 2009.

I

Setting the Scene
Chapter 1
Introduction
Charles Forceville and Eduardo Urios-Aparisi
All discourse is persuasive in the sense of aiming for some sort of cognitive,
emotional or aesthetic effect, or all three together, in its envisaged audience.
But purely verbal messages and texts in (mass) communication are nowa-
days often complemented, or even superseded, by information in other signi-
fying systems. Printed material (advertisements, manuals, instruction books,
maps, graphics, cartoons, etc.) usually combine, and establish interactions
between, verbal and pictorial information, while most films and TV pro-
grams in addition draw on music and non-verbal sound. Internet sites com-
bine text with pictures and sound, and pay attention to graphic lay-out. Spo-
ken language is often accompanied by gestures, while modern product design
involves not only what products look like, but also how they sound (e.g.,
cars motors, their closing doors) or even smell.
Such developments reverberate in scholarly research. Classic language
and literature faculties in the humanities are on the wane or get transformed
and relabeled as media or cultural studies departments. Academic research in
the humanities is beginning to shift from a focus on exclusively verbal text to
discourses in which language is but one albeit still highly important
communicative mode. This inescapable trend toward multimodality, whether
applauded or bemoaned, clearly transpires from the rapidly growing number
of papers, books, and conference panels with multimodal or one of its
cognates in the title.
In the current volume this important development in humanities research
is studied from the perspective of another, somewhat older paradigm shift:
the claim that metaphor is not primarily a matter of language, but structures
thought and action. This view was first systematically presented, at least in
the English-speaking world, by two book-length studies: Andrew Ortonys
(1979) edited volume Metaphor and Thought, which had its second life in a
revised and expanded edition in 1993, and George Lakoff and Mark John-
sons monograph Metaphors We Live By (1980; see also Lakoff and John-
son 2003).
4 Charles Forceville and Eduardo Urios-Aparisi
We believe that the book you have in your hands is pertinent to scholars
in both metaphorology and multimodality. Clearly, metaphorists considering
themselves adherents of the Conceptual Metaphor Theory (CMT) initiated
by Lakoff and Johnson need to take seriously at least one crucial conse-
quence of the tenet that metaphor [is] not a figure of speech, but a mode of
thought (Lakoff 1993: 210): that metaphor can occur in other modes than
language alone. Indeed they must do so, for if researching non-verbal and
not-purely-verbal metaphor does not yield robust findings, this jeopardizes
the Lakoff-and-Johnsonian presupposition that we think metaphorically.
After all, in that case the supposedly metaphorical nature of human thinking
would turn out to be a misconception: what has been presented as the CON-
CEPTUAL level of metaphor would then simply be verbal metaphor under a
different name, disguised in SMALL CAPITALS. Mark Johnson appears to
agree, arguing that lurking behind an exclusive focus on language is the
prejudice that meaning is only to be found in words. He emphasizes that the
processes of embodied meaning in the arts are the very same ones that make
linguistic meaning possible (2007: 209). Of course work to correct the one-
sided emphasis on verbal manifestations has already been done, notably on
gesture and pictures, both by authors represented in this book and by others.
What is new in this book is that it focuses not so much on non-verbal meta-
phor per se, but on multimodal metaphor, that is, on metaphors whose target
and source are rendered exclusively or predominantly in two different
modes/modalities (the terms mode and modality are currently both in
use; it is unclear at present which one will catch on) and in many cases the
verbal is one of these. The definition of a mode is an extremely thorny one
(for more discussion, see Forceville 2006/this volume). For present pur-
poses, the modes to be taken into account are two or more of the following:
(1) written language; (2) spoken language; (3) static and moving images; (4)
music; (5) non-verbal sound; (6) gestures. Since what can be conveyed in
terms of facts, emotions, and aesthetic pleasure differs from one mode to
another, the choices for (one) particular mode(s) over (an)other(s) that the
producer of a multimodal metaphor has to make is/are bound to affect its
overall meaning. One modes potential to render meaning can never be
completely translated into that of another mode and sometimes transla-
tion is downright impossible. For this reason alone, a healthy theory of (cog-
nitive) metaphor must systematically study non-verbal and multimodal
metaphor. It may well be indeed it is very probable that the excessive
emphasis on the verbal manifestations of metaphorical thought has blinded
researchers to dimensions of the latter that quite simply cannot be cued by
the verbal mode.
Introduction 5
But researchers in the field of multimodal discourse can in their turn
benefit from the work done by interdisciplinary-oriented (but often linguisti-
cally trained) metaphor scholars. It is true that semiotics, rooted in the
structuralism of the 1960s and 1970s, deserves credit for being the first dis-
cipline to have conducted sustained research into non-verbal communication,
at least if we discount art history, which has necessarily always had a more
restricted focus. It is therefore also no coincidence that some of the contribu-
tors in this volume propose to marry insights from semiotics to those of cog-
nitivist linguistics and neither is the recent foundation of a journal called
Cognitive Semiotics. However, multimodal discourse is a vast territory,
comprising a multitude of material carriers (paper, celluloid, videotape, bits
and bytes, stone, cloth ), modes (written language, spoken language,
visuals, sound, music, gesture, smell, touch), and genres (art, advertising,
instruction manual; or at a more detailed level, say, comedy, film noir,
Western, science fiction), many of these being further categorizable. It
seems at this moment in time impossible, therefore, to provide anything ap-
proaching a holistic blueprint of multimodal discourse although attempts
have been made (Kress and Van Leeuwen 1996/2006, 2001; Baldry and
Thibault 2006; OHalloran 2004; but see Ventola et al. 2004 for more fo-
cused approaches). By contrast, systematically tracing the possible manifes-
tations of a specific concept such as metaphor across various material
carriers, modes, and genres, will signpost promising scholarly avenues, we
trust, for how to analyze yet other aspects of multimodal discourse.
One way to date the conception of this book is to say that its seed was
planted at The pragmatics of multimodal representations panel that we, the
editors, organized at the 9
th
International Pragmatics Conference (Riva del
Garda, Italy, 1015 July 2005). In the call for papers we had emphasized we
were particularly interested in multimodal metaphor, and in the end the ma-
jority of the submissions focused specifically on this topic. Along with these,
other scholars we knew to have the expertise to bridge cognitive linguistics
and the budding discipline of multimodal discourse were approached with
the request to submit an abstract. They were given detailed guidelines about
the books concept, and about how we envisaged each contribution fitting in.
In order to ensure internal coherence, it was suggested that all prospective
contributors take their cue for the definition of multimodal metaphor from
the position paper by Forceville (2006/this volume) or else that they make
clear why and how they deviated from it. Moreover, we requested that pro-
spective contributors apply theoretical concepts systematically to one or
more real-life case studies, the idea being that this procedure would fruitfully
force them to face problems that mere introspective reasoning often circum-
vents (cf. Haser 2005: 50). In addition, each chapter is thereby expected to
spawn ideas how the proposed procedure can be deployed to analyze other
multimodal representations than those examined there. Contributors were
also encouraged to present (some of) their conclusions in a form that allows
for empirical testing. Most of those we approached responded positively, and
of the latter, the majority of the delivered chapters displayed the quality we
had in mind. Early drafts of the chapters were extensively commented upon
both by the editors and by one other contributing author.
The guiding principle running through the chapters is a consideration of
which modes play a role in the identification and interpretation of the meta-
phors studied. Almost invariably, this entails taking into account the genre to
which the discourse featuring a multimodal metaphor belongs: advertise-
ments, political cartoons, comics, animation, musical compositions, oral
conversations and lectures, feature films. A third recurring dimension is the
extent to which a metaphor is not only embodied but also governed by the
cultural or professional community in which it functions. We will now
briefly introduce each of the chapters in the book.
Chapter 2 is a slightly updated version of the position paper on pictorial
and multimodal metaphor by Forceville (2006). This paper provides and
discusses the definition of multimodal metaphor that contributors to the cur-
rent volume were asked to use or else explain why they opted for an alter-
native definition.
The first cluster of chapters pertains to multimodal metaphor in advertis-
ing. It makes sense to begin with this topic, since advertising has been the
subject of a number of studies pertaining to pictorial metaphor the variety
of non-verbal metaphor that hitherto has attracted most scholarly attention.
This is not surprising, for advertising constitutes a body of texts and prac-
tices that is persuasive par excellence. It allows bringing into play the modes
of language, visuals, and sound/music. The first contribution in this cluster,
Brand images: Verbal and visual metaphor in corporate branding mes-
sages, by Veronika Koller (chapter 3), charts how the logos, visuals, and
layouts that are used to create companies corporate identities often require
or invite the construal of metaphors. Tying in with the pervasive BRANDS
ARE LIVING ORGANISMS metaphor, visual elements often subtly encourage
the inference of positive corporate qualities that are not necessarily verbal-
ized. Identifying the metaphorical mechanisms deployed to achieve this goal
points the way to how the inevitably biased nature of companies self-
portraits can be critically examined.
Chapter 4 is Rosario Caballeros Cutting across the senses: Imagery in
winespeak and audiovisual promotion. The chapter is part of an ongoing
Introduction 7
research project which is partly based on an impressive corpus of 12,000
wine tasting notes in professional journals, and here takes into account Span-
ish and French wine advertisements as well. Clearly, since taste and smell
wines most important characteristics cannot be directly represented, their
verbal and visual descriptions must rely on synaesthesia and metaphor. An
important issue in the chapter is the difficulty of the translation of these
hardly theorized modes of taste and smell into a shared vocabulary of
pictures and words. Another pertinent issue is the role of the cultural back-
ground governing both the choice of source domain in purely verbal meta-
phors describing wines and the choice of visuals in the advertisements.
Eduardo Urios-Aparisis Interaction of multimodal metaphor and me-
tonymy in TV commercials: Four case studies (chapter 5) discusses in-
stances of Spanish television commercials. He addresses how Forcevilles
(2006/this volume) multimodal metaphor interacts with metonymical map-
pings, and applies the taxonomy found in Ruiz de Mendoza and Dez
Velasco (2002) to multimodal advertising texts, identifying their cognitive
value and communicative strategies within this genre. He shows how meta-
phor and metonymy fulfil different cognitive and discursive roles, serving to
identify the target of a metaphor, to limit the correspondences between the
domains, or to expand and create new meanings.
In Nonverbal and multimodal manifestations of metaphors and metony-
mies: A case study (chapter 6), Ning Yu provides an in-depth analysis of a
single educational message (a non-commercial commercial, if you like)
broadcast on Chinese national TV in terms of two conceptual metaphors
whose purely verbal varieties have often been discussed: LIFE IS A JOURNEY
and LIFE IS A STAGE. He shows how aspects of these metaphors, which in
some passages are blended (Fauconnier and Turner 1998, 2002), surface
in various modes. In several scenes, moreover, other conceptual metaphors
such as UNDERSTANDING IS SEEING and SUCCESSFUL IS UP are shown to
play a role, as well as a range of metonymies. The analysis makes clear that
whereas thanks to the visuals, the embodied aspects of the metaphors are
presumably universally comprehensible, many details can only be fully ap-
preciated by viewers aware of specific Chinese myths and beliefs (cf. Force-
ville et al. 2006).
The second cluster of chapters pertains to a different textual genre: politi-
cal cartoons. While a crucial presupposition in advertising is that, one way
or another, a positive claim is made about the product, service, or idea ad-
vertised, political cartoons, by contrast, are characterized by the convention
that something critical or negative is conveyed about one or more persons, or
a state of affairs, in the world. Chapter 7, Visual metaphor versus verbal
metaphor: A unified account, by Francisco Yus, mounts the argument that
verbal and visual metaphors are rooted in the same cognitive mechanism.
Drawing on Sperber and Wilsons (1995) relevance theory and Fodors
(1983) modularity of mind theory, he takes the CMT claim that textual
surface manifestations of metaphors can be traced back to conceptual meta-
phors to imply that there is no substantial difference between how verbal,
pictorial, and multimodal metaphors are processed. Analyzing a number of
cartoons by the Spanish artist El Roto, Yus demonstrates that the interpreta-
tion of each creative metaphor, irrespective of the mode(s) in which it is
presented, depends on the formation of ad hoc concepts and on emergent
properties (Gineste et al. 2000; Fauconnier and Turner 2002).
Elizabeth El Refaies Metaphor in political cartoons: Exploring audi-
ence responses (chapter 8) further illuminates the reader about the cartoon
genre by investigating two British specimens. As in Yus case study, the
source-path-goal schema, with its LIFE IS A JOURNEY manifestation, is em-
phatically present. Since in both Yus educational commercial and El
Refaies cartoons purposiveness as well as temporal development needs to
be conveyed, this is hardly unexpected. After providing her own interpreta-
tion of the cartoons which turns out to be consonant with their creators
intentions El Refaie reports part of a larger research project in which these
two cartoons were presented to, mainly non-native, British youngsters. She
finds that these adolescents are often seriously deluded about what is hap-
pening in the cartoons, with consequences for their interpretations that are as
alarming as they are humorous.
Norman Tengs Image alignment in multimodal metaphor (chapter 9)
addresses the role of patterned visual entities in cartoons. One way of creat-
ing similarities between different visual elements is by presenting them as
featuring the same orientation, color, size or any other saliently shared
aspect of design. Teng discusses how such alignments can play a role in
multimodal metaphors. Examining six cartoons by Clay Bennett, he more-
over suggests that alignment may be the preferred design choice to convey
the abstract concept of similarity between two or more items. Tengs chap-
ter, finally, suggests avenues for research into other multimodal tropes be-
sides metaphor.
Joost Schilperoord and Alfons Maes discuss a variety of Dutch cartoons
in chapter 10, Visual metaphoric conceptualization in editorial cartoons,
arguing that for an appropriate understanding of the metaphors in cartoons
image schema-based reasoning needs to be complemented by taxonomic
reasoning, since the latter is often the crucial trigger in interpreting the
critical stance expressed in editorial cartoons. The authors thus focus not so
Introduction 9
much on the pragmatic knowledge a viewer brings to a cartoon, but on the
text-inherent information that guides metaphor interpretation, which they
believe will permit the identification of textual genre-patterns. Examples of
three subtypes of pictorial metaphor are examined in detail, and a number of
source domains that appear to be particularly popular in cartoon metaphors
are identified, such as hospitals, marriage, funerals, and boxing.
Based on work by Kvecses (1986, 2000) and Forceville (2005), the next
two chapters examine how emotions, specifically the paradigm case of an-
ger, are visualized in comics, and to what extent there is cultural variation
in such renderings. This cluster shifts the focus from advertising and politi-
cal cartoons to comics and animation, retains the cross-cultural dimension,
and addresses the notion of structural (in contrast to creative) metaphors. In
chapter 11, Anger in Asterix: The metaphorical representation of anger in
comics and animated films, Bart Eerden compares Forcevilles findings not
only to those surfacing from the analysis of another Asterix album, but also
to the data elicited from two animation films based on Asterix albums. After
all, since the medium is the message, it is likely that the visual signs commu-
nicating an emotion in animated film are not completely identical to those
found in comics. Kazuko Shinohara and Yoshihiro Matsunaka pursue the
investigations of the EMOTIONS ARE FORCES metaphor in chapter 12, Picto-
rial metaphors of emotion in Japanese comics, but they provide a novel
perspective by analyzing Japanese manga rather than Western comics. As a
consequence, they are able to shed light on which visual signs reflect pre-
sumably universal aspects of the metaphor, and which are manifestations of
knowledge that is tied to a specific culture. Both chapters in this cluster
strongly suggest that conceptual metaphors find expression in visual signs in
ways that are not always translatable into language, and therefore may be
direct manifestations of these conceptual metaphors, unmediated by lan-
guage.
Spoken language and gestures are so closely interdependent that they
really should be studied together (McNeill 1992, 2005; Cienki 1998). It is
thus to be expected that multimodal metaphor frequently and naturally oc-
curs in face-to-face communication. In the next cluster, two chapters discuss
metaphors drawing on the gestural and spoken language modes. In chapter
13, Words, gestures and beyond: Forms of multimodal metaphor in the use
of spoken language, Cornelia Mller and Alan Cienki distinguish between
various types of monomodal and multimodal metaphor that are possible in
spoken language accompanied by gestures, giving examples of each. In addi-
tion, they argue that intonation is an under-researched area of conceptual
metaphor. Their work supports the central CMT idea that metaphor is a
conceptual phenomenon, but also demonstrates that specific modes each
have their own affordances and limitations for conveying dimensions of such
conceptual metaphors. Irene Mittelberg and Linda Waugh show in chapter
14, Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture, that gestures may
manifest dimensions of conceptual metaphors that are not found in the co-
occurring speech and that, moreover, in gesture awareness of metonymy
should be considered as an indispensable stage in the process of accessing
metaphor.
The chapters in the next cluster are specifically devoted to the musical
and sonic contributions to multimodal metaphors. Lawrence Zbikowski dis-
cusses in Music, language, and multimodal metaphor (chapter 15) how
significant aspects of conceptual metaphors in a number of classical and
popular music fragments depend exclusively on the musical, as opposed to
the verbal, mode. Zbikowski is careful to point out, however, that for these
musical elements to be experienced as metaphorical, they need to be consid-
ered in conjunction with the theme of the piece. Moreover, not only map-
pings from language to music are possible, but also vice versa. Zbikowski
concludes that to do full justice to the respective contributions of text and
music to the various musical pieces scrutinized, in a number of cases a mul-
timodal blending approach (Fauconnier and Turner 2002) provides a better
model than a multimodal metaphor construal. In both, he maintains, music
appears particularly suitable in supplying sonic analogs to dynamic proc-
esses. In chapter 16 in the cluster, The role of non-verbal sound and music
in multimodal metaphor, Charles Forceville considers what sonic and musi-
cal sources contribute to the identification and interpretation of multimodal
metaphors in two genres, commercials and fiction films. Whereas Zbikowski
sometimes considers the combinations of text and music best theorizable in
terms of blends, Forcevilles cases, drawing on visuals and music often in
conjunction with texts appear all to impose a clear directionality for map-
pings from a source to a target, and hence can typically be considered mul-
timodal metaphors. He ends the chapter with a series of preliminary claims,
to be tested in further research in this field.
The chapters in the final cluster have been written by scholars with a
cognitivist film theory rather than a cognitivist linguistics background. Mats
Rohdin, in Multimodal metaphor in classical film theory from the 1920s to
the 1950s (chapter 17), reminds us that reflection on non-verbal metaphor
has a long tradition in film studies. He examines a series of classic texts that
discuss cinematic metaphor, and considers to what extent the various ap-
proaches are consonant with the multimodal metaphor model adhered to in
Introduction 11
this volume. Rohdin thus is the only contributor to present a diachronic per-
spective on the issue of multimodal metaphor. Moreover, he draws attention
to the fact that cinematic metaphors may acquire extra meanings because
through visual styling they can create intertextual references to other films
and phenomena familiar from everyday life. Finally, Rohdin finds that, con-
trary to expectation, the silent cinema was particularly rich in multimodal
metaphors of the verbo-pictorial variety, due to the creative use of intertitles.
The final chapter, co-authored by Gunnar Eggertsson and Charles Force-
ville, is titled Multimodal expressions of the HUMAN VICTIM IS ANIMAL
metaphor in horror films (chapter 18). Its key argument is that human vic-
tims in extreme horror films are typically abused as if they were animals.
The findings shed light on metaphor theory, the genre of horror films, but
they also encourage reflection on the issue of animal rights for, in the spirit
of Kvecses (2005) we can adapt a famous dictum and say: show me your
metaphors and I will tell you who you are.
The division in clusters and chapters chosen loosely on the basis of gen-
res and modes could have been made in different ways, since many other
thematic patterns can be detected across the chapters of the book. Without
elaborate discussion, we will briefly list some of these patterns, presenting
them as something with a status that hovers between hypothesis and research
program. Some of the issues have been discussed in relation with verbal
metaphors, but often their importance has been underestimated in that realm;
others appear to reveal themselves precisely thanks to the multimodal nature
of the metaphors that are the specific focus of attention here.
Many metaphors are mini-narratives. The paradigmatic NOUN A IS
NOUN B formula disguises the dynamic nature of metaphor. Human beings
move literally through space and figuratively through time, and it is within
these parameters that they need to make sense of their lives. This sense-
making happens through real or imagined metaphor actions; it would per-
haps be better to conceive of metaphor as A-ING IS B-ING, since metaphor is
always metaphor in action. The A IS B format which maybe became popu-
lar also because CMT long discussed only decontextualized metaphors that
already came in a ready-made verbal A is B form is no more than a con-
venient short-hand for what Andreas Musolff calls a metaphor scenario
(Musolff 2006). And of course we should not forget that Paul Ricoeur
(1977) already strongly emphasized the discursive character of metaphor.
Though not always explicitly, all chapters in the volume tie in with this no-
tion of a scenario or a narrative.
Target and source in multimodal metaphor may both be concrete enti-
ties. Classic CMT has always stressed that human beings can only come to
grips (sic) with the abstract by metaphorically coupling it with the concrete
i.e., with that which is perceptible. But the chapters in this volume are re-
minders that only a target that is concrete is, for instance, depictable, which
is important in advertising a product, satirizing a politician in a cartoon, or
conveying information about a character in a film. The focus on verbal
manifestations of conceptual metaphors, that is, has had as an unfortunate
side effect that for instance the stylistic dimensions of metaphors and other
tropes have been somewhat ignored by cognitivist scholars (but cf. Semino
and Culpeper 2002). Many illuminating (aesthetic as well as persuasive)
multimodal metaphors convey something about this specifically styled target
in terms of this specifically styled source. Moreover, while the embodied
nature of conceptual metaphors is one of the basic tenets of CMT, Caballero
(this volume) correctly points out that the embodied domains of smell and
taste need rather than provide metaphorical sources. The strong focus on a
bottom-up approach (from attested textual manifestations to formulations
of the conceptual metaphors which supposedly underlie them rather than the
other way round) may also be the reason why in several of the chapters there
is some interference of the terminology associated with Max Blacks (1979)
interaction theory. Black whose early contributions to cognitive theories of
metaphor have insufficiently been acknowledged by most CMT theorists
anticipated that metaphor could be a matter of thought rather than language,
but discussed specific, creative metaphors in terms of features that were
projected or transferred from source to target. CMT favors referring to this
process as the partial mapping of entities and knowledge structures from
source to target, resulting in a (temporary) understanding of the target in
terms of the source but the occasional lapse into Blacks terminology is a
healthy reminder that sometimes no more than a single aspect (feature) of
the source is mapped.
It is impossible to study metaphor without addressing metonymy. Me-
tonymy has over the past decade begun to receive sustained attention from
cognitive linguists (Barcelona 2000; Dirven and Prings 2002; Kristiansen
et al. 2006). Clearly, each property or feature that is mapped from a source
to a target must first have been metonymically related to that source. Of
course, a metonym can be an ad hoc one, created by a particular context or
shared by a specific community of users (cf. Yus, this volume). In addition,
a metonym may have a strong emotional or evaluative relation to its source
and it may well be this latter that is the rationale for the metaphor in the first
place. Secondly, a given phenomenon may double as the source domain in a
metaphor and as metonymically related to the target. If this is the case, the
consequence may be that a construal of the relation between two things as
Introduction 13
metaphor is invited rather than forced; after all there may be a realistic,
metonymic motivation for the sources presence on the grounds of expected
contiguity in the domain of the target. The interaction between metaphor and
metonymy is explicitly addressed in the chapters by Urios-Aparisi, Yu, and
Mittelberg and Waugh.
Non-verbal and multimodal metaphors may make salient certain as-
pects of conceptual metaphors that are not, or not as clearly, expressible in
their verbal manifestations. The role of for instance size and spatial dimen-
sions in source domains (e.g., in POWERFUL IS BIG, HONEST IS STRAIGHT) is
more noticeable in visual discourses than in verbal ones. Music, in turn,
affords for example scalarity and loudness in ways that can be made produc-
tive in source-to-target mappings, and the same holds for a voices timbre or
an intonational pattern. Arm-and-hand gestures, both in face-to-face interac-
tion and in the stylized varieties characterizing protagonists behaviors in
comics, manga, and animation are embodied actions whose metaphorical
exploitation communicates perspectives and emotions not (readily) available
in verbal metaphors. A consequence of this is that any translation of these
non-verbal and multimodal metaphors into verbal ones necessary for in-
stance to enable scholarly discussion as in this book inevitably is an ap-
proximation at best. Metaphor scholars should be acutely aware of this, and
reflect on what the choice for one verbalization of a multimodal metaphor
over another may entail. The verbal short-hands of multimodal metaphors
suggest an explicitness and precision that may well be absent in their origi-
nally non-verbal or multimodal, forms. Aspects of this issue are addressed in
the chapters by Eerden, Shinohara and Matsunaka, Yu, Yus, El Refaie,
Mittelberg and Waugh, Mller and Cienki, Teng, Rohdin, Zbikowski, and
Forceville.
Personification is a crucial variety of multimodal metaphor no less than
of verbal metaphor. Living organisms and animals are attractive choices as
source domains both for human target domains and for phenomena such as
organizations and cars. This makes sense for a variety of reasons: as hu-
mans, we find fellow humans as well as animals provide rich opportunities
for the mapping both of idiosyncratic features (snails are typically slow,
peacocks proud and beautiful) and for what Black called implicative com-
plexes (Black 1979) and Gentner and Loewenstein (2002) aligned struc-
tures. To a considerable extent, the place of humans and animals in the
medieval hierarchy of the Great Chain of Being (see Tillyard 1976 [1943],
Lakoff and Turner 1989) is still pertinent today, but creatures status can
also be strongly influenced by cultural myths (think of the connotations of
the dragon in Western versus Chinese culture). Finally, it is attractive that
people and animals move, which allows for numerous ways in which a
metaphor producer can focus attention on mappable features particularly
in film. Chapters in which this issue of the animal realm, and of living or-
ganisms more generally, as source domain, receives attention are those by
Koller, Caballero, Urios-Aparisi, Schilperoord and Maes, Forceville, Roh-
din, and Eggertsson and Forceville.
Under what circumstances can or must a multimodal metaphor be con-
strued? This is a difficult but crucial issue, particularly where a conceptual
metaphor is assumed to be present. This can be rephrased as the following
question: is the phenomenon under consideration necessarily to be interpreted
as a metaphor, i.e., as one thing presented in terms of something that, given
the context, belongs to a different category, or are other, non-metaphorical
construals of their co-occurrence possible or even likely? This is a critical
question for metaphor scholars. If the central tenet of CMT that in essence
we think metaphorically is correct, metaphor scholars, working on verbal,
non-verbal and multimodal specimens alike, should be able to demonstrate
its truth, or at least probability, by showing that the phenomena under con-
sideration can be best explained by postulating that human beings make
sense of them by consciously or automatically construing metaphors. But
even identifying verbal metaphors as such is no simple affair, although the
Pragglejaz Group (2007) has started to develop a procedure for this. To
make further progress on this issue it is necessary that alternative hypotheses
are specified that might account for the phenomena under discussion (Gibbs
and Perlman 2006: 217; for an alternative proposal see Haser 2005: 149 et
passim), so that metaphorical and alternative explanations may be coolly
juxtaposed and critically debated. This task, no easy matter to start with, is
further complicated in the case of metaphors occurring in artistic discourses.
Often, in such discourses, coupling two things metaphorically is not nec-
essary to make the segment of discourse in which they occur meaningful,
since alternative explanations for their co-occurrence are available. That is,
a discourse producer may have reasons not to emphasize that a metaphor is
to be construed. Evading censorship, avoiding litigation, or simply wanting
to create a polyvalent discourse for aesthetic pleasure can motivate a maker
not to produce a strongly signaled metaphor (cf. Forceville 1999: 19196).
We are fully aware that many problems still have to be solved in the
realm of multimodal metaphor, but we are confident that the present volume
will give a substantial boost to its further theorization.
Introduction 15
References
Baldry, Anthony, and Paul J. Thibault
2006 Multimodal Transcription and Text Analysis: A Multimedia Toolkit
and Coursebook. London/Oakville: Equinox.
Barcelona, Antonio (ed.)
2000 Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
tive. Berlin/New York: Mouton de Gruyter.
Black, Max
1979 More about metaphor. In Metaphor and Thought, Andrew Ortony
(ed.), 1943. Cambridge: Cambridge University Press.
Cienki, Alan
1998 Metaphoric gestures and some of their relations to verbal meta-
phoric expressions. In Discourse and Cognition: Bridging the Gap,
Jean-Pierre Koenig (ed.), 189204. Stanford, CA: Center for the
Study of Language and Information.
Dirven, Ren, and Ralf Prings (eds.)
2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New
York: Mouton de Gruyter
Fauconnier, Gilles, and Mark Turner
1998 Conceptual integration networks. Cognitive Science 22: 133187.
2002 The Way We Think: Conceptual Blending and the Minds Hidden
Complexities. New York: Basic Books.
Fodor, Jerry
1983 The Modularity of Mind. Cambridge, Mass.: The MIT Press.
Forceville, Charles
1999 The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pin-
ters, and Paul Schraders The Comfort of Strangers. Metaphor and
Symbol 14: 17998.
2005 Visual representations of the Idealized Cognitive Model of anger in
the Asterix album La Zizanie. Journal of Pragmatics 37: 6988.
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: Agendas for research. In Cognitive Linguistics: Current Ap-
plications and Future Perspectives, Gitte Kristiansen, Michel
Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez
(eds.), 379402. Berlin/New York: Mouton de Gruyter.
Forceville, Charles, Ed Tan, and Paul Hekkert
2006 The adaptive value of metaphors. In Heuristiken der Literatur-
wissenschaft. Einladung zu disziplinexternen Perspektiven auf
Literatur, Uta Klein, Katja Mellmann, and Steffanie Metzger (eds.),
85109. Paderborn: Mentis.
Gentner, Dedre, and Jeffrey Loewenstein
2002 Relational language and relational thought. In Language, Literacy,
and Cognitive Development, Eric Amsel and James P. Byrnes
(eds.), 87120. Mahwah, NJ: Erlbaum.
Gibbs, Raymond W., Jr., and Marcus Perlman
2006 The contested impact of cognitive linguistics research on the psy-
cholinguistics of metaphor understanding. In Cognitive Linguistics:
Current Applications and Future Perspectives, Gitte Kristiansen,
Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza
Ibez (eds.), 211228. Berlin/New York: Mouton de Gruyter.
Gineste, Marie-Dominique, Bipin Indurkhya, and Vronique Scart
2000 Emergence of features in metaphor comprehension. Metaphor and
Symbol 15: 117135.
Haser, Verena
2005 Metaphor, Metonymy, and Experientialist Philosophy: Challenging
Cognitive Semantics. Berlin/New York: Mouton de Gruyter.
Kvecses, Zoltn
1986 Metaphors of Anger, Pride and Love. Amsterdam/Philadelphia:
Benjamins.
2000 Metaphor and Emotion: Language, Culture, and Body in Human
Feeling. Cambridge: Cambridge University Press.
2005 Metaphor in Culture: Universality and Variation. Cambridge: Cam-
bridge University Press.
Kress, Gunther, and Theo van Leeuwen
1996 Reading Images: The Grammar of Visual Design. 2nd ed. Revised
edition published in 2006. London/New York: Routledge.
2001 Multimodal Discourse: The Modes and Media of Contemporary
Communication. London: Arnold.
Lakoff, George
1993 The contemporary theory of metaphor. In Metaphor and Thought,
2nd ed. Andrew Ortony (ed.), 202251. Cambridge: Cambridge
University Press.
Lakoff, George, and Mark Johnson
1980 Metaphors We Live By. Chicago: University of Chicago Press.
2003 Afterword, 2003. In Metaphors We Live By, 243276. Chicago:
University of Chicago Press.
Lakoff, George, and Mark Turner
1989 More Than Cool Reason: A Field Guide to Poetic Metaphor. Chi-
cago: University of Chicago Press.
McNeill, David
1992 Hand and Mind: What Gestures Reveal about Thought. Chicago:
2005 Gesture and Thought. Chicago: University of Chicago Press.
Introduction 17
Musolff, Andreas
2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21:
2338.
OHalloran, Kay L. (ed.)
2004 Multimodal Discourse Analysis. London/New York: Continuum.
Ortony, Andrew (ed.)
1979 Metaphor and Thought. Cambridge: Cambridge University Press.
1993 Metaphor and Thought. 2nd ed. Cambridge: Cambridge University
Press.
Pragglejaz Group
2007 MIP: A method for identifying metaphorically used words in dis-
course. Metaphor and Symbol 22: 139.
Ricoeur, Paul
1977 The Rule of Metaphor: Multi-disciplinary Studies of the Creation of
Meaning in Language. Trans. by R. Czerny, Kathleen McLaughlin,
and John Costello. Toronto: University of Toronto Press.
Ruiz de Mendoza, Francisco Jos, and Olga Isabel Dez Velasco
2002 Patterns of conceptual interaction. In Metaphor and Metonymy in
Comparison and Contrast, Ren Dirven and Ralph Prings (eds.),
489532. Berlin/New York: Mouton de Gruyter.
Semino, Elena, and Jonathan Culpeper (eds.)
2002 Cognitive Stylistics: Language and Cognition in Text Analysis. Am-
sterdam/Philadelphia: Benjamins.
Sperber, Dan, and Deirdre Wilson
1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Black-
well.
Tillyard, E.M.W.
1976 [1943] The Elizabethan World Picture. Harmondsworth: Penguin.
Ventola, Eija, Cassily Charles, and Martin Kaltenbacher (eds.)
2004 Perspectives on Multimodality. Amsterdam/Philadelphia: Benja-
mins.
Chapter 2
Non-verbal and multimodal metaphor in a
cognitivist framework: Agendas for research
1

Charles Forceville
Abstract
Conceptual metaphor theory (CMT) has over the past 25 years amply sought to
underpin the claim that humans pervasive use of verbal metaphor reflects the fact
that they think largely metaphorically. If this tenet of CMT is correct, metaphor
should manifest itself not just in language but also via other modes of communica-
tion, such as pictures, music, sounds, and gestures. However, non-verbal and mul-
timodal metaphor have been far less extensively studied than their verbal sisters.
The present chapter provides a review of work done in this area, focusing on a
number of issues that require further research. These issues include the proposal
to distinguish between monomodal and multimodal metaphor; reflections on the
difference between structural and creative metaphor; the question of how verbali-
zations of non-verbal or conceptual metaphors may affect their possible interpreta-
tion; thoughts as to how similarity between target and source is created; and sug-
gestions about the importance of genre for the construal and interpretation of
metaphor.

Keywords: Monomodal and multimodal metaphor, pictorial metaphor, structural
versus creative metaphor, similarity in metaphor, genre.
1. Introduction
Andrew Ortonys edited volume Metaphor and Thought (1979) and Lakoff
and Johnsons monograph Metaphors We Live By (1980) were milestone
publications in the sense that they marked the switch from research into
metaphor as a primarily verbal to a predominantly conceptual phenomenon.
The conceptual metaphor theory (CMT), as the Lakoffian-Johnsonian
20 Charles Forceville
model is habitually referred to, has been a very productive one (e.g., Gibbs
1994; Johnson 1987, 1993, 2007; Kvecses 1986, 2000, 2002; Lakoff 1987,
1993; Lakoff and Johnson 1999, 2003; Lakoff and Turner 1989; Sweetser
1990; Turner 1996). A key notion in this theory is that the mind is inher-
ently embodied, reason is shaped by the body (Lakoff and Johnson 1999: 5;
Chapter 3). Very briefly, what this means is the following. Human beings
find phenomena they can see, hear, feel, taste and/or smell easier to under-
stand and categorize than phenomena they cannot. It is perceptibility that
makes the former phenomena concrete, and the lack of it that makes the lat-
ter abstract. In order to master abstract concepts, humans systematically
comprehend them in terms of concrete concepts. Thus abstract concepts such
as LIFE, TIME, and EMOTIONS are systematically understood in terms of con-
crete phenomena. LIFE is understood as A JOURNEY (Hes without direction
in his life; Im at a crossroads in my life) but also, for instance, as A
STORY (Tell me the story of your life; Lifes a tale told by an idiot ...).
TIME is comprehended in terms of SPATIAL MOTION (The time for action has
arrived; Time is flying by; He passed the time happily). Emotions are
typically represented by drawing on the domain of FORCES. (I was over-
whelmed; I was swept off my feet; examples from Kvecses 2000; Lakoff
1993; Lakoff and Johnson 1980). Conceptualizations of many phenomena,
CMT proposes, have deeply entrenched metaphorical forms, in which the
metaphors target (topic, tenor) is abstract and its source (vehicle, base) is
concrete. A metaphors interpretation boils down to the mapping of perti-
nent features from the source to the target; a mapping that in the case of
entrenched metaphors such as the above occurs automatically. Since con-
creteness is apprehended perceptually, metaphorical source domains are
strongly rooted in the functioning of the human body. Metaphorical reason-
ing is thus governed by the arch metaphor MIND IS BODY (Lakoff and
Johnson 1999: 249). A more recent development rooted in CMT is blending
theory (Fauconnier and Turner 2002). Rather than postulating a target and
a source domain, it presents two (or more) input spaces. The input spaces
have both shared and unique characteristics, and it is this combination that
allows for the construal of a so-called blended space. Blending theory is a,
mainly descriptive, model claiming to be superior to metaphor theory in be-
ing able to account for ad hoc linguistic creativity, metaphorical and other-
wise. Hitherto it cannot quite convince (for a critical review, see Forceville
2004a), but new work, taking into account pragmatic rhetorical factors, is
promising (see Coulson and Pascual 2006; Terkourafi and Petrakis, forth-
coming).
Metaphor in a cognitivist framework 21
CMT has inspired conferences (e.g., those organized by the International
Cognitive Linguistics Association, and the Researching and Applying Meta-
phor [RaAM] association), journals (e.g., Metaphor and Symbol, Cognitive
Linguistics), as well as empirical research (for references see Gibbs 1994,
2006; Lakoff and Johnson 1999: 58788). Its importance is evident: if CMT
is basically correct, it provides crucial insights into what, thanks to embodi-
ment, lays claim to being universal in human cognition, and what is rooted
in, and shapes, (sub)cultural differences.
However, CMT is restricted in at least the following very important di-
mension. Even though Lakoff and Johnsons (1980: 5) characterization of
metaphors essence as understanding and experiencing one kind of thing in
terms of another emphatically avoids the word verbal or linguistic, the
validity of CMTs claims about the existence of conceptual metaphors de-
pends almost exclusively on the patterns detectable in verbal metaphors.
This entails two dangers: in the first place, there is the risk of a vicious cir-
cle: cognitive linguistic research suffers from circular reasoning in that it
starts with an analysis of language to infer something about the mind and
body which in turn motivates different aspects of linguistic structure and
behavior (Gibbs and Colston 1995: 354; see also Cienki 1998). Clearly, to
further validate the idea that metaphors are expressed by language, as op-
posed to the idea that they are necessarily linguistic in nature, it is impera-
tive to demonstrate that, and how, they can occur non-verbally and multimo-
dally as well as purely verbally. Secondly, an exclusive or predominant
concentration on verbal manifestations of metaphor runs the risk of blinding
researchers to aspects of metaphor that may typically occur in non-verbal
and multimodal representations only. This latter awareness, of course, ex-
emplifies a more general principle. Ever since Marshall McLuhans the
medium is the message (McLuhan 1964: 24 et passim), it is a truism that as
soon as one changes the medium via which a message (including both its
factual and emotive aspects) is conveyed, the content of this message is
changed as well (see also Bolter and Grusin 1999). Each medium here
defined as a material carrier and transmitter of information communicates
via one or more signaling systems. The medium of non-illustrated books, for
instance, exclusively draws on the mode of written language; radio relies on
the modes of spoken language, non-verbal sound, and music; advertising
billboards on written language and visuals; and post-silent film on visuals,
written language, spoken language, non-verbal sound, and music. If, as is
argued here, each of these signaling systems (which will henceforward be
called modes) can cue, independently or in combination, metaphorical
targets as well as metaphorical sources, a full-blown theory of metaphor
cannot be based on its verbal manifestations alone, since this may result in a
biased view of what constitutes metaphor.
In this chapter I will sketch how adopting the view that metaphors can
assume non-verbal and multimodal appearances can and should guide the
research of a new generation of metaphor scholars. I will do so partly by
bringing to bear multimodal perspectives on issues already familiar from
research by language-oriented metaphor scholars, and partly by discussing
issues that have either been neglected by such researchers or are simply not
pertinent to purely verbal metaphors. The chapter should be seen as a map
of mostly uncharted territory, with only a few details inked in, much of it
reporting theory-driven analyses and informed speculation awaiting empiri-
cal testing. Multimodal metaphor researchers have a vast amount of work to
look forward to.
2. Multimodality versus monomodality
In order to distinguish multimodal metaphor from monomodal metaphor, it
should first be further clarified what is meant by mode. This is no easy
task, because what is labeled a mode here is a complex of various factors.
As a first approximation, let us say that a mode is a sign system interpret-
able because of a specific perception process. Acceptance of this approach
would link modes one-on-one to the five senses, so that we would arrive at
the following list: (1) the pictorial or visual mode; (2) the aural or sonic
mode; (3) the olfactory mode; (4) the gustatory mode; and (5) the tactile
mode. However, this is too crude a categorization. For instance, the sonic
mode under this description lumps together spoken language, music, and
non-verbal sound. Similarly, both written language and gestures would have
to be part and parcel of the visual, since one cannot hear, smell, taste, or
touch either conventionally written language or gestures (although a blind
person feels Braille language and, by touch, can perceive certain gestures
for instance those of a statue). If justice is to be done to these distinctions
(between images and gestures, between spoken and written language, be-
tween spoken language, sounds, and music), other factors need to be taken
into account, such as the manner of production (e.g., printed versus Braille
letters in relief on paper; signs made with parts of the body versus signs
whose use is governed by the grammar and vocabulary rules of a natural
language). There are other problematic issues. For instance, what is music
and what mere sound may differ from one culture, or period, to another.
Similarly, it is impossible to assess objectively where music shades off into
sound effect. And is typeface to be considered an element of writing, of
visuals, or of both? In short, it is at this stage impossible to give either a
satisfactory definition of mode, or compile an exhaustive list of modes.
However, this is no obstacle for postulating that there are different modes
and that these include, at least, the following: (1) pictorial signs; (2) written
signs; (3) spoken signs; (4) gestures; (5) sounds; (6) music (7) smells; (8)
tastes; (9) touch.
We can now provisionally define monomodal metaphors as metaphors
whose target and source are exclusively or predominantly rendered in one
mode. The prototypical monomodal metaphor is the verbal specimen that
until recently was identical with metaphor tout court, and which has
yielded thousands of studies (de Knop et al. 2005; Shibles 1971; Van Nop-
pen et al. 1985; Van Noppen and Hols 1990). A type of monomodal meta-
phor that has more recently become the subject of sustained research is pic-
torial or visual metaphor.
2

An early discussion of metaphor in pictures is Kennedy (1982). This per-
ception psychologist takes metaphor in the all-encompassing sense of what
literary scholars call a trope or a figure of speech (and which Tversky
2001 calls figures of depiction) and identifies some 25 types, including for
instance metonymy, hendiadys, and litotes. Kennedys attempts to
describe an extensive catalogue of figures of depiction using the
tenor/vehicle distinction that Richards (1965) specifically coined for meta-
phor are sometimes strained. There are other problems. It is a matter for
debate whether the names he selects for his examples are always necessarily
the best ones, and each trope is illustrated with one or two examples only,
making generalizations difficult (the same problem also adheres to Durand
1987). This having been said, Kennedy makes a number of points that are
illuminating for a theory of pictorial/visual metaphor-in-the-narrow-sense. In
the first place he argues that for a phenomenon to be labeled a visual meta-
phor it should be understandable as an intended violation of codes of repre-
sentation, rather than as being due to carelessness or error. Secondly, Ken-
nedy emphasizes that target and source are, in principle, irreversible, which
ensures that what he labels metaphor remains commensurate with a gener-
ally accepted criterion in theories of verbal metaphor. Thirdly, Kennedy
introduces the helpful notion of runes: the kind of non-iconic signs used
profusely in comics and cartoons to indicate speed, pain, surprise, happiness,
anger and many other phenomena by means of straight or squiggly lines,
stars, bubbles etc. surrounding characters or moving objects (see also Smith
1996). In later work, Kennedy elaborates on his theoretical work in various
experiments. Kennedy (1993) reports, among other things, how congenitally
blind children metaphorically draw a spinning wheel.
Whittock (1990) describes cinematographic metaphor. While his numer-
ous examples are subsumed under ten subtypes, and thus are less wide-
ranging than those by Kennedy, they still go beyond metaphor-in-the-
narrow-sense, including for instance metonymy and synecdoche. Whittock is
criticized by Carroll (1994) for failing to take into account what the latter
considers the most typical variety of visual or cinematographic metaphor,
the visual hybrid (see also Carroll 1996). Carroll, unlike Kennedy and Whit-
tock, moreover argues that visual metaphors differ from verbal ones in often
allowing for reversal of target and source. In Forceville (2002a), expanding
on earlier work (Forceville 1988, 1994, 1995, 1996, 2000), I in turn ques-
tion Carrolls choice for visual hybrids as core filmic metaphor (Carroll
1996: 218) as well as his proposal for the typical reversibility of target and
source in visual metaphors. My argument rests on the claim that Carroll is
biased by his exclusive reliance on examples rooted in Surrealist art. My
own model, largely developed with respect to advertising representations
(but see Forceville 1988) and based on Blacks (1979) interaction theory of
metaphor (see also Gineste et al. 2000; Indurkhya 1991, 1992), centers on
the answerability of the following three questions: Which are the two terms
of the pictorial metaphor?; which is the target and which is the source?; and
which is/are the features that is/are mapped from source to target? The last
question pertains to the metaphors interpretation: in principle all elements
metonymically associated (by a whole community or by a single individual)
with the source domain qualify as potential candidates for a mapping. The
crucial issue what is actually mapped by a specific addressee in a specific
situation is governed by the relevance principle as developed by Sperber and
Wilson (1995). (For more discussion of the role of metonymy in metaphor,
see various contributions in Dirven and Prings 2002; for the pertinence of
Relevance Theory to the interpretation of pictorial metaphors, see Forceville
1996, chapters 5 and 6.) In this model, Carrolls examples would rank as
one of three (Forceville 2002b) or four (Forceville 2005a, 2007) subtypes of
monomodal metaphor. It is to be noted that the distinction between two of
these types metaphor and simile is also made by Kaplan (1990, 1992;
see also Rozik 1994; 1998).
In contrast to monomodal metaphors, multimodal metaphors are meta-
phors whose target and source are each represented exclusively or predomi-
nantly in different modes. The qualification exclusively or predominantly
is necessary because non-verbal metaphors often have targets and/or sources
that are cued in more than one mode simultaneously. To give a fictive exam-
ple: imagine somebody wants to cue, for whatever reason, the metaphor CAT
IS ELEPHANT pictorially in an animation film. She could do this for instance
by depicting the cat with a trunk-like snout and large flapping ears; by show-
ing the cat with a canopy on its back in which a typical Indian elephant rider
is seated; by juxtaposing cat and elephant in the same salient pose; or by
letting the cat behave (for instance: move) in an elephant-like manner. These
variants would constitute monomodal metaphors of the pictorial kind, featur-
ing hybrid, contextual, simile, and integrated subtypes respectively (see For-
ceville 1996, 2002b, 2005a) and of course these subtypes could be com-
bined. Now imagine the producer wishes to cue the same metaphor
multimodally. She could for instance have the cat make a trumpeting sound
or have another cat shout elephant! to the first one (note that this is not a
case of synaesthesia, since there is no conflation of the two domains). In
these cases the source domain ELEPHANT would be triggered in two modes
(sound and language, respectively) that are different from the target
(visuals). By this token, the metaphor would be truly multimodal. But, as in
the case of the visual mode alone, the producer would of course not have to
choose between any of these modes: she could depict the cat with a trunk-
like snout and large ears and have it trumpet, and have another cat shout
elephant! In this case, the source is cued in three modes simultaneously,
only one of these (namely: the visual) exemplifying the same mode as the
target. In such a case I also propose to label the metaphor multimodal. Of
course the metonymy cueing the source domain in itself is often chosen for
its specific connotations. Both tusks and a trunk trigger ELEPHANT, but the
former connotes, among other things, aggressiveness, whiteness, costliness
and the latter among other things flexibility, sensitivity, and instrument-to-
spray-water-or-sand-with. For examples, as well as more discussion, of
multimodal metaphors involving (moving) images, see Forceville (1999a,
2003, 2004b, 2005b, 2007, 2008). There is also a growing literature on
multimodal metaphors involving language and gestures (Cienki 1998;
McNeill 1992; Mller 2004), in which the gesture-modality cues the source
rather than the target domain (McNeill 2005: 45).
3. Structural versus creative metaphor
Lakoff and Turner (1989) have argued that not only metaphors occurring in
everyday verbal communication can be traced back to conceptual metaphors,
but also those in artistic texts, specifically poetry. Particularly when poems
thematize abstract concepts such as life and death, they cannot but draw on
the same conceptual metaphors that permeate non-artistic language. Thus
Lakoff and Turner cite, and richly illustrate, many passages featuring LIFE
and TIME as metaphorical targets (LIFE IS A JOURNEY, LIFE IS A PLAY, LIFE IS
BONDAGE, LIFE IS A BURDEN; TIME IS A THIEF, TIME IS A MOVER, TIME IS A
DEVOURER) concluding that although human imagination is strong, empower-
ing us to make and understand even bizarre connections, there are relatively few
basic metaphors for life and death that abide as part of our culture (1989: 26).
They acknowledge that the art and craft of good poets resides in finding fresh,
original verbal formulations for these conceptual metaphors, and that these for-
mulations resonate both with the rest of the poem and with the extra-textual
knowledge of the reader. I take it that Lakoff and Turner allow that this, in turn,
may result in temporary readjustments of the basic level conceptual metaphors,
and thus that they would agree that the linguistic level of the metaphor is not a
mere illustration or exemplification of the pre-existing basic conceptual level.
But not all verbal metaphors in poetry, as Lakoff and Turner acknowledge,
reflect basic conceptual metaphors. While conventional metaphors can be
expressed either in common or in idiosyncratic language, modes of thought
that are not themselves conventional cannot be expressed in conventional lan-
guage (1989: 26) and hence require idiosyncratic language. (If I understand her
correctly, Renate Bartsch would probably object to the label conceptual meta-
phor for the metaphorical schema that underlies such novel metaphors. She
stipulates that a phenomenon deserves the name of concept only if it has a
stable interpretation in a community, and hence must by definition have been
linguistically explicated (Bartsch 2002: 50). A stable interpretation requires
that the community agrees on the phenomenons characteristic features and these,
in turn, reveal themselves in the true predications that can be used for it. Novel
metaphors focus attention on non-characteristic features and therefore, in
Bartsch reasoning, cannot (yet) have the status of being conceptual.) Often,
the border between conventional metaphors and idiosyncratic ones is difficult to
draw, not least because conventional metaphors may have idiosyncratic exten-
sions. Lakoff and Turner thus admit that not all poetic metaphors are conven-
tional ones, but the bulk of their examples and discussions pertain to the latter.
This is unsurprising, since their aim is to show that poetic metaphors normally
tap into conventional ways of thinking (great poets can speak to us because they
use the modes of thought we all possess, Lakoff and Turner 1989: xi).
Nonetheless Lakoff and Turners account raises some questions. In the first
place, it is not clear how representative their chosen examples are of poetic
metaphors in general. While they convincingly show that structural meta-
phors pervade poetry, the relative distribution of metaphors may depend on
time and place: older poetry, or non-Western poetry, may feature more, or
less, instances of a given metaphor. Gevaert (2001), basing herself on cor-
pus-based data, questions for instance the embodied, timeless status of
ANGER IS HEAT claimed for conceptualizations of anger in Lakoff (1987).
She demonstrates, among other things, that in the Old English period SWELL-
ING was a much more important source domain in ANGER metaphors than
HEAT, and speculates that the latters growing popularity in more recent
periods may well be due to the humoral theory that dominated mediaeval
times; that is, to cultural no less than embodied knowledge (see also Gevaert
2005). Moreover, a systematic, corpus-based analysis might reveal that
many poetic metaphors are not so easily amenable to conventional ones.
Numerous poetic metaphors may simply not have abstract concepts such as
LIFE, DEATH, TIME, PURPOSE as their target domain. As pointed out by
Grady, they may more often be resemblance metaphors than, possibly
(near) universal, correlation metaphors (Grady 1999). This would not
invalidate Lakoff and Turners impressive findings, but their one-sided em-
phasis on correlation and generic-level metaphors
3
in poetry may inadver-
tently lead to an uncritical acceptance of the view that most poetic metaphors
are of this kind.
One important difference between conventional and idiosyncratic meta-
phors is that the interpretation of the latter is, by definition, far less governed
by entrenched, pre-existing correspondences between the schematic struc-
tures in target and source. It is only by downplaying this difference that La-
koff and Turner can say that the preservation of generic-level structure is, we
believe, at the heart of metaphorical imagination, whether poetic or ordinary
(1989: 83; for critical accounts of this view see also Stockwell 1999; Crisp
2003).
Secondly, we should not forget that a metaphor can also conceptualize
the concrete in terms of the concrete. Lakoff and Turner, to be fair, are
aware of this. They discuss at some length the Elizabethan notion of the
Great Chain of Being (see e.g., Tillyard 1976 [1943]), which endorsed the
idea of natural hierarchies within various types of creatures angels, hu-
mans, birds, mammals, etc. and state that the GREAT CHAIN METAPHOR
can apply to a target domain at the same level on the Great Chain as the source
domain (Lakoff and Turner 1989: 179, emphasis in original). Put differently,
metaphors may have targets as well as sources that are directly accessible to
the senses. But given that CMT puts great emphasis on metaphors role in
conceptualizing the abstract in terms of the concrete, this possibility receives
rather scant attention, while CONCRETE IS CONCRETE metaphors are particu-
larly relevant once we leave the realm of the purely verbal. In the case of
monomodal metaphors of the pictorial variety, both target and source are
depicted. In advertising, metaphorical targets usually coincide with promoted
products and, unsurprisingly, are depicted and hence are necessarily con-
crete: a beer brand is depicted as a wine; an elegant watch as a butterfly, a
close-fitting bathing suit as a dolphins tight and supple skin (examples from
Forceville 1996). The same holds for metaphors in feature films (Forceville
2005b; Whittock 1990). In short, to what extent monomodal metaphors of
the non-verbal variety and multimodal metaphors are amenable to the corre-
lation metaphors that are the center of attention in CMT is an empirical
question. Some of them no doubt do; for instance, the personification of
commodities is a very familiar marketing strategy, and ties in with CMT
views (Lakoff and Turner 1989: 72). But many pictorial and multimodal
metaphors are of the OBJECT A IS OBJECT B type. Traditional CMT has not
much to say about these. Even Lakoff and Turners (1989) invocation of the
Great Chain metaphor is only of limited use here, since it depends on typo-
logical hierarchies that may be subverted, or simply irrelevant, in creative
metaphors, many of which function in contexts creating highly specific, ad
hoc metaphorical resemblances (see Black 1979).
There is a third aspect in which CMT has a one-sided emphasis. As dis-
cussed above, the typical source domains concreteness has in CMT been
traditionally connected to the notion of embodiment. The embodied nature
of source domains emphasizes their physical nature: it is human physical
interaction with the world that familiarizes humans with it to such an extent
that the resulting knowledge structures can in turn be mapped onto abstract
concepts. Knowledge about source domains is not simply a matter of em-
bodiment, however, but also of cultural connotations, as Lakoff and Turner
(1989: 66) acknowledge. More recent studies have demonstrated in a variety
of ways how the structure of source domains and the salient (and hence:
easily mappable) elements in it is influenced by culture (Gibbs and Steen
1999; Kvecses 2005; Shore 1996; Yu 1998). Indeed, the cultural connota-
tions that are metonymically related to a source domain are often more im-
portant for potential mappings to a target than its embodied aspects. In a
Dutch commercial promoting a Gazelle bicycle in terms of a dressage horse
the embodied mapping of riding a horse to riding a bicycle is less impor-
tant for the interpretation of the metaphor than the mapping of the cultural
connotations from the dressage horses owner, champion Anky van Gruns-
ven, to the prospective buyer and user of the bike. Similarly, while advertis-
ing a high-tech Senseo coffee machine in terms of a motorbike certainly has
embodied aspects, the subcultural connotations associated with motorbike-
riding evoked by Steppenwolfs Born to be wild, audible on the commer-
cials soundtrack, are at least as important in the mapping (Forceville
2004b/this volume, 2008; Forceville et al. in preparation). The relevant
similarity that is created between target and source pertains to these con-
notations more than to anything else. The examples bear out Bartsch obser-
vation that in metaphor the role of similarity is not restricted to the identity
of internal properties of objects and situations, rather similarity also is due to
identity of external contiguity relationships between objects, between situa-
tions, and it is due to relationships of objects and situations with emotional
attitudes, desires, and behavioural dispositions of people (Bartsch 2002:
52). Indeed, it might be ventured that a single, embodied correspondence
between target and source is enough to trigger a wide range of further cul-
tural correspondences between target and source, and hence of inferences
about the target (Forceville et al. 2006: 107). The old adage that a picture
tells more than a thousand words should not blind us to the fact that pictures
and other multimodal representations seldom communicate automatically or
self-evidently. As in verbal metaphors, it is connotations rather than denota-
tions of source domains that get mapped in metaphors, and these may sub-
stantially differ from one (sub)cultural group to another (see e.g., Maalej
2001). Even when non-verbal metaphors verge toward the conventional, as
in comics representations of ANGER (Eerden this volume; Forceville 2005c;
see Simons 1995 for multimodal instantiations of structural metaphor in pre-
election TV spots promoting political parties; and Forceville 2006a, Force-
ville and Jeulink 2007, for discussions of the source-path-goal schema and
the LIFE IS A JOURNEY metaphor), it may well be the case that, as in their
verbal counterparts (see Kvecses 1986, 2000, 2005), there is cultural varia-
tion (see Shinohara and Matsunaka 2003, this volume).
The three issues briefly mentioned above (metaphors frequently have
concrete rather than abstract targets; many metaphors with idiosyncratic
surface manifestations do not reflect correlation metaphors; what gets
mapped from source to target domains are often cultural, not embodied,
features) are thrown in relief when the study of purely verbal metaphors
gives way to that of non-verbal and multimodal metaphors. That is, general-
izing observations on metaphor based on the systematic investigation of
verbal specimens need to be considered afresh by testing these observations
in metaphors occurring in other modes.
4. The verbalization of non-verbal metaphor and the nature of
similarity
We have seen that within the CMT paradigm, most surface metaphors
should be amenable to a pre-existing conceptual A IS B format. Inevitably, in
order to discuss the metaphor, this A and B must be named, i.e., rendered in
language. It is by no means a foregone conclusion, of course, that the lan-
guage of thought is actually a verbal language. The convention to verbalize
the image-schematic structures underlying surface metaphors by using
SMALL CAPITALS useful inasmuch as this facilitates analyzing them may
disguise a number of consequences that seem to me more problematic in the
discussion of discourses that are not (exclusively) verbal ones than of purely
verbal ones. One of these consequences is that it is the analysts responsibil-
ity to find an adequate or acceptable verbal rendering of the metaphors un-
derlying image-schematic level, but such a verbalization, even though used
as a convenient shorthand, is never neutral. The design of the Senseo coffee
machine suggests the posture of somebody bending over and modestly offer-
ing something (i.e., a cup) on a plate. But should this awareness result in the
verbalization COFFEE MACHINE IS SERVANT, or is COFFEE MACHINE IS BUT-
LER more appropriate? Although servant and butler share many fea-
tures, they also differ: butler is more specific, and may in some people
(but not in others) evoke connotations of Britishness and standards of service
that servant does not. As a result, the mappings suggested by the two ver-
balizations may differ. Bartsch might conclude that this very inability to
agree on a single verbalization of the source domain that is shared within a
community shows that the source has no conceptual status, and reflects a
quasi-concept at best (Bartsch 2002: 50). However, to the extent that there
is a community that recognizes the source as cueing a serving person, the
source admits predicates understood as true in the community (such as is
there to serve the user, obeys your requests, and is almost always avail-
able). In a visually literate society, a vast number of endlessly repeated and
recycled images (such as famous paintings, photographs, film shots, flags,
logos, animation characters) evoke specific phenomena and events in a cli-
chd, shorthand manner widely shared within a community, and hence ar-
guably aspire to conceptual status. But this speculation leads us far beyond
the concerns of the present chapter and deserves in-depth reflection else-
where.
Another consequence is that verbalization of a non-verbal metaphor is
necessarily a conscious action, and a fairly unusual one at that. It is only the
scholar writing an academic paper who, to be able to discuss a multimodal
metaphor, needs to resort to GAZELLE BICYCLE IS VAN GRUNSVEN DRES-
SAGE HORSE; the metaphor is not verbalized in this form in the commercial,
and it is an open question whether the construal of a non-verbal metaphor
requires its verbalization by the audience. The question can be reformulated
as follows: does comprehension of a non-verbal or multimodal metaphor
imply that recipients mentally verbalize the metaphor? It is an important
question, but difficult to test empirically.
Since non-verbal modes of communicating by definition do not have the
is or is like in order to signal a metaphorical identity relation between
two entities belonging, in the given context, to different categories, one issue
that deserves attention is by what stylistic means the similarity is triggered.
Of course this holds for verbal metaphors that do not have the paradigmatic
A IS (LIKE) B format as well (cf. Brooke-Rose 1958; Goatly 1997). But
whatever means are chosen in this latter case, the cues are themselves of a
verbal nature. In non-verbal and multimodal metaphors, the signals that cue
metaphorical similarity between two phenomena are different, and bound to
differ depending on the mode(s) in which the metaphorical terms are repre-
sented. Here are some possibilities that are deployed in isolation or in com-
bination:
Perceptual resemblance. This can only function as a trigger in the case
of monomodal metaphors: only a visual representation can perceptually re-
semble another visual representation; only a sound can perceptually resem-
ble another sound in volume, timbre, or pitch. In the case of visual resem-
blance, there is a larger range of choices: two things can resemble one
another because they have the same size, color, position, posture, texture,
materiality, etc. Note that the resemblance need not reside in the things
themselves, but may surface in their manner of representation: they may for
instance be photographed from the same unusual angle, or filmed with the
same unusual camera movement.
Filling a schematic slot unexpectedly. Placing a thing in a certain con-
text may strongly, even inescapably, evoke a different kind of thing, namely
the thing for which the given context is the natural or conventional place. Put
differently, we may encounter deviations from typical gestalts or schemas.
For example, when in a musical environment a violin case contains a mon-
key wrench, this may suggest the metaphor MONKEY WRENCH IS VIOLIN.
Simultaneous cueing. If two things are signaled in different modes, meta-
phorical identification is achieved by saliently representing target and source
at the same time. For instance a kiss could be accompanied by the sound of a
car crash, of a vacuum cleaner, or of the clunking of chains, to cue meta-
phorical mappings of, say, disaster, dreary domestic routine, and imprison-
ment, respectively. Alternatively, in a variant on the previous mechanism,
two disparate things can be linked because of an unexpected filling of a slot,
as when a photograph of a kiss has the caption imprisonment.
5. The influence of genre
Human beings in most cases appear to construe a text automatically, very
quickly, and probably largely subconsciously as belonging to one genre
rather than to another. Anecdotal support for this claim is the experience of
channel-surfing: seasoned TV watchers guess in a split second what kind of
program they surf into (and decide at once whether they want to spend time
with it). Another illustration for the claim is the funny, self-reflexive trailer
for the film Comedian (http://nl.youtube.com/watch?v=yXbFuNQwTbs, last
consulted 10 July 2008) by Jerry Seinfeld. A man, Jack, sits in a booth in a
sound studio to record the voice-over for the trailer, but as soon as he has
intoned a few words he is impatiently interrupted by the director:
Jack: In a world where laughter was king
Director: No in a world, Jack
Jack: What do you mean, No in a world?
Director: Its not that kind of movie.
Jack: Oh OK In a land that
Director: No in a land either
Jack: In the time
Director: No, I dont think so.

In a rapid exchange Jack makes one abortive attempt after another One
man ; When your life is no longer your own ; When everything you
know is wrong ; In an outpost ; On the edge of space only to
be cut short by the director straightaway. For present purposes the point to
be made is that, to an audience with expertise in the area of film, the few
words uttered by Jack suffice to cue an entire genre. Finally, Hayward
(1994) offers empirically attested support for the claim that people are able
to decide very fast to what genre a text belongs. Hayward found that almost
80% of experimental subjects, given randomly selected passages of history
or fiction writing, recognized the genre of the work even on the basis of very
short passages (5 to 15 words).
The genre within which a text (in whatever medium) is presented, or the
genre to which it is attributed, determines and constrains its possible inter-
pretations to an extent that is difficult to overestimate (see Altman 1999;
Charteris-Black 2004; Forceville 1999b, 2005d; Steen 1994; Zwaan 1993
for discussion). For this reason, it is important to study how genre has an
impact on the production and interpretation of metaphors (monomodal and
multimodal alike). In advertising, for instance, the targets of metaphors often
coincide with the product promoted (Forceville 1996). This is to be ex-
pected: an advertisement or commercial predicates something about a prod-
uct, brand, or service, and this neatly and naturally fits the metaphors TAR-
GET IS SOURCE format. Moreover, the features mapped from source to target
are positive ones (unless the metaphor is used to disqualify a competitors
brand, in which case the mapped features are typically negative). But in
feature films, there is no phenomenon that in a similar, natural, way quali-
fies as a metaphorical target. Metaphors in artistic narratives pertain to phe-
nomena that, for whatever reason, are deemed salient by their producers.
These phenomena can be protagonists, but also objects, or even events. The
mapped features will often be less clear-cut, and may have a richer aligned
structure (Gentner and Loewenstein 2002), than those in advertising.
Metaphors in artistic representations may also differ in other respects
from those in commercial messages. For instance, while in commercials
there will seldom be a question what is target and what is source in a meta-
phor, an artistic narrative may give rise to two different construals of a
metaphor: both A IS B and B IS A are appropriate. (While Carroll 1994, 1996
calls such metaphors reversible, I prefer to say that, in the given context,
both the metaphors A IS B and B IS A are pertinent, in order to retain the no-
tion that target and source in a metaphor are, in principle, irreversible.)
Commensurate with this, metaphors in artistic contexts presumably allow
for greater freedom of interpretation than do metaphors in commercials (cf.
also Shen 1995).
Another parameter that deserves further research is whether any of the
subtypes of pictorial metaphor or of the manifold varieties of multimodal
metaphor can be systematically related to certain text genres. For instance, it
seems that commercial advertising seldom makes use of the hybrid variety of
pictorial metaphors (in Forceville 1996 these were called MP2s). Again, this
makes sense: if metaphorical targets typically coincide with products, adver-
tisers would want their product portrayed in their entirety, and not in a man-
ner that might evoke connotations of incompleteness or mutilation. Hybridiz-
ing it with a metaphorical source domain would not fit this goal. By contrast,
in animation films, or science fiction films, no such problem arises.
Finally, it would be worthwhile to investigate whether in the case of mul-
timodal metaphors there are any systematic correlations between textual
genres and the modes in which a target and source are represented. In adver-
tisements, the visual mode is typically used for representing the target and
this may well be true for different genres as well. But perhaps alternative
patterns in the choice of mode for the source domain are detectable in differ-
ent types of texts, while this may also change over time within a genre.
6. Concluding remarks
Researching multimodal metaphor, in short, is a natural next step in the
further development of metaphor studies a development in which theoreti-
cal reflection will have to go hand in hand with empirical testing. If creative
and conventional metaphor are key factors in human thinking, and if human
thinking is reflected in more than verbal manifestations alone, investigating
multimodal metaphor is highly worthy of extensive scholarly effort. Given
its long disciplinary tradition, the robust insights of metaphor scholarship
can in turn fruitfully feed into the budding field of multimodality in general
(Kress and Van Leeuwen 1996/2006, 2001; Ventola et al. 2004). Genres to
be investigated include advertising (Forceville 1996, 2003, 2007; McQuarrie
and Mick 2003; Phillips 2003; Wiggin and Miller 2003; McQuarrie and
Phillips 2008), political cartoons (El Refaie 2003), film (Forceville 1999a,
2005b; Rohdin 2003; Whittock 1990), oral speech accompanied by gestures
(Cienki 1998; McNeill 1992, 2005; Mller 2004), and design (Cupchik
2003; Van Rompay 2005). And inasmuch as multimodal representations (in
the form of advertising, videoclips, games, TV-formats, mainstream films,
animation) travel faster and more easily across the world than verbal ones,
examining their metaphorical manifestations will help focus on what remains
stable and what changes in cross-cultural communication. Furthermore, such
work may provide the starting point for how other tropes besides metaphor
can assume multimodal appearances (e.g., metonymy, irony, hyperbole,
oxymoron, see Gibbs 1993; Kennedy 1982; Teng and Sun 2002). Here the
analysis of multimodal metaphor ties in with the study of rhetoric. In a
global society in which media are increasingly used, or abused, as mouth-
pieces for the views of powerful factions (politicians, industry tycoons, reli-
gious leaders), the critical analysis of the tools of persuasive discourse in the
broadest sense constitutes an excellent interface between research in acade-
mia and its possible usefulness in the world beyond its walls.
Acknowledgments
I am indebted to Francisco Ruiz de Mendoza and Ren Dirven for comments and
suggestions on earlier drafts of this paper.
Notes
1. This chapter is virtually identical with the text published as Forceville
(2006b). The changes are reference updates and some minor corrections and
rephrasings.
2. The topic of metaphor and music also has inspired studies over the past few
years, but for lack of expertise in this area I will not dwell on these. For ref-
erences, see Johnson and Larson 2003; see also Cook 1998; Zbikowski 2002;
Thorau 2003; and Spitzer 2004 the last one rather difficult for laymen.
3. Lakoff and Turner describe generic-level metaphors as metaphors which are
minimally specific in two senses: they do not have fixed source and target
domains, and they do not have fixed lists of entities specified in the map-
ping (1989: 81). They introduce the term using the example of the EVENTS
ARE ACTIONS metaphor which they contrast with LIFE IS A JOURNEY, one of its
specific-level instantiations.
References
Altman, Rick
1999 Film/Genre. London: British Film Institute.
Bartsch, Renate
2002 Generating polysemy: Metaphor and metonymy. In Metaphor and
Metonymy in Comparison and Contrast, Ren Dirven, and Ralf
Prings (eds.), 4974. Berlin/New York: Mouton de Gruyter.
Black, Max
Bolter, Jay David, and Richard Grusin
1999 Remediation: Understanding New Media. Cambridge MA: MIT
Press.
Brooke-Rose, Christine
1958 A Grammar of Metaphor. London: Secker and Warburg.
Carroll, Noel
1994 Visual metaphor. In Aspects of Metaphor, Jaakko Hintikka (ed.),
189218. Dordrecht: Kluwer.
Carroll, Noel
1996 A note on film metaphor. In Theorizing the Moving Image, 212
223. Cambridge: Cambridge University Press.
Charteris-Black, Jonathan
2004 Corpus Approaches to Critical Metaphor Analysis. London: Pal-
grave-MacMillan.
Cienki, Alan
Jean-Pierre Koenig (ed.), 189204. Stanford, CA: Center for the
Study of Language and Information.
Cook, Nicholas
1998 Analyzing Musical Multimedia. Oxford: Clarendon.
Coulson, Seana, and Esther Pascual
2006 For the sake of argument: Mourning the unborn and reviving the
dead through conceptual blending. Annual Review of Cognitive Lin-
guistics 4: 153181.
Crisp, Peter
2003 Conceptual metaphor and its expression. In Cognitive Poetics in
Practice, Joanna Gavins and Gerard Steen (eds.), 99113. Lon-
don/New York: Routledge.
Cupchik, Gerry D.
2003 The interanimation of worlds: Creative metaphors in art and de-
sign. The Design Journal 6 (2): 1428.
York: Mouton de Gruyter.
Durand, Jacques
1987 Rhetorical figures in the advertising image. In Marketing and Semi-
otics: New Directions in the Study of Signs for Sale, Jean Umiker-
Sebeok (ed.), 295318. Berlin/New York: Mouton de Gruyter.
Eerden, Bart
this vol. Anger in Asterix: The metaphorical representation of anger in com-
ics and animated films.
El Refaie, Elisabeth
2003 Understanding visual metaphors: The example of newspaper car-
toons. Visual Communication 2 (1): 7595.
Forceville, Charles
1988 The case for pictorial metaphor: Ren Magritte and other Surreal-
ists. In Vestnik IMS 9, Ale Erjavec (ed.), 150160. Ljubljana: Inti-
tut za Marksistine tudije [Slovenia].
1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Ac-
tivity 9: 129.
1995 (A)symmetry in metaphor: The importance of extended context.
Poetics Today 16: 677708.
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
1999a The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters,
and Paul Schraders The Comfort of Strangers. Metaphor and
Symbol 14: 17998.
1999b Art or ad? The effect of genre-attribution on the interpretation of
images. Siegener Periodicum zur Internationalen Empirische Lit-
eraturwissenschaft [SPIEL] 18 (2): 279300.
2000 Compasses, beauty queens and other PCs: Pictorial metaphors in
computer advertisements. Hermes, Journal of Linguistics 24: 3155.
2002a The identification of target and source in pictorial metaphors.
Journal of Pragmatics 34: 114.
2002b Further thoughts on delimitating pictorial metaphor. Theoria et
Historia Scientiarum 6: 21327 [Torn, Poland].
2003 Bildliche und multimodale Metaphern in Werbespots [Trans. from
English by Dagmar Schmauks] Zeitschrift fr Semiotik 25: 3960.
2004a Review of Fauconnier and Turner (2002). Metaphor and Symbol 19:
8389.
2004b The role of non-verbal sound and music in multimodal metaphor. In
Words in their Places: A Festschrift for J. Lachlan Mackenzie,
Henk Aertsen, Mike Hannay, and Rod Lyall (eds.), 6578. Amster-
dam: Faculty of Arts, VU Amsterdam.
2005a When is something a pictorial metaphor? [Lecture 2 in eight-lecture
Course in Pictorial and Multimodal metaphor] http://www.
chass.utoronto.ca/epc/srb/cyber/cforcevilleout.html.
2005b Cognitive linguistics and multimodal metaphor. In Bildwissen-
schaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach
(ed.), 264284. Cologne: Von Halem.
2005c Visual representations of the Idealized Cognitive Model of anger in
2005d Addressing an audience: time, place, and genre in Peter Van
Straatens calendar cartoons. Humor: International Journal of Hu-
mor Research 18: 247278.
2006a The source-path-goal schema in the autobiographical journey docu-
mentary: McElwee, Van der Keuken, Cole. New Review of Film and
Television Studies 4: 241261.
2006b Non-verbal and multimodal metaphor in a cognitivist framework:
Agendas for research. In Cognitive Linguistics: Current Applica-
tions and Future Perspectives, Gitte Kristiansen, Michel Achard,
Ren Dirven and Francisco Ruiz de Mendoza Ibez (eds.), 379
2007 Multimodal metaphor in ten Dutch TV commercials. Public Journal
of Semiotics 1 (1): 1951. http://semiotics.ca/.
2008 Metaphors in pictures and multimodal representations. In The Cam-
bridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr.
Forceville, Charles, Paul Hekkert, and Ed Tan
2006 The adaptive value of metaphors. In Heuristiken der Literatur-
wissenschaft. Einladung zu disziplinexternen Perspektiven auf
Literatur, Uta Klein, Katja Mellmann, and Steffanie Metzger (eds.),
85109. Paderborn: Mentis.
Forceville, Charles, and Marloes Jeulink
2007 The Source-Path-Goal schema in animation film. Paper presented at
10
th
International Cognitivist Linguistics conference (ICLC, July) at
Jagiellonian University, Krakow (Poland).
Forceville, Charles, Michelle Hilscher, and Gerald Cupchik
in prep. The interpretation of multimodal metaphors in Dutch commercials
by Dutch and Canadian subjects [provisional title].
Gentner, Dedre, and Jeffrey Loewenstein
2002 Relational language and relational thought. In Language, Literacy,
and Cognitive Development, Eric Amsel and James P. Byrnes
(eds.), 87120. Mahwah, NJ: Erlbaum.
Gevaert, Caroline
2001 Anger in Old and Middle English: a hot topic? Belgian Essays on
Language and Literature, 89101.
2005 The anger is heat question: Detecting cultural influence on the con-
ceptualisation of anger through diachronic corpus analysis. In
Perspectives on Variation, Nicole Delbecque, Johan van der
Auwera, and Dirk Geeraerts (eds.), 195208. Berlin/New York:
Mouton de Gruyter.
Gibbs, Raymond W., Jr.
1993 Process and products in making sense of tropes. In Metaphor and
Thought, 2nd ed., Andrew Ortony (ed.), 252276. Cambridge:
Cambridge University Press.
1994 The Poetics of the Mind: Figurative Thought, Language, and Un-
derstanding. Cambridge: Cambridge University Press.
2006 Embodiment and Cognitive Science. Cambridge: Cambridge Uni-
versity Press.
Gibbs, Raymond W., Jr., and Gerard J. Steen (eds.)
1999 Metaphor in Cognitive Linguistics. Amsterdam/Philadelphia: Ben-
jamins.
Gibbs, Raymond W., Jr., and Herbert L. Colston
1995 The cognitive psychological reality of image schemas and their
transformations. Cognitive Linguistics 6: 347378.
Symbol 15: 117135.
Goatly, Andrew
1997 The Language of Metaphors. London/New York: Routledge.
Grady, Joseph
1999 A typology of motivation for conceptual metaphor: Correlation vs.
resemblance. In Metaphor in Cognitive Linguistics, Raymond W.
Gibbs, Jr., and Gerard J. Steen (eds.), 79100. Amsterdam/
Philadelphia: Benjamins.
Hayward, Malcolm
1994 Genre recognition of history and fiction. Poetics 22: 409421.
Indurkhya, Bipin
1991 Modes of metaphor. Metaphor and Symbolic Activity 6: 127.
1992 Metaphor and Cognition: An Interactionist Approach. Dordrecht:
Kluwer.
Johnson, Mark
1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination
and Reason. Chicago: University of Chicago Press.
1993 Moral Imagination: Implications of Cognitive Science for Ethics.
Chicago: University of Chicago Press.
2007 The Meaning of the Body: Aesthetics of Human Understanding.
Chicago and London: University of Chicago Press.
Johnson, Mark, and Steve Larson
2003 Something in the way she moves metaphors of musical motion.
Metaphor and Symbol 18: 6384.
Kaplan, Stuart J.
1990 Visual metaphors in the representation of communication technol-
ogy. Critical Studies in Mass Communication 7: 3747.
1992 A conceptual analysis of form and content in visual metaphors.
Communication 13: 197209.
Kennedy, John M.
1982 Metaphor in pictures. Perception 11: 589605.
1993 Drawing and the Blind: Pictures to Touch. New Haven/London:
Yale University Press.
Knop, Sabine de, Ren Dirven, Ning Yu, and Birgit Smieja (eds.)
2005 Bibliography of Metaphor and Metonymy. Amsterdam/ Philadel-
phia: John Benjamins. Available electronically only at: http://www.
benjamins.com/online/met.
Kvecses, Zoltn
Benjamins.
2002 Metaphor: A Practical Introduction. Oxford: Oxford University
Press.
1996 Reading Images: The Grammar of Visual Design (2nd ed. 2006).
London/New York: Routledge.
Lakoff, George
1987 Women, Fire and Dangerous Things: What Categories Reveal about
the Mind. Chicago: University of Chicago Press.
2nd ed., Andrew Ortony (ed.), 202251. Cambridge: Cambridge
University Press.
1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to
Western Thought. New York: Basic Books.
Maalej, Zouhair
2001 Processing pictorial metaphor in advertising: A cross-cultural per-
spective. Academic Research, 1: 1942 [Sfax, Tunisia].
McLuhan, Marshall
1964 The medium is the message. In Understanding Media: The
Extensions of Man, 2nd ed., 2335. New York: McGraw-Hill.
McNeill, David
McQuarrie, Edward F., and David Glen Mick
2003 The contribution of semiotic and rhetorical perspectives to the ex-
planation of visual persuasion in advertising. In Persuasive Im-
agery: A Consumer Response Perspective, Linda M. Scott and Ra-
jeev Batra (eds.), 191221. Mahwah, NJ: Lawrence Erlbaum.
McQuarrie, Edward F., and Barbara J. Phillips (eds.)
2008 Go Figure! New Directions in Advertising Rhetoric. Armonk, NY:
ME Sharpe.
Mller, Cornelia
2004 Metaphors, Dead and Alive, Sleeping and Waking: A Cognitive
Approach to Metaphors in Language Use. Habilitationsschrift, Freie
Universitt Berlin, Germany.
Ortony, Andrew (ed.)
1979 Metaphor and Thought. Cambridge: Cambridge University Press.
Phillips, Barbara J.
2003 Understanding visual metaphor. In Persuasive Imagery: A Con-
sumer Response Perspective, Linda M. Scott, and Rajeev Batra
(eds.), 297310. Mahwah, NJ: Lawrence Erlbaum.
Richards, Ivor Armstrong.
1965 [1936] The Philosophy of Rhetoric. New York: Oxford University Press.
Rohdin, Mats
2003 Summary in English. In Vildsvinet I Filmens Trdgrd: Metaforbe-
greppet inom Filmteorin [The wild boar in the garden of film: The
concept of metaphor in film theory], 318329. Ph.D. dissertation,
Stockholm: Edita Norstedts Tryckeri AB.
Rompay, Thomas van
2005 Expressions: Embodiment in the Experience of Design. Ph.D. diss.,
Technische Universiteit Delft, The Netherlands.
Rozik, Eli
1994 Pictorial metaphor. Kodikas/Code 17: 203218.
1998 Ellipsis and the surface structures of verbal and nonverbal meta-
phor. Semiotica 119: 77103.
Shen, Yeshayahu
1995 Cognitive constraints on directionality in the semantic structure of
poetic vs. non-poetic metaphors. Poetics 23: 255274.
Shibles, Warren A.
1971 Metaphor: An Annotated Bibliography and History. Whitewater,
WI: The Language Press.
Shinohara, Kazuko, and Yoshihiro Matsunaka
2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen:
Journal of Yokohama Linguistic Circle 4: 118.
this vol Pictorial metaphors of emotion in Japanese comics.
Shore, Bradd
1996 Culture in Mind: Cognition, Culture, and the Problem of Meaning.
New York: Oxford University Press.
Simons, Jan
1995 Film, Language, and Conceptual Structures: Thinking Film in the
Age of Cognitivism. Ph.D. dissertation, University of Amsterdam,
The Netherlands.
Smith, Ken
1996 Laughing at the way we see: The role of visual organizing principles
in cartoon humor. Humor: International Journal of Humor Re-
search 9: 1938.
1995 Relevance: Communication and Cognition (2nd ed.) Oxford: Black-
well.
Spitzer, Michael
2004 Metaphor and Musical Thought. Chicago/London: University of
Chicago Press.
Steen, Gerard J.
1994 Understanding Metaphor in Literature: An Empirical Approach.
London: Longman.
Stockwell, Peter J.
1999 The inflexibility of invariance. Language and Literature 8: 12542.
Sweetser, Eve E.
1990 From Etymology to Pragmatics: Metaphorical and Cultural Aspects
of Semantic Structure. Cambridge: Cambridge University Press.
Teng, Norman Y., and Sewen Sun
2002 Grouping, simile, and oxymoron in pictures: A design-based cogni-
tive approach. Metaphor and Symbol 17: 295316.
Terkourafi, Marina, and Stefanos Petrakis
forthc. A critical look at the desktop metaphor 25 years on. In Proceedings
of the 6th International Conference on Researching and Applying
Metaphor (RaAM6), University of Leeds, April 2006. Amsterdam/
Thorau, Christian
2003 Metapher und Variation: Referenztheoretische Grundlagen
musikalischer Metaphorizitt. Zeitschrift fr Semiotik 25: 109124.
Tillyard, E.M.W.
1976 [1943] The Elizabethan World Picture. Harmondsworth: Penguin.
Turner, Mark
1996 The Literary Mind. New York/Oxford: Oxford University Press.
Tversky, Barbara
2001 Spatial schemas in depictions. In Spatial Schemas and Abstract
Thought, Merideth Gattis (ed.), 79112. Cambridge MA: MIT
Press/Bradford book.
Van Noppen, Jean-Pierre, Sabine de Knop, and Ren Jongen (eds.)
1985 Metaphor: A Bibliography of Post-1970 Publications. Amsterdam/
Van Noppen, Jean-Pierre, and Edith Hols (eds.)
1990 Metaphor II: A Classified Bibliography of Publications from 1985
1990. Amsterdam/Philadelphia: Benjamins.
Ventola, Eija, Cassily Charles, and Martin Kaltenbacher
2004 Perspectives on Multimodality. Amsterdam/Philadelphia: Benja-
mins.
Whittock, Trevor
1990 Metaphor and Film. Cambridge: Cambridge University Press.
Wiggin, Amy A., and Christine M. Miller
2003 Uncle Sam wants you! Exploring verbal-visual juxtapositions in
television advertising. In Persuasive Imagery: A Consumer Re-
sponse Perspective, Linda M. Scott and Rajeev Batra (eds.), 267
295. Mahwah, NJ: Lawrence Erlbaum.
Yu, Ning
1998 The Contemporary Theory of Metaphor: A Perspective from Chi-
nese. Amsterdam/Philadelphia: Benjamins.
Zbikowski, Lawrence M.
2002 Conceptualizing Music: Cognitive Structure, Theory, and Analysis.
Oxford: Oxford University Press.
Zwaan, Rolf
1993 Aspects of Literary Comprehension: A Cognitive Approach. Ams-
terdam/Philadelphia: Benjamins.

II

Multimodal Metaphor in Advertising
Chapter 3
Brand images: Multimodal metaphor in corporate
branding messages
1

Veronika Koller
Abstract
This chapter looks at multimodal metaphors in companies communication of
their brand personalities. It is argued that corporate brands are cognitively
structured by the metaphor BRANDS ARE LIVING ORGANISMS, often specifically
BRANDS ARE PEOPLE. As an intangible abstraction, the brand is made comprehen-
sible by being conceptualized as an ideal person endowed with the traits that are
positively evaluated in corporate discourse: growth, flexibility, dynamism, and
connectivity. These metaphorical character traits of the brand personality are
expressed by the interplay of verbal and visual features in corporate discourse, in
particular the illustrations, logos, and layout found in genres such as mission
statements and history brochures. The source domain of the BRANDS ARE LIVING
ORGANISMS/PEOPLE metaphor is encoded visually, while the target domain tends to
be encoded verbally, with occasional additional encoding in the visual mode. It is
argued that this dual encoding reinforces the persuasive intent of corporate genres
by endowing them with an affective component. Corporate decision-makers use
these genres and their multimodal features to communicate the corporate brand to
external stakeholders just as much as to address internal ones, i.e., employees, to
foster identification with, and loyalty towards, the brand.

Keywords: branding, discourse, illustration, layout, logos
1. Introduction
This chapter looks at multimodal metaphor in companies communication of
their brand personalities. It argues that corporate brands are cognitively
structured by a metaphor BRANDS ARE LIVING ORGANISMS, often
specifically BRANDS ARE PEOPLE. As an intangible abstraction, the brand is
46 Veronika Koller
made comprehensible by being metaphorized as an ideal person endowed
with the traits that are positively evaluated in corporate discourse: growth,
flexibility, dynamism, and connectivity. These metaphorical character traits
of the brand personality are expressed by the interplay of verbal and visual
features in corporate discourse, in particular the illustrations, logos, and
layout found in genres such as mission statements and history brochures.
Corporate decision-makers use these genres and their multimodal features to
communicate the corporate brand to external stakeholders such as customers
just as much as to address internal ones, i.e., employees (de Chernatony
2002). In both cases, the intention is to foster identification with, and loyalty
towards, the brand. However, given that the power dynamics between em-
ployees and the company they depend on are very different from those be-
tween the corporation and the customers it seeks to win and retain, it may be
easier for consumers to disregard or cynically reject the brand image (Holt
2002).
2. A cognitive approach to multimodal metaphor
In Cognitive Metaphor Theory (see e.g., Barcelona 2000; Lakoff and John-
son 1999, 2003) metaphor, rather than being a merely decorative literary
device, is regarded as an essentially cognitive phenomenon structuring much
of human thought. In particular, metaphor is the means by which the human
mind conceives of one usually abstract entity in terms of another, usually a
more concrete one. The processes by which these metaphorically structured
mental models are brought about have been theorized as mappings from a
source to a target domain, or blends of two or more input spaces (Faucon-
nier and Turner 2002). In any case, it is only in a second step that these
metaphoric models are realized as metaphoric expressions at the surface
level of language, or of any other semiotic mode.
Metaphoric expressions have been most exhaustively studied in the ver-
bal mode, i.e., as surface-level linguistic expressions of metaphorically
structured mental models. This body of work was followed by research into
visual, or pictorial, metaphor (Carroll 1994; Forceville 1994, 1996). How-
ever, most of those studies still addressed mono-modal metaphor, in that
both source and target domains (or input spaces) were provided in the visual
mode and only reinforced, rather than co-constructed, by the verbal co-text.
Multimodal metaphor, on the other hand, is constituted by a mapping, or
blending, of domains from different modes, e.g., visual and verbal, or visual
and acoustic. With a view to the examples presented in this chapter, it seems
Brand images 47
useful to draw on Barthes (1977: 3841) concept of anchoring, according to
which the verbal elements of a multimodal text serve to cue and thereby
restrict possible interpretations of the visual elements. It will be argued that
such anchoring can also be achieved beyond the immediate co-text, by means
of intertextuality. For instance, the IBMs mission statements webpage is
illustrated with the picture of a DNA helix (figure 2), and while the verbal
co-text itself does not include any references to genes, talk about the DNA
of a company is shown to be a staple of corporate discourse, thus anchoring
an interpretation of the DNA image as the source domain of the metaphor
BRANDS ARE LIVING ORGANISMS.
Forceville (2006) has rightly pointed out that modes cannot simply be
equated with the five senses but that distinctions need to be made between,
for instance, different forms of acoustic mode (speech, music, sound; see van
Leeuwen 1999). Indeed, if one followed a perceptual taxonomy based on the
five senses, it would be impossible to analyze multimodal metaphor as being
constituted by visual and verbal elements; after all, verbal elements can be
perceived visually as written language, acoustically as spoken language, and
haptically as Braille language. Forceville therefore suggests the following
categories in the investigation of multimodal metaphor:

- pictorial signs: These include abstract design elements, as in logos
- written signs
- spoken signs
- gestures: These can be seen as part of the more general category of kines-
thetics, i.e. the movement of the body in three-dimensional space
2

- sounds, to be seen on a continuum with
- music
- smells, which are physiologically and hence perceptually related to
- tastes, and finally
- touch.

This chapter will focus on the interaction of pictorial and written signs, as
they combine into multimodal metaphoric expressions realizing the concep-
tual metaphor BRANDS ARE LIVING ORGANISMS and specifically, BRANDS
ARE PEOPLE.
The duality of conceptual metaphors as both operating on the cognitive
level and being realized in surface level metaphoric expressions leads to
inductive reasoning on the part of the researcher, in that the analysis of
metaphoric expressions leads to inferences about underlying metaphoric
models. In the case of verbal metaphoric expressions, verbalizations of in-
48 Veronika Koller
ferred conceptual metaphors are to some extent motivated by the verbal for-
mulation of the observed expression. For example, the phrase we channeled
investments in parallel with our corporate DNA, said by a company execu-
tive about their financial strategy (Schwartz 2006), allows for a metaphor
COMPANIES ARE LIVING ORGANISMS to be inferred. Obviously, any verbali-
zation reflects the researchers interpretation of the observed expressions,
and will in turn steer the analysis of similar phrases. This bias is aggravated
in the case of multimodal metaphor, especially when it is the source domain
that is expressed in non-verbal form: For example, a companys business
principles might feature pictures of a boat race (e.g. ABN Amro bank,
http://www.group.abnamro.com/about/business_principles.cfm) with no
possible literal interpretation.
3
Should the inferred conceptual metaphor be
worded as COMPANY X IS A RACING TEAM, or, in more general terms, BUSI-
NESS IS A RACE? In the case of a relay race, the metaphors significance
changes again, foregrounding aspects of teamwork rather than competition.
The ABN Amro example is further complicated by the verbal co-text realiz-
ing the JOURNEY metaphor, referring to the business principles as a compass
to guide us on our journey. In view of this complexity, it could be argued
that the traditional A IS B type metaphor as such is limiting, and that inferred
metaphors should be seen as part of a metaphor scenario (Musolff 2006)
with different levels of abstraction as well as cause-effect relations. In our
example, this could mean that both COMPANY X IS A RUNNER and BUSINESS
IS A RACE are viable. In addition, the metaphoric scenario triggered by the
multimodal metaphor could include semi-metaphoric components like com-
pany X delivers faster results and outruns the competition. I will return to
this question when discussing the examples from corporate discourse below.
In a given discourse,
4
only particular models will be relevant, i.e., one
form of structured knowledge of a particular notion or entity along with its
associations, evaluations and affective components will be preferred to oth-
ers. Subsequently, the metaphoric expressions deriving from metaphoric
models and their evaluations become typical of that discourse. Repeated
exposure to the same set of metaphoric expressions, extended and elaborated
as they may be in various modes, can be assumed to activate the metaphoric
mental model underlying them and thereby reinforce it. By the same token,
positive evaluation of particular notions would also be reinforced. Discursive
reinforcements of mental models, including evaluations, have been referred
to as moralized activities, i.e., activities represented by means of abstract
terms that distil from them a quality that triggers reference to positive or
negative values (van Leeuwen and Wodak 1999: 105). An example would
be the metaphoric reference to companies as corporate citizens, which has
Brand images 49
acquired the status of a technical term in corporate discourse and reinforces
the positively connoted idea of companies taking responsibility for the wider
community they are part of. What models rise to prominence in a discourse
is to some extent influenced by the (institutional) power of discourse partici-
pants. In the case of corporate branding messages, their senders hold consid-
erable power when addressing employees, while consumers as addressees
have the power to withhold their attention, interest and, ultimately, money.
This relative power on the part of consumers also grants them easier access
to counter-readings that, often cynically, deconstruct brand messages (see
for instance the practice of ad busting; Rumbo 2002). Following the socio-
cognitive tradition in critical discourse analysis (van Dijk 2002, 2005),
metaphor can, because of its dual nature, thus be seen as located at the inter-
face between cognition and discourse. Moreover, its selective use in dis-
course represents an ideologically vested strategy to shape models of social
reality, e.g., to present and ingrain particular traits but not others as positive
and desirable. By investigating how such a strategy is enacted through mul-
timodal metaphor, the chapter seeks to contribute to the critical study of
multimodality (Lassen, Strunck, and Vestergaard 2006), and multimodal
metaphor in particular.
3. A cognitive approach to corporate brands
It has been noted that inasmuch as pictures are more easily recognized
transnationally than languages, pictorial and multimodal metaphors allow
for greater cross-cultural access than verbal ones (Forceville 2008). This
makes visual elements, including the visual components of multimodal meta-
phor, particularly attractive as a communicative strategy for multinational
corporations. Multimodal metaphor in particular can be used as a tool to
meet the persuasive function of most corporate communication, because it
requires the texts recipient to construct a meaningful reading by processing
verbal and visual elements together. The necessary cognitive effort poten-
tially reinforces a particular conceptualization of the company in the readers
mind. Indeed, much research into visual metaphor to date has investigated
the phenomenon in contexts such as advertising, often with a view to proc-
esses of reception and interpretation (Phillips 2003; Phillips and McQuarrie
2004), or international business magazines (Koller 2005).
Supplementing those studies, this chapter looks at corporate branding as
the interface between marketing and strategic organizational management.
As the outward oriented (i.e., projected) organizational identity (Kapferer
50 Veronika Koller
2002: 185) of a company, a corporate brand is the image a company wishes
to convey to its stakeholders. In addition to Kapferers definition of the cor-
porate brand, it is here also understood as inwardly oriented, i.e., communi-
cated to internal stakeholders such as employees.
5
A corporate brand is a
metonymic condensation of what corporate decision-makers consider most
desirable in their organization (Balmer 2006: 39). As such, the corporate
brand has to be distinguished from product brands: Although some corpora-
tions project both (e.g., Apple as a corporate brand and iPod as one of its
product brands), companies may communicate corporate and product brands
separately. Examples are McDonalds as a corporate brand without particu-
lar product brands and, conversely, Ariel as one of the many product brands
marketed by Procter & Gamble, which is not itself a corporate brand. Pro-
jecting both seems to be the exception rather than the rule: Apart from his-
torical and strategic reasons for keeping product and corporate brands sepa-
rate, a duplication of images could ask too much of the cognitive capabilities
of a consumer who is already faced with innumerable brands. Ever since the
latter half of the 1990s, corporate branding has gradually become more im-
portant than product branding, up to the point where tangible products are
merely the material extension of a brand (Askegaard 2006: 100) and the
battle for positioning takes place in the realm of imagery (Soenen and
Moingeon 2002: 30). Reasons for this shift range from increased corporate
brand value as an asset in negotiating mergers and acquisitions, to the
economies of scale gained by more focused marketing and communication
efforts, and the need for employees to identify with their company (Schultz,
Hatch, and Ciccolella 2006: 142).
In socio-cognitive terms, a corporate brand constitutes the ideal self a
company wishes to communicate to others. Ultimately, this ideal self is
meant to converge with the epistemic social schema (Kristiansen 2008),
i.e., the beliefs a social group, in the present case, stakeholders, has about
the nature of another social group or entity, here, the company. The analysis
carried out in this study will show how corporate brands are further con-
structed by drawing on an ought self (Kunda 1999: 472) or deontic social
schema (Kristiansen 2008), i.e., the perceived view of stakeholders about
what the company should be like. Brands as intangible entities represent the
cognitive-affective concepts stakeholders maintain about a particular prod-
uct, service or, in the case of corporate brands, company. To make these
abstract models graspable, companies have since the 1980s sought to con-
ceptualize themselves and their brands as living organisms (Csaba and
Bengtsson 2006: 122; Morgan 1986), even endowing them with a quasi-
human personality (Christensen and Askegaard 2001; Wee 2004), up to
Brand images 51
the point where brands become metaphorical people interacting with stake-
holders.
6
In a final twist, the BRANDS ARE PEOPLE metaphor implies that a
brand is supposed to incorporate the ideal characteristics the consumer
wishes to be seen as having. In an ultimately narcissistic relationship with
the BRANDS ARE PEOPLE metaphor, the brands characteristics are theorized
to be transferred to the consumer, who thus engages in symbolic consump-
tion (Harquail 2006: 174175), using brands as symbolic resources for
the construction and maintenance of identity (Elliott and Wattanasuwan
1998: 132).
A central role in the projection of corporate brand image is played by ar-
tifacts, or multimodal texts, e.g., logos, architecture, office design, call
centre music etc. (Cappetta and Gioia 2006: 212). Such brand artifacts are
seen as the vehicles for the transfer of meaning from the brand to the con-
sumer, bringing with them emotional and self-expressive benefits for the
consumer (Schultz, Hatch, and Ciccolella 2006: 150). Forceville (2006: 388)
notes that the personification of commodities is a very familiar marketing
strategy, and it is certainly one that can be extended to (corporate) brand
artifacts. Metaphors as a reflection of allegedly shared organizational dis-
course (Cunliffe and Shotter 2006: 135) are used in a range of modes, most
notably visual and verbal, which, often in concert, construct the BRANDS
ARE PEOPLE metaphor and endow it with particular characteristics. Such
multimodal metaphors tend to take particular forms and to cluster in particu-
lar genres.
4. Data selection and analysis
Parts of this study are based on a corpus of corporate mission statements.
These were chosen as representing a companys ideal self, i.e., the basis for
projecting corporate brand images externally and internally. The mission
statements in question were taken from the websites of the 2003 Fortune
Global 500, a list of the largest companies worldwide that is ascribed great
importance by the business community. In particular, the texts comprise the
top 50 and the bottom 50 companies, thus building a corpus of 29,925
words. In terms of text type, mission statements come under the headings of
Our Vision, Message from the CEO (a feature that is particularly popu-
lar with Japanese companies), The essence of our company, Strategy or,
indeed, Our Mission. By using these headings interchangeably, companies
corroborate Collins and Porrass observation of mission and vision state-
ments as a muddled stew of values, goals, purposes, philosophies, beliefs,
52 Veronika Koller
aspirations, norms, strategies, practices, and descriptions (1996: 77, quoted
in Czerniawska 1997: 185). Despite this heterogeneity, however, we can
identify the mission statement as a genre since the various forms it takes
share the communicative purpose of conveying the raison dtre, ambitions
and central values of an organization to its internal and external stakeholders
(Fox and Fox 2004: 32, 43) and thus foster their loyalty to the institution in
question. Defining parameters used to achieve this communicative purpose
are hyperbole as realized in superlatives (the best customer experience),
absolute quantifiers (each, everywhere), inclusive we and high-affinity
deontic modality (it has been necessary to do so) (Fox and Fox 2004: 43,
171; Koller 2008; Swales and Rogers 1995).
The key word search function of the Wmatrix corpus analysis interface
(see Rayson 2008) was used to ascertain lexical items that were, in statisti-
cal terms, significantly overused in the mission statements as compared to
the written sampler of the British National Corpus. At 5 per cent probability
for log likelihood, the list contains a total of 2,102 types. Filtering out proper
names and technical terms, the top 15 items to possibly denote central con-
cepts in corporate discourse are: innovation, respect, excellence, integrity,
performance, trust, teamwork, responsibility, growth, committed / commit-
ments, creativity, competitive, transparency, professionalism and fairness. In
the persuasive genre represented by the texts in the corpus, even the less
obvious of these lexical items are found to be evaluated positively when
linked back to their co-text, for instance:
(1) We will invest in those businesses that promise the greatest value
growth. (Continental AG)
(2) The exceptional quality of our workforce is a valuable competitive edge.
(Mobil) [My emphases, VK]
We can therefore posit that these lexical items represent the predominant
values in corporate discourse, and hence the ideal type traits of a companys
brand personality. How then do verbal and visual modes in corporate dis-
course interact to convey the ideal corporate brand personality? The data
collected to answer this question are taken from a range of genres promoting
the corporate brand, such as brochures and the corpus of mission statements
mentioned above. The texts therefore represent instances of the secondary
corporate communication of marketing (Balmer 2006: 37). In general, the
components and characteristics of the BRANDS ARE PEOPLE metaphor are
seen as conveyed in illustrations, logos and layout, together with their verbal
co-text: The graphic designers brief that corporate design reflect the true
Brand images 53
reality of the company means that ideally, if you analyse a companys
design you can also trace its identity [and] personality (Johansson and
Holm 2006: 140). Although not discussed in this chapter, illustrations, logos
and layout, and other low-sensuality artifacts (Pratt and Rafaeli 2006: 281)
should be understood as acting in concert with other corporate artifacts in
communicating brand concepts. Further, while the present analysis focuses
on multimodal metaphors in which verbal components anchor a metaphoric
interpretation of visual elements, discursive construction and cognitive con-
ceptualization of brands can equally be realized in different modes, e.g.,
sounds and music, and the study can therefore be taken as a starting point
for empirically testing its claims against further data involving different mo-
dalities.
5. Illustration, logo, layout
In the data, multimodal metaphors with visual and verbal components fea-
ture in three ways: Firstly, illustrations on corporate websites support con-
ventional metaphors such as BRANDS ARE LIVING ORGANISMS. Secondly,
logos and logo elements tend to visualize particular characteristics ascribed
to the BRANDS ARE PEOPLE metaphor as a special case of the LIVING ORGAN-
ISMS metaphor. Such characteristics are often expressed using spatial con-
cepts as source domains (e.g., illustrating dynamism, balance, or openness).
Thirdly, layout can also visualize source domains of metaphoric expressions
that are anchored by verbal elements, e.g., when connecting design elements
in the corporate color are used in the brochure of a company that claims to
have a networking personality.

5.1 Illustration
As mentioned above, the About sections on corporate websites typically
include a companys values, vision, mission, philosophy and strategy, with
these terms often being used interchangeably. Ultimately elaborating on the
corporate identity and projecting it as corporate brand image, these sub-
sections are a good starting point to ascertain the multimodal metaphoric
construction of BRANDS ARE LIVING ORGANISMS and its specific form
BRANDS ARE PEOPLE. It is noteworthy how many websites in the corpus
contain images of human beings, mostly children, and of nature.
54 Veronika Koller

Figure 1. Hitachi corporate philosophy (http://www.htachi.com/about/vision/in-
dex.html, accessed 20 August 2007). Copyright in these images is
owned by Hitachi, Ltd. Reproduced with permission.
In the triptych structure of the above illustration (figure 1), a verbal repre-
sentation of the brand name, Hitachi, against the background of what looks
like the inside of a corporate building, appears in the left-hand picture, fol-
lowed by the low-angle shot of a skyscraper a staple of corporate iconog-
raphy and the picture of a tree. The latter is over-determined, in that it
serves, most obviously, as a literal illustration of Hitachis key strategic
theme Sustainable Environmental Solutions. Further, the text is typical of
the mission statement corpus in including metaphoric expressions of growth
([Were] aiming for high growth in the global market), and the tree can be
interpreted as a visual representation of the source domain in the metaphor
FINANCIAL SUCCESS IS NATURAL GROWTH. Finally, the dual visual connec-
tion of the brand with culture, as symbolized by the building, on the one
hand and nature, as represented by the tree, on the other finds a reflection in
the verbal co-text:
Brand images 55
(3) Since its founding in 1910, Hitachi has acted from a corporate philoso-
phy of contributing to society through technology. In the intervening
years, the world and society have changed greatly, but we have never lost
our pioneering spirit, based on the principles of harmony and sincerity
(my emphasis, VK).
By setting up a dichotomy of technology and harmony, the latter is concep-
tually linked to what one would normally expect as the opposite of technol-
ogy, namely nature. Not only does the size of the tree then suggest an age
equaling that of the company, but it may also be used to visualize the notion
of harmony, as linked to nature. In mission statements there is a clear, literal
link between harmony and nature, as evidenced by the following examples:
(4) We will shoulder the responsibility to contribute to a sustainable soci-
ety in harmony with nature. (Asahi)
(5) Stay in harmony with nature; blend in with local societies; and put our
hearts into creating a more vibrant, richer culture. (NYK Line)
(6) To conduct fair and open business operations while acknowledging our
social responsibilities and aspiring for harmony with our global envi-
ronment. (Toppan)
If the picture of the tree is indeed meant to metaphorically represent natural
growth, with the verbal elements further cueing a link between growth and
harmony, then we are dealing with a typical case of partial mappings in
promotional genres, in which all negatively connoted knowledge about the
source such as the disarray and imbalance brought about by growth are
muted (Ungerer 2000). Understood as such, we would be confronted with
a multimodal metaphor in which the source domain (NATURAL GROWTH) is
represented visually, while the verbal co-text relates another concept (HAR-
MONY) to the source domain, linking both to the target domain of the brand.
The relation between visuals and language here makes it possible, if not
necessary, to construe a metaphor FINANCIAL SUCCESS IS NATURAL
GROWTH. In that sense, multimodal metaphors that are based on the inter-
play between verbal and visual elements are not unlike pictorial similes
(Forceville 1996). This unforced metaphorical interpretation may well
serve to make instances like these powerful precisely because the metaphor
is neither explicit nor strongly signaled (Forceville 1999).
56 Veronika Koller
A similar multimodal metaphor BRANDS ARE LIVING ORGANISMS can be
found in the relevant section of IBMs website, with the visual element again
acting as a representation of the source domain (figure 2).

Figure 2. IBM values (available at http://www-03.ibm.com/ibm/history/exhibits/
valueone/valueone_intro.html , accessed 20 August 2007). Reprint cour-
tesy of International Business Machines Corporation, copyright Inter-
national Business Machines Corporation.
The stylized picture of a DNA helix next to the company name here instanti-
ates the BRANDS ARE LIVING ORGANISMS metaphor and represents a com-
panys values as an inherent and inalterable quality. The verbal text includes
the source domain only once (to bring IBMs values to life) but explicitly
mentions the target domain (what IBM has always been about and always
will be about). The juxtaposition of present perfect and future tense points
toward the inalterable nature of what is understood to be the brands core
and thus foregrounds this particular semantic component of the metaphor.
While the text does not include the word DNA as such, the visual repre-
sentation of the term intertextually draws on an established metaphoric ex-
pression in corporate discourse. Witness the following examples:
(7) To be successful at a house, a designer has to have an almost visceral
sense of what is now called the brands DNA: the abstract spirit of the
woman the clothes are made for, and the design vocabulary to communi-
cate it. (Friedman 2004)
(8) We all know that these are highly competitive markets, but I firmly
believe that the stellar attributes which are in the DNA of the brands and
Brand images 57
operating culture will enable the groups businesses to successfully dif-
ferentiate themselves and to compete effectively through a variety of
complementary retail channels. (Simon Fox, chief executive officer at
HMV, quoted in Killgren and Callan 2005)
(9) [The corporate culture] really has gotten into the DNA, says Ms Con-
sidine, who will stay on as executive chairman. I dont say that lightly
I used to be a biochemist. (Beales 2006)
In the last example, the metaphoric expression is given particular weight and
credibility by linking it to its source domain via the speakers professional
background. The genetic metaphor has not escaped the attention of brand
theorists, who have raised the question whether such an essentialist notion is
still viable given the fragmented consumer identities found in postmodern
markets (Csaba and Bengtsson 2006: 124125). On the other hand, essen-
tialist brand identity may be reinforced in corporate discourse to provide
orientation and the idea of reliability another key word in the mission
statement corpus for consumers.
Illustrations on corporate websites suggest that brand values are essen-
tialized as stable and inherent in a brand that is metaphorically conceptual-
ized as a living organism. Corporate logos specify the metaphor to BRANDS
ARE PEOPLE, revealing what allegedly inalterable traits are ascribed to such
a metaphorical human being.

5.2 Logo
Logos are central elements of corporate brand communication. Mostly mul-
timodal in themselves, they combine written and pictorial signs, sometimes
in addition to music in so-called sound logos, i.e., short jingles associated
with a brand. Intended to convey a highly condensed form of the brand im-
age, a logo meets cognitive and affective functions, serving as a visual cue
for the recall of information previously received In addition to the cogni-
tive responses of identifying the brand and remembering information, the
logo can also elicit affective responses, such as recall of pleasurable adver-
tising and positive brand experiences (Pimentel and Heckler 2003: 106).
Companies and the designers employed by them are remarkably verbose
when it comes to explaining the meaning of their logos to the public. Clearly,
the function of logos as instances of condensed brand communication, as
well as the ever increasing importance of brands, have led to a previously
unseen vanity with respect to the signifiers that organizations themselves
58 Veronika Koller
choose to manufacture (Christensen and Cheney 2000: 251). Energy com-
pany Totals visual identity portal is a case in point (figure 3):

Figure 3. Total logo. Reproduced with permission.
The companys logo shows the corporate name underneath an open ball
made up of interlaced tapes in red, yellow, light blue, and navy. The accom-
panying text offers the following interpretation:
(10) The logos spherical shape suggests both the Earth and the international
scope of our business. Constructed from a set of colored, curving lines, it
symbolizes the complexity of an ever-changing world. The logos colors
symbolize multiple energies. The colors chosen naturally evoke the idea
of heat. The intersecting curving lines convey an impression of move-
ment. The combination suggests concentrated, controlled energies serv-
ing heat, light and movement. The logo is airy and light, signifying that
energy is more than just a natural resource, but the product of human-
kinds ingenuity and exchanges.
7

This may provide interesting reading for graphic designers but is unlikely to
rivet the attention of stakeholders. Indeed, it is questionable whether espe-
cially consumers can, in their typically superficial encounters with logos,
really appreciate such symbolic meanings, or if they even wish to do so (see
Pimentel and Heckler 2003: 126). Grandiose explanations are also offered
by car manufacturer Mazda. The Mazda logo features the name underneath
a silver circle surrounding a gently curved element that resembles a stylized
bird. The company offers the following interpretation on their website:
8

(11) The brand symbol expresses Mazdas dedication to continuous growth
and improvement. It is a symbolic development of the Mazda M, and
shows the company stretching its wings as it soars into the future.
The text goes on to trace the meaning of the brand name to the West Asian
deity Ahura Mazda and elaborates the positive values it stands for, before
concluding, rather flatly: It also derives from the name of our founder, Ju-
jiro Matsuda.
Brand images 59
Common denominators in the two logo interpretations above seem to be
openness, lightness, and movement. While the first two of those values can
be literally visualized by open and non-solid circles, static images rely on
metaphor for representing movement as well as its associated qualities of
energy, activity, and dynamism. A metaphorical element used to this end in
many logos is a bow or curve (see figures 4a and 4b).

Figure 4a and 4b. Curved elements in logos. The HVB logo is reproduced with
permission. The Vanteon logo is owned by Vanteon and is used with
permission from Vanteon.
Be they for banks (HVB), sportswear companies (Nike),
9
software consult-
ants (Vanteon) or even not-for-profit organizations such as the British Home
Office,
10
curved logo elements (swooshes) seem ubiquitous, up to the point
of clich (Lindsay 2000). Perceived as conveying global reach [and] impact
full spectrums (Brannon Cashion, senior vice president at Addison
Whitney, quoted in Lindsay 2000: 204), curved logo elements visually repre-
sent one of the key words in the mission statement corpus, global. While
this design feature is not in itself metaphoric representing as it does a styl-
ized version of the globe other possible interpretation such as drive and
impetus are better candidates. Given that dynamic, proactive, and
agile are again key words in the mission statements corpus, curved logo
elements may be interpreted as contributing to a multimodal metaphor of
dynamic speed.
This reading hinges on whether the elements are understood as indexical
or symbolic. Swooshes could be seen as varieties of what Kennedy (1982)
has called runes, i.e., the action lines used in comics to indicate, among
other things, movement. In a later paper (Kennedy, Green, and Vervaeke
1993), such action lines are compared to a trail, as if left by lights on the
moving body, like time-lapse photography (p. 247). Understood as such,
action lines would be indexical in that they point to a moving object. In com-
posite logos, however, curved elements include the brand name instead of the
moving object, making the whole logo a multimodal metaphor that includes
the target domain in the verbal and the source domain in the visual mode.
11

60 Veronika Koller
It has been noted that the logo [has] a significant role in triggering
the cognitive frame that a company wishes to be perceived in (Baruch
2006: 182). In corporate discourse, particular frames will be prioritized over
others. Thus, characteristics such as dynamic and active are overrepre-
sented at the expense of contemplative and receptive, both verbally as
lexical items and visually as logo elements. Given the ubiquity not only of
certain logos but also of pervasively used logo elements such as the
swoosh, such endlessly repeated and recycled images arguably aspire
to conceptual status (Forceville 2006: 390). The mental models of activ-
ity and continuous movement have achieved a currency that makes these
models and their linguistic and visual expressions almost defining notions of
the discourses and other practices of corporations. Due to the prestige and
influence in late capitalism of all things corporate, such models become posi-
tively connoted in other areas as well and are likely to be desired as identity
traits by many consumers. Hence, metaphorically constructing brands as
people is intended to appeal to stakeholders and incite them to interact with
the brand and co-produce its identity (Csaba and Bengtsson 2006: 124), in
the hope of acquiring some of its characteristics by doing so. In how far this
strategy can be successful in the face of widespread consumer indifference
and even cynicism is an open question.

5.3 Layout
We have seen how the Hitachi website (figure 1) uses visual elements to
represent the source domain of the BRANDS ARE LIVING ORGANISMS meta-
phor. Moreover, the brand name was accompanied by visual elements signi-
fying the brands main characteristics. This device points to a third area of
use of multimodal metaphor in corporate branding discourse, namely layout.
Not just nameable objects, such as buildings and trees, but also layout can
assist in construing a metaphor, even though this strategy may require a
visually literate audience in order to be successful.
To take but one example, in various samples of the brand communication
of banking group HSBC (brochures, advertisements, posters, websites; see
Koller 2007) red, one of its corporate colors, acts as a structuring device:
Red sub-headings, red frames and red bullet points all serve layout functions
in arranging pictorial elements. In one brochure, red lines connect para-
graphs about the various places where the company does business. The lines
also frame details of easily recognizable, even clichd local symbols such as
a Chinese dragon or the Eiffel Tower. As such, the layout of the brochure is
Brand images 61
an example of pictorial grouping, understood as symmetric image align-
ment, i.e., the spatial organization of pictorial elements in terms of size,
orientation or distance (Teng and Sun 2002; see also Teng, this volume). If,
as in the HSBC example, the aligned elements depict things of the same
kind, their grouping produce[s] the cognitive effect of inviting the viewers
to see the depicted entities as belonging to the same category (Teng and Sun
2002: 300). Furthermore, the corporate color red is metonymic in that it
stands for the whole HSBC group. At the same time it functions to illustrate
the idea that HSBC connects places and people, by dint of the red lines liter-
ally connecting the symbols of different localities the company is present in.
The metonymy is again cued by its verbal component, as the layout just
described can be found in HSBCs Business Connections brochure. Simi-
larly, in their history brochure, the company makes references to its net-
work 24 times and to connection(s) four times in a total of 11,563 words.
Curiously, both these central terms feature in nominal form only, yet it could
be argued that the vectors
12
and lines of the brochures layout are equivalent
to action verbs (Kress and van Leeuwen 2006: 46).
Another statistically overused word in mission statements is connect-
ing, as in the following instance:
(12) Our brands are connecting with customers in ways that will last a life-
time. (Whirlpool)
Thus, the layout of corporate branding messages is another indication of
how all elements metonymically associated with the source domain qual-
ify as potential candidates for a mapping (Forceville 2006: 384/this vol-
ume). In integrated communications, the source domain of BRANDS ARE
LIVING ORGANISMS as such is provided by illustrations, while logos and
layout specify the conceptualization of corporate brands to BRANDS ARE
PEOPLE and provide visualizations of that metaphorical persons character
traits. Given the persuasive genres involved, such traits will be ideal rather
than real, both for the identity of the corporate brand and for that of the
stakeholders interacting with it.
6. Discussion and conclusion
This leaves us with the question why corporate brands should be personified
in the first place. What are the specific affordances of BRANDS ARE PEOPLE
that make this metaphor appropriate for the persuasive genres of corporate
branding communication? As Lakoff and Turner (1989: 72) note with regard
62 Veronika Koller
to poetic metaphor, [p]ersonification permits us to use our knowledge about
ourselves to maximal effect, to use insights about ourselves to help us com-
prehend abstract concepts. Branding messages use personification to aid
cognitive processing of the brand concept, compressing the complex system
that is a corporation into an abstract bundle of characteristics and making
this abstraction graspable by linking it to human personality as the source
domain. As far as those personality traits are supposed to be desirable, per-
sonification also serves a persuasive purpose. The metaphoric personality
traits, or brand values, are communicated by having verbal elements such as
key words anchor interpretation of visual elements. Indeed, the pattern to
emerge from the above examples is that the target of the BRANDS ARE LIV-
ING ORGANISMS/PEOPLE metaphor is encoded verbally, with the company
name in figure 1 being integrated into a visual element. The source domain
in all cases is encoded visually, and occasionally in additional verbal form,
e.g., in the Hitachi mission statement (figure 1) elaborating on growth, the
IBM values statement (figure 2) talking about bringing values to life or in
the HSBC Business Connections brochure (my emphasis, VK). Dual encod-
ing of the source domain seems to be less common in logos, but this and
other results will have to be tested on a much broader scale.
Multimodal encoding has the potential for further persuasion by reinforc-
ing the desirable characteristics in two modes. The aim of persuasive genres
is therefore less likely to be met if the visuals were omitted, especially since
the positive values that are visualized carry affective components that are
intended to reinforce the brand message. Moreover, brand characteristics can
reflect readers perceived personality, as shown in this description of the
following two (product) brands:
(13) Hachez [chocolate] is the brand of the banker, the businessman in a dark
suit and bowler hat. If it was a person, it would be a distinguished eld-
erly gentleman with a distinct feel for quality and indulgence. (Hasso
Nauck, chief executive officer at Hachez, quoted in Willenbrock 2005:
103, trans. VK)
(14) A volkswagen is like a polite, good-looking person. A nice, friendly
attitude, rather unassuming, definitely not a show-off it is a good-
natured car, a bit moody every now and then, just like a human being. A
loyal, reliable life partner. (Volkswagen customers quoted in Lauden-
bach 2005: 87, trans. VK)
The second of the above quotations is elaborated by the writer of the article,
who remarks that his interview partners seem to be describing themselves
Brand images 63
when listing the attributes of their cars. This narcissistic relationship be-
tween the consumer and the brand is made possible by first ascertaining
social and psychological aspects of the target audience through market re-
search and subsequently constructing the brand personality as mirroring
that of the ideal consumer. This conceptualization is effected through mar-
keting, particularly branding messages and their typical features, such as
multimodal metaphor. Unsurprisingly, given the persuasive thrust of market-
ing, the practice is highly manipulative in that it addresses consumers self-
image and exploits potential lack of self-esteem. On a more positive note,
sociological research into brand communities (Muniz and OGuinn 2001)
has made a case for brands fostering social cohesion among individuals iden-
tifying with a brand, rather than fragmenting societies into narcissistic indi-
viduals engaging in compensatory brand consumption. Given the nature of
logos as condensed branding messages, such symbols obviously play a cen-
tral role as textual resources for such brand communities (Muniz and
OGuinn 2001: 423).
The double-edged nature of branding as a corporate discourse practice is
captured in its description as a (global) ideoscape [that] provides the ideo-
logical basis for the establishment of new meaning systems, new practices
and new identity forms for the members of the consumer culture (Aske-
gaard 2006: 98). Although vocal criticism of branding (Klein 2000) has
ironically become part of branding itself, companies still have a vital interest
in not having their brands demystified. One strategy to mystify brands is
naturalization: Verbally and visually, texts promoting corporate brands draw
on LIVING ORGANISM metaphors to recast social practices as natural kinds
and thus make them less vulnerable to criticism. Rothbart and Taylors
(1992) observation that social categories tend to be viewed in an essentialist
fashion as homogeneous and inalterable natural kinds is crucial in this con-
text. The question has been raised whether essentialist notions of brands are
suitable for postmodern consumer identities (Csaba and Bengtsson 2006:
130131). However, given the persuasive thrust of branding communication,
it can equally be argued that brands illusions of stability and inalterability
not only respond to public mistrust but actually attract stakeholders who
lack orientation in highly fragmented societies, for better or worse.
Yet, with promotional texts, genre is key: The designing institution in
each case is a company whose ulterior motive in producing and distributing
texts is promoting itself as a brand and, ultimately, maximize profits. Audi-
ences can be expected to be aware of, and even cynical towards, the profit
64 Veronika Koller
motive (Messaris 1997: 159). Encountering brand images in recognized
persuasive genres will therefore constrain interpretation and make uncritical
acceptance of the communicated brand image unlikely. In this context, it is
crucial to remember that often the viewer is invited rather than forced to
understand the visuals as encoding the BRANDS ARE LIVING ORGAN-
ISMS/PEOPLE metaphor. Outside the context of the specific instance of brand
communication, there would be little to suggest that illustrations, logos or
layout should be taken metaphorically; it is only in the given genre that they
suggest metaphoric construal, at least to some viewers. Also, the instances
discussed above leave some leeway for alternative interpretations of visual
elements, e.g., the rooflike protectiveness offered by the Home Office logo
(see n. 10).
Finally, as indicated above, at least consumers may simply not be inter-
ested enough to engage with corporate branding messages on more than su-
perficial terms. To be sure, this may be different for other stakeholders:
Branding messages are also directed inwardly, i.e., they communicate the
corporate brand identity to employees with the intention to foster employee
loyalty and identification (Schultz, Hatch, and Ciccolella 2006: 143). Given
the institutional power of the sender vis--vis this particular stakeholder
group, employees may be less able to afford the cynicism and/or indifference
with which consumers may react to branding messages. Even when employ-
ees feel cynical towards their employers brand identity, voicing such cyni-
cism on record is rarely advisable. Given the reports of senior managements
attempts to control ethnographic research into corporate discourse (personal
communication with Jo Angouri, 13 December 2005), it is therefore difficult
to find more than anecdotal evidence for negative or indifferent employee
attitudes toward corporate brands. We can therefore expect corporate brand-
ing efforts to increasingly be directed inwardly, leading to further corporate
self-absorption (Christensen and Cheney 2000) on the part of companies
with their metaphorical brand personalities and their multimodal expressions
thereof.
Notes
1. I would like to thank the two editors as well as Rosario Caballero for their
helpful comments on a draft version of this chapter.
2. The embodiment approach to metaphor (Lakoff and Johnson 1999) sees
kinesthetics as the foundation of basic, primary metaphoric models that are
Brand images 65
acquired early in life, e.g. GOOD IS UP (Grady 1997). However, note the dif-
ference between kinesthetically founded conceptual metaphor as a cognitive
phenomenon and gestural metaphor as a surface-level expression of such a
metaphoric model.
3. The image of the race could be literal as an illustration of, say, a marathon
sponsored by the company in question. A case could be made for sponsoring
practices literalizing aspects of the brand personality the company constructs
for itself, e.g., that of fastness and speed.
4. Discourse is here defined as the total of texts produced, distributed and re-
ceived between members of a particular social field, e.g., the business com-
munity.
5. In branding and organizational theory, an internally communicated corporate
brand image is sometimes referred to as brand identity (Hatch and Schultz
2000: 22).
6. Indeed, the metaphor BRANDS ARE PEOPLE has become so entrenched in cor-
porate discourse that it has been used as a methodological tool. Thus, Davies
and Chun (2002) developed a Corporate Personality Scale to measure the dif-
ference between internal and external images of organizations.
7. http://www.total.com/identite/portail/en/index.htm, accessed 12 February
2009.
8. http://www.mazda.com/profile/vision/, accessed 12 February 2009.
9. http://www.nike.com/nikeos/p/nike/en_GB/, accessed 12 February 2009.
10. http://www.homeoffice.gov.uk/, accessed 20 August 2007. In its combination
of traditional (coat of arms) and modern (curve) elements, the Home Office
logo is a good example of the transition from time-honored heraldic symbols
to contemporary abstract logos (see Baruch 2006, Frick 1996, Mautner forth-
coming, Pimentel and Heckler 2003). The curved element can also be read as
reinforcing the word home over which it arches, with both drawing on no-
tions of safety and protection.
11. The metaphoric nature of such logos becomes even more obvious when the
curve is interpreted as a movement heavenwards.
12. In Kress and van Leeuwens theory of visual grammar, vectors are formed
by depicted elements that form an oblique line, typically a diagonal one that
indicates an actor and the goal acted upon. In the HSBC example, connect-
ing lines without an indicator of directionality mean something like is
connected to, is conjoined to, is related to (Kress and van Leeuwen
2006: 59).
References
Askegaard, Sren
2006 Brands as a global ideoscape. In Brand Culture, Jonathan E.
Schroeder and Miriam Salzer-Mrling (eds.), 91102. London/New
York: Routledge.
66 Veronika Koller
Balmer, John T.M.
2006 Corporate brand cultures and communities. In Brand Culture,
Jonathan E. Schroeder and Miriam Salzer-Mrling (eds.), 3449.
2000 Metaphor and Metonymy at the Crossroads: A Cognitive
Perspective. Berlin/New York: Mouton de Gruyter.
Barthes, Roland
1977 Image, Music, Text. Transl. by Stephen Heath. London: Fontana.
Baruch, Yehuda
2006 On logos, business cards: The case of UK universities. In Artifacts
and Organizations: Beyond Mere Symbolism, Anat Rafaeli and
Michael G. Pratt (eds.), 181198. Mahwah, NJ: Lawrence Erlbaum.
Beales, Richard
2006 Teaching a gorilla to tap-dance. Financial Times 5 July. Available
from http://search.ft.com/searchArticle?page=5&queryText=DNA&
y=0&javascriptEnabled=true&id=060705000146&x=5. Accessed 19
September 2006.
Cappetta, Rossella, and Dennis A. Gioia
2006 Fine fashion: Using symbolic artifacts, sensemaking, and
sensegiving to construct identity and image. In Artifacts and
Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael
G. Pratt (eds.), 199219. Mahwah, NJ: Lawrence Erlbaum.
Carroll, Noel
de Chernatony, Leslie
2002 Would a brand smell any sweeter by a corporate name? Corporate
Reputation Review 5: 114132.
Christensen, Lars Thger, and Sren Askegaard
2001 Corporate identity and corporate image revisited: A semiotic
perspective. European Journal of Marketing 35: 292315.
Christensen, Lars Thger, and George Cheney
2000 Self-absorption and self-seduction in the corporate identity game. In
The Expressive Organization: Linking Identity, Reputation, and the
Corporate Brand, Majken Schultz, Mary Jo Hatch, and Mogens H.
Larsen (eds.), 246270. Oxford/New York: Oxford University Press.
Collins, James C., and Jerry I. Porras
1996 Building your companys vision. Harvard Business Review 74: 65
77.
Csaba, Fabian F., and Anders Bengtsson
2006 Rethinking identity in brand management. In Brand Culture,
Brand images 67
Cunliffe, Ann L., and John Shotter
2006 Linguistic artifacts in organizing and managing. In Artifacts and
Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael
G. Pratt (eds.), 119137. Mahwah, NJ: Lawrence Erlbaum.
Czerniawska, Fiona
1997 Corporate Speak: The Use of Language in Business. London:
Macmillan.
Davies, Gray, and Rosa Chun
2002 Gaps between the internal and external perceptions of the corporate
brand. Corporate Reputation Review 5: 144158.
Elliott, Richard, and Kritsadarat Wattanasuwan
1998 Brands as symbolic resources for the construction of identity.
International Journal of Advertising 17: 131144.
Forceville, Charles
1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Activity
9: 129.
1999 The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters,
and Paul Schraders The Comfort of Strangers. Metaphor and
Symbol 14: 179198.
work: Agendas for research. In Cognitive Linguistics: Current
Applications and Future Perspectives, Gitte Kristiansen, Michel
Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379
2008 Metaphors in pictures and multimodal representations. In The
Cambridge Handbook of Metaphor, Raymond W. Gibbs, Jr. (ed.).
Fox, Renata, and John Fox
2004 Organizational Discourse: A Language-Ideology-Power Perspec-
tive. Westport, CT/London: Praeger.
Frick, Wolfgang
1996 Corporate Design als Identittsstrategie: ber die Relevanz der
Deckungsgleichheit zwischen Selbst- und Fremdbild. [Corporate
design as identity strategy: On the relevance of converging self and
other images] Ph.D. dissertation, University Salzburg.
Friedman, Vanessa
2004 New blood reinvents a venerable DNA. Financial Times 12 October.
Available from http://search.ft.com/searchArticle?page=3&query
68 Veronika Koller
Text=DNA&y=0&javascriptEnabled=true&id=041012000957&x=5
Accessed 19 September 2006.
Grady, Joseph
1997 Foundations of Meaning: Primary Metaphors and Primary Scenes.
Ph.D. dissertation. University of California Berkeley.
Harquail, Celia V.
2006 Employees as animate artifacts: Employee branding by wearing the
brand. In Artifacts and Organizations: Beyond Mere Symbolism,
Anat Rafaeli and Michael G. Pratt (eds.), 161180. Mahwah, NJ:
Lawrence Erlbaum.
Hatch, Mary Jo, and Majken Schultz
2000 Scaling the Tower of Babel: Relational differences between identity,
image, and culture in organizations. In The Expressive
Organization: Linking Identity, Reputation, and the Corporate
Brand, Majken Schultz, Mary Jo Hatch, and Mogens H. Larsen
(eds.), 1135. Oxford: Oxford University Press.
Holt, Douglas B.
2002 Why do brands cause trouble? A dialectical theory of consumer
culture and branding. Journal of Consumer Research 29: 7090.
Johansson, Ulla, and Lisbeth S. Holm
2006 Brand management and design management. In Brand Culture,
Kapferer, Jean-Nol
2002 Corporate brand and organizational identity. In Corporate and
Organizational Identities: Integrating Strategy, Marketing,
Communication and Organizational Perspectives, Bertrand
Moingeon and Guillaume Soenen (eds.), 175193. London/New
York: Routledge.
Kennedy, John M.
Kennedy, John M., Christopher D. Green, and John Vervaeke
1993 Metaphoric thought and devices in pictures. Metaphor and Symbolic
Activity 8: 243255.
Killgren, Lucy, and Eoin Callan
2004 HMV names new chief. Financial Times 18 July. Available from
http://search.ft.com/searchArticle?page=4&queryText=DNA&y=0&
javascriptEnabled=true&id=06071Mautner8003693&x=5. Accessed
19 September 2006.
Klein, Naomi
2000 No Logo: Taking Aim at the Brand Bullies. New York: Picador.
Brand images 69
Koller, Veronika
2005 Designing cognition: Visual metaphor as a design feature in
business magazines. Information Design Journal and Document
Design 13: 136150.
2007 The worlds local bank: Glocalization as a strategy in corporate
branding discourse. Social Semiotics 17: 111130.
2008 Our customers embrace us as an essential partner: Corporate
brands as socio-cognitive representations. In Cognitive
Sociolinguistics: Language Variation, Cultural Models, Social
Systems, Gitte Kristiansen and Ren Dirven (eds.), 389418. Berlin/
New York: Mouton de Gruyter.
2006 Reading Images: The Grammar of Visual Design. 2nd ed.
Kristiansen, Gitte
2008 Idealized cultural models: The group as a variable in the develop-
ment of cognitive schemata. In Body, Language and Mind. Volume
2: Sociocultural Situatedness (Cognitive Linguistics Research 35.2),
Roslyn M. Frank, Ren Dirven, Tom Ziemke, and Enrique
Bernrdez (eds.), 409432. Berlin/New York: Mouton de Gruyter.
Kunda, Ziva
1999 Social Cognition: Making Sense of People. Cambridge, MA: MIT
Press.
1999 Philosophy in the Flesh: The Embodied Mind and Its Challenge to
2003 Metaphors We Live By. 2nd ed. Chicago: University of Chicago
Press.
1989 More Than Cool Reason: A Field Guide to Poetic Metaphor.
Lassen, Inger, Strunck, Jeanne, and Torben Vestergaard (eds.)
2006 Mediating Ideology in Text and Image. Amsterdam/Philadelphia:
Benjamins.
Laudenbach, Peter
2005 Wagen ohne Volk. [Volkswagen: a car without people] brand eins
8: 8490.
Lindsay, Greg
2000 I want a unique logo just like theirs. Fortune 24 July: 204205.
Mautner, Gerlinde
forthc. The spread of corporate discourse to other social domains. In
Language and the Market, Helen Kelly-Holmes and Gerlinde
Mautner (eds.). Basingstoke: Palgrave Macmillan.
70 Veronika Koller
Messaris, Paul
1997 Visual Persuasion: The Role of Images in Advertising. London/
Thousand Oaks, CA: Sage.
Morgan, Gareth
1986 Images of Organization. London/Thousand Oaks, CA: Sage.
Muniz, Albert M., and Thomas C. OGuinn
2001 Brand community. Journal of Consumer Research 27: 412432.
Musolff, Andreas
2338.
2003 Understanding visual metaphor in advertising. In Persuasive
Imagery: A Consumer Response Perspective, Linda M. Scott and
Rajeev Batra (eds.), 297310. Mahwah, NJ: Lawrence Erlbaum.
Phillips, Barbara J., and Edward F. McQuarrie
2004 Beyond visual metaphor: A new typology of visual rhetoric in
advertising. Marketing Theory 4: 113136.
Pimentel, Ronald W., and Susan E. Heckler
2003 Changes in logo design: Chasing the elusive butterfly curve. In
Persuasive Imagery: A Consumer Response Perspective, Linda M.
Scott and Rajeev Batra (eds.), 105127. Mahwah, NJ: Lawrence
Erlbaum.
Pratt, Michael G., and Anat Rafaeli
2006 Artifacts and organizations: Understanding our objective reality.
In Artifacts and Organizations: Beyond Mere Symbolism, Anat
Rafaeli and Michael G. Pratt (eds.), 279288. Mahwah, NJ:
Lawrence Erlbaum.
Rayson, Paul
2008 Wmatrix: A web-based corpus processing environment. Computing
Department, Lancaster University. Available from http://ucrel.lancs.
ac.uk/wmatrix/.
Rothbart, Myron, and Marjorie Taylor
1992 Category labels and social reality: Do we view social categories as
natural kinds? In Language, Interaction and Social Cognition, Gn
R. Semin and Klaus Fiedler (eds.), 1136. London/Thousand Oaks,
CA: Sage.
Rumbo, Joseph
2002 Consumer resistance in a world of advertising clutter: The case of
ad busters. Psychology & Marketing 19 (2): 127148.
Schultz, Majken, Hatch, Mary Jo, and Francesco Ciccolella
2006 Brand life in symbols and artifacts: The LEGO company. In
Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli
Brand images 71
and Michael G. Pratt (eds.), 141160. Mahwah, NJ: Lawrence
Erlbaum.
Schwartz, Jonathan
2006 The five founding principles that drive innovation. Financial Times
September 13. Available from http://www.ft.com/cms/s/17cd2eae-
42c4-11db-8dc3-0000779e2340.html. Last accessed 11 February
2009.
Soenen, Guillaume, and Bertrand Moingeon
2002 The five facets of collective identities: Integrating corporate and
organizational identity. In Corporate and Organizational Identities:
Integrating Strategy, Marketing, Communication and Organiza-
tional Perspectives, Bertrand Moingeon and Guillaume Soenen
(eds.), 1334. London/New York: Routledge.
Swales, John M., and Priscilla Rogers
1995 Discourse and the projection of corporate culture: The mission
statement. Discourse and Society 4: 223242.
2002 Grouping, simile, and oxymoron in pictures: A design-based
cognitive approach. Metaphor and Symbol 17: 295316.
Ungerer, Friedrich
2000 Muted metaphors and the activation of metonymies in advertising.
In Metaphor and Metonymy at the Crossroads: A Cognitive
Perspective, Antonio Barcelona (ed.), 321340. Berlin/New York:
Mouton de Gruyter.
van Dijk, Teun A.
2002 The discourse-knowledge interface. In Critical Discourse Analysis:
Theory and Interdisciplinarity, Gilbert Weiss and Ruth Wodak
(eds.), 85109. Basingstoke/New York: Palgrave.
2005 Contextual knowledge management in discourse production: A
CDA perspective. In A New Agenda in (Critical) Discourse
Analysis: Theory, Methodology and Interdisciplinarity, Ruth
Wodak and Paul Chilton (eds.), 71100. Amsterdam/Philadelphia:
Benjamins.
van Leeuwen, Theo
1999 Speech, Music, Sound. London: Macmillan.
van Leeuwen, Theo, and Ruth Wodak
1999 Legitimizing immigration control: A discourse-historical analysis.
Discourse Studies 1: 83118.
Wee, Thomas Tan Tsu
2004 Extending human personality to brands: The stability factor.
Journal of Brand Management 11: 317331.
Willenbrock, Harald
2005 Der Unterschied. [The difference] brand eins 2: 98103.
Chapter 4
Cutting across the senses: Imagery in winespeak
and audiovisual promotion
Rosario Caballero
Abstract
The present chapter surveys some of the metaphors recurrently used in wine pro-
motion. The starting point is the tasting note, a genre that basically deals with
translating a sensorial and highly subjective experience into comprehensible and,
above all, shareable terms. This endeavor implies the abundant use of figurative
language. If we take into account that tasting notes are the verbal translations of
organoleptic experiences (i.e., perceived by a sense organ), the issue here is
whether the metaphors used are monomodal or multimodal according to the defi-
nition of multimodal metaphor in the present volume.
Some of the metaphorical frames in winespeak are also exploited to promote
wine in commercials and print advertisements particularly in the latter. Both
genres show that imagery may encompass modes other than language but, at the
same time, here the difficulties of communicating smell and taste are particularly
conspicuous. Indeed, should we have to choose one mode of expression for this
particular endeavor, wine advertising points to language as the best albeit also
limited tool at our disposal. In this regard, my second aim in this chapter is to
discuss the problems derived from attempting to translate verbal metaphors into
images in wine advertising the metaphors in this case being unquestionably
multimodal and the weight of culture in using and, above all, interpreting figu-
rative phenomena rendered via images.

Keywords: Winespeak, sensory perception, culture, tasting note, advertising gen-
res, cross-modal metaphor
1. Introduction
Wine is becoming a cultural icon in an emerging hedonistic sub-culture ac-
cessible to an ever larger number of consumers. Among the genres articulat-
ing the discourse of wine, the tasting note (henceforth, TN) has played a
74 Rosario Caballero
critical role in introducing people to wine-tasting procedures and language
and, at the same time, has critically contributed to promoting wine. And, yet,
irrespective of the growing popularity of this beverage, wine jargon
(winespeak) still retains some of the mystique traditionally associated with
the topic a mystique that partly rests on the use of figurative language.
One of the reasons underlying the figurative quality of winespeak is the
shortage of terms available to describe two crucial sensory experiences in
wine tasting, namely, smell and taste. Thus, their verbalization usually in-
volves using metonymies (ripe flavors), similes (it smells like a barnyard),
and synesthetic metaphors (it smells crisp). In other words, since there is no
single lexicon with the expressive potential to cover aroma and flavor nu-
ances, their communication is inextricably linked to metaphor (in the broad
sense of the term). However, hitherto there are few studies devoted to explor-
ing metaphor in wine discourse (although see Lehrer 1975, 1983, 1992,
2006; Peynaud 1987; Bruce 2000; Amoraritei 2002; Gluck 2003). In this
regard, one of my aims in this chapter is to survey the figurative language
used by wine critics in TNs. Particular attention will be paid to the cross-
sensory/modal quality of such language in the genre and, hence, the meta-
phors motivating it. For multimodal metaphors I have adopted Forcevilles
definition as those whose target and source are each represented exclusively
or predominantly in different modes (Forceville 2006: 384/this volume), yet
have broadened its scope to some extent. For if we take into account that
TNs are the verbal translations of organoleptic experiences (i.e., those per-
ceived by a sense organ), the metaphors involved may be equally seen as
cutting across various senses and/or modes of expression i.e., as being
multimodal in the broadest sense of the term.
Some of the metaphorical frames in winespeak are also exploited to ad-
vertise wine in commercials and print advertisements particularly in the
latter. Both genres show that imagery may encompass modes other than
language (as discussed throughout this book) yet, at the same time, here the
difficulties of communicating smell and taste are particularly conspicuous.
In other words, it is one thing to acknowledge the heuristic role of metaphor
in wine discourse, and it is another to ignore the limitations of the modes in
which metaphor is instantiated for sharing sensory experiences with a given
audience. Indeed, should we have to choose one mode of expression for this
particular endeavor, wine advertising points to language as the best albeit
also limited tool at our disposal. In this regard, my second aim in this
chapter is to discuss the problems derived from attempting to translate
verbal metaphors into images in wine advertising the metaphors in this
case being unquestionably multimodal.
Cutting across the senses 75
In the present discussion, major insights have been drawn from an ongo-
ing research project devoted to exploring metaphor in a corpus of 12,000
TNs and 100 print advertisements retrieved from American and British wine
publications. However, for this chapter I have also used a small corpus
comprising texts and adverts in Spanish and French.
1
The main reason for
including these lies in the fact that adverts need not necessarily be translated
into the language of the specialized magazine inserting them (French being a
case in point given its former status as the language every oenologist and
wine lover should master). Moreover, although the metaphors and metony-
mies underlying winespeak largely cut across languages and cultures
(Nedilko 2006), this is not always the case when images are concerned, or
when images and words combine in wine adverts. In this regard, comparing
how wines from different countries are advertised may bring to the fore the
weight of culture in using and, above all, interpreting figurative phenomena
rendered via images. A clarification concerning my use of the term culture in
this chapter is in order at this point. Thus, culture covers both the shared
beliefs, knowledge and world view(s) characterizing national groups as well
as the more specific beliefs, knowledge and view(s) of the community articu-
lated around wine in the latter case, culture cutting across national or re-
gional differences and foregrounding the importance of topic and shared
interests in building up discourse communities or cultures within cultures.
The chapter is organized as follows. It starts with a survey of the figura-
tive language used by wine critics in TNs. This is followed by a discussion
of the cross-modal or multimodal quality of such language and its possible
relationships with other figurative phenomena. Then I discuss the use of
imagery in advertising genres, pointing to their heavy dependence on the
verbal mode and the difficulties arising from this fact.
2. Figurative language in tasting notes
The tasting note is one of the most representative and popular genres in wine
discourse, as well as a key instrument in the process of wine acculturation.
TNs are short texts (from 20 to 200 words) devoted to describing and evalu-
ating wine, and are prototypically organized in three distinct sections that
capture the three canonical steps in any wine tasting procedure, namely, the
assessment of wines (a) color, (b) smell metonymically referred to as the
wines nose/nariz/nez in English, Spanish, and French respectively, and (c)
mouth-feel which subsumes smell, taste, and touch, and is metonymically
referred to as the wines palate/boca/bouche.
Another conspicuous characteristic of TNs is their heavy reliance on
figurative language, which, as happens with their rhetorical structure, is
shared by texts written in the three languages under survey. The figurative
repertoire illustrated in the genre draws upon diverse metaphorical frames,
among which the most salient portray wines as if they were LIVING ORGAN-
ISMS, TEXTILES, or THREE-DIMENSIONAL ARTIFACTS. (As discussed later,
this does not mean that wine critics actually equate wines with such entities,
but the labels may be helpful for surveying the most salient domains in-
volved in wine commentary and will, therefore, be used throughout the chap-
ter.)
Textile renderings of wines in English are conveyed through reference to
wine elements as their cloak, glove, frock, or mantle, that is, as PIECES OF
CLOTHING which may well suggest a personified view of wines. Neverthe-
less, textile terms are mostly used either to describe the structural properties
of wines (e.g., these have a fabric or weft, may burst at their seams, and can
be variously described as a tapestry, open-knit, well meshed, or tightly
wound) or to evaluate their feel in the tasters mouth (e.g. silky, velvety,
satiny, pillowy). In contrast, Spanish TNs yield fewer and less varied textile
expressions, although we also find nouns like capa (robe) used to refer to
the intensity of a wines color, adjectives such as aterciopelado (velvety)
or sedoso (silky) qualifying wines mouth-feel, and descriptions of tannins
as cloth that tapiza(n) la boca (literally, carpet the mouth). Finally,
French critics also refer to a wines color as its robe (the term being, in fact,
borrowed by Spanish and English), and use terms like appret (starch),
soyeux (silky), and velout (velvety) to assess the mouth-feel of wines,
and toff to qualify a full-bodied wine.
2
Textile metaphors are illustrated in
the following examples:
(1) Blackberry jam and pepper were well integrated into the fabric of this
wine
(2) A monster in a beautiful frock. loads of velvety tannins.
(3) Los taninos [del vino] son puro terciopelo. [The tannins are pure
velvet]
(4) Trs belle robe rouge pourpre quilibre d aux tannins fins et soyeux.
[Very beautiful red purple color/robe balance due to fine and silky
tannins]
(5) En bouche, il se dploie comme un tapis de velours sur lequel dansent les
armes. [In the mouth, it unfurls like a tapestry on which aromas dance]
The metaphorical frame WINES ARE THREE-DIMENSIONAL ARTIFACTS may
be further decomposed in two related frames. On the one hand, we find
wines described as GEOMETRICAL ENTITIES which, accordingly, can have
edges (Sp. puntas), layers, contours, backs, and fronts, and are often
evaluated as being square (Fr. carr), angular (Sp. anguloso, Fr.
anguleux), wide (Fr. ample), long (Sp. largo, Fr. long), pointed
(Sp. afilado), deep (Sp. profundo, Fr. profond) or round (Sp.
redondo, Fr. rond). Indeed, according to the French wine expert
Peynaud, the ideal wine should be round or spherical (Peynaud 1987). On
the other hand, some terms suggest a view of wines as BUILDINGS. In Eng-
lish, this metaphorical frame is evoked by reference to wines as edifices or
monuments which are constructed, built, buttressed, backed up or fortified
by all or some of their constituents, or by evaluative terms such as monu-
mental, massive, monolithic, foursquare or skyscraper-like. Although this
architectural frame appears to be less productive in Spanish and French,
terms like armado (reinforced), construido (built), and ensamblado
(assembled) in Spanish, and French charpent (well structured/built)
3

suggest that critics also draw upon the domain of architecture to discuss the
structural properties of wines see also Lehrer (2006) in this respect. By
way of illustration, consider the following passages:
(6) Solidly built yet balanced [wine].
(7) The fruit [in this wine] barely peeks through the wall of chewy tannins
on the finish.
(8) Bien armado, con cuerpo y taninos firmes [Well assembled, with
body and firm tannins]
(9) La bouche a des tanins matures, bien fondus mais qui donnent une
architecture bien dfinie. [The mouth has ripe, well integrated tannins
which provide a well defined architecture]
All in all, however, the most salient metaphorical frame in the three lan-
guages at issue is WINES ARE LIVING ORGANISMS. This is far from surpris-
ing since wine is a mutable entity resulting from an organic process: the juice
extracted from grapes changes considerably along its life inside both oak
casks and bottles a process referred to as breeding or ageing (Sp. cri-
anza, Fr. levage) as well as during the very act of drinking it. Thus,
among the many terms used to describe wines evolutionary state we find
forceful (Sp. vigoroso, Fr. vigoreux), weak (Sp. dbil, Fr. faible),
youthful (Sp. joven or juvenil, Fr. jeune), tired (Sp. agotado, Fr.
fatigu), or old (Sp. viejo, Fr. vieux or senile). Nevertheless, this
organic frame is not only concerned with wines physiological dimension,
but is also used to talk about its structural, behavioral, and social prop-
erties.
Wines structural components are frequently described by means of ana-
tomically biased lexis. For instance, the trickles on the inside of a glass indi-
cating the alcohol concentration of a wine are conventionally referred to as
its legs or tears (Sp. lgrimas, Fr. jambes or larmes), and the general
effect or weight of a wine in the tasters mouth is referred to as its body (Sp.
cuerpo, Fr. corps). Accordingly, wines can be described as light-bodied
(Sp. con cuerpo ligero, Fr. lger) and full-bodied (Sp. con cuerpo, Fr.
cors) or, more conspicuously, as brawny (Sp. musculoso, Fr. mus-
cl), fat (Sp. graso, Fr. gras), flabby (Fr. toff), fleshy (Sp. carno-
so, Fr. charnu), lean or slim (Fr. mince), or thin (Sp. con poco
cuerpo, Fr. maigre). Two other common terms in wine description are
masculine (Sp. masculino or viril, Fr. viril) and feminine (Sp.
femenino, Fr. fminin).
4

The behavioral properties of wines are usually assessed by lexis typical
of the human realm. This personifying view of wines is cued by adjectives
such as aggressive (Sp. agresivo, Fr. agressif), pretty-handsome (Fr.
joli), in-your-face, upfront, or honest (Sp. franco, Fr. franc), sexy,
demure or shy (Sp. hurao, Fr. rserv), or expressive (Sp. expre-
sivo), suave (Sp. amable, Fr. aimable), or civilized (Sp. civilizado).
Finally, English terms like blue blood, clone, pedigree, sister, mate, sibling
or peer, and Spanish primognito (firstborn) or hijo (son) foreground
the social and kinship dimensions of wines. The examples below illus-
trate several instantiations of this anthropomorphic frame:
(10) The big, bold, and sultry 2001 Chevalier-Montrachet is a highly expres-
sive, fleshy, supple wine This white chocolate, truffle, cream, and
spice-scented beauty is medium-bodied, concentrated, and sexy.
(11) [This wine] has a nicely buried backbone of acidity and tannin. ulti-
mate impression of both muscle and flesh.
(12) Like its older sibling, [this wine] will be delicious in its first 34 years of
life.
(13) Un gran tinto de la Ribera, con casta y finura. [A great red from Ribera,
with breed and finesse]
(14) [Este vino] es una bestia encerrada de gran complejidad. Nariz pode-
rosa muy compleja. En boca es musculoso, tiene cuerpo, estructura, po-
tencia carnosidad todava es muy joven Posee raza y fuerza. [a
caged beast of great complexity powerful and very complex nose. In
mouth it is brawny, with body, structure, power, flesh it is still very
young has breed and strength]
(15) Nez expressif, dj bien ouvert Joli nez, assez complexe. Attaque
franche et volumineuse. Joli potentiel de vieillissement. [Expressive
nose, already well open Beautiful nose, quite complex. Straightfor-
ward and voluminous Beautiful potential for aging]
(16) Jambes colores. Trs beau nez Trs belle saveur fruite. ... Charnu.
Du corps. ... Trs beau vin. [Colored legs. Very beautiful nose Very
beautiful fruity flavor Fleshy. Full-bodied. Very beautiful wine]
(17) Rserv, complexe, ferme, charnu, compact et pre, dune longueur
ternelle, frache et race: un grand vin encore dans les limbes qui ne
dvoilera sa vraie nature que dans dix ans. [Reserved, complex, firm,
fleshy, compact and rough, incredibly long, fresh and thoroughbred: a
big wine still in a dream world which will reveal its true character in ten
years]
As pointed out earlier, one of my aims in this chapter is to show that, to-
gether with showing the heuristic role of figurative language in conveying
disparate sensory experiences, the passages seen so far also point to the mul-
timodal quality of wine critics commentary. This is the topic of the next
section.
2. Metaphors in winespeak: Monomodal or multimodal?
A core assumption in cognitive approaches to metaphor is that abstract
thinking is heavily determined by the functioning of the human body (Lakoff
and Johnson 1999). Simply put, our concrete experiences in and with the
world provide the basic data for understanding abstract, non-concrete con-
cepts. Nevertheless, although helping our understanding of the most abstract
via the most concrete is one of the most salient properties of metaphor, this
does not rule out the concreteness of both the source and target in certain
metaphors. A case in point is image metaphor, which involves the mapping
of concrete, topological information across disparate domains (for a detailed
discussion on this type of metaphor see Caballero 2006). Another conspicu-
ous albeit often neglected trait of metaphor is that it can be rendered in
modes of expression other than language (e.g. pictures, sounds, or gestures)
as well as involve various modalities or senses (e.g. vision, hearing, touch, or
taste). An extreme example of the latter is synesthetic metaphor, which
maps information across sensory domains or modes (Day 1996;
Ramachandran and Hubbard 2001; Yu 2003).
Both the concreteness and multimodal quality of some metaphors are im-
portant for discussing the figurative language used by critics in TNs. Thus,
the raison dtre of the genre is to put olfactory and gustatory experiences
into words. Yet, although smell and taste are two of the most basic and first-
hand i.e., concrete human experiences, the verbalization of the subtle
differences among the vast array of aromas and flavors at our disposal is no
easy matter. Figurative language compensates for the poverty of the lexicon
in this respect: in general, the expressions used in wine commentary denote
entities and qualities concrete enough to be of use in this difficult context.
Most expressions, however, defy Lakoff and Johnsons claims that human
beings find phenomena they can see, hear, feel, taste and/or smell easier to
understand and categorize than phenomena they cannot. It is perceptibility
that makes the former phenomena concrete, and the lack of it that makes the
latter abstract (1999: 249). In fact, very often the situation in winespeak
seems to be the reverse: wine critics need to draw upon experiences other
than smell and taste to categorize and verbalize these.
In turn, smell, taste, and language (itself expressible in various ways)
represent three different modalities or modes, which leads to a view of the
metaphors underlying winespeak as intrinsically multimodal rather than
conceptual in the sense the latter term is used in mainstream cognitive lin-
guistics. Take, for instance, terms like nose, body or mute, all of which have
been discussed as rendering a human view of wines personification being a
canonical example of conceptual metaphor in the literature. However, wine
critics use them to describe sensory experience: as has been noted, nose re-
fers to a wines aroma(s), body refers to the weight of a wine as sensed in
the palate, and mute or expressive have nothing to do with the articulatory
properties of the wine at issue, but mean that the elements making up a
wines bouquet and/or flavor are difficult or easy to perceive.
Of course, the description of wine as a living organism or, more specifi-
cally, as a human being in wine discourse is fully congruent with the
physiological properties of that beverage. Nevertheless, many other terms
apparently drawing from the human domain as well as expressions bor-
rowed from the textile or architectural domains are, in fact, used to express
the sensations provoked by wine in the drinkers nose and mouth i.e., in-
volve perception through the senses rather than abstract concepts or quali-
ties. This does not mean that some of the metaphors cannot through re-
peated use in textual interaction be further extended and, thus, conform to
the metaphorical frames discussed earlier. Yet, regarding such frames as
conceptually rather than cross-sensually concerned would betray the nature
of the experience metaphorically portrayed and communicated.
Indeed, the complexity and interest of the figurative expressions dis-
cussed so far arises from the fact that their underlying mappings seem to
involve different modes. Thus, whereas many of the targets of the metaphors
are smell(s), taste(s) and mouth-feel (i.e., physical, perceptual experiences
via sense organs), the sources concern both concrete (three-dimensional arte-
facts or the human body) and abstract (kinship) entities and domains an
idiosyncrasy worth investigating, yet which falls outside the general aims of
the present volume. Nevertheless, since both the source and target in the
figurative expressions are rendered verbally, hence in the same communica-
tive mode, these belong to the monomodal type of metaphor in Forcevilles
(2006) sense. Having said this, let us have a look at wine advertisements,
and assess whether these contain truly multimodal metaphors, that is, meta-
phors whose target and source are (primarily) rendered in different modes
(e.g., via images and language).
3. Imagery in wine advertising
Wine advertising is peculiar in a number of ways. In the first place, it is
fairly conservative in general, particularly when compared with commercials
and adverts promoting other consumer goods. For one thing, most wine ad-
verts show the product in a very straightforward (literal) manner so that
audiences cannot be mistaken as to the brand at issue the underlying as-
sumption being that the brand alone sells. Moreover, when a given winery
enjoys a high status among wine lovers, adverts usually consist of a close-up
of wine labels which suggests that their purpose is to remind readers of the
wines existence rather than persuading them to buy the product (i.e., raise
brand awareness and loyalty). A final conspicuous characteristic of wine
advertising is its heavy reliance on language: when elaborate or innovative
images are used, they appear to be but illustrations of what is expressed
verbally.
Metaphorwise the word-image combination in wine adverts is challenging
for several reasons. In many cases, images may be seen as figurative only
when their accompanying slogans are also taken into account. Pictorial
metaphors may also seriously contradict what is being communicated via the
verbal metaphors in the accompanying body text or, what is worse, show the
literal referent of the entity denoted by a given figurative expression or
term irrespective of its relevance and/or meaning in the wine realm (e.g.,
adverts exploiting anthropomorphic metaphors, as discussed in section 3.1.).
This, in turn, suggests that understanding the pictorial metaphors in wine
adverts is no easy matter, and nearly always requires acquaintance with wine
discourse in general and, of course, with the figurative language used by the
community of wine lovers.
As already pointed out, the approach to multimodal metaphor followed in
this chapter is the one provided by Forceville (2005, 2006, 2007). Two im-
portant steps in Forcevilles scheme are (a) to establish the domains involved
in the metaphors which, in turn, implies determining whether the meta-
phors favor certain source domains, and (b) to see which types of multimo-
dal metaphor are illustrated in the data under analysis. These questions are
addressed in the following sections.

3.1. Image-language combination
Given the fierce competition in the sector and the overuse of certain proce-
dures to enlarge production, most wine adverts are particularly concerned
with explaining the winemaking process followed by the winery so that their
products achieve the sought-after sense of terroir (i.e., how the wine exhibits
the characteristics of the vineyard site). Unsurprisingly, the adverts thus
oriented favor the anthropomorphic frame outlined earlier, playing up the
kinship and physiological aspects of the wine at issue in order to highlight its
pedigree or lineage. When the metaphor is used for this purpose, it is
mostly instantiated through language. A nice example of this is a Spanish
commercial designed for Navarra wines (one of the Spanish guarantees of
origin and quality of wines), where the audience is shown a middle-aged man
who appears to be talking to his departing son in the following terms:
Youre leaving tomorrow. The world awaits you and Im sure that when
they know you theyll love you. Were proud of you at home. We know that
youll become one among the great. Some wont appreciate your worth:
pay no attention. Mmm Others will see a winner in you: dont boast.
Dont be afraid of time: it will make you mature and, when you grow old,
do it with dignity. Always remember the land where you were born. Son,
Ive given you everything I know. Ill miss you. OK, enough. You have a
long trip tomorrow. NAVARRA REDS, SONS OF OUR LAND. [Trans-
lated from Spanish, RC] Bassat. Reproduced with permission.
Only after the last sentence we are shown the son thus addressed: a bottle
of wine resting on a table. The advert is true to its purpose, namely, advertis-
ing a wine-producing region rather than a particularly winery. Accordingly,
the bottle bears no label and the camera is solely concerned with the father,
who metaphorically stands for the region. Of course, the language used is
unmistakably figurative, yet we are aware of this only after we are shown
the bottle-son at the very end of the commercial this delay being a com-
mon advertising strategy that seeks to surprise audiences. The commercial is
a true example of multimodal metaphor in that the target (Rioja wines) is
rendered visually and the source (son) is rendered verbally.

Being the latest genera-
tion in a line of truly
great wines means that
you have evolved from
very solid roots. You
have also grown up shel-
tered by the knowledge
of one family. You are
loyal to one name only
Muga, whose tradi-
tion and character allow
you to grow even further
and develop new forms
of art, hence Torre
Muga. Torre Muga
comes from only the best
vintages and is pampered
throughout its traditional
winemaking process.
[] [translation from
Spanish by RC].
Figure 1. Print ad for the Spanish Torre Muga winery. WINES ARE PEOPLE
Personification is also exploited in the following three examples, all of which
promote specific brands and attempt to render a human view of wines via
language and images. Figure 1 shows several products of a well-known
Spanish winery, yet is particularly concerned with promoting the latest
product as explicitly acknowledged throughout the text namely, Torre
Muga. Following the aforementioned prototypical technique, the excellence
of this wine as well as that of its predecessors is warranted by describing the
procedures followed in their making.
The situation here is somewhat different from that of the advertisement
discussed above: although personification is mainly cued by language, we
could contend that the wines shown in the advert pose for the audience as in
a family portrait. Moreover, since the advert is particularly concerned with
promoting Torre Muga, this is foregrounded in the picture (i.e., it has been
moved to the front and appears larger than the other two).
6
However, the
question remains as to whether the image would be regarded as metaphorical
on its own i.e., without the accompanying text.
The role of language is also paramount in the following two adverts, both
exploiting personification as well, yet in subtly different ways.

With wines it is the other
way round. Only if you
have grown up/been raised
in a basement/cellar within
an oak cask, you are of
noble birth [translation
from Spanish by RC].

Figure 2. Advertisement for the Spanish Montecillo wine: WINE IS BABY.
The Spanish wine in figure 2 is portrayed as a human being by language
focusing on its birth, raising and noble origin. The image does, indeed, rein-
force the verbal metaphor: both the wicker recipient and the white cloth re-
semble a cradle, which helps see the bottle as a newborn baby.
5
However,
should we ignore the text, the image would not be seen as metaphorical but,
rather, as literally showing the way wines are usually presented at the table
in any restaurant.
Likewise, the following French advert plays with the idea of wines being
born. The slogan used here is more complex and playful. Thus, although the
literal translation is We show you our latest nose, the term nez may also
be understood as born should we read the slogan aloud (nez and n being
homophones in French).

Figure 3. Advertisement for Chatau Loville Poyferr
The image in figure 3 may be read in at least one of two ways. On the
one hand, it may be seen as a very theatrical way of introducing the wine to
an audience due to the low angle perspective of the picture: this shows a veil
going up just as curtains do in theatres the genres conventions allowing
for this dramatic introduction since nobody would do this with a wine. On
the other hand, if we use our imagination the veils wrapping the wine may
well be seen as resembling the delicate cloth that prototypically covers new-
born babies which would be commensurate with the playful use of nez/n.
However, although using the term nose to refer to the wine as a whole may
reinforce this human portrayal, the physiological, infantile traits suggested
by the second meaning get lost. Moreover, nothing in the image suggests that
it is the wines aromas that appear to be highlighted in the advert (the nose
of wines being, indeed, one of their most distinctive and praiseworthy traits)
even if this is precisely what the term nose means in winespeak.
Going back to the questions posed at the beginning of this section, the
adverts seen so far illustrate the penchant of advertisers for anthropomorphic
metaphors. They also show the usefulness of this frame to foreground traits
of both product and producer, particularly the aforementioned questions of
lineage, nobility, ageing etc. Finally, should we need to determine the type of
metaphor instantiated in the adverts following Forcevilles (1996) scheme
they would be close to what he calls verbo-pictorial metaphor, that is, a
metaphor whose source is visually represented and the target is verbally
represented or vice versa.
Nevertheless, the exploitation of anthropomorphism in wine advertising is
not always so successful despite its weight in wine commentary. This is
particularly conspicuous when adverts focus on a concrete sensory trait
of the wine at issue such as its flavor, bouquet, texture, length (i.e., after-
taste), etc., and combine both anatomically-biased expressions with images
of human body parts to do so. Adverts playing on the term body are very
revealing in this respect. This is the case in a Spanish campaign which ad-
vertises the same wine by showing either a naked male body or a female one
under a huge label reading Body.
7
This image-word combination is infe-
licitous for two reasons. In the first place, the only trait articulated by body
in winespeak is the weight of wines as perceived in the mouth all other
properties of human bodies being totally irrelevant in this particular context.
In other words, the tactile experience encapsulated in the jargon term body
(however unrelated to touch the term may seem to the layperson) gets lost in
the image, which can be interpreted in a number of ways. Moreover, the
indiscriminate use of male and female bodies to promote the same product
deviates from the way the adjectives masculine and feminine are used in
wine discourse since both are used to describe dramatically different wines.
Among the few exceptional cases where advertisers do try to convey the
sensory properties of wines via images I found the following one:

() They say that then it
started swirling, slowly first
releasing berry notes then
faster its pink freshness turn-
ing violet. And some say its
eyes were those of a woman
() [translation from Spanish
by RC].
Figure 4. Advertisement for Gran Feudo Chivite wine: SWIRLING WINE IS SWIRLING
BALLET DANCER.
In Figure 4 we also find language and image side by side; yet, in this par-
ticular case, the image does not need the help of language to be understood.
Thus, the advertisers have used well-known ballet dancer Tamara Rojo to
visually convey the swirling of a wine within a glass i.e., the necessary
first step to release its aromas and assess its nose. Moreover, swirling also
helps tasters grasp the first impression of wines texture (remember the
aforementioned use of legs or tears in this regard) before this is finally as-
sessed inside their mouths. In other words, the dancers swirling is used to
represent both the wines swirling and its texture. Given the characteristics
of the dancers dress (which, as ross, is pink), this texture appears to be
assessed as fresh, light, smooth and soft or supple. All such traits are char-
acteristic of ros wines like the one here advertised, draw upon tactile ex-
periences typically felt outside rather than inside the mouth (i.e., through
skin contact) and, therefore, may be seen as cases of synesthesia particular
to the wine domain, and usually co-occur with one of the most conventional
figurative terms used in texture assessment: silky. Of course, the image may
be interpreted as also illustrating the metaphor A WINE IS A BALLERINA, yet
all in all it is the releasing of aromas and the wines texture that are at stake
in the advert. As it is, equating a wine with a ballerina may, indeed, reinforce
the aforementioned traits of the wine at issue. In sum, the advert not only is
congruent with some of the figurative terms used by expert tasters in TNs,
but, according to an informal discussion held with some wine connoisseurs,
also appears to successfully convey them in visual form.
Together with illustrating some of the problems derived from attempting
to represent the metaphors in winespeak in visual form, the adverts shown so
far may be used to discuss the pertinence of culture in metaphor interpreta-
tion whether metaphor is represented verbally, visually, or both.

3.2. Culture and wine advertising
As Nedilko points out, the terminology of viticulture and winemaking is
marked by globalization and the interpenetration of different cultures, which
leads to a large number of borrowings and neologisms (2006: 138). This is
also the case with the figurative frames motivating a large number of the
lexical repository in the wine realm. Yet, however useful this uniformity
may be for discussing wine in some genres (e.g., the TN), this does not mean
that cultural idiosyncrasies do not also slip in verbal commentary or that
they cannot be exploited for particular purposes in other genres. Indeed,
images are good exponents of the weight of culture in both producing and
interpreting messages whether this is done through metaphors or otherwise.
Thus, the same image may mean totally different things to people immersed
in Western culture, a national culture within this broader frame, or a more
specific community built upon shared professional or ludic interests and,
more often than not, cutting across national differences.
This is also postulated by Forceville, who acknowledges that the old ad-
age that a picture tells more than a thousand words should not blind us to the
fact that pictures and other multimodal representations seldom communicate
automatically or self-evidently. As in verbal metaphors, it is connotations
rather than denotations of source domains that get mapped in metaphors, and
these may substantially differ from one (sub)cultural group to another
(2006: 389). Although I fully agree with Forcevilles views, I think that
knowledge of the target domain and, above all, of the conventions, beliefs,
needs, etc. of the community whose metaphorization of the world is scruti-
nized should not be underestimated either. Before taking this point further,
let us consider two other wine adverts.

Figure 5. WINES ARE JEANS Figure 6. Osborne arrives at Malpica.
The first advert (figure 5) belongs to a campaign promoting the wines of
Ironstone Vineyards (California, US). Here we have a comparison between
wines and jeans, a simile rendered in pictorial form, yet also cued through
language (just like our wine). Concerning the former, we have both source
(jeans) and target (wines) visually represented in the advert. This textile
portrayal is anchored by language: on the one hand, the adjectives soft,
easy, and comfortable are used to qualify both jeans and wines; on the
other, the verb try on reinforces the wines-jeans equation. As to the prop-
erties involved in the comparison, soft and comfortable point to texture
(i.e., tactile sensations), whereas easy suggests the casual, everyday, un-
complicated quality of both jeans and the wines thus qualified which en-
compasses more things than just sensory experience like, for instance, that
both can be worn/drunk with nearly everything and at any occasion. Indeed,
part of the success of this advert lies in the fact that the aforementioned ad-
jectives are frequently used in wine commentary focused on wines texture.
However uncontroversial the advert may seem at first sight, for people in
their middle age it may seem odd to qualify jeans as soft unless they are
really old and worn out. Of course, soft albeit new jeans are the result of
modern cloth manufacturing procedures such as stonewashing (which may
well be seen as a pun on the winery's name, Ironstone) as well as current
jeans fashion. However, for people in their forties, the connotations of jeans
whether visually or verbally cued cover anything but softness. Age and
fashion considerations apart, the advert could be interpreted as highlighting
the casual, young, uncomplicated, everyday qualities of the wines at issue
a view that fits the North American lifestyle also promoted by other con-
sumer goods such as cosmetics or clothes. Moreover, although softness is
usually a result of ageing, this advert tells us explicitly that Ironstone wines
are ready to be consumed i.e., do not need to be kept inside the bottle for a
while to acquire such a property.
However desirable and sought-after such qualities may be in modern
winemaking and in certain wine markets, the question remains as to whether
wine lovers from other countries would find the wines thus advertised worth
buying. Indeed, one of the assumptions in countries such as France or Spain
as well as the main feature of some famous vintages is, precisely, that great
wines stand out among others for their sense of terroir (which rules out the
connotations derived from adjectives such as easy and comfortable) or
the winemaking and ageing process used (which usually takes time). More-
over, softness may be a desirable quality in a wine (meaning that it is round,
fruity and lacks aggressive tannins) or, rather, be regarded as negative in the
sense that it indicates that the wine at issue provides little impact on the pal-
ate i.e., is somewhat watery.
The advert of a Spanish wine in figure 6 offers a dramatically different
picture, the slogan reading Osborne arrives at Malpica. At first sight,
there is nothing figurative here: what we have is a close up of a grapevine
and a slogan that tells us that a well-known Spanish brand (Osborne) has
been recently established in Malpica (a location in Spain). Yet, the advert
also includes the brands trademark, a bull, which was originally created to
advertise one of the most popular Spanish brandies (Veterano) in billboards,
and has become a distinctive landmark of the Spanish landscape as well as
a national identity symbol if we heed the numerous reproductions of Os-
bornes bull in the T-shirts, tea mugs, etc. sold to tourists. Finally, the sur-
name Osborne in Spain is loaded with connotations that bring to mind wines
as well as bull breeding which may be missed by those people not familiar
with Spanish culture.
In my view, the advert nicely illustrates a pictorial metonymy, that is, an
image that provides access to the entity bull via one of its most prototypi-
cal traits: its head and, more specifically, its horns. Moreover, since I am
acquainted with Spanish TNs, I could not help but interpret the metonymy as
cueing one of the best compliments a wine may be paid, namely, that of be-
ing thoroughbred. For whereas this English term is associated with horses or
dogs, its Spanish equivalent casta is prototypically used to qualify bulls.
This characteristic shows both in the animals behavior and in its appear-
ance, and horns are largely responsible for the latter. My specific claim here
is that any Spaniard could easily interpret the advert as equating wines with
bulls regardless of the presence of the trademark or the connotations verbally
cued by the term Osborne, yet would be seen as merely showing a grapevine
by many audiences outside Spain. A different issue is whether the trait
metonymically mapped (lineage, breed) would be understood by Spanish
people outside the wine realm.
Several related questions are relevant concerning this last point, and
are applicable to any advert. The first one is: would non-expert audiences
understand what is communicated in wine adverts at all? Of course, any
reader acquainted with advertising discourse will know that something posi-
tive is being promoted via language, images, or both. S/he will presumably
make use of her/his background knowledge to interpret the adverts knowl-
edge which may include different kinds of information and, most impor-
tantly, will be culturally biased.
In turn, bringing in cultural knowledge in advert and/or metaphor under-
standing also prompts two more questions: What do we mean when we use
the term culture? How do the different ways in which the term may be ap-
plied affect the understanding of metaphors in adverts? For if culture is un-
derstood as the shared beliefs, knowledge and world view(s) characterizing
national groups, the adverts may of course be understood, yet the interpreta-
tion of the images and, above all, of the metaphors thus rendered may very
well be more open given the heterogeneous nature of the audience (due to
their diverse backgrounds, and hence diverse concerns and expectations). In
contrast, a particular discourse community like the one articulated around
wine (be it composed of professionals or, simply, wine lovers) represents a
subculture within a much broader cultural frame, and is characterized by
specific knowledge schemas, needs and interests. It seems reasonable to ex-
pect, then, that the ways in which the adverts will be understood will differ
radically across various communities. For one thing, wine lovers and aficio-
nados bring in knowledge of the domain being metaphorized in the adverts,
as well as of the jargon used in other promotional genres like the tasting
note. They may also be acquainted with numerous brands from different
wine producing countries, and be aware of the connotations in, for instance,
a name such as Osborne. A different issue is whether the two adverts in fig-
ures 5 and 6 represent successful selling strategies given the possible preju-
dices towards wines being compared to jeans or to bulls according to na-
tional cultural with a capital C differences. The issue, however, asks for
research procedures that go beyond the purposes of this chapter (namely, a
user-centered approach), but is open for further research into how the pro-
duction and interpretation of the imagery used in the audiovisual genres are
affected by the specificity of the audience at which they are aimed.
The discussion in this chapter has been mainly concerned with the experien-
tial dimension of metaphor, that is, with how it is used by wine critics and
advertisers to articulate the sensory experience(s) afforded by wines in dif-
ferent media and communicative modes. Thus, I have provided a brief over-
view of the metaphors found in two promotional genres within wine dis-
course: TNs and wine adverts. Concerning the former genre, I have
explained wine critics abundant use of figurative language as responding to
their need to overcome the difficulties inherent in communicating the various
organoleptic experiences conflating in wine tasting. Concerning the latter
genre, the adverts here shown illustrate a penchant for anthropomorphic
metaphors, and largely fall into Forcevilles verbo-pictorial variety of mul-
timodal metaphor. Nevertheless, the foregoing discussion has also revealed
the problems derived from representing anthropomorphic metaphors in both
verbal and visual form, particularly those concerned with the sensory prop-
erties of wines (the exploitation of the term body being a case in point of the
contradictory, even wrong views often rendered by the image-language com-
bination). In this respect, however attractive the image-language combina-
tion may be to sell the product, it also poses many problems for the meta-
phor researcher since it not always captures the previously discussed cross-
sensory or multimodal dimension of most metaphors in winespeak. A point
also worth noting is the fact that many of the metaphors rendered in pictorial
form are actually the translation of metaphors acquired or learnt con-
sciously or unconsciously via such a cultural manifestation as language
rather than being embodied in the cognitive sense of the term.
A final aim of the chapter has been to draw attention to the role of culture
in understanding the metaphors used to advertise wine: both the broad na-
tional culture of the audience at which the adverts are aimed as well as the
specific culture articulated around wine. For, indeed, as Forceville (2006:
389) acknowledges, the connotations from the source domains in the meta-
phors are important in their interpretation, yet these also leave more room
for interpreting them. In my view, this endeavor also requires knowledge of
the target domain (in the present case, wine) as well as of the schemas under-
lying the particular worldview of the wine community.
Acknowledgments
I am grateful to Veronika Koller, Eduardo Urios-Aparisi and Charles Forceville
for their comments on earlier drafts, as well as to Ernesto Suarez-Toste (Universi-
dad de Castilla-La Mancha) for his insightful remarks on some of the issues ad-
dressed in this chapter as well as for providing the wine magazines used to build
the French corpus.
Notes
1. The project is currently being undertaken by Ernesto Suarez-Toste and
Rosario Caballero. The English examples belong to the corpus used in the re-
search and have been retrieved from the following sources: Decanter, Wine
Enthusiast, Wine Spectator, The Wine Advocate, Wine News, The Wine
Pages, and The Wine Anorak. The Spanish data have been extracted from
Sobremesa, and Gua Proensa de los Mejores Vinos de Espaa. French tast-
ing notes belong to La Revue du Vin de France, and Vinum.
2. This term is derived from toffe, meaning cloth, fabric.
3. The term charpent derives from charpent, which means skeleton,
frame, and framework and is used in French both to describe built arti-
facts and people. In this regard, the term is a nice example of the frequent
use of anatomical lexis in architecture and vice versa. This is also the case
with Spanish esqueleto (which can be translated as skeleton, framework
or, simply, structure). For a detailed account of the relationship between
both domains, see Caballero (2006).
4. Although the terms are used for descriptive purposes, and wine lovers defend
their neutral quality, the sexist connotations of the adjectives are often
pointed out by female scholars as I have personally experienced at various
conferences (see also Amoraritei 2002). Indeed, both terms are perfect exam-
ples of the intrusion of culture (and, in many cases, controversy) in the in-
terpretation of figurative expressions. Thus, in spite of whatever expectations
the readers cultural background may arouse, there is no evaluation inherent
in the masculine/feminine distinction: there is nothing per se good or bad in
being one or the other; if anything, it is undeniable that the current market
trends favor power over elegance (i.e., masculine over feminine), but this is
essentially circumstantial. The terms simply state that the drinkers natural
expectations from a specific wine will not be met because the wine is unusual
for its group in terms of mouth-feel and structure (for a detailed discussion
on how both terms are used by wine experts, see Caballero and Suarez-Toste
forthcoming).
5. Notice also that the winespeak term referring to the wine basket is cradle,
although, in this case, the underlying metaphor if any would involve
physical resemblance, in which case it would illustrate image metaphor
rather than any other type.
6. I am indebted to Raquel Segovia (who collaborated in the aforementioned
research project on wine metaphors) and Eduardo Urios-Aparisi for this in-
sight.
7. The adverts here discussed are not shown for copyright reasons.
References
Amoraritei, Loredana
2002 La mtaphore en Oenologie. Metaphorik.de 3: 112. Available from
www.metaphorik.de/03/amoraritei.htm (last accessed 23 March
2003).
Bruce, Nigel
2000 Classification and hierarchy in the discourse of wine: Emile
Peynauds The Taste of Wine. English for Special Purposes Journal
2326: 149164.
Caballero, Rosario
2006 Re-Viewing Space: Figurative Language in Architects Assessment
of Built Space. Berlin/New York: Mouton de Gruyter.
Caballero, Rosario, and Ernesto Suarez-Toste
forthc. A genre approach to imagery in winespeak: Issues and prospects. In
Researching and Applying Metaphor in the Real World, Alice
Deignan, Lynne Cameron, Graham D. Low, and Zazie Todd, (eds.).
Amsterdam/Philadelphia: John Benjamins.
Day, Sean
1996 Synaesthesia and synaesthetic metaphors. Psyche 2 (32). Available
from www.users.muohio.edu/daysa/types.htm. Last acccessed 7 Oc-
tober 2005.
Forceville, Charles
2005 Cognitive linguistics and multimodal metaphor. In Bildwissen-
2008 Pictorial and multimodal metaphor in commercials. In Go Figure!
New Directions in Advertising Rhetoric, Edward F. McQuarrie and
Barbara J. Phillips (eds), 178204. Armonk, NY: ME Sharpe.
Gluck, Malcolm
2003 Wine language. Useful idiom of idiot-speak? In New Media Lan-
guage, Jean Aitchison and Diana Lewis (eds.), 107115. Lon-
Lehrer, Adrienne
1975 Talking about wine. Language 51: 901923.
1983 Wine and Conversation. Bloomington: Indiana University Press.
1992 Wine Vocabulary and Wine Description. Verbatim: The Language
Quarterly 18: 1315.
2006 Wine and conversation: A new look. In Empirical, Cognitive-based
Studies in the Semantics-Pragmatics Interface, Jzsef Andor and
Pter Pelyvs (eds.). Oxford: CRISPI, Elsevier Science.
Nedilko, Anatoly
2006 Viticulture and winemaking terminology and terminography. Ter-
minology 12: 137164.
Peynaud, mile
1987 The Taste of Wine. Trans. Michael Schuster. London: Macdonald
Orbis.
Ramachandran, Vilayanur, and Edward Hubbard
2001 Synaesthesia A window into perception, thought and language.
Journal of Consciousness Studies 8: 334.
Yu, Ning
2001 Synesthetic metaphor: A cognitive perspective. Journal of Literary
Semantics 32: 1934.
Chapter 5
Interaction of multimodal metaphor and
metonymy in TV commercials: Four case studies
Abstract
This chapter analyzes four prototypical cases of interaction of multimodal meta-
phor and metonymy in television commercials. Three questions are addressed:
how multimodal metaphor and metonymy interact; how this interaction can con-
tribute to meaning creation in the commercial; and how multimodal metaphor and
metonymy interact in the cognitive and persuasive aspects of a multimodal genre.
A holistic analysis of these texts needs to bear in mind the diversity of modes and
submodes which participate in this genre as well as adopt a dynamic perspective
on the cognitive processes and their taxonymy, following Ruiz de Mendoza and
Dez Velasco (2002). The analysis shows how metaphor and metonymy interac-
tion hinges on cognitive as well as communicative roles and motivations.

Keywords: metaphor, metonymy, multimodal metaphor-metonymy interaction,
expansion, reduction, highlighting, advertising discourse, media, TV commercials
1. Introduction: television commercials in the advertising world
The presence and importance of multimodal metaphors for the theory of
cognitive metaphor and figurative language has been well attested by Force-
ville (2006/this volume, 2007, 2008) and by the chapters in this volume. In a
multimodal context, words, images and sounds can represent different do-
mains and establish mappings which result in metaphor-producing relation-
ships. As Radden (2002: 413) points out, those domain mappings can also
create metonymy-producing relationships. Only recently research on meton-
ymy has shown the important role it plays in cognition (Panther and Radden
1999, Barcelona 2000, Dirven and Prings 2002, Panther and Thornburg
2003). Nonetheless, except for Mittelberg and Waugh (this volume), the
references to metonymy in relation to multimodal metaphor have generally
96 Eduardo Urios-Aparisi
been made in passing, although multimodal metonymy has been deemed
worthy of consistent study (Forceville 2008: 298).
In the context of advertising, metonymy is an important cognitive process
and it not only reveal[s] rhetorical strategies (Forceville 2008: 298) but
also has an important role in motivating metaphor, highlighting its map-
pings, and consequently metonymy can define and represent reality and how
the product should be perceived by the audience. In this chapter, I address
the following questions: how multimodal metaphor and metonymy interact,
how this interaction can contribute to meaning creation in the commercial,
and how the relation between cognition and persuasion as multimodal meta-
phor and metonymy are embedded in the genre of advertising.
Except for Forceville (2007, 2008) and Yu (this volume), the study of
conceptual metaphor in advertising has focused mainly on printed advertis-
ing. Printed advertising has been approached from a cognitive and pragmatic
point of view (e.g., Tanaka 1994, Forceville 1996, Teng and Sun 2006),
from a rhetorical point of view (e.g., Scott and Batra 2003, Phillips and
McQuarrie 2002), and from a discourse analysis point of view (e.g., Cook
2001). The four cases analyzed in this paper are prototypical cases of meta-
phor-metonymy interaction within the multimodal context of television
commercials.
In a commercial, metaphor is an integrated experience of words, images,
sounds and meanings. As Kress and Van Leeuwen (1996: 17) suggest, of the
visual and verbal modes, each has its own possibilities and limitations of
meaning. A holistic study of meaning in a multimodal context requires that
particular modes of communication should be seen in their environment, in
the environment of all the other modes of communication which surround
them, and of their functions (Kress and van Leeuwen 1996: 33, italics in
original). It is, therefore, necessary to integrate the visual, verbal and audi-
tory modes and to theorize how they contribute to the overall meaning of the
commercial and the intention of the advertiser to represent the advertised
product in a positive light, so the spectator feels compelled to buy or use it
(cf. Forceville 1996).
In a commercial a metaphor is a primarily unidirectional act meant to de-
fine the product and its benefits for the consumer. In most cases, the claims
made about the product need to be seen as motivated and natural. The crea-
tive team uses cognitive resources in order to fulfill the claim about the
product, motivate the meaning by associating it to the real experience of the
spectator and to his conceptions of the world. In this sense, advertising
works very much like poetry, as Barthes (1988 [1964]) already argued. The
creative team uses the same resources as, for instance, a poet, in order to
Interaction of multimodal metaphor and metonymy 97
ground the metaphors in common experience and in the knowledge of the
consumer.
In the next section, I will discuss the theoretical framework of this chap-
ter. This discussion will lead to the microanalysis of four commercials that
were awarded prizes at the Festival Publicitario de San Sebastin, nowa-
days called Festival El Sol.
1
As mentioned before, television commercials
are particularly good examples of interaction between visual, verbal and
sound modes. The analysis of these multimodal commercials will involve the
identification of target and source, a discussion of how they draw on more
than one mode, and an analysis of the various types of metaphor and meton-
ymy interaction. These types follow and expand on the kinds of conceptual
interaction found in Ruiz de Mendoza (1999) and Ruiz de Mendoza and
Dez Velasco (2002). The analyses of these commercials show how all of
them aim to integrate the advertised product within what Sperber and Wilson
(1995) call the cognitive environment of the audience, using the modal
resources of the cinema and television so that its central message is always
somehow in praise of the product.
2. Multimodal metaphor and metonymy
A multimodal metaphor is a cognitive process in which two domains are
represented in two different modes. In the study of multimodal metaphor, the
surface representation should be taken into account in order to further the
understanding of the metaphorical mappings. As Forceville suggests,
clearly, which channel(s) of information (language, visuals, sound, and
gestures, among others) are chosen to convey a metaphor is a central factor
in how a metaphor is construed and interpreted (2007: 15).
We may be allowed to see either the target or the source, but either of
these can be merely suggested by any of a great variety of devices. For in-
stance, the target (often: the advertised product) can be conveyed by one of
its parts or by its logo or jingle, and the source can be explicitly represented
or implicitly inferred. In fact, research on advertising has found that making
claims about a product by means of indirect representations can create posi-
tive inferences and a more receptive attitude toward the brand by the audi-
ence (McQuarrie and Phillips 2005).
Jakobson (1971 [1956]) situated metonymy and metaphor as two poles of
cognitive processing: the metonymic pole accounts for contiguity relations
between linguistic elements, while the metaphoric pole is the result of simi-
larity relations between two domains (cf. Mittelberg and Waugh this volume
on gestures). The interaction of metonymy and metaphor shows that they are
not two opposite poles, but two parts of a continuum from literalness to
metaphor, as Radden (2002: 409) suggests.
Metonymy is understood here as an internal mapping of a subdomain
within the same experiential domain (Lakoff and Johnson 1980, Radden
2002; for the topic of multimodal metonymy per se, see Yu this volume).
While a metaphorical mapping bridges the distance between entities that are
experienced as belonging to two different domains, in metonymy a mapping
is connected to the mental highlighting or activation of one (sub)domain over
another (Barcelona 2002, Croft 1993). The target and source domains in a
metaphor establish symmetrical correspondences between different concepts
in a way that does not happen in metonymy. For instance, in the metaphor of
LOVE IS A JOURNEY the concepts in the target (lovers, love relationship, etc)
correspond to the concepts in the source (travelers, vehicle, etc) as Lakoff
(1993: 208) has shown. As Barcelona (2002) says, in metonymy this corre-
spondence is asymmetrical: the metonymic source projects its conceptual
structure onto that of a target, not by means of a systematic matching of
counterparts, but by conceptually foregrounding the source and by back-
grounding the target (cf. Barcelona 2002: 226, italics in original). For in-
stance, in case 4, the front of the car stands for the whole car or even the
whole car company. The foregrounding of the source (the car front: bumper,
spoiler with company logo, head and turning lights) highlights features of the
car which the advertiser intends to underscore. In contrast with metaphor,
which can be either referential or predicative (e.g., Warren 2006), metonymy
has been considered to have mostly a referential function (Lakoff and Turner
1989: 103). Other functions can be meaning extension (cf. Taylor 2002:
325) or pragmatic inferencing (cf. Panther and Thornburg 2003).
The distinctions between metaphor and metonymy are fuzzy. An instance
of how metonymy and metaphorical mappings can overlap is found in a
discussion in Forceville (1996). In a printed advert a beer bottle is pictured
in a wine cooler, thus expressing the metaphor BEER IS CHAMPAGNE. This
metaphor is developed from the metonymy which connects both target and
source to a single domain: [alcoholic] drinks.
2
Research on metonymy-
metaphor interaction has led to different typologies. Goossens (1990) was
the first to analyze their interaction in linguistic action expressions and cre-
ated the term metaphtonymy. This term included four types of combina-
tion in cases of meaning extension: metaphor from metonymy, metonymy
within metaphor, demetonymisation inside a metaphor and metaphor
within metonymy.
These combinatory entities may be useful for the explanation of meaning
extension in linguistic expressions, although these categories do not seem
adequate for the kind of combinations found in multimodal texts. In this
context, the categories proposed by Ruiz de Mendoza (1999) and Ruiz de
Mendoza and Dez Velasco (2002: 512) in terms of target and source seem
to be more suitable. They suggest a different view of metonymy which con-
siders that of the three general types of metonymies (PARS PRO TOTO, TO-
TUM PRO PARTE and PARS PRO PARTE, see above and Kvecses and Radden
1998 for a detailed analysis of the different metonymies) only the first two
are in fact metonymies. The names of the two types are TARGET-IN-SOURCE
as in She is taking the pill (pill for contraceptive pill) or SOURCE-IN-TARGET
as in All hands on deck (hands for sailors) (Mendoza and Dez Velasco
2002: 497). Two metonymies can also interact as, for instance, in the exam-
ple Wall Street is in panic: the location of the institution (Wall Street) for
the institution (New York Stock Exchange) (PLACE FOR INSTITUTION) and
the metonymy of the institution for the people who work in it (Ruiz de Men-
doza and Dez Velasco 2002: 512).
These authors also distinguish two processes: expansion and reduction.
These processes are present in five types of conceptual interaction. Meto-
nymic expansion of a metaphoric source can be exemplified in the case To
beat ones breast, where the basis of the source of the metaphor is a meton-
ymy. In this metonymy the source is a person beats his breast and the
target a person beats his breasts in order to show his sorrow about a situa-
tion. This is the source for the metaphor a certain person makes an open
show which may be a pretence in order to express his sorrow about a
certain situation (Ruiz de Mendoza and Dez Velasco 2002: 5201). In
metonymic expansion of a metaphoric target, as in to knit ones brows,
knitting is the source of a metaphor with a metonymy in the target: person
who frowns for person frowning because of anger. Metonymic reduction of
one of the correspondences of the target domain of a metaphor can be
found in to win someones heart where LOVE is the target domain of the
metaphor LOVE IS A PRIZE, and HEART FOR LOVE is the metonymic reduction
within the target domain. Metonymic expansion of one of the correspon-
dences of the target domain of a metaphor can be exemplified by to catch
someones ear. In the source: person catches an object, the object of the
source corresponds to ear in the target and this is the source for meta-
phorical mapping with attention. Finally, a metonymic expansion of one
of the correspondences of the source domain of a metaphor is found in to
bite the hand that feeds you, where hand is the source domain of the
metaphor HELPING IS FEEDING and stands for feeder (2002: 522527).
Meaning in the context of a commercial is not static but progresses either
by creating a short fiction or by layering diverse meanings over the adver-
tised product. In this regard, the interaction of metaphor and metonymy and
the processes of expansion and reduction between domains are similar to
those in the verbal mode. Visual perceptions and these perceptions may be
decoded by the same specialized mental module according to Yus (this
volume). But, as Stckl (2004: 14) says, submodes constitute a mode in
that they provide the building blocks of a modes grammar. The submode
discussed in this chapter is color, but other submodes could include the line
and shape, tone, color, movement and rhythm of the moving image (about
these components of the moving image see Block 2008). The creative team
of a commercial is generally aware of each mode and its submodes. In the
short length of a commercial, every single entitiy of the message is consid-
ered in order to elicit our attention and emotions by simulating various sig-
nificant features of our real-world visual experiences (Messaris 1997: 266).
3. Case studies
3.1 Metonymic reduction of metaphorical correspondences between
domains: New model of car (1986)
In this commercial, the metaphor is explicitly represented in the images: CAR
IS A LONG-JUMP ATHLETE, a variant of the metaphor CAR (MACHINE) IS A
PERSON. As we can see in figure 1, the visual montage shows the long-jump
athlete and the car in cross-cut shots. This metaphor is a clear case of per-
sonification. Within this metaphorical conceptualization of cars, the meta-
phor is represented clearly with an ingenuous use of the montage. This ex-
plicit representation of the metaphor is clearly different from the implicit
meaning of case 4, which I will argue is also a case of personification. None-
theless, this case is a particularly good instance of the use of metonymy to
highlight the features that are being mapped between the two domains.
As we can see from the selected images of this commercial, the source is
shown first at a distance, while the target is only seen in parts until both are
on the racing track ready to do the long-jump. The cross-cutting of images
enhances the identification of the target with its source including mappings
of the athletes running shoes and legs, which highlight other features of the
car such as its sporty properties. The property that is metonymically cued in


Athlete Car Words
1

2 3

4 5

6 7

8 9
Ahora ms
versiones
Now more
versions
10 11
Renault 11
Figure 1. Fasa Renault 1986 01 Salto Renault 11 Salto Jump.
both domains is the power of the cars engine (and metonymically the whole
car), and the power of the athlete who can jump very far. Just before the
jump, the car appears completely and the male voice-over says: Now more
versions The verbal mode explicates the target of the commercial: the
new properties and more powerful motor of a car model.
The attributes by which both domains are cued constrain the metaphori-
cal mappings by highlighting those features which are relevant for the target
domain and the intention of the advertiser. Although both source and target
of the metaphor are identified by the montage of images of the athlete and
the car, the voice-over further anchors the commercials message by clarify-
ing the target domain of the metaphor.
The montage of the images of the athlete and the car identify both target
and source of the metaphor, and verbal anchorage further clarifies the target
of the metaphor. The potential mappings in the metaphor CAR IS PERSON are
limited to the ones the advertiser is interested in activating by creating meto-
nymical visual correspondences between the domains and by the verbal mo-
dality naming those conceptual features.

3.2 Metonymic reduction in target and motivation of metaphor: Tea drink
(1987)

Figure 2. Hipnosis (hypnosis) (1987). Product by CPC Espaa. Advertising
company: J. Walter and Thompson.
This commercial creates the metaphor a TEA TAG IS A HYPNOTIZERS WATCH
metaphor in order to transfer the powers of hypnosis to the relaxing impact
of a brand of tea. In this metaphor the target takes on the role of the hypno-
tizers watch and the movement of the tea tag is one of the cues to the source
domain of the metaphor. The representation of the tea follows two metony-
mies: EMBLEM FOR PRODUCT (Tea Tag for Tea Bag), and EFFECT FOR
CAUSE (Relaxation for Drinking Tea), summarized in Table 1.
The first metonymy highlights the products tag as it can resemble a hyp-
notizers watch and allows for the metaphor TEA TAG IS A HYPNOTIZERS
WATCH to be enacted. The second metonymy EFFECT FOR CAUSE (Relaxa-
tion for Drinking Tea) motivates this metaphor. As hypnosis is associated
with falling into a trance-like state, the commercial enacts the quintessential
feature of tea by identifying and making the tea tag act as a hypnotizers
watch.
Table 1. Metonymies to represent TEA.
Metonymy type Metonymy in commercial Explanation
EMBLEM FOR PRODUCT Tea Tag for Tea Visual representation of
the product
EFFECT FOR CAUSE Relaxation for Drinking
Tea
Highlighted consequence
of drinking tea

Table 2. Modes in metonymy and metaphor for case 2.
Figure Components Visual Words Sound
TARGET Tea drink Meton-
ymy
SOURCE Tea tag hanging
from tea cup
The logo of
the product

TARGET Tea Tag
Hypnotizers
watch
Words a hyp-
notizer says
to a patient
Hushed
voice of the
hypnotizer
Submode 1:
movement of the
watch
The tic-toc
of the watch
Metaphor
SOURCE
Submode 2: Soft
colors and low
modality.
SOFT COLOR IS
WARMTH

The visual mode could be divided into several submodes such as color or
movement. As mentioned above, the submodes are building blocks of each
mode (cf. Stckl 2004: 14). The movement of the tea tag acts out the way a
hypnotizers watch moves. The soft colors in a gold and brown hue and the
slightly out of focus image represent the view of the person who is falling
under the spell of the watch. The hue and colors are stereotypically warm
and thus represent the feeling of wellness that the product is supposed to
give and associate to the metaphor SOFT COLOR IS WARMTH and, thus, to the
metaphor AFFECTION IS WARMTH which can be found in the context of emo-
tional relationships (cf. Kvecses 2000: 93) and, if that is the case, the color
submode would integrate experiences of friendship and emotion with the
product. The visual mode also activates the sound mode with the speakers
hushed tonality which is similar to the color and hues of the image. The
modes and their association with different components in the metonymy and
metaphor in this commercial are listed in table 2.
In conclusion, the metaphor is elaborated through a complex interaction
of modal techniques. Each mode and submode associates the product with
the main metaphor TEA TAG IS A HYPNOTIZERS WATCH and relates it to
sensations of warmth, relaxation and wellness.

3.3 Metonymy in source and expansion by implicit metaphorical mappings

This commercials point is not to advertise any new model, but it is a public
announcement and praise for the safety fixtures and reliability of all the cars
of that brand. Therefore, it is a kind of corporate advertising with a public
service tone, most likely aired around a time of high traffic and traveling, as
the voice-over clearly implies: En estos das mucha gente saldr a la carret-
era (These days a lot of people will go on the highway). The com-
mercials design is very simple: the camera focuses on the front grill of a
Volvo car from the 240 series which was developed in the 1970s.
3

Figure 3. Respuesta (answer) (1987). Product advertised: Volvo car. Advertis-
ing company: CID.
It can be considered an example of metonymy in advertising since all the
mapped features are metonymically motivated, as we will see. The square
and bulky shape of the front with its metallic look, together with the bumpers
and large headlights, is a visual metonymy: CAR FRONT FOR CAR. The front
of the car is a complex image. In it we can see the frame of the car with its
bulky design, the big bumpers and lights, the license plate with the word
Volvo in blue capitals with a white background (the official typeface of the
brand; Egyptian according to http://www.volvoclub.org.uk /history/volvo_
logo.shtml) and the logo which is located on top of the radiator. These two
elements are symbolic metonymies of the product and they have metonymi-
cal (LOGO FOR PRODUCT) as well as other meanings (on logos as multimodal
metaphors, see Koller this volume).
4
The image represents the identity of the
corporation in three aspects: the product, the corporate signs and symbols,
and the main features they want to associate themselves with: reliability,
strength and safety.
Table 3. Metonymies and metaphor in Case 3
Figure Components Visual Words Sound
TARGET Car Metonymy 1
PART FOR
WHOLE
SOURCE Front of the car
Front of the car
with logo and
official typeface

TARGET Cars reliability
and safety
Metonymy 2
PART FOR
WHOLE
SOURCE Front of the car
Changing
weather condi-
tions
Changing
weather
conditions
and sounds
associated
to moving
car
TARGET Car Metaphor
SOURCE Person:
Advice to a
driver by the
voice-over

TARGET Company Metonymy 3
PART FOR
WHOLE
SOURCE Intelligent car,
Logo and official
typeface

After the first image of the car, the camera pulls back and the front of the
car moves through all kinds of weather and driving conditions smoothly and
surely, and this driving is accompanied by metonymical sounds and images
which can be associated with these driving conditions and to passengers and
other persons: the sound of children getting in the car, the door closing, the
noise of the car engine, the rain, thunder and snow (also visible in the im-
ages), the turn signal noise and image, the noise and the image of a ball sud-
denly bouncing in front of the car, the car horn and brakes.
On the one hand, the images are metonymies of the car as a whole, per-
haps referentially highlighting the cars strength, sturdiness and immutability
towards the changes in the road conditions. Also, as mentioned above, the
color submodality would also contribute to these meanings. On the other
hand, the language used does not refer to the product at all. The male voice-
over addresses the consumer directly by giving advice about how to drive
when there is high traffic during vacation time (the images suggest that it is
winter). The words insist that the driver be sensitive, drive safely and pru-
dently, and focus on the importance of the family (referenced metonymically
as carga or load) and on arriving safely (which implies that it is better to
arrive safely than fast).
As we can observe, this commercial features good examples of meto-
nymic references in various modes. Metonymy, in this case, is mostly refer-
ential with respect to the car, its passengers, and the weather and driving
conditions, but this metonymy can create further implicit meanings by the
audiences knowledge of car-safety and brands. Besides these components,
there is one obvious absence in the visual representations: that of the driver
who is directly addressed in the words. Whereas the commercial features
various metonymies, the voice-overs address predicates of the car some of
the properties which generally are associated with a person, the driver of the
car: sensitivity and prudence. Therefore, as part of the creation of the corpo-
rate image, the car seems to be personified (CAR IS PERSON). A personifica-
tion is a kind of ontological metaphor in which the target is understood in
human terms (Lakoff and Johnson 1980). It can be explained along the lines
of Lakoff and Turners (1989: 195) discussion of metaphors like PERSONS
ARE ANIMALS (Achilles is a lion) within the cultural model of the GREAT
CHAIN METAPHOR. According to this metaphor, attributes and behaviors are
associated with animate creatures within a hierarchical scale: the Great
Chain is a scale of forms of being human, animal, plant, inanimate object
and consequently a scale of the properties that characterize forms of being
reason, instinctual behavior, biological function, physical attributes, and so
on (Lakoff and Turner 1989: 167). In this commercial, we understand the
cars behavior as it goes through different driving conditions in terms of
some features that can only be associated with a human beings behavior
(prudence, sensitivity and steadfastness). This mapping is complex since it is
not only between the properties of the source and the target, but also be-
tween the relationships of those properties to their domains. Therefore, hu-
man sensitivity and prudence is intelligent and high-order and so car behav-
ior is also intelligent and high-order.
4
The metaphor of an intelligent car (CAR IS PERSON) is an extension of a
complex metonymy: the front part of the car by juxtaposition to the voice-
over which addressees the audience, while at the same time the front of the
car itself is shown and probably credited with the behaviors described by the
words. The commercial as a whole is metonymically representing the brand,
bestowing on it the features associated with the car.

3.4 Metaphorical expansion of metonymy in the source domain
Commercial 4 advises women to do family planning instead of resorting to
abortion. This commercial is another instance of extension of a metonymy
but in this case in the source domain. The first visual metaphor of this com-
mercial is MAKING A DECISION IS PULLING A THREAD. The camera focuses
on a yellow bootie situated over a soft white light against a diffused dark
background. Shortly after the male voice-over starts speaking, we see two
fingers starting to pull a thread of the bootie and the words explain how
women sometimes have to make a decision, and how this decision is hard
and sometimes traumatic (see the complete text in figure 4).
The first image is easily identifiable with a baby by the metonymy BABY
BOOTIE STANDS FOR BABY, a variant of the metonymy GARMENT FOR PER-
SON. The bootie is the source of the metonymy and also part of the source of
the metaphor that is developed as the hand pulls the thread. Another meta-
phor is more specifically identified by the words and the image: ABORTING IS
UKNITTING BABY BOOTIE. Undoing the baby bootie is at the same time the
act of making a decision, and since this object can be identified metonymi-
cally with the baby, its unraveling is the result of a decision which is consid-
ered difficult or traumatic.
During the commercial, abortion is never mentioned, but it is metonymi-
cally implied by the decision to abort and the experience of going
through the abortion (both PART FOR WHOLE metonymies). These words are
diagrammatically represented in the images. As difcil decisin or siem-
pre dura and traumtica (respectively, shots 3, 5 and 7) are uttered, the
perspective on the bootie changes and the camera shows a close-up of the
thread of the bootie slowing down. The source has been identified by the
visual mode while the target is in the verbal mode throughout the commercial
until the final sentence: no vivas pendiente de un hilo (dont live hanging
by a thread) which is made literal in the image by showing the last thread of
the bootie. The final expression is a conventional metaphor in Spanish and
returns to some extent to the first metaphor: MAKING A DECISION IS PULLING
A THREAD. The commercial leaves the opportunity to reach other conclu-
sions to the audience.

1. Muchas mujeres
Many women
2. se han visto obligadas a tomar
have been obliged to make
3. una difcil decisin
a difficult decision

4. 5. Una decisin siempre dura
A decision, always hard
6. y en ocasiones
and sometimes

7. traumtica.

traumatic
8. Evita esta experiencia

Avoid that experience
9. No vivas pendiente
de un hilo
Dont hang by a thread

10. Planifcate
Make plans

Figure 4. Patuco (baby bootie) (1988) Ministerio de Sanidad (Health Depart-
ment, Spanish Government). Advertising company: Vitruvio.
In table 4, I have summarized the metonymy and the metaphors of this com-
mercial. It is an emotionally charged commercial. The bootie, out of all the
possible pieces of baby clothing, is readily associated with a baby (older
children do not wear booties); also, it shows with more immediacy the shape
of the body, and, finally, it has immediate connections to life and movement:
crawling and walking. The metonymy motivates the metaphor, but also maps
other meanings to the target of the metonymy (the baby). This process to-
gether with the music (the so called Brahms Lullaby) creates another emo-
tional layer to the commercial by reliving the decision making process in the
enactment of undoing a bootie.
Table 4. Metonymy and metaphor in case 4.
Figure Components Visual Word
TARGET BABY Metonymy
GARMENT FOR PERSON SOURCE: BOOTIE
TARGET MAKING A DECISION Metaphor 1
SOURCE UNDOING A
BOOTIE

TARGET ABORTING: meto-
nymically implied
by the words (PART
FOR WHOLE)
Metaphor 2
SOURCE UNKITTING A
BABY-BOOTIE

TARGET TO LIVE OR TO BE IN
DANGER IS
Metaphor 3
No vivas pendiente de
un hilo
Dont live hanging by
a thread (extension of
Metaphor 1)
SOURCE Last thread of
the baby boo-
tie [visual
representation]
TO HANG BY A
THREAD
4. Conclusions
In this analysis I have identified the following metaphor-metonymy interac-
tions:
a) Metonymic reduction limits the possible correspondences between do-
mains by highlighting the features of both domains that are to be matched
metaphorically (case 1). In this case, the metaphor CAR IS PERSON is deter-
mined by the visual identification of the correspondences and thus the com-
mercial takes advantage of the persuasive potentials of a variant of this
metaphor: CAR IS ATHLETE.
b) Metonymic reduction in metaphorical target visualizes the product and
highlights the feature that will motivate the metaphor (case 2).
c) Source-in-target metonymy represents the product and the company
and it is also the target of the verbally expressed source domain (in case 3).
The metonymy highlights the relevant part and lends it to further mappings
because of its visual presence throughout the commercial and because of
background knowledge about this particular car and brand.
d) Source-in-target visual metonymy (GARMENT FOR PERSON) is ex-
panded with a series of metaphors. Each metaphor builds on the previous
one and relates to the first metonymy.
Obviously, this is not a comprehensive list and other interactions are pos-
sible. Perhaps other cases can identify the same kinds of interactions in the
source domain as Ruiz de Mendoza and Dez Velasco (2002) found in their
linguistic corpus. In these cases, metonymy has mostly a limiting role, as it
identifies the target of metaphor or the correspondences between both do-
mains, while metaphor expands this identification with other mappings. In
case 3, the metonymy taps into the background knowledge to suggest further
interpretations. To some extent, it is not limiting or exclusively referential
but creates further correspondences.
In the dynamics of a television commercial, metonymical mappings, on
the one hand, do not only substitute for or represent the product, but they
can link the product to domains which can be relevant for the products
promotion: expanding or constraining the interpretation of metaphorical
mappings (see El Refaie this volume for other uses of a metonymy as an-
choring). Metaphor, on the other hand, can also make the metonymy pro-
gress towards mappings that go beyond the presence of the product and try
to convey additional meanings such as emotional representation and poetic
effects in the cognitive environment of the audience, as mentioned earlier.
The different modes and submodes can contribute to the creation of a
pausible representation of reality and can also be associated to other meta-
phors. For instance, as mentioned in case 2, the hue and colors are warm and
soft. Those colors can represent the metaphor SOFT COLOR IS WARMTH and
further, it can connect to the metaphor AFFECTION IS WARMTH. This is
clearly contrasted in case 3. In this car commercial, mainly in blue and grey
the colors can be associated not only with the difficult driving conditions but
also with the strength and endurance of iron or steel. While in case 4, the
choice of yellow in the color of the baby bootie intentionally avoids colors
like blue or pink, which are gender-specific. The diffused light of the back-
ground highlights the baby bootie and its dramatic undoing. In the case of
metaphor, they include meanings that aim at supporting the visual consis-
tency and the identification with those features which are stereotypical of the
domain.
In brief, I would conclude:
1. Television commercials are dynamic texts in which all modes can con-
tribute to multimodal metaphors either in the source domain or the target
domain.
2. In order to understand how metaphor creates meaning, metaphor needs
to be studied within its embeddedness in the context of the commercial and
the persuasive functions of advertising.
3. The interactions of metaphor and metonymy show that layering of rhe-
torical figures is not random, but follows clear cognitive patterns which re-
strict and define their design and persuasion. As seen in case 2, the meton-
ymy has a double function: represent the target for the metaphor in a way
that can be realistic for the metaphorical representation, and motivate the
message of the commercial. Once metonymical correspondences are mapped,
the commercial can create additional metaphorical mappings. Also, meton-
ymy can identify those entities which are to be transferred from the two do-
mains, as in case 1.
4. As shown, a metaphor expands the meaning by associating new do-
mains with the original metaphor or metonymy. It creates further imagery
that can trigger more emotional or intellectual associations with the product.
5. The grounding of the meaning of a commercial in the viewers knowl-
edge and experience can be accomplished by various means. One of them is
metonymy, and this is consonant with general views on this figure. Meton-
ymy is considered closer to literalness in the literalness-metaphor continuum
(Radden 2002 and Dirven 2002), and is frequently used in realist art (Jakob-
son 1971 [1956]). In narrative it can highlight conventional belief, structure
episode development and, thus, help interpretation (cf. Pankhurst 1997). In
the commercials, the metonymy activates or highlights an aspect of the real-
ity of the product. This feature can be recognized by the audience most eas-
ily or can be productive to provoke implicit positive meanings. As shown, it
also can constrain the amount of possible correspondences in a metaphorical
mapping.
6. The modes in a commercial are structured around a clearly defined
target, and the need to persuade or represent a product in a positive light.
The kind of products which are advertised, and even some elements not ad-
dressed here such as the length of a commercial, all contribute to the need
for interaction of metaphors and metonymies. A comprehensive analysis of
multimodal metaphors needs to identify the genre which shapes them and
steers their interpretation (Forceville this volume). The importance of con-
necting discourse and conceptual metaphor has been shown by Cameron
(2003) in education discourse, Urios-Aparisi (2004) in interaction, Caballero
(2006) in architecture discourse, and Caballero (this volume) in winespeak.
Further research is needed to account for the roles of submodes and how
they contribute to the domain construction mainly in visual modes. As we
have seen, their contribution to the source domains in 2 and 3 is relevant to
the metaphors they form, and in 4 it contributes to the dramatic staging of
the commercial, but to what extent can they create other metaphorical map-
pings?
As we have seen, three of the four cases are personifications. This is
probably no coincidence. Objects, animals and other experiences are best
understood as representations or extensions of human beings. The experience
of objects and vehicles as animated human beings reflect the natural applica-
tion of the cognitive model GREAT CHAIN METAPHOR and also the anthropo-
centric dimension of this cognitive model (cf. Dirven, Polzenhagen, and Wolf
2007: 1228 for this metaphor and the language of oppression). Underlying
cognitive structures are present in the use of the camera, montage, color and
other (sub)modalities which can be construed by the audiences cognitive
resources. The inextricable relation between metaphorical and metonymical
mappings determines how meaning is created. The analysis of these com-
mercials shows how metonymy engages the products representation to the
cognitive environment of the audience as it is conceived by the advertiser.
Such criteria underlie the choices made by the creators of these commercials
regarding the modes and the entities which are used to represent the product.
Acknowledgments
I would like to thank Charles Forceville, Brian Patrick, John Bardem and Cristin
Siebert for their insightful comments on, and thorough revisions of, earlier ver-
sions of this chapter.
Notes
1.

The commercials can be viewed online at the Instituto Cervantes in the
Museo Virtual de Arte Publicitario (Muvap): http://cvc.cervantes.es/actcult/
muvap/in sala V: Creatividad publicitaria audiovisual.
2. A similar commercial but with a juice drink is analyzed by Phillips and
McQuarrie (2004). Both printed ads highlight how beer or juice drinks differ
from wine in their production and suggest that both beer and juice drinks ac-
quire the status of wine when they are produced in a cellar.
3. According to http://www.swedecar.com/volvo_history.htm in 1972: The US
traffic safety administration (NHTSA) purchased a number of Volvo 240s,
which were used to set the safety standards against which all new cars on the
US market were tested. The importance of the safety image is still para-
mount in Volvos marketing. In their webpage section Experience Volvo,
the section on safety is first and within this section they have another special
on how Volvo saved my life, (see http://www.volvocars.us/experience/
safety.htm).
4. The logo is an adaptation of the symbol for iron: a circle with an arrow point-
ing upwards towards the right (cf. http://www.volvoclub.org.uk/history/
volvo_logo. shtml).
5. Further analysis would pertain to the slogan of this commercial Respuesta
segura (sure answer). This slogan is a conventional metaphorical meaning
of the word respuesta as the effect of an action is the response to a ques-
tion: the effect of an action is a verbal response. As part of the personification
of the car, the slogan maps the features of talking and reacting with intelli-
gence associated with humans.
References
Barcelona, Antonio
2000 Introduction: The cognitive theory of metaphor and metonymy. In
Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
tive, Antonio Barcelona (ed.), 128. Berlin/New York: Mouton de
Gruyter.
2002 Clarifying and applying the notions of metaphor and metonymy-
within cognitive linguistics: An update. In Metaphor and Metonymy
in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.),
Barthes, Roland
1988 [1964] The Semiotic Challenge. New York: Hill and Wang.
Block, Bruce
2008 The Visual Story. 2nd edition. Amsterdam, etc.: Focal Press.
Caballero, Rosario
2006 Re-viewing Space: Figurative Language in Architects Assessment
of Built Space. Berlin/New York: Mouton de Gruyter.
this vol. Cutting across the senses: Imagery in winespeak and audiovisual
promotion.
Cameron, Lynne
2003 Metaphors in Educational Discourse. London/New York: Contin-
uum.
Cook, Guy
2001 The Discourse of Advertising. London/New York: Routledge.
Croft, William
1993 The role of domains in the interpretation of metaphors and me-
tonymies. Cognitive Linguistics 4: 335370.
Dirven, Ren
2002 Metonymy and metaphor: Different mental strategies of conceptu-
alization. In Metaphor and Metonymy in Comparison and Contrast,
Ren Dirven and Ralph Prings (eds), 75112 Berlin/New York:
Mouton de Gruyter.
Dirven, Ren, and Ralph Prings (eds.)
Dirven, Ren, Frank Polzenhagen, and Hans-Georg Wolf
2007 Cognitive Linguistics, Ideology and Critical Discourse Analysis. In
The Oxford Handbook of Cognitive Linguistics, Dirk Geeraerts and
Hubert Cuyckens (eds.), 12221240. Oxford: Oxford University
Press.
this vol. Metaphor in political cartoons: Exploring audience responses.
Forceville, Charles
2002 The identification of target and source in pictorial metaphors. Jour-
nal of Pragmatics 34: 114.
work: Agendas for research. In Current Applications and Future
Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and
Francisco Ruiz de Mendoza (eds.), 379402. Berlin/New York:
Mouton de Gruyter.
of Semiotics 1: 1951, available at http://semiotics.ca/.
Barbara J. Phillips (eds.), 272310. Armonk, NY: ME Sharpe.
this vol. The role of non-verbal sound and music in multimodal metaphor.
Goossens, Louis
1990 Metaphtonymy: The interaction of metaphor and metonymy in figu-
rative expressions for linguistic action. In By Word of Mouth: Meta-
phor, Metonymy and Linguistic Action in a Cognitive Perspective,
Louis Goosens, Paul Pauwels, Brygida Rudzka-Ostyn, Anne-Marie
Simon-Vandenbergen, and Johan Vanparys (eds.), 159174. Am-
sterdam/Philadelphia: Benjamins.
Jakobson, Roman
1971 [1956] Two aspects of language and two types of aphasic disturbances, in
Selected Writings II: Word and Language. The Hague: Mouton.
Koller, Veronika
this vol. Brand images: Multimodal metaphor in corporate branding mes-
sages.
Kvecses, Zoltan
2000 Metaphor and Emotion: Language, Culture and Body in Human
2005 Metaphor in Culture: Universality and Variation. Cambridge:
Kvecses, Zoltan, and Gnter Radden
1998 Metonymy: Developing a cognitive linguistic view. Cognitive Lin-
guistics 9: 3777.
1996 Reading Images: The Grammar of Visual Design. London/New
York: Routledge.
Lakoff, George
1987 Women, Fire, and Dangerous Things: What Categories Reveal
about the Mind. Chicago: University of Chicago Press.
1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chi-
Messaris, Paul
1997 Visual Persuasion: The Role of Images in Advertising. Thousand
Oaks, CA/London/New Delhi: SAGE.
McQuarrie, Edward F., and Barbara J. Phillips
2005 Indirect persuasion in advertising: How consumers process meta-
phors in pictures and words. Journal of Advertising 34: 721.
Mittelberg, Irene, and Linda Waugh
this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture
Pankhurst, Anne
1997 Interpreting unknown worlds: Functions of metonymic conceptuali-
zation in William Goldings The Sea Trilogy. Language and Litera-
ture 6: 121131.
Panther, Klaus-Uwe, and Gnther Radden
1999 Metonymy in Language and Thought. Amsterdam/Philadelphia:
Benjamins.
Panther, Klaus-Uwe, and Linda L. Thornburg, (eds.)
2003 Metonymy and Pragmatic Inferencing. Amsterdam/Philadelphia:
Benjamins.
2002 The development, change, and transformation of rhetorical style in
magazine advertisements 19541999. Journal of Advertising 31 (4):
113.
2004 Beyond visual metaphor: A new typology of visual rhetoric in adver-
tising. Marketing Theory 4 (12): 113136.
Radden, Gnther
2000 How metonymic are metaphors? In Metaphor and Metonymy at the
Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 93
Ruiz de Mendoza, Francisco Jos
1999 Introduccin a la teora cognitiva de la metonimia. Granada:
Mtodo ediciones.
2000 The role of mappings and domains in understanding metonymy. In
tive, Antonio Barcelona (ed.), 109132. Berlin/New York: Mouton
de Gruyter.
Ruiz de Mendoza, Francisco Jos, and Olga Isabel Dez Velasco
Comparison and Contrast, Ren Dirven and Ralph Prings (eds),
1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Black-
well.
Scott, Linda, and Rajeev Batra (eds.)
2003 Persuasive Imagery: A Consumer Response Perspective Mahwah,
NJ/London: Lawrence Erlbaum.
Stckl, Hartmut
2004 In between modes. In Perspectives on Multimodality, Elija Ventola,
Cassily Charles, and Martin Kaltenbacher (eds.), 930. Amsterdam/
Tanaka, Keiko
1994 Advertising Language: A Pragmatic Approach to Advertisements in
Britain and Japan. London/New York: Routledge.
Taylor, John
2002 Category extension by metonymy and metaphor. In Metaphor and
Metonymy in Comparison and Contrast, Ren Dirven and Ralph
Urios-Aparisi, Eduardo
2004 Quarrelling about metaphors on love: A pragmatic approach. In
Current Trends in the Pragmatics of Spanish, Maria Elena Placen-
cia and Rosina Mrquez Reiter (eds.), 283310. Amsterdam/ Phila-
delphia: Benjamins.
Warren, Beatrice
2006 Referential Metonymy. Lund: Almqvist and Wiksell.
Yu, Ning
this vol. Nonverbal and multimodal manifestations of metaphors and me-
tonymies: A case study.
Yus, Francisco
this vol. Visual metaphor versus verbal metaphor: A unified account.
Chapter 6
Nonverbal and multimodal manifestations of
metaphors and metonymies: A case study
Ning Yu
Abstract
This paper intends to analyze, within the cognitive linguistic paradigm, the non-
verbal and multimodal manifestations of metaphors and metonymies in an educa-
tional commercial screened on China Central Television (CCTV). Specifically, it
shows how two major conceptual metaphors, LIFE IS A JOURNEY and LIFE IS A
STAGE, are manifested in dynamic visual and aural, as well as verbal, discourse.
The various visual, aural and verbal elements are interactive with and dependent
upon each other when they combine into a conceptual blend with input spaces
in visual, aural and verbal modes. This blend contains conspicuous juxtapositions
of various kinds, simultaneous or sequential, which cast in relief the unity and
contrast between the Chinese and the Western, between thought and action, be-
tween primitivity and modernity, and between tradition and innovation. They all
contribute to the central theme of the commercial that China, thanks to a motiva-
tion for change that originates in her heart, has been undergoing the process of
modernization and globalization while retaining her cultural identity.

Keywords: conceptual metaphor, conceptual metonymy, conceptual blend, primary
metaphor, complex metaphor, nonverbal and multimodal manifestations
1. Introduction
This chapter intends to demonstrate the nonverbal and multimodal manifes-
tations of metaphors and metonymies within the cognitive linguistic para-
digm, focusing on the example of an educational advertisement screened on
China Central Television (CCTV). In contrast with commercial advertise-
ments, on CCTV, educational advertisements are designed to influence peo-
ples way of thinking and understanding for the purpose of promoting public
welfare. They are, therefore, usually called advertisements for public good
120 Ning Yu
(). While educational advertisements on CCTV are distinct from
ordinary TV commercials, which have business promotion as their purpose,
I will still use the term commercial or TV commercial, in its broader
sense, to refer to the one under discussion in this chapter.
In this chapter the term metaphor is used in a broad and a narrow sense.
The broad sense includes both metaphor and metonymy in the narrow sense
of the terms. Actually, the distinction between metaphor and metonymy is
scalar, rather than discrete: they seem to be points on a continuum of map-
ping processes (Barcelona 2000b: 16). According to cognitive linguistics,
metonymy is a more fundamental cognitive phenomenon than metaphor, and
metaphor is very often motivated by metonymy (see Barcelona 2000a;
Dirven and Prings 2002; Panther and Radden 1999). This cognitive linguis-
tics view of metaphor and metonymy will gain further support in the analysis
that follows.
The TV commercial to be analyzed is about two minutes long. It con-
verges on the linguistic presentation of a short verbal message like a motto:

In everyones heart there is a big stage; however big ones heart is, that is
how big the stage is
While this verbal line is itself metaphorical in nature, it serves, I will argue,
as the core of the educational TV commercial that is constructed as a blend
of cultural beliefs and conceptual metaphors presented visually and aurally
as well as verbally. I will show that there are two major conceptual meta-
phors underlying this commercial: LIFE IS A JOURNEY and LIFE IS A STAGE.
Of these two the former is manifested visually, i.e., through visual images,
whereas the latter is realized as a multimodal metaphor, i.e., a metaphor
whose target and source are each represented exclusively or predominantly
in different modes (Forceville 2006a: 384/this volume). I will suggest that
although the TV commercial is centered on a Chinese country girl who
dances all the way from a rural village to a modern metropolis, it can be
interpreted allegorically as containing a personification of China, which has
been undergoing a process of modernization and globalization.
In what follows, I first provide a brief review of the cognitive linguistic
paradigm of metaphor studies. Then, following a synopsis of the TV com-
mercial, I analyze the nonverbal and multimodal manifestations of meta-
phors and metonymies involved to reveal how a multi-faceted metaphorical
complex is achieved via contributions from and constructions in visual and
aural, as well as verbal, modes.
Nonverbal and multimodal metaphors and metonymies 121
2. Cognitive linguistic theory of conceptual metaphor
During the past two decades, cognitive science has seriously challenged the
fundamental assumption that most of our thinking about the world is literal,
directly corresponding to external reality (see e.g., Gibbs 2006; Lakoff and
Johnson 1999). The results of cognitive linguistic studies show that human
minds are embodied in the cultural world, and thinking and reasoning are
largely metaphorical and imaginative, shaped by embodied and acculturated
experiences (e.g., Gibbs 1994, 2006; Johnson 1987; Lakoff 1987; Lakoff
and Johnson 1980, 1999). It is argued that all cognition is embodied in
cultural situations (Gibbs 1999: 156).
According to the conceptual metaphor theory of cognitive linguistics,
metaphor is not merely a figure of speech, but also a figure of thought, giv-
ing rise to understanding one conceptual domain in terms of another concep-
tual domain.
1
Conceptual metaphors in peoples conceptual systems influ-
ence to a considerable extent how they think, understand, reason, and
imagine in everyday life, and many concepts, especially abstract ones, are
structured and mentally represented in terms of metaphor (Gibbs 1999:
145).
It is worth stressing that the experiential basis of conceptual metaphors is
both bodily and cultural. Cognitive linguistics maintains that our minds are
embodied in such a way that our conceptual systems draw largely upon the
peculiarities of our bodies and the specifics of our physical and cultural
environments (e.g., Gibbs 1994, 2006; Lakoff and Johnson 1980, 1999).
According to the cognitive linguistic view, conceptual metaphors emerge
from the interaction between body and culture: they are grounded in bodily
experience, but shaped by cultural understanding.
In order to answer the question why some metaphors are widespread or
even universal and others are culture-specific, the newer version of Concep-
tual Metaphor Theory puts forth a decomposition account based on the
distinction between two kinds of conceptual metaphors: primary metaphors
and complex metaphors (see Grady 1997a, 1997b, 2005; Kvecses 2002,
2005; Lakoff and Johnson 1999, 2003). In short, primary metaphors are
derived directly from experiential correlations, or conflations in everyday
experience that pair subjective experience and judgment with sensorimotor
experience (Lakoff and Johnson 1999: 49), whereas complex metaphors are
combinations of primary metaphors and cultural beliefs and assumptions
and, for that reason, tend to be culture-specific. For instance, Lakoff and
Johnson (1999: 6061) suggest that the complex metaphor A PURPOSEFUL
122 Ning Yu
LIFE IS A JOURNEY is composed of the following cultural belief (reformulated
here as two propositions) and two primary metaphors:

PEOPLE SHOULD HAVE PURPOSES IN LIFE
PEOPLE SHOULD ACT SO AS TO ACHIEVE THEIR PURPOSES
PURPOSES ARE DESTINATIONS
ACTIONS ARE MOTIONS

Whereas the two primary metaphors (PURPOSES ARE DESTINATIONS and
ACTIONS ARE MOTIONS), based on common bodily experience, are likely to
be universal, the complex metaphor (A PURPOSEFUL LIFE IS A JOURNEY) is
not or less likely to be so. This is because its validity in a particular culture
depends on this cultures holding the combination of the two propositions
(PEOPLE SHOULD HAVE PURPOSES IN LIFE and PEOPLE SHOULD ACT SO AS
TO ACHIEVE THEIR PURPOSES) and the two primary metaphors, as listed
above.
A more recent development that is related to Conceptual Metaphor The-
ory is Blending or Conceptual Integration Theory (e.g., Fauconnier and
Turner 1998, 2002; Turner and Fauconnier 1995). Different from concep-
tual metaphor theorys conceptual mapping from a source to a target do-
main, this theory postulates conceptual integration of two or more input
spaces, which share a generic space of common characteristics, into a
new blended space. This multiple-space model can account for various
metaphorical and non-metaphorical aspects of conceptual phenomena (see
also Grady, Oakley, and Coulson 1999; Kvecses 2002; Forceville 2004 for
detailed discussions). A major difference between conceptual metaphor the-
ory and blending theory is that the former is typically concerned with en-
trenched conceptual relationships and the ways in which they may be elabo-
rated whereas the latter often focuses on novel conceptualizations, which
may be short-lived (Grady, Oakley, and Coulson 1999). In an effort to
bridge the theories of conceptual metaphor and conceptual integration, it is
proposed that primary metaphors, as entrenched metaphoric correspondences
between concepts, often function as inputs to metaphoric blends in the proc-
ess of conceptual integration (Grady 2005).
The rise of the cognitive linguistic paradigm of metaphor studies has cre-
ated opportunities for the study of nonverbal and multimodal metaphors.
However, as Forceville (2006a) points out, Conceptual Metaphor Theory
has so far been restricted in an important dimension. While it characterizes
metaphors as primarily conceptual in nature and only secondarily manifested
in language, the validity of its claims about the existence of conceptual
metaphors depends almost exclusively on linguistic evidence in the form of
verbal metaphors. If, as it claims, metaphor fundamentally characterizes
thinking, and is thus not an exclusive attribute of language, it should be able
to produce nonverbal manifestations as well as the purely verbal ones that
have so far been the central concern of conceptual metaphor studies. If
metaphor does not necessarily appear in verbal form, conceptual metaphor
theory can hardly afford to ignore the nonverbal realm. For pioneering work
on nonverbal and multimodal metaphors within the cognitive linguistic para-
digm, readers are referred to Forcevilles empirical and theoretical studies
(e.g., 1994, 1996, 1999, 2002, 2005, 2006a, 2006b).
In the light of Forcevilles (2006a) argument, this chapter is part of the
attempt to demonstrate, within the cognitive linguistic paradigm, that con-
ceptual metaphors can be manifested nonverbally and multimodally as well
as verbally.
3. Synopsis of the commercial
Here is a synopsis of the TV commercial under discussion. At the beginning,
a close-up shot from a low camera angle focuses on a Chinese country girl,
wearing peasant-style attire and posed for Western ballroom dancing (figure
1). With the playing of the slow-tempo music of the Chinese folk song Lan
Huahua,
2
the girl starts dancing elegantly but repetitively, turning around
and around, all alone, in a snow-covered countryside. She keeps turning and
turning, along a country path, and through a village with small country
houses (figure 2). As she dances past, it can be seen that she has a gracious
smile on her face, apparently absorbed in the joy of dancing the Western-
style ballroom dance, despite the fact that she does not even have a dancing
partner. Then, in an urban setting, she dances past a traditional-looking tall
dark-red wall (figure 3) and then a Western-looking sculpture (figure 4), and
finally up to the top of a skyscraper, against the metropolitan background
bathing in the sun (figure 5).
At this point, the line In everyones heart there is a big stage appears
on the screen, getting closer and bigger, as the backdrop turns into a dark-
ened screen (figure 6). All of a sudden, the audio shifts from the slow-tempo
Chinese folk music to fast-tempo Western ballroom dance music. Now, the
country girl is dancing with a male partner in a swallow-tailed tuxedo. To-
gether they make a great variety of beautiful moves and poses (figure 7).
Again, they dance past the Western-looking sculpture (figure 8) and the
traditional-looking dark-red wall (figure 9), and then back up to the round
124 Ning Yu
top of another skyscraper, this time with 24 other pairs of similar-looking
dancers following them. While taking the leading role, the first pair dances
around the top of the skyscraper, followed by the other 24 pairs (figure 10).
Then, as the leading couple dances in the foreground, the remaining 24 pairs
change into a matrix of four by six dancing in the background (figure 11).
At this time, the line However big ones heart is, that is how big the
stage is draws nearer and larger while the background fades into a black
screen (figure 12). After this, the country girl is alone again, turning around
slowly to a stop (figure 13). Finally, as the audio shifts back to the Chinese
folk music of Lan Huahua, the girl stands still, with her back toward the
audience, looking far at the skyline of the modern metropolis (figure 14).
The final scene provides a global view - produced by a long shot from a
high camera angle - of the big city with its many tall buildings.

Figure 1. Posed for dancing Figure 2. Dancing past a village

Figure 3. Dancing past a wall Figure 4. Dancing past a sculpture

Figure 5. Dancing on a skyscraper Figure 6. Stage in the heart

Figure 7. Dancing with a partner Figure 8. Dancing past the sculpture

Figure 9. Dancing past the wall Figure 10. Leading the way

Figure 11. Leading and supporting Figure 12. Size of heart and stage

Figure 13. Dancing alone again Figure 14. Gazing afar standing
4. Analysis
In this section, I analyze the TV commercial to show that its didactic and
aesthetic effects are achieved through, among other things, nonverbal and
multimodal manifestations of two common conceptual metaphors: LIFE IS A
JOURNEY and LIFE IS A STAGE. While the former is realized almost exclu-
126 Ning Yu
sively through moving images, accompanied by musical sounds, the latter
one is manifested visually and aurally, as well as verbally through the lin-
guistic message: In everyones heart there is a big stage; however big ones
heart is, that is how big the stage is. Other than these two conceptual meta-
phors, the TV commercial also contains several conceptual metonymies,
which both motivate and constitute the metaphors, and help set in relief the
unity and contrast of cultural identity and globalization that characterize
contemporary China.
Before I proceed to analyze the conceptual metaphors and metonymies
that structure the TV commercial, I first briefly comment on the Chinese
cultural conceptualization of the HEART and the verbal message that serves
as the core of the advertisement under discussion. In the tradition of Chinese
culture, the heart (xin) is regarded as the organ for thinking and under-
standing as well as feeling, and more generally as the central faculty of cog-
nition (Yu 2003, 2005, 2007a, 2007b). This cultural conceptualization of
the heart contrasts with the Western dualism that maintains the heart-mind
dichotomy, i.e., the heart is the seat of emotions whereas the mind, associ-
ated with the brain, is the center of thoughts.
In light of the above comment, I would like to point out that the verbal
message in the TV commercial is a manifestation of the Chinese conceptu-
alization of the heart as the central faculty of cognition, as well as an in-
stantiation of the popular conceptual metaphor LIFE IS A STAGE. On the
stage of life, people play various roles, some being more important and
successful than others. Ones degree of success in external life (the size of
the stage) is attributed and related to the mental capacity of ones internal
world, the heart (the size of the heart), in a metaphorical fashion. That
is, only when one can think big (i.e., with a big heart) can one act big
on the big stage of life. So interpreted, the verbal message of the TV
commercial reveals the following combination of propositions and meta-
phors:

(1)
a. HEART IS THE THINKING ORGAN THAT DESIGNS ACTIVITY OF LIFE
b. SUCCESS IN LIFE ORIGINATES IN HEART
c. DEGREE OF MOTIVATION FOR SUCCESS IS SIZE OF HEART
d. MORE MOTIVATED FOR SUCCESS IS BIGGER OF HEART

In this group, (1a) and (1b) are two propositions that reflect the Chinese
cultural conceptualization of the heart whereas (1c) and (1d) are meta-
phors that are rooted in the cultural beliefs of the heart as the central fac-
ulty of cognition. The metaphorical nature of the culturally constructed un-
derstanding of the heart is quite obvious. This understanding can be sum-
marized by a more general complex metaphor: ONES MENTAL CAPACITY IS
SIZE OF ONES HEART.

4.1. LIFE IS A JOURNEY
I first analyze the conceptual metaphor LIFE IS A JOURNEY, which is funda-
mentally manifested through moving images. In the TV commercial under
discussion, the girl undertakes a journey, going or, more exactly, dancing all
the way from the field of the snow-covered countryside to the top of a sky-
scraper in a large metropolitan area. This journey, however, is metaphori-
cally designed and constructed to manifest, visually, the common conceptual
metaphor LIFE IS A JOURNEY. In other words, it is not a physical journey
taking place through space, but one that is a metaphor for subjective experi-
ence and abstract advancement in life. The cueing of the dancing as a JOUR-
NEY metaphor for abstract life is achieved mainly through the shifts of
scenes that show spatial, as well as temporal, leaps and bounds across
various physical settings. While the country girl repeats her monotonous but
graceful act of dancing alone, her path extending through conspicuous dif-
ferences in the environment, free from any restrictions, makes a literal read-
ing of her dancing impossible.
The LIFE IS A JOURNEY metaphor, which is based on the SOURCE-PATH-
GOAL schema,
3
creates mappings from the source domain of journey to the
target domain of life and establishes correspondences between various items
within these two conceptual domains, such as those shown in (2). The ar-
rows indicate the direction of the metaphorical mappings from the source to
the target domain.

(2) LIFE IS A JOURNEY
SOURCE TARGET
a. JOURNEY LIFE
b. TRAVELER PERSON
c. STARTING POINT INITIAL STATE
d. TRAVEL ON JOURNEY EXPERIENCE IN LIFE
e. PATH OF JOURNEY WAY OF LIFE
f. DESTINATION GOAL

The experiential motivation for this metaphor is that people do undertake
many journeys as part of their life. In this sense, the metaphor LIFE IS A
128 Ning Yu
JOURNEY has a metonymic basis when JOURNEY is mapped onto LIFE, char-
acterized by a conceptual metonymy PART FOR WHOLE.
In the TV commercial under discussion, the traveler is the country girl.
For her, traveling is dancing a Western ballroom dance that she really en-
joys, even though she does it initially without a partner. For her, the path of
the journey runs from the cold of the snow-covered countryside to the
warmth of the sun-bathed modern metropolis, and from the country field to
the top of a skyscraper in a big city. More abstractly, this is a path of going
upward in spatial conceptualization of success in life.
The correspondences listed in (3) and (4) are some of the entailments of
the mappings given in (2) (see also Lakoff 1993):

(3) EXPERIENCE IN LIFE IS TRAVEL ON JOURNEY
a. DIFFICULT TRAVEL BAD EXPERIENCE
b. EASY TRAVEL GOOD EXPERIENCE
c. FAST MOTION FAST PROGRESS
d. SLOW MOTION SLOW PROGRESS

(4) WAY OF LIFE IS PATH OF JOURNEY
a. PHYSICAL CONDITIONS ABSTRACT STATES
b. BUMPY PATH DIFFICULT WAY
c. SMOOTH PATH EASY WAY

This is how conceptual metaphors systematically transfer inferences or en-
tailments, based on our bodily experience, from the source to the target do-
main.
It is noteworthy that LIFE IS A JOURNEY is a complex metaphor that
represents the combination of some primary metaphors and a number of
cultural beliefs, as shown in (5):

(5) LIFE IS A JOURNEY
a. PEOPLE SHOULD HAVE GOALS IN THEIR LIFE
b. PEOPLE SHOULD ACT SO AS TO ACHIEVE THEIR GOALS
c. STATES ARE LOCATIONS
d. CHANGES ARE MOVEMENTS (FROM ONE TO ANOTHER LOCATION)
e. CAUSES ARE FORCES
f. ACTIONS ARE SELF-PROPELLED MOTIONS
g. PURPOSES ARE DESTINATIONS

Here, (5a) and (5b) present two propositions as the cultural beliefs or as-
sumptions upheld by people who subscribe to the complex metaphor LIFE IS
A JOURNEY; (5cg) are primary metaphors of the so-called Event Structure
Metaphor, which is a metaphor system responsible for the conceptualization
of various abstract events (see Lakoff 1993; Yu 1998, Ch. 5).
In the commercial, the country girl has her goal in life and she acts, per-
sistently, to attain this goal. During the process, she has changed the state of
her life. More specifically, she has made changes by moving (in this case,
dancing a Western-style ballroom dance) from one location (the field of the
backward countryside) to another location (the top of a skyscraper of a mod-
ern city). The stark contrast between these two locations represents, meta-
phorically, the marked difference between two states in her life that the
change has brought to her. The cause for this change is the motivation or
ambition in her heart, which is the force that pushes her to make repetitive
but beautiful moves (keeping turning around in her ballroom dance) in a
consistent and persistent manner, until she reaches her destination, when she
is joined by a male dancing partner, and then followed by 24 other pairs of
dancers. Her destination (her goal) is the top of the skyscraper, which looks
like a highly elevated stage (the STAGE metaphor), from which she can enjoy
a full view of the world around her. Physically, a vantage point on a
higher location enables people to see farther, and that is what happens when
the country girl gazes afar on the top of the skyscraper at the very end of the
commercial. Metaphorically, the physical perception of seeing is mapped
onto the mental function of understanding in the primary conceptual meta-
phor UNDERSTANDING IS SEEING (see Lakoff and Johnson 1999; Sweetser
1990; Yu 2004). Thus, we have a two-step mapping: HIGHER LOCATION
SEEING FARTHER UNDERSTANDING BETTER. Of these two mappings, the
first is metonymic whereas the second is metaphorical. Both mappings are
manifested in the TV commercial visually.
At this point, I want to underscore another aspect of the significance of
the country girl ending up on the top of a skyscraper after dancing all the
way from the country field. This, I believe, is the visual manifestation of a
primary conceptual metaphor SUCCESSFUL IS UP, i.e., A MORE SUCCESSFUL
STATE IS A HIGHER LOCATION, which is combined with the LIFE IS A JOUR-
NEY metaphor to form another complex metaphor, as in (6):

(6) SUCCESSFUL CAREER IN LIFE IS UPWARD MOVEMENT ON JOURNEY
a. LIFE IS A JOURNEY
b. SUCCESSFUL IS UP

The list in (7) below gives some of the conceptual mappings entailed by (6b)
from the concrete domain of space to the abstract domain of success.

130 Ning Yu
(7) SUCCESSFUL IS UP
a. UP SUCCESSFUL
b. HEIGHT OF LOCATION DEGREE OF SUCCESS IN LIFE
c. HIGH LOCATION SUCCESSFUL STATUS IN LIFE
d. HIGHER MORE SUCCESSFUL
e. LOWER LESS SUCCESSFUL

That is, DEGREE OF SUCCESS IN LIFE IS HEIGHT OF LOCATION. HIGHER is
mapped onto MORE SUCCESSFUL whereas LOWER is mapped onto LESS SUC-
CESSFUL. A higher location represents a more successful status in life, and a
lower location represents a less successful status in life.
4

In short, the journey that the country girl has undertaken is a journey
from a small village to a large metropolis, and from backwardness to moder-
nity. At the end of the journey, she can enjoy, at a very high vantage point, a
global view of her world that she could not have had if she had not danced
all the way from the field of the countryside to the top of a skyscraper in a
modern city.

4.2. LIFE IS A STAGE
Now I turn to analyzing the multimodal manifestation of the conceptual
metaphor LIFE IS A STAGE. This conceptual metaphor is, again, a complex
metaphor that represents the combination of a number of components at
different levels. First consider (8) below:

(8) LIFE IS A STAGE
a. PEOPLE ACT TO ACHIEVE SUCCESS IN LIFE
b. PEOPLES ACTION IN LIFE IS EVALUATED BY OTHERS
c. ACTION IN LIFE IS ACTION ON STAGE
d. STATES ARE LOCATIONS
e. ACTIONS ARE SELF-PROPELLED MOTIONS

Here (8c) is the key metaphorical component. Since ACTION IN LIFE IS AC-
TION ON STAGE, after the identical item, ACTION, on both sides of the equa-
tion is eliminated, we have LIFE IS A STAGE. But what is the experiential
motivation for (8c)? I argue that this metaphor is motivated by a more fun-
damental figurative relationship, a metonymy, ACTING ON STAGE STANDS
FOR ACTING IN LIFE, which is a specific instantiation of the more general
conceptual metonymy PART STANDS FOR WHOLE. That is, acting on the
stage is only part of the whole, acting in life, and here we use a part to stand
for the whole. In the above list, (8d) and (8e) represent two primary meta-
phors in the Event Structure Metaphor System on which the complex meta-
phor LIFE IS A STAGE is based. Life is a series of states whereas a stage is a
special kind of location. The actions that people take in life, whether con-
crete or abstract, are generally understood as self-propelled motions through
space. In this particular case, actions taken in life are metaphorically con-
ceptualized as artistic moves of ballroom dancing. Besides, I assume that the
cultures that subscribe to the LIFE IS A STAGE metaphor also hold the propo-
sitions in (8a) and (8b), in combination with (8c). Thus, the conceptual par-
allel is perceived as follows. People act to achieve success in life, just as
performers act to achieve success on the stage; their actions in life are evalu-
ated by others, just as actors and actresses performances are evaluated by
their audience.
The LIFE IS A STAGE metaphor establishes, for instance, the correspon-
dences between the following elements in the two conceptual domains.

(9) LIFE IS A STAGE
SOURCE TARGET
a. STAGE LIFE
b. PERFORMANCE ON STAGE ACTIVITY IN LIFE
c. ROLES ON STAGE PEOPLE IN LIFE

In the TV commercial, the country girl dances all the way from a small vil-
lage to a big city, so the stage of her life is indeed very big. Her perform-
ance can be divided into four phases. In the first phase, she is alone and
starts dancing ballroom dance. Her moves, though graceful, are repetitive,
metaphorically representing, I suggest, her persistence and perseverance in
pursuit of her goal. The fact that the Chinese country girl dances a Western
ballroom dance, rather than a Chinese folk dance, is unexpected and for that
matter really significant. In China, the Western ballroom dance is often re-
ferred to as the international standard dance. During the past 15 years or
so, a goal that China has been trying to achieve to implement the reform
and open-door policy is to be connected with the international track/rail
(), i.e., to meet the international standard. Thus, the country
girls persistence in dancing the international standard dance can be seen
as a visual metaphor of Chinas effort to meet the international standard.
In the second phase, the country girl is joined by a male dancing partner
wearing the standard ballroom dance apparel (i.e., a black swallow-tailed
tuxedo and black leather shoes), as in sharp contrast with her Chinese peas-
ant-style clothing. Together, they make all kinds of beautiful moves and
poses, accompanied by Western ballroom dance music. Their fast-tempo
132 Ning Yu
movements, accompanied by the fast-tempo music, are metaphorical of their
speedy advancement in life. Such a conspicuous juxtaposition of the Chinese
and the Western in the pair dance seems to be a visual cue for Chinas coop-
eration with the outside, especially the Western world. This cooperation is
essential for Chinas process of modernization and globalization.
In the third phase, the country girl, with her dancing partner, plays a
leading role in dancing when followed by 24 other couples, and is meta-
phorically a leader in life. The leading role in the group dance seems to cue
the increased influence of China in the international arena or on the interna-
tional stage. In the fourth and last phase, the girl is alone again, stops danc-
ing, and gazes afar while standing still. This is when she achieves some deep
understanding of life when gaining a far vision, i.e., UNDERSTANDING IS
SEEING (see Lakoff and Johnson 1999; Yu 2004): In everyones heart there
is a big stage; however big ones heart is, that is how big the stage is. While
enjoying the global vision at the high vantage point, the girl stands with
her back toward the viewers, looking ahead into the future (since accord-
ing to the Chinese version of the TIME IS SPACE metaphor the future is ahead;
see, e.g., Lakoff 1993; Yu 1998, Ch. 4). This shot design seems to put the
viewers into a following position, i.e., they are invited to follow suit
looking ahead into Chinas bright future while sharing the girls understand-
ing of life as specified by the verbal message of the advertisement.
The metaphorical mappings in (9b) and (9c), for instance, have the fol-
lowing entailments, transferring further knowledge from the source to the
target domain:

(10) ACTIVITY IN LIFE IS PERFORMANCE ON STAGE
a. SUCCESS ON STAGE SUCCESS IN LIFE
b. FAILURE ON STAGE FAILURE IN LIFE
c. VARIETY IN PERFORMANCE DIVERSITY IN ACTIVITY

(11) PEOPLE IN LIFE ARE CHARACTERS ON STAGE
a. LEADING CHARACTERS ON STAGE MORE SUCCESSFUL
PEOPLE IN LIFE
b. SUPPORTING CHARACTERS ON STAGE LESS SUCCESSFUL
PEOPLE IN LIFE

As shown in (10) and (11), the entailments all contribute to the systematic
mappings from the source to the target domain, activated by the LIFE IS A
STAGE metaphor. In the TV commercial, the country girl is indeed very suc-
cessful in life since, later in the show, she is playing a leading role in the
foreground of the stage.
However, LIFE IS A STAGE is still not sufficient to capture the figurative
meaning of the verbal message in particular and the TV commercial in gen-
eral. It still needs to combine with another primary metaphor, SUCCESSFUL
IS BIG, so as to form another complex metaphor, A SUCCESSFUL LIFE IS A BIG
STAGE. This further combination is given in (12).

(12) A SUCCESSFUL LIFE IS A BIG STAGE
a. LIFE IS A STAGE
b. SUCCESSFUL IS BIG

Given in (13) below are some of the conceptual mappings entailed by (12b)
from the concrete domain of spatial dimensions to the abstract domain of
success.

(13) SUCCESSFUL IS BIG
a. BIG SUCCESSFUL
b. SIZE OF OBJECT DEGREE OF SUCCESS IN LIFE
c. BIG OBJECT SUCCESSFUL STATUS IN LIFE
d. BIGGER MORE SUCCESSFUL
e. SMALLER LESS SUCCESSFUL

That is, the physical size of an object is mapped on to the degree of success.
5

In (12) the OBJECT is specified as a stage. Thus, (14) lists some of the
conceptual mappings entailed by (12):

(14) A SUCCESSFUL LIFE IS A BIG STAGE
a. BIG SUCCESSFUL
b. SIZE OF STAGE DEGREE OF SUCCESS IN LIFE
c. BIG STAGE SUCCESSFUL STATUS IN LIFE

That is to say, the size of ones stage is metaphorically correlated with the
degree of success in ones life: DEGREE OF SUCCESS IN LIFE IS SIZE OF
STAGE. The bigger ones stage is (of course metaphorically), the more suc-
cessful one is in life. It is worth noting that the top of the skyscraper, where
the girl, her dancing partner, and 24 other pairs are dancing, looks very
much like a big stage. Thus, the metaphor A SUCCESSFUL LIFE IS A BIG
STAGE is manifested visually as well as verbally.
In summary, the LIFE IS A STAGE metaphor is manifested visually through
moving images, accompanied by musical sounds, as well as linguistically
through the verbal message appearing on the TV screen. The fact that the
country girl has made remarkable progress in life is revealed via visual
metaphors. First, the change in physical location of her dancing from the
134 Ning Yu
backward countryside to the modern metropolis metaphorically suggests her
abstract advancement in life. Second, the change in the manner of her danc-
ing, from solo dancing, to dancing as a couple, and to dancing as the leading
pair of a group, also metaphorically suggests her general progress in life. It
is worth mentioning that the metaphor A SUCCESSFUL LIFE IS A BIG STAGE
also has its metonymic motivation, which can be expressed by the metonymy
STAGE STANDS FOR PERFORMANCE ON STAGE, or more generally LOCATION
OF ACTIVITY STANDS FOR ACTIVITY. The network of metonymic and meta-
phoric relations involved is shown below:

STAGE FOR PERFORMANCE + LIFE IS A STAGE + SUCCESSFUL IS BIG
STAGE PERFORMANCE ON STAGE ACTIVITY IN LIFE
SIZE OF STAGE DEGREE OF DEGREE OF
SUCCESS ON STAGE SUCCESS IN LIFE
BIG STAGE SUCCESSFUL PERFORMANCE SUCCESSFUL
ON STAGE ACTIVITY IN LIFE

That is, as the first step, STAGE is mapped metonymically onto the PER-
FORMANCE ON STAGE and, as the second step, PERFORMANCE ON STAGE is
mapped metaphorically onto ACTIVITY IN LIFE.

4.3. Multimodal metonymies
Apart from the two major conceptual metaphors discussed above, the TV
commercial has also deployed a number of metonymies to achieve its didac-
tic purpose and artistic effect, as has already been touched upon above. For
instance, in the verbal message In everyones heart there is a big stage;
however big ones heart is, that is how big the stage is, we can say that,
initially, the reference to the stage is a metonymy for the performance on
the stage, i.e., STAGE FOR PERFORMANCE ON STAGE, or more generally
LOCATION OF ACTIVITY FOR ACTIVITY. In this case, the figurative mapping
takes place from one thing to another within the same conceptual domain. It
is through further mapping across the domains that the metaphor ACTIVITY
IN LIFE IS PERFORMANCE ON STAGE is constructed.
In this section, I discuss several other metonymies in the visual and aural
modes. In effect, these metonymies under analysis are all integrated into the
complex of conceptual metaphors. The first visual metonymy is STYLE OF
CLOTHING STANDS FOR CULTURE. The girl wears typical peasant-style cloth-
ing, which is metonymically associated with traditionally agricultural Chi-
nese culture. It fits well into the rural setting at the beginning of the commer-
cial. However, this style of clothing is conspicuously at odds with Western
ballroom dance. The most conspicuous contrast appears when the country
girl in the Chinese peasant-style clothes is dancing the Western ballroom
dance with a male partner in a black swallow-tailed tuxedo, which is the
standard Western-style ballroom dance apparel. It is a conspicuous visual
blend of contrasting Chinese and Western styles. Subsequently, 24 other
pairs of dancers join them, wearing exactly the same attire as they do. I
would suggest that the country girls peasant-style clothing is metonymic for
the cultural identity of the Chinese in general, and it is part of the visual
metaphor for the retention of cultural identity in the process of moderniza-
tion and globalization. Although her surroundings have changed drastically
over time, her Chinese peasant-looking appearance has remained the same.
Another metonymy I want to mention is STYLE OF DANCE STANDS FOR
CULTURE. Ballroom dance is associated metonymically with Western cul-
tures in the developed countries that embody modernity and superiority in
various areas in the world today. In the TV commercial, the country girl
could have danced a Chinese folk dance, which would have been very ap-
propriate for her identity represented metonymically by her Chinese peasant-
looking appearance. Instead, what we see is a conspicuously inappropriate
blend of the Chinese peasant-looking appearance and the Western elegance
of ballroom dance. As is masterfully designed, indeed, the Chinese country
girl dancing Western ballroom dance is a powerful visual metaphor for the
process of modernization and globalization that China has been undergoing
in the past twenty years or so. The metonymy STYLE OF DANCE STANDS FOR
CULTURE, realized visually, is an important component of that complex
metaphor. In the process of mapping, we can trace the following steps of
metonymic mapping governed by the principle of contiguity: BALLROOM
DANCE WESTERN CULTURE DEVELOPED COUNTRIES MODERNIZA-
TION AND GLOBALIZATION. However, if we omit and ignore the two interme-
diate steps, we have a cross-domain mapping that is metaphorical: BALL-
ROOM DANCE IS PROCESS OF MODERNIZATION AND GLOBALIZATION.
The third metonymy to be discussed is STYLE OF PHYSICAL SETTING FOR
CULTURE. This metonymy is visualized mainly by the juxtaposition of two
conspicuous contrasts representing different cultures. The first contrast con-
sists of the countryside versus the metropolis. In the countryside, which is
the physical setting of the first portion of the commercial, we see an open
field covered by white snow, and a small village with small wood houses (in
the Northeast of China). In stark contrast, what we see in the second half of
the commercial is a large metropolitan area with numerous skyscrapers (in
Shanghai). This represents the contrast between the underdeveloped and the
136 Ning Yu
developed, and between primitivity and modernity. The second contrast is
that between the tall dark-red wall and the Western-looking sculpture, which
are both shown twice. The first time, the country girl dances past them
alone, and the second time, the girl and her partner dance past them together.
The tall dark-red wall looks like those enclosing the Forbidden City (in Bei-
jing), the royal palace that was off limits to ordinary people in the last feudal
dynasties of China. It is, therefore, metonymically associated with a tradi-
tional culture of isolation characteristic of China before it was opened up to
the outside world some twenty years ago. The Western-looking sculpture, by
contrast, is a visual metonymy of the influence of Western culture present in
contemporary China following the implementation of its open-door policy. It
is worth pointing out that the visual contrasts brought out by the metonymy
STYLE OF PHYSICAL SETTING FOR CULTURE play an important part in the
visual manifestation of the two conceptual metaphors, LIFE IS A JOURNEY
and LIFE IS A STAGE.
Finally, I turn to the metonymy STYLE OF MUSIC STANDS FOR CULTURE.
Two kinds of music are played through the commercial. At the beginning, as
the country girl starts dancing her ballroom dance, the accompanying music
is not Western ballroom dance music, but the music of the Chinese folk song
Lan Huahua. Playing Chinese folk music instead of Western ballroom dance
music adds to the cultural context and cultural identity created by the visual
images of the country girls Chinese peasant-style attire and the physical
setting of the Chinese countryside. That is, music is used as one of the tools
to create cultural context and cultural identity. After the pair and group
dancing, the country girl is alone again, standing motionless on top of a sky-
scraper, gazing at the panorama of the modern metropolis. The audio track,
at this point, shifts back from the Western ballroom dance music to Lan
Huahua, the Chinese folk song music, for the final seconds of the TV com-
mercial. This shift in musical style is designed, I argue, to suggest, meto-
nymically, the retention of cultural identity despite the fact that the physical
setting has changed from the countryside to the metropolis, and from primi-
tivity to modernity. The country girl has not lost her cultural identity, her
appearance remaining the same, even though her state of life has drastically-
changed, as metaphorically and metonymically represented by the change of
locations and physical settings. She is now embedded in a modernized and
globalized environment, as visually represented by the global view of a
modern metropolis, but her cultural identity is retained, as represented visu-
ally by her Chinese peasant-style attire, and aurally by the Chinese folk mu-
sic played for the last few seconds to complete the commercial.
In sum, the metonymies discussed in this section all fall into one general
pattern, where PROTOTYPICAL ITEMS OF A CULTURE STAND FOR THAT CUL-
TURE or SALIENT FEATURES OF A THING STAND FOR THAT THING. As we
have seen, this conceptual metonymy can be manifested visually and aurally
as well as verbally.
5. Conclusion
One of the crucial insights of cognitive linguistic theory of metaphor is that
verbal metaphors systematically manifest underlying conceptual metaphors.
There is already ample and still growing linguistic evidence, in support of
this claim, discovered by empirical studies of a broad spectrum of world
languages and from a cross-cultural perspective. If, as cognitive linguists
have argued, metaphor is primarily conceptual in nature as a cognitive
mechanism characterizing the mode of thought or the way of thinking, it
follows that conceptual metaphors should emerge in nonverbal manifesta-
tions as well as verbal ones. So far, there are not many studies focused on
nonverbal and multimodal manifestations of conceptual metaphors despite
the fact that such studies are theoretically essential to consolidate the validity
of conceptual metaphor theory (see Forceville 2006a). This overwhelming
preference to the study of verbal over nonverbal manifestations of concep-
tual metaphors needs correcting for the sound development of conceptual
metaphor theory. The present study represents part of the attempt toward
that end.
In this chapter, I have analyzed the nonverbal and multimodal manifesta-
tions of two conceptual metaphors through dynamic visual and aural, as well
as verbal, discourse. I have shown that these conceptual metaphors are com-
plex ones composed of cultural beliefs and assumptions, other complex and
primary metaphors, and metonymies. The various visual, aural and verbal
elements are interactive with, and dependent upon, each other when they
combine into a conceptual blend with input spaces in visual, aural, and
verbal modes. This blend contains conspicuous juxtapositions, simultaneous
or sequential, of contrasting visual, aural or verbal images that are meto-
nymic and metaphoric in nature. These juxtapositions bring to the fore the
unity and contrast between the Chinese and the Western, between thought
and action, between primitivity and modernity, and between tradition and
innovation. They all contribute to the central theme of the TV commercial
that China, thanks to a motivation and ambition for change that originates in
138 Ning Yu
her heart, has been undergoing the process of modernization and global-
ization while retaining her Chinese characteristics.

Table 1. Some possible multimodal mappings of the metaphors LIFE IS A JOURNEY
and LIFE IS A STAGE.
Metaphors Visual Aural Verbal
Source:
JOURNEY
(1) the country girl danc-
ing from the field of the
countryside to the top of a
skyscraper in the me-
tropolis; (2) enjoying the
global view at height and
looking afar and ahead
(1) starting with
slow-tempo music;
(2) switching to
fast-tempo music;
(3) winding down
with slow-tempo
music again

Target:
LIFE
(a) the person/China
undergoing the process of
modernization and glob-
alization; (2) gaining a
deep understanding upon
success and looking into
the future
(1) starting at a
self-propelled pace;
(2) switching to a
fast speed; (3) slow-
ing down to a stop

Source:
STAGE
the girl in peasant cloth-
ing (1) dancing Western
ballroom dance alone; (2)
dancing in pair with a
male partner in Western
tuxedo; (3) dancing as
the leading couple; (4)
still wearing the same
peasant clothing in a
changed environment
(1) starting with
Chinese music;
(2) switching to
Western music;
(3) ending with
Chinese music
again
In everyones
heart there is a
big stage;
however big
ones heart is;
that is how big
the stage is.
Target:
LIFE
The person/China, with
her cultural identity, (1)
making self-propelled
effort toward moderniza-
tion and globalization;
(2) working in coopera-
tion with the West; (3)
advancing as a leader in
development; (4) retain-
ing cultural identity in
modernization
(1) Chinese culture;
(2) international-
ization;
(3) retention of
cultural identity
Success in life
originates
internally in
ones mind;
only when one
thinks big
can one act
big and
achieve big
success in life.
Table 1 summarizes some possible multimodal mappings of the two major
conceptual metaphors, LIFE IS A JOURNEY and LIFE IS A STAGE, studied in
this chapter. As can be seen, the first metaphor, LIFE IS A JOURNEY, is basi-
cally realized in the visual mode, whereas the aural mode performs a meto-
nymic function in support of the visual manifestation of the metaphor in
question, with the tempo of the music indexical of the tempo of the move-
ment on the journey in the visual mode. The role of the verbal mode is absent
in this case. With the second metaphor, LIFE IS A STAGE, the verbal message,
which is only displayed on the screen, is crucial in its manifestation. This
message, however, is very schematic, with all the details furnished in the
visual mode by concrete visual images. In the aural mode, the contrast of
Chinese and Western music is again metonymic for the respective cultures.
In playing this metonymic function, however, the music, switched from Chi-
nese to Western and back to Chinese again, reinforces the visual images that
are metaphorical of the process of modernization and globalization and the
retention of cultural identity. Thus, it is in interaction that the elements in
different modes give rise to nonverbal and multimodal manifestation of
metaphors and metonymies in the TV commercial.
Acknowledgments
An earlier version of this paper, titled Cultural identity and globalization: Mul-
timodal metaphors in a Chinese educational advertisement, was presented at the
12th Annual Conference of the International Association of Intercultural Commu-
nication Studies held in San Antonio in August 2006, and appeared in China
Media Research 3(2): 2532, 2007. I am greatly appreciative of Charles Force-
ville and Mats Rohdin for their insightful comments on the earlier versions of this
chapter, but I am solely responsible for any errors that may remain.
Notes
1. For cognitive linguistic studies of metaphor, metonymy, and figurative lan-
guage in general, see, e.g., Barcelona (2000a), Dirven and Prings (2002),
Gibbs (1994), Gibbs and Steen (1999), Johnson (1987), Kvecses (2002,
2005), Lakoff (1987), Lakoff and Johnson (1980, 1999), Lakoff and Turner
(1989), Panther and Radden (1999), Sweetser (1990), Turner (1991, 1996),
and Yu (1998).
2. More exactly, Lan Huahua is a folk song from the northern Shaanxi Prov-
ince, which belongs to the part of China considered as the place of origin of
140 Ning Yu
Chinese civilization. In China, songs of this kind are known as northern
Shaanxi folk songs.
3. See Forceville (2006b) for a detailed discussion of the SOURCE-PATH-GOAL
schema in multimodal manifestations in three documentary films. The
schema, which structures the concept of JOURNEY (starting point, travel, and
destination) literally, is analyzed as shaping the understanding of such more
abstract concepts as PURPOSEFUL LIFE (ambition, action, and achievement)
and STORY (beginning, middle, and end) by metaphorical extension.
4. The verbal manifestation of the metaphor SUCCESSFUL IS UP or A MORE SUC-
CESSFUL STATE IS A HIGHER LOCATION is extremely rich in Chinese. One lexi-
cal example should suffice here: the compound word gao-di (, literally
high and low) means relative superiority or inferiority (e.g., in a contest
or competition). Thus, zheng ge gao-di (, literally vie for the
high and the low) means vie with each other to see who is better, and nan
fen gao-di (, literally hard to distinguish between the high and the
low) means hard to tell which is better. A nonverbal example, which is
internationally accepted as a standard practice in sports competition, is that
the first, second, and third place winners of an event are distinguished by the
height of the platforms on which they receive their medals or trophies: i.e.,
the champions platform is the highest, and the runner-ups is higher than
the third-place winners (and in addition, the champions platform is usually
in the middle, i.e., IMPORTANT IS CENTRAL). Parallel to this distinction in
height (and centrality) is the difference in height (and centrality) of the three
national flags raised to honor the corresponding three athletic winners.
5. In Chinese, again, the primary metaphor SUCCESSFUL IS BIG, parallel to IM-
PORTANT IS BIG, is manifested linguistically in an extremely rich fashion. For
instance, a high-ranking official (gao-guan) is also called a big offi-
cial (da-guan); a big wrist (da-wan) is a big-brand shining
star (da-pai ming-xing) in acting or singing who has a big name
(ming-qi da), a big shot (da-heng) in business is called a big
sum (of money) (da-kuan), a VIP in general is a big person or big
character (da-renwu), and so on and so forth. Various instances of
nonverbal manifestation of SUCCESSFUL IS BIG can be found in daily life, too.
We can go back to the sports example of the platform in note 4, i.e., the
champions platform should be the biggest if there is a difference in size. Be-
sides, trophies, if there are any, also vary in size with the places of their win-
ners, with the first-place winner given the biggest one. And, if financial
prizes are awarded, they should be related directly to the places of the win-
ners too, with the champion getting the biggest or largest amount of
money on a fattest check. Associated with big stars or big persons in
general are big things, ranging from big cars (limousines) to big
houses (mansions).
References
2000a Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
Barcelona, Antonio
2000b Introduction: The cognitive theory of metaphor and metonymy. In
tive, Antonio Barcelona (ed.), 128. Berlin/New York: Mouton de
Gruyter.
Forceville, Charles
tivity 9: 129.
and Paul Schraders The Comfort of Strangers. Metaphor and Sym-
bol 14: 179198.
2004 Review of Fauconnier and Turner (2002). Metaphor and Symbol 19:
8389.
2005 Visual representations of the idealized cognitive model of anger in
2006a/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
Achard, Ren Dirven and Francisco Ruiz de Mendoza Ibez (eds.),
2006b The SOURCE-PATH-GOAL schema in the autobiographical journey
documentary: McElwee, van der Keuken, Cole. New Review of Film
and Television Studies 4: 241261.
1994 The Poetics of Mind: Figurative Thought, Language, and Under-
standing. Cambridge: Cambridge University Press.
1999 Taking metaphor out of our heads and putting it into the cultural
world. In Metaphor in Cognitive Linguistics, Raymond W. Gibbs,
142 Ning Yu
Jr. and Gerard J. Steen (eds.), 145166. Amsterdam/Philadelphia:
Benjamins.
versity Press.
Gibbs, Raymond W., Jr., and Gerard J. Steen (eds.)
jamins.
Grady, Joseph
1997a Foundation of meaning: Primary metaphors and primary scenes.
Ph.D. dissertation, Department of Linguistics, University of Cali-
fornia at Berkeley.
1997b THEORIES ARE BUILDINGS revisited. Cognitive Linguistics 8: 267
290.
2005 Primary metaphors as inputs to conceptual integration. Journal of
Pragmatics 37: 15951614.
Grady, Joseph, Todd Oakley, and Seana Coulson.
1999 Blending and metaphor. In Metaphor in Cognitive Linguistics,
Raymond W. Gibbs, Jr. and Gerard Steen (eds.), 101124. Amster-
dam/Philadelphia: Benjamins.
Johnson, Mark
1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination,
Kvecses, Zoltn
Press.
Lakoff, George
University Press.
1980 Metaphor We Live By. Chicago: University of Chicago Press.
Panther, Klaus-Uwe, and Gnter Radden (eds.)
Benjamins.
Sweetser, Eve E.
Turner, Mark
1991 Reading Minds: The Study of English in the Age of Cognitive Sci-
ence. Princeton, NJ: Princeton University Press.
1996 The Literary Mind. Oxford: Oxford University Press.
Turner, Mark, and Gilles Fauconnier
1995 Conceptual integration and formal expression. Metaphor and Sym-
bolic Activity 10: 183203.
Yu, Ning
2003 Chinese metaphors of thinking. Cognitive Linguistics 14: 141165.
2004 The eyes for sight and mind. Journal of Pragmatics 36: 663686.
2005 The Chinese heart as the central faculty of cognition. Paper pre-
sented at the 9th International Cognitive Linguistics Conference,
Seoul, Korea. July 2005. To appear in Culture, Body, and Lan-
guage: Conceptualizations of Internal Body Organs across Cultures
and Languages, Farzad Sharifian, Ren Dirven, Ning Yu and Su-
sanne Niemeier (eds.). Berlin/New York: Mouton de Gruyter.
2007a Heart and cognition in ancient Chinese philosophy. Journal of Cog-
nition and Culture 7: 2747.
2007b The Chinese conceptualization of the heart and its cultural context:
Implications for second language learning. In Applied Cultural Lin-
guistics: Implications for Second Language Learning and Intercul-
tural Communication, Farzad Sharifian and Gary B. Palmer (eds.),
6585. Amsterdam/Philadelphia: Benjamins.

III

Multimodal Metaphor in Political Cartoons
Chapter 7
Visual metaphor versus verbal metaphor: A unified
account
Francisco Yus
Abstract
Multimodal metaphors are those whose target and source are each represented
exclusively or predominantly in different modes (Forceville 2006: 384/this vol-
ume), mainly with a verbal-visual interface of source and/or target. When multi-
modality is analyzed in metaphors, the verbal and visual inputs are wrongly
treated as different phenomena demanding different interpretive strategies when
searching for a metaphoric interpretation. In this chapter, on the contrary, it is
claimed that the comprehension of verbal, visual and multimodal metaphors in-
volves similar mental procedures. Although the perception of images differs from
linguistic decoding, reaching an interpretation of metaphors entails similar ad-
justments of conceptual information of texts and images and multimodal combina-
tions, regardless of the modal quality of the input.

Keywords: Relevance theory, conceptual upload, ad hoc concepts, ad hoc pointers,
visual-conceptual interface
1. Introduction
In this chapter, visual metaphor comprehension is compared to verbal meta-
phor comprehension and analyzed mainly from a cognitive pragmatics point
of view (specifically within relevance theory, as proposed by Sperber and
Wilson 1995 [1986]), but also with reference to cognitive linguistics where
appropriate. The main claim underlying this chapter is that the comprehen-
sion of verbal and visual metaphors involves similar mental procedures.
While the perception of images is obviously different from linguistic decod-
ing, reaching an interpretation of visual metaphors also entails an adjustment
148 Francisco Yus
of conceptual information a stage during comprehension that will be called
conceptual upload in the same way as verbal metaphors.
Therefore, although it is not denied that the combination of visual and
verbal inputs in multimodal metaphors can indeed generate interesting inter-
pretive outcomes, as the chapters in this book demonstrate, in many studies
of multimodality visual and verbal metaphors are wrongly treated as differ-
ent phenomena. By contrast, in this chapter it will be argued that both types
of metaphor (and also multimodal metaphors with combinations of text and
image) are decoded by specialized mental modules, which deliver sche-
matic information that has to be enriched inferentially in order to obtain the
intended interpretation. Besides, as will be illustrated below with several
Spanish political cartoons by El Roto, visual metaphors can be arranged on
a scale depending on the gap existing between the prototypical referent of the
image and the cartoonists intended referent, which has to be adjusted infer-
entially in the same way as in verbal metaphors, whose interpretation in-
volves the hearers inferential adjustment of the concept that the speaker
encodes in order to obtain the speakers intended interpretation.
In this sense, the cartoons analyzed in this chapter contain metaphors of a
pictorial/visual nature, whereas multimodal metaphors, as defined by Force-
ville, are metaphors whose target and source are each represented exclu-
sively or predominantly in different modes (Forceville 2006: 384). How-
ever, it follows from my central thesis in this chapter that the model
presented here is capable of accommodating multimodal metaphors in the
same way as metaphors with only verbal or only visual inputs.
2. Relevance theory and ad hoc concept formation
Sperber and Wilsons (1995 [1986]) relevance theory (henceforth RT) pre-
dicts that human comprehension follows two stages:

i. Following a path of least effort, test interpretive hypotheses (disambigua-
tions, reference resolutions, enrichments, implicatures) in order of acces-
sibility.
ii. Stop when the interpretation satisfies the current expectation of rele-
vance.

For instance, an advertisement by London Transport quoted in Tanaka
(1994) only contained the text Less bread, no jam. The reader of this ad
will follow a path of least effort and conclude, initially, that bread and
Visual metaphor versus verbal metaphor 149
jam have to do with food, but faced with the incongruity between food
and the advertised transport company, the reader will continue testing
interpretive hypotheses concluding, at a second stage, that bread is a collo-
quial word for money and jam refers to traffic jams. The reader will
now be satisfied at this interpretation and stop processing here.
Specifically, for RT, comprehension does not normally start in a commu-
nicative vacuum, but takes place against a context of previous utterances
whose interpretation (stored in the short-term memory) works as a back-
ground against which new information is processed. Cognitive linguistics has
also drawn attention to the role of context for the right comprehension of
metaphors. For example, for Conceptual Metaphor Theory, metaphors may
be activated as part of the hearers understanding of context, and this will
make the interpretation of metaphors easier at subsequent stages in dis-
course. Similarly, conceptual blending theory stresses the role of context in
metaphor comprehension: because cognitive activity mediates the relation-
ship between words and the world, the study of meaning is the study of how
words arise in the context of human activity, and how they are used to evoke
mental representations (Coulson, quoted in Tendahl and Gibbs 2008:
1843).
This cumulative background context of previous utterances in the con-
versation is normally absent in the processing of visual metaphors, which are
inserted in media discourses such as newspapers, billboards or magazines,
and hence the viewers
1
have to interpret them from scratch, without this
readily available short-term memory store of information. This does not
mean that visual metaphors do not require a great deal of background
knowledge for their satisfactory interpretation. The metaphors used in car-
toons, such as the ones analyzed in this chapter, are often related to recent
news-worthy events whose knowledge is essential to get the right extent of
the metaphoric mappings (cf. Peamarn 1996; El Refaie this volume; Schil-
peroord and Maes this volume; Forceville 2005).
RT predicts two clear-cut phases during interpretation: one of decoding
and one of inference. The first one is in charge of the language module of
the mind (Fodor 1983), which apprehends a linguistic sequence and yields a
de-contextualized but grammatical logical form which has to be enriched
in order to be meaningful. By contrast, cognitive linguistics disregards
modularity in favor of what is called the embodied-mind hypothesis, accord-
ing to which the same neural mechanisms used in perception and bodily
movement play a role in all forms of conceptualization, including the crea-
tion of lexical fields and abstract reasoning (Ruiz de Mendoza 2005: 36).
150 Francisco Yus
For RT, language does not encode thoughts, but only clues that help the
hearer access the speakers thoughts, which are often more complex than the
literal meaning encoded by the utterances. For example, the (b) versions of
the following utterances are closer to the thoughts that the speaker intended
to communicate with them than the schematic (and communicatively useless)
(a) versions, the ones actually uttered:

(1) a. Its too wide.
b. The table that I bought yesterday is too wide to go through the door.
(2) a. The cinema is some distance from here.
b. The cinema is too far to go walking.

Secondly, and following a relevance-seeking criterion, the hearers mind
undertakes an inferential process of mutual parallel adjustment of explicit
content, implicatures and context (including the information from preceding
utterances) until a satisfactory interpretation is achieved, at which point
processing stops. In my opinion, despite the apparent differences (see El
Refaie 2003: 8590), this model of utterance interpretation is applicable to
visual and multimodal metaphor comprehension in the same way as to verbal
metaphor comprehension.
As pointed out above, there is always a greater or lesser informational
gap between what the speaker says (what is encoded) and what the speaker
intends to communicate with the utterance. This task often involves an ad-
justment of the conceptual information encoded, that is, interpretation in-
volves the creation of ad hoc concepts during interpretation (see Carston
2002; Pilkington 2000). Since we store many more concepts in our mind
than words to encode them, inevitably there is a greater or lesser amount of
adjustment of encoded concepts needed in order to grasp the speakers in-
tended interpretation. This is applicable to almost every concept, not just the
adjustment of concepts regarding metaphor comprehension. Consider, for
instance, the examples provided in (3) below (Vega-Moreno 2004: 317):

(3) a. The sofa is soft.
b. Baby skin is soft.
c. The cat is soft.

The hearer of (1a-c) is expected to adjust the encoded concept soft into a
more appropriate and contextualized type of softness that specifically ap-
plies to sofas, skins and cats respectively, that is, adjust into more relevant
ad hoc concepts SOFT*, SOFT**, and SOFT***.
2

Ad hoc concepts can be formed either by a process of narrowing of the
encoded concept (what the hearer interprets is more specific than the en-
coded concept) or a process of broadening of the encoded concept (the hearer
interprets more general or broader information than that encoded), all of
them resulting from a relevance-seeking procedure.
3
These processes are
illustrated in (4) and (5) respectively:

(4) a. The fish attacked some people near here (FISH* = dangerous fish, e.g.,
sharks).
b. The boy has a temperature (TEMPERATURE* = higher temperature than
normal).
c. It will take some time to fix the car (SOME TIME* = longer than it usu-
ally takes).
(5) a. The steak is raw (RAW* = undercooked).
b. She is a genius! (GENIUS* = not literally a genius, but having some of
his/her qualities).
c. It was quiet in the street last night (QUIET* = with very little noise).

Within RT, verbal metaphor comprehension also involves a pragmatic ad-
justment (broadening, narrowing, or both simultaneously), and RT suggests
a different approach to metaphor analysis than does cognitive linguistics.
Briefly, the former is more interested in the role of metaphor for communica-
tion and hence in the role of context favoring a certain adjustment of con-
cepts, while the latter mainly focuses on the cognitive motivation for certain
metaphors, their conceptual organization and the inference patterns involved
in their creation. But, as such, the accounts are not mutually exclusive (see
Ruiz de Mendoza and Prez Hernndez 2003; Ruiz de Mendoza 2005; Ten-
dahl and Gibbs 2008).
In Vega-Moreno (2004: 208), three main types of ad hoc concepts are
proposed for verbal metaphors:
(a) Ad hoc concepts which contain qualities which are applicable to all
the prototypical referents of the encoded concepts and also to a range of
other referents, as in (6) below:

(6) A. Why does your boyfriend want you to go with him everywhere?
B. Because he is a baby.
(BABY* denotes a person who cannot be independent, cannot look after
himself, cant do things alone, etc. These are qualities applicable to all
babies (as prototypical referents) and also to some adults such as the
speakers boyfriend).

152 Francisco Yus
(b) Ad hoc concepts which contain qualities which are applicable to some
of the prototypical referents of the encoded concepts and also to a range of
other referents, as in (7) below:

(7) Being the only boy, Dave has always been the prince of the house.
(PRINCE* denotes a subset of princes who are spoilt and do as they
please, as well as a set of young boys who are not princes but are spoilt
and do as they please).

(c) Ad hoc concepts which contain qualities which are applicable to none
of the prototypical referents of the encoded concepts but are applied to other
referents, as in the utterance quoted in (8):

(8) I tried to persuade him to change the essay topic but there was no way.
He is an iron bar.
(IRON BAR* denotes people who are difficult to convince, persuade, etc.,
qualities which are not found in iron bars as prototypical referents).

In my opinion, these three cases are not only inherent to verbal metaphor
comprehension, but are also found in the processing of visual metaphors. In
this sense, case (c) is interesting because it gives rise to the so-called emer-
gent features or emergent properties which apparently do not belong to the
target domain of the metaphor but seem to emerge during comprehension
(Gineste, Indurkhya, and Scart 2000; Wilson and Carston 2006). These
emergent properties might appear to be found only in the interpretation of
verbal metaphors but, as will be argued below, they are also frequent in
visual metaphor comprehension (cf. Yus 2003a) and in any multimodal
combination of text and image.
Many explanations have been suggested for the creation of these emer-
gent properties.
4
I will follow an interesting proposal by Vega-Moreno
(2004) within a relevance-theoretic point of view, and I will argue that this
proposal is applicable, in a similar way and with the necessary adjustments,
to the processing of emergent properties in visual or multimodal metaphors.
More generally, I will show to what extent conceptual assessment is involved
in visual metaphor comprehension, basically through what will be called
stable versus innovative conceptual upload. The analysis will be divided
into several steps that the reader is expected to go through during the inter-
pretation of a visual metaphor. Comparisons with verbal metaphor compre-
hension will be made where necessary, and there is an inherent claim in this
proposal: that combinations of text and image in multimodal metaphors de-
mand similar interpretive procedures involving conceptual adjustment of
encoded information. Variations are only found in the way schematic infor-
mation is obtained by mental modules (in a more holistic way in pictures, in
a more linear way in texts).
3. Stages in visual metaphor comprehension: A proposal
3.1. Perception: Visual versus verbal
My claim is that processing visual metaphors does not differ substantially
from processing verbal metaphors, the main difference being the way in
which the input is transferred to the central inferential processor. RT pre-
dicts, following Fodors (1983) theory of the modularity of mind, a context-
free decoding of a linguistic string by the language module, which sends a
de-contextualized string of linguistic information to the central processor in
order to be enriched inferentially into a fully contextualized (and optimally
relevant) interpretation that supposedly matches the speakers intended one.
Visual information, on the other hand, is decoded by another module: the
perceptual module. The language module and the perceptual module share
similar properties: (a) they are fast and automatic (i.e., they are capable of a
high-speed transference of information, and they are automatically activated
by the appropriate type of input: linguistic and visual respectively); (b)
they are domain-specific (both modules are only activated by a specific type
of input); (c) they are part of our genetic endowment (i.e., they are not
learned and possess an evolutionary quality); and (d) they have a uniform
path of development (unfolding) across individuals and cultures. In short,
these mental modules get activated automatically when the appropriate type
of input reaches them, and both yield de-contextualized pieces of evidence of
the senders intention to communicate some information. This context-free
information is then enriched inferentially in order to obtain a fully satisfac-
tory (i.e., relevant) interpretation of the verbal or visual input.
However, these qualities of modules do not entail that no choices are ever
made during this phase of verbal or visual decoding. For instance, the lan-
guage module often has to choose between two possible logical forms for the
same linguistic string.
5
In the same way, it has been demonstrated that al-
though the perceptual module seems to engage in a one-to-one matching
between object and referent, it also has to make choices as to what visual
information it is actually processing. Specifically, when readers interpret a
visual metaphor, they start by perceiving the image, that is, by identifying
154 Francisco Yus
the visual input. This is done through a subconscious or sub-attentive com-
parison with previously stored information on the visual attributes of the
object or objects depicted (see McMahon 2003; Kriegel 2004). When the
image is supposed to be intentionally communicated to the readers beyond a
simple perceptual recognition, processing moves one step beyond into a more
conscious stage of interpretation, loaded with inferential activity (see be-
low).
6
This mental storage of prototypical referents that we possess is made
up of two basic types of information which undergo a constant process of
updating and stabilization through subsequent visual perceptions:
(a) Prototypical visual referent: encyclopedic entry containing visual
elements and attributes that an item depicted in an image is typically made
of. For instance, the prototypical referent of an image of a cat would contain
visual attributes that are stored as typical of cats (type of hair, colors, ears,
whiskers, paws) and which allows for an easy visual identification.
(b) Prototypical visual syntax: other items typically associated with an-
other object depicted in an image. In general, processing is faster if the vis-
ual arrangement of objects in the image fits our storage of prototypical vis-
ual syntax for these objects, a sort of visual schema that precedes and
influences actual perception (cf. McMahon 2003: 266). In the above cat
example, we will expect to find visual representations of this animal in
specific scenarios with objects forming a prototypical visual syntax regard-
ing its representation (e.g., cat on a branch, on a mat, playing with wool).
In general, as the number of visual features of the image which belong to
the prototypical visual referent increases and its prototypical visual syntax
fits stored schemas, the effort involved in its processing will decrease ac-
cordingly.
7
Highly iconic images are normally filled with features fitting the
prototypical visual referent of the image that the reader possesses, but there
can be other images containing less prototypical features, generating so-
called scales of iconicity.
Besides, visual perception shares a bottom-up and a top-down quality. It
is bottom-up because the reader constructs and integrates the prototypical
visual referents from the available visual elements (as claimed by Gestalt
theory). But at the same time it is top-down because readers test the visual
input against their mental storage of prototypical visual referents, anticipat-
ing, as it were, and even influencing the recognition of the object depicted in
the image.
8
The perceptual module draws on a conceptual repertoire that
contains a range of visual referents and is subject to constant revision and
updating through subsequent visual perceptions of similar images. Percep-
tion is never isolated, and past exposure to objects constrains future percep-
tions (Villafae and Mnguez 1996: 100). Hence, each perception of the
physical object or picture to which the referent is associated helps the reader
to update the prototypical referent that was created when the reader first
perceived it.

3.2. Ad hoc pointers
The previous section on visual perception is important to understand why
certain images are interpreted metaphorically. In my opinion, the key to a
shift from a purely denotative interpretation of the image, often sub-
attentive, to a connotative metaphoric interpretation, loaded with inferential
processing, lies in the detection of an incongruity that turns up between the
activation of the stored prototypical visual referents during perception and
the actual visual configuration of the image or images making up the visual
metaphor (Forceville 1996: 115). This kind of incongruity has been labeled
ad hoc pointer (Yus 2005), in the sense that an ad hoc visual arrangement
or configuration created by the author with specific communicative purposes
points towards a connotative interpretation, alerts the reader towards a con-
notative interpretation, often a metaphorical one. This idea entails an in-
creased mental effort in moving beyond a sub-attentive visual perception into
an effort-demanding inferential activity in search of the right metaphoric
mapping from what we can label the source image to what we can call
the target image.
Of course, as we enter this inferential phase, the reader takes the respon-
sibility of grasping the intended metaphoric interpretation (or his/her own
personal interpretation) and the author of the image can only hope that the
reader will be able to select the appropriate encyclopedic features associated
with the visual referents of the images and infer which are the ones involved
in the metaphoric interpretation (El Refaie 2003: 81; this volume).
Incidentally, there may be visual ambiguity, in which a metaphoric inter-
pretation of the image is intended but a purely denotative interpretation is
also valid, that is, occasions on which there is no apparent ad hoc pointer
and hence the metaphoric interpretation may not be accessed. In these cases,
it is the readers search for an optimally relevant interpretation that will
guide them beyond a purely denotative interpretation. In general, visual
metaphors are integrated in other discourses (images in advertisements, car-
toons in the press) and the readers know that these images are intended to
communicate specific, non-denotative information, and hence they will not
be cognitively satisfied at a purely denotative level.

156 Francisco Yus
3.3. Visual-conceptual interface
Upon detecting the ad hoc pointer, the reader of the image(s) enters another
stage in interpretation, which I will call visual-conceptual interface, in-
between a sub-attentive perception of the images and a fully inferential ex-
traction of a relevant connotative (i.e., metaphoric) interpretation of the im-
age. At this stage, the reader aiming at an optimally relevant interpretation
has to raise a number of preliminary hypotheses concerning the intended
relationship that holds between the depicted images and the encyclopedic
(conceptual) information stored about the referents of these images, mostly
of a stereotypical quality. In short, the readers would ask themselves ques-
tions such as the following:
1. Which are the two images related metaphorically? Are both present in
the picture? Visual metaphor involves a mapping of information transferred
from one image to another, which we have called source image and target
image respectively. Often both images are present in the picture (either fused
together or separated) but sometimes one of them normally the source
image is absent. Therefore, there are different degrees of mental effort
involved in processing visual metaphors depending on whether both the
source image and the target image are depicted in a metaphoric visual con-
figuration, or one of the images is absent and is only accessible through an
inferential operation regarding the encyclopedic information on its prototypi-
cal referent. At the same time, some mental effort has to be devoted to identi-
fying the source and target images in the first place, which are not always
clearly distinguishable, even when both images are present.
2. What kind of visual arrangement is there between the images? The
reader is also expected to infer what relationship holds between the previ-
ously identified source image and target image. Is the target image sup-
posed to be like the source image? Is it opposed to the source image? (cf.
Phillips and McQuarrie 2004.)
3. Are the prototypical encyclopedic referents of the images themselves
the ones that are going to undergo inferential adjustment in order to obtain a
metaphoric interpretation or do the images stand for a different encyclopedic
referent? I believe that visual metaphor comprehension, in a similar way to
verbal metaphor comprehension, also involves an access to and adjustment
of conceptual information stored in or attached to the encyclopedic proto-
typical referents of the image or images depicted. This implies that it is of
utmost importance to determine whether the author intends the most accessi-
ble referents of the images to undergo metaphoric processing or whether the
intended sources of metaphor have to be found elsewhere. This is the case of
images that stand metonymically for other referents. For instance, in the
visual metaphor described in (9), it is the prototypical referent of the image
buildings that works as target domain for metaphoric mapping, whereas in
(10), the visual referent of book page is not expected as target domain,
since it stands for culture in general, which is the intended target domain.

(9) Images depicted: Lorry throwing buildings into a garbage dump.
Metaphor: BUILDINGS ARE RUBBISH (El Roto, El Pas, 296-2003).
(10) Images depicted: A page of a book as a paper serviette inside a dispenser.
Metaphor: CULTURE IS A PAPER SERVIETTE (photograph by Chema
Madoz).

Figure 1. Cartoon by El Roto, El Pas, 6 June 2003.
Similarly, in a cartoon by El Roto (figure 1), a syringe is depicted with a
television tower instead of the needle. An incongruity in the visual syntax of
the image works as an ad hoc pointer alerting the readers to a metaphoric
interpretation.
9
The readers search for relevance will lead them to dismiss
the conceptual features of the prototypical referents of syringe and televi-
sion tower as the ones undergoing metaphoric assessment, and they will
probably infer, instead, that syringe stands metonymically for drugs in
general, and that television tower stands metonymically for television in
general as a mass medium, and the conceptual features of drugs and tele-
vision are the intended source and target of this visual metaphor. The car-
toon also includes the text la gran droga (the big drug), which works as an
anchorage (in Barthes 1977 sense) of the image facilitating the metonymic
relationship between syringes and drugs in general. Using the cognitive lin-
guistics terminology that distinguishes source-in-target metonymies and
158 Francisco Yus
target-in-source metonymies (Ruiz de Mendoza and Dez Velasco 2002), in
this case we would, instead, encounter examples of source-in-source meton-
ymy and/or target-in-target metonymy between prototypical referents depi-
ced and the intended referents. Specifically in figure 1, both the source image
(tower) and the target image (syringe) stand metonymically for the actual
source and target referents undergoing metaphoric interpretation (televi-
sion/drugs).

3.4. Conceptual upload
The ad hoc pointer and the preparatory phase of visual-conceptual interface
lead to a fully inferential stage in the processing of the image or images,
which will be called conceptual upload. Since this stage is centered upon the
inferential assessment and adjustment of conceptual information attached to
the prototypical encyclopedic referents of the images (or the referents in-
tended through metonymy), the distinction between visual and verbal input to
metaphoric interpretation no longer matters (we are now at a cognitive, fully
inferential phase of interpretation). The reader has now entered a fully infer-
ential stage that takes either the information from the verbal utterance or the
identified visual images as blueprints or clues for an optimal metaphoric
interpretation.
At this stage of conceptual upload, and following a relevance-oriented
path of accessibility, the reader will compute conceptual features stored in
the encyclopedic information of the intended referents of the images (either
the prototypical referents of these images themselves, or the referents to
which these images point metonymically, as commented upon above) and
will try to find the ones that can be applied to the other image, a mental pro-
cedure which can be called ad hoc choice of image-associated conceptual
features. This is a similar inferential activity to the one intended to obtain
metaphoric mappings in verbal and multimodal metaphors.
Sometimes this assessment of possible ways in which the images can be
related does not result in any metaphorical outcome despite the visual incon-
gruity, because the reader is unable or unwilling to find any metaphoric con-
nections between these images. On other occasions, though, the readers
search for relevance will lead to a metaphoric interpretation and to a selec-
tion of features which can be mapped from the source image to the target
image. In this sense, two possible types of conceptual upload can be identi-
fied:
(1) Stable conceptual upload of image-associated conceptual features
takes place when interpreting the visual metaphor involves an adjustment of
one or several features belonging to the prototypical encyclopedic referent of
the image depicted. The maker of a visual metaphor fitting this type will
expect all the readers to have a similar store of conceptual information filling
up the prototypical referent of the image or images depicted. But, crucially,
in this type of conceptual upload the encyclopedic feature of the referent
associated with the source image maintains its conceptual stability, and the
reader only has to broaden, as it were, its denotation to include the referent
associated with the target image.
The types 1 and 2 of ad hoc concept formation that were introduced in
section 1 above for verbal metaphors would belong to this type of conceptual
upload:
Type 1. When one or several conceptual features of the prototypical en-
cyclopedic referent associated with the source image that the author intends
the reader to apply to the target image can be found in all the prototypical
referents represented in the source image. In this case, by means of a process
of conceptual broadening, an ad hoc CONCEPT* is created and applied to the
target image. For instance, in the aforementioned cartoon by El Roto depict-
ing a lorry throwing buildings into a huge garbage dump, one or several of
the features of the prototypical encyclopedic referent rubbish (for example
being useless, having no quality, etc.) are mapped onto the encyclopedic
referent buildings associated with the target image. These conceptual fea-
tures form an ad hoc concept RUBBISH*, which is the result of broadening
the prototypical concept rubbish in order to cover the unusual referent
buildings of the target image.
Type 2. One or several conceptual features of the prototypical encyclope-
dic referent of the source image that the author intends the reader to apply to
the target image can be found in some but not all of the prototypical ency-
clopedic referents represented in the source image (or referred to by this
image). Again, a process of conceptual broadening is required so that an ad
hoc CONCEPT* is created.
For example, in another cartoon by El Roto (figure 2) the reader can see
a man on a surfboard sliding on a huge wave, but the wave is made of build-
ings, instead of water, and this unusual wave looks as if it is about to break
on the surfer. The author probably intends to communicate metaphorically
that the urge to build houses in Spain (the construction wave or construction
bubble) will eventually break (or burst) and harm us in the same way as the
huge wave is about to break on the surfer and probably harm him as well.
The harming quality of breaking waves can be found in some (but not all) of
160 Francisco Yus
the prototypical encyclopedic referents of the image depicted, specifically
only those waves which are big enough to break onto surfers and harm them,
and a new ad hoc concept WAVE* is created as a result of an adjustment
(broadening) in order to fit the new encyclopedic referent construction bub-
ble of the target image.

Figure 2. Cartoon by El Roto, El Pas, 19 October 2003.
(2) Innovative conceptual upload of image-associated conceptual fea-
tures. No conceptual features that the author intends the reader to apply to
the referent associated with the target image seem to be found in the proto-
typical encyclopedic referent associated with the source image. These con-
ceptual features are not stabilized in the prototypical referents but arise, in
the same way as do emergent properties in verbal metaphors, as part of the
readers relevance-seeking interpretation procedure. This kind of conceptual
upload fits the third type of ad hoc concept construction introduced in sec-
tion 1 above for verbal metaphors.
The reader will consider encyclopedic features not directly applicable to
the referent of the target image. Inevitably, this mental operation entails the
adjustment of one or several of these features and, as a consequence, they
will be deprived of their conceptual stability when attributed to the referent
associated with the target image. Unlike cases 1 and 2 above, in which the
features were minimally adjusted in order to include the new referent but
maintained their conceptual quality, in this third case a substantial adjust-
ment of the features is required and only by losing their stability can they be
applied to the new referent. In fact, the resulting emergent properties are not
directly applicable to the target, and hence a deep process of adjustment is
required, in the same way as are emergent properties that arise in case 3 of
conceptual adjustment during verbal metaphor comprehension.
For example, another cartoon by El Roto (El Pas, 9 June 2002) depicts a
goal keeper clearing a book (instead of the expected ball) with his fists. In
the metaphor THE BOOK IS A BALL, none of the features of the prototypical
referent ball seems to be applicable to the referent book of the target
image. Whatever metaphoric interpretation the reader chooses (e.g., football
makes people reject books or football stops people from getting real cul-
ture if book stands metonymically for culture) will emerge during the
processing of the referents of the images, and will involve more inferential
effort than a simple broadening of one or several features of the prototypical
encyclopedic referent of the source image in order to obtain an ad hoc con-
cept BALL*. More inferential activity will have to be devoted to adjusting
some feature(s) that belong to the prototypical encyclopedic referent.
Vega-Moreno (2004: 318f) exemplifies this possible explanation of
emergent properties with the verbal metaphor communicated in (11) below:

(11) Jane: I know I have to speak to my boss but I am afraid of him. He is
such a bulldozer!

In this metaphor, our knowledge of bulldozers does not include information
about them being stubborn or disrespectful, and hence this metaphoric qual-
ity of the boss emerges during interpretation. In a nutshell, the hearer can
select, as a starting point, the assumptions that bulldozers are machines and
are used to remove obstacles in their way. These are not directly applicable
to the boss (as it would be in stability-preserving cases 1 and 2 of ad hoc
concept formation). Cognitive linguistics would explain this metaphor by
claiming that there is an underlying conceptual metaphor THE MIND IS A
MACHINE.
10
For RT, though, the hearer creates an ad hoc concept BULL-
DOZER* which involves a radical adjustment of its denotation so that it also
includes a kind of removal, a type of obstacle, and a range of situations that
warrant the derivation of a relevant metaphoric interpretation. In other
words, an ad hoc concept REMOVE OBSTACLES IN THE WAY* is created with
an adjustment which not only applies to machines, but also includes the act
of despising, rejecting, undermining peoples feelings and peoples thoughts.
As will be seen in section 4.2 below, there is a similar conceptual adjustment
in the case of type-three visual metaphors and, in principle, in any multimo-
dal combination of text and image.
162 Francisco Yus
4. Visual metaphor comprehension: Some examples
4.1 Examples of visual metaphor comprehension involving stable
conceptual upload

Figure 3. Cartoon by El Roto, El Pas, 23 June 2002.
(a) THE EARTH IS A SAUCEPAN (figure 3)
1. The reader finds figure 3 in a newspaper and infers that its author in-
tends to communicate some information by means of this wordless cartoon.
2. He perceives the iconic signs of the cartoon by a bottom-up and top-
down matching with previously stored prototypical visual referents of the
item(s) depicted. A saucepan is identified. The continents of the earth are
also identified. These are superimposed on the saucepan.
3. An incongruity arises during the perception of the elements of the im-
age. The earth and a saucepan cannot be fused in one image. There is an
anomalous visual arrangement regarding the mental storage prototypical
combinations of objects depicted together (what above was labeled visual
syntax of the image) that works as an ad hoc pointer that alerts the reader
to an intended metaphoric interpretation beyond the simple depiction and
perception of the drawing in the cartoon.
4. The reader enters a visual-conceptual interface, in which a number of
hypotheses are made and the so-called prototypical visual referents of the
images (already dealt with by the perceptual module) are contrasted with the
parallel prototypical encyclopedic referents of these images.
The readers hypotheses at this stage will prepare the ground for a fully
inferential stage of visual metaphor comprehension, and should include the
following conclusions: (a) the saucepan is the source image; (b) the earth is
the target image (a likely conclusion obtained by the reader in a relevance-
seeking procedure); (c) the encyclopedic referent EARTH has qualities of the
encyclopedic referent SAUCEPAN; and (d) these are the prototypical referents
intended by the author; the images of the earth and the saucepan do not stand
metonymically for other referents.
5. The reader starts computing assumptions in order of accessibility fol-
lowing a relevance-guided procedure, beginning with the ones which belong
to the prototypical encyclopedic referent depicted in the source image:
SAUCEPAN.
6. One of the encyclopedic features of the prototypical encyclopedic ref-
erent SAUCEPAN seems to be directly applicable to the earth: heats up
gradually, since the earth, due to global warming and the so-called green-
house effect is also heating up gradually. This involves the creation of an
ad hoc concept SAUCEPAN* whose quality heats up gradually remains
relatively stable in the metaphoric process (i.e., undergoes a minimal ad-
justment via broadening). This new concept is applicable to all saucepans
and also metaphorically to the new encyclopedic referent EARTH. In this
sense, this could perhaps be a case of ontological metaphor in Lakoff and
Johnsons (1980) terminology, since in this case an inherent quality of
saucepans is attributed to the target image. It would also fit Ruiz de Men-
dozas (1998) one-correspondence metaphor, since in this case only one
correspondence between the source and the target is exploited.
This visual metaphor would fit case 1 of ad hoc concept formation, in
which the intended conceptual feature is found in all the prototypical refer-
ents depicted by the image plus a number of other entities included through
broadening.
7. The presence of SAUCEPAN* in what can be called the explicit content
of the referents attached to the items depicted in the cartoon warrants the
derivation of a number of possible implicated conclusions. Cartoons are a
good example of a medium in which current news-worthy events play a part
in the generation of implicated conclusions. In this case, if the reader knows
about the fact that, at the time the cartoon was published, there was a debate
on the Kyoto protocol and whether Japan and Australia would sign it (i.e., if
this information is manifest to him/her, in RT terms), this information will
influence both the accessibility to the visual metaphor and the mental effort
devoted to its processing.
(b) THE BALLOT BOX IS A DICE (figure 4)
Steps 1. and 2. as above.
3. The reader enters a visual-conceptual interface, in which a number of
hypotheses are made concerning the encyclopedic referents of the images,
once the prototypical visual referents have been perceived, again preparing
164 Francisco Yus
the ground for a fully inferential stage. Some conclusions should be derived:
(a) the dice is the source image; (b) the dice stands metonymically for gam-
bling with dice and more generally for all types of gambling (and hence
the encyclopedic referent intended by the author to undergo metaphoric
transference is not DICE, but GAMBLING); (c) the ballot box is the target
image (facilitated by the readers background knowledge about the fact that
the cartoon was published in a time of political elections); (d) the ballot box
is in a metonymic relationship to political elections (and hence the referent
intended by the author to undergo metaphoric transference is not BALLOT
BOX, but POLITICAL ELECTIONS in general or more specifically the ones tak-
ing place in the near future); (e) the political elections have qualities of gam-
bling.

Figure 4. Cartoon by El Roto, El Pas, 19 February 2005.
4. The reader starts computing assumptions in order of accessibility fol-
lowing a relevance-guided procedure, beginning with the ones which belong
to the prototypical encyclopedic metonymic referent of the source image:
GAMBLING.
5. One of the encyclopedic features of the prototypical referent gam-
bling seems to be directly applicable to political elections: involves an
unpredictable outcome. Strictly speaking, this cartoon should be included in
type 1, if we consider that all dice involve unpredictability. However, we
may also hypothesize that since there is also a kind of gambling that gener-
ates a wholly predictable outcome (for instance gambling involving the use
of loaded dice which always produce the desired result) in this case not all
the referents for image of a dice would contain this quality. In this more
unlikely case, the reader would be expected to create an ad hoc concept
GAMBLING* whose feature would involve an unpredictable outcome appli-
cable to most (but not all) kinds of gambling and also to other unpredictabil-
ity-loaded events such as political elections. Consequently, this visual meta-
phor would now fit case 2 of ad hoc concept formation during visual meta-
phor comprehension.
6. The presence of GAMBLING* in what can be called the explicit content
of the image depicted in the cartoon warrants the derivation of a number of
possible implicated conclusions, some strongly implicated (e.g., the unpre-
dictability of Spanish elections) and other weakly implicated (e.g., the more
emotion-connoted implication that no Spanish party is trustworthy).

4.2. Example of visual metaphor comprehension involving innovative
conceptual upload
(a) THE TENNIS RACKET IS A SHARKS JAW
Phillips and McQuarrie (2004: 123) reproduce an advertisement for a tennis
racket in which a sharks jaw has been superimposed on a racket frame.
How would the audience process this ad? Basically steps 1 to 3, as proposed
for the previous examples, would also apply here. The unusual fusion of
images and the anomalous visual syntax (i.e., bringing together visual ele-
ments racket and jaw which are not stored as part of the prototypical
arrangement of objects such image can normally be made of or surrounded
by) work as an ad hoc pointer to a non-denotative interpretation of the im-
ages.
4. The reader then enters the so-called visual-conceptual interface, in
which a number of questions are asked regarding the relationship between
the images depicted (already obtained via perception) and the encyclopedic
information attached to them. Among others, some conclusions that the
reader would be expected to derive are these: (a) the jaw is the source image;
(b) the racket is the target image (a conclusion facilitated by the context of
the advertisement, in which it is made clear that the author of the ad intends
to characterize the racket in some way, so that it is eventually purchased);
(c) the jaw stands metonymically for SHARKS in general; (d) the racket
stands metonymically for THE PLAYERS SKILL; (e) the players skill acquires
qualities of a shark (aggressiveness, fearful attack).
5. The reader then starts computing assumptions in order of accessibility
following a relevance-guided procedure, starting with the ones listed in the
prototypical encyclopedic referent made accessible by the source image:
SHARK.
6. Again, although intuitively the reader can feel that the intended inter-
pretation has to do with some form of aggressiveness, there is nothing in the
166 Francisco Yus
behavior of sharks that can be directly applicable to a tennis players skill
(although, as a stereotypical feature of sharks, it would also be labeled as
ontological metaphor, in Lakoff and Johnsons 1980 terminology), and
therefore all the eventual transference of information will inevitably involve
emergent properties arising in the relevance-guided comprehension proce-
dure. Whatever conceptual information ends up being applied to the ency-
clopedic referent associated with the target image will involve a loss in the
stability of the storage of this information.
7. The readers then assess qualities of the encyclopedic referent associ-
ated with the source image and will adjust their denotation drastically in
order to fit not only the aggressiveness of an animal, but also the aggressive-
ness of a tennis player. These qualities include the information that sharks
are aggressive animals. These are possible constituents of the ad hoc con-
cept SHARK* but not directly applicable to playing skill, so the reader has to
adjust (broaden) the quality AGGRESSIVE* that belongs to SHARK* so that it
also covers the tennis players skill. Other features such as the way sharks
chase and attack their preys may also be adjusted (depending on the
readers willingness to devote additional mental resources to this task), lead-
ing to the ad hoc concepts CHASE* and ATTACK*.
8. These adjustments are necessary in order to draw the implicated con-
clusion that the user of this racket will play with an unusual degree of ag-
gressiveness. As above, the adjusted concepts warrant the derivation of a
number of possible implicated conclusions.
5. Verbal before visual or vice versa?
From the examples analyzed in the previous section, we can conclude that
interpreting visual metaphors also involves a great deal of conceptual upload
and adjustment following a criterion guided by a search for relevance. In this
sense, it is worth commenting that many visual metaphors are original in the
way they create a metaphoric link through an anomalous visual arrangement,
while others seem to include an anchorage of previously used verbal meta-
phors which are simply transferred to a visual medium and were probably
stored previously as conventionalized metaphors. In these cases, the meta-
phor-seeking conceptual assessment can indeed be speeded up by the fact
that a particular feature of the prototypical encyclopedic referent of the
source image has been made prominent by previous use through verbal
means, or even facilitated by the fact that the visual metaphor only exists
because there is an underlying verbal one. In a way, this is the counterpart of
the well-studied fact that verbal metaphors also draw on a conceptual reper-
toire of visual sensory schemas which aid in the metaphoric attribution, to
the extent that these images often end up becoming conventionalized in the
language and deprived of their sensory metaphoric power.
11

Therefore, it comes as no surprise that so many metaphors involve the
mediation of visual sensory information contained in image schemas (see
Lakoff 1987), or involve what can be called re-visualization of convention-
alized verbal metaphors, which is extensively used by cartoonists. Several
steps are involved in the comprehension of the cartoons that fit this quality:
(a) initially, an image is much more effective (i.e., vivid) than the range of
coded options available to communicate a thought. (b) A metaphor is created
that contains a schema as a referent. This schema contains visual sensory
information.
12
(c) Repeated use of the metaphor makes it lose its sensory
vividness and it ends up becoming conventionalized and hence people stop
regarding it as a metaphor. (d) The cartoonist takes this conventionalized
metaphor and re-visualizes it, as it were, forcing the reader to re-incorporate
into its processing all the sensory vividness that the metaphor had already
lost.
An example is a cartoon drawn by El Roto (El Pas, 15 April 1996) de-
picting the earth split into two parts, and with a big gap between the North
and the South hemispheres. There are people trying to jump from the South-
ern hemisphere onto the Northern one but they inevitably fall into the huge
gap. This metaphor, which can be described as there is an abyss between
the North and the South, reproduces the aforementioned steps: (a) the sen-
sory information of an abyss is more vivid than other coded options to com-
municate depth and distance between A and B; (b) a metaphor THE DIFFER-
ENCE BETWEEN NORTH AND SOUTH IS AN ABYSS is created containing visual
sensory information; (c) the metaphor ends up becoming conventionalized
and loses its sensory power (people stop seeing an abyss when uttering the
metaphor); (d) the cartoonist re-visualizes the information contained in the
metaphor, forcing the reader to see the sensory qualities of the image schema
that had been lost due to conventionalization.
Interpreting visual metaphors does not differ substantially from verbal meta-
phor comprehension. Both kinds of metaphor are decoded by a specialized
mental module (Fodor 1983) which delivers schematic information that has
to be enriched inferentially in order to obtain the intended interpretation (an
168 Francisco Yus
optimally relevant one). The de-contextualized perception of images is not
relevant enough (some incongruity in the image or images works as an ad
hoc pointer directing the viewer towards a metaphorical interpretation) and
the reader has to engage in subsequent interpretive steps involving the access
to encyclopedic information either directly related to the referent depicted in
the image, or made prominent by metonymic relationship to other encyclope-
dic referents. At this stage, the reader will adjust the conceptual information
in his or her search for relevance and will generate appropriate ad hoc CON-
CEPTS*. This is the same kind of conceptual adjustment that takes place in
the interpretation of verbal metaphors. In short, conceptual information has
to be accessed and adjusted in any type of metaphor. It is only the type of
decoded input feeding the inferential processor that makes a difference. In-
deed, the mode in which the reader is presented with the coded information
that has to be adjusted inferentially plays a major role in the quantity and
quality of metaphoric conclusions derived. Normally, pictures have a more
powerful impact on the reader due to their holistic gestalt-like processing and
are good for visualizing conventionalized concepts such as abyss in the
example above. Utterances, on the other hand, are linear, and readers make
interpretive hypotheses as text is processed in a word-by-word integration
into phrases and sentences, which entails differences in the way literal and
implicated meanings are generated.
Notes
1. Since all the visual metaphors analyzed or referred to in this article are taken
from newspapers, from now on the viewer of the visual metaphor will be re-
ferred to as reader.
2. I will use the convention of adding asterisks to stress the fact that the accom-
panying word is an ad hoc concept and not an encoded concept.
3. In fact, context plays an important role aiding the addressee in determining
the kind of CONCEPT* that the speaker or the author intends. Normally, in the
course of a conversation, previous utterances and background knowledge
about the speaker work as an important short-term-memory storage of in-
formation against which new utterances are interpreted. Vega-Moreno (2004:
317) explains this with the metaphor my boss is a shark. If it is clear from
previous turns of the conversation or from general encyclopedic information
about the speaker that he is happy with his boss, the concept AGGRESSIVE
may be adjusted to denote a kind of (positive) aggressiveness that involves
energy and assertiveness (represented as AGGRESSIVE*). However, processing
the metaphor on the assumption that the speaker is afraid of his bosss tactics
and techniques, the concept AGGRESSIVE would be adjusted to denote a kind
and level of (negative) aggressiveness (AGGRESSIVE**).
4. One famous explanation is provided by the so-called conceptual blending
theory (Fauconnier and Turner 1998). It suggests that a subset of the attrib-
utes and relational structure from the source and target domains are imported
into a blended space where they can be combined and supplemented with in-
formation from encyclopedic knowledge. These hybrid models, or blends, are
useful in explaining emergent properties.
5. The existence of a choice of two possible logical forms for the same linguistic
input is often exploited by humorists in some of their jokes. In Yus (2003b:
1304), it is claimed that the source of humor in some jokes lies in the fact
that the language module of the addressee has to choose between two possible
de-contextualized logical forms extracted from the humorists utterance, as in
the following example: Postmaster: Heres your five-cent stamp. Shopper
(with arms full of bundles): Do I have to stick it on myself? Postmaster:
Nope. On the envelope.
6. It is utterly important to identify the image as intentionally communicated
(ostensive in RT terms) and not simply as visual information unintentionally
exuded, as it were, from the environment. The former carries a presumption
of eventual relevance which the latter lacks. This stage of intention ascription
is important because it constrains all the subsequent inferential activity de-
voted to the processing of the visual stimulus and the amount of effort that
the reader will be willing to devote to this inference beyond a purely sub-
attentive identification of the image.
7. In fact, there is experimental evidence that the visual system uses principles
of coherence to detect whether the visual information corresponds to a unique
object or belongs to separate, interrelated objects, and different specialized
brain cells are devoted to these tasks (see Humphreys and Heinke 1998).
8. The fact that we normally perceive only one interpretation very rapidly indi-
cates that we see far more than the immediate information falling on our ret-
ina. The highly accurate guesses and inferences that we make rapidly and
unconsciously are based on a wealth of knowledge of the world and our ex-
pectations for the particular scene we are seeing (Cavanagh 1998).
9. The change in mode from drawing to photographic style in this example may
indicate that the visual syntax not only points towards the metaphor but also
the humor of the vignette by the unexpected association between the build-
ing and the syringe. Perhaps the decoding of the photographic mode versus
the drawing modes connected in this picture provoke the search for extra im-
plicit meanings besides the anomalous visual arrangement. I would like to
thank E. Urios-Aparisi for pointing this out to me.
10. Specifically, what guarantees the interpretation is the existence of an under-
lying conceptual mapping from bulldozer to human being whereby we
understand [the bosss] behavior in terms of the figurative behavior (i.e. the
170 Francisco Yus
way the machine functions) that we attribute to a bulldozer (Ruiz de Men-
doza and Prez Hernndez 2003: 29).
11. Tendahl and Gibbs (2008: 1834) assert that the motivation for metaphorical
language is found in recurring sensorimotor patterns of experience that are
continually enacted as neural processes in the moment of thinking, speaking,
and understanding. Such recurring sensorimotor patterns at least motivate
the existence and continued use of many conventional metaphors and some
novel extensions or elaborations of these in creative metaphorical language.
12. As correctly argued within cognitive linguistics, this is an example of the
typical mental operation that helps people conceptualize vague or abstract
domains of knowledge in terms of more specific and familiar knowledge such
as the one provided by sensory input.
References
Barthes, Roland
1977 Image, Music, Text. London: Fontana.
Carston, Robyn
2002 Thoughts and Utterances: The Pragmatics of Explicit Communica-
tion. Oxford: Blackwell.
Cavanagh, Patrick
1998 Top-down processing in vision. In Encyclopedia of Cognitive Sci-
ence (MITECS), 839840. Cambridge, MA: MIT Press.
2003 Understanding visual metaphor: The example of newspaper car-
Fodor, Jerry
1983 The Modularity of Mind. Cambridge, MA: The MIT Press.
Forceville, Charles
2005 Addressing an audience: Time, place, and genre in Peter van
Straatens calendar cartoons. Humor 18: 247278.
work: Agendas for research. In Current Applications and Future
Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and
Francisco Ruiz de Mendoza (eds.), 379402. Berlin/New York:
Mouton de Gruyter.
Symbol 15: 117135.
Humphreys, Glyn, and Dietmar Heinke
1998 Spatial representation and selection in the brain: Neuropsychologi-
cal and computational constraints. Visual Cognition 5: 147.
Kriegel, Uriah
2004 Perceptual experience, conscious content, and non-conceptual con-
tent. Essays in Philosophy 5 (1): 114.
Lakoff, George
1980 Metaphors We Live By. Chicago: The University of Chicago Press.
McMahon, Jennifer A.
2003 Perceptual constraints and perceptual schemata: The possibility of
perceptual style. Journal of Aesthetics and Art Criticism 61 (3):
259272.
Peamarn, Cristina
1996 El humor grfico y la metfora polmica. La Balsa de la Medusa
3839: 107132.
2004 Beyond visual metaphor: A new typology of visual rhetoric in adver-
Pilkington, Adrian
2000 Poetic Effects: A Relevance Theory Perspective. Amsterdam/
Ruiz de Mendoza, Francisco
1998 On the nature of blending as a cognitive phenomenon. Journal of
Ruiz de Mendoza, Francisco
2005 Linguistic interpretation and cognition. In Cultural Matrix Re-
loaded. Romanian Society for English and American Studies. Sev-
enth International Conference, Elena Croitoru, Daniela Tuchel and
Michaela Praisler (eds.), 3664. Bucarest: Didactica Si Pedagogica.
Ruiz de Mendoza, Francisco, and Olga I. Dez Velasco
Comparison and Contrast, Ren Dirven and Ralph Prings (eds.),
Ruiz de Mendoza, Francisco, and Lorena Prez Hernndez
2003 Cognitive operations and pragmatic implication. In Metonymy and
Pragmatic Inferencing, Klaus-Uwe Panther (ed.), 2349. Amster-
172 Francisco Yus
Schilperoord, Joost, and Alfons Maes
this vol. Visual metaphoric conceptualization in editorial cartoons.
1995 [1986] Relevance: Communication and Cognition. Oxford: Blackwell.
Tanaka, Keiko
1994 Advertising Language. A Pragmatic Approach to Advertisements in
Britain and Japan. London/New York: Routledge.
Tendahl, Markus, and Raymond W. Gibbs
2008 Complementary perspectives on metaphor: Cognitive linguistics and
relevance theory. Journal of Pragmatics 40: 18231864.
Vega-Moreno, Rosa
2004 Metaphor interpretation and emergence. UCL Working Papers in
Linguistics 16: 297322.
Villafae, Justo, and Norberto Mnguez
1996 Principios de Teora General de la Imagen. Madrid: Pirmide.
Wilson, Deirdre, and Robyn Carston
2006 Metaphor, relevance and the emergent property issue. Mind and
Language 21 (3): 404433.
Yus, Francisco
2003a Conceptos ad hoc en el procesamiento de la metfora. El caso de las
vietas humorsticas en la prensa, paper delivered at Jornadas sobre
Texto/Imagen, Ciudad Real (Spain): University of Castilla-La Man-
cha, December.
2003b Humor and the search for relevance. Journal of Pragmatics 35:
12951331.
2005 Ad hoc concepts and visual metaphor? Towards relevant ad hoc
pointers, paper delivered at the 9th International Pragmatics Con-
ference, Riva del Garda (Italy), July.
Chapter 8
Metaphor in political cartoons: Exploring audience
responses
Elisabeth El Refaie
Abstract
Using data from a study of young peoples responses to British newspaper car-
toons, this chapter considers the ways in which readers interpret multimodal
metaphors of the verbo-visual variety. One of the central tenets of Conceptual
Metaphor Theory is that many metaphors derive from our bodily experience and
are thus likely to be understood in similar ways by all human beings. But in fact
there is increasing evidence that the interpretation of metaphors is partly depend-
ent upon peoples socio-cultural background, as well as on the contexts in which
the metaphors are used. The results of our study suggest that some metaphorical
mappings in cartoons, such as those between size and power/status, and between
movement through space and the passing of time, might be understood more gen-
erally and at a more intuitive level than more elaborate structural metaphors,
which tend to be interpreted in different ways by different individuals.

Keywords: audience research, embodiment, newspaper cartoons, verbo-visual
metaphor
1. Introduction
This chapter analyzes the use of multimodal metaphors of the verbo-visual
variety in political cartoons and explores the ways in which such metaphors
are understood by viewers from different backgrounds. The discussion draws
on data from a study of young peoples responses to British newspaper car-
toons about the 2004 US presidential elections.
1

Political cartoons offer a good opportunity to explore multimodal meta-
phor, because metaphor is a very common device used by cartoonists (Ed-
wards 1997; Philippe 1982; Morrison 1992; Templin 1999) and most car-
174 Elisabeth El Refaie
toons combine visual and verbal codes. As I argue in section 2, using two
cartoons from our study as examples, newspaper cartoons typically act as a
bridge between fact and fiction, combining actual current events with an
imaginary, make-believe world created by the cartoonist (Edwards 1997: 8).
While this metaphorical process of transferring meaning from the make-
believe to the real world tends to be conveyed predominantly in the visual
mode, most cartoon metaphors also rely to some extent on verbal cues.
Sometimes either the target or the source is represented exclusively through
language, but more frequently verbal labels in cartoons are used as a means
of specifying important aspects of a primarily visual metaphor. In the case
of political cartoons, Forcevilles (2006: 384/this volume) definition of mul-
timodal metaphors as metaphors whose target and source are each repre-
sented exclusively or predominantly in different modes thus needs to be
interpreted in a way that also embraces such asymmetrical verbal-visual
relationships.
Section 3 explores the issue of universality versus individuality in the un-
derstanding of metaphors in cartoons. Conceptual Metaphor Theory is based
on the proposition that metaphor derives from our bodily experience and is
thus an essential part of our everyday patterns of thinking. This suggests that
most instances of metaphor will be understood in similar ways by all mem-
bers of a language community. But in fact it is becoming increasingly clear
that the choice and interpretation of metaphors is partly dependent upon the
participants social and cultural background (Kvecses 2005; Proctor, Proc-
tor, and Papasolomou 2005), as well as on the specific contexts in which the
metaphors are encountered (Ritchie 2004).
In section 4, I describe the data and methods used in our study of young
peoples readings of political cartoons, including the measures we took to
ensure that we did not pre-empt their responses. I also explain our decision
to consult the makers of the two cartoons used in the study, Nicholas Gar-
land and Peter Schrank, about their intentions.
2
Although we do not consider
the artists to be the ultimate arbiters of the meaning of their work, it proved
revealing to compare the intended meanings with the analysis of the cartoons
by the author of this chapter and with the interpretations generated by the
young people participating in the study.
The fifth section of this chapter discusses some of the results of our
study. The main focus is on the way readers recognize and interpret multi-
modal metaphors, but this cannot always be separated from more general
considerations of how people read visual meaning and how their world
knowledge influences the interpretation process. The discussion of the data
therefore sometimes goes beyond the issue of multimodal metaphor in the
Metaphor in political cartoons 175
narrow sense and includes some reflections on wider issues of cartoon inter-
pretation.
In the final section I reconsider the results and propose ideas for further
research.
2. Bridging the gap from fiction to fact
If, as most researchers now accept, metaphors operate at the level of thought
rather than being merely linguistic, then any form of communication can be
seen as an instance of metaphor if it is able to induce a metaphoric thought
or concept (Kennedy, Green, and Vervaeke 1993: 244). Since the early 90s,
researchers have been discovering manifestations of metaphor in various
non-verbal modes, thereby providing additional evidence for the existence of
metaphorical thought patterns (Seitz 1998). It has also been shown that
metaphors can be cued in more than one mode simultaneously (Forceville
2004; 2006).
However, the search for commonalities must not distract from potential
variations in meaning arising from the genre in which a metaphor occurs. As
Sol Worth ([1974] 1981: 161) pointed out in an early discussion of visual
metaphor, [i]t is the fact that we learn the agreed-upon rules for the inten-
tional creation of meaning within specific contexts that makes metaphor
possible. Therefore, genre is likely to have an important influence on the
choice of metaphors by producers, the form these metaphors take, and the
ways in which they are recognized and interpreted by audiences.
Most of the research on visual and multimodal metaphor has so far fo-
cused upon its use in advertising (Forceville 1996; Kaplan 1992; Messaris
1997; Phillips 2003; Scott 1994), where the communicative purpose is obvi-
ous: to attract the attention of potential customers and create (implicit) cog-
nitive links between the product and some desirable abstract quality. Be-
cause of this, visual metaphors in advertising are often highly creative and
unusual. Researchers have tended to focus on this level of explicit meta-
phorical meaning and to disregard the issue of whether adverts also contain
more basic orientational and ontological metaphors, which are thought to
structure human perception and experience at a very fundamental level (La-
koff and Johnson 1980; Kvecses 2002).
Clearly, in the case of political cartoons we are dealing with a completely
different genre, with its own distinctive styles, conventions, and communica-
tive purposes. A political cartoon is an illustration, usually in a single panel,
published on the editorial or comments pages of a newspaper. Generally, the
purpose of a political cartoon is to represent an aspect of social, cultural, or
political life in a way that condenses reality and transforms it in a striking,
original, and/or humorous way. The field of politics is often complex and
bewildering, and cartoons offer a way of explaining the significance of real
life events and characters through the means of an imaginary scenario. As
Edwards puts it, in cartoons people and events are depicted as something
that they are not in order to arrive at a new definition of what they are (Ed-
wards 1997: 128). The frame around a cartoon functions as an implicit
metacomment, signaling to the newspaper reader that it is to be viewed as
part of the dramatic cartoon world, as opposed to the real world of serious
news reports, commentaries and newspaper photographs (Baldry and Thi-
bault 2006: 17). Although cartoons often depict clearly ludicrous situations,
they draw on readers real-life experiences and rely on their wider interpre-
tive competences (Dines-Levy and Smith 1988: 244). A generic convention
of cartooning, in contrast to advertising, is that the goal is generally to ex-
pose something bad or shameful rather than to highlight the positive. Not
surprisingly, cartoonists will thus often fall back on stereotypes and system-
atic metaphorical concepts that represent the complexity of the world in
simpler and often very negative terms.
Orientational metaphors, which link spatial orientation with more ab-
stract meanings, seem to be particularly common in political cartoons. Ac-
cording to Lakoff and Johnson (1980: 14ff), our physical and cultural ex-
perience of the up-down orientation, for instance, motivates a whole range of
common metaphors, including HIGH STATUS IS UP/LOW STATUS IS DOWN and
HAVING CONTROL OR FORCE IS UP/BEING SUBJECT TO CONTROL IS DOWN. In
cartoons, size is commonly used to indicate the relative salience or impor-
tance of the various elements (El Refaie 2003: 85), and, since a difference in
size is closely linked to up-down orientation, it can also imply power differ-
entials. In her analysis of US newspaper cartoons about the 88 Primaries,
for instance, Edwards (1995) found that the Democratic candidates were
frequently represented as the Seven Dwarfs, which very effectively conveyed
the message that they were all equally irrelevant, weak and powerless.
One of the cartoons we used in our study (figure 1) is another good ex-
ample of how size can be imbued with a metaphorical meaning. As cartoon-
ist Peter Schrank explained (personal communication, June 2005), he in-
tended the giant boot to express the utter indifference of the US
administration toward the UK; Tony Blair had been used by them, but hes
powerless. I like the way the cowboy boot as a simple object represents the
whole administrations and current US cultures attitude. In this case the
intention was thus to use exaggerated size to signify the abuse of power and
force, rather than simply to increase the salience of this visual element.

Figure 1. Peter Schrank, Independent on Sunday, 15.10.2004
Some authors believe that there are significant differences regarding what
the visual and the verbal mode are able to convey effectively (Messaris
1994; Kress 2000). For instance, the visual mode differs from language in
that it is simply not possible to represent abstract meaning visually without
recourse to symbols, metonyms, or metaphors (El Refaie 2003; Forceville
2005). So, in our example the concept of an unequal relationship between
the British Prime Minister and the US President could not have been ex-
pressed literally in pictorial form at all and had to be translated into a meta-
phorical image. Conversely, because images always represent a particular
instance of someone or something, they are more specific than words, cap-
turing nuances of meaning that would be hard to convey through language.
Moreover, images often evoke profound emotional responses which are hard
to explain and of which the viewer may not always be entirely conscious
(Zakia 2002: 233255). The precise look of the cowboy boot and Tony
Blairs facial expression in figure 1, for instance, speak volumes about the
cartoonists opinion of both politicians.
The cartoon in figure 2, which was published on the day of the 2004 US
Presidential elections, uses size and foregrounding to indicate salience: even
at a glance, we can recognize that the larger-than-life matchbox is clearly
essential to the argument the cartoonist is trying to make. The central meta-
phor in this cartoon is based on the representation of George Bush as a small
child, which could be verbalized as BUSH IS TODDLER. In formal terms, this
can be described as a monomodal metaphor of the pictorial variety, or, more
specifically, as a hybrid (Forceville 1996: 163) or fusion (Carroll 1996)
metaphor, where the target and the source are visually amalgamated into one
spatially bounded object. Such fusion is very common in political cartoons,
since caricatures often overlay the features of a famous personality onto any
imaginable being or object (Bell 2004). Provided the caricatured personality
is recognized, this type of visual metaphor should work perfectly well with-
out the support of a verbal label. Here the face of George Bush, the target of
the metaphor, is visually amalgamated with the body of a toddler, which
represents the source. Once both target and source have been identified, the
reader is invited to map properties of a prototypical toddler (playful, irre-
sponsible, stubborn, impetuous) onto the American president.

Figure 2. Nicholas Garland, Daily Telegraph, 2.11.2004
However, this example demonstrates that newspaper cartoons are typically
about more than just the characteristics of a person. Rather, they tend to
represent a particular situation, event or action in terms of something else,
which in this case was paraphrased by the artist Nicholas Garland (personal
correspondence, June 2005) in the following way: It is dangerous to give a
small child a box of matches to play with President Bush is not to be
trusted to be sensible and wise in my opinion. This concept is clearly more
complex, and it requires the cartoon to be read as a narrative, representing
not just participants, but also events and temporal sequence.
Although still images are not particularly suited to the task of expressing
action and events (Kress 2000), even single-panel images such as political
cartoons are able to convey some narrative meaning through the depiction of
movement that is frozen in the instance of representation (Schirato and
Webb 2004: 87). This sense of activation can be increased by the use of
vectors, strong, often diagonal lines formed obliquely by depicted objects or
people, which indicate the direction of an action (Kress and van Leeuwen
1996: 4378) and, in some cases, through conventionalized motion lines
(Horn 1998: 136) leading to or from a moving element. The cartoonist thus
relies on every readers ability to complete in his or her head what is sug-
gested by an image, including the actions that precede and follow the de-
picted moment (Edwards 1997: 53).
Since static images are also unable to express chronology directly, car-
toonists are bound to use spatial relations to indicate the passing of time.
According to Lakoff and Johnson (1980: 4144), we are in any case used to
thinking about time in terms of space, with the future in front of us, the pre-
sent right by us, and the past behind. We imagine the passing of time in two
different ways: in the TIME IS A MOVING OBJECT metaphor, we are facing
toward the future, which creeps up on us and which we have to meet head
on. However, time can also be conceptualized as a stationary landscape
through which we move in the direction of the future (were approaching the
end of the year). Although these two metaphors are not consistent, they are
nevertheless coherent by virtue of being special cases of the same underlying
metaphor TIME PASSING IS MOTION (Lakoff 1993: 217).
In the two cartoons shown above, action is implied through frozen
movement and strong diagonal vectors, while chronology is suggested
through the relative position of the main active participants, who, in accor-
dance with the MOVING OBSERVER metaphor, are shown traveling through
time from the past to the future. In figure 1, the past is in the foreground and
the future in the background, so that the viewer is facing in the same direc-
tion as the main participants and, like them, is confronted with the prospect
of an uncertain future. The image of someone walking along and, reaching a
fork in the road, having to decide which way to go, is an instantiation of the
LIFE IS A JOURNEY metaphor, which has been described as one of the most
conventional ways of conceptualizing the link between space and time in
Western cultures (Lakoff 1993). Perhaps partly because of the conventional-
ity of this metaphor, which is firmly rooted in the embodied source-path-
goal schema (Johnson 1987), visual information alone may suffice to cue
the idea of two destinations representing future decisions.
However, a more specific reading of this cartoon, which could perhaps be
verbalized as FUTURE US FOREIGN POLICY IS BOOT CRUSHING OTHER COUN-
TRIES, requires more detailed information about the various slots in this
schema; this additional information is at least partially provided by verbal
tags. The slot of moving observer, for instance, is further specified
through a doubly coded metonym: Since the American President is fre-
quently referred to as W in order to distinguish him from his father, the
use of this letter in the cartoon is likely to support the reading of the cowboy
boot as a visual metonym for Bush, a reading principally triggered by his
Texan background and his famous predilection for Western-style casual
dress. The detailed drawings of the wall and nuclear power station, meto-
nyms for Irans controversial nuclear program and the long-running territo-
rial conflict between Israelis and Palestinians, provide additional information
about the precise goals of the boot, as do the visual symbols of the vulture
and the dove, representing war and peace respectively. The verbal elements
on the signs pointing to Israel + Palestine and Iran + North Korea pro-
vide important support in the identification of these visual elements in the
background, which may otherwise be difficult to recognize. This example
clearly demonstrates that reading political cartoons often requires readers to
draw on several different kinds of interpretive strategies, including the ability
to recognize visual symbols and the targets of metonyms.
Similarly, in figure 2 the basic interpretation of a conceptual link between
space and time involves only the visual mode and is not dependent upon
verbal information. Again, the image of the toddler crawling towards some-
thing cues the MOVING OBSERVER metaphor, although here the orientation is
reversed, so that the background of the image represents the past and the
foreground the future. This places the viewer in a knowing position where,
in contrast to Bush-the-toddler, he or she is already able to see only too
clearly what the next few years are likely to bring.
This general orientational schema provides the basis for a more complex
structural metaphor, which we might verbalize as ACHIEVING RE-ELECTION
IS GETTING HOLD OF A BOX OF MATCHES. In this case, only the source is
visually present, while the target must be gleaned from the pictorial context,
any verbal clues, or more generally the world knowledge of the viewers
(Forceville 1996: 109). The inscription on the box of matches is instrumental
in pointing the viewer towards a more exact interpretation of the general
SPACE-IN-FRONT IS FUTURE metaphor, since the precise concept of SPACE-
IN-FRONT IS FOUR MORE YEARS IN OFFICE could not have been rendered
exclusively in the visual mode and needed to be supplied partially as a verbal
message.
3
In this context, the box of matches thus denotes Bushs bid to be
re-elected and the opportunity this would offer him to cause even more
havoc, while the flames are clearly meant to represent some of the Presi-
dents foreign policies, such as the invasion of Iraq and the launch of his so-
called war on terror, which by the time of the cartoons publication were
already being seen by many as disastrous.
In the case of the complex structural metaphors FUTURE US FOREIGN
POLICY IS BOOT CRUSHING OTHER COUNTRIES and ACHIEVING RE-ELECTION
IS GETTING HOLD OF A BOX OF MATCHES I believe we are dealing with ex-
amples of true multimodal metaphors, even though, strictly speaking, they
are not covered by Forcevilles (2006: 384) definition, which stipulates that
in multimodal metaphors target and source must be represented exclusively
or predominantly in different modes. In our examples, the verbal tags offer
additional but nevertheless essential information, which helps the reader
identify the precise source or target of a complex metaphor. In order to em-
brace such typical instances of multimodal metaphor in political cartoons,
Forcevilles definition would thus have to be extended to include cases where
target and source are partially represented in different modes.
3. Universality versus individuality
According to the standard view of metaphor in a cognitivist framework,
metaphors are based on embodied human experience (Lakoff and Johnson
1999; Turner 1996), which is seen as universal. For example, since we all
feel hot as a result of physical exertion or excitement, metaphors based on
the concept of INTENSITY IS HEAT seem entirely natural to us (Kvecsec
2005: 18). Similarly, we all experience a connection between our movement
through space and the passing of time. This would suggest that everybody
understands such basic ontological and orientational metaphors intuitively
and, very often, at the level of unconscious or barely conscious thought
processes.
While primary metaphors are thought to be universal, they can be com-
bined to form more complex mappings between different domains of experi-
ence, and at least some of these structural metaphors are assumed to be lan-
guage or culture specific. For instance, there is evidence that in many
languages anger is conceptualized as a fluid or gas under pressure in a con-
tainer, but in Zulu it is apparently also sometimes understood as OBJECTS IN
THE HEART (Kvecses 2005: 69).
However, such a neat split between universal primary metaphors and
more culturally influenced structural metaphors may be deceptive. It is per-
fectly possible that some universal experiences do not lead to universal
metaphors and that bodily experience may in some cases be overridden by
both culture and cognition (Kvecses 2005: 4). Moreover, some metaphors
may not be based on bodily experience at all, but rather on purely cultural
considerations and cognitive processes (Talebinejad and Dastjerdi 2005).
Even more importantly, perhaps, the choice of metaphors by a communi-
cator and the recognition and interpretation of these metaphors by audiences
are likely to be strongly dependent upon the social context in which they are
used. As Ritchie (2004: 277279) points out, the same metaphorical state-
ment, my wife is an anchor, may be used to mean completely different things
in different conversational contexts, implying, for instance, either feelings of
contentment and stability or a sense of boredom and frustration.
Building on the concept of working memory, Ritchie (2004) suggests that
people interpret metaphors by drawing on readily accessible elements from
their long-term memory, as well as all the ideas that were recently activated
through the communicative interaction, including the perceived relationship
between participants and the degree to which they expect their perceptions
and ideas to be shared (Ritchie 2004: 272). If peoples cognitive representa-
tions of the common ground are radically different, then they may well inter-
pret the same metaphor in very different ways. Whether the processing of a
metaphor stops as soon as an initial interpretation is reached or whether it
continues to generate various metaphorical entailments will depend on the
nature of the conversation, the relationship between participants, the avail-
able time, distractions and perceived importance. Metaphor elaboration may
also be encouraged by the intrinsic pleasure involved in thinking and solving
puzzles.
In the case of political cartoons, it is perfectly possible that some basic
ontological and orientational metaphors are understood in a similar way,
perhaps even intuitively, by all members of a culture or language commu-
nity, irrespective of their background and prior knowledge. However, it
seems more likely that the interpretation of all metaphors in cartoons is at
least to some extent dependent upon the context in which they are used and
the working memory of the individual reader.
Since political cartoons typically refer to topical events, it is clear that
they will be read differently if taken out of their original context and viewed
at a later date. Apart from lacking knowledge about a particular event or
political issue, readers may also be unfamiliar with aspects of the fictional
world on which the cartoon is based. In Western cultures, at least, it is
common for cartoonists to use metaphors drawn from everyday life, sport
and popular culture in order to ensure that they are in fundamental harmony
with the cultural literacy of their public (Fischer 1996: 122; Mumford 2001:
xi). But cartoonists clearly sometimes misjudge their public, and the back-
ground and experiences of some individual readers may lead them to inter-
pret a cartoon in very different ways to the one intended.
Metaphors are by their very nature open to more than one interpretation,
which makes their use more risky for communicators, but also potentially
more interesting for interpreters. In fact, if cartoons were too easy to under-
stand, they would probably not provide the same sense of pride and
achievement to those viewers who manage to solve these intriguing mental
puzzles.
4. Data and Methods
This chapter is based on data collected in phase one of our study into the
ways in which young people understand cartoon images, including the mul-
timodal metaphors they often contain. For this purpose, we conducted one-
to-one interviews with 25 young people between the ages of sixteen and nine-
teen in Bradford, a city with a large British Asian population. Of the 13 men
and 12 women, 13 were Muslims from a Pakistani or Bangladeshi back-
ground, while two each were Hindus and Sikhs from Indian families. The
family of one young man had come to Britain from Afghanistan, and there
were also one British-Indian, one black, and six white British-born young-
sters who described themselves either as Christian or Atheist. Most of our
respondents were doing A-levels, whereas six of the participants were taking
vocational courses. The majority of the interviews were conducted in July
2005 and a small number in November of the same year. We used five car-
toons on the US Presidential elections published several months previously,
in October or November 2004.
Since the participants had volunteered to take part in the study, we ex-
pected them to be on average a little more interested in political issues and
more likely to follow the news than others in their peer group. Although
more than half of our interviewees initially described themselves as not very
or not at all interested in politics, during the interviews the Muslim students
in particular actually turned out to hold quite passionate views about those
geopolitical issues that they felt concerned them directly, such as the inva-
sion of Iraq and the conflict between Israel and Palestine (Hrschelmann
2008). If they read any newspapers at all, the respondents tended to read the
local paper or, more rarely, a tabloid or the occasional broadsheet newspa-
per. In the UK, local newspapers and tabloids tend not to carry many car-
toons of an explicitly political nature.
4
Not surprisingly, many of the young
participants were thus not very familiar with the political cartoon genre.
The semi-structured one-to-one interviews, which lasted between 30 and
45 minutes each, took place at the Technology College and local Further
Education College where the young people were studying. After explaining
that the cartoons were all about the recent Presidential elections in the USA,
the cartoons were discussed one by one, using the entire newspaper page in
order to ensure that at least some of the context was preserved. We encour-
aged respondents to read any of the headlines that they would normally read
in conjunction with the cartoon before describing the drawing and then at-
tempting an interpretation. In fact, while the articles below the cartoons did
generally comment on the US Presidential elections, none of them related
directly to the content of the cartoons. This meant that, in the few cases that
the respondents did avail themselves of the chance to read the headlines, this
did not necessarily assist them in their interpretation of the cartoons.
In order to ensure that the interview data were comparable, we used the
same basic interview schedule, consisting of a small number of open and
non-leading questions such as: Can you recognize any of the depicted char-
acters?; How would you describe their mood/feelings?; What do you
think the cartoonist wanted to say with this cartoon?; What are your own
thoughts and feelings when you look at this cartoon?
While taking care not to prejudice the participants in their responses, we
occasionally used additional prompts in order to elicit more details or to
clarify their answers. For instance, if respondents seemed to be struggling to
describe the mood of the figures in the drawing, we might ask them to de-
scribe the relationship between the depicted characters and to imagine what
they might be feeling. If they overlooked certain important details, such as
the inscription four more years on the matchbox in figure 2, we would
generally draw their attention to these and ask what they might be referring
to. In order to preserve the anonymity of the interviewees, responses were
coded with a letter from A to Y. ER indicates that I conducted the inter-
view, and KH stands for my collaborator, Kathrin Hrschelmann. The
symbol (.) in the transcriptions indicates a hesitation or pause in the respon-
dents delivery.
We also sent short questionnaires to the creators of the cartoons used in
the study, asking them to describe the intended meanings of their drawings.
In addition to completing the questionnaire, most of the artists agreed to a
telephone interview, during which they talked about their general working
conditions and practices, and explained in more detail what had given them
the inspiration for the cartoons used in the study, why they had chosen par-
ticular symbols and metaphors, and what they were trying to express
through their work.
Clearly, the meanings intended by the cartoonist are not the only ones
that a particular cartoon or, indeed, a particular multimodal metaphor is
able to generate, nor are his or her intentions to be equated with its correct
or most valid interpretation. By its very nature, the cartoon genre is ambigu-
ous and open to multiple readings, and the process of making sense of a
cartoon is likely to be strongly dependent upon the individual interpreters
background and experiences. In fact, our own detailed analysis of the car-
toons corresponded very closely with the artists intended meanings. This
may be linked to the fact that we probably represent the sort of politically
minded regular newspaper readers that the cartoonists apparently had in
mind when creating their work. In contrast to this, the young peoples read-
ings of the cartoons and the multimodal metaphors they contained reflected
their very different interests and preoccupations, as well as perhaps a degree
of unfamiliarity with cartoon conventions.
5. Results: Interpreting cartoon metaphors
As I showed in section 2, two of the cartoons used in our study contain sev-
eral layers of metaphorical meaning, from the basic connection between
movement through space and the passing of time to the more complex asso-
ciations between a make-believe mini narrative and events in the real world.
In the case of the cartoon by Peter Schrank, published in the Independent on
Sunday (figure 1), another basic metaphor that seemed to evoke a common
response was that of the darkened sky, which for the four participants who
mentioned this aspect indicated a sense of danger and impending disaster.
The orientational TIME IS MOTION THROUGH SPACE metaphor also seemed to
be understood in a similar way by all the respondents, since nearly every-
body equated the concept of the boot walking towards Iran and North Korea
with future actions, with the two signposts pointing in opposite directions
representing alternative decisions.
One exception to this general rule was respondent I, who described the
boot as coming from Palestine and other Muslim countries, where it has
caused a lot of destruction. For this young man, the figure stuck to the sole
of the boot embodied the innocent people in Afghanistan who have suffered
George Bushs wrath or so-called wrath, people that suffered in Iraq. Simi-
larly, respondent J, a young Muslim woman who was clearly very concerned
with the plight of Palestinian civilians, thought the small figure represented a
Palestinian:
J I think he is trying to show all (.) all this war going on (.) on in Pales-
tine (.) thats been going on for the past year or so (.) its trying to (.)
show that (.) its George Bushs fault and you know he is just crush-
ing people for nothing without realizing it and people are yelling at
him look what you are doing to our country but hes not taking no no-
tice and he is just crushing people under his feet (.) and now hes
been to Palestine then hes going to Iran and so hes going across the
world doing it to all the countries
In fact, eleven respondents did not recognize Tony Blair, seeing the drawing
instead as symbolizing an ineffective politician or, more generally, the weak
and powerless: So there is a big authority leader going towards the way he
wants and there are smaller people getting stuck (F). Many of the partici-
pants said it was the W on the boot that allowed them to make the connec-
tion to George Bush, while a few seemed to rely more on the general context
or the Texan connotations of the cowboy boot to draw this conclusion. Even
among the eight respondents who did not make the metonymic connection
between the boot and George Bush there was still unanimous agreement
about the fact that the person symbolized by the boot was powerful and ruth-
less. Participants talked about him having the ability to go where he likes
or do as he likes and his actions were described as standing, stepping,
crushing, or trampling on someone less powerful: sort of dont care
who he steps on (M). Thus, the metaphorical link between size and power,
status, or force was understood in a similar way by everybody, although the
degree to which the boot was seen as ferocious and sadistic seemed to de-
pend on whether or not Blair and Bush were recognized and the degree to
which the respondent identified with the figure squashed under the boot.
Clearly, only the eleven respondents who recognized the references to
both politicians were able to read the cartoon in the way it was intended by
the cartoonist, namely as a comment on the relationship between the two
men and their respective countries:
R Tony Blair is basically just (.) hes following George Bush round like
a lost sheep (.) even though that Tony Blair does have his own (.) he
wants to do his own things (.) hes like scared that (.) he wont let go
cause hes scared of losing America (.) and thats why I think that
Tony Blair he just isnt a suitable leader
This excerpt also demonstrates that, when discussing the meaning of a par-
ticular metaphor, respondents often introduced new metaphors or similes to
express their thoughts. The description of Tony Blair following Bush around
like a lost sheep evokes a very different image from the one represented in
the cartoon and shifts the focus from the actions of the US President to those
of the British Prime Minister.
The young people struggled to recognize other aspects of the cartoon im-
age as well. For instance, most of our interviewees read out the place names
on the signposts but were unable to make sense of the little drawings in the
background. The wall behind the signpost pointing to the left was described
by six respondents as a wall or fence, while four students realized that the
drawing of a power station on the other side had something to do with nu-
clear energy or nuclear weapons. The latter drawing reminded two inter-
viewees of the oil industry, while others described it as a crane, buildings,
satellites and communication (B), a building with a big ball on top of it
(N) and a door of a mosque and buildings (L). Only two of the respon-
dents (K and U) were sufficiently well informed about the related geopoliti-
cal issues to be able to refer to these in their interpretation of the cartoon. In
Ritchies (2004) terms, most of the young people did not have these facts
stored in their long-term memory, or at least they were not able to access
them when cued by the cartoon image. Unsurprisingly, this influenced their
readings of the two metonyms WALL IS ISRAEL-PALESTINE CONFLICT and
POWER STATION IS NUCLEAR ISSUE.
21 of the participants were also not able to name either the vulture or the
dove (or both), referring to them instead as a big bird (T), a turkey (J),
an eagle (K), and as a normal (smaller) bird (J, T) or pigeon (P) re-
spectively. With some respondents, it was simply that they could not think of
the correct term, but in many other instances the students were apparently
not aware of the conventional meanings associated with these symbols.
However, for these respondents the size and other physical features of the
birds nevertheless seemed to act as powerful metaphors for particular atti-
tudes and characteristics. Respondent S, for instance, described the bird on
the left as homeless and the one on the right as selfish, greedy.
As pointed out above, this lack of political background knowledge and
familiarity with common cultural symbols did not prevent the vast majority
of our respondents from understanding that the fork in the road represented a
choice of future actions according to the source-path-goal schema. This
would seem to support a tentative conclusion that basic orientational meta-
phors are generally quite widely and easily understood, particularly if they
are represented through a very conventional visual image, like the one in
figure 1. By contrast, the interpretation of metaphors of a more elaborate
structural nature (e.g. FUTURE US FOREIGN POLICY IS BOOT CRUSHING
OTHER COUNTRIES) requires readers to use any verbal tags, pictorial sym-
bols and/or metonyms to assemble more precise information about target and
source and to apply the general schema to a specific social or political sce-
nario.
The cartoon by Nicholas Garland (figure 2) was again recognized by al-
most all the participants to be a narrative image, and spatial relations were
used to comprehend the sequence of events. However, in this case an under-
standing of the nature of future action was more tightly connected to grasp-
ing the implications of the inscription (four more years) on the matchbox.
23 out of 25 respondents understood that the matches had something to
do with the future, but this interpretation was generally not forthcoming until
after the inscription had been noted and read out in the course of the inter-
views. Only respondent X seemed not to understand that the four more
years referred to the future: the matchbox (.) four years (.) is that how
long he has been president? Another young person was unsure about how to
read the sequence of events: it looks like he is going for the matches but
theres already a fire lit (.) no idea what thats supposed to mean (H). Five
of the participants explicitly made the connection between this inscription
and the length of a Presidential term in office in the US, with one young man
drawing attention to the number of matches lying on the ground: there are
four matches out representing four more years (U). The great majority (18)
seemed to grasp from the context that the inscription must refer to George
Bushs re-election, even though they did not explicitly refer to the length of
the US Presidential term and may not have been aware of this fact.
As I discussed in section 1, the more elaborate structural metaphor BUSH
IS TODDLER is a monomodal metaphor of the visual variety, which is com-
pletely independent of the verbal mode. However, it does require the reader
to identify the caricature of the American President. Six people did not rec-
ognize George Bush as the target of this metaphor, in which case they tended
to read the cartoon in a rather literal way, as a depiction of the threatening
nature of fire. One young woman, for instance, was clearly worried by the
picture of a baby in extreme danger and speculated that the cartoon was
trying to convey a message do not leave matches out for little babies (E).
In this context, the interviewee was puzzled by the inscription on the box,
speculating that it might refer to the age of the child or to the longevity of a
particular brand of quality matches:
KH have a look at the box of matches (.) what do you think four more
years means?
E never actually noticed that before (.) four more years four more years
(.) it could have a double meaning four more years (.) unless a child is
four years old or could live for few more years rather than putting the
child in danger (.) or it could be branding matches could last a long
time
In addition to the respondents who did not recognize the US President, 12 of
the participants did not seem to see the figure in the drawing as childlike at
all, which meant that the source of this metaphor was also not perceived in
the same way by everyone. In fact, one participant described the figure as
an old woman (F) and another as a gentleman:
B okay theres an image of a gentleman crawling towards a box of
matches which says four more years on them (.) so Im guessing its
to do with the election (.) theres smoke and fire in the background
which means perhaps the gentleman is going to strike a match and
add more fire to the background make it even worse (.) thats what it
shows
ER hmm (.) and can you identify who the character (.) do you think its
anyone specific?
B no (.) no
ER and can you describe the the =
B = Ive met him I cant remember his name Ive met him (.) no Im an
ex member of youth parliament and so Ive met all of these top guys
(.) but I cant remember his name

As this example shows, by no means can it be taken for granted that all
newspaper readers will recognize the caricatures of even the most well-
known politicians. One young woman taking part in our study, for example,
did not appear to be at all sure what the US President looked like, although
she thought she had seen him on TV once in a blue moon (O). Respondent
B, by contrast, clearly had a keen interest in politics and had even met some
politicians face-to-face. He nonetheless seemed unable to see any similarity
between this drawing and the US President and was instead trying to name a
British politician who might fit the bill; in our subsequent conversation it
turned out that he was probably thinking of John Prescott, the British La-
bour Party politician and Deputy Prime Minister. Cartoonists employ a par-
ticular shorthand style when drawing famous politicians and it is therefore
hardly surprising that people who are not familiar with this genre or with a
particular artists work will struggle to recognize some caricatures.
But even among those respondents who identified both the source and the
target of the metaphor BUSH IS TODDLER, the similarities they saw between
the two domains often diverged from or went beyond the meanings intended
by Garland. For example, one woman (Y) initially explained this metaphor
by pointing out that people actually see Bush as a small child and not very
intelligent, but she later went on to generate more entailments, suggesting
that, like a baby, he seems to need taking care of: whenever you see him he
is always surrounded by so many I dont know bodyguards and everything
(.) he likes the attention if you like. Respondent R focused on the idea that
the more something is perceived to be naughty, the more a child wants to do
it, while another participant suggested that the cartoonist had drawn Bush as
a grinning baby in order to show his absolute confidence in his victory: es-
pecially at a young age (.) like possibly a toddler crawling towards four
more years in power (.) he could be really confident (.) and because of that
hes showing that he can do it without all his American citizens (I).
While most people saw the fire as a metaphor for general problems, con-
flict, war and destruction, or more specifically the invasion of Iraq or 9/11,
respondent A interpreted the fire in the background as a reference to the
difficulties George Bush had to face in order to be re-elected: he has been
through hell to get four more years. Participant S, by contrast, thought the
figure crawling towards the match box looked like John Kerry and saw the
matches as representing his being under stress: he looks so pressurized
cause hes got a flame up the back and hes trying to reach to the matches
cause that shows that hes got tension (.) and hes got a lot of stress due to
the voting four more years to work (.) he has to organize it present it every-
thing so it all goes well. Although most of the interviewees did make a con-
nection between the matches, fire and danger of some sort, only four explic-
itly mentioned the idea that children should not be allowed to play with
matches and that if you play with matches you get burnt (U).
6. Conclusion and suggestions for further research
Multimodal metaphors, particularly those involving pictures and language,
offer the opportunity to explore different peoples understanding of the same
material, by asking them to verbalize their interpretation process. In our
study we were able to compare and contrast our own analysis of the various
levels of metaphor with the ways in which the same metaphors were read by
a group of young people. We were also at least to some extent able to relate
the various responses to the young peoples background, prior knowledge,
and experiences. This allowed us to explore the different roles of the verbal
and the visual mode in multimodal metaphor and to begin to address the
thorny issue of universality versus individuality in metaphor understanding.
However, in view of the relatively small number of participants in this study,
the following are just tentative conclusions, which would require further
investigation.
The results of our study indicate that some metaphorical mappings in po-
litical cartoons, such as those between size and power/status and between
movement through space and the passing of time, might be understood more
generally and at a more intuitive level than more elaborate structural meta-
phors. These orientational metaphors are also generally monomodal, in the
sense that visual information alone is sufficient to cue the conceptual link
between a more concrete and an abstract meaning.
However, reading a cartoon usually requires the various slots of basic
schemata to be filled in with more specific and detailed information. The
interpretation of the complex structural metaphors, ACHIEVING RE-ELECTION
IS GETTING HOLD OF A BOX OF MATCHES and FUTURE US FOREIGN POLICY IS
BOOT CRUSHING OTHER COUNTRIES, for instance, requires readers to draw
on verbal clues (four more years, place names on the sign posts) as well.
This, I propose, justifies their inclusion in the category of multimodal meta-
phors, in which case Forcevilles (2006: 384) definition would need to be
slightly extended to comprise all instances of metaphor where the target and
source are represented exclusively, predominantly, or partially in different
modes.
The data also reveal that the more complex structural metaphors could
only be read in the way intended by the cartoonist if respondents had a cer-
tain level of general knowledge and a familiarity with current affairs and
particular political figures. Clearly, the identification of metaphors is only
part of the process of cartoon interpretation as a whole, which means that
the study of how people understand multimodal metaphors must also include
a consideration of more general processes of meaning-making, which in turn
are closely linked to an individuals education, background, and experience
of the genre in which the metaphor occurs. For instance, since most of our
respondents were not accustomed to the cartoon genre, it is hardly surprising
that many of them struggled to identify the caricatures of even very well-
known politicians.
Even if the target and the source of a metaphor were identified in the way
they were intended by the cartoonist, the exact interpretation of it and the
entailments it generated varied a great deal from one interviewee to another
and reflected their current interests and preoccupations. This is consistent
with the view that people interpret metaphors by drawing on elements from
their working memory (Ritchie 2004: 272). The Muslim respondents, for
instance, appeared to be particularly conscious of the suffering of the civil-
ian population in the Middle East, which was reflected in the way many of
them tended to map properties of extreme cruelty and ruthlessness from the
cowboy boot onto the American President. Another good example of the
important role of personal experience in the interpretation process is the way
in which respondents applied completely different features of toddlers to
George Bush in the context of his bid for re-election.
Unfortunately, asking people to verbalize visual meaning is also not
without its theoretical and methodological problems. If, as many authors
believe, the verbal and the visual mode differ with respect to what they are
able to communicate, then any translation from one mode to the other will
involve some loss of meaning. Thus, verbal responses may not provide en-
tirely reliable data for judging what people are actually thinking when they
look at a visual metaphor, particularly if the respondents are unaccustomed
to such essentially artificial verbalization tasks. Perhaps future researchers
will be able to devise methods of exploring multimodal metaphor that are not
wholly dependant upon respondents verbal dexterity.
Another difficulty concerns Ritchies (1994) observation that the same
metaphor can assume entirely different meanings when encountered in dif-
ferent contexts. Although we tried to replicate the ordinary reading situation
as closely as possible, for instance by providing the whole newspaper page,
the interview situation was in fact far from ordinary. Most of the young
people were not particularly avid newspaper readers, and they tended not to
be very familiar with the political cartoon genre. Under normal circum-
stances they would probably never have encountered these cartoon images,
or, if they had, they may not have paid them much attention. Apart from a
couple of interviewees who were unable or unwilling to continue to discuss
the two cartoons beyond a very superficial description, our respondents were
nevertheless very cooperative and tried hard to glean some meaning from the
images. Prompted by our questioning, they were often prepared to generate
further metaphorical entailments, which they would otherwise perhaps not
have considered at all. Although we tried to keep our interventions to a
minimum and not to ask any leading questions, it is probably inevitable that
our own interests to some extent influenced their readings of the cartoons
and the metaphors within them.
In spite of these limitations, the study provides striking new evidence for
the unpredictability of metaphor interpretation in real life situations. Our
results suggest that analysts of all types of metaphor, be they verbal, visual
or multimodal, must guard against taking their own interpretations for
granted and must pay more attention to the pragmatics of metaphor use in
particular contexts.
Notes
1. Elisabeth El Refaie and Kathrin Hrschelmann, Editorial cartoons and
geopolitical perceptions, April 2005April 2007. We would like to thank
the British Academy for supporting this project (Grant No. SG-39469).
2. We are very grateful to Peter Schrank and Nicholas Garland for their kind
permission to reprint their cartoons and for agreeing to be interviewed.
3. For some readers, the verbal mode might play another, less explicit, role in
this cartoon; for them, the drawing might evoke the verbal expression play-
ing with fire. In this case, it could be said to represent a striking instance of
a cartoon metaphor in which the cartoonist merely secures what language
has prepared (Gombrich 1971: 128; cf. El Refaie 2009). None of the par-
ticipants in our study referred to this expression.
4. The other three cartoons used in the study were taken from the Guardian, the
Independent and the Daily Mail respectively.
References
Baldry, Anthony, and Paul J. Thibault
2006 Multimodal Transcription and Text Analysis: A Multimedia Toolkit
and Coursebook. London/Oakville: Equinox.
Bell, Steve
2004 Drawing Tony. The Guardian 17 July: 11.
Carroll, Noel
1996 A note on film metaphor. Journal of Pragmatics 26: 809822.
Dines-Levy, Gail, and Gregory W. H. Smith
1988 Representations of women and men in Playboy sex cartoons. In
Humour in Society: Resistance and Control, Chris Powell and
Gregory E. C. Paton (eds.), 234235. Basingstoke/London: MacMil-
lan.
Edwards, Janis L.
1995 Wee George and the Seven Dwarfs: Caricature and metaphor in
campaign 88 cartoons. Inks 4 (May): 2634.
1997 Political Cartoons in the 1988 Presidential Campaign: Image,
Metaphor, and Narrative. New York/London: Garland.
2009 Multiliteracies: How readers interpret political cartoons. Visual
Communication 8 (2): 181205.
Fischer, Roger A.
1996 Them Damned Pictures: Explorations in American Political Car-
toon Art. North Haven CT: Archon Books.
Forceville, Charles
2004/this vol. The role of non-verbal sound and music in multimodal meta-
phor. In Words in their Places: A Festschrift for J. Lachlan
Mackenzie, Henk Aertsen, Mike Hannay, and Rod Lyall (eds.), 65
78. Amsterdam: Faculty of Arts, VU Amsterdam.
2005 Visual representations of the idealized cognitive model of anger in
Gombrich, Ernst H.
1971 The cartoonists armoury. In Meditations on a Hobby Horse and
Other Essays on The Theory of Art. 2d ed., 127142. London/New
York: Phaidon.
Hrschelmann, Kathrin
2008 Youth and the geopolitics of risk after 11 September 2001. In Fear:
Critical Geopolitics and Everyday Life, Rachel Pain and Susan J.
Smith (eds.), 139152. Aldershot: Ashgate.
Horn, Robert E.
1998 Visual Language: Global Communication for the 21st Century.
Washington: MacroVU.
Johnson, Mark
1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination
Kaplan, Stuart Jay
1992 A conceptual analysis of form and content in visual metaphors.
Communication 13: 197209.
Kennedy, John M., Christopher D. Green, and John Vervaeke
1993 Metaphoric thought and devices in pictures. Metaphor and Symbolic
Activity 8: 243255.
Kvecses, Zoltn
Press.
Kress, Gunther
2000 Text as the punctuation of semiosis: Pulling at some of the threads.
In Intertextuality and the Media: From Genre to Everyday Life, Ul-
rike H. Meinhof and Jonathan Smith (eds.), 132154. Manchester:
Manchester University Press.
1996 Reading Images: The Grammar of Visual Design. 2nd ed. 2006.
Lakoff, George
2d ed., Andrew Ortony (ed.), 202225. Cambridge: Cambridge
University Press.
1980 Metaphors We Live By. Chicago/London: The University of Chi-
cago Press.
Messaris, Paul
1994 Visual Literacy: Image, Mind, and Reality. Boulder, CO/Oxford:
Westview.
1997 Visual Persuasion: The Role of Images in Advertising. Thousand
Oaks, CA: Sage.
Morrison, Susan S.
1992 The feminization of the German Democratic Republic in political
cartoons 198990. The Journal of Popular Culture 25 (4): 3552.
Mumford, Alan
2001 Stabbed in the Front: Post-War General Elections Through Politi-
cal Cartoons. Canterbury: Centre for the Study of Cartoons and
Caricature.
Philippe, Robert
1982 Political Graphics: Art as Weapon. Oxford: Phaidon Press.
2003 Understanding visual metaphor in advertising. In Persuasive Im-
agery: A Consumer Response Perspective, Linda M. Scott and Ra-
jeev Batra (eds.), 297310. Mahwah, NJ/London: Lawrence Erl-
baum.
Proctor, Tony, Stella Proctor, and Ioanna Papasolomou
2005 Visualizing the metaphor. Journal of Marketing Communications
11 (1): 5572.
Ritchie, David
2004 Metaphors in conversational context: Toward a connectivity theory
of metaphor interpretation. Metaphor and Symbol 19: 265287.
Schirato, Tony, and Jen Webb
2004 Understanding the Visual. London/Thousand Oaks/New Delhi:
Sage.
Scott, Linda M.
1994 Images in advertising: The need for a theory of visual rhetoric.
Journal of Consumer Research 21 (2): 252274.
Seitz, Jay A.
1998 Nonverbal metaphor: A review of theories and evidence. Genetic,
Social and General Psychology Monographs 124 (1): 95119.
Talebinejad, M. Reza, and H. Vahid Dastjerdi
2005 A cross-cultural study of animal metaphors: When owls are not
wise! Metaphor and Symbol 20: 133150.
Templin, Charlotte
1999 Hillary Clinton as threat to gender norms: Cartoon images of the
First Lady. Journal of Communication Inquiry 23 (1): 2036.
Turner, Mark
1996 The Literary Mind. Oxford: Oxford University Press.
Worth, Sol
1981 Studying Visual Communication. Philadelphia: University of Penn-
sylvania Press.
Zakia, Richard D.
2002 Perception and Imaging. Boston/Oxford: Focal Press.
Chapter 9
Image alignment in multimodal metaphor
Norman Y. Teng
Abstract
This chapter focuses on how image alignment as a design strategy figures in the
construction of multimodal metaphors. Six editorial cartoons from The Christian
Science Monitor are used as illustrative examples. Image alignment can take
many forms. It can be linear, curvilinear, or exhibit a two-dimensional pattern. It
works by making some constituent components of the alignment salient, surpris-
ing, evocative, or otherwise noticeable, or by making the shape of the overall
alignment conspicuous and unexpected. Sometimes it is only implicitly involved
in a design choice. How non-pictorial elements in a multimodal metaphor interact
with the aligned pictorial components is explained by concrete examples. As to
the conceptual basis for image alignment as a design strategy, a tentative thesis is
put forward for future research: image alignment renders the abstract concept
SIMILARITY visible on the basis of the experiential correlation that motivates the
primary metaphor SIMILARITY IS ALIGNMENT.

Keywords: cartoon, design, image alignment, primary metaphor, multimodal
metaphor
1. Introduction
This chapter focuses on image alignment and how it figures in multimodal
metaphor. Image alignment as a design strategy was explored in Teng and
Sun (2002), which elaborated and extended Forcevilles (1996) account of
pictorial simile. The core idea of this strategy is this: when pictorial compo-
nents are approximately aligned with one another with respect to size, orien-
tation, and distance, the alignment thus formed is apt for expressing an idea
that connects these pictorial components. For example, when the pictorial
components depict things of different kinds, the alignment is apt for express-
ing pictorial simile. As another example, if the components depict things that
198 Norman Y. Teng
can be seen as incompatible, the alignment can express an oxymoron in pic-
torial terms. A quick look at the following cartoon (figure 1) may give us a
good sense of how image alignment figures in pictorial representation. This
cartoon features a pattern of image alignment apt for expressing pictorial
simile. A newspaper is placed in alignment with books on a shelf. The news-
paper is positioned in the middle of this alignment, and the books that flank
it on both sides are all well-known horror novels. The front-page headline of
the newspaper indicates that the news is about the United States. News and
horror stories are of different narrative styles and presumably belong in dif-
ferent genres; however, the alignment suggests some similarity between the
components. This cartoon, then, suggests that news about the US is similar
to a horror story. The word HORROR engraved on the front of the upper
shelf further supports this reading. Following the standard A IS LIKE B for-
mat of simile, this pictorial simile can be labeled AMERICAN NEWS IS LIKE
HORROR NOVEL (see Teng 2006: 7374 for further discussion of this exam-
ple). The expressiveness of image alignment as shown in figure 1 is not con-
fined to pictorial representation, but also plays an important role in the de-
sign of multimodal metaphor.

Figure 1. The Horror Show, by Clay Bennett, The Christian Science Monitor, May
13, 2004, page 8. 2004 The Christian Science Monitor
(www.csmonitor.com). All rights reserved. Reprinted with permission.
Multimodal metaphor is a newly formed research topic proposed and ex-
plored by Forceville (2006/this volume, 2008). This newly defined research
topic incorporates Lakoff and Johnsons (1980a, b) insight that the occur-
rence of metaphors is not restricted to language. It directs attention to the
Image alignment in multimodal metaphor 199
phenomenon that metaphorical thoughts can be present in different modes of
representation, and, more importantly, in a diverse range of combined ver-
sions of different modes of representation. Briefly, a metaphor is multimodal
when its target and source are each represented exclusively or predomi-
nantly in different modes of representation, or when its targets and/or
source are cued in more than one mode simultaneously (Forceville 2006:
384). I shall restrict my discussion to a subtype of multimodal metaphor,
previously called verbo-pictorial metaphor (Forceville 1996: 148162),
and examine how image alignment as a design strategy may figure in it.
Six examples are discussed in the next section. All of them are taken
from The Christian Science Monitor, for which they were created by Clay
Bennett, the papers editorial cartoonist. (The reader may survey Bennetts
cartoons, which are full of verbo-pictorial metaphors, by visiting his per-
sonal website at http://www.claybennett.com.) The purpose of the following
discussion is to give a robust sense of how image alignment participates in
the construction of multimodal metaphors. In the final section, a tentative
thesis concerning the conceptual basis for image alignment as a design strat-
egy is put forward for future research.
2. Image alignment as a design strategy
Let us begin with a simple form of image alignment juxtaposition. Con-
sider figure 2, which juxtaposes two images. The one in the left panel is a
blurred image of a garbage can brimming with a mixture of rubbish and
surrounded by scattered emptied tins and wasted food. The one in the right
panel is essentially the same as that in the left panel, but it has such high
resolution that people can see more clearly what it is about. The alignment
suggests that the two images are connected via some idea. The word Tele-
vision printed at the bottom of the left panel and the phrase High-
Definition Television printed at the bottom of the right panel further suggest
what that idea is: TV programs are piles of garbage, and watching programs
on a high-definition television will not change that. This alignment and the
printed words jointly form a metaphor which, following the standard A IS B
format, can be labeled TV PROGRAMS ARE GARBAGE. This metaphor is
multimodal in that the target TV PROGRAMS is chiefly represented through
the verbal representations Television and High-Definition Television,
with the aid of the pictorial element that marks out the difference in image
resolution, and the source GARBAGE is exclusively represented in the picto-
rial mode of representation.
200 Norman Y. Teng

Figure 2. High-Definition Television, by Clay Bennett, The Christian Science
Monitor, June 15, 1998, page 8. 1998 The Christian Science Monitor
Consider next figure 3. Again there is a pattern of alignment. The image in
the left panel depicts a rhino, and the words Southern Africa above it indi-
cate where the rhino is located. The image in the middle panel depicts a
panda, and the words Western China above it indicate where the panda is
located. The image in the right panel depicts a dove with an olive branch in
her beak, and the words Middle East above it indicate where the dove is
located. The words endangered species are printed on a rectangular box,
which is superimposed on the upper parts of the panels. The label endan-
gered species, positioned this way, suggests that the alignment puts the
three depicted creatures in the same category, and conveys the message that
rhinos in Southern Africa, pandas in Western China, and doves in the Mid-
dle East are all endangered species. It is assumed that people know that rhi-
nos in Southern Africa are endangered, as are pandas in Western China.
However, doves are not really endangered species, and they are not indige-
nous to the Middle East. Joining the doves to the endangered species sug-
gests a metaphorical reading, and the readers are expected to take notice of
the fact that doves and olive branches are often used as a symbol of peace in
Western cultures. Taken together, this design suggests the metaphorical
reading that peace in the Middle East, like rhinos in Southern Africa and
pandas in Western China, is an endangered species. This metaphor is multi-
modal in that the target PEACE IN THE MIDDLE EAST is chiefly represented
by the image of a dove holding an olive branch in her beak, and the source
ENDANGERED SPECIES is chiefly represented by the category label endan-
gered species, aided by the depiction of a rhino and a panda that serve as
illustrative examples of endangered species. Following the standard A IS B
format, this metaphor can be labeled PEACE IN THE MIDDLE EAST IS AN EN-
DANGERED SPECIES.

Figure 3. Endangered Peace, by Clay Bennett, The Christian Science Monitor,
May 24, 2002, page 10. 2002 The Christian Science Monitor
Alignment in figures 1, 2, and 3 is linear along the horizontal axis, but it can
also exhibit other patterns. Consider figure 4. This cartoon depicts scores of
worshippers aligning themselves in a two-dimensional pattern, which is un-
derlined by the layout of the carpets. The white gowns that the worshippers
wear and the decorative patterns on the carpets are further cultural elements
that suggest to the readers that the worshippers are Muslims. The worship-
pers are on all fours, and most of them point their bodies in the same direc-
tion. At the center of this alignment, however, one worshipper, bearing a
rifle and a pistol, holds his body in the opposite direction. The overall align-
ment suggests the interpretation that all the worshippers belong in the same
group; they all believe in Islam and live according to its rules. However, the
depiction of the opposite body-orientation at the center of this alignment
suggests that the rifle-bearing worshipper is an exception. This cartoon is an
editorial cartoon from The Christian Science Monitor, and the readers
surely understand this important detail. While admiring the cartoons gentle
sense of humor and the empathy it expresses with Muslims, the readers may
also take in its political stance and read the message as follows: unlike the
rest of Muslims, the rifle-bearing man is not really a true believer, or his
religion is not true. The word terrorism printed on his belt and the bewil-
dered look of another worshipper beside him reinforces this interpretation.
The metaphor embedded in this overall interpretation can be labeled MIS-
GUIDED BELIEF IS AN OPPOSITE BODY ORIENTATION. Many readers would
undoubtedly arrive at the same interpretation without noticing the word on
202 Norman Y. Teng
the belt of the rifle-bearing man, because of the prototypical depiction of a
terrorist. (I thank Francisco Yus for pointing out this to me.) This implies
that the word terrorism does not play any essential role in their interpreta-
tion, and, as a result, the metaphor in this cartoon is hardly multimodal. It is
worth emphasizing, however, that terrorism plays an important role from a
design point of view. It guides peoples interpretations, and for the people
who do not take the depiction of the rifle-bearing man as a prototypical de-
piction of a terrorist, it frames their interpretation. (For further discussion on
the guiding and framing of image interpretation, which Barthes examined
under the concept of anchoring, see Barthes 1985 [1964]: 2830, and Force-
ville 1996: 7174.) From this viewpoint, MISGUIDED BELIEF IS AN OPPOSITE
BODY ORIENTATION can be considered a multimodal metaphor, in that it
becomes multimodal when the target MISGUIDED BELIEF is chiefly repre-
sented through the word terrorism, aided by the depiction of a rifle-bearing
man, and the source OPPOSITE BODY-ORIENTATION is exclusively repre-
sented by a pictorial component in the image alignment. Whether a design is
multimodal depends on how it achieves its effect, and it is important to spec-
ify in what way a particular metaphor is, or becomes, a multimodal meta-
phor.

Figure 4. Call to Prayer, by Clay Bennett, The Christian Science Monitor, Sep-
tember 24, 2001, page 8. 2002 The Christian Science Monitor
Figure 5 exhibits alignment in a curvilinear form. In figure 5, two army offi-
cers are pondering possible scenarios and what action the military should
take. The American flag and the US badge on their upper arms indicate that
they are US army officers. The words war on terrorism on the door win-
dow depicted in the upper left corner of the picture indicate that it is military
strategies for war on terrorism that they are pondering. They seem to come
to terms with a difficult situation, and reach a decision: an endless deploy-
ment of armed forces and non-stop military operations throughout a vast
geographical area. The metaphor embedded in this interpretation is chiefly
represented by the model tanks on the map, which line up into a curving
shape reminiscent of the mathematical symbol of infinity . The model
tanks metonymically represent the deployment of armed forces, the map
represents a vast geographical area, and the curving alignment reminiscent
of the mathematical symbol of infinity metaphorically represents what the
strategic deployment will lead to. This is a multimodal metaphor, which can
be labeled THE DEPLOYMENT OF ARMED FORCES AGAINST TERRORISM IS AN
INFINITY DEPLOYMENT IN A VAST GEOGRAPHICAL AREA.

Figure 5. Terrorism Strategy, by Clay Bennett, The Christian Science Monitor,
September 13, 2004, page 8. 2004 The Christian Science Monitor
As shown by the above examples, image alignment can take many forms. It
can be linear along the horizontal axis (figures 1, 2, and 3), a two-
dimensional pattern (figure 4), or a curvilinear form (figure 5). Despite the
diverse forms it can take, alignment of any of these forms can be strategi-
cally deployed in the construction of multimodal metaphors through its cen-
tral use for connecting a set of pictorial components in a way that is apt for
expressing a certain idea. Figures 25 show how this strategic deployment
may work. In figure 2, the two pictures of garbage cans are essentially the
same except the contrast in image resolution between them. This contrast is
a design choice made within the framework defined by the overall alignment.
In figure 3, the dove, unlike the rhino and the panda, cannot properly be
labeled endangered species; it is much smaller in size than the other two
animals; and, more important, it is symbolic of peace. This contrast, again,
is a design choice in the framework defined by the overall alignment. It is
204 Norman Y. Teng
worth noting here that alignment is defined in terms of iconic features such
as size, orientation, and distance, and, as a result, the symbolic representa-
tion in figure 3 is anchored to the alignment via the iconic function of the
pictorial components. In figure 4, the worshipper at the center of the align-
ment is in stark contrast to the rest of worshippers with respect to their
body-orientation. This is a design choice, too. All these choices make some
constituent components of the alignment salient, surprising, evocative, or
otherwise noticeable. In figure 5, the model tanks line up into a shape remi-
niscent of the mathematical symbol of infinity. This is a design choice that
makes the shape of the overall alignment conspicuous, and, probably, unex-
pected in that context. In contrast to figure 3, the symbolic representation in
figure 5 is achieved via the overall shape of the alignment, rather than a
particular aligned component. Design choices of the sort just described point
to the directions in which the readers should take their interpretations, but
they do not determine their interpretations. The above discussion of figures
25 shows how one may proceed to work out pertinent and well-balanced
interpretations.

Figure 6. Its an Escher-type Economy, by Clay Bennett, The Christian Science
Monitor, November 5, 2003, page 8. 2003 The Christian Science
Monitor (www.csmonitor.com). All rights reserved. Reprinted with per-
mission.
Figures 6 and 7 show that image alignment may be only implicitly involved
in a design choice. In figure 6, a business person and a laborer each stand in
an erect position, watching a zigzag arrow on a large sheet, which covers
most of the wall. The word economy printed on the arrow indicates that it
is a graphic report on the economy. The Y shaped lines, which represent
the edges and the corner, define an unconventional three-dimensional spatial
frame. The business person and the laborer are in positions orthogonal to
each other within this spatial frame. The business persons position defines a
viewpoint from which he sees a growing trend in the economy. The laborers
position defines another viewpoint from which he sees a downward trend in
the economy. Notice that the business person and the laborer in figure 6
would stand next to each other if they were aligned in a normal, spatial
frame. It is against this implicit understanding of the normal alignment that
the unconventional spatial frame and the consequent opposing viewpoints are
made possible and salient. This design gives a succinct, metaphorical ac-
count of the economic situation. The target is the economic trend; it is repre-
sented by the arrow and the word economy. The source is the direction
that the arrow is supposed to point in. It is represented by the arrowhead and
its spatial relations to the positions of the business person and the laborer.
Connecting the target to the source yields a metaphor that cannot but be
formulated in somewhat laborious terms: THE DIRECTION OF THE ECONOMIC
TREND IS THE DIRECTION OF THE ARROW AS IT IS VIEWED FROM EITHER THE
BUSINESS PERSONS POSITION OR THE LABORERS POSITION IN THE UNCON-
VENTIONAL SPATIAL FRAME. This metaphor expresses a critical stance on
the economy by reminding people of the existent alternative perspectives,
and using the unconventional spatial frame, instead of a normal spatial
frame, metaphorically suggests that the conventional assumption that a
growing economy will eventually benefit all people is not, or no longer,
valid.
Alignment implies an orderly arrangement of pictorial components, and
the examples we have encountered up to this point are effective in exposing
how an orderly arrangement of pictorial components may figure in the con-
struction of multimodal metaphors. Nonetheless, a disorderly array of picto-
rial components may also participate in the construction of multimodal meta-
phors, especially if such an array is intended for a particular effect against a
backdrop of some understood, orderly alignment. Figure 7 is a case in point.
In this cartoon, five sheets of paper are arranged in a rather untidy way.
Moreover, the four sheets in the background have been damaged some
letters were cut out from them. The sheet in the foreground is intact, and the
letters cut out from the background sheets have been pasted on it. The scis-
sors, the glue, and the scattered shreds give further evidence of the clipping
and pasting. The symbol SeCuRitY thus created on the foreground sheet is
a jumble of both lower case and upper case letters, and the letters are not
lined up in an orderly way. One can tell from the context that the symbols
printed on the background sheets were Liberty, Justice, Equality, and
Freedom, but, because of the clipping and pasting, they are now in bad
shape. This design suggests the following interrelated interpretations: (a) the
206 Norman Y. Teng
cuttings metaphorically mean that liberty, justice, equality, and freedom have
been severely and dangerously compromised; (b) the clipping and pasting
metaphorically mean that liberty, justice, equality, and freedom have been
curtailed in the service of security; and (c) the jumbled form of SeCuRitY
metaphorically mean that security measures have been badly managed. On
top of all this, the fact that the five sheets of paper are not well arranged
suggests the metaphorical reading that issues concerning liberty, justice,
freedom, equality and security have not been handled carefully. It is worth
emphasizing that disorder need not be a poorly thought-out design choice. It
can be carefully crafted so as to suggest that an action has been performed.
The disarray in figure 7 testifies to such a design choice. It is the implied
actions, rather than the things portrayed, that figure importantly in this con-
struction of multimodal metaphor.

Figure 7. Homeland Security, by Clay Bennett, The Christian Science Monitor,
October 11, 2001, page 8. 2001 The Christian Science Monitor
3. Conclusions and suggestions for further research
In this chapter, I have focused on image alignment as a design strategy in the
construction of multimodal metaphors. Various uses of this strategy and the
design choices that point out the directions the interpretations should take are
described. How verbal and other non-pictorial elements in a multimodal
metaphor interact with the aligned pictorial components is explained by con-
crete examples. A few further suggestions that are more amenable to empiri-
cal evaluation may be considered. In figure 2, Television and High-
Definition Television are essential to the interpretation of the cartoon. Most
people probably will have no clue how to read the garbage images if the
verbal representations are removed from the picture. It is also likely that
many people will have no idea what figure 3 is intended to mean if endan-
gered species, Southern Africa, Western China, and Middle East are
deleted from it. By contrast, if the target audiences are already familiar with
the political cartoon genre, terrorism in figure 4 is probably dispensable
because of the prototypical depiction of a terrorist. Economy in figure 6 is
also likely dispensable because of the typical portrayal of a boss and a la-
borer. As to the mathematical symbol of infinity in figure 5, if the model
tanks were lined up into a shape other than the mathematical symbol for
infinity, the metaphorical meaning would be lost. Finally, the lower and up-
per case letters in figure 7 are definitely verbal elements, but they are also
pictorial components by design. In addition to the texts guiding and framing
readers interpretation, the garbage in figure 2, the dove-with-olive branch in
figure 3, the terrorist in figure 4, the American flag in figure 5, and the boss
and the laborer in figures 6 are all pictorial components that carry symbolic
or cultural meanings familiar in the Western world. When the target audi-
ences are conversant with the symbolic and cultural meanings, those picto-
rial components can be good choices for communicative purposes. (For an
empirical study of audience responses to political cartoons, see El Refaie,
this volume)
It is worth noting that the cartoons metaphors are mainly pictorial in
their mode of representation, and the verbalizations of the metaphorical
meanings can sometimes be a laborious task. Figure 6, for example, is a case
in point (see Forceville 2006: 390392 about the implications of this phe-
nomenon for metaphor research). From a design perspective this should not
be a surprise, since words and pictures belong to different modes of repre-
sentation and are suitable for different communicative purposes. The idea of
image alignment as a design strategy discussed above offers a perspective
from which different modes of representation (here pictures and language)
can be deftly combined and coherently understood.
It is also worth noting that the patterns of image alignment discussed
above should not be taken to be instances governed by the invariance princi-
ple, one version of which runs as follows: Metaphorical mappings preserve
the cognitive topology (that is, the image-schema structure) of the source
domain, in a way consistent with the inherent structure of the target domain
(Lakoff 1993: 215; for a slightly different version, see Turner 1991: 172
182; for a critical discussion, see Brugman 1990, Lakoff 1990, Turner
1990; for a discussion of how this principle fares against blending theory,
see Turner 1996: 108109; for a discussion of how this principle should be
208 Norman Y. Teng
further revised, or abandoned, see Lakoff and Johnson 2003: 253254). The
patterns of image alignment, despite their image-schematic and topological
nature, are not something to be preserved or overridden in the metaphorical
mappings. Or to put it differently, the idea of target domain overrides is just
not appropriate to frame the issues concerning the role of image alignment in
the construction of multimodal metaphors. Instead, it is more suitable to
consider them as constructional schemas, and the design choices described in
the previous section as elaborations of the schemas. (For a recent discussion
of constructional schemas in cognitive grammar, see Langacker 2008: 167
174; for a discussion of elaboration, see Langacker 2008: 198205.) This
line of thinking gives us a way to apply cognitive grammar and its usage-
based approach to design research. (I thank Francisco Ruiz de Mendoza and
an anonymous reviewer for prompting this clarification.) It also suggests the
hypothesis that primary metaphors may well serve as a common conceptual
basis for multimodal constructions. A metaphor is primary in the sense that
the association of the target with the source is directly based on an experien-
tial correlation between them (Grady 1997a: 4748; see also Grady 1997b,
1999, 2005, Grady and Johnson 2000, Johnson 2007: 178179, Lakoff and
Johnson 1999: 4573). Moreover, the source and target concepts refer to
basic dimensions of experience, the shared structure of which coincides
largely with parameters relevant to the characterization of basic grammatical
categories in cognitive grammar, and the metaphorical mappings appear not
to be governed by the invariance principle. (Grady 2005: 16061607).
One entry in Gradys (1997a: 281299) list of primary metaphors is
SIMILARITY IS ALIGNMENT. The linguistic examples and the experiential
motivation for this metaphor are given below:

Motivation:
Objects may be oriented in the same way because they serve similar func-
tions, are involved in similar processes or acted on by similar forces.
And/or, orientation is a basic parameter for perceptual/cognitive classifi-
cation.

Examples:
Her new dress is very much in line with those worn by her co-workers.
There are stunning parallels between these two novels (Grady 1997a:
283).

Let us suppose that Grady is correct about this metaphor (and I think he is).
One may ask what is the relationship between this metaphor and the image
alignment discussed in this chapter. My guess is that this metaphor provides
the requisite conceptual underpinning for image alignment (I thank Charles
Forceville for suggesting this point to me). It seems then that image align-
ment renders the abstract concept SIMILARITY visible on the basis of the
experiential correlation that motivates the primary metaphor SIMILARITY IS
ALIGNMENT. In the last resort, it is this primary metaphor that sustains the
framework for the design choices described above. One may further hy-
pothesize that each design strategy that enables people to render an abstract
concept visible on the basis of the relevant experiential correlation in fact is
based on a primary metaphor. More data from a diverse range of designs
have to be examined before one can substantiate this conjecture about pri-
mary metaphors and design strategies.
Acknowledgments
This study is supported by a grant from Taiwans National Science Council. I
have greatly benefited from Charles Forcevilles extensive comments and sugges-
tions on earlier versions of this chapter. I thank Kathleen Ahrens, Sewen Sun,
Francisco Yus, Francisco Ruiz de Mendoza, and an anonymous reviewer for their
helpful critical remarks, and Judd Kinzely and Jeffrey Cuvilier for their hints on
how to improve the writing. I was kindly given an opportunity to present an ear-
lier version of this chapter at the International Symposium on Language, Culture
and Cognition, Taiwan. I thank William Croft and I-wen Su for their comments
and suggestions on that occasion. Another version was given as a lecture talk at
the Institute of European and American Studies, Academia Sinica. I thank Lee-
Joy Cheng, Wan-Chuan Fang, Der-Chin Horng, Richard Hwang, Chyong-Fang
Ko, Te-Hsing Shan and Ruey-Ling Tzeng for their probing questions and sugges-
tions on that occasion.
References
Barthes, Roland
1985 [1964] Rhetoric of the image. [Trans. by Richard Howard] In The Re-
sponsibility of Forms: Critical Essays on Music, Art, and Represen-
tation, 2140. New York: Hill and Wang.
Brugman, Claudia
1990 What is the invariance hypothesis? Cognitive Linguistics 1: 257
266.
210 Norman Y. Teng
Forceville, Charles
2008 Metaphor in pictures and multimodal representations. In The Cam-
Grady, Joseph E.
1997a Foundations of meaning: Primary metaphors and primary scenes.
Ph.D. diss., University of California, Berkeley.
1997b THEORIES ARE BUILDINGS revisited. Cognitive Linguistics 8: 267
290.
1999 A typology of motivation for conceptual metaphor: Correlation vs.
resemblance. In Metaphor in Cognitive Linguistics, Gerard J. Steen
and Raymond W. Gibbs, Jr. (eds.), 79100. Amsterdam/
2005 Primary metaphors as inputs to conceptual integration. Journal of
Grady, Joseph E., and Christopher Johnson
2000 Converging evidence for the notions of subscene and primary
scene. In Metaphor and Metonymy in Comparison and Contrast,
Ren Dirven and Ralf Prings (eds.), 533554. Berlin/New York:
Mouton de Gruyter.
Johnson, Mark
Lakoff, George
1990 The invariance hypothesis: Is abstract reason based on image-
schemas? Cognitive Linguistics 1: 3974.
1993 The contemporary theory of metaphor. In Metaphor and Thought
University Press.
1980a Metaphors We Live By. Chicago: University of Chicago Press.
1980b Conceptual metaphor in everyday language. Journal of Philosophy
77: 453486.
2003 Metaphors We Live By (with a new Afterword). 2nd ed., Chicago:
Langacker, Ronald W.
2008 Cognitive Grammar: A Basic Introduction. Oxford: Oxford
University Press.
Teng, Norman Y.
2006 Metaphor and coupling: An embodied, action-oriented perspective.
Turner, Mark
1990 Aspects of the invariance hypothesis. Cognitive Linguistics 1: 247
255.
1991 Reading Minds: The Study of English in the Age of Cognitive
Science. Princeton: Princeton University Press.
1996 The Literary Mind: The Origins of Thought and Language. Oxford:
Oxford University Press.
Chapter 10
Visual metaphoric conceptualization in editorial
cartoons
Joost Schilperoord and Alfons Maes
Abstract
Typical of metaphors in editorial cartoons is that they not only require somehow
the mapping of features from one object or domain to another, as all metaphors
do, but their interpretation also includes a critical stance towards a particular
socio-political situation, event or person. We will argue that the full interpreta-
tion of editorial cartoon metaphors can best be accounted for on the basis of the
combination of two cognitive interpretation strategies, i.e., schematic vs. taxo-
nomic reasoning, following the two types of source domains distinguished by
Shen (1999). In the chapter, we argue that schema-based reasoning tends to trig-
ger the rich variety of features to be mapped from source to target domain, and
that taxonomic reasoning is often the crucial trigger in interpreting the critical
stance expressed in editorial cartoons.

Keywords: editorial cartoons, visual metaphor, schema theory, critical viewpoint,
categorical metaphor interpretation
1. Introduction
Metaphors are employed to pursue a range of communicative goals, for ex-
ample to explain things to people (a heart is like a motor), to persuade peo-
ple (this car is as fast as a leopard) or to carry away readers (my thoughts
are playing in my head like young lions in the evening sun). Since the
seminal work of Lakoff and Johnson (1980), metaphors are considered a
conceptual rather than a linguistic phenomenon. But, thus far, language has
been the privileged input modality when it comes to studying metaphoric
conceptualization. Over the last decades, cognitive linguists, cognitive psy-
chologists and philosophers have studied the ways in which different types of
214 Joost Schilperoord and Alfons Maes
metaphor are conventionalized in language, what meaning operations are
involved, how new metaphors emerge and how language users process meta-
phors triggered by linguistic cues (e.g., Bowdle and Gentner 2005;
Deutscher 2005; Gentner, Bowdle, Wolff, and Boronat 2001; Gibbs 2005,
2006; Lakoff and Johnson 1980, 1999). Despite many different theoretical
positions and claims with respect to metaphor interpretation, there is broad
agreement on the basic template of metaphor. It is considered a rhetorical
figure in which one particular entity (an object or domain), i.e., the target, is
conceptualized in terms of another entity (object or domain), i.e., the source.
The processing of a metaphor thus implies the understanding or experiencing
of the target in terms of the source.
This understanding and experiencing involves a dual process: the two
entities have to be identified as source and target respectively, and corre-
spondences between the entities originating from two distinct domains have
to be found in order to establish what aspects of the source also apply to the
target. The latter process is usually called cross-domain mapping.This proc-
ess underlies for example Gentners career of metaphor theory (Bowdle and
Gentner 2005) as well as Glucksbergs (2001) class inclusion idea.
The conceptual nature of metaphor implies that not only language can be
used as a stepping stone for metaphoric conceptualization. Pictures, sound
and gestures can trigger metaphors as well (e.g., Cienki and Mller 2008,
Forceville 2006/this volume, Zbikowski 2002). In recent years, we have
witnessed a rapidly growing research interest in visual rhetorical figures,
with a special focus on visual metaphors and similes (Forceville 1996, 2002;
Kenney and Scott 2003; McCarthy and Mothersbaugh 2002; McQuarrie and
Mick 1992, 1996, 1999, 2003a, 2003b; McQuarrie and Phillips 2005, 2008;
Mothersbaugh, Huhmann, and Franke 2002; Phillips 2000, 2003; Phillips
and McQuarrie 2002, 2004; Scott 1994; Scott and Batra 2003; Teng 2006;
Teng and Sun 2002; Tom and Eves 1999; Van Mulken 2003; Van Mulken,
Van Enschot-van Dijk, and Hoeken 2005). The research efforts include
analyses of the form-meaning relationships involved in visual rhetoric, pro-
posals for a typology of visual rhetoric, studies into readers responses asso-
ciated with different types of visual rhetoric, and reflections on the medium
(in)dependence of visual rhetoric.
To our knowledge, the better part of the research into visual metaphors
takes commercial advertisements as the main context of study. Obviously,
commercial advertisements abound in metaphorical images (Phillips and
McQuarrie 2002) and therefore offer a natural habitat and a rich and prom-
ising resource of data on visual metaphors, on how they are shaped and
combined with verbal rhetoric. But editorial cartoons, too, are a metaphor-
Visual metaphoric conceptualization in editorial cartoons 215
rich communicative area. For example, Wikipedia observes that most [edi-
torial cartoons] use visual metaphors and caricatures to explain complicated
political situations, and thus sum up a current event with a humorous pic-
ture.
1
The political cartoonist for the Daily Telegraph, Nicholas Garland
(cited in Plumb 2004: 432) tellingly portrays the force of an editorial cartoon
as being derived from the vehicle itself which, besides caricature, requires
some or all of a mixture of caricature, metaphor, distortion, surrealism, de-
liberate misunderstanding and mockery.
The communicative functions of editorial cartoons and advertisements
differ considerably, which makes it interesting to investigate whether and
how these differences are reflected in the metaphoric conceptualizations
found. In this chapter, we will discuss a number of building blocks which we
consider crucial in constructing a cognitively plausible interpretation of
metaphors in editorial cartoons. Section 2 briefly characterizes editorial
cartoons. In section 3, the method of selecting and defining metaphors in
editorial cartoons will be discussed. Section 4 discusses four cases of edito-
rial cartoons, representing different structural types of visual metaphor. In
section 5, we will present two interpretative ingredients which we consider
crucial in metaphoric cartoons and which enable us to account not only for
their metaphoric conceptualization, but also for their critical stance. On the
one hand, metaphoric cartoons tend to evoke scenarios and thus require a
schematic interpretation of the source domain, in which clusters of (rather
than individual) attributes or relations are mapped from source to target
domain. On the other hand, they almost exclusively propagate a critical
stance towards a certain topic, which as we will argue requires a cate-
gorical interpretation of the source domain. We conclude the chapter with a
discussion on the multimodal nature of editorial cartoons.
2. Metaphorical editorial cartoons
According to Wikipedia, an editorial cartoon is an illustration or comic
strip containing a political or social message, that usually relates to current
events or personalities. For our purposes, we would specify this definition
in that the social message implies that a certain socio-political event or per-
sonality is depicted and commented on. That way, an editorial cartoon can
be said to have a descriptive and an evaluative communicative function.
Editorial cartoons are widely studied, especially in different areas of sociol-
ogy and journalism. Most of these studies focus on cartoon selections related
to particular political situations (such as presidential elections, world wars,
or the cold war, e.g., Becker 1996; Edwards 2001), particular countries
(such as the US, Japan or Austria, e.g., El Refaie 2003; Feldman 1995) or
particular societal issues (such as the role of women in society, e.g., Gilmar-
tin and Brunn 1998; Morrison 1992), or they study the work of a particular
cartoonist (e.g., Plumb 2004).
Although the role of metaphor is often acknowledged in these studies, and
some of them extensively explain metaphors within the pragmatic (political)
context in which they thrive (e.g., Edwards 2001; El Refaie 2003), to our
knowledge there has been no attempt to account for the generic structure of
metaphoric conceptualization as it takes shape in editorial cartoons.
As already indicated, editorial cartoons differ considerably from com-
mercial advertisements. Cartoons aim at affecting states of minds, beliefs,
points of view, and perspectives on socio-political affairs, rather than at
changing or influencing behavior. In addition, the processing and interpreta-
tion of cartoons requires a complex mix of political, cultural, historical, and
contextual knowledge. Most importantly, however, they almost invariably
express a particular critical, if not radically negative stance towards the
topic. Whereas commercial ads intend to evoke positive attitudes and feel-
ings with respect to the target, the reverse often holds for editorial cartoons.
One goal of this paper is to demonstrate the necessity of making this point of
view explicit in order to properly analyze the way metaphors operate in car-
toons (cf. Groarke 2002).

Figure 1. The Mobil oil advertisement (Forceville 1996).
Figures 1 and 2 may serve to illustrate these points. Figure 1 (taken from
Forceville 1996) shows a bottle of Mobil motor oil, positioned upside down,
with liquid pouring out of it. Together with the header text Intensive Care,
the following metaphoric X IS Y scheme can be derived: MOBIL OIL IS INTRA-
VENOUS DRIP. Since the message is about Mobil motor oil, convention tells
us that this entity should be identified as the target. The ad thus expresses
the message that the product is the life-blood of ones car (Forceville
1996: 159). The viewer/reader is invited to map positively oriented elements
like professional care and expert treatment from source to target. The con-
text of medical care is attributed to the maintenance of ones car: MOBIL OIL
IS TO A CAR WHAT AN INTRAVENOUS DRIP IS TO THE HUMAN BODY.

Figure 2. The Mobil oil cartoon (Bernhardt Willem Holtrop, De Nieuwe Linie,
1977)
Now look at figure 2: an editorial cartoon drawn by the Dutch artist Bern-
hardt Willem Holtrop (Willem), which was published in the magazine De
Nieuwe Linie in 1977. The cartoon shows us Ian Smith, leader of the former
Rhodesian apartheid regime (presently Zimbabwe). Smith is portrayed in
quite a deplorable physical condition, sitting in a wheelchair and connected
to several bottles of intravenous drips. On these bottles the names of various
Dutch and multinational companies are shown, amongst others Mobil. We
can paraphrase the metaphor in this cartoon as INTERNATIONAL SUPPORT OF
SMITH IS INTRAVENOUS DRIP, or, specified to Mobil as one of the support-
ers: MOBIL IS INTRAVENOUS DRIP, i.e., Mobils support of the Smith-regime
is to that regime what an intravenous drip is to the human body. Now sup-
pose a viewer, presented with this image, interprets it by mapping the same
elements to the target object as intended by the Mobil oil ad. That is, s/he
infers that Mobil oil is like a doctor to the Smith regime, providing profes-
sional care and expert treatment. S/he would then clearly misunderstand the
essence of this cartoon. The central tenet of the message seems to be that the
Rhodesian apartheid regime can only survive by (an overkill of) external
treatment, as it is no longer able to stand on its own, and to take care of its
existence itself. In other words, whereas the Mobil advertisement in figure 1
aims at cross domain mappings that should lead to a favorable attitude to-
wards Mobil, figure 2 aims at precisely the opposite. In our view, this short
analysis suggests a systematic interpretative difference between editorial
cartoons and commercial advertisements and brings to light that the same
visual metaphor serves widely distinct purposes in these two communicative
genres.
3. Sample and selection of editorial cartoons
We used two main sources to collect the sample of editorial cartoons that we
discuss in this chapter. Most of the cartoons were taken from collections of
cartoons published as separate books (containing work) by various famous
Dutch cartoon artists, such as Fritz Behrendt, Opland, Willem, Tom
Janssen, Jos Collignon and Peter van Straaten. Our decision to focus mainly
on Dutch cartoons is practical as it offers the best guarantee that we are able
to understand as precisely as possible the public events and persons the car-
toons refer to. In addition, we consulted a number of websites with collec-
tions of (international) editorial cartoons.
In collecting cartoons, we did not systematically control cartoons per art-
ist, time period, publication medium or type of public event satirized, as we
are not interested in (representative analyses of) particular periods, artists or
event types. Rather, we intend to characterize the dominant generic structure
of metaphoric conceptualizations in editorial cartoons and to analyze the
way visual metaphors are employed to make a point or express a viewpoint.
Therefore, our main criterion in selecting cartoons is that they should contain
a visual metaphor, or a combination of visual and verbal metaphor.
2
That is,
the image is to refer to (at least) two domains X and Y that have a meta-
phorical relationship (rather than a literal analogy) X IS Y, where one domain
(X) serves as target and the other (Y) as vehicle.
3
This way, we sampled a
corpus of 117 Dutch and 27 international cartoons, large enough to make
observations and cautious hypotheses about representative characteristic
features and generalizations. Many of the selected cartoons obviously con-
tain more than just a metaphor. For example, many also contain caricatures
(Bush-as-elephant, Russia-as-bear) or other rhetorical figures, such as hy-
perboles. However, our analyses will primarily focus on metaphorical con-
ceptualizations of target domains.
It is obvious that the interpretation of cartoon metaphors is to a certain
extent driven by genre-related, pragmatic considerations. For example, view-
ers will more or less expect cartoons to criticize or ridicule situations or
persons, and they will therefore most probably arrive at an interpretation of
a cartoon to that effect. Also, the interpretation of cartoons crucially depends
on the viewers ability to recognize persons, objects, and situations and on
their knowledge of the relevant facts, such as news events, historical facts,
cultural habits, and the like. This knowledge is indispensable for a full inter-
pretation of the intended meaning associated with metaphors in cartoons.
Hence, in analyzing cartoons, we take the position of a fully informed
viewer, that is, a viewer who not only knows the conventions of the genre,
but also recognizes the relevant persons, objects and situations and has suf-
ficient factual knowledge to understand the gist of the cartoon.
This being said, in analyzing and discussing the cartoon metaphors, we
take a cognitive rather than a pragmatic point of view. We intend to account
for the metaphoric conceptualization and critical stance of editorial cartoons
in terms of what is actually shown in the image, rather than in terms of the
cultural and general discourse context. This means that we focus on the in-
formation contained in cartoons that viewers are to process, rather than on
the prior, contextual/pragmatic knowledge they bring to bear on interpreting
the cartoon. Clearly, such an approach has its flaws. For example, as El
Refaie (this volume) demonstrates, contextual/pragmatic knowledge varies
from viewer to viewer, and this might cause interesting and complex inter-
pretation problems problems that are fully worth to be studied in their own
right. Nevertheless, we will ignore such variation, and concentrate on unrav-
eling the source-to-target mapping mechanism, and the accomplishment of
the evaluative position, thereby assuming (rather than assessing) all avail-
able and relevant background knowledge. That way, we hope to detect ge-
neric meaning ingredients and interpretation mechanisms related to meta-
phoric conceptualization in editorial cartoons.
4. Analyzing the cartoons: Four case studies
For each cartoon, we carried out the following analytical steps.
We defined the conceptual content of a cartoon by paraphrasing the
metaphor using the X IS Y template, thus determining target and
source domain and the aspects to be mapped.
We determined the way in which the two domains are realized. Fol-
lowing Forceville (1996), Philips and McQuarrie (2004) and Teng
and Sun (2002), we tried to distinguish cases in which only one do-
main is expressed visually (so called replacements), cases in which
the two domains are expressed separately (i.e., juxtapositions), and
cases in which the two domains are visually integrated (i.e., fu-
sions).
Following the practice of Groarke (2002) in analyzing the argumen-
tative structure of editorial cartoons, we reconstructed as objectively
as possible the point of view expressed.
In this section we will illustrate these analytical steps by discussing four
cartoons. We will show that the three structural classes are relevant in de-
tecting relations and attributes to be mapped and argue that cartoons need an
additional interpretation step in which the critical point of view is extracted.

Figure 3. FOOTBALL IS SUN (BAS, Tachydromos, www.cartoonweb.com, retrieved
June 2006).
Figure 3 shows us a solar system of two celestial bodies. On the smaller
object latitude and longitude lines are projected, a conventional way of visu-
alizing the earth, which suggests that we are to interpret the large object as
the sun. This qualifies the cartoon as an example of replacement in that the
sun is replaced by a football. Hence, the metaphorical objects involved here
can be paraphrased as FOOTBALL and SUN. The cartoon dates back to 2002,
the year in which the world champion soccer tournament took place in Japan
and South Korea, so its topic is football, or the tournament. Hence, the ob-
ject FOOTBALL is to be labeled the target object, and SUN the source, and the
metaphorical structure can be rephrased as FOOTBALL IS SUN.
What comment does this analogy express? The sun very often appears as
source object in commercial ads, for example in ads for fruit juices, where it
attributes positively oriented notions like health, giving energy, and
providing light and warmth to the product. Evoking such positive attitudes
is not what this cartoon intends to do. Rather, the cartoon ironically com-
ments on the worldwide impact of the championship, suggesting that it is the
most important thing in the world. Two pictorial details are important in this
interpretation. First, the complete replacement of the sun by a football may
make the impact of the latter object stronger than a fusion or juxtaposition of
these two domains would have done. If the objects were juxtaposed or fused,
the viewer might infer that at least the sun would still be there, so to speak,
whereas now it has completely disappeared from the scene. Second, the pro-
portions of a solar system with a tiny earth and a large football moreover
hint at the ironical stance towards the topic, expressing the artists opinion
that, to many people, football has become as important as the sun is to life
on earth.

Figure 4. TSUNAMI IS SHARK (Tom Janssen, www.tomjanssen.net, retrieved Janu-
ary 2007)
Figure 4 is a case of fusion: two objects are fused into one hybrid visual
image which expresses the metaphor TSUNAMI IS SHARK. The cartoon ap-
peared right after the news of the disastrous tsunami in Indonesia in Decem-
ber 2004. Although natural events are hardly suitable to form an opinion
about, the metaphor expresses the horror of the event by fusing the tidal
wave and shark teeth. The cartoon thus exploits the source entity (SHARK) as
a prototypical member of the set of horrible things. As we will discuss
further in section 5.2, this requires a categorical interpretation: by using
sharks as a source, the cartoon expresses the message that from now on we
should also include tsunamis as members of the set of horrible things.
Figure 5 shows a case of juxtaposition. We see a dog and a television
both tied to a tree, with the accompanying text (to be translated as And now
we repeat last years summer broadcasting programs). Note that the two
objects DOG and TV-SET are placed next to each other, and this way of
grouping them invites readers to solve the cognitive dissonance caused by
the apparent differences between the objects and to look for relevant connec-
tions between the objects (cf. Teng and Sun 2002). Distilling the critical
comment in this cartoon requires some contextual knowledge about the two
objects. First, summer broadcasting in the Netherlands (and elsewhere) is
often criticized for mainly repeating old programs. Second, many people
have the deplorable habit of going on holiday unwilling to take proper care
of their pets. Pets are often left on their own in just the way this cartoon
depicts. This knowledge enables us to identify the metaphorical objects as
SUMMER TV PROGRAMMING and LEFT-BEHIND PET, and to determine the
crucial element which connects the two objects: they represent comparable
examples of contemptuous human behavior. Although the phenomena are
equally deplorable, only the image of the dog refers literary to a real-world
state of affairs. We may conclude that this should be the source object, and
that the metaphorical structure can be rephrased as SUMMER TV PROGRAM-
MING IS (LIKE) LEFT BEHIND PET. By metaphorically grouping these objects,
the artist thus expresses the idea that Dutch broadcast organizations who are
responsible for summer TV programming relate to their audience the way
dog owners who leave behind their dogs relate to their pets.

Figure 5. TELEVISION IS (LIKE) DOG (Tom Janssen, www.tomjanssen.net, retrieved
January 2007)
As a final example, we consider a cartoon (published in the 1960s) in which
there is more than one metaphor present within the same image (see figure
6). This cartoon refers to the situation in a class room where an angry
teacher punishes a bad school boy by putting him in the corner. The name
Cuba projected on the back of the boy allows us to identify the first meta-
phor as CUBA IS LITTLE BAD BOY, whereas the stars on the sleeve of the
pointing figure tell us the second metaphor is AMERICA IS ANGRY TEACHER.
What is special about this cartoon is that the two metaphors together trigger
a scenario or scheme, as is in fact the case with a large number, if not all of
the cartoons in our corpus (see section 5). In this case the cartoon not only
maps attributes of bad boys and angry teachers onto the target domains
Cuba and America respectively, but it also maps certain typical roles. As
such the cartoon expresses the idea that Cuba relates to America like a little
bad boy relates to his angry teacher.

Figure 6. CUBA IS LITTLE BAD BOY, AMERICA IS ANGRY TEACHER (www.politiekes-
potprenten.nl, retrieved March 2003).
It is not easy to distill the critical stance of the cartoon, as it only seems to
tell the reader: This is what the political situation of Cuba and the US is like.
So the image merely seems to signify some complex political situation to the
reader, rather than to comment on it. However, note that the metaphorical
scenario evokes the inherently unequal teacher-pupil relationship, empha-
sized by the visual perspective taken in the cartoon. This may hint at its
critical stance, i.e., the US acting wrongly because it treats Cuba like a
naughty boy. In international affairs there is no actor who is inherently supe-
rior to the other. All nations are equal, so if one nation (the US) considers
itself as superior with respect to another nation (Cuba), then that first nation
is wrong. Like the cartoon in figure 3, it is the very choice of the meta-
phoric scenario, together with pictorial details, that accounts for the car-
toons critical stance.
To conclude this section, the four cases show that the domains involved
in visual metaphors can be realized or represented differently: by the re-
placement of one visual domain by the other or by a combination of two
visual domains, integrated (fused), or juxtaposed. Apart from that, we sug-
gested two characteristics which indicate how in a cartoon a visual metaphor
is involved in visual argument, both in signifying a certain state of affairs
and in commenting on it. Pictorial details in cartoons may trigger schema or
scenario domains involving prototypical relations between objects (the little
earth and the large football; the pupil and the teacher), and they may trigger
category domains including prototypical members (i.e., sharks and left pets).
The next section discusses these characteristics in more detail.
5. Two basic interpretation ingredients
5.1 Signifying states of affairs: schematic source domain interpretation
Especially the Cuba-US cartoon suggests a pervasive characteristic of edito-
rial cartoon metaphors, i.e., their scenario character. The majority of car-
toons in our corpus introduce a familiar scenario as a basis of the visual
metaphor. A typical example is given in figure 7.

Figure 7. Hospital scenario metaphor (Fritz Behrendt, De Telegraaf, 1980; www.
socsci .kun.nl/ped/owk/activeworlds/prenten/, retrieved January 2007).
This cartoon was published in 1980, during the heyday of the former Polish
free labor union Solidarnosc (Solidarity), the first free labor union under a
communist regime. It depicts union leader and Solidarnosc founder Lech
Walesa, the former Polish leader Wojciech Jaruzelski, and Karl Marx, who
gather around the sickbed of a patient, the personification of Poland. Walesa
is holding a syringe and Marx a copy of his book Das Kapital. The cartoon
triggers an entire conceptual domain that we may call a hospital scenario.
The scenario contains several elements: persons (staff, patients), roles (doc-
tor, patient), relations (doctors and medical instruments) and objects (in-
struments, a drip), and attributes (being sick). Hence, with respect to the
topic of this cartoon the scenario serves as a supplier of various metaphori-
cal relations:

WALESA IS DOCTOR
JARUZELSKI IS DOCTOR
MARX IS DOCTOR
POLAND IS PATIENT
DAS KAPITAL IS MEDICINE
SOLIDARNOSC IS MEDICINE
POLITICAL DOMAIN/PUBLIC AREA IS HOSPITAL ROOM

Not only do scenario elements serve as source objects, but the scenario also
imposes an additional conceptual structure onto the image by virtue of the
conceptual relations that hold between the scenario elements. According to
Lakoff and Johnson (1980), scenario metaphors can be seen as Idealized
Conceptual Models, i.e., cognitive networks with causal, temporal and other
sorts of relationships between persons, roles, locations and attributes which
all are more or less fixed, and conventionally known by all members of the
cultural community in which they appear.
A more fine-grained account of this type of metaphor can be found in
Shens (1999) notion of schematic source domain. With this notion, Shen
provides an explanation why, in processing metaphorical messages, certain
features are mapped from source to targets in preference to others. For in-
stance, as also noted by Gentner et al. (2001), in comprehending a metaphor
like plant stems are drinking straws, people tend not to map properties like
tubular or hollow from source to target. Rather, their understanding of the
metaphor is likely to involve the notion of a drinking device, and the relation
between the person/object that drinks and the liquid being drunk. Shen ac-
counts for this preference by proposing that the source domain of the drink-
ing straw evokes a schema that not only contains the object itself, but also
entities such as the person using the straw, the liquid being drunk, the recep-
tacle, and more in general the affordances, experiences, and functions asso-
ciated with these objects grouped together in this particular constellation.
Besides sources-as-schemas, he also distinguishes sources-as-taxonomic
categories. We return to this type of source domains in the next section
where we attempt to account for the critical stance of editorial cartoons.
A schematic source domain thus represents a cognitive schema, i.e., a
knowledge structure consisting of a set of entities which are connected to
each other via relations of contiguity, such as thematic, causal, spatial and
temporal relations (Shen 1999: 1633). In interpreting metaphors on the
basis of this schema paradigm, the mapping of relations is preferred over the
mapping of isolated attributes, and higher order relation mappings are pre-
ferred over lower order relation mappings, an idea which is consistent with
Gentners structure-mapping theory (Gentner et al. 2001).
4

The hospital or medical scenario-as-metaphor is a recurrent one. In our
sample, eight cartoons exploit this scenario (see also figure 2). Equally fre-
quent is the cooking-scenario, an example of which is given in figure 8.

Figure 8. Cooking scenario metaphor (Tom Janssen, Trouw, April 2004)
The cartoon was published in April 2004. At that time, the former Dutch
minister of administrative innovation Thom de Graaff was deeply involved in
defending and implementing his proposal for a new system for electing may-
ors and Dutch congress representatives. The cartoon shows us De Graaff,
who is busy cooking the new election system (see the Dutch label nieuw
kiesstelsel on the kitchen pot). Promoting the reorganization of the Dutch
election system was the main reason for founding De Graaffs party D66 in
1966. As in the medical scenario, there are different roles here that may
serve as individual metaphors:

MINISTER IS COOK
POLICY IS DISH
MEASURE IS INGREDIENT
MEASURE IS COOKING EQUIPMENT

Once the schema is triggered, it is not difficult to infer other metaphors such
as PEOPLE ARE CONSUMERS OF DISH.
Other frequently encountered metaphorical scenarios in our corpus are
marriage, funerals, and (boxing) games. The marriage scenario is often em-
ployed to portray coalitions or political treaties. The sports game scenario is
well suited to metaphorically depict political opponents or election cam-
paigns. An example is figure 9, published during the 2004 US presidential
election campaign, depicting George Walker Bush and John Kerry as boxers
in a boxing ring.

Figure 9. Boxing game scenario metaphor (KAL, The Economist, www.cartoon
web.com, retrieved, June 2006)
Finally, there are some prototypical source objects (not scenarios) that we
observed on several occasions, especially ships and animals. Ships are often
used as source domains for nations, states or political parties. In one case a
ship (quite a marked one, in fact) is even used for a person, i.e., the Titanic
as a source domain for former US president Bill Clinton. Laws, treaties or
other products of politicians are often portrayed as animals. The use of ani-
mals in cartoons justifies a study in its own right, as they are used meta-
phorically to evoke a wide range of semantic categories (see also figure 4).
For example, we found two Dutch cartoons, dating from March 2005, both
commenting on the special anti-terrorist act of the Dutch minister of Justice
by portraying the act and terrorism as two equally big, blood-thirsty dogs,
thus expressing the opinion that in this case the cure may be worse than the
illness. Another animal that is frequently used as metaphorical source do-
main is the snail, often employed to express the opinion that some measure
or action has been undertaken too late, or comes too slowly.

5.2 The critical viewpoint: categorical source domain interpretation
As we already noted, editorial cartoons differ from commercial advertise-
ments in that they often express some critical point of view. A full account
of visual metaphor in cartoons requires this point of view to be made ex-
plicit. If we look again at figure 8, it is striking that De Graaffs kitchen is a
mess and that black smoke evaporates from his kitchen pot. These pictorial
aspects suggest that there is something wrong with both the dish and the way
he is preparing it. For example, De Graaff is portrayed as a child, which
suggests that he is not able to cope with his job. Such details allow us to
conclude that the cartoon expresses a critical standpoint towards both the
minister and his policy. But on what grounds precisely do we conclude this?
In this section, we discuss an interpretation mechanism, which, we be-
lieve, enables us to explain the critical stance of editorial cartoons. It is
based on the second type of source domain proposed by Shen (1999), i.e.,
the domain as category, which is reminiscent of Glucksbergs (2001) class
inclusion idea. According to Shen, sometimes source domains require a
taxonomic or categorical interpretation, which is based on a set-member
relationship, rather than a part-whole (i.e., schematic) relation. The meta-
phorical expression Bush is a hawk, for example, is based on the taxonomic
interpretation of a hawk as a particular type of bird. That is, the interpreta-
tion is based on how hawks relate to the set of birds, rather than how they
relate to other objects or events in a scenario. Animals often evoke this type
of categorical interpretation in editorial cartoons. The categorical interpreta-
tion is guided by the so-called diagnosticity principle, stating that the map-
ping of properties with high diagnostic value is preferred over the mapping
of properties with low diagnostic value, an idea which is congruent with
Ortonys salience imbalance theory of metaphor understanding (Ortony
1979). Applied to the Bush is a hawk metaphor, the categorical attributes of
a hawk with a high diagnostic value in the given context are its ferocity and
invulnerability.
In the remainder of this section, we will reanalyze some of the cartoons
discussed above, in an attempt to substantiate the plausibility of the relation-
ship between categorical reasoning and the critical stance in cartoons. Con-
sider again the difference between the Mobil metaphors in figures 1 and 2.
The tenet of the Mobil ad in figure 1 seems to rely primarily on the underly-
ing medical scheme: by using Mobil oil a car owner takes care of his car like
a doctor takes care of his patient. The medical schema evokes entities like
doctor, medicine and patient, and the interpretation of the metaphor is based
on the standard relations of contiguity between these entities in the medical
scheme: actor, patient, instrument, and so on. Obviously, a drip may also be
seen as member of the category of things used to treat sick patients, and even
as a member with a high diagnostic value, but this adds little to what the
viewer already knows, namely that he is to form positive attitudes towards
the advertised product. Now consider figure 2. This cartoon triggers exactly
the same medical schema. But in order for the viewer to distill the critical
tenet of the cartoon, additional interpretation is required, otherwise, s/he
would simply feel invited to understand the international support of the
Smith regime in terms of medical care. In our view, this additional interpre-
tation is critically dependent on categorical reasoning about the medical
source domain. In particular, the medical care domain has to be viewed as a
member of the set of external forces or assistance from outside. With re-
spect to the target domain (political leaders or regimes), this categorical
interpretation has a rich diagnostic potential. Political leaders or statesmen
are good to the extent that they are supported by internal forces and assis-
tance from inside, for example their own governments and people. Con-
versely, political leaders who can only stay in power due to external support
are considered bad. Hence, the diagnostic potential of the categorical inter-
pretation of medical care as a member of the set of external assistance offers
the crucial trigger for the critical point the cartoon attempts to make (i.e.,
international support for the Smith regime is bad). Note that pictorial
details are in line with this interpretation (e.g., the facial expression and
posture of Smith, the relative large size of the drips, and the visual perspec-
tive of the dangling wheelchair).
A similar categorical interpretation can explain the critical position ex-
pressed in the Poland cartoon (figure 7). A strictly schematic interpretation
(as provided in section 5.1) actually does little more than signifying the state
of affairs in Poland in 1981: Walesa, Jaruzelski, and Marx relate to Poland
like a doctor relates to a patient. The critical tenet of the cartoon, however,
rests on a multiple categorical interpretation of the source domain element
doctor. Two opposite diagnostic properties of the class of doctors (or the
superset of caregivers) are crucial. A doctor can either be a member of the
set of caregivers who help and cure other people or he can be a member of
the set of caregivers who mistreat patients and destroy their lives. So who
belongs to which category in this cartoon? Our knowledge of the political
circumstances in Poland by the time, and the political points of view of the
cartoonist will lead us to infer that Walesa is the good doctor. But this same
conclusion is also warranted by pictorial elements, especially by the different
facial expressions of the three doctors. So, the intended critical interpreta-
tion, we believe, is carried by the diagnostic potential of the categorical in-
terpretation of the source domain element doctor. Furthermore, metaphorical
entailment allows us to make the following inference: Because good doctors
have good medicines, and bad doctors have bad medicines, Solidarnosc is a
good medicine, and Das Kapital a bad one.
Similar observations can be made with regard to the cooking source do-
main. The cartoon in figure 8 rests on a cooking schema to the extent that it
clarifies how this minister relates to his policy and the people affected by it.
However, only a categorical interpretation evokes the diagnostic property
which reveals the critical stance of the cartoon: the cook is to be seen as a
member of what can be termed the set of bad cooks, i.e., cooks who are not
up to the task of organizing their work in the kitchen and preparing a proper
meal. This categorical interpretation is supported by a number of visual
details, such as the messy kitchen, or the black smoke, and reveals the rele-
vant properties to be mapped to the target domain.
In sum, it is evident that factual pragmatic knowledge is an essential con-
dition to come up with these fine grained interpretations of cartoons. But the
point that we want to pursue is that a generic cognitive mechanism (i.e., set-
member classification) is at work which accounts for the evaluative aspect of
visual metaphor in cartoons as it is triggered by pictorial details. We are
only beginning to understand the kind of pictorial devices that can be inter-
preted in terms of point of view. Bodily expressions and the scaling (i.e., the
relative size) of objects are good candidates. For example, the bad cook idea
in figure 8 seems to rely on the relative size of the cook and his cooking pot,
in that the small cook cannot be supposed to master his much too large cook-
ing pot. Figure 9 is another example. At first sight, the cartoon just seems to
portray the presidential election as a boxing game with Bush and Kerry as
boxers in the ring, and thus to require only a schematic interpretation. But a
closer look reveals a bigger glove on Kerrys right hand with the word Iraq
projected on it. This enlarged glove suggests a categorical interpretation, i.e.,
that it is a member of the set of decisive weapons, which results in the inter-
pretation that Kerry has the better weapon; he can attack Bush with the war
in Iraq and therefore he is likely to win the elections (or at least, he himself
believes this to be so). So, again, the gist of the message, and the critical
position of the author is accounted for by a categorical aspect of the source
domain, highlighted by pictorial details.
The recognition of this set-member mechanism does not necessarily mean
that the critical stance in any cartoon is based on this same mechanism.
Many more cartoons have to be analyzed to justify this conclusion. But the
analysis shows that categorical reasoning can be used successfully as a heu-
ristic in finding the critical stance in editorial cartoons.
If indeed categorical interpretations are responsible for the (often nega-
tive) critical stance of editorial cartoons, they may be rare in commercial ads
in which such a critical position is absent. Again, an empirical account of
this hypothesis requires a large scale study of metaphors in commercial ads.
For the time being, we contend that categorical source domain interpreta-
tions are relatively rare in commercial ads, but certainly not absent. As we
discuss in more detail in Maes and Schilperoord (2008), there seem to be at
least two types of commercial ads for which a categorical source domain
interpretation is crucial, but not related to a critical stance. The first type is
represented by ads in which one attribute of a product is highlighted instead
of a network of relations and attributes within a schema. Take for example
an ad for a four-wheel drive vehicle representing the car together with some
hippos in the water, thus evoking the idea of imperturbable robustness. In
cases like these, the artful deviation of the image is based on the activation
of an ad hoc class, in this case the class of robust things, with a hippo as a
prototypical member.
Another type of advertisements that seem to require a categorical inter-
pretation are the so-called real thing ads. Commercial ads for products like
orange or tomato juice, mineral water or canned fish often express the mes-
sage that the products are as fresh and tasty as the real thing. Such ads at-
tempt to counter the commonly held opinion by consumers that such prod-
ucts are just a surrogate. In our view, the real thing idea is based on a
categorical source domain interpretation. The viewer is stimulated to believe
that the juice, salmon, or mineral water is a member of the set prototyped by
the real thing.
In sum, the notion of categorical source domains seems to offer a suitable
interpretation mechanism for explaining the critical stance expressed in edi-
torial cartoon metaphors. The mechanism is assumed to come on top of a
basic schematic source domain interpretation, as is illustrated in the analyses
above. In a particular schema, specific roles trigger the construction of ad
hoc categories enabling viewers to find the diagnostic property which is re-
sponsible for the critical stance of the cartoon. The interpretation mechanism
is not exclusive for editorial cartoons and can be used to trigger negative as
well as positive diagnostic properties. But in interpreting editorial cartoons,
it most probably is the dominant mechanism accounting for the critical
evaluative stance.
6. Conclusion
In this chapter we have argued that most schematic source domains of meta-
phors in editorial cartoons require an additional categorical source domain
interpretation, which results in the detection of the critical stance which is
typical of editorial cartoons as a genre. The ultimate message a cartoon
communicates is criticism, or a particular political stance towards a certain
public topic, and this is, we believe, best accounted for by the categorical
aspect of the source domain

6.1 The persuasive force of message form
In its essence, metaphor is argumentative, so if we are to properly character-
ize the role metaphor plays in editorial cartoons, the notion of persuasion
should be focused on. Socio-psychological theories of persuasion mostly
stress the importance of exchanging information in persuasion processes.
Fishbein and Ajzens (1975) famous model of persuasion is essentially an
information processing model of influencing attitudes. In their view, attitudes
have an informational base. However, from a rhetorical point of view, per-
suasion is first and foremost the process of directly trying to regulate the
behaviors or points of view of an audience. This raises the question of the
role metaphor plays in persuasion.
Metaphor has long been considered an ornamental addition to the thing
that really matters in communication: i.e., its content. Ever since Lakoff and
Johnson (1980), we know that metaphor is the content of the message and
this is, we believe, rhetoric in its most essential appearance. Applied to our
rhetorical case here, this means that especially the choice of the source do-
main is responsible for the type of conclusion we are supposed to draw on
the basis of the image. One famous Dutch cartoon (one we already alluded to
in section 5) comments on the measures that the then Dutch minister of Jus-
tice, Donner, took upon him to combat terrorism in the Netherlands. The
cartoon depicts the minister as a small figure, standing between two large,
black, blood-thirsty dogs, one that has the word terrorism on its back and
the other the word anti-terrorism. Because the opposite forces in the target
domain are linked to the same source domain (of blood-thirsty, dangerous
dogs), the cartoon expresses the message that in this case the cure is at least
as harmful as the pain. In addition, by depicting the owner of the anti-
terrorism dog as much smaller than his dog, Donner is presented as someone
who is not capable of mastering his own dog (let alone the other one). Cer-
tainly, there is a message involved in this cartoon (with his new antiterror-
ism policy, the minister is playing with fire), but much more important here
is the persuasive impact of the message form (sameness, big versus small)
and the conclusions we have to draw from it. These are determined by the
rhetorical choices, especially the strategic choice of the source domain, and
the visual details providing us with the conceptualization of the problem and
the critical position towards it. That way, we think that the interpretation of
metaphors in editorial cartoons nicely illustrates the pervasive force of mes-
sage form in establishing meaning.

6.2 The multimodal nature of cartoon metaphors
In the preceding, we did not explicitly address the issue whether metaphors
in editorial cartoons are a multimodal phenomenon. El Refaie (this volume)
does address this issue. She contends that in the cartoons she has analyzed,
the viewer should probably succeed in identifying the target objects from the
caricatured features of the politicians appearing in her cartoons. Moreover,
viewers will also draw on broader contextual information in doing so. In
both cartoons, she states, the verbal messages also play a role in pointing the
viewer towards the precise targets. She has in mind the verbal signs that
often appear on persons or objects depicted in a cartoon, as the names of the
multinationals in figure 2, the surname Thom and the noun new voting sys-
tem in figure 8, etc. So, if indeed these verbal signs point the reader towards
the target entities, this would suggest the following metaphors: MULTINA-
TIONALS ARE DRIPS, THOM DE GRAAF IS CHILD, NEW VOTING SYSTEM IS
COOKING POT.
To address the question whether these metaphorical relations are real
multimodal metaphors, we should first ask ourselves what multimodal meta-
phors are. Forceville (2006) states that multimodal metaphors are metaphors
in different modes. His example is the cueing of the metaphor CAT IS ELE-
PHANT in an animation film. The designer could do this by depicting a cat
and have it make a trumpeting sound. In this case, the target would be trig-
gered visually, and the source by means of sound. And because vision and
sound are different modes, the resulting metaphor would be truly multimo-
dal.
If we apply this view to the cartoons in this chapter one may simply con-
clude that the above-mentioned cartoons with verbal cues are multimodal
metaphors, because visual and verbal codes are different representational
modes. But this is probably too simple. Look at the little cook in figure 8. If
the viewer misses the necessary factual knowledge and does not know who
secretary De Graaff is, the surname label (Thom) on the trousers will not be
helpful, and will not make his metaphoric conceptualization multimodal. So,
we should read and apply Forcevilles characterization in such a manner that
metaphors are multimodal only if the identification of a metaphorical term
really depends on the presence of that label. This is exactly the case in the
CAT IS ELEPHANT example. The removal of one of the modes (either the im-
age or the sound) would simply cause the metaphor to evaporate. All that
would remain is a picture of a cat or the sound of an elephant. If, on the
other hand, the proper name Thom is removed from the trousers of the cook-
ing child, this would not destroy the metaphor DE GRAAFF IS CHILD. So, it
would seem that the visual metaphors with proper noun label are not auto-
matically multimodal. All the proper name does, or is intended to do, is to
provide the viewer with an additional aid to make sure that the person is
recognized.
Not all verbal labels have the same function. Take the MEASURE/POLICY
IS COOKING POT metaphor in figure 8. At first sight, this metaphor appears
to be truly multimodal (albeit not a very exciting one), as there seems to be
only one way of identifying the source (the visual sign) and the target (the
verbal label sign on the cooking pot). Nonetheless, we believe there are at
least two reasons to doubt this characterization. The first has to do with the
metaphorical scenario itself. As soon as the viewer identifies the image as
metaphorical (an aspect of interpretation that is evidently triggered by prag-
matic considerations and knowledge of the genre) s/he would have little
trouble identifying the cooking pot as some policy, e.g., on the basis of a
straightforward entailment: if this secretary is portrayed as a cook, then the
cooking pot and the stuff he is preparing must be his policy.
The exact identification will, of course, depend on various external
sources of knowledge, such as the time of publication and broader contextual
information, for example the context of the ongoing public discussion in
news media about the new voting system. This pragmatic knowledge should
direct the viewer at the precise target (the new voting system), but not at the
type of target (policy making), since the latter is to be figured out on the
basis of the scenario evoked. And since that immediate context is evoked
visually as well, the metaphor NEW VOTING SYSTEM IS COOKING POT is at
least not entirely multimodal.
Our second reason for doubting the multimodal nature of this cartoon has
to do with the possibility that the cooking scenario is actually a broad visual
manifestation of one basic conceptual metaphor in the sense of Lakoff:
IDEAS ARE FOOD, which instantiates the more general conceptual metaphor
IDEAS ARE PHYSICAL OBJECTS. Verbal expressions that relate ideas to food
are omnipresent and well documented (see e.g., Gibbs 2005), which suggests
that this type of metaphoric conceptualization is cognitively anchored. We
are only beginning to ask questions regarding the way basic and complex
visual metaphors are anchored cognitively, but it does not seem too far-
fetched to hypothesize that making food (cooking), so often employed in
cartoons, is particularly well suited to metaphorically conceptualize abstract
concepts such as political coalitions and policy making. If so, it may well be
that the verbal label in figure 8 is merely a secondary supporting feature
designating the exact type of policy hinted at, rather than the verbal part of
the multimodal metaphor.
To conclude, we are convinced that there is such a thing as multimodal
metaphor, but its precise delineation awaits further (preferably empirical)
research. At the very least, it seems too hasty to conclude that since meta-
phors in editorial cartoons use both visual and verbal signs, they are multi-
modal.
Notes
1. See en.wikipedia.org/wiki/Editorial_cartoons, consulted 2007-02-19.
2. The difference between these two types is not unproblematic, as we suggest
in the concluding section.
3. As we will discuss in the conclusion, not all of these metaphoric cartoons fit
in with Forcevilles (2006) definition of multimodal metaphor.
4. In the analyses to follow, we adopt the theoretical framework proposed by
Shen (1999) where two kinds of source domains are distinguished (see next
sections). This does not mean that Shens framework is the only one suitable
for the analysis of metaphorical conceptualizations in editorial cartoons. One
obvious alternative is Blending Theory (e.g., Coulson and Oakley 2001; Fau-
connier and Turner 2002). Many cartoons can in fact be considered a concep-
tual blend of (at least) two so-called input spaces, representing disparate con-
ceptual domains. For example, the Cuba cartoon in figure 6 blends the
conceptual domains of the schools (teachers, pupils) and that of international
political relationships into one integrated conceptual domain. Although
Blending theory is not a theory of metaphor proper, but a general theory of
cognitive conceptualization, it has been shown to be fruitfully applicable to
(visual) metaphors as well (Coulson in press). The blended conceptual do-
main consists of structural analogies between the two input spaces to the ef-
fect of establishing partial correspondences between distinct conceptual do-
mains (such correspondences are modelled to constitute a so called generic
space). The crucial structural analogy between the input spaces in the Cuba
cartoon pertains to the relation between pupils and teachers in the school in-
put space, and the Cuba and US relation in the international politics input
space. The same kind of conceptual integration applies to the other cartoons
as well. The football cartoon, for example, blends the conceptual domain of
celestial systems and of international football tournaments, while the cartoon
in figure 7 can be seen as a blend of the conceptual domains of hospitals and
politics.
Our reason for nonetheless adopting Shens framework is strictly practi-
cal, as it emphasizes the role of conceptual/cognitive schemata in the process
of interpreting (visual) metaphors. As we hope to make clear, his idea of
source domains as cognitive schemas may be very useful in order to account
for the way metaphor is involved in depicting certain political and socio-
cultural events and persons, while his concept of categorical source domains
allows us to account for the way the artist evaluates such events or persons.
However, in most cases it is not difficult to envisage an analysis grounded in
Blending Theory, because both frameworks strongly employ Gentners sys-
tematicity principle (see Gentner et al 2001), e.g., the fact that in order to es-
tablish correspondences between disparate conceptual domains, people will
primarily look for structural analogies between the various elements and enti-
ties within the two domains.
References
Becker, Jonathan A.
1996 A disappearing enemy: The image of the United States in Soviet
political cartoons. Journalism and Mass Communication Quarterly
73: 609619.
Bowdle, Brian F., and Dedre Gentner
2005 The career of metaphor. Psychological Review 112: 193215.
Cienki, Alan, and Cornelia Mller (eds.)
2008 Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins.
Coulson, Seana
in press Whats so funny: Conceptual blending in humorous examples. In
The poetics of cognition: Studies of Cognitive Linguistics and the
Verbal Arts, Vimala Herman, (ed.). Cambridge: Cambridge Univer-
sity Press.
Coulson, Seana, and Todd Oakley
2001 Blending basics. Cognitive Linguistics 11 (34): 175196.
Deutscher, Guy
2005 The Unfolding of Language: An Evolutionary Tour of Mankinds
Greatest Invention. New York: Metropolitan Books.
Edwards, Janis L.
2001 Running in the shadows in campaign 2000: Candidate metaphors in
editorial cartoons. American Behavioral Scientist 44 (12): 2140
2151.
Feldman, Ofer
1995 Political reality and editorial cartoons in Japan: How the national
dailies illustrate the Japanese prime minister. Journalism and Mass
Communication Quarterly 72: 571580.
Fishbein, Martin, and Icek Ajzen
1975 Belief, Attitude, Intention and Behavior: An Introduction to Theory
and Research. Reading, MA/London: Addison-Wesley.
Forceville, Charles
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame
Gentner, Dedre, Brian F. Bowdle, Philip Wolff, and Consuela Boronat
2001 Metaphor is like analogy. In The Analogical Mind: Perspectives
from Cognitive Science, Dedre Gentner, Keith J. Holyoak, and
Boicho N. Kokinov (eds.), 199253. Cambridge, MA: MIT Press.
2005 Embodiment in metaphorical imagination. In Grounding Cognition.
The Role of Perception and Action in Memory, Language and
Thinking, Rolf A. Zwaan and Diane Pecher (eds.), 6592. Cam-
bridge: Cambridge University Press.
2006 Metaphor interpretation as embodied simulation. Mind and Lan-
guage 21: 434458.
Gilmartin, Patricia, and Stanley D. Brunn
1998 The representation of women in political cartoons of the 1995
World Conference on Women. Womens Studies International Fo-
rum 21: 535550.
Glucksberg, Sam
2001 Understanding Figurative Language: From Metaphors to Idioms.
Groarke, Leo
2002 Toward a pragma-dialectics of visual argument. In Advances in
Pragma-Dialectics, Frans van Eemeren (ed.), 137151. Amsterdam:
Sic Sat.
Kenney, Keith, and Linda M. Scott
2003 A review of the visual rhetoric liturature. In Persuasive Imagery. A
Consumer Response Perspective, Linda M. Scott and Rajeev Batra
(eds.), 1756. Mahwah, NJ/London: Lawrence Erlbaum.
1980 Metaphors We Live by. Chicago: University of Chicago Press.
Western Thought. New York, NY: Basic Books.
Maes, Alfons, and Joost Schilperoord
2008 Classifying visual rhetoric. Conceptual and structural heuristics. In
Go Figure! New Directions in Advertising Rhetoric, Edward F.
McQuarrie and Barbara J. Phillips (eds.), 227253. Armonk, NY:
ME Sharpe.
McCarthy, Michael S., and David L. Mothersbaugh
2002 Effects of typographic factors in advertising-based persuasion: A
general model and initial empirical tests. Psychology and Marketing
19: 663692.
McQuarrie, Edward F., and David G. Mick
1992 On resonance: A critical pluralistic inquiry into advertising rhetoric.
Journal of Consumer Research 19 (2): 180197.
1996 Figures of rhetoric in advertising language. Journal of Consumer
Research 22 (4): 424438.
1999 Visual rhetoric in advertising: Text-interpretive, experimental, and
reader-response analyses. Journal of Consumer Research 26: 3754.
2003a The contribution of semiotic and rhetorical perspectives to the ex-
planation of visual persuasion in advertising. In Persuasive Im-
agery. A Consumer Perspective, Linda M. Scott and Rajeev Batra
2003b Visual and verbal rhetorical figures under directed processing ver-
sus incidental exposure to advertising. Journal of Consumer Re-
search 29: 579587.
McQuarrie, Edward F., and Barbara J. Phillips
2005 Indirect persuasion in advertising. Journal of Advertising 34: 720.
McQuarrie, Edward F., and Barbara J. Phillips (eds.)
2008 Go figure! New Directions in Advertising Rhetoric. Armonk, NY:
ME Sharpe.
Morrison, Susan S.
1992 The feminization of the German Democratic Republic in political
cartoons 198990. Journal of Popular Culture 25: 35.
Mothersbaugh, David L., Bruce A. Huhmann, and George R. Franke
2002 Combinatory and separative effects of rhetorical figures on consum-
ers effort and focus in ad processing. Journal of Consumer Re-
search 28: 589602.
Ortony, Andrew
1979 Beyond literal similarity. Psychological Review 86:161180.
2000 The impact of verbal anchoring on consumer response to image ads.
Journal of Advertising 29: 1524.
2003 Understanding visual metaphor in advertising. In Persuasive Im-
agery. A Consumer Perspective, Linda M. Scott and Rajeev Batra
2002 The development, change, and transformation of rhetorical style in
magazine advertisements 19541999. Journal of Advertising 31: 1
14.
2004 Beyond visual metaphor: a new typology of visual rhetoric in adver-
Plumb, Steve
2004 Politicians as superheroes: The subversion of political authority
using a pop cultural icon in the cartoons of Steve Bell. Media, Cul-
ture and Society 26: 432439.
Scott, Linda M.
1994 Images in advertising: The need for a theory of visual rhetoric.
Journal of Consumer Research 21: 252273.
Scott, Linda M., and Rajeev Batra (eds.)
2003 Persuasive Imagery: A Consumer Response Perspective. Mahwah,
NJ/London: Lawrence Erlbaum.
Shen, Yeshayahu
1999 Principles of metaphor interpretation and the notion of domain: A
proposal for a hybrid model. Journal of Pragmatics 31: 16311653.
Teng, Norman Y.
2006 Metaphor and coupling: An embodied, action-oriented perspective.
Tom, Gail, and Anmarie Eves
1999 The use of rhetorical devices in advertising. Journal of Advertising
Research 39: 3943.
Van Mulken, Margot
2003 Analyzing rhetorical devices in print advertisements. Document
Design 4: 114128.
Van Mulken, Margot, Renske Van Enschot-van Dijk, and Hans Hoeken
2004 Puns, relevance and appreciation in advertisements. Journal of

IV

Metaphors of Emotion in Comics, Manga,
and Animation
Chapter 11
Anger in Asterix: The metaphorical representation
of anger in comics and animated films
Bart Eerden
Abstract
This chapter analyses anger in two Asterix comics albums and two Asterix
animated films. This will on the one hand yield, enrich, and qualify Forcevilles
(2005a) earlier findings on the visual representation of the Idealized Cognitive
Model of anger in the Asterix album La Zizanie and on the other hand enable
insights into medium-specific representations of the ICM of anger. In
investigating the structural part of the emotion metaphors, handbooks for comics
and animated films will be taken into account as well, thus allowing for
comparisons between theories developed in metaphor scholarship and the
practices of animation artists. The methodological framework constructed for the
analysis in this paper is applicable to (a) other emotions and (b) other pictorial
manifestations of emotions in static and moving images.

Keywords: Idealized Cognitive Model, anger, emotions, comics, animation
1. Introduction
In cognitive metaphor research, metaphorical expressions are the verbal
manifestations through which Idealized Cognitive Models (ICMs) or Folk
Models (Lakoff and Johnson 1980, 1999; Lakoff 1987) can be charted.
Zoltn Kvecses has systematically analyzed the ICMs of various emotions
(Kvecses 1986, 1990, 1995, 2000, 2002 and 2005). Based on verbal evi-
dence in several languages, Kvecses argues that people structurally concep-
tualize emotions metaphorically. For example he is doing a slow burn and
he spat fire can be traced back to the concept ANGER IS FIRE. By contrast,
he has a ferocious temper and he unleashed his anger draw on ANGER IS
244 Bart Eerden
A CAPTIVE ANIMAL (Kvecses 2000: 21). Over the past decade the research
of visual and multimodal metaphors within the cognitivist paradigm has
taken shape in the work of Carroll (1996), Forceville (1996, 2002, 2005b,
2006/this volume, 2008), Kennedy (1993), Khordoc (2001), Whittock
(1990), but most of this work focuses on creative rather than on structural
metaphors. Kvecses model, however, provides a good starting point for the
investigation of structural emotion metaphors in non-verbal and multimodal
representations.
Elaborating on findings by Kennedy (1982, 1993), Forceville (2005) ex-
amines visual representations of anger in an Asterix comic. Comics provide
good source material for research, because of their rich use of pictorial
metaphors to convey a vast array of emotions (see also Fein and Kasher
1996; Khordoc 2001). Forceville (2005) introduces various pictorial signs
such as red face, spirals and bulging eyes that are frequently used in
the Asterix comic to depict anger. The nature and use of these signs appear
to confirm that these are not just creative metaphors in the sense of Black
(1979) and Forceville (1996) but indeed manifestations of structural meta-
phors (like the he spat fire example) in the sense of Lakoff and Johnson
(1980, 1999).
Whereas Forceville confined himself to anger in one comics album, this
chapter, building on Eerden (2004), will analyze anger in two other Asterix
comics albums and two Asterix animation films. This will on the one hand
enrich and qualify Forcevilles earlier findings and on the other hand enable
insights into how yet another medium (animated film) can represent the ICM
of anger. In investigating the structural part of the emotion metaphors, hand-
books for comics and animated films will be taken into account as well, thus
allowing for checking theories from the realm of metaphor scholarship
against the practices of comics and animation artists. The methodological
framework constructed for the analysis in this paper is intended to be subse-
quently applicable to (a) other emotions and (b) other pictorial manifesta-
tions of emotions in static and moving images.
2. Non-verbal metaphor
The perception psychologist Kennedy is one of the first to mention cartoons
as a source of non-verbal metaphors (Kennedy 1982). Consider the wavy
lines used by a cartoonist to depict smoke. These lines realistically represent
a visible phenomenon. Now imagine the same wavy lines used in a cartoon
to represent the strong smell of garbage. These lines are not realistic but
Anger in Asterix 245
exemplify a metaphorical representation of (invisible) smell (example from
McCloud 1994: 128). In the same way, comics use realistic and non-realistic
signs to depict emotions. Much of the importance of Kennedys work resides
in the fact that he argues how the signs used in comics need not always be
derived from verbal metaphors: there may be pictorial devices which are
metaphoric but which have no clear equivalent in language (Kennedy 1982:
600).
Kennedy introduces the term pictorial rune for non-realistic visual
metaphors. According to Kennedy, pictorial runes are often used to depict
abstract concepts which are difficult to depict literally. Emotions are a good
example of such abstract concepts: States such as anxiety and pain are
difficult to depict []. Cartoonists often turn to pictorial runes to show
these states (Kennedy 1982: 600). Kennedys observation of the representa-
tion and understanding of abstract concepts through concrete perceptible
phenomena in fact reflects one of the central principles of Cognitive Meta-
phor Theory.
Kennedys concept of pictorial runes is further fine-tuned by Forceville
(2005a). Forcevilles analysis of the non-verbal representation of anger in
the Asterix album La Zizanie (Goscinny and Uderzo 1970) introduces two
categories of visual representations of anger, based on a metonymic relation
between sign and emotion. One category consists of pictorial runes, a term
reserved for non-realistic (not perceptible in real life) signs. The second
category is formed by indexical signs. The indexical sign differs from the
pictorial rune in that it is a realistic sign (although often exaggerated). Apart
from its realism, the indexical sign seems to function much like a pictorial
rune. Both pictorial rune and indexical sign signify anger through a meto-
nymic relation (as opposed to arbitrary signs or literal depiction).
3. Anger in comics
Before presenting the results of the analysis of anger in two Asterix comics,
it is important to look at the results of Forcevilles (2005a) earlier analysis.
Forcevilles analysis of anger in La Zizanie will be the reference point for
the current analysis. La Zizanie contains 103 angry characters, and anger is
expressed through twelve different signs (Forceville 2005a: 7577). The
twelve angry signs can be further divided in two categories: pictorial runes
(ex-mouth,
1
spirals around the head, smoke above a head, bold face and
jagged text-balloon lines) and indexical signs (bulging eyes, tightly closed
246 Bart Eerden
eyes, wide mouth, tightly closed mouth, red face, arm/hand position and
shaking.
Table 1. Signs of anger in percentages of total number of angry characters per
album. (Percentages in the table are rounded to whole percents. For ex-
ample, bulging eyes occur 47 times in La Zizanie on a total of 103 an-
gry characters. This means that in 46 (45,63) percent of all angry char-
acters in La Zizanie bulging eyes is present.)
La Zizanie Asterix Lgionnaire Asterix et Latraviata
Pictorial runes
Smoke/steam 1 2 -
Helmet - 3 -
Ex-mouth 13 21 16
Bold face 38 34 48
Jagged line 31 16 4
Spirals head 44 50 35
Indexical signs
Bulging eyes 46 42 59
Closed eyes 37 50 41
Wide mouth 4
0
34 48
Tight mouth 1
3
15 4
Red face 1
7
32 7
Hand/arm 5
0
59 70
Shaking body 6 5 1

The pictorial runes in La Zizanie, Forceville argues, are metonymically mo-
tivated, just as the indexical signs (Forceville 2005a: 82). More importantly,
the runes in La Zizanie, in some cases, appear to be direct manifestations of
the ICM of anger, thus confirming Kennedys claim that pictorial metaphors
do not necessarily have a verbal counterpart (Kennedy 1982: 600). However,
Forceville acknowledges the possibility that the runes under scrutiny here
have become conventionalized as, somehow, visual translations of verbal
manifestations of ICMs (Forceville 2005a: 83). Both indexical signs and
pictorial runes are commensurate with the important central conceptual
metaphor ANGER IS A HOT FLUID IN A PRESSURIZED CONTAINER (as dis-
cussed by Kvecses 1986).
In this chapter, two other Asterix comics, Asterix et Latraviata (Uderzo
2001) and Asterix Lgionnaire (Goscinny and Uderzo 1967) are analyzed to
test the insights of Forceville (2005a).
2
Asterix et Latraviata is a relatively
recent album whereas Asterix Lgionnaire was published in 1967, three
years before La Zizanie. With almost twenty-five years between the two
albums, they represent a cross-section of the Goscinny/Uderzo oeuvre.
The signs depicting anger in Asterix Lgionnaire and Asterix et La-
traviata correspond largely, in form and in frequency, to the signs in La
Zizanie. See table 1 for the results, including Forcevilles.
As is noticeable from table 1, the two eyes, mouth, and the
hand/arm categories are the most frequently used signs in the three Asterix
albums. Helmet is a new pictorial rune, which does not appear in Force-
villes list. A helmet flying off a head as well as a character with smoke or
steam around its head appear only three times but these signs are certainly
worth mentioning.
3
They are good examples of the depiction of different
categories or stages of anger. The helmet flying off is a sign that occurs with
the outburst of anger. Smoke as a sign occurs right before the explosion of
anger when the emotion is (still) suppressed. This stage of anger is described
by Forceville as typically [comprising] a tightly closed mouth, often em-
phasized by clamped teeth and the hidden arm/hand position (Forceville
2005a: 8383).
The hand/arm sign is a rather large category of different arm positions
occurring in different stages of anger. Forceville (2005a) uses a threefold
distinction to differentiate the arm/hand category: (i) fisted hands; (ii)
hands/arms emphatically close to the body; and (iii) pointing with the index
finger. Positions (i) and (ii) are particularly associated with attempts to con-
trol anger whereas position (iii) suggests the eruption of anger. A closer look
at the hand/arm category in the two Asterix albums reveals two additional
elements of this sign, upright position and stretching forward. The first
element (upright) seems to appear with Forcevilles description of
arm/hand position (i) and (ii), with fisted hands or arms close to the body (as
if to keep the anger inside the body-container). These positions of arm/hand
often occur in combination with an upright position of the body in which the
head is tilted with the nose pointing upwards.
4
The second element (stretch-
ing forward) typically co-occurs with the outward pointing arm/hand posi-
tion (iii) in an outburst of anger. The forward stretched body appears to
convey the eruption of the pressurized container. Moreover, stretching for-
ward seems to be preceded by the upright position, thus conveying a
248 Bart Eerden
sequence of anger consisting of two stages, (a) the rising or suppression of
anger, followed by (b) the explosion of anger. However, in the static images
of a comics album, upright and stretching forward are difficult to rec-
ognize. This further distinction of the hand/arm sign will become relevant
in the discussion of animation film in section 5.
4. Handbooks for comics and animation films
Kvecses (2000, 2005) and Forceville (2006) as well as Lakoff and Turner
(1989) and Gibbs and Steen (1999) emphasize the influence of culture and
context on (the representation of) ICMs and the interpretation of source do-
mains, which is an important issue for the current analysis. After all, only
three comics albums by the same artist have been examined in this chapter.
Previous research (Kvecses 2005) shows evidence of quite fundamental
differences between for example the verbal manifestations of the Chinese
and Western container metaphor for anger. Yu (1998) describes how the
Chinese container metaphor, contrary to its Western equivalent, does not
involve a hot fluid but a gas which produces pressure on the body container
and which is not associated with heat. Shinohara and Matsunaka (2003)
demonstrate that in Japanese the metaphor ANGER IS GASTRIC CONTENTS IN
THE DIGESTIVE SYSTEM is saliently present. But within Western culture,
differences between verbal (and visual) metaphors in different languages
seem minor, suggesting that the most important conceptual metaphors of an
ICM transcend at least national/language borders.
Handbooks for comics and animated films in western culture can prove to
be helpful in further investigating the structural part of the anger representa-
tions. The results of a short survey of handbooks (Blair 1994; Maestri 1999,
2002; Thomas and Johnston 1981; Williams 2001) not only show a great
similarity between the advice proffered by the different handbooks among
themselves, but also between the handbooks and the findings on anger in the
three Asterix comics investigated. The use of, and emphasis on, the same
signs to represent anger in different examples suggests the existence of em-
bodied concepts that are, to a certain extent, cross-culturally represented
through the same signs in comics and animation.
Blair (1999) describes a number of prototypical characters and facial ex-
pressions pertaining to standard emotions. His description of the standard
anger face strikingly resembles Forcevilles description of the visualization
of anger in La Zizanie. According to Blair, a prototypical angry character
has fisted hands, clamped teeth, V-shaped heavy brows and small, black
pupils located in the corner of the eyes (Blair 1999: 24, 52, 53). All these
elements appear in Forcevilles analysis as well as in the current analysis of
two new Asterix albums (arm/hand (i), tight mouth and bulging eyes).
My earlier claim (Eerden 2004) about the existence of strong signs
which prototypically represent a certain emotion, and weak signs which
can only represent a certain emotion in combination with other signs, is con-
sistent with Thomas and Johnstons (1981) views. They also emphasize that
emotions more often than not are represented by a certain combination of
signs. While a combination of signs visually represents an emotion, say an-
ger, an isolated sign from such a combination, or such a sign in combination
with other signs, might represent a headache (spirals) or a person who has
eaten spicy food (red face) rather than anger. According to Thomas and
Johnston, emotions are most importantly expressed via the eyes, and again
the similarities with angry eyes in the analyzed Asterix comics are evident.
As a final point, Thomas and Johnston describe how Disney Studios initially
tried to copy facial expressions from real actors. This method proved unsuc-
cessful and Disneys designers turned back to their drawing boards to ex-
periment with and try out other ways to represent emotions visually. This
seems to confirm a metonymically motivated connection between sign and
emotion instead of a sign iconically depicting an emotion.
In digital animation, to conclude this brief survey of handbooks, the same
few pictorial signs of anger can be found. Williams (2001) and Maestri
(1999, 2002) yet again emphasize the eyes (with brows) and the mouth as
the main sites for the expression of emotions. These are among the most
differentiated signs in the Asterix albums.
Although the fact that the handbooks corroborate the theoretical findings
should not lead to sweeping conclusions, it does present data in support of
the idea that pictorial anger signs metonymically instantiate conceptual
metaphors, specifically the ICM of anger. Moreover, the handbooks present
eyes as the most important part of the face for expressing an emotion,
which ties in with eyes being the largest and most differentiated category
in all three Asterix albums under scrutiny here. Interestingly, Kvecses
(1990) gives numerous examples of eyes as the source domain of emotion
metaphors in English and Hungarian. The verbal expressions found by
Kvecses, however, do not correspond with the signs found in the Asterix
albums. Whereas verbal expressions seem to focus on the presence and kind
of emotions (love showed in his eyes, I could see the fear in his eyes),
consistent with the idea of eyes as mirror of the soul (Kvecses 1990:
173), visual signs, at least of anger, appear to focus on representing the in-
tensity of an emotion in, and also around, the eyes (pouches, brows, lines).
250 Bart Eerden
Although different in appearance, both visual and verbal expressions of emo-
tions in eyes can be traced back to the conceptual metaphor EYES ARE CON-
TAINERS FOR THE EMOTIONS (Kvecses 1990; Lakoff and Johnson 1980).
These examples substantiate the hypothesis that visual metaphors can be
direct manifestations of an ICM instead of merely being visual translations
of verbal metaphors.
5. Animated films
This section examines the visual representation of anger in Asterix animation
films. The films under scrutiny here are Asterix et la Surprise de Csar
(Brizzi and Brizzi: France 1985) and Asterix chez les Brtons (van Lams-
weerde: France 1986). The first animated Asterix film is an adaptation of the
album Asterix Lgionnaire (Goscinny and Uderzo 1967). Since neither of
the two other albums exists as animated film, the second film was selected
more or less randomly. Although based on the original comics album Asterix
Lgionnaire (Goscinny and Uderzo 1967), Asterix et la Surprise de Csar is
very different from the original. Not only has the story been altered but,
more importantly for present purposes, there are salient differences in the
visual representation of anger due to the fact that animated film, unlike com-
ics, can actually show moving characters and moreover include sound. The
analysis of the representation of anger through different media is an impor-
tant way to determine the influence of a medium on its message, in this case
the pictorial metaphors. In the following paragraphs I will discuss the most
salient differences between these two media, and some consequences for the
analysis of anger.
First of all, comics are static and films are animated. This obvious fact
has a great influence on the appearance and interpretation of certain anger
signs. For example, the famous pictorial runes speed lines and shape
changes, used to indicate speed or motion in static images (Kennedy 1982:
501), are much rarer in animated films. The fact that animated films can
convey motion also sheds new light on certain anger signs. For example,
the hair of the characters turned out to be a difficult sign in comics. It is for
instance hard to see if the braids of Obelix (one of the main characters in
Asterix) swing up to express an emotion or just because they follow the
movement of his head. Hair as a sign is for that reason not included in the
analysis of the comics. In the animated films, by contrast, it is easy to see
how Obelix braids and moustache sometimes unrealistically swing up when
he expresses anger. Another example of different visualization in animated
films is the position or posture of a characters body. In section 3 the new
categories stretched forward and upright were introduced as subcatego-
ries of Forcevilles hand/arm sign. But in the context of comics these pos-
ture signs are difficult to define. A change in posture that is caused by an
emotion is difficult to recognize in static images, where every panel shows a
more or less new situation. In contrast, animated films consist of shots and
sequences instead of panels, which makes it easy to detect a change in body
position.
A second big difference is related to the moving aspect of animation, too.
Since a comics album consists of a sequence of separate panels, readers can
decide for themselves how long they want to look at a panel. In fact the panel
can be seen as the smallest unit of a comic. For the average viewer the
smallest unit of an animated film is one shot (since normally a viewer can
not discern a single frame but only the succession of frames). This makes it
difficult to analyze and compare the two animated films because the length
of shots in Asterix et la Surprise de Csar and Asterix chez les Brtons
varies from less than one second to more than ten seconds. This means that
mutually exclusive signs (in comics) such as wide mouth and tight
mouth can easily appear multiple times in a single shot with a single char-
acter. Moreover, the comparison between a sign in a panel and the same sign
in a shot is difficult to express in numbers. If bulging eyes appears in a
panel it counts as one appearance, but how to count bulging eyes in a shot
of half a second or ten seconds, and what if bulging eyes appears five
times in a single shot? The inevitable conclusion is that reliable quantitive
comparison between both media is impossible. However, it is possible to
analyze the use of emotion signs across the different animated films and to
compare similarities and differences between emotion signs used in different
media in a more general sense.
One last important difference has to do with the multimodality of the dif-
ferent media. According to Forcevilles (2006) list of modes, comics contain
pictorial signs, written signs, and gestures. Animated films contain the same
three modes plus spoken signs, sounds, and music. Since the present study
focuses only on visual representations of anger, spoken signs, sounds, and
music will not be included in the analysis.
6. Signs of anger in animated films
The new indexical sign stretched forward returns in the expression of an-
ger in the two animated films. This sign typically appears in combination
252 Bart Eerden
with the other newly introduced sign upright. These signs usually appear
one after the other, expressing the build-up, followed by the release of anger.
The upright position is a manifestation of the building up of pressure in the
container, usually in combination with arms close to the body and/or
clenched fists, both associated with keeping the pressure in the container
(Forceville 2005a: 81). The stretched forward sign expresses the release of
pressure from the body container. Stretched forward usually appears in
combination with stretched arms and pointing or fisted hands. The outward
pointing arms also suggest the release of pressure. For examples of both
signs, see figures 1 and 2.
5

Figure 1. Upright (hand-traced still from Asterix et la Surprise de Csar).
Apart from stretched forward and upright, which could already be seen
in comics, although not very clearly, some other completely new signs mate-
rialize in the animated films. These signs either do not appear in comics at
all or vary substantially from their equivalents in comics. In the following
analysis these signs will be examined more closely. An important aim of the
analysis is to find out if these new signs can be attributed to a conceptual
metaphor and if so, to which one.

6.1 Wide mouth
As with bulging eyes the wide mouth sign in animated films also ap-
pears to differ from the same sign in comics because of the multimodality
(especially the presence of the sonic modality) and the motion aspect of the
animated film. Because animated film is not static, it is even better able to
build up a sign through different stages. In the Asterix comics, a mouth
counts as wide if at least two of the following are visibly present: (i) the
tongue; (ii) teeth; (iii) (a) line(s) running over the cheek from the nose to the
corners of the mouth (Forceville 2005a: 76). In animated films, by contrast,
the wide mouth requires the presence of only one of these three features to


Figure 2. Stretched forward (hand-traced still from Asterix et la Surprise de
Csar).
communicate anger, and moreover is not restricted to the expressed anger
stage. This has to do with animated films ability to express the building up
of an emotion through time in a continuous sequence. This feature of ani-
mated film along with the presence of the sonic modality makes it much
easier to identify the emotion anger in a sequence. A wide mouth typically
represents the release of pressure from the body-container. An example of
wide mouth in animated film is provided in figure 3.

Figure 3. Wide mouth (hand-traced still from Asterix et la Surprise de Csar).

6.2 Bulging eyes

The two animated films show a wide variety of bulging eyes. All signs,
however, include a V-shaped brow with pouches and frowns as already de-
scribed by Forceville in the Asterix comics album (Forceville 2005a: 75).
The medium animated film and its specific features (moving images) possi-
bly plays an important role in the variation in pictorial signs of anger. The
animated films show a greater range in the depiction of bulging eyes. The
two animated films for example have bulging eyes wide open but also with
one eye closed and the other wide open or all the traits of bulging eyes with
254 Bart Eerden
the eyelids still slightly closed. Bulging eyes are a manifestation of the (re-
lease of the) internal pressure of the body-container.

6.3 Shaking
This sign expresses anger in the same way as does the shaking sign in
comics which consists of (i) multiple superimposition of a character and/or
(ii) a non-moving character depicted as loose from the ground (Forceville
2005a: 76). There is, however, a difference in form. Shaking can appear in
animated film without the superimposition described for the comics version
of shaking, and in other cases the head of the character is the only shaking
part. The absence of superimposition or extra lines inevitably means that
shaking is only recognized in a sequence of frames, but not in a single
(freeze)frame.
Shaking is a manifestation of the (immense) internal pressure of the
container. The shaking sign appears especially at moments where sup-
pressed anger turns into an outburst or when an outburst ends. An earlier
study of pictorial signs representing love in animated films suggested that
shaking is also used as a sign when love strikes a person or when a per-
son instantly falls out of love (Eerden 2004: 53). This particular example of
the shaking sign is limited to the head only.
The shaking of the head can be traced back to the folk theory of emo-
tions. Kvecses (1990, 2000, 2005) describes the folk theory according to
which an emotion can be characterized as a five-stage scenario. In this cog-
nitive model the emotion affects the self as a force that causes a change of
state. As a result the self loses control and at a final stage responds to the
emotion with emotional behavior (Kvecses 2000: 5859). The new sub-
category of shaking, which focuses specifically around the head, marks the
sudden entering of one of the final stages of an emotion. The character
instantly loses control.

6.4 Ex-mouth
In comics this sign has straight lines emitting from the mouth (Forceville
2005a: 77). The lines express something forcefully coming out of the mouth.
This could be explained as the release of pressure, which makes ex-mouth
a pictorial rune. On the other hand, the straight lines might simply represent
inadvertent spitting as an expression of anger. In that case ex-mouth
should fall in the category of indexical signs.
The ex-mouth sign appearing in the animated films obviously has
something forcefully coming out of the mouth but without the straight
lines visible in comics albums. These lines, it seems, are replaced by sound
(shouting) and an unrealistic stream of air emitting from the mouth. The
stream of air is perceptible in the waving of clothes or hair of the victim
who is shouted at. These unrealistic traits confirm the idea of ex-mouth
being a pictorial rune. There is no example of spitting with anger to be found
in either animated film. As with shaking, this is again an example of a sign
which loses all its speed lines or superimpositions when translated from
comic to animation. Probably the waving of clothes is a motion hard to de-
pict in comics, hence support is needed in the form of lines to accentuate the
release of pressure. This is a good example of how pictorial signs have me-
dium-specific manifestations. In animated films the lines disappear and are
replaced by sound. For an example of ex-mouth in animated films, see
figure 4.

Figure 4. Ex-mouth and the waving movement of clothes in the Roman soldiers
cape (hand-traced still from Asterix et la Surprise de Csar).
6.5 Upright

As an indexical sign for anger, upright has an erect position of the body
with the back of the head usually pressed against the neck and the nose
pointing upwards. The upright position of the body signals the build-up of
internal pressure in the body-container.
256 Bart Eerden
6.6 Stretched forward
Stretched forward is particularly associated with the outburst of anger and
is often preceded by the upright sign. Stretched forward usually appears
with arms pointing outwards and clenched fists or pointing hands. This sign
can be interpreted as the violent release of pressure from the container.

6.7 Low angle
At first glance, it appears as if the low angle-shot functions as an arbitrary
sign for anger.
6
This shot (see figures 5 and 6) is often used to show angry
or threatening characters. Film theorists such as Bordwell and Thompson
stress, however, that framing does not automatically possess absolute or
general meaning (Bordwell and Thompson 2004: 263). According to Bord-
well and Thompson the context of a film will determine the function of a low
angle shot.
If low angle is considered neither a Peircean symbol sign nor a
Peircean iconic sign, the question will be whether low angle can be ex-
plained as the visual representation of a conceptual metaphor of anger, mak-
ing it an indexical sign. Possibly, low angle is the representation of the
conceptual metaphor ANGER IS A DANGEROUS ANIMAL (an animal to which a
character looks up from a low angle). Yet Kvecses (1990: 63) does not
derive metaphors from this concept that confirm the above hypothesis. His
outline of ANGER IS A DANGEROUS ANIMAL focuses on the growing or height
of the angry person whereas low angle seems to convey the experience of
the victim watching a huge angry character.
Perhaps the sign low angle does not express anger, but fear. But then
again the analysis resulted in only one example of a point of view shot from
a frightened character. Possibly, the low angle sign represents a part of the
ICM of anger that is not, or at least differently, expressed in language. Since
ICMs have hitherto mainly been charted on the basis of verbal expressions,
the analysis of visual expressions might result in new representations of
known conceptual metaphors or possibly even new conceptual metaphors.
Kvecses (2005), following Grady, offers a possible explanation for cul-
tural variations between conceptual metaphors and their verbal representa-
tions. According to Grady, complex conceptual metaphors are built up from
a combination of simpler primary metaphors. In this way the complex meta-
phor ANGER IS A HOT FLUID IN A CONTAINER is for example based on a
combination of primary metaphors such as INTENSITY IS HEAT, INTENSITY IS
QUANTITY and INTENSITY IS SPEED (Kvecses 2005: 27). Although the pri-
mary metaphors seem universal, combinations of primary metaphors vary
between different cultures, thus resulting in different complex metaphors,
which accounts for cultural variation in verbal expressions of anger. This
same phenomenon seems to be relevant for different modes of communica-
tion. I suggest that different modes of communication can account for varia-
tions in conceptual metaphors in much the same way as cultural context
does.

Figure 5. Low angle (hand-traced still from Asterix et la Surprise de Csar).

Figure 6. Low angle (hand-traced still from Asterix et la Surprise de Csar).
7. Results
Many of the signs found in the analyzed comics and films are commensurate
with the results of previous research (Forceville 2005a, Eerden 2004). The
analysis of comics and animated films reported in this chapter has also re-
sulted in the identification of some new signs of anger. Some of these oc-
curred in the two Asterix comics albums, but were not reported by Forceville
258 Bart Eerden
(2005a); others appear to be specific for the medium of animation. The ques-
tion here is whether different signs of anger downplay or highlight different
parts of the ICM of anger in verbal, comics, and animated form, respec-
tively. Further analysis is necessary to chart the conceptual metaphors which
are expressed through the newly found animated signs.
The eyes and mouth signs, followed by arm/hand are omnipresent
in both comics and animated films. The analysis of handbooks confirms the
important role of the three largest categories of signs and the rich variations
existing within each category (Blair 1999; Maestri 1999, 2002; Thomas and
Johnston 1981; Williams 2001). These three categories are the most used
and most differentiated signs in the comics, animated films, and handbooks
examined. But however important these signs are in visual representations,
verbal equivalents seem hard to find.
An example of the differences between verbal and visual representations
can be found in the source domains for eyes. The verbal expressions
largely pertain to EYES AS CONTAINER FOR EMOTIONS with the emotion
being visible in the eyes (I could see the fear in his eyes, his eyes were
filled with anger, and love showed in his eyes). Comics and animated
films, by contrast, are able to express not only the presence of a certain emo-
tion, but also the intensity of an emotion like anger, as well as its stage. This
again confirms the idea that complex conceptual metaphors might be con-
structed and expressed differently in different media and modes of communi-
cation.
The representation of anger in animated films results in the identification
of at least one new sign (low angle) which does not seem to fit in with the
central conceptual metaphor of anger (ANGER IS A HOT FLUID IN A PRESSUR-
IZED CONTAINER) as derived from verbal expressions. While all the other
signs can be explained as referring to embodied metaphors, low angle has
no relation to the body of the angry person, which explains why it is not
compatible with the container metaphor. Kvecses also presents verbal ex-
pressions that do not refer to the container metaphor but to other conceptual
metaphors, such as ANGER IS A DANGEROUS ANIMAL or ANGER IS AN OPPO-
NENT. However none of these examples seem to be related to the low angle
sign. The low angle sign originates from the unique possibilities (such as
framing and motion) of the animated medium. Also low angle seems to
focus more on the perspective of a victim experiencing or witnessing the
anger expressed by another character. When a victim-character is not
present in the story, it is the viewer who experiences the anger through a
virtual point-of-view-shot.
Animated films and comics in general contain yet other examples of signs
conveying emotions that are not related to the body. One might think of a
light-bulb or rain cloud above someones head. Certainly this area of picto-
rial metaphor needs more research. Shinohara and Matsunaka (this volume)
give examples of such external signs as thunder, wind, or flowers often oc-
curring in Japanese Manga. They also give examples of background scenes
of panels which are used to express the emotional state of the character in
the panel. A similar version sometimes appears in the Asterix comics. This is
not included in the current analysis, as the signs do not seem to appear struc-
turally, but further research is important here. One such sign is green text
balloons, which appears in La Zizanie. Forceville labels this sign as arbi-
trary and thus excludes it from his analysis, using the sign as an independent
indicator of anger, since the green text balloons appear in over 50 percent of
the anger panels (Forceville 2005a: 75). The green text balloons seem even
more arbitrary because they do not appear in the next two albums. However,
a closer look at the emotion anger in Asterix et Latraviata and Asterix et la
Surprise de Csar shows many unrealistically colored backgrounds in pan-
els with angry characters. These background and text balloon signs might be
arbitrary, but in light of Shinohara and Matsunakas research it is possible
that these signs actually express certain conceptual metaphors.
The results in table 1 present hand/arm and eyes as the largest two
categories of signs, followed by mouth signs. On average, eyes appears
in 85 percent of anger panels, for hand/arm the average is 63 percent, and
the category mouth can be found in 51 percent of the anger panels. The
three categories are not only the largest in numbers but also the most differ-
entiated signs, both in comics and animated films. The three signs are at the
very least commensurate with the conceptual metaphor ANGER IS A HOT
FLUID IN A PRESSURIZED CONTAINER. I would venture the stronger claim
that the container metaphor is at the center of the representation of anger in
the analyzed comics as well as the animated films.
Kvecses emphasizes the role of the container concept as central to the
concept of anger. It seems warranted to conclude that certain complex and
central embodied concepts such as CONTAINER and FORCE play a central
role in metaphors, irrespective of medium or mode. However, based on ver-
bal evidence, the concept of anger consists of a number of other important
metaphors such as ANGER IS A BURDEN, ANGER IS AN OPPONENT IN A
STRUGGLE and ANGER IS A CAPTIVE ANIMAL (Kvecses 2000: 21). These
metaphors are good examples of concepts that are not primarily based on
embodiment but seem more related to behavioral aspects. It seems that vis-
ual representations of anger focus almost entirely on embodied container
260 Bart Eerden
concepts whereas verbal expressions regularly draw on non-embodied, be-
havioral concepts. Since culture and context influence behavior rather than
physical aspects, it appears that verbal expressions more strongly reflect
culture-specific expressions of anger. The visual part of comics and ani-
mated films, which draws heavily on embodied concepts, seems less influ-
enced by culture-specific metaphors.
The current analysis of pictorial signs in comics and animated films sup-
ports Forcevilles (2005a) and my own earlier (Eerden 2004) assumption
that pictorial signs provide medium-specific representations of the ICM of
anger. Presumably, the charting of ICMs through verbal expressions alone
creates blind spots. To achieve a complete insight in an ICM means one
has to study the metaphorical representations in every mode of communica-
tion. The emphasis on embodied concepts and the temporal development of
an emotion are examples of new insights from analyses of clearly defined
corpora, in this case three comics albums and two animated films. Future
research into other modes of communication will certainly yield new aspects
of ICMs that have hitherto been downplayed in verbal (and visual) contexts.
Tying in with the blind spots of the charting project is the idea that a
particular medium draws on specific medium-related possibilities and im-
possibilities to represent a certain emotion. Such medium-specific represen-
tations appear to occur on two levels. A specific medium draws heavier on
certain unique combinations of primary metaphors to construct complex
conceptual metaphors. Since animated film (as opposed to comics) can de-
pict movement, it will draw more on those primary metaphors concerning the
progression of an emotion through different stages. Moreover, some concep-
tual metaphors are in fact limited to a specific mode of communication, or
are at least very rare in other modes. Consider for example an indexical sign
that is tied to animated film such as low angle. This sign is all-pervasive in
the analyzed films and is an important sign in the representation of anger.
This metaphorical sign, however, does not seem to have a verbal equivalent.
8. Further research
Since research into non-verbal and multimodal metaphor is relatively new,
further research can move into largely unexplored areas. In addition, it is
important to verify or falsify the results of previous research. This is one of
the main reasons for me to use a clearly defined corpus of research. The
more eclectic approach of Kvecses or Lakoff and Johnson makes it more
difficult to compare results. Considering the current analysis and results,
some important research projects come to mind.
First of all, more research into the visual representation of anger and
other emotions is needed to substantiate and verify the results reported in the
current chapter. Research should, however, not be limited to comics and
animation alone. Other forms of communication and especially multimodal
communication should be included. Forceville (2006) distinguishes nine
different modes of metaphorical representation, including at least pictorial
signs, written signs, spoken signs, gestures, sounds, music, smells, tastes and
touch. Further research into these modes can present important insights. The
current analysis already suggests that certain medium-specific signs in com-
ics appear as sonic signs in animated films.
A particularly interesting area is the representation of emotions in various
new media. One can think of the representation of emotions through emoti-
cons and different fonts in internet communication or the use of emotion-
specific avatars; and of the representation of emotions in Massive Multi-
player Online Role Playing Games (MMORPGS) such as Second life.
MMORPGS can prove especially interesting for research into cultural varia-
tions because MMORPGS create a new cross-cultural online community.
Finally, Gibbs and Steen (1999) as well as Kvecses (2005) underline the
important influence of culture on ICMs. This suggests cross-cultural differ-
ences in the conceptualization and representation of emotions. Complex non-
western media such as manga and anime are important source material for
further investigation in cultural variations (Shinohara and Matsunaka 2003,
this volume). Especially for this kind of research and its comparison with
earlier findings, the use of a clearly defined corpus is important. Moreover,
the use of empirical research and verifiable experiments is crucial if we are
to present evidence about the structural use of metaphorical signs and con-
ceptual metaphors.
Notes
1. This sign shows straight lines emitting from the mouth.
2. The albums used here are Dutch translations (since language is not a part of
the analysis) of Asterix Lgionnaire (Goscinny and Uderzo 1967) and Asterix
et Latraviata (Uderzo 2001).
3. Both signs occur in Asterix Lgionnaire and steam/smoke occurs once in
Latraviata. The helmet can be seen on page 19 and 23 and the steam/smoke
occurs on page 15 and 16 and in Latraviata on page 17.
262 Bart Eerden
4. Forceville (2005a) also recognizes an upward tilted head in combination with
anger and suggests that it might correspond with pride.
5. For copyright reasons and in order to emphasize the relevant details, all
figures are hand-traced stills from Asterix et la Surprise de Csar (Brizzi and
Brizzi: France 1985).
6. Symbol is the term used by Charles Sanders Peirce. A symbol is character-
ized by the arbitrary link between sign and referent; for example words in a
language are symbolic signs.
References
Black, Max
Blair, Preston
1994 Cartoon Animation. Laguna Hills: Walter Foster.
Bordwell, David, and Kristin Thompson
2004 Film Art: An Introduction. 7th ed. Boston: McGraw-Hill.
Carroll, Noel
Cienki, Alan
Jean-Pierre Koenig (ed.), 189204. Stanford: Center for the Study of
Language and Information.
Eerden, Bart
2004 Liefde en woede: De metaforische verbeelding van emoties in
Asterix. [Love and anger: The metaphorical visualization of emo-
tions in Asterix.] MA thesis, Department of Media Studies, Univer-
sity of Amsterdam, The Netherlands.
Fein, Ofer, and Asha Kasher
1996 How to do things with words and gestures in comics. Journal of
Forceville, Charles
bol 14: 17998.
2005a Visual representations of the Idealized Cognitive Model of anger in
2005b Cognitive linguistics and multimodal metaphor. In Bildwissenschaft
zwischen Reflexion und Anwendung, Klaus Sachs-Hombach (ed.),
264284. Cologne: Halem.
work: agendas for research. In Cognitive Linguistics: Current Ap-
2008 Metaphor in pictures and multimodal representations. In The Cam-
Gibbs, Raymond W., Jr., and Gerard J. Steen (eds)
jamins.
Goscinny, Ren, and Albert Uderzo
1967 Astrix Legionnaire. Paris: La Hachette.
1970 La Zizanie: Une Aventure dAstrix. Neuilly-sur-Seine: Dargaud.
Khordoc, Catherine
2001 The comic books soundtrack: Visual sound effects in Asterix. In
The Language of Comics: Word and Image, Robin Varnum and
Christina T. Gibbons (eds.), 156173. Jackson: University Press of
Mississippi.
Kennedy, John M.
1993 Drawing and the Blind: Pictures to Touch. New Haven/London:
Yale University Press.
Kvecses, Zoltn
Benjamins.
1990 Emotion Concepts. New York: Springer.
Press.
Lakoff, George
264 Bart Eerden
Maestri, George
1999 Digital Character Animation: Volume 1 Essential Techniques. Indi-
ana: New Riders.
2002 Digital Character Animation: Volume 2 Advanced Techniques.
Indiana: New Riders.
McCloud, Scott
1994 Understanding Comics: The Invisible Art. New York: HarperPeren-
nial.
2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen:
Journal of Yokohama Linguistic Circle 4: 118.
Thomas, Frank, and Ollie Johnston
1981 Disney Animation: The Illusion of Light. New York: Abberville
Press.
Uderzo, Albert
2001 Astrix et Latraviata. Paris: Les Editions Albert Ren.
Whittock, Trevor
Williams, Richard
2001 The Animators Survival Kit. London: Faber and Faber.
Yu, Ning

Chapter 12
Pictorial metaphors of emotion in Japanese comics
Kazuko Shinohara and Yoshihiro Matsunaka*
Abstract
In this chapter, we analyze pictorial metaphors of emotion in Japanese comics
(manga) within the framework presented by Forceville (1994, 2005) and support
his argument that many of the metaphors expressed through the verbal modality
and those expressed through other modalities appear to share the same fundamen-
tal motivation, that is, mappings between conceptual domains (Lakoff and John-
son 1980). Our analysis of the data including emotion types such as anger, happi-
ness, love, anxiety, surprise, and disappointment shows that (1) the conceptual
metaphor ANGER IS A HOT FLUID IN A CONTAINER is shared by verbal and visual
modalities, that (2) differences can emerge due to the different properties of each
modality though basic conceptual mappings are shared by both modalities, and
that (3) culture-specific aspects found in verbal emotion metaphors in Japanese
may also be found in pictorial emotion metaphors in Japanese comics. We can
conclude that metaphor is a matter of concept and cognition not limited to lan-
guage.

Keywords: pictorial metaphor, indexical signs, emotion, manga, conceptual map-
pings
1. Introduction
Since the foundation of Conceptual Metaphor Theory (CMT) by Lakoff and
Johnson (1980), verbal metaphors that are motivated by cross-domain con-
ceptual mappings have attracted researchers attention and the theory itself
has experienced revision and elaboration (e.g., Lakoff 1990, 1993; Grady
1997; Lakoff and Johnson 1999; Kvecses 2000, 2002, 2005). This theory
has had a great impact on the view of metaphor, changing its fundamental
understanding from a rhetorical to a cognitive one. Though metaphor tends
to be thought of as a device of poetic ornamentation that is added to ordinary
266 Kazuko Shinohara and Yoshihiro Matsunaka
language, CMT claims that metaphor is pervasive in everyday life, not just
in language but in thought and action, and that [o]ur ordinary conceptual
system, in terms of which we both think and act, is fundamentally meta-
phorical in nature (Lakoff and Johnson 1980: 3). In CMT, a conceptual
metaphor is defined as the systematic mapping between two different con-
ceptual domains, and it is assumed that surface linguistic expressions of
such a metaphor emerge from that conceptual mapping. The central tenet of
CMT is that metaphor, by its very nature, not only affects surface linguistic
expressions but also characterizes cognitive/conceptual structure.
This tenet of CMT, however, has not been fully explored, although some
researchers have started to apply this idea to the study of multimodal meta-
phors. Forceville (1994) suggests that metaphors are not limited to linguistic
ones, and he maintains that to further validate the idea that metaphors are
expressed by language, as opposed to the idea that they are necessarily
linguistic in nature, it is necessary to demonstrate that, and how, they can
occur non-verbally and multimodally as well as purely verbally (2006:
381/this volume, emphasis in original). Thus, in recent developments of the
study of metaphor, researchers attention is being drawn to the new field of
multimodal metaphors. Metaphors expressed through the verbal modality
and those expressed through other modalities appear to share the same fun-
damental motivation, and thus they cannot be regarded as totally separate
phenomena. Although, of course, multimodal metaphors that are based on
nonverbal sources have unique properties that can be traced back to the na-
ture of the mediums modalities (Forceville 1994, 1996, 1999, 2005; Carroll
1994), they share the underlying mapping between conceptual domains with
verbal metaphors. Thus, previous studies on multimodal metaphors strongly
support the argument that metaphor is a matter of concept and cognition not
limited to language.
This chapter builds upon these previous studies, especially the one by
Forceville (2005), which deals with pictorial metaphors of anger in a French
comics album. Following Forcevilles view, we try to demonstrate that at
least some manifestations of anger are found in visual no less than in verbal
metaphors. We also try to extend Forcevilles view to other types of emo-
tions such as happiness, love, anxiety, surprise, or disappointment. To attain
this goal, we examine pictorial metaphors of emotion in Japanese comics.
The source of this type of metaphor is pictorial or visual, where meanings
are conveyed via pictorial or visual representations. The target is emotion,
which belongs to a more abstract domain of psychological experience. In this
metaphor, what is expressed as a picture can be interpreted as representing
some emotion. Data are taken from some of the present day popular Japa-
Pictorial metaphors of emotion in Japanese comics 267
nese comic (manga) books written by several different authors. Innumerable
comic books are published in Japan now, but despite the diversity, the data
and the analysis we are providing in this chapter will clearly show that there
are pictorial metaphors that manifest conceptual metaphors that are also
expressed verbally.
2. Previous studies and issues
2.1 Emotion metaphors in Japanese
Since Lakoff and Johnson (1980) established CMT in cognitive linguistics,
efforts have been made to establish generalizations for various emotion
metaphors. It has been argued that there are structural correspondences be-
tween the source and the target domains of each metaphor. For anger meta-
phors, in particular, it has been suggested that the correspondences are
mainly based on folk theories of anger that are commonly accepted within a
speech community.
1
For example, Lakoff (1987: 381382) describes the
common folk theory of anger in English-speaking communities as follows:
The physiological effects of anger are increased body heat, increased inter-
nal pressure (blood pressure, muscular pressure), agitation, and interfer-
ence with accurate perception. As anger increases, its physiological effects
increase. There is a limit beyond which the physiological effects of anger
impair normal functioning.
Many of the anger metaphors suggested by researchers are based on this or
similar folk theories. One such metaphor is ANGER IS A HOT FLUID IN A
CONTAINER (Kvecses 2000: 21), which is manifested in expressions of
anger.
2
The following are examples from English.

(1) a. You make my blood boil.
b. She is blowing off steam.

This is not, of course, limited to English. In Japanese, we can find examples
like the following.

(2) a. atama-kara yuge-ga deru
head-from steam-Nom. come-out
Steam comes out of (ones) head.
b. harawata-ga niekurikaeru.
gut-Nom. boil-up
Gut boils up.
This emotion metaphor is observed cross-linguistically. Kvecses (2000,
2002) examines several basic emotions in English, Hungarian, Chinese, and
Japanese, and arrives at a single underlying master metaphor: EMOTION IS
FORCE. These four languages and cultures are remarkably similar with re-
gard to their basic structure of emotion metaphors (Kvecses 2000: 146),
and this similarity is explained in terms of the same physiological processes
all humans experience when they are in the state of a certain emotion
(Kvecses 2000: 156). At the same time, it is observed that the general meta-
phor is elaborated in different ways at a more specific level of metaphor in
each language.
In Kvecses argument of the master metaphor EMOTION IS FORCE, a
great deal of attention is paid to the internal pressure in a container, which is
assumed to be the source concept of force. Previous studies on Japanese
metaphors confirm that Japanese also has such emotion metaphors (Matsuki
1995, Shinohara and Matsunaka 2003).

(3) a. kanashimi-de mune-ga harisakeru.
sorrow-by chest-Nom. tear
Chest tears with sorrow.
b. yorokobi-de mune-ga ippai-ni natta.
joy-by chest-Nom. full-Loc. became
Chest was filled up with joy.
c. nikushimi-ga mune-ni uzumaku.
hatred-Nom. chest-Loc. billow
Hatred billows in chest.

As these examples show, emotions such as sorrow, joy and hatred are
thought of as contents in the container (the chest). They either exert a kind of
force upon the container, which makes it tear apart or become full, or they
stay in the container in an unstable state, having a certain impact on the
container. Matsunaka and Shinohara (2001b) call this kind of force the in-
ner force, since the force affects the container from inside. In addition to
this, Matsunaka and Shinohara demonstrate that there are at least some ex-
amples of metaphors of emotions in Japanese that imply an outer force
rather than an inner force.

(4) a. kanashimi-ni uchi-nomes-areru.
sorrow-by beat-flat-Passive
(Self) is beaten by sorrow.
b. kanashimi-ni mune-ga shime-tsuker-areru.
sorrow-by chest-Nom. screw-put-Passive
Chest is screwed up by sorrow.
c. huan-ga mune-ni noshi-kakaru.
anxiety-Nom chest-Loc. make.flat-hang
Anxiety hangs over chest.
d. kyouhu-de mune-ga oshi-tsubus-areru.
terror-by chest-Nom. push-squash-Passive
Chest is squashed by terror.
e. huan-ga mune-ni oshi-yoseru.
anxiety-Nom. chest-Loc. push-approach
Anxiety inundates chest.
f. kyouhu-ni osow-areru.
fear-by attack-Passive
(Self) is swept by fear.

These examples show that the metaphorical force of emotion can exist out-
side the self or the part of the body that is regarded as the container of emo-
tion. The force of emotion affects the container from the outside, making its
shape change, or even breaking it down. Though previously it was assumed
that only very intense emotions are conceptualized as outer forces such as
natural forces (Kvecses 2000: 72), it has been found that weak or mild
emotions can also be conceptualized in terms of natural or meteorological
phenomena in Japanese. The following Japanese examples illustrate that
both intense and mild emotions can be expressed in terms of natural or mete-
orological phenomena (Matsunaka and Shinohara 2001b, Shinohara and
Matsunaka 2001).

(5) a. kokoro-ni honokana hikari-ga sasu.
heart-Loc. faint light-Nom. shine
A weak light brightens heart. (One feels relieved a little
bit.)
b. kumo-yuki-ga ayashii.
cloud-go-Nom. strange
The weather is getting squally. (A person is getting bad-
tempered.)
c. kuro-kumo-ga mune-ni ooi-kabusaru.
black-cloud-Nom. chest-Loc. cover-lap
A black cloud (=anxiety) covers chest.
d. kimochi-ga harebaresuru.
feeling-Nom. become.fine.weather
My heart clears up. (I feel very happy.)
e. kokoro-wa doshaburi da.
heart-Topic heavy.rain be
It rains heavily in (my) heart. (I am very sad.)
f. ikari-no tsunami-ga oshi-yoseru.
anger-Gen. tsunami-Nom. push-approach
Tsunami of anger inundates (self).
g. kokoro-ni arashi-ga hukiareru.
heart-Loc. storm-Nom. bluster
Storm (of anger) blusters in (my) heart.
h. kaminari-ga ochiru.
thunder-Nom. fall
Thunder falls. (A person gets very angry and scolds
another.)

As the above examples show, the intensity of natural phenomena corre-
sponds to the intensity of the emotion. Severe natural phenomena such as
tsunami or storms are mapped onto intense emotions. Less intense natural
phenomena like clouds, rain, or sunshine are likely to be mapped onto less
intense emotions. In general, anger may be intense and sadness or anxiety
may be less intense, but each emotion seems to have a range of intensity.
Through analyses of these examples, Shinohara and Matsunaka (2003)
suggest the existence of an emotion metaphor that they name EMOTION IS
EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS
THE SELF, at least in the Japanese language. Here, the mappings between
natural/meteorological phenomena and emotions seem to be experientially
motivated. Changes of weather can affect mental or physical states of human
beings. For example, depression can be caused by low atmospheric pressure
or by lack of sunshine. This is called seasonal affective disorder (Nihon-
Seikishoogakkai 1992: 4243). Moreover, this metaphor may also be attrib-
uted to the socio-cultural background of the Japanese. As Yamanaka (2003)
argues, Japan has a long tradition of regarding the heart as a microcosm,
which appears in many old Japanese poems. In such poems, the outside
natural phenomena reflect inner emotions of the poet, or the inner emotions
are regarded as natural/meteorological phenomena.

ikadekawa tori-no nakuran hito-shirezu
omou-kokoro-wa mada yo-hukaki ni (anonymous)
[Why does a morning bird chirp, even though my heart that yearns for you in
private is still deep in night?]

sabishisa-ya omoi yowaru-to tsuki mireba
kokoro-no sora-zo aki hukaku-naru (by Kujou, Yoshitsune)
[Feeling sad, in despair, and looking at the moon, my hearts sky is deepened
into autumn.]
takitsuse-ni hito-no kokoro-o miru-koto-wa
mukashi-ni ima-mo kawarazarikeri (by Go-suzaku-in)
[It is unchanged in all ages to regard river rapids as ones heart/mind.]

These wakas (Japanese old-style poems) date back to the 11th or 12th cen-
tury. Since that time, our hearts have been thought of as experiencing night,
seasons, or landscapes such as rivers. This cultural tradition of Japan may
be one of the background factors that motivate EMOTION IS EXTERNAL ME-
TEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF in
Japanese. If this emotion metaphor is culture-specific, then it may be due to
this type of tradition in Japanese society. However, the mappings between
natural/meteorological phenomena and emotions do not seem to be totally
arbitrary or unmotivated, but rather, they seem to be supported by the bio-
meteorological tendencies mentioned above. This tendency may not be so
universal as the rise of blood pressure and temperature during the experience
of anger and thus its physiological motivation for the conceptual metaphor of
emotion may perhaps be rather weak. Consequently, metaphors based on this
tendency may be culture-specific.
To sum up this section, we have argued that (1) the general schema of the
anger metaphor ANGER IS A HOT FLUID IN A CONTAINER is applicable to
Japanese emotion metaphors; (2) there is an elaborated distinction between
inner and outer force in Japanese emotion metaphors; (3) the source domain
of natural force is highly elaborated in Japanese emotion metaphors and
displays various levels of intensity; and (4) some Japanese emotion meta-
phors have mappings from natural or meteorological phenomena to emo-
tions, which are basically experientially motivated but allow for the possibil-
ity of culture-specificity. As the discussion proceeds, we will examine
whether these findings from a cognitive linguistic viewpoint also occur in
pictorial/visual metaphors of emotion in Japanese manga.

2.2 Semiotic characteristics of pictorial metaphors
In the previous section, we have reviewed the cognitive linguistic analysis of
emotion metaphors, which are verbal in nature. In recent developments of the
study of pictorial and multimodal metaphors analyses of the expressions of
emotion in comics have provided intriguing data and insights. An important
study in this line is the one by Forceville (2005). He takes data from a
French comic album, La Zizanie, and analyzes the pictorial expressions of
anger. The data are classified into two categories. Category I (straightfor-
ward indexical signs) includes bulging eyes, tightly closed eyes, wide
mouth, tightly closed mouth, pink/red face, arm/hand position, and
shaking. These are indexical signs since we recognize them as symptoms
accompanying anger from our everyday experience (Forceville 2005: 77).
Category II (pictorial runes, a term coined by Kennedy 1982) includes spi-
rals, ex-mouth, smoke, bold face, and jagged line. These are not
perceptible in real life, and their indexicality is therefore less evident than
those in Category I (2005: 77). Through the analysis of the collected exam-
ples of expressions of anger in La Zizanie, Forceville concludes, the picto-
rial runes signaling anger appear indeed to be Peircean indices rather than
Peircean symbols, since they are motivated rather than arbitrary signs
(2005: 82).
Here, motivatedness is an important notion. In CMT, it has been repeat-
edly argued and demonstrated that the most basic linguistic metaphors are
motivated in such a way that the motivation can be traced back to our per-
ceptual or bodily experience. One of the theses of cognitive linguistics is that
language is embodied and therefore our mind is fundamentally embodied.
The above-mentioned study of pictorial metaphors by Forceville and other
researchers along this line provide strong support for this view of mind. If
we can find additional evidence that demonstrates parallel structures between
linguistic metaphors and pictorial and multimodal metaphors, this will
strengthen the claim that metaphorical mappings are not merely a matter of
language but reside in a deeper layer in our mind, thus enabling manifesta-
tions in multimodality. Eerden (this volume) demonstrates that Forcevilles
arguments also apply to animated films. This constitutes evidence to support
this line of ideas. In our study we aim at the same goal, but use a different
kind of data. We will add evidence taken from works of Japanese comics
written by several authors.

2.3 Issues and goals
What we try to do in this study is two-fold. First, we will analyze pictorial
emotion metaphor in Japanese manga using the two categories suggested by
Forceville, Category I (indexical signs) and Category II (pictorial runes).
Our analysis will basically support Forcevilles (2005: 82) discussion that
the pictorial runes signaling anger are Peircean indices rather than Peircean
symbols. In addition to this, some of our examples will show how pictorial
runes can deviate from indexicality (being related to physical states of a
person in anger that actually occur). Some of the visual signs of anger in
Japanese manga, which are experientially motivated, can be drawn in physi-
cally impossible places (e.g., veins in the air). Such cases of deviation
sometimes cannot be verbally expressed, or they become hard to understand
if they are directly translated into words. Though they share cross-domain
mappings with the verbal expressions of conceptual metaphor ANGER IS A
HOT FLUID IN A CONTAINER, these pictorial anger metaphors have a broader
range of use than verbal expressions of this metaphor. Thus, we can see that
some pictorial metaphors have different and novel ways of representing the
conceptual metaphor of anger from verbal ones.
Second, a possibly culture-specific Japanese metaphor, EMOTION IS EX-
TERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE
SELF (Shinohara and Matsunaka 2003) is not restricted to verbal metaphors
but is also instantiated pictorially in Japanese comics. Thunder, wind, rain,
cloud, fog, flowers, birds, and other naturally occurring or existing phenom-
ena are used as the background-scene of panels, and can express emotional
states of a person described in a panel.
The sources of data that are used in this study are as follows.

(i) Kashimashi: Girl Meets Girl Vol. 1, by Akahori [story] and Katsura [art],
2005.
(ii) Azumanga Daiou, Vol.1, by Azuma, 2000.
(iii) Azumanga Daiou, Vol. 4, by Azuma, 2002.
(iv) Yotsubato, Vol. 5, by Azuma, 2006.
(v) Ichigo Mashimaro, Vol. 1, by Barasui, 2003.
(vi) Ichigo Mashimaro, Vol. 4, by Barasui, 2005.
(vii) Black Jack, Vol. 3, by Tezuka, 1993.
(viii) Crayon Shinchan, Vol. 1, by Usui, 1992a.
(ix) Crayon Shinchan, Vol. 3, by Usui, 1992b.
(x) Crayon Shinchan, Vol. 16, by Usui, 1997.

The first three authors (Akahori, Azuma, and Barasui) have been popular
among young people in recent years, and though originally written as comics
for girls, their readership extends beyond gender differences. Most of their
works selected for this study are about young heroines school and family
lives. The last two authors, Tezuka and Usui, and their pieces used in this
study, are well known long-selling comic artists for broader age groups in
Japan. Usuis works, which describe the daily life of a kindergartener and his
parents, are more vulgar and comical than Tezukas serious medical drama.
3. Anger metaphors in Japanese manga
3.1 Category I: Indexical signs
As described in 2.2, Forcevilles (2005) Category I is defined as indexical
signs, which are recognizable as symptoms that we often perceive in daily
life as accompanying the emotion of anger. Bulging eyes, tightly closed
eyes, wide mouth, tightly closed mouth, pink/red face, arm/hand
position, shaking, and ex-mouth/spit are reported to have been ob-
served in La Zizanie (2005: 77). These are metonymic rather than meta-
phorical, since we can perceive them when we see an angry person. For ex-
ample, a red face stands for anger of the person metonymically. So does a
shaking body. We find such indexical signs of anger in the Japanese manga
we have investigated. Some of them are included among the items that For-
ceville finds in La Zizanie, and others are not, but they are all indexical in
the sense that they are recognizable physical symptoms of anger.

Figure 1. Slanted open eyes representing anger (Yotsubato vol. 5: 80, Kiyohiko
Azuma).
A typical pictorial expression of an angry person in Japanese manga is
slanted eyes (figure 1) or slanted eyebrows (figure 2). These eyes can be
either wide open or narrow/closed. They correspond to Forcevilles bulging
eyes and tightly closed eyes respectively, but the angle (slant) seems more
noticeable than size, since half-open slanted eyes can also express anger. The
slanted shapes of eyes and eyebrows are so expressive that, as figure 1
shows, the detailed parts of eyes like the pupils are sometimes omitted but
still the reader can recognize that the person is angry. Slanted eyes and eye-
brows are sometimes accompanied by wrinkles near their inner edge or be-
tween them, which symbolizes the knitting of eyebrows in anger. Figures 1
and 2 also show wide mouth, as found by Forceville.

Figure 2. Slanted eyebrows, tightly closed eyes, and wide mouth representing
anger (Yotsubato vol. 5: 85, Kiyohiko Azuma).
Shrunken or round pupils are also typical pictorial expressions of anger in
Japanese comics. In figure 3, shrunken round pupils are used to express the
persons anger. A tightly closed mouth and shaking body also appear in this
picture, like the ones that Forceville (2005) found in La Zizanie. (The girl in
figure 3 has also Y-shaped signs on her forehead and cheek, which will be
analyzed in 3.2 as pictorial runes.)

Figure 3. Shrunken pupils, slanted eyes, tightly closed mouth, and shaking body
representing anger (Yotsubato vol. 5: 82, Kiyohiko Azuma).
Arm/hand position, as suggested by Forceville, is also expressive of anger.
In figure 4, the girl on the left side raises her arm and clenches her fist. Even
though her face is not drawn, readers can easily understand that she is angry.
This may be because the final stage of the scenario of the folk theory about
anger is that an angry person loses control of herself and vents her anger in
the act of retribution (Lakoff 1987: 397398), and the raised hand and fist
represent this act, showing that the angry girl is about to hit, or at least
threatening to hit, the other girl.

Figure 4. Raised arm/fist representing anger (Azumanga Daiou vol. 1: 112,
Kiyohiko Azuma).
The types of pictorial expressions of anger shown in this section are, as men-
tioned earlier, metonymic in nature. That is, they are perceivable physical
states of a person who is angry. They may be, in that sense, not purely
metaphorical. However, as figures 13 show, the pictorial expressions are in
most cases exaggerated and schematized to some extent. For example, the
actual shapes of eyes of an angry person are not like the ones in the figures,
but the pictures can invoke the typical physical change in the shape of eyes
or eyebrows of an angry person. Thus, what are included in Category I (in-
dexical signs) constitute examples of pictorial metaphor of anger.

3.2 Category II: Pictorial runes
The other category examined by Forceville is called pictorial runes. It is
Kennedys (1982) label but Forceville uses it in a broader sense. According
to Forcevilles definition, pictorial runes are not perceptible in real life, and
their indexicality is therefore less evident than those in the first category
(2005: 77). Examples found in La Zizanie are spirals, ex-mouth,
smoke or fire, bold face, and jagged line. Forceville (2005: 82) ex-
plains them as follows.
The fan-shape of the spirals surrounding the characters head appears to
convey the effect of its risen temperature, its almost bursting with the exer-
tion of either trying to suppress the anger, or with its expression.
If the something coming out of the mouth is understood as signal-
ing loud noise, or indeed as a non-physical phenomenon, this is a pictorial
rune, which is explicable as the release of pent-up pressure built up within
the body-container in the HOT FLUID metaphor.
Smoke (in other comics also often fire) is clearly an effect of the heating
up of the fluid or gas in the body-container.
Bold face and larger fonts [i.e., in the letters in the text balloons] can be
seen as equivalent to saying he spoke very, very loudly. The large
fonts and bold face, then, cue loudness via a more generic metaphor; and
loudness is metonymically associated with (expressed) anger.
The angularity of the jagged line connection [i.e., linking the text bal-
loon to the speaking character] is a less obvious cue for anger, but if we
characterize it as non-smooth as opposed to the rounded and hence
smooth way of connecting balloon to character that is the default, we may
hypothesize that it fits in with a whole category of tense behaviors.
In the Japanese manga we have investigated, spirals were not found. In-
stead, radial straight lines are often drawn as emanating from the front of the
entire angry person, as seen in figure 5.

Figure 5. Radial straight lines representing anger (Azumanga Daiou vol. 1: 30,
Kiyohiko Azuma).
These straight lines may have the same function as Forcevilles ex-mouth,
that is, signaling loud voice or the release of pent-up pressure in the body-
container in the HOT FLUID metaphor (2005: 81). Whether it is temperature
or voice, emission of some kind of energy seems to be expressed by these
radial lines. Thus, it shares motivations with verbal anger metaphors.
Smoke (steam) is also seen in Japanese manga. There is only a little
variation in the shape of smoke (steam), as in figures 6, 7, 8.

Figure 6. Steam representing anger (Crayon Shinchan vol. 3: 16, Yoshito
Usui).

Figure 7. Steam representing anger (Black Jack vol. 3: 98, Osamu Tezuka).

Figure 8. Steam representing anger (Ichigo Mashimaro vol. 4: 136, Barasui).
The small cloud-like shape and the line(s) under it represent that the angry
person is blowing off steam from the head. It seems to be steam rather
than smoke, since the same shape can be used to express steam coming out
of a boiling pan. Indeed, we have a verbal metaphor of anger using the word
for steam (example (2a) in Section 2.1, repeated here).

(6) atama-kara yuge-ga deru
head-from steam-Nom. come-out
Steam comes out of (ones) head.

As often argued in CMT, the use of smoke or steam in this metaphor is
motivated by the folk theory that assumes the rising of temperature in an
angry persons body.
Fire is quite another kind of sign in Japanese manga. It comes out the
whole body, as if the persons body is in flames. An example is shown in
figure 9.

Figure 9. Fire representing anger (Crayon Shinchan vol. 1: 98, Yoshito Usui).
We have verbal metaphorical expressions corresponding to the concept of
fire in Japanese as in (7). Again, the motivation for the use of fire is
provided by the folk theory about anger.

(7) Taro-wa ikari-ni moe-teiru
Taro-Topic anger-Loc. burn-State
Taro is burning with anger.

Figures 2, 5, 8 and 9 include examples of bold face as well. It may repre-
sent a loud voice and is often accompanied by an exclamation mark. As
Forceville argues, there may be a mediating generic metaphor LOUDER IS
BIGGER, and a loud voice may metonymically stand for anger.
As for jagged line, we have a similar kind of signal in Japanese manga,
but not exactly the same as the one Forceville presented as an example. In
the examples taken from La Zizanie, what is jagged is mainly the connecting
line between the balloon and the person, but most cases in Japanese manga,
it is the balloon itself that is jagged.
3
We can see some examples in figures 2,
5, 7, and 8. Though Forceville (2005: 82) suggests that the jagged line is
related to tense behaviors in general, these Japanese cases seem to represent
a loud voice or shouting, as radial straight lines seem to do. Since loud voice
is not physically jagged, this is a kind of metaphor, and loud voice meto-
nymically stands for anger.
We have so far examined the items of pictorial runes suggested by Force-
ville (2005) and demonstrated that the same or similar kinds of runes are
observed in Japanese manga. Though several differences were identified
between Forcevilles findings and ours, those differences do not exceed the
scope of the conceptual metaphor ANGER IS A HOT FLUID IN A CONTAINER
(Kvecses 2000) or ANGER IS THE HEAT OF A FLUID IN A CONTAINER (La-
koff 1987), nor the scenario in the folk theory of anger.
However, we find in addition a very peculiar pictorial rune in Japanese
manga which is not found in Forcevilles study. This rune has a very simple
shape that represents pressurized veins on ones temples. Figure 10 illus-
trates two variants of this rune.

Figure 10. Two variants of pressurized veins.
This sign is very commonly used in Japanese manga to express anger. The
most basic use is to put this sign on a characters temple.
4
Physically, when
a persons blood pressure is very high, the veins near the temples sometimes
stand out. This may be seen with old or middle-aged people, whose facial
skin and hypodermic tissues have decayed so that their veins are no longer
hidden. Indeed, there is a verbal metaphor in Japanese that refers to ones
veins in the brain, as illustrated by example (8). This is an example of AN-
GER IS A HOT FLUID IN A CONTAINER.

(8) kekkan-ga kire-sou
veins-Nom. cut-likely
My veins are just about to snap (Im very angry).
Being based on the physically occurring states, this sign might be classified
as an indexical signal of anger. However, the shape of the sign, especially
the cross-shape, does not coincide with the shape of the veins in the temple.
Moreover, this sign is indiscriminately used even for young girls or children,
whose temple veins are not likely to stand out. Thus, it seems reasonable to
classify this sign as a pictorial rune.
The sign of pressurized veins can be seen in most of the above figures of
manga. However, it may be noticed that the location of the sign varies. This
displacement is a very interesting phenomenon that deserves discussion. We
will examine it more closely in the next section.

3.3 Deviation of pictorial runes
As mentioned in the previous section, one of the most typical and productive
pictorial runes of anger in Japanese manga is a sign of pressurized veins.
This pictorial rune can be detached from its original (physically occurring
and perceivable) place, that is, ones temples, and can be drawn in places
that seem to be physically impossible. For example, in figures 3 and 6, it is
drawn on the forehead and cheek. In figure 11, the signs are on the hands of
the angry mother who is pinching the boys face.
5
In figure 12, it appears on
the neck (or the lower part of the cheek). In figure 13, four of the people
waiting in the queue have this sign on their head but three of them have no
detailed face. So it is not certain where the pressurized vein is, but we can
easily conclude that they are all angry from the narrative and the utterances.

Figure 11. Displaced sign of anger (Crayon Shinchan vol. 3: 16, Yoshito Usui).


In all the above cases, the signs of the pressurized veins appear on a per-
sons face, except in the case of hands. However, this sign can also be wit-
nessed in unexpected places. In figures 4 and 8, it is on the back of the head,
or on the hair. It is definitely impossible to see pressurized veins standing out
on ones hair. At this stage, the sign seems to have been freed from its fea-
ture of appearing on a persons skin. In figure 5, it is in the air, just above
the head. This indicates that the sign can even become detached from the
human body. Finally, in figures 9 and 11, it is in the balloon, beside the
words, as if indicating the intonation or tone of voice with which the words
are uttered.
Thus, even in places other than temples, the sign of pressurized veins can
express anger. Though this sign can be traced back to its original status as a
metonymy, it seems that it has become a sort of independent sign represent-
ing anger and thus can enjoy free displacement and deviation of a kind that is
not seen in verbal metaphors. In verbal metaphors, example (8) implies hem-
orrhage in the brain and nowhere else in the body. To verbally express the
location of the sign such as on hands, on the back of the head, in the air, and
so forth, would make no sense. It is only in the visual mode that this sign can
be displaced. It seems that the free deviation of this pictorial rune is made
possible by the visual properties of the medium in which it is used. This
suggests that pictorial metaphor is not a mere substitution for or equivalent
of verbal metaphor, though these two obviously share experiential motiva-
tions and the same cross-conceptual mappings. As Forceville (2006: 281)
maintains, an exclusive or predominant concentration on verbal manifes-
tations of metaphor runs the risk of blinding researchers to aspects of meta-
phor that may typically occur in multimodal representations only. Dis-
placement of the pictorial rune of pressurized veins may be an instantiation
of the aspects that Forceville argues. It is a visual manifestation of a con-
ceptual metaphor of anger and its meaning cannot be fully covered by the
corresponding verbal metaphor.
So far, we have seen that there are the same or similar kinds of pictorial
runes in Japanese comics as Forceville finds in the French album La Zizanie.
They are all compatible with the verbal expression of the ANGER IS A HOT
FLUID IN A CONTAINER metaphor or its background folk theory and the sce-
nario of anger events. The pictorial runes are not straightforward expres-
sions of physically perceivable states of an angry person, so they are not
straightforward indexical signs in Peircean semiotic theory. However, as
Forceville (2005: 82) argues, they are not symbols in Peircean sense either,
since they are not arbitrary but are motivated. Considering the importance of
the notion of motivation in the study of metaphor, we agree with Forceville
in maintaining that pictorial runes should be classified as indexical signs.
For this discussion, Forceville provides examples from a French comics
album. The present study provides further supports for this claim by giving
evidence from Japanese comics. Motivated pictorial runes are not confined
to French, but are realized in at least two language communities.

3.4 Atmosphere and natural phenomena
As mentioned in 2.1, Japanese emotion metaphors have an elaborated dis-
tinction between inner and outer force, and there is a conceptual mapping we
call EMOTION IS EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON
THAT SURROUNDS THE SELF. In this conceptual mapping, the metaphorical
force of emotion is conceptualized as being outside the self or the body-part
that is regarded as the container of emotion. Not only intense emotions like
anger, but also weak or mild emotion types such as happiness, sadness, dis-
appointment, or anxiety can be conceptualized in terms of natural or mete-
orological phenomena. To our knowledge, English does not have as rich a
repertoire of this kind of highly elaborated mappings between natural phe-
nomena and emotional states of a person as Japanese does.
6
As Matsunaka
and Shinohara (2003) argue, it seems that the rich realization of this emotion
metaphor in Japanese is motivated by the traditional Japanese view that the
heart is a microcosm and the outside natural phenomena reflect inner emo-
tions, or inner emotions are regarded as natural/meteorological phenomena
(see 2.1).
Instantiations of this possibly culture-specific conceptual metaphor are
not restricted to verbal ones; we also find them in pictorial expressions. In
Japanese manga, meteorological phenomena such as thunder, wind, rain,
cloud, or fog, and some life forms in nature like flowers or birds can be used
to express a persons emotional state. They are drawn as the background
scene of a panel, by which readers can recognize how the person in that
panel is feeling. The following figures illustrate some of such uses.

Figure 14. Cloud or fog representing anger (Yotsubato vol. 5:79, Kiyohiko
Azuma).

Figure 15. Thunder representing anger (Yotsubato vol. 5: 84, Kiyohiko
Azuma).

Figure 16. Thunder, wind, and rain representing anger (Crayon Shinchan vol. 1:
100, Yoshito Usui).
In the background scene of figure 14, a cloud or fog-like substance is de-
scribed. Since the setting of this scene is in the living room at sunny lunch-
time, this cloud or fog cannot be a physical occurrence. It expresses the
girls anger, which is getting more and more intense. In figures 15 and 16,
thunder appears, which expresses that the angry person is releasing his/her
anger, and in typical cases, shouting out. Thunder may be associated with
emission of intense energy, perhaps with a loud noise, which may be meta-
phorically mapped onto the loudness of the persons shouting voice.
Figure 16 has wind and rain besides thunder. The slant lines and the
onomatopoeia written on the left side (which reads Gooooo) represent
strong wind. The lines may also represent rain, together with the drops be-
tween them. They all combine to represent a storm. Since there is no ac-
tual storm in this scene, the storm is here a metaphorical expression of the
persons emotion. The image of storm may be associated with intense energy
and destructive power that will affect the person nearby. It may be noticed
that this storm is drawn only around the mother, not around the boy. This
clearly means that the mother is very angry but the boy does not care at all.
The effect of the storm does not reach the boy. In this way, visual represen-
tation of meteorological phenomena can be used very effectively and expres-
sively in manga.
So far, we have observed meteorological or natural phenomena in picto-
rial metaphors that express the emotion of anger. These kinds of pictorial
metaphors are, however, not limited to anger. Other types of emotions like
love, happiness, surprise, disappointment, or anxiety can also be expressed
by meteorological or natural phenomena that are not physically present. The
following are some examples.

Figure 17. Lightning representing surprise (Ichigo Mashimaro vol. 4: 58, Bara-
sui).

Figure 18. Darkness representing disappointment/anxiety (Azumanga Daiou vol.
4: 137, Kiyohiko Azuma).

Figure 19. Birds representing joy/happiness (Azumanga Daiou vol. 4: 158,
Kiyohiko Azuma).

Figure 20. Flowers representing joy/happiness (Ichigo Mashimaro vol. 1: 51,
Barasui).

Figure 21. Flowers representing love (Kashimashi vol.1: 152, Yuukimaru Ka-
tsura).

Figure 22. Flowers representing love (Crayon Shinchan vol. 16: 12, Yoshito
Usui).

Figure 23. Dropping petal of a flower representing disappointment (Ichigo Mashi-
maro vol. 1: 42, Barasui).
In figure 17, two girls are holding their heads in surprise or shock. This emo-
tional state is emphasized by the background lightning. The motivation be-
hind this pictorial metaphor may be that the impact of surprise or shock on
ones heart is similar to that of lightning on ones eyes, in temporal aspect
(abrupt, unexpected occurrence), and its intensity.
As a more lasting and static environment, darkness is expressed by nar-
row vertical lines in figure 18. This is expressive of the persons emotional
state of disappointment or anxiety. In this scene, the girls are talking about
their exams, in which they have no confidence. The relation between disap-
pointment/anxiety and darkness is observed also in verbal metaphors. That
these negative emotions are experienced as dark may be experientially
motivated by a biometeorological tendency, as mentioned in 2.1, or just by
what we tend to feel in actual darkness.
Figures 1923 differ from figures 17 and 18 in that they use natural life
forms instead of meteorological phenomena to express emotional states. In
figure 19, the girls are happy and full of joy, and this is symbolized by the
birds surrounding them. Since this is an outdoor scene, the presence of birds
is not totally implausible. This is an example of what Forceville (2006) calls
a realistically motivated metaphor. Consistent with what he argues, the
metaphor is subtler in this case than the case of thunder, whose presence is
very unlikely. Since birds can fly in the air, the conceptual mapping HAPPY
IS UP may motivate this use of birds.
Figure 20 represents a girl drinking beer after taking a bath. She is very
relaxed and happy. This state of emotion is expressed by surrounding flow-
ers. The relationship between happiness and flowers is obvious, since it is a
typical reaction for us to feel relaxed and happy when we see flowers. Thus,
this use of flowers is experientially motivated.
In figures 21 and 22, flowers represent love. This is also observed in ver-
bal metaphor in Japanese, in which a rich vocabulary about flowers is asso-
ciated with love. This may hold in other languages as well.
Figure 23 has a more complex structure. This pictorial metaphor consists
of two successive panels. In the first panel, the flower behind the girl has a
complete shape. However, after she hears what other persons say, which is
shown in the two balloons, one petal of the flower drops, as in the second
panel. This change of state of the flower represents the girls emotional state,
that is, how she became disappointed when she heard those utterances. The
petal dropping indicates incompleteness of the flower, deficiency, or lack of
something. Since flower represents a happy state, this deficiency means de-
stroyed happiness. It is, however, not a severely destructive experience
because the flower is still a flower even when it has lost one of the petals.
Thus, this pictorial metaphor is highly expressive of such dynamic state of
ones emotion, and it illustrates that dynamic or complex use of visual signs
is possible in this kind of media.
To sum up, meteorological or natural phenomena used in pictorial emo-
tion metaphors we have examined are experientially motivated. Flowers can
represent happiness or love, but they do not represent anxiety or shock.
What makes pictorial metaphors of this kind comprehensible and interpret-
able resides in the motivation or grounding that supports their mappings.
Considering these properties, pictorial metaphors in Japanese manga should
be classified as Peircean indices rather than Peircean symbols.
However, it should be noted that the use of meteorological or natural
phenomena to express emotional states may be, at least to some extent, cul-
ture-specific. As already mentioned, to regard natural scenes as reflections
of ones internal or emotional states may be an aspect of Japanese socio-
cultural tradition which may not be shared by other cultural groups. We
have argued elsewhere that experiential motivation does not imply universal-
ity. Even when a conceptual mapping has physical motivation, it could still
be culture-specific. Pictorial runes of meteorological or natural phenomena
in multimodal emotion metaphors may constitute an instantiation of this
argument.
4. Conclusion
In this chapter, we have examined some works of Japanese manga to demon-
strate that there are pictorial metaphors that are motivated by the same con-
ceptual mappings as verbal metaphors. We have shown that the conceptual
metaphor ANGER IS A HOT FLUID IN A CONTAINER is shared by verbal and
visual modalities. Readers/viewers of the pictorial expressions of anger such
as the ones discussed in this chapter require no verbal information, either in
the form of talk by a character in the panel, or in the surrounding narrative
context, to assess the emotion depicted. That is, readers know what kind of
emotion is expressed from only pictorial signs of the types we have pre-
sented. (Emotions expressed by meteorological/natural phenomena are some-
times ambiguous and thus need to be aided by facial expressions of a charac-
ter, though these do not require verbal information either.) The parallel
structures between verbal and visual metaphors indicate that metaphors are
not merely a surface linguistic phenomenon but rooted more deeply, structur-
ing our cognition and concepts.
However, we have also observed that verbal and visual metaphors are not
completely equivalent. Some examples of the pictorial metaphors are spe-
cific to the visual mode. Even though basic conceptual mappings are shared
by both modalities, it seems that differences can emerge due to the different
properties of each modality.
Moreover, culture-specificity may be observed in both verbal and visual
metaphors. By examining EMOTION IS EXTERNAL METEOROLOGICAL/
NATURAL PHENOMENON THAT SURROUNDS THE SELF, we have argued that
culture-specific aspects found in verbal emotion metaphors may also be
found in pictorial emotion metaphors. This point, however, needs more in-
vestigation. Researchers of multimodal metaphors, especially of other lan-
guages and cultures, are invited to this promising field.
Notes
* We would like to acknowledge Charles Forceville and Eduardo Urios-Aparisi
for their constructive comments. We are also indebted to Kotaro Yamamoto
for his role in collecting many of the panels we used as the data.
1. Discussions on anger metaphor within the field of cognitive linguistics can
be found e.g., in Geeraerts and Grondelaers (1995), Gevaert (2002), Kotze
(2005).
2. In this metaphor, fluid is mainly assumed to be blood, as Kvecses (2000:
159) says [I]t is reasonable to assume that it is mainly blood (but perhaps
some other body fluids as well) that accounts for the fluid component in the
container metaphors. Other than blood, qi (gas) in Chinese (Kvecses
2000, Yu 1998) and gastric contents in Japanese (Matsunaka and Shinohara
2001a) have been suggested as instantiations of the fluid.
3. It is suggested that jagged balloons sometimes appear in also European com-
ics as well (Forceville, personal communication).
4. As McCloud (2006: 97) points out, this sign is one of the symbolic expres-
sions which vary from culture to culture, though it has found its way to Eng-
lish language comics.
5. The veins on ones hands may stand out when he/she strains the muscles of
the hands. In that case, this example may be an indexical sign of physically
occurring states.
6. There are a few similar cases of this kind in English (Kvecses 2000: 71),
but they are confined to strong emotions.
References
Akahori, Satoru, and Yuukimaru Katsura
2005 Kashimashi [clamorous]: Girl Meets Girl. Vol. 1. Tokyo: Media
Works.
Azuma, Kiyohiko
2000 Azumanga Daiou [Azumanga the Great]. Vol. 1. Tokyo: Media
Works.
2002 Azumanga Daiou [Azumanga the Great]. Vol. 4. Tokyo: Media
Works.
2006 Yotsubato [With Yotsuba]. Vol. 5. Tokyo: Media Works.
Barasui
2003 Ichigo Mashimaro [Strawberry Marshmallow]. Vol. 1. Tokyo: Me-
dia Works.
2005 Ichigo Mashimaro [Strawberry Marshmallow]. Vol.4. Tokyo: Media
Works.
Carroll, Noel
Forceville, Charles
tivity 9: 129.
bol 14: 179198.
2006/this vol. Non-verbal and multimodal metaphors in a cognitivist frame-
Geeraerts, Dirk, and Stefan Grondelaers
1995 Looking back at anger: Cultural traditions and metaphorical pat-
terns. In Language and the Cognitive Construal of the World, John
Taylor and Robert MacLaury (eds.), 153179. Berlin/New York:
Mouton de Gruyter.
Gevaert, Caroline
2002 The evolution of the lexical and conceptual field of ANGER in Old
and Middle English. In A Changing World of Words: Diachronic
Approaches to English Lexicology and Semantics, Javier E. Daz
Vera (ed.), 275299. Amsterdam: Rodopi.
Grady, Joseph
1997 Foundation of Meaning: Primary Metaphors and Primary Scenes.
Ph.D. diss., University of California at Berkeley.
Kotze, Zacharias
2005 Humoral theory as motivation for anger metaphors in the Hebrew
Bible. Southern African Linguistics and Applied Language Studies
23 (2): 205209.
Kvecses, Zoltn
2000 Metaphor and Emotion. Cambridge: Cambridge University Press.
Press.
2005 Metaphor in Culture. Cambridge: Cambridge University Press.
Lakoff, George
1990 The Invariance Hypothesis: Is abstract reason based on image-
1993 The contemporary theory of metaphor. In Metaphor and Thought.
2nd ed. Andrew Ortony (ed.), 202251. Cambridge: Cambridge
University Press.
Matsuki, Keiko
1995 Metaphors of anger in Japanese. In Language and the Cognitive
Construal of the World, John Taylor and Robert MacLaury (eds.),
137151. London: Methuen.
Matsunaka, Yoshihiro, and Kazuko Shinohara
2001a ANGER IS GASTRIC CONTENTS: Japanese anger metaphor revisited.
Paper presented at 4th International Conference on Researching and
Applying Metaphor, Tunis, Tunisia.
2001b Emotion and outer force schema: A perspective from Japanese.
Paper presented at The First Seoul International Conference on Dis-
course and Cognitive Linguistics, Seoul, Korea.
2003 Clouds and sunshine in mind: Meteorology-based Japanese emotion
metaphors. Paper presented at 5th International Conference on Re-
searching and Applying Metaphor, Paris, France.
McCloud, Scott
2006 Making Comics: Storytelling Secrets of Comics, Manga and
Graphic Novels. New York: HarperCollins Publishers.
Nihon-Seikishoogakkai [Japanese Society of Biometeorology]
1992 Seikishougaku no Jiten [Encyclopedia of Biometeorology]. Tokyo:
Asakura Shoten.
2001 Is emotion really force? Japanese metaphor of sorrow. Paper pre-
sented at The Seventh International Cognitive Linguistics Confer-
ence, Santa Barbara CA, USA.
2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen
[language and human being]: Journal of Yokohama Linguistic Cir-
cle 4: 118.
Tezuka, Osamu
1993 Black Jack. Vol. 3. Tokyo: Akita Shoten.
Usui, Yoshito
1992a Crayon Shinchan. Vol. 1. Tokyo: Futabasha.
1992b Crayon Shinchan. Vol. 3. Tokyo: Futabasha.
1997 Crayon Shinchan. Vol. 16. Tokyo: Futabasha.
Yamanaka, Keiichi
2003 Waka no Shigaku [The Poetics of Classical Japanese Poetry]. Tokyo:
Taishuukan Shoten.
Yu, Ning

V

Metaphor in Spoken Language
and Co-Speech Gesture
Chapter 13
Words, gestures, and beyond: Forms of multimodal
metaphor in the use of spoken language
Cornelia Mller and Alan Cienki
Abstract
This chapter offers a systematic account of the forms that mono- and multimodal
metaphors may take in face-to-face communication. The account is based on the
relation of source and target domains expressed either in one modality only (thus
forming a monomodal metaphor) or in two modalities (forming a multimodal
metaphor). We will then illustrate the inherent dynamic nature of metaphors when
used in spoken interaction, pointing out more specifically how metaphors are
being elaborated within and across modalities. We will focus particularly on
metaphors that are realized in speech and/or gesture, but point out the relevance
of studying metaphors in other articulatory forms such as stress and intonation.
The different forms of multimodal metaphors are systematically based on different
relations between metaphoric and gestural expressions. Finally, implications for
metaphor theory and for the dynamic aspects of thinking for speaking are dis-
cussed, suggesting that multimodal metaphors in spoken language are products of
the process of creating metaphoricity (by a speaker/gesturer and ideally also by a
listener/perceiver), which is essentially independent of modality and expressive
form.

Keywords: Activation of metaphoricity, gestures, gestural metaphors, thinking for
speaking and gesturing, multimodal metaphor, monomodal metaphors, verbal
metaphors, verbo-gestural metaphors
1. Introduction
The situation which has been most influential for the form that spoken lan-
guages have is arguably the face-to-face encounter. We take it as a scenario
that has been described by sociologists, social psychologists, anthropologists
and linguists, and which rather unanimously has been characterized as a
communicative situation that is inherently multimodal.
298 Cornelia Mller and Alan Cienki
Adam Kendon, inspired by Goffman (1967), was interested in the interac-
tional aspects of this situation and he has devoted many of his early studies
to finding responses to Erving Goffmans (1967) call for a study of the
ultimate behavioral materials of interaction. [] That is, [] the glances,
gestures, positions and verbal statements that constitute the stuff of face-to-
face encounters (Kendon 1990: ix). David Efron, one of the pioneers of
gesture studies, directed attention to the fact that the hand gestures people
seem to use unwittingly and very regularly when they converse with each
other are so deeply intertwined with spoken language that in his empirical
investigation of cross-cultural differences between Jewish and Italian immi-
grants to New York City he distinguishes spatio-temporal aspects, i.e.,
gesture simply considered as movement, from their referential aspects,
i.e., gesture envisaged as language (Efron 1972: 67, emphasis in original).
Herb Clark, in turn, has described the face-to-face situation of communica-
tion as the canonical encounter of human beings:
From the social psychologists viewpoint, man is a social animal, who enjoys,
perhaps even needs, to interact socially with other people. What are the char-
acteristics of the most usual interaction between two people, John and Mary?
For our purposes, the most important property is that they will be facing each
other a short distance apart. It is in this position that John and Mary are situ-
ated for the optimal perception of messages both verbal and nonverbal
from the other person. John is in Marys positive perceptual field, and Mary is
in Johns. If John and Mary were side by side, or back to back, or back to
front, or in any other position, these conditions would no longer be optimal. It
is no accident that normal conversations are carried out face-to-face. This
face-to-face situation is what I would like to refer to, for convenience, as the
canonical encounter (Clark 1973: 3435).
Granting that the canonical encounter as described above does not imply that
co-participants literally face each other,
1
it is a form of interaction which is
extremely common in the cultures of many areas of the world and which
does imply reciprocal audibility and visibility (although to varying degrees).
This face-to-face encounter between two people will be the context in which
we will treat the use of spoken language in this chapter. As Clark indicates,
the use of spoken language in this context is inherently a process of multi-
modal communication, involving not only the oral production of sound and
its aural reception, but also the production of various kinds of bodily motion
in space, which the addressee can perceive visually. The multimodal nature
of spoken communication has been especially emphasized in recent years by
those researching spontaneous gesture with speech, suggesting that gesture
Words, gestures, and beyond 299
and speech are visible and audible actions that form one single utterance
(e.g., Kendon 2004) or proposing that gesture and speech are dynamically
based in different forms of thought but constitute one integrated system (e.g.,
McNeill 1992, 2005). Given that spoken language involves multiple modali-
ties, it makes sense that metaphor should have the potential for multimodal-
ity when used in this form of communication; and indeed over the past years
quite a substantial body of research on metaphor in gesture, speech, and sign
language has been carried out (cf. Bouvet 1997, 2001; Calbris 1998, 2000,
2003; Cienki 1998, 2005b; Cienki and Mller 2008a, 2008b; McNeill 1992;
Mittelberg 2006; Mittelberg and Waugh, this volume; Mller 1998, 2008;
Nez 2004; Nez and Sweetser 2006; Webb 1996; Wilcox 2000, 2004).
The topic we want to explore here in particular is the different forms that
multimodal metaphors may take in face-to-face communication. We will
specifically concentrate on the kinds of relations between metaphors that are
realized in speech and/or gesture. It is not by accident that the study of
metaphor is increasingly taking data from gesture studies into account (e.g.,
Cienki and Mller 2008a, 2008b; Mller 2008) and this chapter offers a
systematic account of the forms of metaphors that occur either in speech or
in gesture or in both modalities at the same time. However, we would also
like to point out that beyond gesture, there are additional properties of spo-
ken communication which have received much less or no attention in terms
of their implications for the expression of metaphor, including prosodic fea-
tures, such as stress and intonation, and the time course in which all of these
expressive forms are used during acts of speaking (for the latter point see
Mller 2007, 2008).
In order to clarify what we are discussing, we will restrict the term mo-
dality to two dimensions of face-to-face communication: one will refer to
what is expressed orally and perceived primarily aurally as sound (the
oral/aural modality), and the other will refer to bodily forms and movements
in space which are primarily perceived visually (the spatial/visual modality).
In this sense, we will see that gesture/word combinations can constitute mul-
timodal metaphors. Within each modality, there are various forms which can
be used for expressive purposes. In the oral/aural modality, intonation and
stress can be discussed separately from each other and separately from the
words being articulated. We will refer to these as different articulatory forms
within this modality. Similarly within the spatial/visual modality, eye gaze,
body shifts, manual gestures, etc., can all be considered different expressive
forms. Our understanding of articulatory form partially overlaps with Force-
villes use of the term mode. In his critical stance towards giving a satisfac-
tory definition of mode or of compiling an exhaustive list of modes,
Forceville argues that this fundamental difficulty is no obstacle for postu-
lating that there are different modes and that these include, at least, the fol-
lowing: (1) pictorial signs; (2) written signs; (3) spoken signs; (4) gestures;
(5) sounds; (6) music; (7) smells; (8) tastes; (9) touch (Forceville 2006:
3823/this volume). In short, spoken words and gestures are articulatory
forms or modes, which are realized in an aural/oral or spatial/visual modal-
ity.
After presenting an overview of what appear to be the most common
ways in which the use of metaphor can play out in the oral/aural and spa-
tial/visual modalities and articulatory forms, we will point out the inherent
dynamic nature of metaphors when used in spoken interaction. Eventually
we will suggest that these observations indicate that multimodal metaphors
are products of the process of creating metaphoricity (by a speaker/gesturer
and ideally also by a listener/perceiver), which is essentially independent of
modality and articulatory form, if metaphoricity is a matter of understand-
ing one idea (or domain) in terms of another. However, we will also argue
that the different modalities and forms that are involved in spoken interaction
afford the use of different expressions for metaphors. What one can express
via a given modality and expressive form will have an effect on what one
will express using that modality. We will conclude by considering the impli-
cations this has for how we can think with metaphors while we are speaking,
or attending to someone who is speaking.
2. Monomodal and multimodal metaphors in words and gestures
To begin with we need to clarify our understanding of mono- and multimodal
metaphors. Following Forceville (2006: 383), we will consider as monomo-
dal those metaphors whose target and source are exclusively or predomi-
nantly rendered in one mode. This means we will distinguish monomodal
verbal metaphors from monomodal gestural metaphors. We will consider as
multimodal those metaphors whose target and source are each represented
exclusively or predominantly in different modes (Forceville 2006: 384), and
for the present chapter this means that we will document and discuss verbo-
gestural metaphors. In fact, and even more precisely, this means that we are
actually talking about verbal and gestural, or verbo-gestural metaphoric
expressions, since the phenomenological level we are concerned with in our
analysis is that of verbal, gestural, or verbo-gestural utterances. If, and if so
how, these expressions relate to a general level of conceptual metaphors
(such as LOVE IS A JOURNEY) remains unaddressed. We do, however, con-
sider the meaning of those metaphoric expressions be they verbal, gestural,
or a combination of both to be conceptual (adhering to the cognitive lin-
guistic assumption that meaning in general is conceptual). Thus for the
sake of brevity only we will henceforth use the terms metaphor and meta-
phoric expressions as synonyms in this chapter, unless otherwise indicated.
Second it is central to illustrate what we mean by gesture. Here we will
be focusing on visible, effortful movements of parts of the body whose pri-
mary purpose is apparently not that of self-adjustment (for example, as with
grooming behavior) or object manipulation (such as lifting a cup to take a
drink). One could focus on many different parts of the body, such as head
gestures, eye movements, foot gestures, body shifts, shoulder shrugs, and so
on. Here we will focus on positions, orientations, and movements of the
hands and forearms; these are what we will mean henceforth with the term
gesture, unless specified otherwise.
Which gestures will be considered metaphoric? Here we will restrict the
discussion to gestures whose primary function can be identified as abstract
reference. Mller (1998: 1101) notes that referential gestures can refer
either to physical objects, properties, actions, or relations, or to abstract
notions in terms of such physical means. Thus the same two-handed gesture
with thumb and index finger on each hand forming a 90 degree angle could
be used when talking about a picture frame or when describing the frame-
work of a theory. Abstract referential gestures are inherently metaphoric by
virtue of rendering a non-physical idea in terms of a physical, spatio-
temporal representation. We can note that a concrete referential gesture can
also be metaphoric in certain contexts (e.g., when someone imitates an ani-
mal referring to a person in a derogatory way), but we will not focus on such
usage here, as we have found it extremely rare in our research to date.
We should also mention our criteria for identifying verbal expressions as
metaphoric. For this we rely on the procedure developed by the Pragglejaz
Group (2007), with language-specific adaptations, as appropriate. To put it
briefly, the procedure has been designed for the identification of (a) when a
word is being used in a given context with a meaning which is different from
another physically more basic meaning that it may have, and (b) when the
contextual meaning is interpreted via comparison with the more basic mean-
ing. It is a maximally inclusive procedure, intended to identify words which
may even potentially be understood metaphorically in the given context of
use. For a detailed account of the procedure see Pragglejaz Group (2007). In
the examples that follow, we will indicate words so identified via this proce-
dure with underlining. The examples below are from our qualitative analyses
of videorecorded conversations from four different languages: American
English, German, Cuban Spanish, and Russian. The conversations in each
language were elicited from pairs of native speakers, and were on abstract
topics so as to increase the likelihood of use of metaphoric expressions. The
English and Russian conversations were between pairs of university students
who were talking about how they take exams at their universities (in the
United States and Russia, respectively). The German data come from mid-
dle-aged German women who were talking with the researcher (the first au-
thor) about their first love relationship. The Cuban data were all recorded in
Havana. Participants were asked to give accounts of important events in
their lives (a wedding) or to talk about life in Cuba in general. They were
roughly of the same age (in their thirties, with one exception of a speaker
who was 54 years old), and mixed with regard to gender.
Each example cited here comes from a different conversation, unless
noted otherwise. Each line in the transcript indicates a new intonation unit
(see Chafe 1994 on intonation units as units of analysis for spoken dis-
course). A comma (,) indicates an intonation unit with a terminal pitch that
signals continuation, a period/full stop (.) marks a final intonation unit fal-
ling to a low pitch at the end, and a double dash (--) shows that the intona-
tion unit was truncated. A longer pause by the given speaker is indicated
with three dots (), and a shorter one with two (..). In the transcription of
the examples, each separate gesture is indicated by a number (G1, G2, etc.);
hands involved in gesturing are noted as rh, lh, bh, indicating right, left or
both hands; the onset of gesture notation is synchronized in the transcript
with the beginning of the gestural movement and bold face indicates the en-
tire duration of gestural movement (preparation, stroke, retraction, cf. Ken-
don 2004; McNeill 1992).
Building on the possible relations that Cienki (1998) has found between
metaphors expressed in words and in gestures and on Mllers (2008) dis-
cussion of different realms of metaphor, we will now give an overview of
what is known about the manners in which metaphor may be expressed in
speech in real time, that is: in one modality (spoken words or gestures) or in
a combination of the two. Put differently, we will offer a sketch of what
appear to be common forms of mono- and multimodal metaphors in words
and/or gestures.
3. Monomodal metaphors: source and target within one modality
First, we can confirm that we often find the use of metaphoric verbal expres-
sions without co-occurring metaphoric gestures. For example, one American
student talks about how people may verbalize certain beliefs about honesty,
but sometimes might not behave in accordance with them. At this point she
says (Example 1):

Example 1:
Just because of the pressure,
the peer pressure,

The word pressure was coded as metaphoric in this context because peer
pressure normally involves behaviors other than physical pressing the
more basic physical meaning of the word. Even if peer pressure involved
physical contact, the word still can be understood with the abstract sense of
coercive behavior (it has potential metaphoricity). Although the word was
coded as metaphorically used, the speaker made no gestures while saying it
either time, keeping her hands resting on her leg as she was sitting. We might
refer to such a use of metaphor purely on the verbal level as monomodal
metaphor or as verbal metaphoric expression.
We also find the converse monomodal pattern of metaphor use: meta-
phors expressed in gestures without metaphors in the co-occurring speech,
that is, gestural metaphoric expressions that are used concurrently with
speech. Example 2 comes from one of the Russian students talking about
how they take exams at their university. An English translation is provided
below the transliterated transcript of the Russian. The student is trying to
characterize the Russian concept of chestnost, which may be translated
as honesty.

Example 2 (from Russian):
Dlia menia chestnost eto nekaia absoliutnaia kategoriia.
For me chestnost is a kind of absolute category.
G1 preparation
bh raised in front of torso, flat in vertical
plane, fingers pointing out
Kogda vot iest situatsiia,
When theres this situation,
G1 stroke
bh move straight
downward slightly
seichas postupit chestno tak.
then [you need] to act honestly like this.

The student begins seated, hands at rest in his lap, and starting when he says
situatsiia [= situation], he lifts his two hands in front of his torso, the right
hand somewhat higher than the left, palms and fingers flat in the vertical
plane, fingers pointing forward. He holds his hands there until he gets to the
word tak [= like this], at which point he moves them both slightly
downward in unison, maintaining their position relative to each other and flat
in the vertical plane. Here the speaker explicitly uses the gesture to make
reference if you dont see the gesture, you dont know what he thinks it
means to act honestly (chestno). The speaker uses a verbal deictic particle
to direct the attention of his co-participant to the gesture: like this points to
the gesture, and his hands move in temporal coordination with the verbal
deictic. Thus verbally he makes clear that the gesture contains relevant in-
formation, but there is no verbal mention of a metaphoric source. In this
instance, manner of behavior (honest) is expressed gesturally as a physical
form (flat/straight) with a certain motion (brief and straight). (See Cienki
1999 for further discussion of this and related examples.)
Note that this is a particularly interesting case, since not only are the ges-
tures source and target independent from any verbal metaphoric expression
(in fact there is none in the concurrent speech), but the gestural metaphoric
expression is used in place of words. Thus we might speculate whether this
is an instance of a multimodal utterance consisting of a monomodal gestural
metaphoric expression which is being inserted into a verbal utterance.
Another type of gestural metaphoric expression that is very common
among different cultures involves gestures which perform a speech-act or
more generally a communicative activity. These are gestures that recur in
form and function over a large amount of contexts and therefore we term
them recurrent gestures (cf. Bressem and Ladewig in prep.; Ladewig in
prep.; Teendorf in prep. a, b). Examples are the palm-up-open-hand gesture
(cf. Kendon 2004; Mller 2004; Streeck 1994), the ring gesture (Fatfouta in
prep.; Kendon 2004; Morris 1977; Neumann 2004), or the brushing aside
gesture (Mller and Speckmann 2002; Speckmann 1999; Teendorf in prep.
a, b). These gestures all share a common origin, in that they are all meto-
nymic derivations of everyday actions (cf. Mittelberg 2006; Mittelberg and
Waugh, this volume; Mittelberg and Mller in prep.; Mller 1998, in prep.
a, b; Mller and Haferland 1997; Streeck 1994): presenting, offering or
receiving something (the palm-up-open-hand gesture); picking up small ob-
jects with the index finger and thumb (the ring gesture); or brushing aside
small objects. What we observe in these gestures is a two-step semiotic
process as identified and described by Mittelberg and Waugh (this volume),
in which the metonymic target of the sign-formation process turns into the
source of the metaphoric gesture (see also Mittelberg 2007). The targets of
the metonymic process in our cases are the modulated actions: i.e., part of
the action stands for the action as a whole, thus consituting a classical
instance of a synecdochic relation (Mller in prep. a, b). This modulated
action is used now as a metaphoric source for symbolizing abstract issues
such as presenting a discourse object on the palm-up-open-hand, indicating
the preciseness of arguments, or brushing aside unpleasant topics. For in-
stance the brushing aside gesture is widely used to express negative assess-
ments, and this is what we will see in the next example. Example 3 comes
from free conversations recorded in Cuba (Mller and Speckmann 2002;
Speckmann 1999). (For a detailed analysis of the brushing aside gesture
used by speakers of the Iberian Peninsula, see Teendorf in prep. a, b).
In Example 3 the speaker thinks out loud about the possible conse-
quences of what it would be like to have four instead of two TV-channels in
Cuba. He is convinced that this would disturb family life by causing endless
discussions about which program to watch. He describes the big arguments
this would raise in a very lively way, and in doing this becomes himself part
of such an imaginary situation: he imagines himself standing in the living
room, and he indicates three different places, each of them relating to a dif-
ferent person voting emphatically for another program: Yo quiero vel
aquello yo quiero ver lo otro yo quiero ver esto (I want to see this one, I
want to see that one, I want to see the other one). The more programs to
choose from, the more arguments you have in your family this is the moral
of the speakers imagined scenario. It is clearly not desirable to have four
channels on Cuban TV and correspondingly he concludes his discussion with
a negative assessment performed gesturally as a brushing aside gesture.

Example 3 (from Cuban Spanish):
G1
rh point to the right
yo quiero vel aquello,
I want to see that one

G2
lh points straight
yo quiero ver lo otro,
I want to see the other

G3 G4
rh points straight rh brushes aside
yo quiero ver esto (.)
I want to see this one (.)

The first three gestures the speaker uses relate to the propositional content of
the utterance; G1, G2, and G3 point to three different places in the imagined
apartment, localizing three different persons with three different wishes. The
brushing aside gesture (G4) is located in a micro-pause at the end of this
utterance and clearly assesses this imagined situation as an undesirable
one. There is no verbalization of a negative assessment; the gesture takes
over the entire communicative burden. It is noteworthy that the gesture is
placed at the end of the phrasal unit, precisely where a verbal evaluative
particle could have been placed. Instead the speaker pauses and produces a
gesture with similar content. It seems as if the brushing aside gesture would
do the same job as a verbal particle would (which is why Mller and
Speckmann, suggested the term gestural particle). It gives a negative
evaluation of a situation being described, and the gestural meaning is derived
from the negative connotation of the practical action. (For a detailed cogni-
tive semiotic analysis of this process, see Teendorf in prep. a, b.)
What the brushing aside gesture shares with the other recurrent gestures
mentioned before is that it has a performative or (more general pragmatic)
function rather than a referential one, and it is obvious that metaphor plays a
different role here than in example 2. In the second example the communica-
tive function is metaphorical reference, whereas in example 3 the gestures
function is the performance of a communicative action. Hence the first use
of the metaphoric gestures belongs to the realm of semantics while the other
one belongs to pragmatics. The difference is not a simple matter, but for the
sake of brevity it might be characterized as a difference between gestures
contributing information to the propositional content of the utterance and
gestures contributing meta-communicative information. While in the second
example the metaphoric gestures expressed aspects of the propositional con-
tent (honesty as a physical form and movement) in the third example the
metaphoric gestures are used for meta-communicative purposes (they qualify
the propositional content), telling us how the propositional content (the
choice of various programs) is being assessed by the speaker. Thus while
metaphor as well as metonymy are clearly involved in these gestures, they
come in at the level of the semiotic process of sign formation rather than on
the level of communicative function.
We may conclude that monomodal metaphors are frequent in words but
they also can be found in gestures. As for gestural metaphoric expressions
we have found two different kinds: on the one hand there seem to be gestures
that are more likely to be created on the spot (such as example 2, honesty
with a straight gesture), and others that appear to recur with a relatively
stable form and function (the brushing aside case, example 3). These two
kinds of gestural metaphors furthermore seem to fall into two different func-
tional groups: one of them expressing parts of the propositional content, the
other one performing meta-communicative acts in the widest sense. It seems
that the latter ones show a tendency for conventionalization, which is why
we are able to put together repertoires of those forms but not of the sponta-
neous referential gestures, created ad hoc.
4. Multimodal metaphors: source and target in two modalities
Turning to multimodal metaphor, it is not only interesting to note that once
again we encounter quite some variation with regard to which articulatory
mode expresses which aspects of the metaphor. The kind of variation and
distribution of duties over the two modalities involved tell us something
about the nature of the collaboration of words and gestures in spoken lan-
guage, and it offers insights into the cognitive activation of metaphoricity
during speaking. Of the three theoretically possible variations, two are com-
mon, whereas one is extremely rare. We find that the same source and target
in two modalities, as well as different source and same target in two modali-
ties, are very common forms of multimodal metaphors in words and ges-
tures, whereas same source and different target appears to be rare.

4.1 Same source and same target in two modalities
A very common form of gesture word collaboration in expressing metaphor
is, as one might expect, when the source domain of a conceptual metaphor
appears simultaneously in both verbal and gestural form. Consider example
4, in which a young woman is describing how her teenage love became more
and more clingy (klebrig) and intense (heftig):

Example 4:
G1
open palms touching each other repeatedly
also da hab ich schon gemerkt naja
So there I had already realized, well,

G1 continued
des is ganz schn klebrig.
this is pretty clingy,
G1 continued
(..) oder heftig.
(..) or intense.

Here clingyness is being expressed verbally and gesturally. The speakers
flat hands repeatedly touch each other, moving apart and then back to stick-
ing together. It is as if the palms were sticky and it was hard to separate
them. The gesture enacts the source domain of the verbal metaphoric expres-
sion, indicating that metaphoricity of this expression was activated or in the
foreground of the speakers attention. However, the semantic co-
expressiveness that we observe in this example does not imply that the ges-
tural and the verbal parts of such a multimodal metaphor must also be ex-
pressed simultaneously in real time. As is widely known from gesture stud-
ies, gestures often precede words they are co-expressive with; sometimes
they are held up and wait for speech, and sometimes they continue past the
expression. This is precisely what happens here. The gestural metaphor en-
acting clingyness begins with the first line in the transcript. Well I already
realized and it is held through the pause in the third line and recycled while
she offers another metaphoric qualification of the relation (strong, in-
tense). Put differently, even before the speaker actually verbalizes the
metaphoric expression clingy she begins to enact clingyness gesturally.
This is a case of a verbo-gestural metaphor in which the gesture enacts the
source domain of the verbal metaphoric expression, but it does so signifi-
cantly before the verbal part of the metaphor is uttered. In fact the temporal
overlap of the verbo-gestural metaphor is surrounded by an ongoing gestural
enactment of the metaphor. Put differently, gestures may dynamically fore-
shadow and maintain verbal metaphors over longer stretches of discourse.
Moreover, they indicate activation of metaphoricity of conventional and
transparent metaphors (Mller 2003, 2007, 2008). Gestures expressing the
source domain of a verbal metaphoric expression therefore indicate that at
this very moment in the production of a verbo-gestural metaphor, the
speaker had activated metaphoricity, and we may therefore characterize this
metaphor as waking for that very speaker at this moment in time. These
cases appear to be widespread and to occur frequently, and we will consider
another, similar example below.
2

In Example 5 a young man challenges his co-participants opinion on the
future implications of ones first job after graduating from university. He
thinks that the first job one takes on determines the path of ones future ca-
reer, and in order to make his point he uses a German idiomatic expression
die Weichen stellen, literally setting the tracks. Note that when he is
using this idiom for the first time, he does not gesture. Gesturing begins with
his elaboration and illustration of this metaphoric argument.

Example 5:
nein es is nich so,
no this is not the case,

aber es stellt natrlich Weichen.
but it obviously sets tracks.

das is das Problem.
this is the problem.

G1, 2 joined flat hand point towards left
es is schon ne Weiche-
it does set tracks

The speaker begins to develop his alternative viewpoint with a very common
rhetorical pattern in German conversations, the nein aber (no but) pat-
tern, in which a preceding suggestion is first confirmed and then challenged.
The confirmation in our example is verbalized in the first line: no, this is
not the case, hereby confirming his interlocutor's point of view, which is
then followed by the counterargument in line two, beginning with but: but
it obviously sets tracks. He verbally formulates his alternative viewpoint,
and he does this metaphorically: it obviously sets the tracks. No gesture is
produced along with this first formulation of his counterargument; he only
begins to gesture with his first reformulation of the verbal metaphoric ex-
pression. Having had no ratifying reaction from his co-participant he begins
to elaborate his argument. And with this elaboration he performs a pointing
gesture towards his left. Note that the pointing gesture is one in which the
extended palms, held vertically, are used to indicate a certain direction. Note
that there is a systematic variation of form and function in pointing gestures.
Kendon and Versante (2003) show that in Neapolitan conversations speakers
use the index finger to point out objects, whereas the flat hand is used to
indicate directions. In our case the vertical open palms of the two hands are
joined to indicate one direction of a future career. In short, we see here an-
other example of a source being expressed in words and in gestures; the goal
of a track is to lead the train into a certain direction, and the gesture visual-
izes and spatializes this aspect of directionality of the source: gesturally the
future career is located to the left hand side of the speaker. That this pointing
gesture is a metaphoric one only becomes clear when considering the words
with which it is co-expressive, and these entail a verbal metaphoric expres-
sion Weiche (tracks). Words and gestures share source and target of a
metaphoric expression, and these cases are what Mller (2008) terms
verbo-gestural metaphors (cf. Forcevilles 1996, 2002 concept of verbo-
pictorial metaphors).

4.2 Different source and same target in two modalities
Here we encounter two different types of multimodal metaphor: one in which
there is a gestural metaphoric expression with a target that is verbalized in a
non-metaphorical fashion, and another one in which a gestural metaphoric
expression goes along with a verbal metaphoric expression. Thus in both
cases the target is shared, but only in one case is it metaphorically conceptu-
alized in both modalities. We begin with an example in which the target of
the gestural metaphoric expression is verbalized non-metaphorically. The
example comes from conversations between American students in which they
discussed honesty as a moral value in the context of taking exams, and the
student describes honesty as a kind of abstract thought.

Example 6:
y'know,

G1
bh in front of chest, palms facing self, fingers curled
a- as far as an abstract thought of honesty is,

As soon as the speaker says far she lifts her two hands up and places them
next to each other with the palms of her hands basically facing herself, and
turned slightly towards each other. Both hands are cupped, with the fingers
tense and curled inward halfway towards the palms. The shape is as if her
hands were surrounding a medium-sized ball that she were squeezing. Given
that she holds her hands in this position for the entire phrase abstract
thought of honesty is (making rhythmic beats on the syllables far, -
stract, and hon-) we argue that this is a way in which she physically char-
acterizes this abstract thought in gesture. We therefore find the metaphoric
target domain in her words and the source domain (a solid form like that of a
round object) in the gesture. Note that there is no metaphoric expression on
the verbal level.
We sometimes see verbal and gestural metaphoric expressions being ut-
tered at the same time, each using a different source to express the same
target. In example 7 there is a color metaphor expressed verbally with a
spatial metaphor expressed gesturally (see also the discussion of this exam-
ple in Cienki 2008, and Cienki and Mller 2008b). Here the speaker from
Example 6 above continues the thought which was begun there describing
honesty as something that does not have gradations: instead it is character-
ized by clear oppositions: right or wrong, black or white.

Example 7:
I mean--
y'know,

y'know,
G1
bh in front of chest, palms facing self, fingers curled
a- as far as an abstract thought of honesty is,
y'know,

G2
bh palms together, flat in horizontal plane, lh palm up,
rh sweeps left to right across palm of lh
ther- there is no gradations.

G3
lh flat and palm up, rh flat,
outer edge taps palm of lh (v marks tap),
alternating slightly to the left (L) and to the right (R)
v L v R v L v R
Either you're right you're wrong y'r black 'r white y'know.

While verbally describing these oppositions (G3), she moves her left hand
out in front of her, palm up and open. She holds her right hand above it, flat,
with the palm held vertically, and taps the right edge of her palm against her
open left hand in time with the speech as she says each of the words right,
wrong, black, and white. Her right hand taps the left hand first
slightly on the left side of her palm (while saying right), then slightly to the
right side (while saying wrong), and repeats these left and right taps when
saying black and white, respectively. In one sense the gesture appears to
be the dividing line, separating the space on the palm of her left hand into
two parts (left and right spaces); but at the same time it indicates those very
spaces, the left and right sides of the palm of her hand, by tapping them.
Whereas the gesture indicates each member of the two sets of opposing cate-
gories as two spaces, the words invoke an opposition between black and
white. While colors (or the lack of them) would be difficult to represent in an
iconic way with gestures, spatial concepts are easily rendered, and conse-
quently the metaphor used in gesture (different spaces) is different than the
one used in words (different colors). We see here how the specific character-
istics of the expressive modality may inform the type of metaphors ex-
pressed, leading in this case to multimodal metaphoric expressions that have
different sources but share the same metaphoric target (different categories
of behavior). These expressions might be tentatively termed verbo-gestural
metaphoric compounds. They differ from verbo-gestural metaphors (source
and target are shared) in that they work together in expressing the same tar-
get metaphorically but do so with different means, i.e., by using different
sources.

4.3 Same source and different target in two modalities
It is interesting to note that, although theoretically possible, this variant does
not appear to be used at all at least insofar as our sets of data are con-
cerned. One could picture a situation where somebody talks about brushing
off crumbs of potato chips (crisps) from ones sweater while doing a dismis-
sive brushing aside gesture, characterizing this as a negative aspect of eating
potato chips.
3
Cases like these seem to be extremely rare. What we do find
instead is that the gestures appear to have a tendency to follow the seman-
tics and pragmatics of the verbal utterance, if they can (i.e., if the verbalized
content is gesturable, recall the color metaphor example). It seems, there-
fore, that when the source is shared the target is also shared. This is theoreti-
cally interesting because it puts the source information rendered gesturally
into a specific light. We will return to this highlighting of source information
through gestures later.

4.4 Discussion
If we consider the examples of metaphoric expression in gesture, described
above, we find certain aspects which are qualitatively different than what we
see in metaphoric expression in words. First, words are part of the symbolic
system of a language. We generally accept that individual words and the
phrases they comprise have ascertainable meanings motivating their use.
While within a given culture there may be recurring forms which many ges-
tures take, or certain parameters that they share, most gestures do not have
highly codified symbolic form-meaning pairings. (The well-known excep-
tions are the emblem gestures which can substitute for words, such as the
thumbs-up gesture used as a positive response to something, at least in many
European cultures.) In this way they contrast with the manually produced
signs of a signed language. Because of the spontaneously determined form,
placement, and duration of their use, and obviously because they consist of
physical forms and movements in space, we find they have properties when
serving as expressions of metaphor source domains which are different from
those of words expressing metaphoric source domains.
In addition, it is well known that many gestures present abstract ideas,
which are being mentioned in the speech, as concrete entities in front of the
speaker: the gestures indicate particular spaces and locations for the idea, or
the hands appear to hold an idea, as if it were an object. But this reification
is not a simple reification of ABSTRACT AS CONCRETE (what Lakoff and
Johnson 1980 called ontological metaphors). The gestures also show us cer-
tain properties of the objectified ideas or topics their size, relative location
as imagined by the speaker in the space before him/her, perhaps even their
metaphorical evaluation as good or bad by their placement in a high or low
space (respectively). For example, in Example 6 when the speaker says ab-
stract thought of honesty, she holds her two hands out in front of her, the
pinkie-finger sides next to each other with her palms facing herself, the
hands half curled in a tense position, as if the hands were cupping and hold-
ing an object about the size of a grapefruit. Here the abstract thought is
shown to be something quite concrete, discrete, and of a size comparable to
that of objects we manipulate with our hands every day. As this description
shows, tidy characterization of these metaphors for research purposes is
problematic, particularly when using a theory like Conceptual Metaphor
Theory, in which the formula X IS Y (e.g., GOOD IS UP) is the standard way
of analyzing metaphoric mappings. We will return to this dilemma towards
the end of the chapter. In addition, we find that metaphors do not necessarily
occur as single units, but they can also extend over time, and can add up to
complex structures, and as this volume demonstrates they may appear in a
broad range of media and modes among them film, photography, painting,
and sculpture (Forceville 2006; Gilot and Lake 1964; Mittelberg 2002;
Mller 2007).
5. The dynamic nature of metaphoric expressions in the flow of
discourse
Metaphors can be successively elaborated and specified. This holds for
monomodal verbal metaphors as well as for multimodal metaphors realized
in spoken language as well as for those instantiated in other media, such as
for instance film or cartoons (Forceville 1999, 2005, 2006; Mller 2007,
2008).
Example 8 is a continuation of Example 5 and it shows such a successive
multimodal elaboration of a verbal metaphoric expression. In this example
the co-participants are discussing their differing viewpoints regarding the
future implications of ones first job after graduating from university. The
current speaker rejects his co-participants position and argues for the im-
portant consequences that the first job may have for the path of ones future
career. Recall that he uses a German idiomatic expression die Weichen
stellen, literally, setting the tracks, when expressing his point verbally.
He only begins to gesture as he offers a succession of examples which illus-
trate and elaborate his understanding of setting the tracks by choosing a
specific job after graduation.

Example 8:
nein es is nich so,
no this is not the case,

aber es stellt natrlich Weichen.
but it obviously sets tracks.

das is das Problem.
this is the problem.

G1
2 joined flat hand point towards left
es is schon ne Weiche-
it does set tracks
G2
flat hands point forward
es is wieder ne Weiche--
it sets another track

G3
flat hands point upward
wenn de sachst ich studiere Medizin--
when you say I will study medicine

G4
2 flat hands point towards left
oder Germanistik--
or German studies
G5
2 flat hands point towards right
oder Landwirtschaft--
or agriculture
G6
2 flat hands point upwards and clap during pause
oder (..) werde Tennislehrer.
or become a tennis coach

des is schon ne Weichenstellung.
this is a kind of setting the tracks

G7
1 flat hand points forward twice, held through pause
nachm Studium mute dir wirklich berlegen welche--(...)
after graduating [from university] you really have to think carefully
which...

In the first part of his response he expresses his alternative viewpoint with a
verbal metaphoric expression: it does set tracks. We see no gesture going
with this first formulation; rather this counterargument is highlighted ver-
bally through a meta-comment: this is the problem. These first verbal
moves set the stage for a sequence of verbal illustrations and gestural enact-
ments of the verbal metaphoric expression. Subsequently the verbally ex-
pressed metaphoric concept of setting the tracks is illustrated by listing
three job alternatives medicine, agriculture, tennis each one being gestur-
ally situated in a different direction: medicine is the path to the left, agricul-
ture to the right, and tennis is located in the upward direction. The gestures
visualize the source of the metaphoric expression setting the tracks, they
embody directionality, and they locate the different future career paths in
three alternative directions in the gesture space (left, right, up). But this is
not the end of the speakers argument. After verbalizing three alternatives
and enacting three different directions for three different jobs, he summarizes
and comes back to the verbal metaphoric expression that he had used ini-
tially: this is a kind of setting the tracks; once again this verbal metaphoric
is not accompanied by a gesture. By returning to his initial expression he
retrospectively frames his verbo-gestural elaborations as examples for the
metaphoric expression he had used to challenge his co-participants argu-
ment while at the same time preparing his last and now fully explicit re-
formulation of his counter-argument: after graduating you really have to
think carefully which-- (). This last re-formulation ends with a gestural
expression of the idiom that replaces the words and is inserted into the
speech-pause. He uses yet another pointing gesture, but this one is per-
formed with one hand only and it is directed forward a direction which has
not yet been occupied by any of his preceding examples. Moreover the
gesture is highly articulated in shape: it is supported by the left hand, di-
rected towards the recipient, repeated twice, and held through the speech
pause at the end of the turn; and with this gesture the counter-argument and
the turn end.
To sum up, in this segment of talk we find a verbal metaphoric expres-
sion at the onset which is further verbally illustrated with concrete examples,
enacted and elaborated in gestural metaphoric expressions, and completed
with a final gestural metaphoric expression at the end of this counterargu-
ment. This example nicely illustrates that metaphoricity is a dynamic feature
which may trigger metaphoric elaborations in multiple modalities succes-
sively in time, and which may provide grounds for the ad hoc creation of
new metaphoric gestures, doing different jobs. We may argue that when
the verbal metaphoric expression was uttered first, metaphoricity was not in
the foreground of the speakers attention; we find no indication that meta-
phoricity was particularly active for the speaker at that point in time. Put
differently, at this moment the metaphor was sleeping; only as the speaker is
moving on is he building his elaborations in words and gestures on this
sleeping metaphoric expression, thus using it as source. Doing this makes
clear that metaphoricity becomes successively more active, as he moves
along with his argument, such that we may now speak of waking metaphors.
Formulated in McNeills terms, what we may find here is a metaphorical
growth point that structures a whole unit of discourse (McNeill 1992;
McNeill and Duncan 2000).
4
For conceptual metaphor theory this raises
questions about how to account for metaphoricity as a dynamic property,
which can be more or less highlighted (Mller 2008). Again, the formula of
TARGET IS SOURCE problematically reifies the two domains as static entities.
6. Metaphors beyond words and gestures
But there is more to multimodal metaphor in spoken language use than
words and gestures. Let us return to the various expressive forms involved in
the oral/aural modality. The metaphoric possibilities of prosodic expression
have received less attention in the literature. However, some of the existing
research on prosody (e.g., Pierrehumbert and Hirschberg 1990) may be rein-
terpreted in terms of revealing the potential of intonation for metaphoric
significance. As an example, we can take Pierrehumbert and Hirschbergs
discussion of a well-known role played by intonation in speech, that of ex-
pression on the metanarrative level. Thus the speakers belief about whether
or not s/he shares information mutually with the hearer may be expressed
metaphorically via low or high pitch accents, respectively. Looking at these
findings from the perspective of metaphor studies, might they ultimately be
grounded in the metaphorical patterns of reasoning (known since Lakoff and
Johnson 1980) concerning what is KNOWN AS DOWN and what is UNKNOWN
AS UP? (In fact these themselves are secondary metaphors based on our
Western metaphorical understanding of pitch along a vertical scale, but we
will not dwell on that here.)
Other research shows a connection between the lexical semantics of
words rated as positive or negative, and the relative pitch with which they
were produced in experimental settings, correlating with the metaphors
GOOD IS UP and BAD IS DOWN. For example, Herold (2006) found in her
study that words with positively rated meanings (like happy and yummy
[tasty]) were produced with a higher fundamental frequency (pitch) than
words with negatively rated meanings (like sad and weak).
In terms of metaphoric expression in speech which is independent of
metaphoric verbal semantics, think of vocalizations that are not lexical
words and how their interpretation can differ by the intonation with which
they are uttered. In response to a question about whether someone likes
something or not, one (at least an American English speaker) can utter
Mmm starting with a high pitch accent, and then letting the pitch fall, to
indicate a positive reply. But one could also say Mmm with a level low
tone, which could indicate a negative reaction, or at least non-confirmational
uncertainty. Thus perhaps metaphorical mappings such as POSITIVE IS UP
and NEGATIVE IS DOWN may appear in the use of intonation, even without
accompanying words that have corresponding lexical meanings.
One study (Cienki in prep.) provides some evidence about the degree to
which individuals interpret the quality of prosodic features (stress and into-
nation contours) in metaphoric terms. The study involved having 20 partici-
pants categorize a series of phrases which they simultaneously heard and
read using a set of image schemas as descriptors. The term image sche-
mas is being used here in the sense of Johnson (1987) to refer to simple
patterns which frequently recur in various aspects of peoples everyday ex-
perience (especially visual, tactile, and force-dynamic experience). The set of
image-schema names from which participants could choose was limited to
the following: container, cycle, force, object, and path, plus the alternative of
other. For comparison in this study, another 20 participants performed the
same task of having to characterize the same phrases using the given set of
image schema names, but this second group only read the phrases and did
not hear the recordings of the speakers voices uttering them. The experi-
ments were actually conducted as controls for another experimental setting
in which the participants saw and heard the video clips in which the speakers
uttered these phrases and made co-verbal gestures (Cienki 2005a). Since the
utterances were chosen because they were ones which occurred with gestures
of various kinds, the words and phrases themselves were rather random,
ranging from more substantive ones, such as their tests are difficult and
its like youre performing, to comments and interjections, such as no,
not really and like.
After completing the categorization task, the participants in the first
group were asked to write a sentence or two explaining how they used the
image schemas to categorize the phrases they heard. The results revealed
that they sometimes categorized some of the phrases according to their
acoustic properties, rather than referring to the meanings of the lexical items.
Consider the following response as an example: a phrase where the tone
rose and fell back again seemed cyclical, whereas when the tone steadily rose
it seemed like a path. We see how metaphor may play a role in interpreting
how an utterance was spoken. As a side note, this could be important for
metaphor researchers in terms of setting up stimuli for experiments on meta-
phor interpretation. The findings underscore the importance of considering
the mode of presentation of experimental stimuli (in oral versus written
form) because of the effect it may have on the interpretation of the same
linguistic expressions.
It is worth noting with these examples of metaphor in intonation that we
are not dealing with verbal semantics, but with metaphor on the pragmatic
level what the speaker meant with the use of a given intonation contour.
Interestingly, we find a parallel phenomenon of metaphor on the pragmatic
level in gesture. The primary function of some gestures appears to be to
highlight interactive or interpersonal relations, to parse the discourse, or to
accomplish a performative act (Kendon 2004: ch. 9). Mller, referring to
unpublished observations by Jrgen Streeck, discusses the pragmatic func-
tions of the palm-up open-hand (PUOH) gesture, which can serve to pre-
sent an abstract, discursive object as a concrete, manipulable entity (2004:
233). The gesture can indicate that what the speaker is saying is to be inter-
preted as an idea to be discussed, a proposal, or a question (Kendon 2004:
159). In terms of conceptual metaphor theory, we might say that this gesture
uses the pragmatic metaphor of INTRODUCING AN IDEA IS PRESENTING AN
OBJECT. Here as in the other recurrent gestures discussed above, the meta-
phor does not simply work on the level of what the speakers words express
semantically. Rather, it works on the pragmatic level, in that the source is
expressed in the gesture, and the target is what the speaker is doing with
his/her words as well as with his/her gestures.
7. Implications for metaphor theory
A major conclusion we can draw from the fact that metaphors can be real-
ized in multiple modalities is that metaphoricity is modality-independent. It
documents that the establishment and creation of metaphoricity is a cognitive
process with products in various modalities, thus offering strong support for
Lakoff and Johnsons initial idea of moving metaphor(icity) out of the realm
of literary discourse into the mundane world of everyday thought (Mller
2003, 2007, 2008). However, this also has critical implications for metaphor
theory in that it calls for refined empirical methodology as well as for a new
theoretical understanding of the different forms of multimodal metaphors and
their constitutive semantic relations. It also directs our attention to the neces-
sity of including a cognitive-semiotic analysis of metaphoric, as well as of
metonymic, processes (see Mittelberg 2006, 2007; Mittelberg and Mller in
prep. a; Mittelberg and Waugh this volume). A major implication of the
insights gained through the analysis of multimodal metaphors in the use of
spoken language is the fact that as spoken language is inherently dynamic, so
is multimodal metaphor.
As already indicated above, the study of metaphors as expressed in the
dynamic processes of speaking presents us with metaphoric source domains
which are themselves contingent on time for their realization. This raises a
problem, given the traditional means of conceptual metaphor analysis,
namely that it involves the static verbal formula of TARGET IS SOURCE (an
issue raised long ago by the anthropologist Bradd Shore, personal communi-
cation). Various authors in recent research have suggested alternatives to try
to overcome the limitations of this analytic device.
For some types of source domains, one solution is to characterize them
by using schematic images. An example described in Cienki (2005b) is that
when Al Gore was a candidate for U.S. president in 2000 he used the same
gesture at several points during the televised debates: a gesture with one or
both hands palm up and cupped slightly with the fingers slightly curved, as if
he were holding a small ball. This gesture occurred with phrases such as
enable us to project the power for good, shepherds that economic
strength, the power of example is Americas [greatest power] in the
world (with square brackets indicating the timing of the gesture in the last
example). We would argue that in the examples from Gore, the gesture
serves basically the same purpose as the PUOH gesture discussed by Mller
(2004), but that there is an added element here indicated by the cupped shape
of the hand. In a physical situation, such a hand shape would be used not
only to support a small object in the hand, but also to prevent it from falling
off the extended hand, thus protecting it in a way. Thus the gesture not only
suggests that the speaker is treating AN IDEA AS AN OBJECT, which he is
presenting to the addressee (the moderator of the debate and, by extension,
the television audience), but that he is also showing something about his
attitude toward the idea he is presenting, perhaps that it is something good
which he wants to support (all three utterances expressed positive ideas
which Gore espoused). In light of the meaning added by the cupped hand
shape, the manner of presenting is significant, and (as argued in Cienki
2005b) could be indicated by a diagram or schematic image see the ones
Efron (1972) used in his analysis of the linguistic properties of the gestures
used by Italians as compared to Eastern-European Jewish immigrants to
New York City. See also Calbris (2003) schematizations of gesture hand
shapes and motions in diagrammatic form. Finally, the increasing use of
digital publishing (online or on CDs or DVDs) allows for video characteriza-
tion of source domains which are dynamic in nature, in that they can be pre-
sented as moving schematic images, for example as small animations.
8. Implications for thinking for speaking and gesturing
Slobin (1987, 1996) argues that there is a special form of thought which is
mobilized in the process of talking, which he calls thinking-for-speaking. As
he describes it, Thinking for speaking involves picking those characteris-
tics [of a perceived event, CM and AC] that (a) fit some conceptualization of
the event, and (b) are readily encodable in the language which the speaker is
using at the moment (Slobin 1987: 435). Thus the lexical and grammatical
means of expression available in a language are used by speakers already as
they are anticipating how to utter what they want to utter. McNeill and Dun-
can (2000) suggest that gesture needs to be taken into account in this process
as well. They discuss how the idea units which we are continually develop-
ing and unraveling for expression while we talk, what McNeill (1992) has
called growth points, combine both imagery and linguistic-categorical con-
tent. In the process of thinking while speaking, which McNeill and Duncan
(2000: 157) note is perhaps a more accurate way to refer to the phenomenon,
the imagistic content receives partial expression in the gestures that the
speaker produces. Indeed, which imagery is expressible in gesture is a factor
in how the verbo-gestural utterance is produced. Consequently we (Mller
and Cienki 2006) have discussed the process as thinking-for-speaking-and-
gesturing. Both the nature of the available linguistic forms as well as the
expressive potential of hand-gestures which one can use in the expression of
ones thoughts while speaking are significant for what thoughts ultimately
get expressed. This means that there are important connections between
which single or multiple modality/ies are at ones disposal for expression and
the kind(s) of metaphoric ideas which one ultimately conceptualizes and
expresses either monomodally or multimodally.
9. Conclusion
The insights gained through the analyses of multimodal metaphors in lan-
guage use have rather far-reaching consequences for a theory of metaphor.
Not only do they underline the stance of Conceptual Metaphor Theory with
regard to the principally modality-independent nature of metaphoricity (this
means metaphor as a cognitive mechanism); they uncover that the hitherto
static view on metaphor in thought and language must be supplemented by a
dynamic view on metaphor in thinking, speaking and gesturing (cf. Mller
2003, 2007, 2008; Mller and Cienki 2006;). Such a dynamic view takes
into consideration the procedural nature of meaning creation in situations of
face-to-face communication, including the elaboration of metaphoric expres-
sions in the discourse as well as the dynamic activation of metaphoricity,
which for a specific and often short moment in time may turn sleeping meta-
phors into waking ones (Mller 2003, 2007, 2008). Put differently, these
observations indicate that we need a theory that distinguishes between prod-
ucts and processes (Gibbs 1992, 1998, 1999; cf. also Cameron and Low
1999), and also between system and use (Mller 2008; Steen 2006; Steen
and Gibbs 1999) or one that cross-cuts these dichotomies as Steen (2007)
has recently proposed; that one distinguishes metaphors in grammar and
usage, be they approached as symbolic structure or as forms of behavior.
Further consequences of realizing the multimodal nature of metaphoric
expression in the use of spoken language include that in co-speech gesturing
as well as in prosodic features of speech we may see manifestations of the
imagistic or, more generally, the embodied nature of many metaphoric
source domains. We also see that the metaphoric process is not a unidirec-
tional one, one in which a preconfigured thought is being translated into
gesture, word, or sound; rather we must conceive of it as an interactional
process which takes into consideration the nature and the expressive poten-
tial of the respective modalities (colors do not lend themselves to expression
in gesture, but for spatial relations the opposite holds true). Compare for
instance what is known about metaphoric expression in another use of the
manual modality, namely in sign language. In both cases, gesture and sign
language, the iconic nature of visual/manual expression affords different
potentials than aural/oral expression does (Mller in prep.; Taub 2001),
although gestures with speech are normally co-verbal, as opposed to consti-
tuting linguistic signs in and of themselves. However, in the process of
communication or to put it in Wallace Chafes (1994) terms, in the flow of
discourse these modality-specific properties can be exploited to varying
extents in any given event of speaking.
A dynamic approach to linguistic theory (such as that proposed by
McNeill 2005) or to metaphor theory (as in Mller 2008) which can ac-
commodate the multimodal potential of language production and reception
can provide a more complete picture of the complexity of this form of human
behavior than the static views of language, metaphor, and thought which
currently dominate the field of cognitive linguistics and beyond. In conclu-
sion, for researchers of spoken language, moving beyond the level of the
words can uncover many facets of metaphoricity that had previously lain
hidden.
Acknowledgements
We are grateful to Charles Forceville, Irene Mittelberg, and Linda Waugh for
insightful comments on an early draft of this chapter.
Notes
1. In his work on F-Formations (facing formations) Kendon (1990) indicates
that facing formations may actually show different forms, varying with the
amount of participants involved but also within dyadic encounters: In F-
formations of two indidviduals, for example, we may see arrangements that
vary from a direct face-to-face pattern, to an L-shaped pattern, or even a side-
by-side pattern (Kendon 1990: 250). Kendon (personal communication) has
also observed that Australian aboriginals conduct conversations in certain
situations all facing the same direction, i.e., in a side-by-side configuration.
Tzeltal speakers appear to quite commonly choose a side-by-side configura-
tion in dyadic situations (Stephen Levinson, personal communication). For
the interactive structure of establishing such F-formations, see Mller and
Bohle (2007).
2. For a more detailed account of this and other similar examples, see Mller
(2008).
3. This example is inspired by a case reported by Teendorf, in which the
brushing aside movement is used to brush aside crumbs of potato chips, func-
tioning here as an object manipulation. For the metonymic and metaphoric
links necessary to transform this action into a pragmatic gesture see Teen-
dorf (in prep. a, b).
4. For further discussion and more examples of dynamic metaphoricity in ges-
ture and speech see Mller (2003, 2007, 2008) and Cienki and Mller
(2008a).
References
Bouvet, Danielle
1997 Le corps et la mtaphore dans les langues gestuelles: A la recher-
che des modes de production des signes. Paris: LHarmattan.
2001 La dimension corporelle de la parole: Les marques posturo-mimo-
gestuelles de la parole, leurs aspects mtonymiques et mtaphori-
ques, et leur rle au cours dun rcit. Paris: Peeters.
Bressem, Jana, and Silva Ladewig (eds.)
in prep. Handmade patterns: Recurrent Forms and Functions of Gestures.
Amsterdam/Philadelphia: Benjamins.
Calbris, Genevive
1990 The Semiotics of French Gestures. Bloomington, IN: Indiana Uni-
versity Press.
1998 Geste et images. Semiotica 118 (1-2): 105120.
2003 From cutting an object to a clear cut analysis: Gesture as the repre-
sentation of a preconceptual schema linking concrete actions to ab-
stract notions. Gesture 3: 1946.
Cameron, Lynne, and Graham Low (eds.)
1999 Researching and Applying Metaphor. Cambridge: Cambridge
University Press
Chafe, Wallace
1994 Discourse, Consciousness, and Time: The Flow and Displacement
of Conscious Experience in Speaking and Writing. Chicago: Uni-
versity of Chicago Press.
Cienki, Alan
phorical expressions. In Discourse and Cognition: Bridging the
Gap, Jean-Pierre Koenig (ed.), 189204. Stanford, CA: Center for
the Study of Language and Information.
1999 Metaphors and cultural models as profiles and bases. In Metaphor
in Cognitive Linguistics, Raymond. W. Gibbs, Jr. and Gerard J.
Steen (eds.), 189203. Amsterdam/Philadelphia: Benjamins.
2005a Image schemas and gesture. In From Perception to Meaning: Image
Schemas in Cognitive Linguistics, Beate Hampe (ed.), 421441.
Berlin/New York: Mouton de Gruyter.
2005b Metaphor in the Strict Father and Nurturant Parent cognitive
models: Theoretical issues raised in an empirical study. Cognitive
Linguistics 16: 279312.
2008 Why study metaphor and gesture? In Metaphor and Gesture, Alan
Cienki and Cornelia Mller (eds.), 525. Amsterdam/Philadelphia:
Benjamins.
in prep. The (image) schematicity of concrete versus abstract referential
gestures and their accompanying words.
2008a Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins.
Cienki, Alan, and Cornelia Mller
2008b Metaphor, gesture, and thought. In The Cambridge Handbook of
Metaphor and Thought, Raymond W. Gibbs, Jr. (ed.), 483501.
Cambridge: Cambridge University Press.
Clark, Herbert
1973 Space, time, semantics, and the child. In Cognitive Development
and the Acquisition of Language, Timothy E. Moore (ed.), 2763.
New York: Academic Press.
Efron, David
1972 Gesture, Race, and Culture. (First published (1941) as Gesture and
Environment). Berlin/New York: Mouton de Gruyter.
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge
1999 The metaphor COLIN IS A CHILD in Ian McEwans Harold Pinters,
bol 14: 17998.
the Asterix album La Zizanie. Journal of Pragmatics 37: 6988
2006 Non-verbal and multimodal metaphor in a cognitivist framework:
Agendas for research. In Cognitive Linguistics: Current Applica-
tions and Future Perspectives, Gitte Kristiansen, Michel Achard,
Ren Dirven, and Francisco J. Ruiz de Mendoza Ibez (eds.), 379
1993a Process and products in making sense of tropes. In Metaphor and
Thought (2nd ed.), Andrew Ortony (ed.), 252276. Cambridge:
1998 The fight over metaphor in thought and language. In Figurative
Language and Thought, Albert N. Katz, Cristina Cacciari, Raymond
W. Gibbs, Jr., and Mark Turner (eds.), 119157. New York/Oxford:
Oxford University Press.
1999 Researching metaphor. In Researching and Applying Metaphor,
Lynne Cameron and Graham Low (eds.), 2947. Cambridge: Cam-
Gilot, Francoise, and Carlton Lake
1964 Life with Picasso. New York: McGraw-Hill.
Herold, Debora
2006 Acoustic correlates to word meaning in infant directed speech.
Ph.D. diss., Emory University.
Johnson, Mark
Kendon, Adam
1990 Conducting Interaction: Patterns of Behavior in Focused Encoun-
ters. Cambridge: Cambridge University Press.
2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge Uni-
versity Press.
Kendon, Adam, and Laura Versante
2003 Pointing by hand in Neapolitan. In Pointing: Where Language,
Culture and Cognition Meet, Sotaro Kita (ed.), 109137. Hillsdale,
NJ: Lawrence Erlbaum.
Ladewig, Silva
in prep. Metonymic relations in a recurrent polysemous gesture. In Meton-
ymy in Gesture and Signed Languages, Irene Mittelberg and Corne-
lia Mller (eds.), special Issue of CogniTextes.
McNeill, David
McNeill, David, and Susan Duncan
2000 Growth points in thinking-for-speaking. In Language and Gesture,
David McNeill (ed.), 141161. Cambridge: Cambridge University
Press.
Mittelberg, Irene
2002 The visual memory of grammar: Iconographical and metaphorical
insights. Metaphorik.de 2/2002: 6989.
2006 Metaphor and Metonymy in Language and Gesture: Discourse Evi-
dence for Multimodal Models of Grammar. Ph.D. diss., Cornell
University.
2007 Internal and external metonymy in co-verbal gesture: Jakobsonian
and cognitivist views on contiguity relations within and across mo-
dalities. Paper presented at the 10th ICLC, Krakow, July 2007.
multimodal figures of thought in co-speech gesture.
Mittelberg, Irene, and Cornelia Mller (eds.)
in prep. Metaphor and Metonymy in Gesture and Sign Languages. Special
Issue. CogniTextes.
Morris, Desmond
1977 Manwatching. London: Jonathan Cape.
Mller, Cornelia
1998 Redebegleitende Gesten. Kulturgeschichte Theorie Sprachver-
gleich. Berlin: Berlin Verlag Arno Spitz.
2003 Gestik als Lebenszeichen toter Metaphern. Tote, schlafende und
wache Metaphern. Zeitschrift fr Semiotik 1-2: 6172.
2004 Forms and uses of the Palm Up Open Hand: A case of a gesture
family? In The Semantics and Pragmatics of Everyday Gestures,
Cornelia Mller and Roland Posner (eds.), 23356. Berlin: Weidler.
2007 A dynamic view of metaphor, gesture and thought. In Gesture and
the Dynamic Dimension of Language, Susan Duncan, Justine Cas-
sell, and Elena Levy (eds.), 109116. Amsterdam/Philadelphia:
Benjamins.
2008 Metaphors Dead and Alive, Sleeping and Waking: A Dynamic
View. Chicago: University of Chicago Press.
in prep. a Creative devices in gestures and sign languages. In Cornelia Mller,
Ellen Fricke, Silva Ladewig, Irene Mittelberg, Sedinha Teendorf
(in prep.) Gestural Modes of Representation as Mimetic Devices
Revisited. Gesture.
in prep. b How hand movements turn into gesture: Gestural modes of repre-
sentation as metonymic resources of gesture creation. In Metaphor
and Metonymy in Gesture and Sign Languages. Special Issue of
CogniTextes, Irene Mittelberg and Cornelia Mller (eds.).
Mller, Cornelia, and Ulrike Bohle
2007 Das Fundament der Interaktion: Zur Vorbereitung und Herstellung
von Interaktionsrumen durch krperliche Koordination. In
Koordination: Analysen zur multimodalen Interaktion, Reinhold
Schmitt (ed.), 129165. Tbingen: Narr.
Mller, Cornelia, and Alan Cienki
2006 Verbal to gestural, and gestural to verbal, metaphoric expression:
Activation and development of metaphoricity. Paper presented at the
sixth conference on Researching and Applying Metaphor (RaAM
6), Leeds, UK, April 2006.
Mller, Cornelia, and Harald Haferland
1997 Gefesselte Hnde. Zur Semiose performativer Gesten. Mitteilungen
des Germanistenverbandes 3: 2953.
Mller, Cornelia, and Gerald Speckmann
2002 Gestos con una valoracin negativa en la conversacin cubana. De
Signis 3: 91103.
Neumann, Ragnhild
2004 The conventionalization of the ring gesture in German discourse. In
The Semantics and Pragmatics of Everyday Gestures, Cornelia
Mller and Roland Posner (eds.), 217224. Berlin: Weidler.
Nez, Rafael
2004 Do real numbers really move? Language, thought and gesture: The
embodied cognitive foundations of mathematics. In Embodied Arti-
ficial Intelligence, Fumiya Iida, Rolf Pfeifer, Luc Steels, and Yasuo
Kuniyoshi (eds.), 5473. Berlin: Springer.
Nez, Rafael, and Eve Sweetser
2006 With the future behind them. Convergent evidence from Aymara
language and gesture in the crosslinguistics comparison of spatial
construals of time. Cognitive Science 20: 149.
Pierrehumbert, Janet, and Julia Hirschberg
1990 The meaning of intonational contours in the interpretation of dis-
course. In Intentions in Communication, Philip R. Cohen, Jerry
Morgan, and Martha E. Pollack (eds.), 271311. Boston, MA: MIT
Press.
Pragglejaz Group
2007 MIP: A method for identifying metaphorically used words in dis-
course. Metaphor and Symbol 22: 139.
Slobin, Dan I.
1987 Thinking for speaking. Proceedings of the 13th Annual Meeting of
the Berkeley Linguistics Society, 435445.
1996 From thought to language to thinking for speaking. In Rethink-
ing Linguistic Relativity, Stephen C. Levinson and John Gumperz
(eds.), 7096. Cambridge: Cambridge University Press.
Speckmann, Gerald
1999 Sprachbezug und Konventionalitt von Gesten am Beispiel
kubanischer uerungen. MA diss., Freie Universitt Berlin.
Steen, Gerard J.
2006 Metaphor in applied linguistics: Four cognitive approaches. Docu-
mentao de Estudos em Lingstica Terica e Aplicada
D.E.L.T.A. 22 (1): 2144.
2007 Finding Metaphor in Grammar and Usage: A Methodological
Analysis of Theory and Research. Amsterdam/Philadelphia: Benja-
mins.
Steen, Gerard J. and Raymond W. Gibbs, Jr.
1999 Introduction. In Metaphor in Cognitive Linguistics, Raymond W.
Gibbs, Jr. and Gerard J. Steen (eds.), 18. Amsterdam/Philadelphia:
Benjamins.
Streeck, Jrgen
1994 Speech-handling: The metaphorical representation of speech in
gestures. A cross-cultural study. Ms. University of Texas at Austin.
Sweetser, Eve
1998 Regular metaphoricity in gesture: Bodily-based models of speech
interaction. Actes du 16e Congrs International des Linguistes
(CDRom). Amsterdam: Elsevier.
Teendorf, Sedinha
in prep. a From everyday action to gestural performance: Metonymic motiva-
tions of a pragmatic gesture. In Metaphor and Metonymy in Gesture
and Sign Languages. Special Issue of CogniTextes, Irene Mittelberg
and Cornelia Mller (eds.).
in prep. b Pragmatic and recurrent: The case of the brushing aside gesture. In
Recurrence in Coverbal Gestures Investigating Form and Func-
tion, Jana Bressem and Silva Ladewig (eds.). Amsterdam/ Philadel-
phia: Benjamins.
Taub, Sarah
2001 Language from the Body: Iconicity and Metaphor in American Sign
Language. Cambridge: Cambridge University Press.
Webb, Rebecca
1996 Linguistic Features of Metaphoric Gestures. Ph.D. diss., University
of Rochester, NY.
Wilcox, Phyllis P.
2000 Metaphor in American Sign Language. Washington, DC: Gallaudet
University Press.
2004 A cognitive key: Metonymic and metaphorical mappings in ASL.
Cognitive Linguistics 15: 119147.
Chapter 14
Metonymy first, metaphor second: A cognitive-
semiotic approach to multimodal figures
of thought in co-speech gesture
Irene Mittelberg and Linda R. Waugh
Abstract
Based on spoken academic discourse and its accompanying gestures, this chapter
presents a cognitive-semiotic approach to multimodal communication that assigns
equal importance to metaphor and metonymy. Combining traditional semiotics
with contemporary cognitivist theories, we demonstrate how these two figures of
thought jointly structure multimodal representations of grammatical concepts and
structures. We discuss Jakobsons view of metaphor and metonymy, and particu-
larly his distinction between internal and external metonymy, thus discerning
various principles of sign constitution and indirect reference within metaphoric
gestures (whether or not the concurrent speech is metaphorical). We then intro-
duce a dynamic two-step interpretative model suggesting that metonymy leads the
way into metaphor: in order to infer the imaginary objects or traces that gesturing
hands seem to hold or draw in the air, a metonymic mapping between hand
(source) and imaginary object (target) is a prerequisite for the metaphorical map-
ping between that very object (source) and the abstract idea (target) it represents.

Keywords: metaphor, metonymy, gesture, semiotics, cognitive theory
1. Introduction
Work done by scholars in many disciplines has shown that metaphor and
metonymy rely on general cognitive processes of conceptualization and asso-
ciation that may materialize in modalities other than spoken and written
words, e.g., in gesture. While the chapters in this volume contribute to a
unified approach to the role of metaphor in multimodal representations, we
will show here that it is both metaphor and metonymy that, by working to-
gether in multimodal communication, function to convey complex meanings,
330 Irene Mittelberg and Linda R. Waugh
just as they motivate, to various degrees, all processes of signification (Ja-
kobson 1990 [1956]). Thus, we agree with many other scholars that more
attention should be paid to metonymy: it is equal in importance to metaphor
but crucially different from it in its contribution to communication (e.g.,
Barcelona 2000; Croft 1993; Dirven and Prings 2002; Gibbs 1994;
Goossens et al. 1995; Lakoff 1987; Panther and Radden 1999; Panther and
Thornburg 2003; Turner and Fauconnier 2002; Wilcox 2004).
We will show this by focusing on a multimodal combination of verbal
and bodily communication namely, spoken academic discourse and ac-
companying manual gestures (see also Mller and Cienki, this volume). The
discourse here is classroom lectures by linguists about grammar and syntac-
tic theory and, as we will demonstrate, both metaphor and metonymy play a
crucial role in presenting these abstract phenomena. Multimodal representa-
tions of knowledge domains such as grammar have a long-standing tradition
and visual metaphor, in particular, has widely been used to illustrate linguis-
tic explanations (see Mittelberg 2002 on Early Modern printed images of
grammar). Besides figurative language, pictorial metaphor has been studied
as a way of expressing an analogy between two entities (e.g., Carroll 1994;
Forceville 1994, 1996, 2002, 2005; Kennedy 1982). However, if we com-
pare gesture to solid sculptures or pictorial signs, we realize that, just as
speech in the auditory domain is inherently dynamic and fleeting, so with
gestures the interplay of time, space, and motion typically engenders tran-
sient rather than lasting images that have to be understood quickly in time.
This difference is crucial for an understanding of gesture.
In the gestural representations of grammar we will investigate here, the
hands and arms serve as a resource for making meaning as they correlate
with the on-going speech through co-speech gestures (McNeill 2005). Cogni-
tive linguists have provided a detailed picture of the different ways in which
the human body serves as a (re)source for a large array of metaphors (e.g.,
Kvecses 2002; Lakoff and Johnson 1980, 1999). We are primarily inter-
ested here in the ways in which the gesture modality generally can provide
insights into situated cognition because it helps to externalize concepts,
structures and practices through the use of both space and the body
thereby creating an ex-bodiment of these concepts and structures through
motor actions and other forms of interaction with the physical and social
environment (Mittelberg 2006). Thus, this work rests on the premise that
concepts (and other aspects of conceptual structure such as image schemata)
are embodied and are themselves multimodal, including kinesthetic features
(Hampe 2005; Johnson 1987; Lakoff 1987; Lakoff and Johnson 1980, 1999;
Sweetser 1990, 1998). One of our aims here is to present ways in which co-
Metonymy first, metaphor second 331
speech gesture provides additional evidence for conceptual metaphor and
metonymy by revealing in a dynamic fashion dimensions of schemata not
necessarily expressed in the concurrent speech (Calbris 2003; Cienki 1998,
2005; Cienki and Mller 2008; McNeill 1992, 2005; Mittelberg 2008;
Mller 1998, 2008; Sweetser 1998, 2007; Taub 2001). Another is to show
that some of the claims about the relationship between source and target
domains made on the basis of multimodal representations without gestures
fail to account for gestures satisfactorily, since the metaphorical meaning-
making processes cannot be elucidated without taking into consideration how
they interact with metonymic modes. We will also show that metonymy as-
sumes distinct functions regarding, for instance, gestural sign constitution
through synecdoche and indirect reference, as others have suggested (see
Bouvet 2001; Gibbs 1994; Mller 1998; Wilcox 2004; Wilcox and Morford
2007).
Building on these considerations, we present an approach to multimodal
communication, and in particular to multimodal metaphor and metonymy,
that has been shaped by both theoretical motivations and by the nature of our
data. Our approach combines contemporary cognitivist theories (Gibbs
1994, 1999; Johnson 1987; Lakoff 1987, 1993; Lakoff and Johnson 1980,
1999; Sweetser 1990) with the older but still relevant work of C. S. Peirce
(1960, 1991, 1992, 1998) and Roman Jakobson (1956, 1987a, b, 1990;
Jakobson and Pomorska 1983). The Jakobsonian and Peircean semiotic ap-
proaches continue to provide rich perspectives for multimodal research and
lend themselves to being applied to gesture, for they are broader in scope
than linguistic theories and have been used to analyze a wide variety of dy-
namic, discursive phenomena such as theatre, cinema, myths, rituals, music,
poetry, etc. Since we have treated in detail elsewhere Peirces theory (Waugh
1992, 1998; Waugh et al. 2004) and Peircean perspectives on the gestural
sign (Mittelberg 2006, 2008; see also Fricke 2007; McNeill 1992), our fo-
cus here will be on Jakobsons approach and its relevance for gesture re-
search from a cognitivist perspective.
Our rationale for combining these theoretical perspectives rests on the
fact, firstly, that Cognitive Metaphor Theory (CMT) and Peirces semiotic
share central assumptions about the link between image-schematic structures
and metaphorical (and diagrammatic) projections, as well as about habitual
patterns of experience and interpretation (Danaher 1998; Hiraga 1994,
2005; Mittelberg 2006, 2008). Second, Jakobson emphasized the importance
of metaphor and metonymy as different semiotic and cognitive strategies that
structure both verbal and non-verbal signs and messages (Jakobson 1956,
1987a, b; Lodge 1977; Waugh 1998, 2000; Waugh and Monville-Burston
1990; Waugh et al. 2004). Thirdly, both Jakobson and Peirce stress the
point of view of the interpreter of a given message: in similar fashion, our
aim will be to explicate how gestures are interpreted by the viewer (e.g., the
student, the gesture analyst) through both metaphor and metonymy.
2. Characterizing the data: Semiotic idiosyncrasies of metalinguistic
discourse and its accompanying gestures
Connecting the body, language, cognition, society and culture, gestures can
provide a window into thought processes and their pragmatic and ecological
anchorage (Goodwin 2003; Ochs et al. 1996; Streeck 2002). From a cogni-
tivist viewpoint, i.e., taking the perspective of the speaker, research has
shown how hands (and arms) may reveal, consciously or unconsciously,
thoughts and attitudes that speech might conceal (McNeill 1992: 246). We,
on the other hand, are trying to see how the gestures help the viewer to un-
derstand the conceptualization of abstract ideas that the speaker/gesturer is
communicating.
The spontaneous gestures we will be analyzing here are not part of an
elaborated sign system but are created by the speaker as he/she speaks, and
thus gesture and speech can produce very different effects, including junc-
ture or disjuncture, redundancy, complementation, or mismatch (Goldin-
Meadow 2003; McNeill 2000, 2005). A gesture may disambiguate linguistic
information and thus make meaning more precise (for instance, by pointing
at a concrete referent that is linguistically only referred to via an unspecified
pronoun), or it may add components of meaning not expressed in the speech
it accompanies (Kendon 2000).
1
Often, however, spontaneous gestural signs
tend to be polysemous and need a contextual support to be correctly inter-
preted; thus, discourse-pragmatic factors and concurrent speech help to dis-
ambiguate them (Calbris 1990; Kendon 2004; McNeill 2005; Mller 1998).
As Jakobson noted, a pointing gesture at a package of cigarettes could be
interpreted to mean this package in particular, or a package in general, one
cigarette or many, a certain brand or cigarettes in general, or, still more gen-
erally, something to smoke. The viewer does not know if the pointer is
simply showing, giving, selling, or prohibiting the cigarettes. The only
way to know is through the accompanying speech (Jakobson 1953: 567). A
single gesture could also fulfill several functions at once: e.g., from represen-
tational to deictic, or from accentuating the rhythm of the speech to attract-
ing attention and managing interaction between the interlocutors. Gestures
are thus visuo-spatial motor signs (Jakobson 1987a: 474) that derive their
locally-situated meaning from the very human body that articulates them, the
speech they accompany, and the socio-cultural and material environment the
person interacts with. Consequently, a gestural sign does not exist, and can-
not be analyzed, detached from either the human body or the here and now of
the speech event (the origo in Bhlers terms, see Fricke 2007). This means
that in order to understand the gestures under discussion here, we first need
to characterize the speech they accompany in terms of its genre and func-
tions.
As indicated above, our data come from one specific spoken genre:
metalinguistic academic discourse in lecture format, from a corpus of such
lectures by four professors (three women and one man), all native speakers
of American English, while they were teaching introductory linguistics to
undergraduate and graduate students at two major American universities.
The lectures were videotaped in a naturalistic setting, that is, regularly
scheduled classes where neither the teacher nor the students knew about the
purpose of the taping (in particular, they did not know that the analysis was
to focus on gesture). Thus, the assumption is that the gestures used by the
professors were not affected by the videotaping (for a detailed description of
methods of collecting, editing and transcribing the data, including the coding
and annotation systems used, see Mittelberg 2006, 2007). Now, in the typi-
cal classroom setting there are other visual modalities: e.g.,
black/green/white boards with writing and other visuals on them, handouts,
slides and power point projections. However important these are for the
communication of information in the classroom, what is unique to gestures is
that they are conveyed by the body of the lecturer and correlated with the
speech that is emanating from that same body.
2

The speech that is at issue here is highly complex. It has multiple func-
tions: it conveys information about language that reflects the beliefs of the
speaker and is directed at the audience (the students in the class) with the
aim that the students will gain at least an understanding of, and perhaps also
a belief in, the concepts being discussed. The gestures have the same com-
plex multifunctionality as they contribute to the communication and under-
standing of the lectures. Our focus will be on what Mller (1998: 110113)
calls referential gestures, that is, gestures that depict objects, attributes of
objects and people, actions, or behaviors, whether concrete or abstract
(Mller 1998; see also Cienki 2005). More specifically, the gestures ana-
lyzed here are all attempts at making fairly abstract grammatical concepts
and aspects of the syntactic structure of sentences more understandable for
the listener/viewer, by turning them into (partial) visuo-spatial and embodied
manifestations of these concepts.
3. Jakobson: Metaphor and metonymy (internal and external),
similarity and contiguity, selection and combination
Roman Jakobsons view of metaphor and metonymy has been successfully
utilized in the analysis of a wide variety of monomodal and multimodal texts
(Bradford 1994; Jakobson 1956; Jakobson and Pomorska 1983; Shapiro
1983; Whittock 1990) and is particularly valuable and compatible with con-
temporary, cognitively-oriented accounts of metaphor and metonymy since it
is in fact one of their predecessors. In recent publications (e.g., Dirven and
Prings 2002), cognitive linguists revisit and offer a great deal of evidence
for Jakobsons theory. This chapter attempts to show that adopting his bal-
anced approach and exploring the interplay of these two different mental
strategies of conceptualization (Dirven 2002: 75) has the potential to illu-
minate the semiotics of gesture and of multimodal communication more gen-
erally.
3

Jakobson (1956) contends that metaphor and metonymy are two different
modes of association that structure both linguistic and non-linguistic signs.
While until not too long ago metaphor consistently received much more
scholarly attention than metonymy, Jakobson paid equal attention to both
tropes. In his view, metonymy is not a sub-type of metaphor, but the two are
in opposition with each other and thus create a fundamental polarity that is
at the root of all symbolic processes, cultural manifestations, and human
thought in general. Thus, studies concerned with metaphor ought to pay
(more) attention to its interaction with metonymy, a view that is also present
in the work of quite a large number of cognitive linguists.
In defining the difference between metaphor and metonymy, Jakobson
was particularly inspired by Peirces famous trichotomy of signs: icon, index
and symbol. According to Peirce, similarity is at the root of iconic relation-
ships between the sign and the object it represents and he includes meta-
phor as a specific sub-type of icon (Peirce 1960, 1992, 1998). Contiguity,
on the other hand, is inherent to the index, deictic categories, and, as Jakob-
son (1956, 1966) also emphasized, metonymy. For Jakobson, similarity and
contiguity are bipolar opposites, representing the two essential structural
relations between signs that permeate all of language (Shapiro 1983: 194).
Thus, similarity is the basis for metaphor, as well as synonymy, paraphrase,
antonymy, analogy, etc, and contiguity underlies metonymy, as well as spa-
tial and temporal neighborhood (both proximity and remoteness), cause-
effect relations, etc. In addition, Jakobson differentiated between two major
subtypes of metonymy: (1) external metonymy (metonymy proper), in
which the name of an object is replaced by the name of an attribute, or of
an entity related in some semantic way (e.g., cause and effect; instrument;
source) (Wales 2001: 252), e.g., the term the White House when refer-
ring to the President of the U.S. (place for person); and (2) internal meton-
ymy(synecdoche), in which the name of the referent is replaced strictly by
the name of an actual part of it (Wales 2001: 252) or by the name of the
whole of which it is a part; e.g., part stands for whole and whole for part
(e.g., all hands on deck, in which hands stands for the whole body). That
is, Jakobson integrated synecdoche as an important sub-type of metonymy,
and as we will see, these two types of metonymy are crucial to the study of
gesture. Most importantly, Jakobson insisted that similarity and contiguity
and metaphor and metonymy are not mutually exclusive: just as signs can
exhibit both similarity and contiguity in differing hierarchies (Jakobson
1966: 411), so the nature of a given sign is dependent on the preponderance
of one of the two modes over the other (see Jakobson 1956: 130).
According to Jakobson (1956: 117, see also Waugh and Monville-
Burston 1990), the similarity/contiguity relations between signs are different
from the basic types of operations by which any linguistic utterance is con-
structed by the speaker. Any act of utterance formation involves the selec-
tion of certain linguistic entities from the code (e.g., words) and their combi-
nation into linguistic units of a higher degree of complexity (e.g., phrases
and sentences). Understanding by the addressee implies the reverse order of
operations: the combination of units of greater complexity has to be dis-
solved into the individual linguistic entities selected. Both modes of ar-
rangement (Jakobson 1956: 119) reflect the structural reality of language:
selection relies on the organization of the linguistic system, while combina-
tion is evidenced in the fact that every sign is made up of constituent signs
(sentences, words, morphemes, phonemes, features) and serves as the con-
text for other signs. Jakobson (1956: 119) referred to this kind of semiotic
contextualization as contexture, e.g., the process by which any linguistic
unit at one and the same time serves as a context for simpler units and/or
finds its own context in a more complex linguistic unit. [ C]ombination
and contexture are two faces of the same operation. In the case of multimo-
dal messages, signs from more than one mode are selected and combined to
constitute the contexture for one another: for example, gesture combined
with speech. Such combinations may be concurrent and/or sequential: so, a
given gesture is concurrent with the simultaneously occurring words, and the
way in which gestures unfold in time (with or without speech) is an example
of sequential combination.
4. Conceptual metaphor and metonymy in gestural representations of
grammar
Offering new insights into multimodal instantiations of conceptual metaphor,
previous work on metaphoric gestures has shown that they are not random,
unsystematic hand movements, but exhibit recurrent forms and form-
meaning mappings (Bouvet 2001; Calbris 1990, 2003; Cienki 1998, 2005;
Cienki and Mller 2008; McNeill 1992, 2005; Mittelberg 2006, 2008;
Mller 1998, 2008; Nuez and Sweetser 2006; Parrill and Sweetser 2004;
Sweetser 1998; Webb 1996). For each metaphorical meaning construal, it is
necessary to determine locally whether the underlying metaphor is material-
ized in the speech and/or in the manual modality and how the relationship of
source and target domains can be defined. For example, language and ges-
ture do not necessarily exhibit the same metaphorical understanding, and
gesture and speech may be motivated by different but compatible metaphors
(see also Cienki and Mller 2008; Mller and Cienki this volume). More-
over, gesture may reveal metaphorical understandings even if the concurrent
discourse is non-metaphorical (Mittelberg 2008; Mller 2003, 2008). These
observations attest to the importance of gesture as a rich data source in cog-
nitive linguistics in general (Sweetser 2007) and for embodiment theory in
particular (Gibbs 1994, 2003, 2006; Lakoff and Johnson 1999; Taub 2001).
Much less work has been done on metonymy in gesture. The objective of
this section is two-fold. First, we will show that the interpretation of the
gestures in our corpus is anchored in metonymy. In referring to an abstract
notion, for instance, a metaphorical gesture relies on metonymic principles of
sign formation: for example, via synecdoche, the hands may depict only the
locally essential elements (parts) of the object or action (whole) in question
(cf. Bouvet 2001; Mittelberg 2006; Mller 1998). Secondly, we suggest that
due to its spontaneous and ephemeral nature co-speech gesture allows in-
sights into the dynamics of figurative thought, and our analysis contributes
to existing views of the definition of multimodal metaphor and the relation-
ship between, e.g., source and target domain (for implications regarding
static versus moving images, see Forceville 2003, 2005, 2006/this volume).

4.1 Non-metaphorical discourse and ad-hoc metaphorical visualizations in
gesture
As indicated earlier, spontaneous referential gestures tend to be polysemous
and often need contextual support to be correctly interpreted; for example,
one and the same gestural form may potentially refer to either a concrete or
an abstract entity. Depicting via metonymy contextually pertinent features of
objects or actions, referential gestures may either portray predominantly
iconic sign-object relationships (representing concrete objects or move-
ments), or they may rely on metaphorical sign-object relationships (involving
abstract entities) and thus call forth a metaphorical interpretation. For ex-
ample, a gesture with two hands may trace the frame of a painting or the
frame of a theory. In both interpretations, the gesture is synecdochic since it
provides only some aspects of the frame by rendering the parts that are
pragmatically salient in the given discourse context. When used non-
metaphorically, the synecdochic gesture can be interpreted as referring to a
spatial, physical structure (e.g., the essential panels of the frame itself, not
the other elements that hold the painting in place). In the case of a meta-
phorical interpretation, the synecdochic gesture further represents, in
Peirces (1960: 157) terms, some sort of parallelism or similarity be-
tween the form and function of a physical frame and the form and function
of an abstract frame structure (Kller 1975). Adopting a cognitivist perspec-
tive, we can say that the gesture is interpreted with respect to the metaphori-
cal concepts IDEAS ARE OBJECTS and CONCEPTUAL STRUCTURE IS GEOMET-
RIC PHYSICAL STRUCTURE (Lakoff and Johnson 1980; Sweetser 1998).
Stated in McNeills (1992) terms, such images of abstract ideas (originally
called metaphorics) represent both source and target domain information (in
this example, the gesture would be regarded as representing both the concep-
tual frame and the physical frame). However, as we will see, the metaphori-
cal gestures we will be discussing here are not directly iconic of the concrete
source domain they involve. In fact, what is common to all of the metaphori-
cal interpretations is that they rest on a first interpretation of the gesture
through metonymy: e.g., the traces in the air have to be interpreted as mean-
ing a frame of some sort; only then can the metaphorically-motivated object
be accessed.
Let us look at some examples from the data to determine how metaphor
and metonymy are manifested in gesture and how source and target domains
play out in the two modalities. The gesture represented in figure 1 is an ex-
ample of a frequently occurring form that has several potential interpreta-
tions. Looking simply at the morphology of the gesture, we see that it con-
sists of two, relatively relaxed, open hands held fairly far apart with palms
facing each other (the right hand is partly closed because it contains a piece
of chalk). If the speaker was referring linguistically to the length of a physi-
cal object such as a large box, the gesture would receive a concrete interpre-
tation through metonymy, as if the speaker were holding an elongated object
like a box between his hands. However, in this case, the speaker is referring
to a sentence and represents the sentence metonymically by the hands, which
are assumed to be marking the beginning and end of its projection in space.
The sentence is conceptualized metaphorically as bounded space or a large,
elongated object. Thus, the gesture may be said to reflect some basic meta-
phorical concepts proposed in the cognitive linguistic literature: e.g., IDEAS
ARE OBJECTS; CONTENTS ARE CONTAINERS; CATEGORIES ARE CONTAINERS;
CONCEPTUAL STUCTURE IS GEOMETRIC PHYSICAL STRUCTURE (Lakoff and
Johnson 1980; Sweetser 1998). To get to the idea of a sentence, however,
the viewer first has to take a metonymical path from the hands to the space
or the imaginary object. This is a case of external metonymy, because the
speaker is holding the imaginary object between his hands, which are exter-
nal (i.e., adjacent) to the object. And then to get from the object (or the space
extending between the hands) to the sentence, the viewer has to take a meta-
phorical path from the imaginary concrete entity (or space) to the abstract
entity (the sentence).
4

( (1 1) ) ( (s se en nt te en nc ce es s) )
. .. .. . S Se en nt te en nc ce es s, , \ \
G G1 1
p pv vo oh h- -b bh h f fa ar r a ap pa ar rt t
( (. .. .) ) [ [w wh hi il le e t th he ey y r re e m ma ad de e u up p o of f w wo or rd ds s, , _ _

G G1 1 b be ei in ng g h he el ld d
( (. .. .. .) ) a ar re en n t t m ma ad de e u up p o of f w wo or rd ds s, , \ \] ]

Figure 1. A sentence as an elongated object held (or space extending) between two
hands
It should be noted here that while the term sentence is non-metaphorical,
its gestural portrayal is first metonymical in nature and then interpreted
metaphorically. In other words, there are two interpretative moves needed to
get to the imaginary object: (1) the hands represent, via (external) meton-
ymy, the object held between them; (2) the object is a metaphorical represen-
tation of a sentence (which is a non-metaphorical linguistic expression).
The imaginary object being held is metonymically inferred through the ges-
ture itself (ACTION FOR OBJECT INVOLVED IN ACTION, Panther and
Thornburg 2004). But the underlying metaphorical mapping, involving the
target domains CONCEPTUAL STRUCTURE, IDEA, or CATEGORY and the
source domains PHYSICAL STRUCTURE, OBJECT, or CONTAINER respec-
tively, can only be inferred by a metaphorical interpretation of the meto-
nymically conveyed object. Forceville suggests that [b]y contrast to mono-
modal metaphors, multimodal metaphors are metaphors whose target and
source are each represented exclusively or predominantly in different modes
(Forceville 2006: 384). For this example, this definition holds: the target
domain (sentence) is expressed linguistically and the source domain (ob-
ject) is conveyed manually.
The gesture above recurs in the data in slight variations referring to lin-
guistic units of different degrees of complexity (words, phrases, constituents,
sentences, etc.). By contrast, single words, units below the word level, and
grammatical categories such as noun and verb are often represented by a
single hand, for example by an open hand with the flat palm turned upward,
thus forming a surface on which to present something to the addressee (see
Mller 2004 for a detailed discussion of this gesture type). According to our
analysis, the gesture is interpreted metonymically to mean that there is an
object on the hand, and then, through metaphor, that object is interpreted as
a word, a morpheme, a noun, or a verb. In example 2 (figure 2), this open
hand is combined with a closed fist. The speaker, who is talking about mor-
phological structure, illustrates the fact that the English noun teacher con-
sists of two morphemes by forming two closed fists held next to each other.
His left fist seemingly contains the lexical morpheme teach- and his right
fist, which opens up into a relaxed palm-up open hand during the demonstra-
tion, contains the grammatical morpheme -er. Although the interpretation
of the linguistic expressions relies on neither metaphor nor metonymy, the
two figures of thought again are involved in a two-step process in this semi-
otically complex instance of indirect gestural reference. Here, however, there
is no direct similarity (i.e., image iconicity) between the form of the gesture
and the objects it refers to (as in the frame example discussed above). In-
stead, the enclosed fist is interpreted metonymically as containing, and the
open hand as holding, small physical objects, e.g., LOCATION FOR OB-
JECT; ACTION FOR OBJECT INVOLVED IN ACTION; REPRESENTATION
FOR REPRESENTED (Panther and Thornburg 2004; Wilcox 2004). Thus,
the left hand serves as a CONTAINER and the right hand as a SUPPORT
structure for the imagined objects; they evoke, independently of the speech
content, these two basic image schemas (cf. Johnson 1987; Mandler 1996).
5

And in both cases, these imagined objects are metaphorically construed as
being the two morphemes (IDEAS ARE OBJECTS, Lakoff and Johnson
1980).

(2) (the teach-er)
our understanding of this is as speakers of English you know

G1.1 1.2 1.3
[that the teacher] consists [of the] [and teacher (..)]
1.4 1.5 1.6
and [teacher] [consists of teach] [and er]

1.7
[not]

1.8 1.9 1.10 1.11
[the teacher con]sists [of the] [and teach] [and er].

Without going into the theoretical views regarding the differences between
metaphor and metonymy currently debated in the cognitive linguistics litera-
ture, it should be noted that one of the received understandings holds that
whereas metaphor is based on cross-domain mappings, metonymy consists
of mappings within the same experiential domain (cf. Barcelona 2000a; see
also Croft 1993; Radden 2000).
6
In light of this domain-based definition of
metonymy, we can say that both manual actions constitute common experi-
ential domains of holding objects, and thus the gesturer can expect the view-
ers to easily relate to the action from their own experience and to build the
basis for accessing the metaphorically construed objects. According to the
two interpretative moves we introduced above, metonymy again comes first:
the gestural vehicles (e.g., the hand configurations) serve as visible meto-
nymic sources, that is, reference points (Langacker 1993); they point to
the invisible target concepts (teach- and -er, sitting in/on the hands) that
are mentioned in the concurrent discourse. These are instances of external
metonymy, since the imagined objects are adjacent to (contained in or sitting
on), but external to, the hands. The gestural form embodies the source, thus
making it perceivable and present in the immediate context and pointing to
the unperceivable target. So while the associative relation between visible
source and associated invisible target is based on conceptual contiguity, the
abstract notions are metaphorically construed as imaginary objects. Meton-
ymy is also based on the fact that the two hands positioned somewhat near
each other hold associated objects the metaphorically inferred lexical and
grammatical morphemes referred to in the speech that together make up a
word (PART FOR WHOLE, PART FOR PART, Jakobson 1956, 1963).

Figure 2. Morphemes as small objects on open hand or in closed fist
We are now in a position to define the relationship between source and target
domains more clearly. The perceivable, manual modality triggers cognitive
access to the abstract target via two interrelated mappings in which chunks
of space extended between two hands or the imagined physical objects serve
as a juncture between metonymy and metaphor. In figure 2, the imaginary
object, presented on the right hand of the speaker, serves as both the target
of the metonymic mapping (the hand stands for the object resting on it) and
the source of the metaphoric mapping (a morpheme is a small object). The
same holds for the gesture shown in figure 1, in which the speaker seems to
be holding a large object between his two hands. Again, the hands (meto-
nymic source) point to the object or the space (metonymic target) extending
between them; or, put differently, the action of holding an object (metonymic
source) stands for the object (metonymic target) itself. Thereby, the meto-
nymically accessed imaginary object or, the chunk of space extending be-
tween the two hands is both the metonymic target and the metaphoric
source, since it stands for the sentence (metaphoric target).
The assumption that some metaphors are grounded in metonymy (Barce-
lona 2000a: 33; Geeraerts 2002; Goossens 1995: 171; Jakobson 1956, 1960;
Lodge 1977: 111; Radden 2000: 93) holds in all the examples in our corpus.
Whereas in these cases the speech itself is for the most part non-figurative
(i.e., consists of technical grammatical terminology), figurative principles
guide the interpretation of these dynamic multimodal representations: first
metonymy and then metaphor contribute to the meaning-making processes
linking the manual modality to cognitive processes of association and imagi-
nation via contiguity and similarity.
As we hope has become clear in our discussion, in order to interpret these
spontaneous expressions conveyed through the manual modality, we rely on
our embodied cognitive and sensory-motor abilities and schemata to see and
feel the contiguity between the hands and the objects they seem to manipu-
late (or the hands and different amounts of space extending between them),
which stand metaphorically for the ideas concurrently mentioned in the dis-
course. In other words, in order to arrive at the meaning of these gestures,
the viewer can be assumed to perform an act of pragmatic inferencing (see
work on metonymy in language by Panther and Thornburg 2003, 2004). To
conclude this section, what is important to note here is that although, in the
examples discussed so far, the concurrent speech is non-metaphorical (sen-
tence, teach-, er), the gesture depicts the image schema (OBJECT) un-
derlying the metaphorical projection (IDEAS ARE OBJECTS). The bodily
modality thus spontaneously and dynamically expresses a metaphorical un-
derstanding of abstract entities as imaginary graspable objects. In the next
section, we will examine multimodal representations of more elaborate theo-
retical constructs based on a well-defined set of conventionalized metaphors.

4.2. Metaphorical discourse and theory-based metaphorical visualizations
Having discussed some of the most basic gestural forms that recur in the
data, we will now turn to more complex multimodal representations of syn-
tactic structure that are based on a specific model of linguistic structure,
namely generative grammar. In these cases, there are ready-made metaphori-
cal visualizations provided by the theory (tree structure diagrams to depict
syntactic structure) that can then be referred to by the gestures. For example,
when explaining dependent clauses in English, a speaker employing this
framework used the right hand to sketch a branch of a tree structure diagram
extending toward the lower right of her body. Figure 3 shows such a diago-
nally descending movement that is meant to represent an embedded clause.
The speaker illustrates the idea of subordination (G1) by repeatedly moving
her right hand first up to eye-level and then downward to her right side,
thereby making a wave-like movement by tilting the hand from side to side.
This can be assumed to roughly sketch out, through synecdoche, an elabo-
rated tree structure, which is a diagrammatic metaphor used in generative
grammar for the structure of complex sentences with subordinate clauses.
More importantly, such tree structures are used in linguistic textbooks for
learners and in research articles by scholars and in the case discussed here,
there is a tree diagram behind the speaker on the white board. The fact that
the speaker (who is left-handed) is talking about English, a right-branching
language, may motivate the use of her right hand for the gesture even fur-
ther.

(3) (wavy embedded clauses)

G1 (G1 repeated) (G1 repeated)
rh diagonal wavy line from head downward to the right

[but this is gonna be another one with embedded sentences

(G1 repeated) G2
rh extended arm and index finger
point toward ground
coming in verb phrases] [all the way down].

Figure 3. An embedded clause as a wavy diagonal line
The gesture is synecdochic (i.e., an instance of internal metonymy), but its
proper interpretation entails a metaphorical interpretation of the metonymi-
cally given object, which is then inferred to be the same as the metaphors in
the textbook or on the board. In the next gesture in the same utterance (G2)
the speaker uses in her speech the metaphor (also tied to the tree diagram)
all the way down to indicate the fact that in certain cases embedded sen-
tences may continue almost without stopping. At this point the speaker ex-
tends her right arm towards the floor and points with her index finger
straight to the ground in a deictic gesture (for a Peircean approach to deictic
gestures see Fricke 2007). Without the background knowledge of the theory
and its canonical metaphors and diagrams, the gestures could not be inter-
preted correctly. This is different from the more intuitive examples in section
4.1, in which the speech was non-metaphorical (sentence, morphemes) and
the gesture rendered a metaphorical understanding of abstract entities as
objects or chunks of space without any ready-made visualization to fall back
on.
Since linguistic theories are often built on many specific metaphors, in-
teractions of more than one metaphorical understanding can also be observed
in the data. The subordination gesture (G1) in example (3) represents, as we
just saw, the notion of embedded sentence (mentioned in the concurrent
speech) as a wavy line descending in a diagonal toward the floor. Subordi-
nated (embedded) entities are thought of in generative grammar as below the
ones that dominate them. This indicates that the theory the speaker has in
mind when talking about sentence structure motivates the form the gestures
take. Moreover, the theory of syntactic structure proposed within generative
grammar rests on a combination of spatial metaphors (i.e., the tree diagram)
and power relations (i.e., dominance, control, etc.). The question that arises
is whether, and if so, how, these two different source domains are made
manifest in the verbal and/or manual modalities.
In the example above, relations of dominance are not alluded to linguisti-
cally, but let us look at another sequence where the same speaker makes
reference to the idea of dominance in the speech modality. Just as in figure 3,
the gesture derives its meaning from the movement and the virtual traces left
in the air. As shown in figure 4, the speaker draws a tree chunk in the form
of a triangle in the air, with both hands starting out at the center top (the
node) and then tracing diagonals outward and downward to either side of the
body. The gesture is a synecdochic depiction that is metaphorically inter-
preted as meaning several technical terms (nodes alpha and delta, domina-
tion, and branching). Outside of this theoretical model, these terms do not
necessarily entail spatial relationships, or if they do, then they might not be
represented in exactly the same way (e.g., branching does not necessarily
have a downward orientation). An interesting moment occurs when the
speaker realizes that she was talking about a node dominating elements
without actually having introduced the idea of dominance. In the speech
modality, she quickly changes from the hierarchical understanding of domi-
nance back to the spatial tree metaphor involving a node being on top of
two things. Thus, the speech here is metaphorical in two compatible ways;
compatible because spatial and social hierarchies both draw on spatial rela-
tions such as UP and DOWN, with certain values attached to each location in
the corresponding system (e.g., POWER IS UP, see Lakoff and Johnson
1980). Although in the speech modality there is, for a moment, a slight hesi-
tation about which metaphor to use, the gesture modality consistently and
repeatedly represents the spatial features of the tree model and thus is moti-
vated by the corresponding spatial metaphor, which is a conventional part of
the theory.

(4) (branching, domination)
No=des, /
alpha and delta, /

G1 bh, branch triangle, branching movement x2
[branch, \ (...7)]
okay? /
so thats a technical term, _

G2 bh, branch triangle, branching movement x2
(..) [when the nodes]--, /

G3 rh branch G4 bh triangle, branching movement x2
[a node] [dominates--], /

(..) woops I said a technical term too soon,

G5 rh draws triangle G6 bh triangle branching
(..) when [the node is on top] [of two things] or more, /

(..) it branches. \

Figure 4 represents (1) the very beginning of the branching gesture (hands
are joined at the top, the node, of the triangle) and (2) the repeated down-
ward movement that reinforces the idea of an active branching process.
As for the underlying conceptual metaphors, these gestural diagrams all
reflect the metaphor SYNTACTIC STRUCTURES ARE GEOMETRIC PHYSICAL
STRUCTURES, based on CONCEPTUAL STRUCTURE IS GEOMETRIC PHYSICAL
STRUCTURE (Sweetser 1998) discussed above. Given its specific semiotic
affordances, gesture, a semiotic system exploiting space, provides a spatial
projection of compatible metaphors stemming from the domains of physical
structures and social hierarchies with a built in up-down orientation
(POWER/HIGH STATUS IS UP; HAVING CONTROL IS UP; BEING SUBJECT TO
CONTROL IS DOWN; LOW STATUS IS DOWN; Lakoff and Johnson 1980;
Sweetser 1998). Following canonical tree diagrams, these gestures depict
logical relations between entities (which is one of the central functions of
diagrams; see Peirce 1960; Mittelberg 2008; Waugh et al. 2004). While
within the domain of meta-grammatical discourse dominance is indeed
used metaphorically, in the manual modality the theory-driven spatial con-
ceptualization of linguistic structure seems to be the predominant, overriding
metaphorical understanding that motivates the representation of the behavior
and relationship of elements in a sentence. As in the examples discussed in
section 4.1, we can also discern a double mapping here: (1) there is external
metonymy between the hand (metonymical source) and the trace left in the
air (metonymical target), whereby the hand and the line drawn are part of the
same experiential domain of drawing a tree structure (whether it is on paper,
on a blackboard, or in the air); and (2) through metaphorical projection these
spatial tree structures depicted by the gesture (metaphorical source) simulta-
neously represent the abstract conceptual structure (metaphorical target).
The difference between the examples discussed in this section and section
4.1 is that whereas in 4.1 the speech is non-metaphorical, here it is meta-
phorical and the metaphors referred to are associated with theory-based ca-
nonical visualizations of abstract structure that can be easily mimicked by
gesture. Also, the speaker had probably drawn many such tree diagrams on
blackboards or paper before depicting them gesturally.

Figure 4. Sentence structure as a laterally branching tree chunk
5. Concluding remarks: Cross-modally achieved, intertwined figures of
thought
What all the examples in this paper show is that whether the metaphorical
interpretation of the metonymic gesture is simple and easily accessible (e.g.,
a sentence, morphemes) or complex and only understandable in the context
of a given linguistic theory (e.g., a tree structure, branching, dominance),
they still adhere to the general principle of metonymy first, metaphor second,
or to say it in another way, metonymy, whether external (adjacency/contact)
or internal (synecdoche), leads the way into metaphor. Due to the abstract
nature of the subject matter in linguistics courses, the objects in question are
conceptualized via metaphor. But metonymy is needed to access the meta-
phor: e.g., external metonymy (contiguity through adjacency/contact) be-
tween, for example, the fist and the small object it seems to enclose, or be-
tween the hand drawing a line in the air and the imaginary trace that this
movement leaves behind. In the latter case, there is also synecdoche between
the diagram on the blackboard, for instance, and the sketchy hand movement
representing it. Using the terms of contemporary metonymy theory, we have
also claimed that the hands and the actions they perform constitute a com-
mon experiential domain and that the imaginary objects or traces are prag-
matically inferred from the performed actions.
Taking the material side of gestures as a point of departure, we thus iden-
tified associative processes involving two intertwined mappings leading from
the form of the gesture to the metaphorically construed entity it stands for.
The metonymic mapping functions as follows: in the case of the closed fist
that co-occurs with the mention of the morpheme teach, the perceivable
fist serves as a metonymic source triggering cognitive access to the imagi-
nary object inside of it, that is, the metonymic target. In the ensuing meta-
phoric process, the metonymic target, i.e., the object, becomes the meta-
phoric source that is mapped onto the metaphoric target, that is, the
linguistic unit teach. Both of these figurative, multimodally achieved ef-
forts are needed to make abstract entities and conceptual structures visible
(e.g., when the teacher is unable to point to words or diagrams written on the
board), thus grounding them in the immediate teaching context and making
them graspable for the student audience. In order to arrive at what is referred
to, the addressee of these dynamic multimodal representations needs to inter-
pret a combination of not only speech and gesture, but also metonymy and
metaphor, in that order.
We also saw that metaphorically motivated gestural forms do not always
coincide with metaphorical speech. Whereas in the examples of the sen-
tence or the morpheme teach-, discussed in section 4.1, the speech is
technical rather than metaphorical, the gestural illustrations are metaphori-
cally motivated, featuring imaginary physical objects or assigning meaning
to chunks of space extending between the manual articulators. These ad-hoc
gestural metaphors stand in contrast to the sequences discussed in section
4.2, in which the speech is metaphorical; however, the metaphors used in the
speech are given by the theory of generative grammar, which also provides
conventional ways of diagramming syntactic structure in the form of inverse
tree diagrams. Accordingly, the gestures that depict aspects of embedded
sentences or dominance are more or less sketchy (i.e., synecdochic) rendi-
tions of those ready-made visualizations.
In the light of the importance that those who work on multimodal mani-
festations of figurative thought place on the specific materiality and logic of
each modality (e.g., Cienki and Mller 2008; Mller and Cienki, this vol-
ume; Forceville 2006; Kress and van Leeuwen 2006; Kress et al. 2001; Mit-
telberg 2002, 2006), it is interesting to realize that making sense out of what
a speaker-gesturer is trying to convey involves our imaginative abilities as
much as our visual and auditory senses. Interpreting gestures entails combin-
ing perceivable visual and verbal materialized information; but the manual
configurations and movements also appeal to our capacity during the process
of interpretation to assign meaning to empty space and to fill in missing in-
formation, for example, when inferring objects and actions from gestures
involving closed fists, open hands, or lines drawn in the air.
In the multimodal manifestations of metaphor and metonymy examined
above, source and target meanings are not always neatly distributed across
the two modalities (see Forceville 2006), and gesture may be the only modal-
ity in which the metaphor is expressed (especially when it is spatial meta-
phor). Of course, there are also instances in the data in which the speech is
metaphorical but there is no gesture. As we saw, source and target domains
of a mapping are not necessarily co-present in a given instance of multimo-
dal representation: they may need to be inferred by interpretative hypotheses
(Peirce 1991, 1992, 1998) from the discourse and/or physical context
(neighborhood/contexture; Jakobson 1956), or the knowledge of the linguis-
tic theory talked about.
Since gesture is a largely unconscious, spontaneous means of expression,
the multimodal metaphors discussed here can hardly be compared with
elaborated and consciously chosen metaphorical messages in cartoons or
advertisements (see Forceville 1996, 2002, 2005; El Refaie 2003, this vol-
ume; Yus, this volume; Schilperoord and Maes, this volume). And the ques-
tion of whether the linguistic explanations and the linguistically expressed
metaphors could be recognized and understood by the audience without the
gestural support is not answerable on the basis of our data. However, the
interplay between metaphor and metonymy deserves, as has been shown
already (e.g., Bouvet 2001; Forceville 2005; Gibbs 1994; Whittock 1990), a
more detailed scrutiny in other forms of multimodal communication. The
Jakobsonian (and Peircean) notions, combined with contemporary cognitivist
approaches, are a way to account for not only the materialized dimensions of
figures of thought motivating multimodal discourse, but also for their cogni-
tive and imaginative dimensions.
Acknowledgements
We wish to thank the editors of this volume, Charles Forceville and Eduardo
Urios-Aparisi, as well as Alan Cienki and Cornelia Mller for insightful com-
ments on an earlier version of this chapter. We are also grateful to Allegra Gio-
vine, Joel Ossher, and Daniel Steinberg for their help with data coding and to
Yoriko Dixon for providing the artwork.
Notes
1. This is very similar to Barthes concept of anchoring (see for discussion
Forceville 1996: 71).
2. For reasons of space, the functions fulfilled by the additional modalities and
artifacts in the class environment and by the teachers facial expressions or
gaze cannot be included in the analysis here (see Kress et al. [2001] and
Ochs et al. [1996] for work on multimodality in the science classroom).
3. See Furuyama (2001) regarding Jakobsons concept of the poetic function in
gesture.
4. The abbreviations used in the transcript are to be read as follows: puoh
stands for palm-up open hand, pvoh stands for palms vertical open
hand, bh stands for both hands. As for gesture-speech synchrony, the
speech segments that coincide with a gesture are set off by square brackets,
speech segments highlighted in bold face represent the gesture stroke (the
peak of a gestural expression), and underlined speech segments indicate a
post-stroke gesture hold. G1 in example 1 stands for Gesture 1. For more
details see Mittelberg (2006, 2007).
5. According to Johnson (1987: xiv), an image schema is defined as a recur-
ring, dynamic, pattern of our perceptual interactions and motor programs
that gives coherence and structure to our experience.

See Mittelberg (2006,
forthcoming) for a complete list of the image-schematic and geometric sche-
mata that emerged from the data.
6. Barcelona (2000a: 4) gives the following definition: Metonymy is a concep-
tual projection whereby one experiential domain (the target) is partially un-
derstood in terms of another experiential domain (the source) included in the
same common experiential domain (italics in original).
References
Barcelona, Antonio
2000a The cognitive theory of metaphor and metonymy. In Metaphor and
Metonymy at the Crossroads: A Cognitive Perspective, Antonio
Barcelona (ed.), 128. Berlin/New York: Mouton de Gruyter.
2000b Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
Bradford, Richard
1994 Roman Jakobson: Life, Language, Art. London/New York:
Routledge.
Bouvet, Danielle
2001 La Dimension Corporelle de la Parole. Les Marques Posturo-
Mimo-Gestuelles de la Parole, leurs Aspects Mtonymiques et M-
taphoriques, et leur Rle au Cours dun Rcit. Paris: Peeters.
Calbris, Genevieve
1990 The Semiotics of French Gestures. Bloomington: University of Indi-
ana Press.
2003 From cutting an object to a clear cut analysis: Gesture as the repre-
sentation of a preconceptual schema linking concrete actions to ab-
stract notions. Gesture 3 (1): 1946.
Carroll, Noel
1994 Visual metaphor. In Aspects of Metaphor, J. Hintikka (ed.), 89218.
Dordrecht: Kluwers.
Cienki, Alan
Jean-Pierre Koenig (ed.), 189204. Stanford: CSLI Publications.
2005 Image schemas and gesture. In From Perception to Meaning: Image
Schemas in Cognitive Linguistics, Beate Hampe (ed.), in coopera-
tion with Joseph Grady, 421442. Berlin/New York: Mouton de
Gruyter.
2008 Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins.
Croft, William
1993 The role of domains in the interpretation of metaphor and meton-
ymy. Cognitive Linguistics 4: 335370.
Danaher, David
1998 Peirces semiotic and Cognitive Metaphor Theory. Semiotica 119
(12): 171207.
Dirven, Ren
2002 Metonymy and metaphor: Different mental strategies of conceptu-
alisation. In Metaphor and Metonymy in Comparison and Contrast,
Ren Dirven and Ralf Prings (eds.), 75111. Berlin/New York:
Mouton de Gruyter,
Forceville, Charles
2002 The identification of target and source in pictorial metaphors.
2003 Bildliche und multimodale Metaphern in Werbespots. Zeitschrift fr
Semiotik 25 (12): 3960.
Fricke, Ellen
2007 Origo, Geste und Raum Lokaldeixis im Deutschen. Berlin/New
Furuyama, Nobuhiro
2001 De-syntacticizing the theories of reference maintenance from the
viewpoint of poetic function of language and gesture: A case of
Japanese discourse. Ph.D. diss., University of Chicago.
Geeraerts, Dirk
2002 The interplay of metaphor and metonymy in composite expressions.
In Metaphor and Metonymy in Comparison and Contrast, Ren
Dirven and Ralf Prings (eds.), 435465. Berlin/New York: Mouton
de Gruyter.
1999 Speaking and thinking with metonymy. In Metonymy in Language
and Thought, Karl-Uwe Panther and Gnter Radden (eds.), 6176.
Amsterdam/Philadelphia: Benjamins.
2003 Embodied experience and linguistic meaning. Brain and Language
84: 115.
versity Press.
Goldin-Meadow, Susan
2003 Hearing Gesture: How Hands Help Us Think. Cambridge,
MA/London: The Belknap Press of Harvard University Press.
Goodwin, Charles
2003 The semiotic body in its environment. In Discourses of the Body,
Justine Coupland and Richard Gwyn (eds.), 1942. Basing-
stoke/New York: Palgrave-Macmillan.
Goossens, Louis
1995 Metaphtonymy: The interaction of metaphor and metonymy in figu-
rative expressions for linguistic action. In By Word of Mouth: Meta-
phor, Metonymy and Linguistic Action in a Cognitive Perspective,
Goossens, Louis, Paul Pauwels, Barbara Rudzka-Ostyn, A.-S.
Simon-Vandenbergen, and Jan Vanparys (eds.), 159174. Amster-
Hiraga, Masako
1994 Diagrams and metaphors: Iconic aspects in language. Journal of
Pragmatics 22: 521.
2005 Metaphor and Iconicity: A Cognitive Approach to Analysing Texts.
Basingstoke/New York: Palgrave-MacMillan.
Jakobson, Roman
1953 Results of a joint conference of anthropologists and linguists. In
Selected Writings II: Word and Language, 554567.
1956 Two aspects of language and two types of aphasic disturbances. In
On Language, Linda Waugh, and Monique Monville-Burston (eds.),
115133.
1960 Linguistics and poetics. In Language in Literature, Krystyna Po-
morska, and Stephen Rudy (eds.), 6294.
1963 Parts and wholes in language. In On Language, Linda Waugh and
Monique Monville-Burston (eds.), 110114.
1966 Quest for the essence of language. In On Language, Linda Waugh,
and Monique Monville-Burston (eds.), 407421.
1971 Selected Writings II: Word and Language. The Hague: Mouton.
1987a On the relation between auditory and visual signs. In Language in
Literature, Krystyna Pomorska and Stephen Rudy (eds.), 467473.
1987b Language in Literature, Krystyna Pomorska and Stephen Rudy
(eds.). Cambridge, MA: Harvard University Press.
Jakobson, Roman, and Krystyna Pomorska
1983 Dialogues. Cambridge, MA: MIT Press.
Johnson, Mark
Kendon, Adam
2000 Language and gesture: unity or duality? In Language and Gesture,
David McNeill (ed.), 4763. Cambridge: Cambridge University
Press.
versity Press.
Kennedy, John M.
Kller, Wilhelm
1975 Semiotik und Metapher. Untersuchungen zur grammatischen
Struktur und kommunikativen Funktion von Metaphern. Stuttgart:
Metzler.
Kvecses, Zoltn
Press.
Kress, Gunter, and van Leeuwen, Theo
2006 Reading Images: The Grammar of Visual Design. 2nd ed. Lon-
Kress, Gunter, Carey Jewitt, Jon Ogborn, and Charalampos Tsatsarelis
2001 Multimodal Teaching and Learning: The Rhetorics of the Science
Classroom. London/New York: Continuum.
Lakoff, George
About the Mind. Chicago: University of Chicago Press.
University Press.
Langacker, Ronald W.
1993 Reference-point constructions. Cognitive Linguistics 4: 138.
Lodge, David
1977 Two Modes of Modern Writing: Metaphor, Metonymy, and the Ty-
pology of Modern Literature. Ithaca, NY: Cornell University Press.
Mandler, Jean
1996 Preverbal representation and language. In Language and Space,
Paul Bloom, Mary A. Peterson, Lynn Nadel, and Merrill F. Garrett
(eds.), 365384. Cambridge, MA: MIT Press.
McNeill, David
Chicago University Press.
2005 Gesture and Thought. Chicago: Chicago University Press.
McNeill, David (ed.)
2000 Language and Gesture. Cambridge: Cambridge University Press.
McNeill, David, Francis Quek, Karl-Eric McCullough, Susan D. Duncan, Nobu-
hiro Furuyama, Rolbert Bryll, and Rashid Ansari
2001 Catchments, prosody and discourse. Gesture 1 (1): 933.
Mittelberg, Irene
2002 The visual memory of grammar: Iconographical and metaphorical
insights. Metaphorik.de, 02/2002: 6989.
2006 Metaphor and metonymy in language and gesture: Discourse evi-
dence for multimodal models of grammar. Ph.D. Diss., Cornell Uni-
versity.
2007 Methodology for multimodality: One way of working with speech
and gesture data. In Methods in Cognitive Linguistics, Monica Gon-
zalez-Marquez, Irene Mittelberg, Seana Coulson, and Michael
Spivey (eds.), 225248. Amsterdam/Philadelphia: Benjamins.
2008 Peircean semiotics meets conceptual metaphor: Iconic modes in
gestural representations of grammar. In Metaphor and Gesture,
Alan Cienki and Cornelia Mller (eds.), 115154. Amster-
forthc. Geometric and image-schematic patterns in gesture space. In Lan-
guage, Cognition, and Space: The State of the Art and New Direc-
tions, Vyvyan Evans and Paul Chilton (eds.). London: Equinox.
Mller, Cornelia
1998 Redebegleitende Gesten. Kulturgeschichte Theorie Sprachver-
gleich. Berlin: Berlin Verlag A. Spitz.
2003 Gesten als Lebenszeichen toter Metaphern. Zeitschrift fr
Semiotik 25 (12): 6172.
2004 Forms and uses of the Palm Up Open Hand: A case of a gesture
family? In The Semantics and Pragmatics of Everyday Gesture: The
Berlin Conference, Cornelia Mller and Roland Posner (eds.), 233
256. Berlin: Weidler.
2008 Metaphors. Dead and Alive, Sleeping and Waking. A Cognitive
Approach to Metaphors in Language Use. Chicago: Chicago Uni-
versity Press.
this vol. Words, gestures, and beyond: Forms of multimodal metaphor in the
use of spoken language.
Nez, Rafael, and Eve E. Sweetser
2006 Aymara, where the future is behind you: Convergent evidence from
language and gesture in the cross-linguistic comparison of spatial
construals of time. Cognitive Science 30: 149.
Ochs, Eleanor, Patrick Gonzalez, and Sally Jacoby
1996 When I come down Im in the domain state: Grammar and
graphic representation in the interpretive activity of physics. In In-
teraction and Grammar, Eleanor Ochs, Emanuel Schegloff, and
Sandra Thompson (eds.), 329369. Cambridge: Cambridge Univer-
sity Press.
Panther, Klaus-Uwe, and Gnter Radden (eds.)
Benjamins.
Panther, Klaus-Uwe, and Linda L. Thornburg, (eds.)
2003 Metonymy and Pragmatic Inferencing. Amsterdam/Philadelphia:
Benjamins.
2004 The role of conceptual metonymy in meaning construction. Meta-
phorik.de 06/2004: 91113.
Parrill, Fey, and Eve E. Sweetser
2004 What we mean by meaning: Conceptual integration in gesture
analysis and transcription. Gesture 4 (2): 197219.
Peirce, Charles Sanders
1960 Collected Papers of Charles Sanders Peirce (19311958), vol. I:
Principles of Philosophy, vol. II: Elements of Logic, Charles Harts-
horne and Paul Weiss (eds.). Cambridge: The Belknap Press of Har-
vard University Press.
1991 Peirce on Signs: Writings on Semiotic, James Hoopes (ed.). Chapel
Hill: University of North Carolina Press.
1992 The Essential Peirce: Selected Philosophical Writings, vol. 1 (1867-
1893), Nathan Houser and Christian Kloesel (eds.). Indiana: Indi-
ana University Press.
1998 The Essential Peirce: Selected Philosophical Writings, vol. 2 (1893-
1913), Nathan Houser and Christian Kloesel (eds.). Indiana: Indi-
ana University Press.
Radden, Gnter
2000 How metonymic are metaphors? In Metaphor and Metonymy at the
Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 93
Schilperoord, Joost, and Alfons Maes
Shapiro, Michael
1983 The Sense of Grammar: Language as Semeiotic. Indiana: Indiana
University Press.
Sweetser, Eve
1998 Regular Metaphoricity in Gesture: bodily-based models of speech
interaction. Actes du 16e Congrs International des Linguistes (CD-
ROM), Amsterdam: Elsevier.
2007 Looking at space to study mental spaces: Co-speech gesture as a
crucial data source in cognitive linguistics. In Methods in Cognitive
Linguistics, Monica Gonzalez-Marquez, Irene Mittelberg, Seana
Coulson and Michael Spivey (eds.), 201224. Amsterdam/Phila-
delphia: Benjamins.
Streeck, Jrgen
2002 A body and its gestures. Gesture 2 (1): 1944.
Taub, Sarah
2001 Language from the Body: Iconicity and Metaphor in American Sign
Language. Cambridge: Cambridge University Press.
Turner, Mark, and Gilles Fauconnier
2002 Metaphor, metonymy, and binding. In Metaphor and Metonymy in
Comparison and Contrast, Ren Dirven and Ralf Prings (eds.),
Wales, Katie
2001 A Dictionary of Stylistics, 2nd ed. London: Longman.
Waugh, Linda R.
1998 Semiotics and language: The work of Roman Jakobson. In Hi Fives:
A Trip to Semiotics, Roberta Kevelson (ed.), 85102. New York: Pe-
ter Lang.
2000 Jakobson, Roman (18961982). In Concise Encyclopedia of Prag-
matics, Jacob Mey (ed.) Oxford: Pergamon.
Waugh, Linda R., and Monique Monville-Burston
1990 Roman Jakobson: His life, work and influence. In Jakobson on
Language, Linda R. Waugh and Monique Monville-Burston (eds.),
145. Cambridge, MA: Harvard University Press. Updated version
published as 2002, Introduction to Roman Jakobson: Selected Writ-
ings, I, third edition, v-xcviii.
Waugh, Linda R., Norma Barletta, Susan Smith, Elizabeth Specker, Shawn
Steinhart, and Jue Wang
2004 Peircean theory, diagrammatic iconicity and academic texts: Global
structure, abstracts, and the role of narrative. Logos and Language.
Journal of General Linguistics and Language Theory 6: 3962
(special issue on Aspects of Iconicity in Contemporary Linguistics).
Webb, Rebecca
1996 Linguistic Features of Metaphoric Gestures. Ph.D. Diss., University
of Rochester.
Whittock, Trevor
Wilcox, Phyllis P.
2004 A cognitive key: Metonymic and metaphorical mappings in ASL.
Cognitive Linguistics 15 (2): 197222.
Wilcox, Sherman, and Jill Morford
2007 Empirical methods in signed language research. In Methods in Cog-
nitive Linguistics, Monica Gonzalez-Marquez, Irene Mittelberg, Se-
ana Coulson and Michael Spivey (eds.), 171200. Amsterdam/

VI

Metaphor Involving Music and Sound

Chapter 15
Music, language, and multimodal metaphor
Lawrence M. Zbikowski
Abstract
This chapter considers the topic of multimodal metaphor from the perspective of
cross-domain mappings between the musical and the linguistic domains. Begin-
ning with an example of what musicans call text painting (in which music is
used to paint an image related to the text of a vocal work), I explore the differ-
ent ways music and language structure thought. Examples of musical passages
from Palestrina, Biber, Bach, Schubert, and Jerome Kern are used to demonstrate
how music contributes to meaning construction and thus may serve as a source
domain for a multimodal metaphor. I conclude with a brief discussion of how
conceptual blending theory can be used for the analysis of text-music relations,
and the multimodal metaphors that may result.

Keywords: music, text-painting, conceptual blending, popular song, musical
grammar, Bach, Schubert
1. Introduction
Giving voice to an idea that took a number of forms in his later work,
Ludwig Wittgenstein, near the end of his Philosophical Investigations,
wrote, Understanding a sentence is much more akin to understanding a
theme in music than one may think (Wittgenstein 2001: frag. 527). The
close relationship between language and music suggested by Wittgensteins
observation is borne out by similarities between the two: both are unique to
the human species, both unfold over time, both have syntactic properties,
and both make use of sound. There are also, of course, notable differences:
musical meaning is on the whole much less precise than linguistic meaning;
music often involves simultaneous events, where language does not; and
there is more of a sense of play in ordinary music than there is in ordinary
360 Lawrence M. Zbikowski
language. On the one hand, these differences suggest that language and mu-
sic belong to two different conceptual domains. On the other hand, the simi-
larities between the two suggest that language and music may recruit some
of the same cognitive resources, and that structure from one domain may be
readily mapped to the other to create meaning. Understanding a sentence is
like understanding a musical theme because both language and music offer
possibilities for constructing meaning, possibilities that can be exploited
through multimodal metaphors.
As an example, consider the passage from the Credo of Giovanni Pier-
luigi da Palestrinas Pope Marcellus Mass (printed 1567) given in figure 1.
1

The text Palestrina sets here is Qui propter nos homines et propter nostram
salutem descendit de clis (Who for us men, and for our salvation, came
down from heaven). With the first statement of the word descendit, each
voice begins a scalar descent. Christs descent from heaven is thus repre-
sented with a cascading fall through musical space, a series of overlapping
movements down the musical scale. This representation exploits the com-
mon construal of musical pitches as situated in vertical space, a construal
that follows from the characterization of pitches as high or low with
respect to one another.

Figure 1. Giovanni Pierluigi da Palestrina, Credo of the Pope Marcellus Mass,
mm. 53-58.
Music, language, and multimodal metaphor 361
Although this characterization seems quite natural, it is actually rather arbi-
trary. For instance, when you play higher notes on the piano, you move
your hand to the right; when you play higher notes on a cello, you move
your hand down toward the ground.
2
Describing musical pitches in terms of
high and low is in fact a product of mapping structure from one domain
of knowledge (relationships among points in vertical space) onto another
(relationships among musical pitches). From the perspective provided by the
contemporary theory of metaphor (Lakoff 1993), this mapping relies on the
conceptual metaphor PITCH RELATIONSHIPS ARE RELATIONSHIPS IN PHYSI-
CAL SPACE. Language, of course, facilitates this mapping. In that language
belongs to one mode (that of either spoken or written signs) and music to
another, the metaphorical description of pitches as high or low is multi-
modal in at least a minimal sense. If, however, we take a closer look at Pal-
estrinas compositional choices, a somewhat more textured view of multimo-
dal metaphors that involve music emerges. Were Palestrina interested only in
portraying a move from high to low, he could have used a single falling
interval rather than his stepwise descending scale. Using the resources of-
fered by six voices, he also reinforced and inflected his sonic image of de-
scent through a series of multiple, and subtly different, descending gestures,
all of which culminate in the important cadential arrival of measure 58.
What Palestrina gives us, then, is the sound of descent, realized as an or-
derly, stately process. Contrast his setting of descendit with that of
Heinrich Biber at the corresponding point in the Credo from Bibers Missa
Christi resurgentis, written a little over a hundred years after Palestrinas
mass. As shown in figure 2, the path sketched by Biber is much more in-
volved, comprising fifteen notes and proceeding through a series of twisting
turns before reaching its goal. Palestrina, for his part, takes only eight notes,
and never changes direction.
3
The sonic image of descent offered by Biber is
consequently quite different than that offered by Palestrina. Bibers descent

Figure 2. Comparison of melodic passages from Heinrich Biber, Credo of the
Missa Christi resurgentis, mm. 51-57 and Giovanni Pierluigi da Pal-
estrina, Credo of the Pope Marcellus Mass, mm. 53-55.
is a leisurely, almost caressing, affair in which the journey is at least as im-
portant as the goal. Indeed, when Biber approaches the cadence that articu-
lates the end of this section of the Credo he leaves behind the melodic mate-
rial of figure 2 and adopts a much more proclamatory style, effectively
bracketing off the sensation of descent from its goal.
Palestrinas and Bibers settings of descendit demonstrate, in a concise
way, the different resources offered by language and music for the construc-
tion of multimodal metaphors. Language gives access to a rich network of
conceptual frameworks. High and low, for instance, are used not only
for orientation in physical space, but also as evaluative terms: Christs de-
scent is from heaven, where everything is good, to earth, where it is not, a
perspective that relies on the conceptual metaphor STATE OF BEING IS ORI-
ENTATION IN VERTICAL SPACE.
4
As another example of the conceptual
frameworks to which language offers access, the continuum of vertical space
can be divided up into a series of points with distances between them. When
these points are mapped onto musical pitches (through the conceptual meta-
phor PITCH RELATIONSHIPS ARE RELATIONSHIPS IN PHYSICAL SPACE) it be-
comes possible to measure the distance between any two notes. I would
emphasize that neither of these conceptual frameworks has immediate rele-
vance for Palestrinas or Bibers setting of descendit, yet both are accessi-
ble through the conceptual networks to which the high and low of musi-
cal space is connected. Music, for its part, infuses the process of meaning
construction with a crucial dynamic aspect. It is not simply the concept of
descent through vertical space that we draw from the musical domains set up
by Palestrina and Biber but particular kinds of descent, each with its own
texture and shape. Where language provides us with only the bare prompt of
the word descendit, a notion we fill out with our own experiences with the
process of descent, music actively sketches different sorts of descents
through carefully arranged sonic materials.
In this chapter I want to explore in greater detail multimodal metaphors
that involve language and music from both theoretical and practical perspec-
tives. The theoretical perspective, which is developed in the first main sec-
tion that follows, concerns the different functions of language and music in
human culture. I should note that in this section, and indeed throughout this
chapter, I shall be concerned with the ways musical materials actually give
rise to meaning rather than the meanings with which they may become asso-
ciated.
5
The practical perspective, which is developed in the second main
section, begins with a treatment of my opening examples in terms of the
theoretical perspective for which I have argued and expands the discussion
to consider relationships between text and music in greater detail. In a con-
cluding section I shall return to Wittgensteins observations about under-
standing in language and music, and consider what multimodal metaphors
can tell us about how we understand language and music.
2. Function and structure in language and music
As was noted above, there are numerous similarities between language and
music, and there are any number of cultural practices that blur the bounda-
ries between the two modes of communication (Boiles 1967; Forns 2003).
It is still the case, however, that we do not live our lives inside the equivalent
of a grand opera or Broadway musical, with every utterance sung and every
action accompanied by an orchestra. So why is it that human cultures have
developed both language and music? Although there is any number of ways
to approach this question, I take the position that language and music have
different functions within human culture. In doing so I am influenced by the
work of the developmental psychologist Michael Tomasello, who situates the
emergence of language in our species within the broader development of
human culture. In Tomasellos view, the primary function of language is to
direct the attention of another person to objects or concepts within a shared
referential frame (Tomasello 1999: chap. 5). I would argue that music is
similarly part of a cultural framework unique to our species, but one whose
primary function is to represent through patterned sound various dynamic
processes that are common in human experience. Chief among these dy-
namic processes are those associated with the emotions (which, following
recent work by Antonio Damasio, can be construed as sequences of physio-
logical and psychological events that subtend feelings [Damasio 1999,
2003]) and the movements of bodies including our own through space.
The difference in function between these two modes of communication is
matched by a difference in the forms through which the functions are real-
ized. In the case of language, this is accomplished through symbolic units
that correlate with functions such as those represented by nouns and verbs,
as well as with the many other parts of speech recognized by grammarians
(Croft 2001: chap. 2). Through the use of these symbolic units we can direct
the attention of another person to objects or concepts within a shared refer-
ential frame.
In the case of music, the basic formal unit is what I call a sonic analog,
which represents through patterned sound the central features of some dy-
namic process.
6
Descent, for instance, is one such dynamic process: Pal-
estrina, in the example shown above, provides one sonic analog for this
process; Biber, in the Credo from his mass, provides another. I would hasten
to add that neither musical passage stands for descent in a direct or un-
equivocal fashion. Instead, the phenomenal properties of these passages are
such that they can serve as analogs for the process of descent.
7

Given this perspective, it should be clear that mappings between the do-
mains of language and music will involve structures that are fundamentally
different in kind. Language tends to focus on objects (whether real or imag-
ined) and relationships between objects. Language can direct our attention to
a process (that is, the noun descent picks out a dynamic process that in-
volves a traversal of space), but it is less common for language to embody
such a process. When it does when, for instance, we imitate the sound of a
horses step with the words clip-clop, clip-clop language starts to be-
come more like music. Music, for its part, does not tend to be involved with
the rich symbolic systems typical of language.
8
In those cases where music
does exploit this sort of symbolic system as, for instance, through musical
topics of the type employed by Mozart and Haydn (Ratner 1980; Allanbrook
1983; Agawu 1991, 1999) its dynamic aspect tends to recede in impor-
tance. In sum, then, mappings from language to music will tend to focus on
static aspects of the musical domain; mappings from music to language will
draw out the dynamic aspects of the domain of language.
Although any attempt to determine a crisp boundary for what counts as
language or what counts as music may be an endeavor destined to generate
more heat than light, it seems clear that, at least in their characteristic usage
in the contemporary world, language and music have different functions.
While the range of language functions is broad, primary among these is the
use of symbolic tokens to direct the attention of another person to objects
and relations within a shared referential frame. Music, by contrast, provides
sonic analogs for a wide range of dynamic processes that are marked in hu-
man experience, especially those associated with the regulation of emotions.
Multimodal metaphors that involve language and music draw on both of
these resources, as the analyses in the next section shall make clear.
3. Text Painting
Because so much of our communication is done through language, the con-
tribution of non-linguistic forms of communication to the construction of
meaning often goes unnoticed. However, as work on the gesture that accom-
panies speech has shown (Goldin-Meadow 2003a; Goldin-Meadow 2003b;
Kendon 2004; McNeill 1992, 2005; Mller and Cienki this volume; Mittel-
berg and Waugh this volume), non-linguistic forms of communication are an
important way humans shape their thought. Evidence for a similar role for
prosody has been provided by a recent set of experiments that demonstrated
the use of pitch inflection as an analog for motion through space, and rate of
delivery as an analog for the speed of an object (Shintel, Nusbaum, and Ok-
rent 2006). With regard to the latter, when subjects were asked to character-
ize the motion of a dot that moved horizontally across a screen at one of two
speeds by saying either It is going left or It is going right, they spoke
more rapidly if the dot was moving quickly than if it was moving slowly.
Moreover, listeners were able to make accurate judgments about the speed of
the dot when they heard recordings of these statements.
In the case of multimodal metaphors that involve language and music, a
consequence of the emphasis on language has been that mappings from lan-
guage to music have received the most attention (Zbikowski 2008). Map-
pings from music to language which function, as do gesture and prosody,
to shape thought have been, by comparison, rather neglected (although see
Lidov 2005 for a recent corrective). The compositional technique of text
painting provides an opportunity to explore both ways of mapping in more
detail, and to develop a better understanding of multimodal metaphors that
involve music.
The basic idea of text painting is simple enough. When a particularly
strong or compelling image occurs in the text for a musical work, the com-
poser writes the accompanying music to suggest, or paint, the image. Thus
if the text mentions a galloping horse, the music coincident with the text
might imitate the sound and action of a horse proceeding at full speed. The
passages from Palestrinas and Bibers masses that I discussed in my intro-
ductory comments are typical examples of text painting, but with one illumi-
nating characteristic: there is in fact nothing in Palestrinas or Bibers music
that imitates the sound of an actual descent. Indeed, the sounds associated
with descents have little in common with a descending scale, and in some
cases for instance, walking down a hill descent may be virtually sound-
less. Text painting is, in consequence, not so much about imitating some
naturally-occurring sound as it is about providing sonic analogs for various
dynamic processes. In Bibers and Palestrinas masses, the music provides a
sonic analog for the dynamic process associated with descent (correlating
descending pitch with a decrease in potential energy), and our understanding
of this analog is structured by the accompanying text.
In terms of the contemporary theory of metaphor, text painting occurs
when concepts prompted by the text for a musical work (which serves as the
source domain) are mapped on to a series of musical events (which serves as
the target domain). This mapping structures our understanding of the musi-
cal domain: we hear the sounds as descending. But as I suggested in my
introductory comments, the sonic analog provided by the music also shapes
our understanding of the text, for the music gives the delivery of the words a
specific contour and duration.
9
Again, given our tendency to give priority to
linguistic domains, the notion that the seemingly indefinite and nonconcep-
tual domain of music could be used to structure thought may seem at best
little more than a passing curiosity, and at worst downright nonsense. But
consider three situations: descendit spoken; descendit sung by Pal-
estrinas singers; and descendit sung by Bibers singers. If there are any
differences between these three utterances, they come from the structure
music can impose on language.
Two further examples of text painting can help to elaborate this point.
The first of these comes from the fourth movement of Johann Sebastian
Bachs Advent cantata Nun komm der Heiden Heiland (BWV 61). The
text and music for each of the three preceding movements of the cantata have
all focused on the coming of Christ (as befits the Advent theme). In the
fourth movement Christ is suddenly before us, speaking words from the third
chapter of Revelation: Behold, I stand at the door, and knock. If any man
hear my voice, and open the door, I will come in to him, and will sup with
him, and he with me. Bach sets this text as an accompanied recitative for
baritone, with the strings playing pizzicato throughout; a shorthand version
of the string parts and solo bass melody for the first four measures is given
in figure 3.

Figure 3. Measures 1-4 of the fourth movement (Recitativo) from J. S. Bachs
cantata Nun komm der Heiden Heiland (BWV 61).
Bachs text painting centers on the words und klopfe an that is, and
knock. Bach uses three compositional techniques to paint this activity.
First, he summons the repetitions we associate with the act of knocking by
repeating the words, and by using three notes to set the first syllable of the
initial klopfe (a device called a melisma) such that the articulation of the
syllable is repeated. Second, he uses staccato marks on the three notes of the
melisma, which place silences between these notes; these silences are similar
to those that fall between knocks on a door. Third, he sets the words with a
broken chord (or arpeggio). This places a kind of distance between each
successive note but also allows us to hear all as belonging to a single con-
nected gesture.
Again, as an out-and-out imitation of the act of knocking Bachs setting
of und klopfe an leaves something to be desired. Knocks are usually un-
pitched, for instance, but Bach gives us different pitches for each blow.
Knocking is not usually accompanied, but here we have pizzicato strings
pulsing in the background.
10
The reason we hear these musical events as
knocking (to the extent that we do) is that our understanding of them is
structured by mapping concepts from language onto music. Again, once we
hear these musical events as the sound of knocking our understanding of the
text starts to change, for the music creates a rather specific analog for the
dynamic process of knocking on a door. This sonic analog relies on the
rhythmic and pitch resources of music: the delivery of und klopfe an is at
first halting and then (when it is repeated) hurried; the rhythmic structure of
the passage is coordinated with a notable expansion of pitch space (from the
span of B3 to D#3 in measures 1-2 to the span of C4 to B2 in the first part
of measure 3) which is then compressed by the final F#3-B3-F#3-G3. These
resources in turn shape our understanding of the text. We not only know that
someone is knocking at the door, but how they are knocking: first tentatively,
and then with more urgency.
My final example of text painting is from Franz Schuberts song
Gretchen am Spinnrade, which takes as its text a scene from Goethes
Faust. In the scene we overhear Gretchen as, alone in her room, she de-
scribes her love for or perhaps enchantment by Faust. The song begins,
as shown in figure 4, with the briefest of introductions by the piano, a swirl-
ing sixteenth-note figure that circles around a D minor chord, a few sparse
and repetitive bass notes sounding beneath. This pattern, or some version of
it, continues throughout the entire 120 measures of the song, with but one
interruption (to which we shall return in a moment). At first glance, the text
(the first lines of which translate as My peace is gone, / My heart is heavy,
/ I will find it never / and never more) may seem to have little to do with
this monotonous accompaniment. The link is provided by the title: this is
Gretchen at the spinning wheel. Schuberts accompaniment is, of course,
meant to evoke the sound of the wheel in action, with the swirling sixteenth
notes summoning the wheel itself and the repetitive, off-beat accents in the
middle voice representing the clack of the bobbin, but for modern listeners
this sonic image will not typically resolve itself until we have mapped con-
ceptual structure from the domain of the text onto the domain of the music.
The text thus serves a function similar to what Roland Barthes has called
anchoring, rendering transparent an aural image that might otherwise re-
main opaque.

Figure 4. Measures 1-13 from Franz Schuberts Gretchen am Spinnrade, op. 2
(D. 118).
Again, Schuberts accompaniment does not, in any direct way, imitate the
sound of a spinning wheel (which is unpitched) although it does provide a
surprisingly accurate analog for the act of spinning. Typical treadle speeds
start at about 60 treadles per minute, with the main wheel turning around
once with each push of the treadle.
11
Each complete pattern of sixteenth
notes in Schuberts accompaniment (with two complete patterns per meas-
ure) takes a bit less than a second to complete at standard performance tem-
pos, meaning that Schuberts wheel spins at approximately the same speed
as does Gretchens. And, as does Gretchens wheel, Schuberts continues
uninterrupted until, midway through the song, Gretchen seems to forget her-
self and the accompaniment momentarily breaks off. The resumption of the
musical pattern and, presumably, Gretchens spinning is at first halting,
getting under way only with the return to the music shown in figure 4.
In my other examples of text painting, the mappings between language
and music were more or less focused: a particular musical passage provided
a sonic analog for descent, or for the act of knocking. In Gretchen am
Spinnrade the mappings are rather less focused: the correlation is not be-
tween a word or cluster of words and a particular musical passage, but be-
tween the situation described in the title and the accompaniment pattern that
permeates the song. As a result, the music informs our understanding of not
just one word from the text but the text as a whole. Through the music we
can hear Gretchens obsession with Faust (in the relentless patterns of the
accompaniment) and sense its fevered intensity: over the course of the song
Schubert makes sparing use of the sort of normative harmonic syntax used
to suggest progress, and relies instead on quicksilver gestures toward various
harmonic centers indicative of a mind that cannot settle down to anything.
The curious situation evident in this last case of text painting, where a
sonic analog informs our understanding of an entire scene rather than just a
single word, points to an interesting feature of multimodal metaphors that
involve language and music, and that can be illuminated by a brief consid-
eration of directionality in metaphor. As has often been noted, metaphors are
directional: the statement The hippopotamus is a ballerina is rather differ-
ent from the statement The ballerina is a hippopotamus. In the first case,
attributes associated with a ballerina are mapped on to the hippopotamus; in
the second case, attributes associated with a hippopotamus are mapped on to
the ballerina. One relatively straightforward explanation for the obvious
differences between these two mappings views metaphor as a special case of
analogy, in which the correlated domains are conceptually distant from one
another (Holyoak and Thagard 1995: 213). From this perspective, the direc-
tionality of metaphor can be viewed as a consequence of mapping conceptual
structure from a source domain to a target domain when there is relatively
little conceptual overlap between the two; when the mapping is reversed, the
large amount of new information introduced creates a second metaphor
markedly different from the first. In the case of linguistic metaphors, the
more abstract structure of the correlated domains is nonetheless retained:
objects are mapped to objects (ballerina to hippopotamus) and relations
to relations (dances to lumbers). Given the account of the different func-
tions of language and music that I have offered, however, even this aspect of
correlation drops away in the case of multimodal metaphors that involve
language and music: the objects and relations of language do not have an
obvious correlate among musics sonic analogs. The directionality of map-
pings between the domains of music and language will, in consequence, take
on an added dimension. When language is used to structure our understand-
ing of music, the result will be a view of music that is static and reflective.
(This is a view basic to much musical analysis, one which Hans Keller 1994
tried to counter with his wordless musical analyses.) When music is used to
structure our understanding of language, the result will be a view of lan-
guage that emphasizes moment to moment transitions and semiotic indeter-
minacy.
In the preceding analyses of text painting I have deliberately tried to be
even handed: I have described how language can structure our understanding
of music, but I have also tried to describe how music can structure our un-
derstanding of language. While this strategy reflects my long acquaintance
with music music for me does not simply accompany thought, but actively
shapes it it raises two problems. First (and returning to my initial exam-
ples), are the concepts summoned by descendit and the passages by Pal-
estrina and Biber really different from one another? The answer is yes and
no. No in that the meaning structures activated by all share certain ele-
ments and relations. But yes in that the way these elements and relations
are configured is decidedly different in each case. The second problem is one
of source and target. Do the novel meaning structures created by these map-
pings really reflect mapping either from language to music, or from music
to language, or do both domains contribute more or less equally to the proc-
ess of meaning construction? The answer is, it depends. On just what it de-
pends can be made clearer by exploring the perspective on cross-domain
mapping provided by the theory of conceptual blending.
4. Conceptual blending and song
As was contemporary metaphor theory, the theory of conceptual blending
was developed primarily with respect to linguistic phenomena.
12
Early on,
however, its application was extended to non-linguistic areas (Zbikowski
1999; Cook 2001), in part because blending involved mental spaces (Fau-
connier 1994) rather than conceptual domains (with the former understood
as ephemeral and pragmatic and the latter as relatively stable and abstract.)
In the following, I would like to explore a conceptual blend in which one of
the input spaces is set up by language, but the other is set up by music. My
focus here will not be on one moment, or even one basic image, but on con-
cepts that are developed over the course of an entire song.

Figure 5. Lead sheet for Jerome Kern and Dorothy Fields The Way You Look
Tonight Polygram International Publishing, Inc., and Aldi Music.
Jerome Kern and Dorothy Fields The Way You Look Tonight is typical
of many of the tunes produced during what some have called the Golden Age
of American song. As can be seen from the lead sheet given in figure 5, Kern
chose an AABA form for the tune. That is, the music for the first sixteen
measures (the A section) is re-used with little alteration for the second six-
teen measures. Contrasting music (the B section, often called the bridge) is
introduced in the third sixteen-measure unit (measures 33-48), and this is
then followed in measure 49 by a return of the A section (in a slightly modi-
fied form, with a tag added at the end). Kerns musical form meshes with
that of Fields lyrics, in which the third stanza contrasts with the others in
both the length of its lines and its rhyme scheme. In the following, I would
like to concentrate on the first verse and explore the mental space set up by
the lyrics, the mental space set up by the music, and then the blended space
that results from their combination.
Fieldss lyrics open with a characteristic space builder: Someday.
13

Here, the space builder establishes a mental space focused on a future state
of affairs rather than on the present. The followings lines fill out the picture:
the speaker, beset with rather dire circumstances (When Im awfully low /
And the world is cold) will be comforted by the remembrance of the object
of his affections and, more specifically, by the way she looks on this particu-
lar night (I will feel a glow just thinking of you / And the way you look
tonight). As is hinted at by the transformative effect of the appearance of
the beloved (an effect confirmed by the second verses There is nothing for
me but to love you), what is involved here is not simply a kind of passive
looking, with one person gazing on another, but an intimacy of association
that has both power and depth. The mental space established by the first
verse thus develops into a scene in which what is of moment is not some
future opportunity to look back to the present as a golden past but the cen-
trality of the way you look tonight to a highly charged romantic relation-
ship. The Someday space builder is thus somewhat misleading: what is
important is not the future but the present.
The melody for the song also begins with a space builder, but in this case
the falling fifth A5-D4. Although these pitches could be understood in a
variety of ways, the simplest interpretation (and one supported by the open-
ing D major chord) is as the fifth and first notes in a D scale. In the music
that follows the registral space between D4 and A5 is filled in by a sequence
of arch-like figures that flesh out the musical topography with notes from the
key of D major (rather than, for instance, D minor). These figures ultimately
move past A5 (in measure 7) and arrive on the high note of the melody (D5)
of measure 9. This arrival coincides with a return to the rhythm of the open-
ing gesture, but with the A5-D4 falling fifth replaced by a D5-D4 descend-
ing octave. It is worth noting that while the D4 of measure 2 and the D4 of
measure 10 are the same pitch, their context is quite different: the registral
space above the D4 of measure 10 has been expanded through the sequential
figures of measures 3-8, and the temporal space between the two whole
notes has been filled in by the moving quarter-notes in these same measures.
Measure 10 is followed by a final passage that owes something to the com-
positional strategies that filled out the musical space in measures 3-8, but
which now lead directly to the D4 that concludes the melody of the A section
in measure 13. From a musical perspective, then, the song opens with a
space builder roughly equivalent to the Someday of the words, after which
the space is filled out with notes instead of words to solidify and stabilize
the conceptual realm prompted by the space builder.
The melodic process that leads to the arrival on D4 in measure 13 is rein-
forced by rhythmic and harmonic processes that are to some extent inde-
pendent of the melody.
14
Over the course of measures 1-8 the number of
shorter-duration notes increases until it reaches its maximum density in
measures 7-8. This is followed (as has already been noted) by the whole
notes of measures 9-10. The contrast between shorter- and longer-duration
notes is then revisited in a more orderly fashion, with the quarter notes of
measure 11, half notes of measure 12, and whole note of measure 13 (tied
over into the quarter note of measure 14). While the harmonies used by Kern
a four-measure pattern repeated (with slight variation) four times are
quite typical in American popular music (what musicians would call a I-VI-
II-V pattern), two details contribute to the overall dynamic shape of the A
section. First, although the D major harmony of measure 5 represents a re-
turn to the opening harmony, the music is kept moving by the melodic se-
quence that accompanies this harmony. Second, although D major returns
once more in measure 9 (and supports long notes that recall the long notes of
the opening measures), Kern destabilizes the chord by turning it into a domi-
nant seventh. The overall effect of all of these processes melodic, rhyth-
mic, and harmonic is to make measures 11-13 a goal for the A section, a
goal whose culmination is the D4 of measure 13, the note which correlates
with the arrival on the last syllable of The way you look tonight.
Essential features of the mental spaces set up by the lyrics and the music
for the first verse of The Way You Look Tonight are given in the concep-
tual integration network diagrammed in figure 6, with each mental space
represented by a circle. Where the mental space set up by the text is con-
cerned primarily with objects and relations (namely, the appearance of the
beloved and its importance for romance) the mental space set up by the
music is concerned with a set of coordinated processes that lead to the final
phrase in the section. When these two spaces are correlated with one another
(as they are in the song) aspects of their structure are projected into the
blended space to yield a dynamic representation of the development of an
intimacy of association. Guiding the integration of these concepts is the ge-
neric space (represented in the top circle of the diagram), which defines the
core cross-space mapping and basic topography for the network. The generic
space, which reflects the insights captured in the invariance principle (Lakoff
1990; Turner 1990), is centered on the idea that focused attention is a form
of intimacy. This sort of attention is behind the fixed gaze of the lover, and it
is also behind our discomfort when confronted with the unbending stare of a
stranger.

Figure 6. Conceptual integration network for the first verse of The Way You
Look Tonight.
The development of this sort of intimacy was at the heart of the scene from
the 1935 movie Swing Time (starring Fred Astaire and Ginger Rogers)
where The Way You Look Tonight first appeared. In this scene Astaire is
trying to win Rogers over: he has managed to get in to her hotel suite, and,
even though she has repeatedly spurned him, gives it his best effort by sing-
ing this song (accompanying himself at the piano). Rogers, for her part,
has locked herself in her room and has started shampooing her hair. As the
song unfolds, however, she emerges from her room in a bathrobe with hair
lathered, first smiling on Astaire and then soundlessly walking over to stand
behind him at the piano, where she rests her hand on his shoulder. This ges-
ture coincides with Astaires arrival at the music of measure 61 (the penul-
timate statement of The way you look tonight), and in response to it he
sings the words one more time, turning to look into Rogers eyes. At the
conclusion of the song his gaze becomes quizzical as he notices the lather on
her head and she, observing this change, turns to a mirror and discovers her
appearance. This comedic moment is, of course, a play on the way you look
tonight, but Astaire and Rogers have just shared an encounter typical of a
much more intimate relationship than they have enjoyed thus far. This inti-
macy is one that is worked out not only in the first verse of Kern and Fields
song, but over its entire course. The second verse moves from generalities of
appearance to specifics (With your smile so warm / And your cheek so
soft), the bridge (with its music momentarily suspending the process en-
acted by the first two A sections) adds the detail of that laugh that wrinkles
your nose, and the final verse finds the singer speaking directly to the be-
loved: Never, never change.
From the perspective of conceptual blending theory, then, in some cases
words and music will prompt the construction of two independent but corre-
lated mental spaces. Both of these spaces contribute structure to a third men-
tal space, in which concepts drawn from each of these two input spaces are
blended. This new space typically serves as a site for the imagination. For
the conceptual blend created by the words and music for The Way You
Look Tonight, we might well imagine that the intimacy established between
the lovers is one that would lead them to dance together, or to exchange lov-
ing words, or perhaps just to stare into each others eyes. None of these pos-
sibilities concludes the scene from Swing Time Rogers, aghast at her ap-
pearance, rushes back to her boudoir but subsequent scenes do make clear
that Rogers and Astaires characters are now a pair.
The mappings associated with a conceptual blend of this sort are different
from those associated with a metaphor in two important ways. First, blend-
ing typically involves highly fluid and thoroughly pragmatic mental spaces
rather than established domains. Metaphorical mappings often yield systems
of metaphor; while blends may exploit such systems, they may also destabi-
lize them by extending the system in novel ways, and thus push against the
boundaries of the domain. Second, rather than one domain (the source) pro-
viding structure for the other (the target) a mapping that gives rise to the
directionality of metaphor noted above in a blend correlated spaces each
contribute to structure that is mapped onto the blend. But whether metaphor
or conceptual blending is involved, I hope I have demonstrated the resources
for meaning construction provided by these two different modalities. Most
readers will find mapping from language to music to be simple enough, not
the least because language is the primary means most humans use to struc-
ture their understanding of the world. Mapping from music to language
might seem a stranger alternative, but I believe it is a real possibility, espe-
cially when language is for some reason ambiguous (as it is in the opening of
Schuberts Gretchen am Spinnrade). Additional evidence that music can
serve as a proper conceptual domain, and thus have at least the potential to
structure our understanding of language, comes from instances of conceptual
blending where music provides one of the input spaces for a conceptual
blend. (I discuss further instances in Zbikowski 2002: chaps. 2 and 6.)
5. Conclusion
Although Wittgenstein believed that how we understand music provided
important insights into how we understand anything in general and lan-
guage in particular he himself struggled with what it meant to understand
music. In a fragment that dates from 1948 he wrote, If I now ask So what
do I actually experience when I hear this theme and understand what I hear?
nothing occurs to me by way of reply except trivialities. Images, sensations
of movement, recollections and such like (Wittgenstein 1980: 69e-70e).
Multimodal metaphors that involve language and music have the potential to
provide crucial insights into this question, for such metaphors make clear the
different functions of language and music in human culture, and the different
ways they construct meaning. Language gives us the means to represent
symbolically objects and relations, and through these representations we can
direct the attention of another person to things within a shared referential
frame. Music, by contrast, provides us with sonic analogs for dynamic proc-
esses, processes that include movement through space (such as descent),
physical gestures (like knocking), and emotional states (such as obsession or
the development of intimacy). A place to begin understanding how we under-
stand music is with such sonic analogs.
Although the established models of cross-domain mapping can tell us
much about the different contributions language and music make to multi-
modal metaphors, the methodology of conceptual blending, and the some-
what more fluid construct of a mental space basic to this methodology, of-
fers one way to capture the unique contribution of each mode of
communication to the process of meaning construction. Conceptual blending
raises at least as many questions as it answers, but it has the potential to
place language and music on an equal footing so that we may better under-
stand the contribution of each to multimodal metaphor.
As Wittgenstein suggested, understanding a sentence is much more akin
to understanding a musical theme than we might first think. This is not to
say that language and music accomplish their ends by the same means I
have in fact argued just the contrary but that they draw on some of the
same cognitive resources for constructing meaning. A key to understanding
how this is possible is offered by multimodal metaphors that involve both
language and music, for such metaphors give us compelling insights into the
rich and varied world of meaning within which humans dwell.
Notes
1. I discuss this passage at greater length in Zbikowski (2002: chap. 2).
2. Further discussion of ways to characterize relationships between musical
pitches can be found in Zbikowski (2002: chap. 2) and Ashley (2004).
3. For the sake of concision I have given only one of Bibers melodic strands.
At this point in the Credo there are nine vocal parts, and together with the
instrumental ensemble the melodic contour shown in example 2 is replicated
no fewer than four times (in most cases with pairs of voices). The temporal
span of Bibers setting of descendit de clis is also significantly longer
than that of Palestrina: Bibers setting runs to forty-five seconds, where Pal-
estrinas is only about twenty seconds long.
4. This conceptual metaphor is a variant of STATES ARE LOCATIONS discussed by
Lakoff and Turner (1989).
5. With regard to the associational meanings of music, it is worth noting that
one of the most well-known depictions of the pounding of horses hooves in
American popular culture the use of a portion of Gioacchino Rossinis
overture to Guillaume Tell for the theme music for the radio and television
series The Lone Ranger does not have any associations with horses in its
original context, but with general ideas about victory.
6. As I use the term, representation does not have to entail a full semiotic
system. This perspective is similar to that adopted by Naomi Cumming
(2000: chap. 3). A dynamic process may be provisionally defined as a coher-
ent sequence of phenomena that is distributed over time and typified by pa-
rametric modulation or change.
7. Sonic analogs are akin to Charles Peirces notion of an icon, and in particu-
lar to the form of an icon he called a diagram, which represent the relations
of the parts of one thing by analogous relations in their own parts (Peirce
1960, 1:277).
8. I draw my perspective on the unique symbolic aspect of language from Dea-
con (1997, 2003).
9. The rough equivalent in prosody would combine the two parameters studied
by Shintel and her colleagues, yielding something like It is going
dooooowwwwnnn spoken with a descending inflection.
10. A few writers have gone so far as to interpret the steady plucking of the or-
chestra in this movement as a further embodiment of knocking, but this
seems something of a stretch. Not only are the attack points too widely
spaced to sound much like knocking but the effect is far too persistent, more
like Edgar Allen Poes telltale heart than a summons from the Savior.
11. This information was gleaned from http://www.hjsstudio.com/espinner.html
(accessed 26 September 2006).
12. For an overview of the theory see Fauconnier and Turner (1998, 2002). For a
review of the latter, see Forceville (2004).
13. Fauconnier describes a space builder as follows: Linguistic expressions will
typically establish new spaces, elements within them, and relations holding
between the elements. I shall call space-builders expressions that may estab-
lish a new space or refer back to one already introduced in the discourse
(1994: 17).
14. Music scholars often analyze music in terms of three primary parameters:
melody, harmony, and rhythm. In the same way that the very notion of mu-
sic varies broadly across cultural practices, the manifestation of these pa-
rameters is not always obvious or unequivocal. In The Way You Look To-
night, however, both the harmony (indicated by the chord symbols above the
staff) and the rhythmic frame (indicated by the notated durations of pitches
and by barlines) are relatively clear-cut.
References
Agawu, Victor Kofi
1999 The challenge of semiotics. In Rethinking Music, Nicholas Cook
and Mark Everist (eds.), 13860. Oxford: Oxford University Press.
1991 Playing with Signs: A Semiotic Interpretation of Classical Music.
Princeton: Princeton University Press.
Allanbrook, Wye Jamison
1983 Rhythmic Gesture in Mozart: Le Nozze di Figaro and Don Gio-
vanni. Chicago: University of Chicago Press.
Ashley, Richard
2004 Musical pitch space across modalities: Spatial and other mappings
through language and culture. In Proceedings of the 8th Interna-
tional Conference on Music Perception and Cognition, Scott D.
Lipscomb, Richard Ashley, Robert O. Gjerdingen, and Peter Web-
ster (eds.), 6467. Adelaide, Australia: Causal Productions.
Boiles, Charles L.
1967 Tepehua thought-song: A case of semantic signaling. Ethnomusi-
cology 11 (3): 26792.
Cook, Nicholas
2001 Theorizing musical meaning. Music Theory Spectrum 23: 17095.
Croft, William
2001 Radical Construction Grammar: Syntactic Theory in Typological
Perspective. Oxford: Oxford University Press.
Cumming, Naomi
2000 The Sonic Self: Musical Subjectivity and Signification. Blooming-
ton: Indiana University Press.
Damasio, Antonio R.
1999 The Feeling of What Happens: Body and Emotion in the Making of
Consciousness. New York: Harcourt Brace.
2003 Looking for Spinoza: Joy, Sorrow, and the Feeling Brain. Orlando,
FL: Harcourt.
Deacon, Terrence W.
1997 The Symbolic Species: The Co-Evolution of Language and the
Brain. New York: W.W. Norton.
2003 Universal grammar and semiotic constraints. In Language Evolu-
tion, Morten H. Christiansen, and Simon Kirby (eds.), 111139.
Fauconnier, Gilles
1994 Mental Spaces: Aspects of Meaning Construction in Natural Lan-
guage. 2nd ed. With a foreword by George Lakoff and Eve Sweet-
ser. Cambridge: Cambridge University Press.
Forceville, Charles
2004 Review of Gilles Fauconnier and Mark Turner, The Way We Think:
Conceptual Blending and the Minds Hidden Complexities. Meta-
phor and Symbol 19: 8389.
Forns, Johan
2003 The words of music. Popular Music and Society 26: 3751.
Goldin-Meadow, Susan
2003a Hearing Gesture: How Our Hands Help Us to Think. Cambridge,
MA: Harvard University Press.
2003b The Resilience of Language: What Gesture Creation in Deaf Chil-
dren Can Tell Us About How All Children Learn Language. New
York: Psychology Press.
Holyoak, Keith J., and Paul Thagard
1995 Mental Leaps: Analogy in Creative Thought. Cambridge, MA: MIT
Press.
Keller, Hans
1994 The musical analysis of music. In Essays on Music, Christopher
Wintle (ed.), 12628. Cambridge: Cambridge University Press.
Kendon, Adam
versity Press.
Lakoff, George
1990 The invariance hypothesis: Is abstract reason based on image-
2nd ed. Andrew Ortony (ed.), 20251. Cambridge: Cambridge Uni-
versity Press.
Lidov, David
2005 Is Language a Music? Writings on Musical Form and Signification.
Bloomington: University of Indiana Press.
McNeill, David
1992 Hand and Mind: What Gestures Reveal About Thought. Chicago:
Mittelberg, Irene, and Linda R. Waugh
this vol. Multimodal figures of thought: A cognitive-semiotic approach to
metaphor and metonymy in co-speech gesture.
this vol. Words, gestures and beyond: Forms of multimodal metaphor in the
Peirce, Charles Sanders
1960 Collected Papers of Charles Sanders Peirce, Charles Hartshorne
and Paul Weiss (eds.). Cambridge, MA: The Belknap Press of Har-
Ratner, Leonard G.
1980 Classic Music: Expression, Form, and Style. New York: Schirmer
Books.
Shintel, Hadas, Howard C. Nusbaum, and Arika Okrent
2006 Analog acoustic expression in speech communication. Journal of
Memory and Language 55: 16777.
Tomasello, Michael
1999 The Cultural Origins of Human Cognition. Cambridge, MA: Har-
Turner, Mark
1990 Aspects of the invariance hypothesis. Cognitive Linguistics 1: 247
55.
Wittgenstein, Ludwig
1980 Culture and Value. Georg Henrik von Wright and Heikki Nyman
(eds.), Peter Winch (trans.). Chicago: University of Chicago.
2001 Philosophical Investigations. 3rd ed. Gertrude Elizabeth Margaret
Anscombe (trans.). Oxford: Blackwell.
1999 The blossoms of Trockne Blumen: Music and text in the early
nineteenth century. Music Analysis 18: 30745.
(AMS Studies in Music.) New York: Oxford University Press.
20023 Music theory, multimedia, and the construction of meaning.
Intgral 1617: 25168.
2008 Metaphor and music. In The Cambridge Handbook of Metaphor
and Thought, Raymond W. Gibbs, Jr. (ed.), 50224. Cambridge:
Chapter 16
The role of non-verbal sound and music in multimodal
metaphor
1

Charles Forceville
Abstract
Any object or phenomenon that evokes clear-cut connotations for a community of
users can function as the source domain of a metaphor, since these connotations
qualify for mapping on the target domain. The activation of connotations, in turn,
presupposes that the source domain is identified by means of one or more
modes/modalities. While the linguistic mode, in its written and spoken varieties,
has received ample attention, and the visual mode is also now theorized more
broadly, non-verbal sound and music as (source) domain-cueing modes/modalities
still await exploration. The present chapter demonstrates that sound and music
can play a role in multimodal metaphor (1) by cuing a source domain; and/or (2)
by triggering mappable connotations of a source domain signaled in a nonverbal
mode. Ten examples of multimodal metaphors involving sound and music from
two different genres (advertising and film) are discussed. The chapter ends with
hypotheses that invite further testing in theoretical and experimental research.

Keywords: non-verbal sound, music, genre, advertising, film
1. Introduction
As defined in Forceville (2005, 2006/this volume, 2007, 2008), multimodal
metaphors are metaphors whose target and source domains are predomi-
nantly or entirely presented in different modes, these modes including mini-
mally visuals, written language, spoken language, non-verbal sound, and
music. According to this definition, the verbo-pictorial metaphors in For-
ceville (1996: 14862) belong to this category, but so do verbal metaphors
whose target and source are presented in written and spoken form respec-
tively. The present chapter is an exploration of how non-verbal sound and
music can play a role in the construal and interpretation of multimodal meta-
phor. Its aim is to chart parameters that need to be taken into account in a
full-fledged theory of multimodal metaphor by discussing ten case studies.
Specifically, I will focus on (a) the role of sound and music to (help) identify
the metaphors source domain; (b) the role of sound and music to (help)
identify features that can be mapped from source to target. Five examples
originate in advertising, with its clearly specifiable genre-convention of at-
tempting to persuade an audience of positive qualities adhering to a certain
product, brand, or service; and five are fragments from fiction films, a genre
which is supposed let us say with Horace to delight, instruct, and move.
The analyses are guided by the three questions that need at the very least be
capable of being answered for something to count as a metaphor:
(1) What are its two domains?
(2) What is its target domain, and what its source domain?
(3) Which feature or (structured) cluster of features can/must be mapped
from source to target? (see Forceville 1996: 108)
2. Case studies
Example 1 (Hi): MOBILE PHONE IS PIANO. A Dutch commercial from KPN
for Hi, a mobile phone service, shows an attractive woman playing an in-
strument that looks and sounds like a piano, but on closer inspection turns
out to be a visual hybrid between a piano and a mobile phone (figure 1).
Playing the piano, the woman sings the following song fragment:

Je bent de liefde van mijn leven/ voor altijd verbonden.
You are the love of my life/ connected forever.
Jij hebt mij vrienden en vrijheid gegeven/ k kan niet meer zonder.
You have given me friends and freedom/ I can no longer do without
Ik bel en SMS, met heel hart en ziel/ Ik ben volslagen mobiel.
I call and SMS, with [my] whole heart and soul/ I am completely
mobile.

(My translation, here and throughout the chapter, ChF)

The fragment (composed by Sander Baas and Iwan den Boestert) sounds like
a hit song of the kind popular in Holland at the time of writing, but was in
fact specifically composed for the commercial (see http://www. megamedia-
magazine.nl/mvtr.php [accessed 9 July 2008] under KPN Telecom Volsla-
gen Mobiel).

The role of non-verbal sound and music in multimodal metaphor 385

Figure 1. An attractive woman plays on a mobile phone that is shaped like a
piano (still from HI mobile commercial, The Netherlands).
After the last line, a female voice-over says, Turn completely mobile, too;
turn Hi-subscriber. This month as from no more than 10. Look on Hi.nl,
followed by an audiologo (So Hi iii!). It is to be noted that the original
Dutch text has (near-)rhymes: leven/gegeven; zonder/verbonden;
ziel/mobiel, contributing to the sense that this is a song. Since both do-
mains (mobile phone and piano) are cued in the visual mode, neither written
text, nor spoken text, nor music is necessary for their identification, and
hence this specimen is strictly speaking a monomodal metaphor of the picto-
rial variety. However, non-verbal sound does play a role in the construal and
interpretation of the metaphor. In the first place the fact that the text is sung
rather than spoken adds emotion to the womans appeal to the mobile phone,
so that her ode becomes aligned with numerous pop songs in which singers
romantically address their lovers. Secondly, the music enhances the similar-
ity that is created between mobile phone and piano. The visual homology
between striking a pianos keys and pressing a mobile phones buttons is
complemented by the effect of this action: singing a song while accompany-
ing oneself on the piano (a source domain action), it is suggested, in the tar-
get domain transforms mobile-phoning into a playful, artistic act with aes-
thetic effects. The commercials makers could hardly have spelled out these
mappable features verbally on penalty of appearing ridiculous; but as non-
verbalized features they strengthen and enrich the MOBILE PHONE IS PIANO
metaphor.
Example 2 (Tuc): COOKIE IS PERSON DOING FITNESS EXERCISES. A com-
mercial for a salty cookie called Tuc shows a Tuc cookie moving from left
to right, and going up and down (figure 2). We interpret this movement as
self-propelled due to the commands of the male voice-over (And left!
Two, three, right! And up! . And relax). This text evokes the domain
of doing fitness exercises, but so does the rhythmic beat accompanying the
voice-over. Thus the target domain is cued visually alone. If the sound is
switched off, and all the verbal cues are (mentally) eliminated, the personifi-
cation of the cookie might still be inferred by some viewers: after all it seems
clear that the cookie jumps to and fro entirely of its own accord. However,
the specification of the source as not simply a person but as a person doing
fitness exercises is cued by the voice-over text as well as by the rhythmic
beat. Each of these would suffice alone for this specification, although no
doubt their combination facilitates and probably quickens comprehension of
the metaphor COOKIE IS PERSON DOING FITNESS EXERCISES.

Figure 2. A Tuc cookie jumps up and down like a fitnesser. Text: But Tuc is most
of all a lot more tastier (still from Tuc Commercial, The Netherlands).
Example 3 (Shell): CARS ARE FISH. The images in the commercial are those
of beautifully colored, animated fish apparently swimming just above the
bottom of the sea (reminiscent of the animation film Finding Nemo, USA
2003). We see a school of fish all stopping in mid-swim (figure 3), then
moving on, and a small fish darting away just in time from a swordfish try-
ing to stab it. The sounds we hear are traffic sounds revving motors,
screeching tires, claxons, a siren. The voice-over reinforces the traffic do-
main:
What is the advantage, in everyday traffic, of a gasoline that has been de-
veloped in collaboration with the people at Ferraris? V-Power is a new
gasoline that ensures better performance. Thanks to a better combustion.
So that your car can respond faster when necessary. And where do you
find a gasoline that guarantees such a good performance as V-Power?
Shell. Where you stop, we go further.

Figure 3. Fish halt suddenly as if cars stopping before traffic lights, suggested by
street sounds (still from Shell commercial, The Netherlands).
The traffic domain is thus cued by both non-verbal sound and spoken lan-
guage. Once the audience has accessed this domain, some movements of the
fish, for instance their completely synchronized stopping and moving, can be
interpreted as signaling the traffic domain (here: stopping before a traffic
light), but these visual cues alone would not suffice to ensure its recognition.
Moreover, the sound adds liveliness and precision to the images: for in-
stance, the little fishs escape from the swordfish is emphasized not just
visually by the quick movement, and verbally by the brief pause before
when necessary, but also by the revving sound. Since speed embodies a
quality claimed to be facilitated by the product advertised (V-Power), this is
meaningful.
In view of the claim that in principle metaphors, irrespective of the mo-
dalities they draw on, do not allow for reversal of target and source (Force-
ville 1995, 2002), it is interesting that in this metaphor its distribution is not
immediately clear. In the first instance, before the voice-over is audible, I
suspect viewers hypothesize that the fish are, or belong to the domain of, the
product promoted. That is, they might at this stage speculate, for instance,
that this is a commercial for a zoo, or perhaps an amusement park with an
aquarium. At the moment they hear the words traffic and gasoline,
words that strongly connote the realm of cars and driving, the assumption
that the fish are to be taken literally is probably discarded. Indeed, it is not
until this moment that viewers will reinterpret the fish as the source domain
of what is to be construed as a metaphor. I submit that just as inferences
made on the basis of the visual track tend to prevail over those made on the
basis of the non-verbal soundtrack, inferences on the basis of verbal lan-
guage override those originating in the visual track. It is thus the verbal
track, supported by the traffic sounds, that makes viewers postulate the
metaphor CARS ARE FISH. The target-status of cars is further reinforced once
we hear the name of the brand advertised: Shell and see one of a series of
shells on the sea bottom transform into the Shell logo.
Example 4 (Iglo): CORNCOB AND FRENCH BEAN ARE BRIDE AND
GROOM. An Iglo instant-meal commercial features a mini-corncob and a
French bean, walking together toward an Iglo package (figure 4). Their
movements alone already suggest anthropomorphizing, but it is the tune of
the Wedding March which metaphorizes the two into bride and bridegroom
(and makes us realize that the Iglo package actually resembles a church). A
voice-over tells us:
In our newest Iglo dish there are mini-corncobs and French beans a com-
bination of young and crispy. And what should of course never be missing
on such a joyful occasion? Exactly! And together with chicken filet
and soy sauce these make up the delicious new Iglo dish.

Figure 4. A min-corncob and a French bean walk as a married couple, the Wed-
ding March being audible on the sound track (still from commercial for
Iglo meals).
At the moment the male voice-over says exactly we hear a faint voice say
yes, now!, followed by a kissing sound, and we see, under cheers and ap-
plause, rice showered upon the couple. The metaphor CORNCOB AND
FRENCH BEAN ARE BRIDE AND GROOM is cued mainly by the Wedding
March tune, since the anthropomorphizing of the two does not suffice to turn
them into a wedding couple. If the sound were turned off, most viewers
would not access the source domain on the basis of visual cues until the rice
shower, if then. Note that despite the reference to a joyful occasion, the
verbal text alone, too, would not be sufficient to cue WEDDING at all. That
is, there is no unambiguous verbal reference to the source domain. Both
target and source are cued pictorially, but it is the Wedding March tune that
turns the vegetables not just into humans, but into bride and groom even
before we see the rice-shower.
Example 5 (Senseo): COFFEE MAKER IS MOTORCYCLE. A Philips coffee
maker, Senseo, shows the metallic machine first in a number of extreme
close-ups defying recognition. We hear a throbbing motorcycle engine and a
fragment of Steppenwolfs Born to be Wild featuring the line Get the motor
running, hit it on the highway made famous by the opening sequence of
Easy Rider (Dennis Hopper, USA 1969). The metaphor COFFEE MAKER IS
MOTORCYCLE is further emphasized by concurrently showing a finger push-
ing the coffee makers on/off button and sounding a kick-starting motorcy-
cle. The shiny black covering of the coffee machine, moreover, looks like the
surface of a motorcycle and the dripping coffee resembles a drop of oil (fig-
ure 5). The following phrases are superimposed, one after the other, on the
images: designed with a vision developed with passion makes of each
moment a sensation that delights all senses. The metaphor in this com-
mercial is cued at least as much aurally as visually. One mappable feature is
clearly the high-tech design, but more importantly the music evokes connota-
tions such as living life in the fast lane, freedom, unconventionality, youth,
sixties counterculture a whole range of qualities nostalgically associated
with Easy Rider motorbiking that are potentially mapped to making your
coffee with a Senseo machine. That these connotations are considered impor-
tant by the commercials maker is confirmed by the written pay-off line at
the end of the commercial: three years old, and already a legend at least in
the kitchen. Switching off the sound of the commercial presumably elimi-
nates the most important cue for the source domain, Easy Rider motorbike.
This commercial, therefore, is best classified as a multimodal metaphor. (I
owe this example and part of the analysis to Victor 2004.)
Watching commercials is governed by the strong genre-expectation that
some product, service, or brand name is promoted, which enormously facili-
tates and constrains the preferred interpretation of everything visible and
audible in them, including the metaphors. In most metaphors in commercials,
the product is the target and the source is something else, which means that
it is positive features that are mapped from source to target (Forceville 1996:
104). However, it is important not to theorize multimodal metaphor exclu-
sively on the basis of case-studies exemplifying a single genre, since this
might lead to a mistaken conception of pictorial and multimodal metaphors
prototypical characteristics (Forcevilles 2002 criticisms of Carrolls 1994,
1996 theory of pictorial/ visual metaphor expose this misconception). It is
commendable, therefore, to consider specimens from a different genre as
well. Metaphors in art are usually not amenable to the kind of clear-cut,
singular intentions found in advertising. One crucial artistic convention is
that it is, in the words of Siegfried Schmidt (1991), more poly-interpretable
(see for discussion of Schmidts fact versus aesthetic and monova-
lence versus polyvalence convention Fokkema and Ibsch 2000: chap. 2
and 5; see also Forceville 1999a). Here are, therefore, five examples of mul-
timodal metaphors from fiction films. (For discussions of multimodal meta-
phors in the cartoon genre, see El Refaie this volume; Yus this volume;
Schilperoord and Maes this volume.)

Figure 5. A drop of what looks like oil an impression reinforced by the accom-
panying Born to be Wild song is in fact a drop of coffee (still from a
commercial for Senseo, The Netherlands).
Example 6 (The General): REAL CANNON IS CIRCUS CANNON. Buster Kea-
tons brilliant film The General (1927), set during the American Civil War,
features Keaton as the train driver Johnnie Gray, whose locomotive is stolen
by enemy Northern soldiers. Johnnie gets hold of another locomotive, hitches
on to the loc a wagon with a cannon, and single-handedly goes in pursuit
along the same railway track. When he has the Northerners in sight, he loads
the cannon (figure 6), ready to fire over the cabin of his own loc. Just before
Johnnie fires the cannon, we hear the soft-and-quick drum on the soundtrack
that we recognize as a clich device to create tensive expectation, while the
firing of the canon ball itself is accompanied by a very unrealistic popping
sound (the music and sound of the version discussed here is by Konrad Elf-
ers). As such, there is nothing much metaphorical about this, but the conven-
tional drums anticipating the firing will no doubt remind many viewers of a
similar situation in circus acts, where a human cannon ball is fired. Hence
construal of the metaphor REAL CANNON IS CIRCUS CANNON is invited. The
mapped features in this case are the connotations evoked by a circus context,
such as pleasurable excitement, risk-taking, the idea of watching a perform-
ance and of course the mapping onto real-cannon firing turns Johnnies
action into slapstick.
Incidentally, the fact that this film predates the sound era (Al Johnsons
The Jazz Singer, 1927, is conventionally credited with being the first sound
film), has an interesting consequence from the point of view of metaphors
involving music and sound. In the pre-sound era, the typical program had
musical accompaniment. In the more modest presentations, a pianist might
play; in vaudeville theatres, the house orchestra provided music (Thompson
and Bordwell 1994: 13). This, then, gave ad-libbing musicians opportunities
to create multimodal metaphors involving sound where these may not have
been envisaged by the films makers.

Figure 6. Johnnie (Buster Keaton) loads a cannon, while tensive circus-act mu-
sic is audible (still from Buster Keaton, The General, USA 1927).
Example 7 (If ): PUBLIC SCHOOL IS ARMY. Lindsay Andersons If .... (UK
1968) is a satire on the perverse aspects of British public school life. The
verbal and visual references to battle and war, combined with the violence
and the expectation of blind discipline in a hierarchical system gives rise to
the metaphor PUBLIC SCHOOL IS ARMY a conceptual metaphor that is rein-
forced throughout the film. Starting out realistically, the film gradually be-
gins to show bizarre and surrealistic events that, commensurate with the
counterfactuality suggested by the title, destabilize the status of the real.
Indeed, the film increasingly literalizes the metaphor, resulting in a climactic
machine-gunning of the established order by a small band of rebels. The very
last scene of the film can be construed as a multimodal metaphor in which
music provides the source domain. The rebels, led by Mick (the first major
role of actor Malcolm MacDowell), are positioned, heavily armed, on the
roofs surrounding the colleges quadrangle. Because of a celebration, pupils,
staff, parents, and various officials (tellingly: representatives of the royal
family, the army, and the church) are gathered in the college chapel, until
they are smoked out by the rebels. Outside they are gunned down helplessly
(figure 7) until they manage to access a weapon depot in the college (!) and
are able to retaliate. During the last images of cross-firing, the same melody
is audible that had been sung in the chapel.
2
This hymn-like melody meto-
nymically connotes the rituality of a religious service, and given the films
consistent focus on cruel, ritualized behavior (canings and other humiliating
punishments such as cold showers and pupils being hung upside down in a
toilet pot), it is possible to construe the multimodal metaphor MASSACRE IS
RELIGIOUS RITUAL, potential mappable features being the circumstances that
in a ritual event everything is anticipated, constitutes the performance of a
script, and has profound meaning.

Figure 7. The massacre in the college quadrangle (still from If , Lindsay Ander-
son, UK 1968).
Example 8. (The Godfather I): MENTAL STATE IS FAST TRAIN. Michael (Al
Pacino) in The Godfather I (Francis Ford Coppola, USA 1972) prepares to
take revenge on his familys enemies, Sollozzo and McCluskey. He has
planned for a gun to be hidden in the toilet in the restaurant where they will
be having dinner. Since he is not an experienced killer at this stage of his
life, this is an emotional moment for Michael. When he returns from the
toilet, with the gun, the sound track features a rumbling sound, which be-
comes identifiable as that of a riding train. The sound is in this scene non-
diegetic, that is, it does not emanate from the actual events portrayed at this
moment. That this is so is made clear by the fact that the sound is not con-
tinuous: it is audible in the shots when we see Michael in the toilet, but not in
the shots of Sollozzo and McCluskey left behind at the restaurant table. The
sound is thus used to convey Michaels mental state, suggesting the meta-
phor MICHAELS MENTAL STATE IS FAST TRAIN. Possible mappings are such
a trains unstoppability, the inexorable rhythm of its progress, the circum-
stance that its noise drowns out other sounds which in the target domain
translate as, say, Michaels determination, or his refusal to reconsider his
plan to kill. Lena Chatzigrigoriou (to whom I owe the example and part of
the analysis) interprets the scene as follows: The sound of the shrieking
train breaks gradually, overpowering Sollozzos voice. This sound tells us
that Michael is not listening anymore, he is ready for action (Chatzigri-
groriou 2006: 13). That the sound is a trains is confirmed in the DVD
commentary track by Francis Ford Coppola, who observes that there was an
elevator train in the vicinity of the restaurant where the scene was shot, and
that it was, in fact, the sound of this train that provided the idea for its usage
in the scene. This comment also suggests that we indeed already heard the
same train sound earlier, but then diegetically, namely during the car trip to
the restaurant. As in the If scene discussed in example 7, then, the sound
used as source domain in the metaphor had been cued realistically in an ear-
lier scene.
Even if the analysis is accepted, it is nonetheless clear that presumably
few spectators will consciously construe the concept TRAIN as a metaphori-
cal source domain in this highly suspenseful scene. They may, instead, con-
strue the sound more generically as swelling rumble or something like that.
This leaves intact the claim that the sonic source domain brings to the fore to
the audience what Michael thinks which is not directly made visible (al-
though Al Pacinos facial expression arguably also helps the audience inter-
pret what he is thinking). In Indurkhyas (1991) terms, this would be an
example of a projective metaphor (Indurkhya 1991: 16), a type in which a
source domain structures a largely unstructured target domain. But the ex-
ample points up an issue that is particularly pertinent to multimodal meta-
phors whose source, for whatever reason, is difficult to verbalize unambigu-
ously. Should we say that members of the audience who failed to cue TRAIN,
but who did cue, say, SWELLING RUMBLE, and somehow mapped associa-
tions adhering to that source to the target MICHAELS MENTAL STATE have
or that they have not processed the metaphor? Or have they processed differ-
ent metaphors? (See Bartsch 2002 and some responses to Bartsch in Force-
ville 2006.) This issue deserves sustained discussion.

Figure 8. Adam as dog, biting the hand of his creator (still from Adam, Peter
Lord, UK 1991).
Example 9 (Adam and Remember to Keep Holy the Sabbath Day): HUMAN
BEING IS DOG. Animation films are rich in metaphors in which sound plays a
role. A simple example is the juxtaposition of a human(oid) creature and a
recognizable animal sound. Adam (Peter Lord, UK 1991) is a short clayma-
tion which playfully refers to the fact that the eponymous hero derives its
name from Hebrew Adamah, meaning reddish clay. Its Godlike animator
keeps ordering Adam about on his miniature planet earth. At one moment
Adam, confused what behavior his creator expects of him, goes down on all
fours, barks, and snaps at his creator (figure 8), so that for a moment the
viewer is invited to entertain the metaphor ADAM IS DOG. A similar situation
occurs in one of Phil Mulloys bleak animations, Remember to Keep Holy
the Sabbath Day in the Ten Commandments series: Ezekiel Mittenbender
kneels down and begins to bark. Note, incidentally, that the metaphor is used
for different narrative purposes: in the first case, the metaphor is deployed to
indicate the heros temporary confusion; in the second, to convey his insan-
ity. Furthermore, while redundant to cue the source domain animal (the
fact that both Adam and Ezekiel go down on all fours provides sufficient
visual cues for that), the barking narrows down the animality to that of a dog
thus potentially activating in the audience a whole range of connotations
adhering to DOG and hence contributes information not available in the
image track.
Example 10 (Robin Hood): TENT IS TRAIN. In the animation film Robin
Hood (Wolfgang Reitherman, USA 1973) at one moment a group of crea-
tures in a tent scurries over a plain, only their feet being visible. The phe-
nomenon reminds us of a fast-moving, but unspecified vehicle, but the
rhythmic percussion steers us into understanding it as a train an interpreta-
tion that is further supported when, upon the tents approach of a porch,
we hear a train whistle. Without the sound, we might have opted for the
metaphor TENT IS VEHICLE, but the sound specifies the vehicle as a TRAIN,
and transforms the porch into a TUNNEL. I owe this example to Bensdorp
and Vergeer (2004), who discuss many more sound metaphors in animation
films.
3. Discussion and conclusions
While it impossible to make sweeping generalizations on the basis of the
examples discussed above, the case studies enable the formulation of obser-
vations and hypotheses that can be tested in further research, both theoretical
and empirical, pertaining to sound in multimodal metaphor.
Whereas language has the means to correlate verbal information with
visual, sonic, or musical information via deixis (e.g., this man or the
train), correlations between non-verbal modes depend on well-timed si-
multaneity. That is, in order to suggest a multimodal metaphor that does not
draw on spoken or written language, the identity relationship between target
and source must be triggered by making them visible/audible at the same
moment. Such identity can, of course, be enhanced by many devices: a sound
that is lip-synchronous with a character opening her mouth; visible move-
ments that correspond with sonic rhythms (due to its abundant use in main-
stream animation this is called Mickey mousing); or a montage pattern
that has the same rhythm as the accompanying sound or music.
The non-verbal and musical sounds that play a role in the metaphor
tend to cue its source rather than its target domain. The visual component,
that is, cues the literal target under discussion, whereas the sonic component
metaphorically transforms this target. The Shell example suggests that the
spoken language variety of the sound track, in turn, takes precedence over
the visual track in cueing the target rather than the source (the same holds,
more spectacularly, for the CMG commercial discussed in Forceville 2003).
The sound is seldom alone responsible for the identification of the
source domain. Usually, the pictorial and/or verbal information provides
hints about the identity of the source domain (see also Forceville 1999b;
Eggertsson 2006; Eggertsson and Forceville this volume). The sound, how-
ever, facilitates the identification of the source. In this respect Gibbs remark
that even a cursory examination of theories of linguistic interpretation reveals a
tremendous diversity in the emphases on the different temporal points at which an
utterance or text has supposedly been understood (1994: 115116) is no less
pertinent for multimodal metaphors than for verbal ones. Time is an impor-
tant factor here: it is likely that the sound in multimodal metaphors, even if it
does not contribute mappable features, aids the speed with which a source
domain is identified. This is particularly pertinent in films and commercial
advertising, which do not, under normal viewing circumstances, allow for
the kind of backtracking that for instance written verbal metaphors permit.
In addition, as we have seen, in several examples the soundtrack provides
information that steers identification of the source domain on a more specific
level, or allows that identification to be made with greater confidence than on
the basis of the visual and/or verbal track alone: in the Iglo example, it is the
Wedding March that is the first cue that the viewer is not just to see the
mini-corncob and the French bean as persons, but as bride and groom.
A complete analysis of a multimodal metaphor involving sound as in-
deed of any kind of metaphor requires an assessment whether the source
domain cued is diegetic or non-diegetic. If the source domain, or part of it, is
diegetic (that is: it belongs to the events in the story world as presented at
that moment) or quasi-diegetic, the metaphor is naturalized because there
is a metonymic link between the source and the target domains. As a conse-
quence, the metaphor is less obtrusive, and possibly less easily identifiable
as such than when the source domain is plainly non-diegetic. It is noticeable,
for instance, that in four of the commercial examples, the source domains
are metonymically linked to the target domain, and thus in some quasi-logic
literalized: in example 1 the piano and its music are not only the source
domain for a mobile phone: they also help produce the love song supposedly
addressed to the phone. In example 2, the Tuc cookie is not only, metaphori-
cally, a fitnesser; fitnessers also do well, the voice-over tells us, to eat Tuc
cookies because they are so light (i.e., presumably calories-poor). In ex-
ample 3, the underwater world of fish leads naturally to a shot of a number
of shells, one of which transforms into the Shell logo. In example 4 the rice
showered over the wedding couple is part of the Iglo product promoted. Only
in example 5, there is not really a diegetic motivation for the source domain,
motorcycling, although the slogan already a legend is supposed to bridge
the distance between Easy Rider-motorbiking and Senseo-coffeemaking.
In the artistic metaphors, too, there is sometimes a degree of naturaliza-
tion of the source domain. In Lindsay Andersons fragment from If , for
instance, the organ music that helps metaphorize, in my reading of it, the
final scene of the film, may not be particularly noticeable because the same
organ song was used just before in a literal context. That is, in the earlier
scene it was diegetically motivated music, which in the latter scene has be-
come non-diegetic (but recalls its diegetic use). The same holds true for the
fragment from The Godfather, in which the elevator train had been diegeti-
cally audible in an earlier scene. The effect, I propose is similar to the effect
in the commercials: it reduces the sense of artificiality necessarily associated
with the presentation of a source domain that has no realistic motivation
whatsoever.
Bordwell and Thompson point out that sound can achieve very strong
effects and yet remain quite unnoticeable (1997: 315). Inasmuch as sound
is less consciously registered than images, metaphors with sonic source do-
mains may exercise their persuasive or narrative influence more subtly than,
for instance, metaphors whose terms are both presented in pictorial terms.
Experimental research on multimodal metaphors with a sound dimension in
which sound tracks are suppressed or altered (see Chatzigrigoriou 2007) is
imperative to gain more insight into the working of sonic metaphors.
Not all examples presented here are metaphorical with a high degree of
explicitness. As always, the signals used to cue the source domain must be
comprehensible by the envisaged audience for a metaphorical construal to be
possible in the first place. In fact, one could venture that, because they never
have a verbal is to explicitize them, multimodal metaphors tend to be less
explicit than purely verbal ones (for more discussion on implicitly versus
explicitly signaled metaphors, see Forceville 1999b).
Finally, the genre to which a representation belongs steers the possible or
most plausible interpretations of any element in it, including metaphor (see
Charteris-Black 2004 for corpus analysis of metaphors in genres such as
sports news, political manifestos, and religious texts in the Bible and the
Koran; Caballero 2006 for an in-depth examination of metaphor use in ar-
chitectural building reviews; and Caballero, this volume, for their use in
wine-tasting notes). Systematic investigation of sonic metaphors in different
genres (e.g., commercials, art films, horror films, video clips, computer
games), or sequences within films (dreams, hallucinations, flash-backs) may
reveal patterns specific for such genres or sequences.
Notes
1. This chapter is a revised and expanded version of a paper that appeared ear-
lier of Forceville (2004). I am indebted to Eduardo Urios-Aparisi for com-
ments on an earlier draft of this new version.
2. The melody of the song is that associated with J. Threlfalls Hosanna, Loud
Hosanna, but the words sung, so far as they can be deciphered, do not fit
that hymns text. Lindsay Anderson himself wrote about the melody that he
had originally asked Marc Wilkinson to write some music for the final on-
slaught, where Mick alone on the roof tries to hold at bay the attacking forces
of Establishment, but we found when we played with the sound tracks in the
cutting room that a simple organ version of the College song, which fortu-
nately I had recorded when we were on location, fitted the sequence much
better (Anderson 1975). Apparently, then, the hymns melody was used for
a college song, whose text I have not been able to locate. I am indebted to
Thomas Elsaesser, Erik Hedling, and Andrew Webber for help in my hunt
for clues on the song.
References
Anderson, Lindsay
1975 Using music. [Typed draft, Library collection University of Stirling,
UK] http://www.is.stir.ac.uk/libraries/collections/anderson/music.
php (last accessed 8 July 2008).
Bartsch, Renate
2002 Generating polysemy: Metaphor and metonymy. In Metaphor and
Metonymy in Comparison and Contrast, Ren Dirven and Ralf
Bensdorp, Thomas, and Arjan Vergeer
2004 Geluid en de Populaire Animatiefilm: Een Cognitivistisch
Onderzoek naar Betekenisvorming. [Sound and the popular anima-
tion film: A cognitivist approach to meaning formation.] MA diss.,
Department of Media Studies, University of Amsterdam, The Neth-
erlands.
Bordwell, David, and Kirstin Thompson
1997 Film Art: An Introduction. 5th ed. New York: McGraw-Hill.
Caballero, Rosario
2006 Re-Viewing Space: Architects Assessment of Built Space. Berlin/
New York: Mouton de Gruyter.
this vol. Cutting across the senses: Imagery in winespeak and audiovisual
promotion.
Carroll, Noel
1994 Visual metaphor. In Aspects of metaphor, Jaakko Hintikka (ed.),
1996 A note on film metaphor. In Theorizing the moving image, 212
Charteris-Black, Jonathan
2004 Corpus Approaches to Critical Metaphor Analysis. London: Pal-
grave-MacMillan.
Chatzigrigoriou, Eleni
2006 Take-home exam pictorial and multimodal metaphor. Manuscript.
Department of Media Studies, University of Amsterdam, The Neth-
erlands.
2007 Trans-Diegetic Sound: In-Between Spaces in Post-Classical Cin-
ema. MA diss., Department of Media Studies, University of Am-
sterdam.
Eggertsson, Gunnar Theodr
2006 Animal Horror: An Investigation into Animal Rights, Horror Cin-
ema and the Double Standards of Violent Human Behaviour. MA
diss., Department of Media Studies, Universiteit van Amsterdam.
Eggertsson, Gunnar Theodr, and Charles Forceville
this vol. The HUMAN VICTIM IS ANIMAL metaphor in extreme horror films.
El Refaie, Elizabeth
Fokkema, Douwe, and Elrud Ibsch
2000 Knowledge and Commitment: A Problem-Oriented Approach to
Literary Studies. Amsterdam/Philadelphia: Benjamins.
Forceville, Charles
1995 (A)symmetry in metaphor: The importance of extended context.
Poetics Today 16: 677708.
1999a Art or ad? the influence of genre-attribution on the interpretation of
images. SPIEL 18: 279300.
1999b The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters,
bol 14: 17998.
2003 Bildliche und multimodale Metaphern in Werbespots [Tr. from
2004 The role of non-verbal sound and music in multimodal metaphor. In
Words in their Places: A Festschrift for J. Lachlan Mackenzie,
Aertsen, Henk, Mike Hannay, and Rod Lyall (eds.), 6578. Amster-
dam: Faculty of Arts, VU Amsterdam.
of Semiotics 1: 1951. http://semiotics.ca/.
Barbara J. Phillips (eds.), 272310. Armonk, NY: ME Sharpe.
Indurkhya, Bipin
1991 Modes of metaphor. Metaphor and Symbolic Activity 6: 127.
Schilperoord, Joost and Alfons Maes
Schmidt, Siegfried J.
1991 Literary systems as self-organizing systems. In Empirical Studies of
Literature, Elrud Ibsch, Dick Schram, and Gerard Steen (eds.),
413424. Amsterdam/Atlanta, GA: Rodopi.
Thompson, Kristin, and David Bordwell
1994 Film History: An Introduction. New York: McGraw-Hill.
Victor, Paul
2004 The Metaphorical Use of Sound in Film. MA diss., Department of
Media Studies, University of Amsterdam, The Netherlands.
Yus, Francisco
this vol. Visual metaphor versus verbal metaphor: A unified account.

VII

Metaphor and Film
Chapter 17
Multimodal metaphor in classical film theory from
the 1920s to the 1950s
Mats Rohdin
Abstract
This chapter presents the discussion of multimodal metaphor in classical film
theory from the 1920s to the 1950s. Many famous film metaphor examples from
the silent era are multimodal since they involve not only moving images but also
written texts (intertitles, that is, shots inserted in the film providing explanatory
text or dialogue). The breakthrough of sound film in the late 1920s enhanced the
construction of multimodal metaphors since speech, music and sound effects were
added to images and written texts. Throughout this paper questions of identifica-
tion and interpretation of film metaphor are discussed, and it is claimed that for-
mal criteria are not sufficient to adequately describe the rich variety of multimodal
metaphors.

Keywords: Conceptual metaphor, film metaphor, film theory, monomodal meta-
phor versus multimodal metaphor.
1. Introduction
Arthur Danto (1992: 74) has suggested that a possible explanation why art
criticism never developed a concept for pictorial metaphor in ancient Greece
was that scholars were too much concerned with mimesis. Dantos opinion is
no less pertinent to film scholars during the first decades of the twentieth
century, when the new medium was regarded as the culmination of the old
mimetic tradition, almost as reality representing itself (Hake 1993: 277).
The close connection between film and perceptual reality is probably one
important reason why there is not any systematic account of metaphor in
film theory until the late 1920s. Another reason is, of course, that many
404 Mats Rohdin
writers regarded the concept of metaphor as exclusively a verbal matter
(e.g., Arnheim 1933: 260270; Stanford 1936: 95).
In this chapter I will focus on multimodal metaphors in classical film the-
ory from the 1920s to the 1950s. Multimodal metaphors are metaphors
in different modes (Forceville 2006: 384/this volume). In fiction film these
modes include five different signs: (1) pictorial signs, (2) written signs, (3)
spoken signs, (4) non-verbal sounds, and (5) music. Classical film theory
refers to a theory of cinematic representation premised upon mechanical
duplication (upon the existential relation between film and referent) and is
therefore based upon an extensional semantic theory of meaning (Buckland
2000: 26). Principally I will elaborate on some tenets from my Ph.D. Vilds-
vinet i filmens trdgrd. Metaforbegreppet inom filmteorin [The wild boar
in the garden of film. The concept of metaphor in film theory] (Rohdin
2003), where I examine film theory texts that explicitly include the term
metaphor. For this occasion my investigation concerns the question
whether some of these metaphors would qualify as multimodal metaphors.
Worth remembering is that many examples discussed are difficult to fully
describe due to the lack of information from the writers as to what guides the
identification (what are the two terms? which are target and source, respec-
tively?) and interpretation (what is mapped from source domain to target
domain?) of the metaphor. Consequently some examples of identification and
interpretation may be more rightly attributed to my inferences and guesses
than to the author in question. As Forceville stresses, such verbalization is
never neutral (2006: 390).
When dealing with the identification and interpretation of multimodal
metaphors I attempt to delineate three questions, which according to Force-
ville (1996: 6566, 2002: 23, 2005: 266), have to be answered: (1) Which
are the two terms of the multimodal metaphor? (2) Which is the metaphors
target domain and which is the metaphors source domain? (3) Which fea-
tures are mapped from the source domain to the target domain?
One reason why I have chosen to examine classical film theory is that the
definitions and film examples presented by these theorists, especially in the
1920s and 1930s, have formed the canon to which film historians and theo-
rists frequently return. These examples of metaphor have later been repeated
extensively in film studies (e.g., in film theory and film history, as well as in
film dictionaries), thus fulfilling a function similar to the classic examples in
handbooks of rhetoric. Above all, these examples have illustrated for genera-
tions of university students the formal characteristics, which constitute crite-
ria for defining film metaphor.
Multimodal metaphor in classical film theory 405
2. Classification of multimodal film metaphors up to the late 1950s
In silent cinema during the 1920s experiments were done in different coun-
tries in order to find filmic equivalents of verbal metaphor. Soviet montage
cinema, and especially Sergei Eisenstein, is particularly renowned in this
respect. Eisenstein (1996: 222) even claimed that experiments dealing with
montage tropes, including film metaphor, was Soviet montage cinemas most
important contribution to film history.
As a matter of fact, many of the canonized examples of film metaphor
from the silent film period are multimodal, since they involve not only mov-
ing images (visual signs) but also intertitles (shots inserted between live-
action shots, providing explanatory text or dialogue). The breakthrough of
sound film in the late 1920s further enhanced the possibility of multimodal
metaphors since three more modes were added to the standardized film prod-
uct: speech, sound, and music.
When it comes to the question of how to identify a verbal metaphor, early
theories often assumed that metaphor was a thing or an object in the text,
which could be identified thanks to some sort of deviation or anomaly at the
surface level, for example, semantic anomaly or logical absurdity when
taken literally. In film theory the equivalent has often been a disruption of or
deviation from a mimetic form of representation (for example by a superim-
position, that is when two or more images are combined into one over the
whole area of the frame by double printing or double exposure in the cam-
era, Salt [1992: 329]) or against narrative codes (for example by using a
nondiegetic insert in a mainstream fiction film, that is a shot or series of
shots cut into a sequence, showing objects that are represented as being out-
side the world of the narrative [Bordwell and Thompson 2007: 480]). How-
ever, modern verbal and non-verbal metaphor theories often claim that there
does not have to be any deviation or incoherence at all on the surface level of
the representation in order to identify a metaphor (Black 1993: 34; Cohen
1997: 223225, 1999: 399; El Refaie 2003: 7980).
In the writings of classical film theory between the 1920s and 1950s, a
preliminary classification of different formal characteristics which are said
to cue the spectator in identifying the film metaphor would, according to my
analysis, look like this:
- Superimposition (e.g., Arnheim 1933; Carroll 1996a)
- Verbal image (e.g., Balzs 1930, 1952; Ejxenbaum 1981; Eisenstein
1987a, 1987b, 1996)
- Montage (primarily the nondiegetic insert) (e.g., Arnheim 1997; Balzs
1930, 1952; Bazin 1967; Eisenstein 1974, 1996)
406 Mats Rohdin
- Cinematography (e.g., Balzs 1930; Bazin 1981; Eisenstein 1991, 1996;
Tynjanov 1981)
- No formal characteristics on surface level (e.g., Balzs 1952; Bazin
1971, 1973)

2.1 Superimposition
In general, the superimposition is part of a dissolve, which is a transition
between two shots during which the first image gradually disappears while
the second image gradually appears; for a moment the two images blend in
superimposition (Bordwell and Thompson 2007: 478, emphasis in origi-
nal). This type of metaphor is often considered an anomaly in popular fiction
film because two elements belonging to different categories are presented as
a homospatially unified figure. In the early years of silent cinema, superim-
positions were done in the film camera, which required a lot of skill from the
photographer. A famous example of this is when Sergei Eisenstein in Strike
(Stachka, 1925) blends an image of a police spy with that of a fox, thereby
proposing the spectator to map aspects from the source domain, FOX, to the
target domain, POLICE SPY, for example cunning and shrewdness. Thus the
superimposition suggests the conventional metaphor MAN IS AN ANIMAL
(figure 1). When the optical printer was invented in the late 1920s superim-
positions could be produced in laboratories during postproduction, which
was less complicated. However, at that time this type of film metaphor al-
most disappeared from popular fiction film.
Another example of superimposition, discussed in Carroll (1996a), ap-
pears in Fritz Langs futuristic film Metropolis (1927), when the young
protagonist visits the gigantic underground power plant. In a horrific vision,
produced by a dissolve (thus including superimposition), he sees a machine
transforming into a man-devouring monster. An intertitle explains that the
monster is Moloch (figures 24). Thus the metaphor could be expressed as
MACHINE IS MOLOCH: the machine is the target (pictorial signs) and Moloch
is the source (pictorial and written signs). In the source the machine has been
transformed in a number of ways, for example, the stairs have now become
Molochs tongue and the former opening at the top of the machine is now the
monsters mouth. The intertitle was probably considered necessary for the
spectator in order for him or her to identify Moloch, the ancient deity from
the Old Testament (Lev 20: 25; Jer 32: 35). There would of course still be
a metaphoric relationship without the intertitle, namely, MACHINE IS MON-
STER, but the source would in this case be confined to man-devouring mon-
sters in general, thereby losing specific connotations that are attached to the
Moloch figure. Thus the pictorial signs and the intertitle, I would suggest,
qualify the metaphor as multimodal. Mappable features from the source
domain are, for example, how innocent people in ancient times were sacri-
ficed into the fire-burning opening of Moloch, which can be compared to
how workers in the film slave to death at the machines, and so on. Moloch is
literally eating people, while modern industrial machines are metaphori-
cally eating human beings, and thereby killing them.

Figure 1. Superimposition in Sergei
Eisensteins Strike.
Figure 2. In Fritz Langs Metropolis ...

Figure 3. the intertitle helps the
spectator to identify the
transformation of the ma-
chine to Moloch
Figure 4. the man-devouring deity
from the Old Testament.

This metaphor could be read as reflecting the basic or conventional meta-
phor DEATH IS A DEVOURER (Lakoff and Turner 1989: 77), but the specific
context triggers mappings not available from the devourer scheme as such,
408 Mats Rohdin
for example, extra-textual mappings dealing with metafilmic connotations
from the intertitle of the source domain. That is to say, the graphics of the
intertitle, oblique and vertical lines, resemble those used in another famous
German Expressionist film, The Cabinet of Dr. Caligari (Das Cabinet des
Dr. Caligari, Robert Wiene, 1920). This sort of genre mixing (science fic-
tion, horror) facilitates new mappings from the source domain MOLOCH to
the target domain MACHINE, for example, connotations like madness, tyr-
anny, disorder, death, evil supernatural forces out of mans control, and so
on.
A good example of how metaphors can generate new interpretations is the
way in which Janet Ward (2001: 164165) connects Moloch in Langs film
to the institution of cinema during the Weimar period in German history:
[T]he mouth of the devouring Moloch-machine into which workers are led
parallels the electrically lit entrances of the movie palace, into which the new
white-collar employees would stream, blissfully unaware, in their leisure
time (164). Thus I will claim that we have a new metaphor: (THE INSTITU-
TION OF) CINEMA IS MOLOCH. This means that from the same source (MO-
LOCH) an implicit target domain (CINEMA) has been constructed, which is
not part of the film. Although Ward does not stretch her interpretation any
further, it is easy to map more features from the source domain MOLOCH to
the new target domain CINEMA. For instance, the fact that it was mostly
children who were sacrificed in the ancient Moloch rituals could be paral-
leled to the censorship debate in Germany during the early decades of 1900,
which often centred on films allegedly detrimental influence not only on the
working class but also on children (for early German film, see Hake 1993).
Although Wards example is taken from a recently published book, and
thereby does not belong to the period under study, it is illuminating since it
shows how metaphors can build on earlier interpretations, underpin, chal-
lenge, revise, or revitalize them, etcetera, thereby changing meaning in the
course of time.

2.2 Verbal image
A verbal image is a filmic expression building on a verbal expression (meta-
phor, clich, pun, or proverb), which it visualizes literally (e.g., Carroll
1996b). Soviet montage cinema during the 1920s is famous for this sort of
experiment (e.g. Ejxenbaum 1981; Eisenstein 1974, 1987a, 1987b, 1991,
1996; Tynjanov 1981). Often these verbal clichs and metaphors were ap-
propriated from other media and art forms, for example, poster art and pup-
pet theatre, which were very popular at that time because of illiteracy
amongst the majority of the people. Sergei Eisenstein often used these sorts
of clichs when portraying capitalists, kulaks, priests, policemen, etcetera.
Thus the film viewer could use experiences from other media and art forms,
which shared codes and conventions with film. As a result, the spectators
processing of the film narrative was facilitated considerably, which was
important due to the typically fast editing of Soviet cinema at that time.
Sergei Eisensteins film debut Strike includes many scenes built upon the
visual expression of verbal metaphors and clichs. The famous slaughter-
house sequence at the end of the film, for example, is preceded by a scene in
which the police chief gets furious and knocks a bottle of ink over a map
showing the streets of the striking workers district, thus literalizing the
metaphor the streets running with blood (Bordwell 1993: 5859) (figure
5). The verbal expression is target, while the ink on the map is source. The
map metonymically stands for the streets, while the spilled ink metaphori-
cally stands for the workers blood, which gives the metaphor STREETS
RUNNING WITH BLOOD ARE MAP SPILLED WITH INK.

Figure 5. When the police chief in Sergei Eisensteins Strike rages and knocks a
bottle of ink over a map, the verbal image streets running with blood
is visualized.
A mappable feature from the source domain is the manner in which the
spilled ink overflows the streets of the map, which correlates with how the
next scene shows the massacre of the workers, as the streets running with
blood. In order to understand the metaphor the interpreter has to construct
the implicit target domain, STREETS RUNNING WITH BLOOD, which requires
awareness of the metaphoric expression, either in spoken or written form.
410 Mats Rohdin
Thus the source is presented before the implicit target, but the target is then
visualized in the following scene, a common method used by Eisenstein when
dealing with verbal images.
The Russian formalists, who also worked as film critics and scenario
writers, were fond of using verbal images, which may not be surprising con-
sidering their literary background. Boris Ejxenbaum (1981: 79), for exam-
ple, claimed that verbal images, or internal speech as he called it, were the
only way to express metaphors in film. One advantage of visualizing verbal
images, according to some critics, was that they revitalized conventional and
worn out metaphors and clichs (Balzs 1952: 111112, 127), a method
used in different media of today as well, for example in political cartoons
(e.g., El Refaie 2003: 89). A change in the medium will also, of course, in
one way or another, affect the content, which is always important to remem-
ber. Nevertheless, many critics and filmmakers (e.g., Clair 1972: 48) con-
demned this way of metaphorical filmmaking as too literary, a common
accusation at a time when proponents of film claimed the medium to be an
independent art form. Quite soon, indeed, filmmakers realized that meta-
phors in the form of verbal images would forever subordinate film to lan-
guage. The result of this would be that novel metaphors could not be created
in film since they had to exist verbally in order for the spectator to be able to
identify them at all. For this reason, filmmakers began to seek other ways to
express metaphors.

2.3 Montage (primarily the nondiegetic insert)
Filmmakers in the early years soon realized how important montage was to
achieve filmic signification. Some critics suggested that the film metaphor
could be created within the story world (diegesis) by juxtaposing different
objects or things by montage. For example, Balzs (1952) paid attention to
this type of metaphorical montage, as he called it, which during silent
cinema often was visual and monomodal, since the two domains belonged to
the diegetic world. He exemplifies this subtype using a sequence from Eisen-
steins Battleship Potemkin (Bronenosets Potemkin, 1925), where close ups
of the faces of the sailors are crosscut with the ships engines: Such re-
peated juxtaposition compels comparison. A visual parallel inevitably con-
jures up a parallel in the mind. The angry, resolute faces of the sailors trans-
fer their own expression to the wheels and cranks. Yes, they are fighting side
by side in a common struggle (Balzs 1952: 126). This suggests the con-
ventional metaphor MACHINES ARE PEOPLE (Lakoff and Turner 1989: 132)
that attributes human properties to machines. Thus Balzs wanted to stress
how even the mechanic world revolts against injustice by joining the revolu-
tionaries. Other critics did not accept montage images such as these as meta-
phors, since they did not disrupt the mimetic model of the film representa-
tion. Instead they claimed that these expressions ought to be named
similes, since the cut juxtaposed the two objects or things as the words is
like do in verbal similes (e.g., Balzs 1930: 5153).
However, when the montage juxtaposition included a nondiegetic insert it
was often labeled a film metaphor. In Rohdin (2003) I claimed that it is the
nondiegetic insert that is usually referred to when questions concerning film
metaphor are discussed. The fact that the nondiegetic insert does not belong
to the story world is the anomaly or conflict that cues the spectator to carry
out a metaphorical interpretation. References are often made to Sergei
Eisensteins films in the 1920s, for example, Strike (1925), October (Okty-
abr, 1928) and Old and New (Staroe e novoe, 1929). The ending of Eisen-
steins Strike includes a well-known example of a nondiegetic insert when
images of the militarys massacre of defenceless workers are intercut with
scenes from the slaughter of an ox: WORKERS ARE SLAUGHTERED OXEN.
Another often-quoted nondiegetic insert appears in Eisensteins October
(1928), where shots of the politician Kerensky outside the Tsarinas bed-
room in the Winter Palace are intercut with images of a mechanical peacock:
KERENSKY IS A PEACOCK. Both these examples include intertitles and elabo-
rate montage but could be referred to as monomodal since in both examples
target and source, as well as their metaphorical coupling, can be identified
visually.
The breakthrough of sound film in the late 1920s stirred an array of de-
bates. In Soviet film, for example, proponents of the montage cinema were
worried that sound film would be detrimental to their own film style, built
upon fast editing. In a manifesto Sergei Eisenstein, Vsevolod Pudovkin, and
Grigori Alexandrov (1988) claimed that sound film could be justified only if
sound was used contrapuntally: The first experiments in sound must aim at
a sharp discord with the visual images (Eisenstein et al. 1988: 114, em-
phasis in original).
An innovative use of sound in line with the Soviet directors beliefs, in-
cluding a nondiegetic insert, appears in the comedy/musical The Million (Le
Million, 1931) by the French director Ren Clair. When a couple of men in a
theatre are fighting about a coat containing a valuable lottery ticket, it looks
as though they are playing rugby. On the sound track a nondiegetic sound
from a sport arena is heard, that is, noise from the rugby play as well as
from the spectators (figures 67). Clair explained the metaphor as the men
412 Mats Rohdin
throwing the coat like a football, a writer would say (cited in Arnheim
1933: 264265). This, I would argue, qualifies as a multimodal metaphor:
MEN CHASING COAT ARE MEN PLAYING RUGBY. Perhaps it would be possi-
ble to argue that this is a monomodal metaphor because the metaphor is
possible to understand without the nondiegetic sound. Although the source
domain men playing rugby is cued by both visual and aural elements, I
believe that many viewers would miss this metaphor if the obtrusive non-
diegetic sound were missing.

Figure 6. When a couple of men in
Ren Clairs The Million
are chasing a coat inside a
theatre
Figure 7. a nondiegetic sound from a
sport arena is inserted to give
the impression that they are
playing rugby.

Clair (cited in Arnheim 1933) characterized the experiment as a sound-
filmic metaphor equivalent to a metaphor in literature (265). Rudolf Arn-
heim, the German gestalt psychologist and film theorist, by contrast, criti-
cized the scene, which he found unacceptable to the spirit of the film me-
dium: The result? Not a metaphor, but a fake (Arnheim 1997: 41).
Arnheims reaction is a good example of his and other contemporary theo-
rists view that the fiction film must obey a mimetic form of representation.
Nondiegetic inserts, superimpositions, and so on, disturb the films represen-
tation of reality and thus, by definition, must be condemned.
The nondiegetic sound in Clairs film helps the spectator to trigger the
source faster, which may be important in film and its temporal development
under normal viewing circumstances. Forceville (this volume) has observed
that non-verbal and musical sound which influence the multimodal metaphor
tend to cue its source rather than its target domain, which is also true of the
example in The Million. Furthermore Forceville claims that a multimodal
metaphor including sound tends to be more unobtrusive and naturalized
when part of the source domain is diegetic. This is also confirmed in the
example from The Million, since the source is signaled both aurally and
visually. Surprisingly, Arnheim, who is so negative about the nondiegetic
sound insert, praises the scene as skilful: [T]he things that do not go to-
gether are fit together so cleverly that the viewer cannot help combining them
psychologically (1997: 41).
Hollywood also experimented with film metaphors of the nondiegetic type
a couple of times, and two films made by former European directors are
famous in this regard. In Fury (1936), Fritz Lang dissolves from a shot of
gossiping women to a nondiegetic insert showing cackling fowl: GOSSIPING
WOMEN ARE CACKLING FOWL. It could be considered a monomodal meta-
phor of the visual variety, since the visuals seem to suffice to cue both target
(the gossiping women) and source (the cackling fowl). However, it is impor-
tant to note that the transition to the nondiegetic insert has been prepared for
in a number of ways in the film, by including other modes in addition to the
visuals. For example, when the gossips start among the women in the vil-
lage, nondiegetic music is heard simultaneously on the soundtrack, which
mimics not only the voices of the women but also, as it will turn out later,
the sound of cackling fowl. After a short while the womens voices fade out
and become inaudible, while the volume of the mimicking music increases.
Later, when another group of women is gossiping, the nondiegetic sound of
cackling fowl can be heard on the soundtrack a moment before the non-
diegetic insert appears (showing the fowl), which further emphasizes the
connection between the womens chatter and the noise of the fowl. At the
same time we hear the mimicking music in the background. In sum, four
different modes are working together to create this metaphor; visuals, music,
speech, and, finally, noise. The use of these various modes speed up recogni-
tion, but worth taking into consideration is also that without the aural cues it
would presumably be unclear or less clear, at least instantaneously during
film viewing, what is supposed to be the mappable feature. To sum up, this
example from Fritz Langs Fury could indeed be considered a multimodal
metaphor.
Charlie Chaplins Modern Times (1936) is also interesting, because it
raises important questions concerning the criterion governing the distinction
between monomodal and multimodal metaphor. At the beginning of the film
Chaplin uses a nondiegetic insert showing penned sheep, which is followed
by a dissolve to people in a hurry coming up from the subway in the rush
hour going to work (figures 8-10). Thus the two domains of the metaphor
are SHEEP (source) and WORKERS (target); consequently, the metaphor can
414 Mats Rohdin
be expressed as WORKERS ARE SHEEP. Since the identification of both target
and source can be made on the basis of the visual images alone, this meta-
phor would be monomodal. The mapping may be a comparison between the
workers who are forced to take employment in industry in order to survive in
the same way as the sheep are forced into the pen. Both workers and sheep
can be seen as victims of modern large-scale industry and its inhuman as-
sembly line production. The black sheep among the white flock is a visu-
alization of the proverb to be a black sheep, and finds its counterpart in
the little tramp (Chaplin), dressed in black, that cannot fit into the system
among loyal industry workers and other law-abiding citizens in the film.

Figure 8 Charlie Chaplins Modern
Times, a nondiegetic insert
of penned sheep
Figure 9 dissolves into a shot show-
ing workers

Figure 10 hurrying in the early
morning rush.

But the metaphor, I propose, may also draw on the subtitles that precede the
nondiegetic insert and the accompanying intense and aggressive music. The
intertitles Modern Times. A story of industry, of individual enterprise
humanity crusading in the pursuit of happiness are superimposed on a clock
whose second hand is moving relentlessly. Both the written text and the mu-
sic may play important roles when deciding on mappable features to be
transferred from source domain to target domain. For example, the religious
connotations in the word crusading imply a war, and where there is a war
there are victims. The sequence may also be read as an intertextual homage
to Walter Ruttmans Berlin, Symphony of a Great City (Berlin, die Sym-
phonie der Grostadt, 1927), and the controversial sequence where workers
in the rush hour on their way to work are crosscut with a herd of cows being
pushed through the city streets in the morning, heading for the slaughter-
house. Thus we have within a period of a few years, in films such as Me-
tropolis, Berlin, and Modern Times, a number of remarkable metaphors that
represent a fierce critique of modern industrial society and its inhuman
treatment of workers.
To summarize, in the 1920s and early 1930s the film metaphor is consid-
ered as one of the most interesting novelties of the fiction film. But in the
middle of the 1930s things change, and Roman Jakobson writes that there is
an increasing aversion to filmic metaphor (1987: 464), which reflects the
general view among contemporary film critics and theorists. The main prob-
lem, according to me, is the insistence on a mimetic model of film represen-
tation, which makes it almost impossible to integrate metaphor into the nar-
rative since metaphor, according to these theories, is characterized by an
explicit anomaly or deviation disrupting the spatio-temporal and narrative
continuity of traditional fiction film. Metaphors built on superimpositions or
nondiegetic inserts supposedly ruin the impression of a closed diegetic reality
in the fiction film.

2.4 Cinematography
When filmmakers realized that nondiegetic inserts and superimpositions did
not match with the norms of popular fiction film, they tried other solutions to
create some deviation or anomaly on the surface level. A common way of
producing these deviations was aided by cinematography, that is, all the
manipulations of the film strip by the camera in the shooting phase and by
the laboratory in the developing phase (Bordwell and Thompson 2007:
477). Devices commonly used include different sorts of camera angles, cam-
era lenses creating perspective distortions, distance of framing, and so on.
Balzs (1952: 111114), for example, paid much attention to this type of
metaphor in his writings. Among other things, he points out how the camera
416 Mats Rohdin
set-up makes the film image look real in the diegetic story world, but how
it also can express metaphoric meanings: In such metaphoric sequences the
objects photographed are real the set-up merely gives them a deeper
meaning, a second, symbolical significance, without depriving them of their
own, real, normal meaning. The shot would be comprehensible, as the detail
of an ordinary film scene, even to those who failed to grasp this second
meaning (1952: 113).
According to Balzs such metaphors show the skillfulness of the artist,
the ability to take advantage of the film images capacity to embrace two
different meanings at the same time. This type of metaphor could be based
upon unusual camera angle and framing, as when Vsevolod Pudovkin in The
End of St. Petersburg (Konets Sankt-Peterburga, 1927) shows the rulers of
Russia with their heads cut off by the frame (figure 11). Balzs commen-
tary is, as usual, concise, and not very informative: A pictorial metaphor
whose meaning is obvious (Balzs 1930: 36, my translation, MR). How-
ever, the intra-textual context includes a lot of intertitles, which describe
how Mother Russia calls the people to enter World War I with enthusi-
asm. Probably Mother Russia (the rulers) is target and the image of the
rulers with their heads excluded is source, thus probably some sort of visu-
alization of a verbal image: MOTHER RUSSIA IS HEADLESS PEOPLE. This
ought to be labeled multimodal, since the intertitles are important to aid the
spectators to process the rapidly changing images showing cheering people,
marching soldiers, people sitting on chairs with their heads cropped by the
frame, etcetera. The source domains image of the rulers of that time, with
their heads cut off by the frame, indicates their headless politics when Rus-
sia entered Word War I, features which are mapped to the target domain
Mother Russia.
Another example of this type of metaphor cued by cinematography comes
from Eisensteins Old and New (1929), were the director uses a close-up
with wide-angle lens and low camera angle to criticize the Soviet bureauc-
racy. The intra-textual context is recounted with various shots and intertitles:
the farmers are applying for a loan to buy a tractor in order to be able to
bring in the harvest. But the local authorities deny them this, because the
rules state that the farmers can only be given the loan after they have com-
pleted the harvest. Eisenstein wanted to disparage the system of bureauc-
racy, and he did this in different shots by showing objects (writing machine,
pencil sharpener, law book) in gigantic close-ups made with a wide-angle
lens, which dwarfs the clerks in the background (figure 12). In the shot in
question a man is barely visible behind a gigantic book, perhaps a literal
visualization of the verbal image, to hide behind bureaucracy. Thus the
metaphor can be expressed as follows: AUTHORITIES HIDING BEHIND BU-
REAUCRACY IS CLERK BEHIND BOOK. From the source domain CLERK BE-
HIND BOOK mappings are made to the target domain AUTHORITIES HIDING
BEHIND BUREAUCRACY, for example, just as the clerk hides behind the book
so we can barely see him, in the same way authorities hide behind bureauc-
racy, behind rules and regulations so that we get the impression that there
are no humans any longer making decisions, and so on.

Figure 11. Cinematography (e.g.,
camera framing) cues the
spectator to identify a meta-
phor in Vsevolod Pudovkins
The End of St. Petersburg.
Figure 12. An example from Sergei
Eisensteins Old and New
where cinematography (e.g.,
close-up and camera angle)
indicates a metaphor.

Pudovkins and Eisensteins examples show the importance of distance of
framing, especially in the close-up. According to Eisenstein, the close-up
makes an image more abstract than, for example, the long shot or medium
shot. In Dickens, Griffith and ourselves, he speculates on the reasons
which made the Russians realize the importance of the close-up: In Russian
we talk about an object or a person being shot on a large scale [krupnym
planom], i.e., large. An American says: closely (the literal meaning of the
term close-up). Americans talk about the physical conditions of seeing. But
we talk of the qualitative side of the phenomenon that is linked to its signifi-
cance For an American, the term is associated with vision. For us, with
the evaluation of what is seen (Eisenstein 1996: 226, emphasis in original).
Thus Eisenstein seems to imply that there are verbal reasons which made
the Russians connect the close-up with an evaluation of what is seen in-
stead of linking it to the physical condition of seeing, as we do in the West.
The result in the former case, according to Eisenstein, is that a metaphorical
interpretation is facilitated. This example shows that a medium and its tech-
418 Mats Rohdin
nology, due to cultural circumstances like language, may be understood in
different ways in different parts of the world, which is a healthy reminder of
the extent to which metaphors are governed by culturally embedded rather
than universal knowledge.

2.5 No formal characteristics on surface level
This type of film metaphor challenges the old demand for a deviation on the
surface level since there are no formal characteristics that cue the spectator
to perform a metaphoric interpretation. The following example is taken from
the writings of the French film critic Andr Bazin in the 1950s. Among film
scholars he is frequently regarded as one of the most important proponents
of a realistic film style in film history. Nevertheless, Bazin did speak favora-
bly about cinema and its plastic reproduction of reality as an art of po-
tential metaphor (1981: 151). A good example of his metaphoric interpreta-
tions occurs in an article when he writes about a scene at the end of Federico
Fellinis The Swindle (Il bidone, 1955). The story is about an old, tired
crook, who suddenly decides to become a better man and start anew. After
the last swindle, he tries to cheat his accomplices out of a large sum of
money. They find out his plans, however, beat him unconscious, and leave
him to die on a mountain slope. The last thing he sees before he dies is a
group of women and children passing by on the road, carrying bundles of
sticks on their backs. Bazin interprets this as angels pass, which he sees as
a typical element of Fellinis preoccupation with the angel metaphor (1971:
89) (figures 1314).
The two domains are the PEASANT WOMEN (target) and ANGELS (source)
respectively: WOMEN ARE ANGELS. Thus the source domain is absent, but is
cued by the similarity between the womens bundles of sticks on their backs
and angels wings. But it is also possible that Bazins religious interpretation
rests upon his knowledge of the works of Dante, for example Vita Nuova
and La Divina Commedia. The former deals with a mans wish to start
anew, as is the case in Fellinis film, and so does the latter, which begins
with the following words: In the middle of lifes road / I found myself in a
dark wood. La Divina Commedia has a geographical composition (hell,
purgatory, and paradise), which has a correspondence in the films journey
from the hectic city nightlife to the top of the mountain near heaven. This
can be compared to well-known conceptual metaphors like LIFE IS A JOUR-
NEY, DEATH IS DEPARTURE, and DIVINE IS UP (Lakoff and Turner 1989: 9
11, 150151). Dante is guided into paradise by Beatrice in the same way as
the group of women lead the crook to heaven after his last words: Wait, Im
coming. In early Christian belief, angels are regarded as messengers be-
tween humans and God, especially after death when every human is guided
to heaven by angels (Nationalencyklopedin [The National Encyclopedia]
1996: 408409). The crook is ready to meet the Lord. Moreover, when he
dies, non-verbal sounds of dry leaves scattering in the wind are heard, thus
connoting his death. But what is most important in this religious interpreta-
tion is the sound of the wind, which activates ideas of God. In the Bible, God
sometimes appears in disguise as the wind, for example in Genesis (8:1). To
summarize, I would categorize this scene from The Swindle as a multimodal
metaphor since speech and non-verbal sounds play such an important role in
identifying the absent source domain, angels.

Figure 13. Federico Fellinis The
Swindle ends, according to
Bazin, with a metaphor
Figure 14. showing no formal char-
acteristics on the surface level
cueing a metaphoric interpre-
tation.

It is interesting to note that this type of film metaphor is reconcilable with
the norms of invisible storytelling of classical Hollywood cinema. Bazins
metaphor example does not show any explicit deviation on the surface level
from a realistic motivation. Instead his interpretation is based on different
contextualizations which underpin the metaphoric interpretation, for exam-
ple, Fellinis earlier films, thematic elements, stylistic expressions, graphic
correspondences, and so on. What Balzs and Bazin have in common is that
they connect the identification of the metaphor with the style of the film.
Style has always been important in film due to the mediums technological
base, which makes the history of film an ongoing story of technological
changes in the form of improvements and innovations (e.g., sound film, film
420 Mats Rohdin
stock, moving camera, zoom lens, color film, cinemascope, stereo sound,
etcetera), which gives each period and each film a particular appearance.
Bazins way of using the concept of style is restricted and pertains mainly
to formal characteristics, that is, the repeated and salient uses of film tech-
niques characteristic of a single film or a group of films (Bordwell and
Thompson 2007: 481). Bazins opinion about style is subtle, and can be
compared to Arthur Dantos view on style attribution in The Transfigura-
tion of the Commonplace: A Philosophy of Art: The structure of a style is
like the structure of a personality. And learning to recognize a style is not a
simple taxonomic exercise. Learning to recognize a style is like learning to
recognize a persons touch or his character (1981: 207, emphasis mine,
MR).
3. Summary and concluding remarks
One of the most interesting findings in this chapter, I propose, is the impor-
tant role of intertitles in the canonic examples of film metaphors in the silent
cinema. Consequently, many of these examples qualify as multimodal meta-
phors (of the verbo-pictorial variety). This may be surprising considering
that the film medium during this period often was characterized as moving
images or visual Esperanto (e.g., Lindsay 1916: 177). Evidently, many
film scholars seem to have proceeded from the conviction that film meta-
phors of silent cinema had to be monomodal, that is, both target and source
were visuals. The reason for this is, of course, that the predominance of the
visual mode in film was taken for granted, which made the written text auxil-
iary by definition. But many of the metaphors taken from the silent era listed
above are to a high degree dependant on intertitles when spectators identify
and/or interpret these metaphors.
A possible explanation why early film theorists put so much emphasis on
the visual mode is that, in order to prove that film could be accepted as an
independent art, intertitles had to be downplayed. When early film theorists
wrote about metaphors they often presented them out of context, thereby
ignoring the intertitles, which often were of vital importance for identifica-
tion and/or interpretation of the metaphor. At the same time it should not
come as any surprise that, for example, metaphor experiments in Soviet
montage cinema were dependent upon intertitles, since the spectator needed
help to understand the logic of the story when suddenly superimpositions,
nondiegetic inserts, and unfamiliar camera set-ups were invading the screen.
With the breakthrough of sound film in the late 1920s it was possible to
include new modes in metaphorical compositions. Sound films like The Mil-
lion, Fury, Modern Times, and The Swindle show that speech, sound, and
music played an important role in both the identification and interpretation
of multimodal metaphors.
Often different modes interacted in order to make it easier for the specta-
tor to quickly identify and interpret the metaphor, which of course was im-
portant considering the irreversibility of the spectacle (the spectator can not
stop the film during projection in the theatre to think over or re-examine
obscure passages). Sound could also produce meanings that were difficult to
represent visually, for example in The Swindle, where the notion of God in
the barren mountain landscape is conveyed by the sound of the wind. Never-
theless, elements such as speech, sound, and music are often neglected in
early film theory when metaphors are discussed, as though the visuals alone
are sufficient to make up the metaphor in question.
Another pertinent question in this chapter is how target and source have
been presented in these examples. In most of them, both target and source
are visually present. An exception is the example from Metropolis, CINEMA
IS MOLOCH, where the target is supplied by extra-textual information. Symp-
tomatically, this is an interpretation from our time. My opinion is that this is
quite uncommon in classical film theory texts, but much more accepted to-
day, when reader-response theories have influenced criticism and interpreta-
tions in, for example, film studies at universities. In fact, much current
metaphor interpretation deals with how to find a target to match a source in
order to create a metaphor (Johnson and Malgady 1980: 266). This is also in
line with the view of metaphor/figurative processing as a reader strategy in
contexts which stress novelty and originality, for example, in academic writ-
ing (Gibbs 1994: 448449; see also Steen 1994).
The only metaphor example discussed in this chapter that draws on an
absent source, which is rare, is Fellinis WOMEN ARE ANGELS, where the
angels are implied by extra-textual interpretation. An alternative interpreta-
tion could be that the metaphor should be reversed, that is, angels are
women. This would correspond to the view that the source in linguistic
metaphors is always supplied by the description (the text) (Indurkhya
1992: 1819). Another suggestion for how to decide on target and source is
to choose as target the domain with the strongest denotation or which be-
longs to the story (Whittock 1990: 3132), that is, in the Fellini film, the
women. However, both Indurkhya and Whittock put forth principles that are
more prescriptive than descriptive, which blind them to the rich variety of
manifestations of metaphor where target and source are decided on different
422 Mats Rohdin
grounds (for example, when extra-textual knowledge is involved). Various
contexts make interpreters choose different ways of determining target and
source, without giving priority to a certain mode of procedure determined in
advance in the way Indurkhya and Whittock do. Therefore, in the Fellini film
my suggestion is that it is the women who are the target, and not the angels,
since we use the angels to predicate something about the women.
This chapter also shows the rich variety of different features that are
mappable from the source domain to the target domain. Since these exam-
ples deal with creative metaphors, the mappings may build on connota-
tions and contextual factors of all sorts, for example, other films, genre ex-
pectations, socio-cultural knowledge, etcetera (for questions concerning what
features are to be mapped, see Forceville 2005: 279).
Notable in these examples from classical film theory is that a majority of
the film examples are of the metaphor type CONCRETE A IS CONCRETE B.
This makes the type different from conventional verbal metaphors, which,
according to Lakoff and Johnson, are ABSTRACT A IS CONCRETE B. As
scholars have pointed out novel metaphors in art often belong to the former
type, that is, CONCRETE A IS CONCRETE B (e.g., Forceville 2006: 387). How-
ever, it is important to notice that the interpreter of a non-verbal multimodal
metaphor may influence the categorizing when suggesting a verbal rendering
of the metaphor. For example, in The End of St. Petersburg and Old and
New I proposed two abstract targets, MOTHER RUSSIA and AUTHORITIES,
instead of two more concrete targets that also would have been possible, for
example, THE RULERS and THE CIVIL SERVANTS respectively. The reason for
this did not rest upon any objective fact, but was motivated by my decision
to adhere to the vocabulary of Cognitive Metaphor Theory.
Another interesting finding is that many film metaphors in classical film
theory could be summed up in the conceptual metaphor MAN IS ANIMAL, for
example, spy and fox (Strike), workers and oxen (Strike), politician Keren-
sky and peacock (October), women and fowl (Fury), workers and sheep
(Modern Times). But also other common source domains theorized in Cogni-
tive Metaphor Theory (Kvecses 2002: 1620) are found in the film meta-
phors discussed in this chapter, for example, THE HUMAN BODY (The End of
St. Petersburg), MACHINES and TOOLS (Metropolis), GAMES and SPORT
(The Million), and MOVEMENT and DIRECTION (The Swindle). According to
Kvecses the source domain of animals is extremely productive (2002:
17). Are we to look for embodied or for cultural explanations? Many of
these examples come from the Soviet Union in the 1920s, where politicized
art and ideology went hand in hand, influenced by poster art, caricatures,
political cartoons, and so on. Also worth considering is that the fables of
writers like Aesop and La Fontaine have always been popular in authoritar-
ian societies like Russia and later the Soviet Union.
In the introduction of this chapter I posed three different questions,
which, according to Forceville (1996: 6566; 2002: 23; 2005: 266), have
to be answered: (1) Which are the two terms of the multimodal metaphor?
(2) Which is the metaphors target domain and which is the metaphors
source domain? (3) Which features are mapped from the source domain to
the target domain? Although my classification in this chapter mostly deals
with formal characteristics (for example, superimpositions, and nondiegetic
inserts), it has also shown that the question of how to identify the metaphor,
to quote Danto about style, is not a simple taxonomic exercise. Although
some sort of deviation or surface anomaly frequently holds, especially in the
early examples, it is not a necessary condition. Therefore, I agree with the
philosopher Ted Cohen when he asserts: If we have learned anything since
the explosion of interest in metaphor that began about twenty-five years ago,
it is that the classical and standard definitions of metaphor are unreliable,
and so every sensible person, I think, must suppose that we are dealing with
formally undefined phenomena (Cohen 1999: 399, emphasis mine, MR).
In contemporary metaphor theory, writers seem to agree that any linguis-
tic material can be used to make up a metaphor (e.g., Kittay 1989: 103;
Entzenberg 1998: xviiixxvii). In Rohdin (2003) I have demonstrated that
the same holds true for any filmic material. This is in accordance, I argue,
with Lakoff and Johnsons well known definition of metaphor: The essence
of metaphor is understanding and experiencing one kind of thing in terms
of another (Lakoff and Johnson 1980: 5, emphasis in original). However,
this should not prevent us from examining which filmic expressions have
been used in metaphorical interpretations throughout film history. Therefore
I think that corpus-based studies of metaphors in different genres, move-
ments, national cinemas, and among individual directors, would yield inter-
esting results. Worth remembering when dealing with questions like these is
Forcevilles critique of Nol Carrolls restricted metaphor classification:
Categorizing and name-giving should follow analysis of phenomena, and
not the other way round (2002: 9).
Finally, when the history of non-verbal metaphors is outlined, references
often start with I. A. Richards The Philosophy of Rhetoric: Thought is
metaphoric, and proceeds by comparison, and the metaphors of language
derive therefrom (1965: 94, emphasis in original). But it is interesting to
note that the term metaphor already had been used for several decades in
film theory since the 1910s (for example, Lindsay 1916: 172; Epstein 1974
vol. II, 68). It is important to remember that with film we have a new me-
424 Mats Rohdin
dium and art form whose development we have been able to monitor from its
birth, in the late 1890s, which makes it exceptional compared to other, older
art forms like theatre, literature or painting. This not only gives us important
knowledge about the film medium itself, and how it has changed over time,
but also demonstrates that metaphor, including what we today call multimo-
dal metaphor, had been introduced, discussed, questioned, and, lastly, estab-
lished in a non-verbal art such as film, from its earliest beginnings.
Acknowledgment
I would like to thank Charles Forceville for insightful comments on an earlier
draft of this chapter.
References
Arnheim, Rudolf
1933 Film [1932]. L. M. Sieveking and Ian F. D. Morrow (transl.), Lon-
don: Faber and Faber.
1997 Sound film by force [1931]. In Film Essays and Criticism [1977],
3942. Brenda Benthien (transl.), Madison/London: The University
of Wisconsin Press.
Balzs, Bla
1930 Der Geist des Films. Halle: Verlag Wilhelm Knapp.
1952 Theory of the Film: Character and Growth of A New Art [1945],
Edith Bone (transl.), London: Dennis Dobson.
Bazin, Andr
1967 The evolution of the language of the cinema [195055]. In What is
Cinema? Vol. I [195859], Hugh Gray (ed. and transl.), 2340.
Berkeley/Los Angeles/London: University of California Press.
1971 Cabiria. The voyage to the end of neorealism [1957]. In What is
Cinema? Vol. II [196162], Hugh Gray (ed. and transl.), 8392.
Berkeley/Los Angeles/London: University of California Press.
1973 The French Renoir. In Jean Renoir [1971], Franois Truffaut (ed.),
W. W. Halsey II and William H. Simon (transl.), 7491. New York:
Simon and Schuster.
1981 On LEspoir, or style in the cinema [1945]. In French Cinema of
the Occupation and Resistance: The Birth of a Critical Esthetic
[1975], Franois Truffaut (ed.), Stanley Hochman (transl.), 145
157. New York: Frederick Ungar.
Black, Max
1993 More about metaphor. In Metaphor and Thought. 2nd ed. Andrew
Ortony (ed.), 1941. Cambridge: Cambridge University Press.
Bordwell, David
1993 The Cinema of Eisenstein. Cambridge, MA/London: Harvard Uni-
versity Press.
Bordwell, David, and Kristin Thompson
2007 Film Art: An Introduction. 8th ed. Boston: McGraw Hill.
Buckland, Warren
2000 The Cognitive Semiotics of Film. Cambridge: Cambridge University
Press.
Carroll, Nol
1996a A note on film metaphor. In Theorizing the Moving Image, 212
1996b Language and cinema: Preliminary notes for a theory of verbal im-
ages [1980-1981]. In Theorizing the Moving Image, 187211. Cam-
bridge: Cambridge University Press.
Clair, Ren
1972 In the beginning was the image. In Cinema Yesterday and Today, R.
C. Dale (ed.), Stanley Appelbaum (transl.), 4080. New York: Do-
ver.
Cohen, Ted
1997 Metaphor, feeling, and narrative. Philosophy and Literature 21 (2):
223244.
1999 Identifying with metaphor: Metaphors of personal identification.
Journal of Aesthetics and Art Criticism 57 (4): 399409.
Danto, Arthur C.
1981 The Transfiguration of the Commonplace: A Philosophy of Art.
Cambridge, MA/London: Harvard University Press.
1992 Metaphor and cognition. In Beyond the Brillo Box: The Visual Arts
in Posthistorical Perspective, 7387. Berkeley/Los Ange-
les/London: University of California Press.
Ejxenbaum, Boris
1981 Problems of cinema stylistics [1927]. In Russian Formalist Film
Theory, Herbert Eagle (ed.), 5580. Ann Arbor: University of
Michigan.
Eisenstein, Sergei
1974 Zur Komposition des Streik-Finale [1933]. In Schriften 1: Streik,
Hans-Joachim Schlegel (ed.), 274276. Mnchen: Carl Hanser.
1987a Nonindifferent nature [1945]. In Nonindifferent Nature, Herbert
Marshall (transl.), 216396. Cambridge: Cambridge University
Press.
426 Mats Rohdin
1987b Pathos [1947]. In Nonindifferent Nature, Herbert Marshall (transl.),
1991 Montage 1937 [1937]. In Selected Works. Vol. II, Towards a Theory
of Montage, Michael Glenny and Richard Taylor (eds.), Michael
Glenny (transl.), 1158. London: British Film Institute.
1996 Dickens, Griffith, and ourselves [1942]. In Selected Works. Vol. III,
Writings 193447, Richard Taylor (ed.), William Powell (transl.),
193238. London: British Film Institute.
Eisenstein, Sergei, Vsevolod Pudovkin, and Grigori Alexandrov
1988 Statement on sound [1928]. In Sergei Eisenstein, Selected Works.
Vol. 1, Writings 19221934, Richard Taylor (ed. and transl.), 113
114. London/Bloomington: BFI/Indiana University Press.
2003 Understanding visual metaphors: The example of newspaper car-
toons. Visual Communication 2 (1): 7595
Entzenberg, Claes
1998 Metaphor as a Mode of Interpretation: An Essay on Interactional
and Contextual Sense-Making Processes, Metaphorology, and Ver-
bal Arts. Ph.D. diss., Uppsala University.
Epstein, Jean
1974 crits sur le cinema 19211953. 2 vol. Paris: Seghers.
Forceville, Charles
2006/ this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
Hake, Sabine
1993 The Cinemas Third Machine: Writing on film in Germany 1907
1933. Lincoln, NE/London: University of Nebraska Press.
Indurkhya, Bipin
1992 Metaphor and Cognition: An Interactionist Approach. Dordrecht:
Kluwer.
Jakobson, Roman
1987 Is the film in decline? [1933]. In Language in Literature, Krystyna
Pomorska and Stephen Rudy (eds.), 458465. Cambridge
MA/London: Harvard University Press.
Johnson, Michael G., and Robert C. Malgady
1980 Toward a perceptual theory of metaphoric comprehension. In Cog-
nition and Figurative Language, Richard P. Honeck, and Robert R.
Hoffman (eds.), 259282. Hillsdale, NJ: Lawrence Erlbaum.
Kittay, Eva Feder
1989 Metaphor: Its Cognitive Force and Linguistic Structure. Oxford:
Clarendon Press.
Kvecses, Zoltn
2002 Metaphor: A Practical Introduction. New York: Oxford University
Press.
cago/London: University of Chicago Press.
Lindsay, Vachel
1916 The Art of the Moving Picture. New York: MacMillan.
Nationalencyklopedin
1996 Nationalencyklopedin [The National Encyclopedia] VITRY.
Hgans: Bokfrlaget Bra Bcker.
Richards, I(vor) A(rmstrong)
1965 The Philosophy of Rhetoric. Oxford: Oxford University Press.
Rohdin, Mats
2003 Vildsvinet i filmens trdgrd. Metaforbegreppet inom filmteorin
[The wild boar in the garden of film: The concept of metaphor in
film theory] Summary in English 318329. Ph.D. diss., Stockholm:
Edita Norstedts Tryckeri AB.
Salt, Barry
1992 Film Style and Technology: History and Analysis. 2nd

ed. Starword:
London.
Stanford, W(illiam) B(edell)
1936 Greek Metaphor: Studies in Theory and Practice. Oxford: Basil
Blackwell.
Steen, Gerard
1994 Understanding Metaphor in Literature: An Empirical Approach.
London/New York: Longman.
428 Mats Rohdin
Tynjanov, Jurij
1981 On the foundations of cinema [1927]. In Russian Formalist Film
Theory. Herbert Eagle (ed.), 81100. Ann Arbor: University of
Michigan.
Ward, Janet
2001 Weimar Surfaces: Urban Visual Culture in 1920s Germany. Berke-
ley/Los Angeles: University of California Press.
Whittock, Trevor
Chapter 18

Multimodal expressions of the HUMAN VICTIM IS
ANIMAL metaphor in horror films
Gunnar Theodr Eggertsson and Charles Forceville
Abstract
In this chapter, we analyze the structural metaphor human victim is animal in
three modern horror films, and argue that the metaphor is characteristic of the
horror genre. It is shown that its pervasive manifestations, in both monomodal
and multimodal form, create what Musolff (2006) calls a metaphor scenario. In
the concluding section we discuss some implications of our study for metaphor
theory, genre, and animal rights.

Keywords: structural metaphor, horror film, animal rights
1. Introduction
Within the medium of film, or rather moving images in general, scholarly
studies focusing specifically on non-verbal and multimodal metaphor are still
relatively rare (Whittock 1990; Rohdin 2003, this volume; see also Carroll
1996; Forceville 1999a, 2003, 2005, this volume). These latter studies,
moreover, discuss metaphors that are at the creative end (see Black 1979)
rather than at the structural end of the continuum (Lakoff and Johnson
1980). However, assessing the validity of the Conceptual Metaphor Theory
(CMT) paradigm launched by Lakoff and Johnson requires not only that
non-verbal and multimodal expressions of conceptual metaphors are studied
as such, but also that the extent of their systematic occurrence is investi-
gated. After all, a central tenet of CMT is that human beings conceive cer-
tain phenomena systematically in terms of certain other phenomena, allow-
ing for numerous different surface manifestations of a single conceptual
metaphor. It is suggested in Forceville (2006a/this volume) that examining
multimodal metaphors (as indeed many other discourse phenomena) is best
430 Gunnar Theodr Eggertsson and Charles Forceville
done by focusing on specific genres, since the meanings that can or must be
gleaned from representations are strongly steered and constrained by the
audiences awareness of the genre to which a discourse belongs (Forceville
1996, 1999b, 2006b). The current chapter aims (a) to highlight the impor-
tance of the structural metaphor HUMAN VICTIM IS ANIMAL in the modern
horror film; (b) to demonstrate how this metaphor is expressed multimo-
dally, and to formulate some tentative generalizations about the surface
manifestations of the metaphor.
2. What should count as a multimodal metaphor?
Before addressing the central metaphor itself, we want to clarify briefly our
use of the labels multimodal and monomodal metaphor. According to
Forceville (2006a), monomodal metaphors present target and source do-
mains both in the same mode. In film, the list of modes includes at least:
visuals, written language, spoken language, sound, and music. Although this
is not an exhaustive list (Mittelberg and Waugh this volume and Mller and
Cienki this volume extensively discuss the gestural mode), we will in the
present chapter restrict ourselves to these. The best-known variant of mono-
modal metaphor is verbal metaphor, which until Lakoff and Johnson (1980)
was considered the only type of metaphor anyway. Another variant of
monomodal metaphor is the pictorial one, which cues both target and source
visually (Forceville 1996). By contrast, multimodal metaphors are meta-
phors whose target and source are each represented exclusively or predomi-
nantly in different modes (Forceville 2006a: 384). A pure multimodal
metaphor thus presents a target in one mode/modality, and one mode only,
and the source in another mode/modality only. But such pure multimodal
metaphors are distinguished for analytical purposes only: there is nothing
inherently better about pure than about impure multimodal metaphors.
Indeed, as acknowledged by Forceville (2006a: 385), probably by far the
majority of multimodal metaphors in moving images cue target and/or
source in more than one mode simultaneously (we could label these multi-
modal metaphors in the broad sense, as opposed to multimodal metaphors in
the strict or narrow sense). The broad definition of multimodal metaphor is
particularly pertinent if a metaphor is not a creative metaphor that surfaces
only once in a multimodal text, but is an expression of a conceptual meta-
phor that keeps appearing throughout a narrative or argument. In Musolffs
(2006) terminology, HUMAN VICTIM IS ANIMAL would be a scenario.
1
In
such a situation, the same conceptual metaphor can well be expressed in a
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 431
variety of ways in each individual instantiation. That is, in one instance it
can surface as a monomodal metaphor of the verbal variety, in the next as a
narrowly defined multimodal metaphor of the pictorial-sonic type, and in the
third as a broadly defined multimodal metaphor with a target that is cued in
visual terms only and a source that is cued primarily verbally, but with some
musical underpinning as well.
It would of course have been possible to assess for each individual mani-
festation of the HUMAN VICTIM IS ANIMAL metaphor in the three films to be
discussed (a) whether it exemplifies a monomodal or a multimodal meta-
phor; (b) if the latter, whether it is a multimodal metaphor in the strict or in
the broad sense; and (c) which mode(s) is/are involved in the target and the
source domains, respectively. But since, as we claim, the HUMAN VICTIM IS
ANIMAL is a structurally occurring metaphor, the modes in which target and
source of the metaphor are cued are bound to vary across different instantia-
tions (for a similar situation, see the discussion of the structural metaphor
COLIN IS A CHILD in Forceville [1999a]), to avoid tedious cataloguing we
will not systematically list the various types. However, in the concluding
section we will make some tentative general observations about this issue.
3. The modern House of Horror
The year is 1957 and in the small deer hunting community of Plainfield,
Wisconsin, something truly horrifying is waiting to be discovered. Investiga-
tors will enter the house of one Edward Theodore Gein to find the remains of
numerous female bodies, including the woman they were looking for, freshly
killed Bernice Worden. The house will quickly be dubbed the Gein House
of Horrors by the press as reports of strange artistic experiments and dis-
turbing uses of the human body become apparent. Bowls made of human
skulls, human skin suits, furniture upholstered by body parts and a necklace
made of nipples are a few examples of the horrid imagery presented to those
willing to know about such things. The case of Ed Gein becomes infamous
in its time and a subject of infinite conversation and shock among the public.
Atrocious crimes are no longer the subject of far-away places or past reali-
ties. The realization that danger can lurk in the neighborhood and that not
everyone can be trusted becomes the new reality. Ed Gein constitutes the
prototypical serial-killer-next-door and goes on to directly inspire such hor-
ror film legends as Norman Bates, Leatherface, and Buffalo Bill. In fact, the
story becomes a source and influence for at least three generations of horror
cinema, ranging most notably from Psycho (Hitchcock, USA 1960) to The
Texas Chain Saw Massacre (Hooper, USA 1974) and The Silence of the
Lambs (Demme, USA 1991). Other horror films influenced by the Gein
story include Three on a Meathook (Girdler, USA 1972), Deranged (Gillen
and Ormsby, Canada/USA 1974), Maniac (Lustig, USA 1980), Ed Gein
(Parello, USA 2000), and House of 1000 Corpses (Zombie, USA 2003).
Horror scholars generally see the release of Hitchcocks Psycho as marking
the shift from classic to modern horror film. The monsters populating classic
horror films were always foreign in a way mummies, vampires, aliens
but after checking in at Psychos Bates Motel the home became the new
territory for horror. Reynald Humphries claims about modern horror that
the element that needs to be stressed is that we are dealing with the modern,
everyday world, not with one situated in the past. Gone are the mad scien-
tists, the remote islands and settings, the dangers of invasion and radiation
(Humphries 2002: 85). This element can be directly traced to the films that
sprang from the obsession with the case of Ed Gein, where horror found a
new home right next door in Anytown.
But there is more to the Gein influence than meets the eye. By the time of
his death in 1984, Gein had already become a legend in horror folklore, an
inspiration to decades of horror cinema and revered by horror buffs as the
prototype of every knife-, axe-, and cleaver-wielding maniac who has stalked
Americas movie screens for the past thirty years (Internet Zombie 2006,
http://www.houseofhorrors.com/gein.htm last accessed 11.05.07). On the
surface, this seems very obvious. Not only did he bring terror into suburban
reality, and cast a shadow of doubt and paranoia over even the most lovable
neighbor; his violent crimes and methods have been recreated and imitated in
countless cheap horror flicks since the 1960s. But there is a more deep-
rooted argument to be made regarding the extent of Geins crimes one that
will be inspected more closely in this chapter. Sure enough, the shock of
such atrocious and horrific crimes taking place in the peaceful rural commu-
nity had an extreme and long-lasting effect not only on the evolution of hor-
ror art but on the American psyche in general. This shock derives from a
very particular aspect of his violent crimes something that provided the
base for the publics extremely horrific reactions to his actions, and that was
responsible for a crucial element in horror cinema ever since: Ed Gein
treated human beings in precisely the same way human beings treat their
non-human counterparts. For the purpose of this chapter, this statement can
be rephrased as: Ed Gein created a metaphor that has been embraced and
recycled by horror cinema ever since investigators came upon the gutted,
decapitated body of Mrs. Worden hanging upside down in Geins House of
Horrors back in 1957. This metaphor went unnoticed at the time but after
fifty years of roaming wild and pervading the genre it is time to finally ana-
lyze it: HUMAN BEING IS ANIMAL,
2
or to be more descriptive: HUMAN VICTIM
IN HORROR FILM IS ANIMAL USED AS RESOURCE.
4. Metaphorical vulgarity
The idea of human victims presented as animals is not necessarily anything
new in the history of horror. After all, horror is about fear, and fear derives
from something that is out of ones control. A very natural thing for us to
fear is something that has power over us such as a being that holds a
higher place in the food chain. It used to be the case that the human race had
to fear other predators, but those days are long since gone (at least in con-
temporary urban societies) and we have placed ourselves, self-assuredly, on
top of the food chain. No one feeds on us anymore, so ultimately a great deal
of fear comes from the idea that something unexpected can jump in and eat
us. Werewolves, vampires, atomic mutations and maggot-infested living
dead corpses immediately spring to mind, but these monstrosities are not
relevant to the subject of this chapter, as there is no metaphor in how they
are presented. If a monster is hunting a human being for food, then the hu-
man is quite simply a non-metaphorical prey an animal subjugated to a
higher species that fits into the natural chain of events. But if a human
being is hunting another human being, then something different happens. The
victim becomes a metaphorical prey and the killer a metaphorical predator.
Therefore this chapter will concentrate on that special group of serial
killers and madmen in horror films which does not draw any distinctive line
between killing an animal and killing a human being. The focus will be on
three films where human beings abuse other animate creatures, treating them
as animals. The attacker considers his victim purely in terms of utilitarian
value; he is after the victim for its flesh or bones or because he wishes purely
to indulge his sadism or desire to kill. In other words, the human victims
have their status as social human beings reduced to the status of non-human
animals in Western society an expendable resource. If monster movies
were to be included, obvious justifications would be present for most of the
violence in cases where the attackers reasons can be explained away simply
as acting on impulse or by natural instinct in terms of, for instance, self-
defense, territorial interest, or nourishment. Even in cases of monsters that
appear to be very human in nature, such as humanoid vampires, the justifi-
cation can be a purely biological one they are creatures that can not sus-
tain themselves except on blood. The same goes for instance for the crea-
tures from the Alien (Scott, UK 1979) cycle, who need to kill other life
forms in order to procreate. When human beings attack other human beings,
the actions are more difficult to justify.
In many cases hardly even an attempt is made in the films themselves to
provide proper justification. The genre of extreme horror is one famous for
poorly constructed narratives, usually existing solely in order to string to-
gether different scenes of gratuitous violence where the pleasure of the killer
is shared by the audience rooting for progressively elaborate effect sequences
of blood and gore. This has led to the claim that it is in fact the special ef-
fects crew that ranks first, above director, screenwriter, producer, or actors,
in such films. The subgenre of extreme horror is furthermore an interesting
field for metaphor studies, as it is generally considered to be of lesser aes-
thetic quality than the more conservative horror movie that style of hor-
ror which actually aims at scaring the audience, keeping up a reasonable
storyline, and generally leaning towards some sort of intellectual process in
the viewing experience. The extreme horror movie, on the other hand, fo-
cuses purely on the bodily experience: the more body-harassing details are
provided, the more clearly does the film belong to the genre. This constitutes
a great part of why these movies are generally frowned upon by the advo-
cates of proper cinema which tends not to show everything. It is always
more in the style of fine arts to metaphorize instead of vulgarly presenting
the subject. But in the genre of extreme horror, metaphors abound be they
intentionally created or not.

4.1 The mapping of the hunt: Wolf Creek
In Wolf Creek (McLean, Australia 2005), three backpackers are driving
through the Australian outback. After spending an afternoon by the Wolf
Creek crater they return to find their car broken. A man comes by on a truck
and offers to tow their car back to his place where he can fix it for them. The
group is hesitant, but ultimately decides to accept his offer. The man, Mick,
drives them in the dark of night to his secluded home, where they sit together
and chat for a while before going to sleep. It is through this evenings dis-
cussion that the group discovers he used to be a professional shooter
getting rid of vermin for clients, such as kangaroos, horses, pigs, buffalo,
you name it, he proudly exclaims. But this veteran hunter has been forced
out of his job because people use poison nowadays instead of hiring shoot-
ers. Therefore he has taken it upon himself to hunt people for his own pleas-
ure the backpackers do not know that yet, but they will soon find out. Dur-
ing their conversation Mick describes different ways of hunting and killing
animals and when one of the girls asks him if he really kills kangaroos he
replies that he is in fact doing people a favor, making the following explicit
analogy: theyre everywhere out here now like tourists! The HUMAN IS
ANIMAL metaphor in Wolf Creek comes from different verbal references to
kangaroos, Micks evening speech about hunting, and from visual cues in the
latter half of the film, which mainly focuses on Mick hunting down two of
the escaped backpackers.

Figure 1. The victims of Wolf Creek are shown driving, through a bullet hole in a
kangaroo warning sign, as part of a running reference to the characters
as kangaroos to be hunted.
The most obvious way the metaphor HUMAN IS ANIMAL surfaces is through
the contextual information of Micks hunting background, combined with his
weapons of choice (including a hunting knife and rifle) and his method of
trapping and caging his prey: he tampers with peoples cars and then offers
them help, thereby luring them to his home where he subsequently drugs his
victims. In a telling scene, one of the victim girls stumbles upon a room full
of hunting memorabilia, including video footage of other people having
fallen prey to Mick in precisely the same way. After trapping and caging the
backpackers, he proceeds to torture and kill them as he sees fit to appease
his sadistic desires. And even though he does not intend for his victims to
escape, he is well prepared for that possibility, thus turning the game into a
proper hunt. When the two girls manage to flee he goes on the road to pursue
and kill them. In a similar manner to most cinematic serial killers of his
genre, Mick gets his first victim up close and personal (Clover 1992: 3132)
when he cuts her up with a big hunting knife, mutilating her hand and sever-
ing her spinal chord in order to disable her. But he steps away from his genre
colleagues when he pulls out his hunting rifle a weapon made to use at a
distance. The second girl manages to flee out to the highway, where she is
picked up by an old man whom Mick shoots from far away. The girl takes
the car and a chase ensues. After having been thrown off the road, Mick
shoots her tires with the rifle. Even as he arrives at the crash site and finds
her still alive, he does not kill her up close but ultimately shoots her from a
few feet away.

Figures 2a and 2b. Kangaroos jump around the outback. The last survivor is
cross-cut with the kangaroos to further emphasize his role as a hunted,
wounded animal (Wolf Creek).
There are a few other hints to bring out the animal metaphor, given through-
out the film as part of a running reference to the characters as kangaroos to
be hunted. Very early on, the group of victims is filmed through a bullet-hole
in a kangaroo warning sign (figure 1). A close-up of the road sign fills the
screen, with the focus on the victims car in the background, seen through
the hole, driving down the road. The focus then quickly shifts to the fore-
ground, showing viewers not only that the sign portrays a shaded image of a
kangaroo, but that the hole in question has been made by a bullet. Through a
juxtaposition of the car, the bullet hole, and the kangaroo image, the specific
metaphor HUMAN VICTIMS ARE KANGAROOS is first introduced. In addition
to Micks explicit reference to tourists (i.e., the backpackers) as kangaroo
vermin, there is another juxtaposition involving the kangaroo domain later in
the film. After being trapped by Mick, the third protagonist victim is not
shown again until after the two girls have been killed. He is nailed to a cross
next to a locked pen of starved dogs, sharing the room with the corpse of a
previous torture victim (eaten, presumably alive, from the waist down). He
manages to free himself and escape into the wild, wandering around until he
is picked up by travelers. It is in these final scenes of the film that for the
first time real-life kangaroos are finally shown roaming the outback, juxta-
posed with the last survivor to further emphasize his role as a hunted,
wounded, and lost animal (figures 2a and 2b).
The metaphor HUMAN VICTIMS ARE KANGAROOS surfaces in two ways
monomodally through the use of language (during Micks speech) and
monomodally through visual juxtaposition (the road sign and the kangaroo
herd). Furthermore, the more general metaphor HUMAN IS ANIMAL is created
through the context of Micks hunting background and the unabashed nature
of his murders, in which the victims could easily be substituted by an animal
and the serial killer by a sports hunter. From this short description it should
be clear that Mick employs explicit methods of animal game hunting, using
terms of trapping, chasing and killing his victims with popular hunting
weapons. Nevertheless he gains no utility from their bodies other than the
pleasure and thrill of the kill he is shown either burning the victims or leav-
ing them to hang and rot in his home. It is possible that he uses the material
possessions stolen from the victims as a source of income, but from what is
shown on-screen it seems that their sole purpose is to substitute for hunting
trophies. Furthermore, stealing would hardly justify the torture and killing
he could easily do that without resorting to his violent methods. Thus he
becomes reminiscent of animal game hunters, where the object of desire is
not the victim itself, but the pleasure gained from the sport of the hunt. And
if Mick the killer constitutes part of the metaphor SERIAL KILLER IS SPORTS
HUNTER, then his victims surely fall into that category of ultimate game
hunting: HUMAN VICTIM IS ANIMAL RESOURCE.

4.2 The mapping of caging: Calvaire/The Ordeal
Calvaire/The Ordeal (Du Welz, Belgium/France/Luxembourg, 2004) pro-
vides a very vivid example of a human being whose rights are violated to
satisfy the most whimsical desires of a fellow human being. The HUMAN IS
ANIMAL metaphor surfaces strongly through the more specific metaphor
PERSON IS CAPTIVE ANIMAL, where the character of Marc is so presented in
order to further emphasize his defenselessness towards his captor. In other
words, by understanding Marc in terms of a trapped animal, the viewer can
become even more sympathetic towards the character it adds to the terror
of Marcs atrocious ordeal and taps into the idea that horror films systemati-
cally explore an inherent fear in humans to be treated in the same way as we
treat animals. Therein lies the true terror of Calvaire.
Cabaret singer Marc Stevens is traveling alone when his van breaks down
in the middle of rural nowhere. He meets Boris, who is out looking for his
dog in fact, he has been looking for his dog for a very long time and seems
quite insane and brings Marc to innkeeper Paul Bartel, who offers him a
place to stay and a helping hand. Bartel is especially interested in Marcs
role as a singer because his wife Gloria, who has left him long ago, was also
a singer. In a hauntingly beautiful scene Marc agrees to sing a song for
Bartel, hoping to get on his way the following morning. But Bartel becomes
convinced that Marc is his lost wife returned, destroys Marcs car, and takes
the singer captive in his home, all the while pretending that Marc is his wife.
He shaves Marcs head (figure 4c), dresses him up in her old clothes, and
keeps him in his bed at night. As the days go by a group of villagers start
showing interest in the affairs at the old inn. They also become convinced
that Marc is Gloria returned and subsequently attack Bartels inn, killing
both Boris and Bartel and attempting to gang-rape Marc, who manages to
escape into the woods. In the final scenes the camera leaves Marc alone in
the woods and takes off to wander aimlessly until the end credits roll per-
haps implying that there is no turning back for our protagonist.
The first example of the animal metaphor is found in the character of Bo-
ris, who is constantly seeking out his dog, Bella. There is an explicit analogy
between Boris and his long lost dog on the one hand, and Bartel and his long
lost wife on the other. The two characters are regularly juxtaposed visually
and both are clearly out of their minds. This analogy is further emphasized
later in the film, when Marc has become stuck in a woodland trap after a
failed escape attempt. With his hands tied together, he is completely unable
to move and lies there, crying, trapped until nightfall. Boris stumbles upon
him, but instead of helping out, he simply sits down and starts stroking his
head and talking about his dog, treating Marc as if he were Bella incarnate.
Marc pleads with Boris to untie him but gets no answer and as if their
positioning is not reminiscent enough of a man and his dog, Marc bites Boris
in the thigh in a desperate act of frustration. Here the specific metaphor
MARC IS BELLA or HUMAN IS DOG surfaces as part of the more inclusive
HUMAN IS ANIMAL metaphor (figures 3a-c).

Figures 3a-c. From top to bottom: Marc ensnared in a trap in the woods; Boris
treats the trapped Marc as if he were his lost dog; Marc is hunted in the
final chase through the woods (Calvaire).
Another manifestation surfaces relatively early in the film, with the added
sexual implication of Marc as an object of desire, when Marc takes a walk
in the woods and comes across a farming area where he hears animal noises
coming from a nearby stable. Silently he sneaks towards the sounds and
spies on a family of male villagers the same ones that will raid the inn later
on and attempt to rape Marc standing around one family member who is
getting his genitals licked by a calf. There is also a pig present and a cow
looking on. A few small dead animals hang from the stable rafts behind the
men, who are all enthusiastically watching their relative lying on the stable
floor getting a bestial blowjob to the sounds of their squealing pet pig. It is a
clear case of an innocent animal being used for sexual pleasure in the same
way as Bartel and these same villagers will use Marc later on. In a way,
as Marc peeps through the wooden boards of the stable, he is looking into
his own future near-fate: to be sexually exploited by a bunch of crazy men
(figures 4a, b). The calf is the source domain of a premonitory metaphor
MARC IS CALF derived from the larger animal metaphor and created from
the contextual information of what lies ahead for Marc (Bartel taking him to
bed and dressing him up, the villagers rape attempt), the visual juxtaposi-
tion of Marc and the calf at the stable, as well as the sounds of the farm
animals during the stable scene, most notably the pig squealing.

Figures 4a-c. From top to bottom: Marc peeks into the barn and into his own
future; the villagers within take sexual pleasure from one of the farm
animals; Marc is shorn by Bartel.
The final noteworthy sub-metaphor is that of the trap and the hunt. During
his failed escape attempt, Marc is caught in an animal trap, as previously
mentioned, and subsequently brought back to his captor, Bartel, and returned
to his cage, where he is locked up, constantly watched over, and subjected
to cruel and degrading treatment. When Boris drives Marc back to Bartel
after freeing him from the trap in the woods (with Marc lying on the back of
the tractor like a hunted animal), they pass two villagers who are in the
process of gutting a boar they have hunted, creating a graphic hunting juxta-
position. Furthermore, during the second escape attempt after the villagers
have killed both Boris and Bartel the viewer is again reminded of a hunt,
where the villagers bear hunting rifles and use their pigs as a tracker or
bloodhound. In the end, Marc is left alone in the woods, completely lost and
out of place in the world, much like the last survivor of Wolf Creek. These
scenes all hint at the metaphor MARC IS HUNTED ANIMAL.
In each manifestation of these sub-metaphors Marc is presented in terms
of animals that are in human captivity thus creating the metaphor MARC IS
CAPTIVE ANIMAL. By presenting the protagonist in such a way, certain ele-
ments become highlighted for his character portrayal. He is defenseless,
suffering, subjugated, and stripped of freedom, but most importantly his life
and well-being depend solely on the will and desire of his captor. All of these
features belong to the social status of nonhuman animals in contemporary
society but they do not usually apply to human beings at least not in a
society that bases its legislation on universally declared human rights.

4.3 The mapping of slaughter: The Texas Chain Saw Massacre
Few horror films deal as explicitly with slaughter or are as full of references
to meat as the original The Texas Chain Saw Massacre (Hooper, USA
1974). Fictionally represented as being based on a true story, it is famously
inspired, at least in part, by the Gein murders. The theme of slaughtering
human beings like animals and putting their bodies to use (e.g., as food,
clothing, and decoration) resonates throughout the film as the metaphor HU-
MAN VICTIM IS ANIMAL RESOURCE comes to life in numerous scenes. The
storyline has become horror movie stock material. Five friends head into the
countryside to visit an old family house that is currently uninhabited. Upon
arrival, one by one they fall prey to the cannibalistic family next door, until
the last surviving girl, Sally, manages to escape into the open road and gets
picked up by a truck driver and wheeled away to safety. The cannibal family
consists of two brothers, their father, and their grandfather, whose business
used to be slaughtering cattle. However, similar to Wolf Creeks killer Mick,
they have lost their jobs due to new farming technologies. The father works
as a gas station attendant and the two boys seem to be employed only to rob
graves and hunt people and animals for the family unit. Grampa is long re-
tired, but retains the claim to fame as one of the greatest butchers this rural
community ever had.
The first shot to greet viewers after the opening credits is that of an ar-
madillo road kill in the street. Driving past the dead armadillo the youngsters
are filmed heading to their imminent doom on the open road. The juxtaposi-
tion clearly mirrors that of the kangaroo warning scene in Wolf Creek and
serves as part of the running HUMAN VICTIM IS ANIMAL RESOURCE metaphor
or in this case, HUMAN VICTIMS IN CAR ARE DEAD ARMADILLO IN STREET.
As they head down the road they pass the old slaughterhouse, at which point
Franklin, whose relatives used to own the house they are driving towards,
reminisces about how his grandfather used to sell his cattle to that place. He
describes old-fashioned cattle slaughter methods in vivid detail, involving
bashing in heads with a large sledgehammer. Although the audience does not
yet know it at this stage unless they are alert to extreme horrors genre
conventions here the abused-animal story helps build up the metaphors
source domain, since the head-bashing will be acted out on some of the hu-
man victims as the film progresses. This lends support to the specific meta-
phor HUMAN VICTIMS ARE CATTLE, which is emphasized by crosscutting the
youngsters discussion with images of cows in small stalls, huddled together
on their way to slaughter, as the car drives past (figure 5a-f). Thus the meta-
phor is monomodally presented through the use of dialogue and visual juxta-
position, respectively, and multimodally through verbal and visual references
to killings with sledgehammers. The scene at the films end, when Sally is
held captive with her head over a basin, and the legendary butcher grandfa-
ther makes vain attempts to bash her head in, is particularly reminiscent of
old-fashioned cattle slaughter, precisely reproducing common procedures as
described by Franklin in the car.

Figures 5a-f. Discussions in the car about slaughtering, cross-cut with cattle (The
Texas Chain Saw Massacre).
Another gruesome scene involves the first two murders of the friends, shortly
after they arrive at the country house. A couple Kirk and Pam go knock-
ing on the cannibal house next door looking for gasoline. Pam stays outside
in a swing as Kirk enters the house, after getting no answers from their
knocking. Inside, a wall is decorated with animal remains and on the sound-
track the constant noise of a squealing pig can be heard. When Kirk enters
the opposite room, Leatherface, one of the brothers, pounds his head with a
sledgehammer. Kirks body twitches and writhes like a freshly slaughtered
animal as Leatherface hits him repeatedly and drags him into the room,
which closes with a heavy steel door, reminiscent of an industrial slaughter-
house. When Pam follows to investigate she goes into a different room where
she falls into a pile of animal bones. The whole room is decorated with ani-
mal remains (human and nonhuman alike, presumably) and to further em-
phasize the metaphor there is a live chicken hanging in a small cage, cluck-
ing as it anxiously tries to move around its confined space. The room is filled
with macabre artworks, such as a couch made of bones and fetishized human
skulls, and shots of animal remains are crosscut with the nervous chicken
and the shocked Pam, who subsequently becomes nauseous. As she stumbles
back out of the room Leatherface emerges to pull her into the slaughter
room, where he sticks her up on a large meat hook hanging from the ceiling.
He lets her hang there, screaming and convulsing, as he begins to cut up her
boyfriends body with a chain saw. Finally, when their friend Jerry comes
looking for them he follows a strange sound into the slaughter room, resonat-
ing from within a big icebox. Upon opening it he finds Pam still twitching,
her body being kept in cool storage. Suddenly Leatherface reappears and
kills Jerry with the sledgehammer.
The way the violence is presented from the telling descriptions of
slaughter by sledgehammer to the storing of meat in an icebox is all done
in terms of animal treatment. Furthermore, it remains obvious through
crosscutting (the dead armadillo, the pent-up cattle, and the caged chicken)
that the victims are symbolized to be at the same level as subjugated ani-
mals, powerless to act against their impending doom as weak prey for a
stronger attacker. The metaphor resounds mostly through the contextual
nature of cattle slaughter as well as repeated hints of juxtaposing. The can-
nibal family hunt their victims and subsequently kill them. Even though in
the case described above the victims all wandered in the house of their own
free will, it is clearly put forth in the film that the brothers job is to catch
people out on the road. The father sells barbecue equipment at his local gas
station and it can be read between the lines that the meat he is selling is most
likely human meat (possibly mixed with that of animals). The family mem-
bers eat other types of meat as well, but make no distinction between eating
humans or animals. Furthermore, they put their human victims to more use
than only for food. Leatherface makes clothes out of human skin, symbolized
by his trademark mask, and the house is filled with monstrous artworks
made from human bones and/or skin, similar to the objects found in the Gein
house of horrors. Unlike the sports hunter of Wolf Creek these human hunt-
ers are interested in more than just the thrill of the kill, and unlike the love-
seeking innkeeper of Calvaire their prey is not merely an object of sexual
desire. To them, it is one of their main sources of income and sustenance,
much like for the people of Plainfield.
The symbolism of people-as-meat is a well known interpretation for The
Texas Chain Saw Massacre, but the traditional interpretation does not in-
clude references to animal treatment; rather it is volunteered in the context of
the Vietnam War as being the great machine that, at the time, was eating
Americas youth (an interpretation presented in Ursula MacFarlane and
David Richardsons Clive Barkers A-Z of Horror, UK 1997). The animal
metaphor around the cannibal house is vividly based on the work of Ed Gein
and serves as a constant reminder of the role our victims play throughout the
film. In the end Sally, the sole survivor of the unfortunate five, has come to
realize what it is like to be treated as an animal for the slaughter and may
bring that realization with her back into society as she rides away on the
truck in the finale scene.
5. Discussion and conclusions
Although we investigated in detail three horror films only, we propose that
the HUMAN VICTIM IS ANIMAL metaphor dominates many more such films.
Other pertinent examples include The Hills Have Eyes (Craven, USA 1977),
exploring the slaughter/meat metaphor; Hostel (Roth, USA 2005), mining
the hunting/trapping metaphor; and Les Yeux Sans Visage/Eyes Without a
Face (Franju, France 1960), revealing the caging metaphor but with the
added dimension of torture and experimentation (see Eggertsson 2006 for
more discussion of these films). We want to end by presenting our findings
in the form of some tentative conclusions that can guide further theoretical
and experimental research.
We have provided further support for the insight that the medium film
can coherently exploit verbal, visual, and sonic modalities to cue a specific
conceptual metaphor (see Forceville 1999, 2006b; Forceville and Jeulink
2007). Sometimes manifestations of HUMAN VICTIM IS ANIMAL take the form
of monomodal metaphors (verbal or pictorial), sometimes they are multimo-
dal in the narrow sense, sometimes multimodal in the broad sense (Forceville
2006a). The conceptual nature of the metaphor transpires largely through its
constant repetition. While we might be in doubt whether to construe the
metaphor on the basis of single occurrences, its reiteration, in many different
stylistic guises, relentlessly drives the insight home. The metaphor is a highly
embodied one (Lakoff and Johnson 1980; Gibbs 2006; Johnson 2007) in
exploiting the mapping of very physical actions (shaving, hunting, trapping,
caging, slaughtering, killing) directed against animals, but also has cultural
dimensions (Gibbs and Steen 1999; Kvecses 2005) inasmuch as it high-
lights behaviors toward animals that are sanctioned or even encouraged by
mainstream society; after all, the scenario-based presuppositions are needed
to understand the inferences suggested in the texts (Musolff 2006: 36).
The HUMAN VICTIM IS ANIMAL metaphor manifests itself structurally, but
in various dimensions in the modern horror film. While in each film the vic-
tims are a resource for desire, the desires are of different kinds. The serial
killer of Wolf Creek quenches his desires through the sadistic joy of killing.
The mentally deranged innkeeper of Calvaire satisfies his sexual desires by
using his victim as an object of pleasure and entertainment. The cannibalistic
family members of The Texas Chain Saw Massacre meet their desires
mostly through their love of meat. All of the victims are presented as animals
to emphasize their subjugated position towards their attackers. The pertinent
mappings in each case thus depend on the nature of the victimizers desires.
Here is an (incomplete) list of these mappings:

Target domain Source domain
HUMAN VICTIM IS ANIMAL
No inherent social right to life
Stripped of freedom and liberty
Subjugated within a hierarchy
Subject to imprisonment
Subject to excessive pain, suffering
Defenseless towards captors
Voiceless towards society
Object of sexual desire
Object of entertainment
Resource for food
Resource for material goods
Life and well-being depends on the will and
desire of captor

Although by and large the mappable features apply across the range of ani-
mal species, in a given instantiation the species may play a role: Wolf Creek
mostly discusses kangaroos, which are typically hunted in Australia, but
makes references to horses, pigs and buffalos as well. Calvaire presents
farmyard animals (pig, cow, calf) and includes a trap that is obviously made
to catch the leg of an animal (probably a rabbit or a fox). The Texas Chain
Saw Massacre mainly presents cattle as the metaphorical source, although it
also refers to a pig squealing as well as to a caged chicken. It is to be noted
that the precise way in which an animal can be abused varies per species
depending on such factors as whether it is wild or domestic, big or small,
edible or non-edible, furred or feathered. Such considerations are important,
because they help counterbalance classic CMTs sometimes one-sided
emphasis on the conceptual level. Yes, the central metaphor in these films is
HUMAN VICTIM IS ANIMAL but the precise mappings vary in many small
and not-so-small ways depending on the precise narrative context in which
the metaphor surfaces. This view ties in with more recent work in CMT.
Thus Kvecses observation that the mappings of the same metaphor may
be different across any two languages or [language] varieties (2005: 123)
needs to be even further refined to accommodate the specificities of ad hoc
context.
The recurrence of the metaphor HUMAN VICTIM IS ANIMAL can be con-
sidered an important indication to characterize a film as a horror film.
Hence the films discussed illuminate that structural metaphors can contrib-
ute to genre theory (Altman 1999; Neale 2000, 2002). Note that this claim
only holds if the metaphor is verbalized in this particular way. The more all-
encompassing HUMAN BEING IS ANIMAL may contribute to the characteriza-
tion of other genres such as, arguably, the fable (although here the thorny
question remains whether the metaphor is HUMAN BEING IS ANIMAL or ANI-
MAL IS HUMAN BEING, perhaps calling for a blending theory approach, see
Fauconnier and Turner 2002). The centrality of the HUMAN VICTIM IS ANI-
MAL metaphor in the horror genre could be further investigated by focusing
on promotion materials such as trailers and posters.
Finally, the consistency with which the metaphor under scrutiny is de-
ployed in horror film inevitably focuses attention on issues of morality, spe-
cifically those pertaining to animal rights. As Lakoff and Johnson point out,
the most fundamental values in a culture will be coherent with the metaphorical
structure of the most fundamental concepts in the culture (1980: 22). In the
cases discussed in this chapter, the horrific mistreatment of humans that is cen-
tral to the horror genres conventions can apparently be no more hauntingly
presented than through the metaphor HUMAN VICTIM IS ANIMAL, in which
behaviors considered more or less conventional and acceptable when displayed
toward non-human creatures become a source of fear, disgust, and repellence
when mapped onto human beings (for more discussion on the role of metaphors
in the constitution of cultural values, see Johnson 1993). The structural nature of
the metaphor under discussion thus reveals the moral double standard that
surfaces when it becomes apparent that it is not the violence itself that cre-
ates horror for the viewer it is rather the type of animal the violence is
administered to (Singer 2001).

Acknowledgment

The authors are indebted to Mats Rohdin for his thoughtful comments on an ear-
lier draft of this chapter, but of course remain entirely responsible for all views
expressed.
Notes
1. Musolff characterizes a metaphor scenario as a set of assumptions made by
competent members of a discourse community about typical aspects of a
source-situation, for example, its participants and their roles, the dramatic
storylines and outcomes, and conventional evaluations of whether they count
as successful or unsuccessful, normal or abnormal, permissible or illegiti-
mate, etc. (Musolff 2006: 28).
2. Arguably, the metaphor should read HUMAN ANIMAL IS NON-HUMAN ANIMAL,
but since the English language differentiates between humans and animals
the more typical anthropocentric phrase will be used.
References
Altman, Rick
1999 Film/Genre. London: British Film Institute.
Black Max
Carroll, Noel
Clover, Carol J.
1992 Men, Women and Chain Saws: Gender in the Modern Horror Film.
London: British Film Institute.
Eggertsson, Gunnar Theodr
2006 Animal Horror: An Investigation into Animal Rights, Horror Cin-
ema and the Double Standards of Violent Human Behaviour. Re-
search MA diss., Media Studies Department, Universiteit van Am-
sterdam.
2002 The Way we Think: Conceptual Blending and the Minds Hidden
Forceville, Charles
1999a The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters,
bol 14: 17998.
1999b Art or ad? The effect of genre-attribution on the interpretation of
images. SPIEL 18: 279300.
2003 Bildliche und multimodale Metaphern in Werbespots [Trans. from
2005 Cognitive linguistics and multimodal metaphor. In
Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus
Sachs-Hombach (ed.), 264284. Cologne: Von Halem.
2006a/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
2006b The Source-Path-Goal schema in the autobiographical journey
documentary: McElwee, Van der Keuken, Cole. New Review of Film
and Television Studies 4: 241261.
2007a Multimodal metaphor in ten Dutch TV commercials. Public Journal
of Semiotics 1: 1951. http://semiotics.ca/
Forceville, Charles, and Marloes Jeulink
2007 The Source-Path-Goal schema in animation film. Paper presented at
10
th
International Cognitivist Linguistics conference (ICLC, July) at
Jagiellonian University, Krakow (Poland).
Grant, Barry Keith (ed.)
1996 The Dread of Difference: Gender and the Horror Film. Austin, TX:
University of Texas Press.
Humphries, Reynald
2002 The American Horror Film: An Introduction. Edinburgh: Edinburgh
University Press.
Johnson, Mark
1993 Moral Imagination: Implications of Cognitive Science for Ethics.
Chicago/ London: University of Chicago Press.
Kvecses, Zoltn
multimodal figures of thought in co-speech gesture.
this vol. Words, gestures and beyond: Forms of multimodal metaphor in the
Musolff, Andreas
2338.
Neale, Steve
2000 Genre and Hollywood. London/New York: Routledge.
Neale, Steve (ed.)
2002 Genre and Contemporary Hollywood. London: British Film Insti-
tute.
Rohdin, Mats
2003 Summary in English. In: Vildsvinet I Filmens Trdgrd: Metafor-
begreppet inom Filmteorin [The wild boar in the garden of film:
The concept of metaphor in film theory], 318329. Ph.D. diss.,
Stockholm: Edita Norstedts Tryckeri AB.
this vol. Multimodal metaphor in classical film theory from the 1920s to the
1950s.
Singer, Peter
2001 Animal Liberation. New York: Harper Perennial.
Whittock, Trevor
Subject index

ad hoc concepts, 8, 12, 147153,
159161, 163166, 168
ad hoc pointer, 149, 155158, 162,
165, 168
advertising, 5, 67, 9, 21, 24, 28,
33, 34, 49, 57, 60, 73, 74, 75,
81, 8392, 9598, 100112,
119, 120, 126, 132, 148, 149,
155, 165, 175, 176, 214, 215,
216, 218, 228, 229, 231, 348,
383, 384, 387, 388, 390, 396
wine ~, 7, 28, 7376, 8184, 86
89, 9092
anchoring, 47, 53, 62, 89, 102, 110,
157, 166, 181, 202, 204, 235,
332, 349, 368
anger, 9, 23, 27, 29, 99, 101, 243
262, 265267, 271285, 290,
291
anthomorphism, 78, 82, 85, 86, 91,
112, 388, 447, See also per-
sonification
Asterix [Alberto Uderzo and Ren
Goscinny], 9, 243, 244255,
257259, 261
audience (readers), 3, 8, 26, 31, 32,
49, 60, 62, 63, 74, 8185,
9093, 96, 97, 106108, 110
112, 123, 124, 131, 148, 149,
152169, 173176, 178182,
184, 187189, 191, 192, 199
201, 204, 207, 213, 214, 217,
222, 223, 232, 233, 251, 273,
274, 276, 284, 290, 320, 333,
347, 348, 375, 384, 387, 393,
394, 397, 421, 430, 434, 442
background of ~, 7, 90, 93, 173,
174, 184, 187, 190, 191, 270
interpretive strategies by ~, 8,
147149, 153, 163, 180, 213,
421

Battleship Potemkin [Sergei Eisen-
stein], 410
Berlin, Symphony of a Great City
[Walter Ruttman], 415
blending theory, 10, 20, 46, 122,
149, 169, 207, 235, 236, 359,
370, 375, 376
~ involving music and language,
10, 359, 370, 375, 376, 446
mental spaces in ~, 20, 46, 119,
122, 137, 169, 235, 236, 370
373, 375, 376,
~ compared with conceptual
domains, 122, 149, 236, 375,
376, 378
brand, corporate, 6, 28, 33, 4547,
4965, 81, 83, 89, 90, 91, 97,
102, 104, 106, 107, 110, 384,
388, 389, See also logo and
layout

The Cabinet of Dr. Caligari [Robert
Wiene], 408
Calvaire [Fabrice Du Welz], 437-
439, 444446
cartoons, 3, 69, 12, 23, 34, 148
149, 155, 157, 173193, 197
199, 201, 202, 205207, 213,
214236, 244, 314, 348, 390,
410, 423
452 Subject index
political ~, 69, 34, 148149,
157, 159165, 173193, 207,
423
~ as genre, 79, 34, 207
editorial ~, 8, 213236
cartoonist, 2, 148, 167, 173, 174,
176179, 182, 184, 189, 186,
191, 192, 199, 215, 216, 230,
244, 245
intentions of ~, 148, 167, 174,
176, 179, 182, 184, 186, 191
cognitive grammar, 208
Cognitive/Conceptual Metaphor
Theory (CMT), 4, 8, 9, 11,
12, 14, 19, 20, 21, 27, 28, 30,
46, 98, 121123, 137, 149,
173, 174, 245, 265267, 272,
279, 313, 316, 318, 319, 321,
331, 422, 429, 446
comics (see also animated film,
manga, and runes), 6, 9, 13,
23, 29, 59, 215, 245, 247
255, 257261, 265269, 271,
273, 283
~ and anger, 9, 29, 245, 247
255, 257260, 275, 277, 283
~ and other emotions, 9, 261,
265269, 271, 283287
conceptual upload, 147, 148, 152,
158162, 165, 166
constructional schema, 208
contiguity, 13, 29, 97, 135, 226,
229, 334, 335, 340, 342, 347
corpus analysis 7, 27, 51, 52, 53, 75,
218, 223, 260, 333, 336, 397,
423
culture, 9, 14, 22, 26, 28, 54, 63, 73,
75, 87, 88, 90, 92, 93, 121,
122, 126, 131, 134137, 139,
153, 157, 161, 179, 181, 182,
200, 248, 257, 260, 261, 268,
291, 298, 304, 312, 313, 332,
363, 376, 389, 446
Chinese ~, 7, 119, 120, 123, 126,
131136, 140, 291,
Chinese vs. Western ~, 14, 119,
131, 135139, 248
Dutch ~, 8, 217, 218, 222, 226,
227, 232, 384, 385
English ~, 21, 27, 7678, 90,
267, 284, 291
French ~, 7, 75, 77, 84, 85, 92,
271, 283, 388
Japanese ~, 9, 51, 248, 259, 265
275, 277281, 283, 284, 289,
290, 291, See also manga
Spanish ~, 7, 7578, 82, 86, 89,
90, 108
~ specific, 9, 12, 92, 121, 181,
260, 265, 271, 273, 284, 289,
290
critical discourse analysis, 49

decoding, 147, 149, 153, 169
deictic categories, 334
design strategy, 197, 199, 206, 207,
209
diagram, 320, 345, 346
~ tree, 342, 343, 346, 347, 348
domain(s), 7, 9, 1214, 20, 24, 25,
2730, 3335, 4548, 5362,
76, 77, 7982, 87, 88, 9193,
95, 97100, 102, 107, 110
112, 121, 122, 127129, 131
134, 152, 153, 157, 169, 170,
181, 189, 207, 208, 213215,
218221, 223233, 235, 236,
248, 249, 258, 265267, 273,
297, 300, 307, 308, 310, 313,
316, 319321, 330, 331, 336,
337, 339, 340, 341, 344349,
359362, 364366, 368370,
375, 376, 383389, 393397,
404, 406410, 412, 413, 415
419, 422, 423, 430, 431, 437,
Subject index 453
439, 442, See also target and
source
conceptual ~, 7, 9, 121, 127,
131, 134, 225, 235, 236, 265,
266
schematic source ~, 27, 224226,
230232
categorical source ~, 213, 215,
221, 228232, 236

embodied knowledge, 9, 27, 28
embodiment (embodied meaning),
4, 6, 7, 12, 20, 21, 27, 28, 64,
92, 121, 135, 149, 179, 181,
185, 248, 258260, 272, 315,
321, 330, 333, 336, 340, 342,
364, 377, 423, 445
The End of St. Petersburg [Vsevolod
Pudovkin], 416, 417, 422

features, 12, 13, 14, 20, 24, 29, 30,
33, 46, 89, 98, 100, 102105,
107, 110, 111, 113, 157161,
163, 166, 213, 225, 384, 385,
389, 391, 392, 396, 404, 407
409, 413, 415, 416, 422, 423,
441, 446
emergent ~ (properties), 8, 152,
160, 161, 166, 169
encyclopedic ~, 155, 159, 163,
164
mappable ~, 14, 225, 385, 389,
392, 396, 407, 409, 413, 415,
422, 446
film, 3, 5, 6, 1012, 14, 21, 24, 28,
3234, 140, 313, 314, 383,
384, 390392, 396, 397, 403
424, 429, 430, 436, 437, 446
animated ~ (also ~ animation),
9, 25, 233, 243, 244, 248,
250261, 272, 386, 394, 395
classical ~ theory, 403405, 421,
422, 424
horror ~, 11, 397, 430446
superimposition in ~, 254, 255,
405, 406408, 412, 415, 421,
423
~ montage, 100, 102, 112, 395,
405, 408, 410, 411, 421
silent ~ (cinema), 11, 403, 405,
408, 410, 420
Fury [Fritz Lang], 413, 421, 422

genre, 511, 19, 3234, 45, 46, 51,
52, 55, 6164, 7376, 80, 85,
87, 91, 95, 96, 112, 175, 183,
184, 189, 191, 192, 198, 207,
218, 219, 232, 234, 333, 383,
384, 389, 390, 397, 408, 422,
423, 429, 430, 433435, 442,
446
gesture(s), 35, 9, 10, 13, 19, 22,
23, 25, 34, 47, 80, 97, 98,
214, 251, 261, 297322, 323,
329349, 364, 365, 376
co-speech ~, 10, 321, 330, 336
deictic ~, 332, 343
functions of~, 309, 318, 332, 333
manual ~, 299, 313, 322, 330,
336, 339, 340, 341, 342, 344,
346, 347, 348
metaphoric ~, 297, 300302,
304306, 309, 310, 316, 329,
336, 337
metonymic ~, 346, 347
spontaneous ~, 298, 307, 332,
336, 342, 348
referential ~, 298, 301, 306, 307,
333, 336, 337
synecdochic ~, 305, 337, 343,
344, 348
~ and words, 316319
grammar, 22, 321, 330, 336
generative ~, 342, 344, 348
musical ~, 359
454 Subject index
Great Chain of Being, 13, 27, 28,
106, 112

happiness, 23, 265, 266, 283, 285
287, 289

indirect reference, 329, 331, 339
image alignment, 8, 61, 197209
image schema, 8, 30, 139, 167, 179,
187, 207, 208, 224, 317, 320,
330, 331, 340, 342, 349
SOURCE-PATH-GOAL ~, 8, 29,
127, 140, 179, 187
interpretation, 6, 810, 19, 20, 26
28, 32, 33, 4749, 53, 55, 58,
59, 62, 64, 87, 9093, 110
112, 147150, 152158, 160
162, 165, 167169, 173175,
180184, 187, 190192, 201
207, 213216, 219221, 224,
228232, 234, 248, 250, 317,
318, 331, 336, 337, 339, 341,
343, 346, 348, 372, 384, 385,
389, 395, 397, 403, 404, 408,
411, 418421, 423, 444
inference(s), 29, 47, 97, 128, 149,
151, 169, 230, 387, 445
integration, conceptual, 122, 236,
373, 374, See also blending
intertextuality, 11, 47, 56, 415
invariance principle, 207, 208, 373

layout, 6, 45, 46, 52, 53, 6061, 64,
201
logo (see also brand), 6, 30, 4547,
5153, 5765, 97, 98, 103,
105, 113, 385 (audiologo),
388, 396
love, 98, 249, 254, 258, 265, 266,
285, 287, 289, 300, 302, 307,
367, 372, 373, 396, 444
manga, 9, 13, 259, 261, 265, 266,
271274, 276281, 284, 285,
289, 290
manifestation, 8, 9, 29, 126, 321,
334, 430, 431, 445
nonverbal and multimodal ~ of
metaphor, 5, 7, 13, 34, 92,
119, 120, 123, 125, 129, 130,
136, 137, 139, 140, 175, 235,
243, 244, 246, 250, 252, 254,
255, 267, 272, 333, 348, 422,
429, 431, 439
verbal ~ of metaphor, 4, 12, 13,
21, 22, 34, 123, 248, 283,
431
mapping(s), 10, 13, 20, 24, 2830,
46, 61, 79, 81, 95, 96, 100,
120, 122, 127, 128, 133, 135,
138, 139, 169, 181, 213, 214,
217219, 226, 228, 265, 266,
270, 271, 273, 283, 284, 289,
290, 336, 340, 341, 348, 359,
361, 364367, 369, 370, 373,
375, 376, 384, 391, 393, 407,
408, 414, 417, 422, 434, 437,
441, 445, 446
double ~, 129, 346, 347
metaphoric(al) ~, 10, 13, 20, 24,
31, 97, 102, 104, 107, 110,
111, 112, 132, 149, 155158,
173, 190, 207, 208, 272, 313,
317, 339, 341, 375
metonymic(al) ~, 7, 98, 99, 110,
112, 134, 329, 341, 347
partial ~, 12, 55
medium, 9, 21, 32, 157, 163, 166,
214, 218, 243, 244, 250, 253,
255, 258261, 266, 283, 403,
410, 412, 418, 420, 424, 430,
444
metaphor
articulatory forms of ~, 299300,
307
Subject index 455
artistic (also metaphor in artis tic
texts / discourses, metaphor,
creative) ~, 396
complex ~, 119, 120122, 127
131, 133135, 137, 180, 181,
191, 235, 256260
conceptual ~, 4, 710, 1214, 19,
21, 25, 26, 30, 47, 48, 65, 80,
96, 112, 119123, 125130,
134, 136, 137, 139, 149, 161,
173, 213216, 218, 219, 234,
235, 246, 248250, 252, 256
261, 265267, 271, 273, 280,
283, 284, 290, 300, 301, 307,
313, 316, 318, 319, 321, 331,
336, 345, 361, 391, 403, 419,
422, 429, 430, 444, 445, See
also Cognitive / Conceptual
Metaphor Theory
construal of ~, 6, 10, 12, 14, 19,
20, 31, 33, 64, 336, 384, 385,
391, 397
conventional (conventionalized)
~, 26, 27, 29, 34, 53, 108,
113, 166, 167, 170, 179, 308,
342, 406, 407, 410, 422
creative ~, 8, 9, 12, 19, 25, 28,
34, 175, 244, 422, 429, 430
diagrammatic ~, 331, 342
diegetic/nondiegetic ~, 393, 396,
410, 411, 413, 415
directionality in ~, 10, 309, 369,
370, 375, See also invariance
principle
dual encoding in ~, 45, 47, 49,
54, 62
dynamic ~, 11, 95, 297, 300,
313, 316, 319, 320, 321
elaboration of ~, 104, 122, 170,
182, 268, 298, 313316, 321
emotion ~, 243, 244, 265268,
270272, 283, 284, 290
gestural ~, 65, 298, 300, 303,
304, 306308, 310, 311, 316,
347, See also metaphor in
gesture
~ in advertising, 6, 7, 9, 28, 33,
34, 48, 74, 81, 82, 86, 87,
9092, 96, 111, 112, 175,
176, 214, 215, 218, 383, 390
~ in animation, 6, 9, 13, 25, 233,
243, 258, 259, 260, 394, 395
~ in artistic texts/discourses, 14,
25, 33, 396
~ in film, 6, 1012, 14, 21, 24,
28, 33, 34, 140, 383, 384,
390392, 396, 397, 403406,
410413, 415, 418420, 429
431, 436, 438, 441, 442, 444
446
~ in gesture(s), 9, 10, 299, 302
309, 312, 316, 318, 321, 323,
329, 336, 337, 346, See also
gestural metaphor
~ in intonation, 9, 13, 297, 299,
302, 316318
~ in speech, 10, 297, 303, 304,
317, 321, 330332, 336, 340,
341
interaction theory of ~, 12, 24
interaction of ~ with metonymy,
7, 13, 95100, 104, 109112,
334, 344
monomodal ~, 9, 19, 2225, 28,
31, 33, 74, 79, 81, 177, 188,
190, 297, 300, 302, 303, 304,
306, 313, 321, 334, 339, 385,
403, 410414, 420, 429431,
437, 442, 445
multimodal ~, 314, 19, 2122,
24, 25, 2831, 3334, 4549,
51, 53, 5557, 59, 60, 6264,
7375, 7983, 88, 91, 92, 96
99, 105, 111, 112, 119, 120,
122, 123, 125, 130, 137, 139,
456 Subject index
140, 147, 148, 150, 152, 158,
161, 173175, 180, 181, 183
185, 190192, 197200, 202,
203, 205, 206, 208, 215, 233
235, 244, 251, 252, 260, 266,
271, 272, 283, 290, 297300,
302, 304, 307, 308, 310, 312
314, 316, 319, 321, 329331,
334, 336, 339, 341, 342, 347,
348, 359, 360365, 369, 370,
376, 383, 384, 389393, 395
397, 403405, 407, 412, 413,
416, 419424, 429431, 442,
445
naturalization of ~, 63, 396
ontological ~, 106, 163, 166,
175, 181, 182, 313
orientational ~, 175, 176, 180
182, 185, 187, 190
pictorial ~, 3, 6, 8, 9, 19, 2225,
28, 33, 46, 49, 55, 81, 82, 92,
148, 177, 187, 197, 199, 202,
243246, 250, 259, 265267,
271273, 276, 283, 285, 288
290, 330, 385, 389, 390, 395,
403, 416, 430, 431, 445
poly-interpretability of ~, 390
primary ~, 119, 121, 122, 128
130, 133, 137, 140, 181, 197,
208, 209, 256, 257, 260
products and processes in ~, 297,
300, 319, 321
projective ~, 393
prosody in ~, 316, 321
sleeping ~ (versus waking ~),
316
source and target in ~, 4, 7, 9
14, 1921, 2325, 2731, 33
35, 4548, 5357, 5962, 74,
79, 8183, 86, 88, 92, 97
102, 104, 107, 108, 110, 111,
120, 122, 127, 128, 132, 147,
148, 155166, 169, 174, 177,
178, 180, 181, 187, 188, 189,
191, 199, 200, 202, 205, 207,
208, 213215, 217222, 224
234, 235, 236, 244, 248, 249,
258, 266, 267, 271, 297, 300,
302, 304, 305, 307310, 312,
313, 315, 316, 319321, 329,
331, 336, 337, 339341, 344,
346349, 359, 365, 369, 370,
375, 383389, 392, 393397,
404, 406423, 430, 431, 439,
442, 444, 446
spatial ~, 128, 311, 312, 344
346, 348
systems of ~, 62, 121, 128, 129,
131, 132, 176, 266,
transparent ~, 308
verbal ~, 4, 68, 1014, 19, 21
24, 26, 29, 31, 34, 46, 47, 49,
73, 74, 81, 82, 84, 87, 88, 90,
123, 137, 140, 147, 148, 150
153, 156, 158161, 166168,
218, 243, 245, 248250, 256,
257, 260, 265267, 271, 273,
278280, 282284, 288290,
297, 300304, 308310, 313
317, 383, 396, 397, 405, 408,
409, 422, 430, 431, 435, 445
verbalization of conceptual ~, 13,
19, 30, 31, 47, 48, 74, 192,
207, 306, 404
verbo-gestural ~, 297, 300, 308,
310, 312, 315, 321
verbo-pictorial ~, 11, 86, 91,
199, 310, 383, 420
verbo-visual ~, 173
visual ~, 6, 8, 23, 24, 31, 46, 49,
54, 107, 131, 135, 147150,
152, 153, 155157, 159, 161
163, 165168, 174, 175, 178,
192, 213215, 218, 223, 224,
228, 230, 234236, 244, 245,
Subject index 457
248, 250, 271, 290, 330, 390,
413
metaphoricity, 297, 300, 303, 308,
316, 319, 323,
activation of ~, 307, 308, 321
~ and attention, 308
~ as cognitive process, 300, 316,
319
creation of ~, 297, 300
foregrounding of ~, 316
modality independent ~, 300,
319, 321
metonymy, 7, 10, 12, 13, 2325, 34,
61, 90, 95112, 119120,
128, 130, 134137, 139, 158,
282, 306, 329343, 346348,
349
~ and metaphor, 7, 10, 12, 13,
24, 25, 95100, 104, 107,
109111, 120, 139, 306, 329
332, 334337, 339341, 347
348
external ~, 329, 334, 338, 340,
346, 347
functions of ~, 98, 111, 139, 347
highlighting in ~, 96, 98, 100,
102, 103, 106, 110, 111
internal ~, 329, 334, 335, 343,
347, See also synecdoche
~ mapping, 7, 96, 98, 100, 110,
112, 120, 129, 135, 329, 340,
341, 346, 347
multimodal ~, 119, 120, 134
137, 139, 334, 348
visual ~ (pictorial ~), 90, 105,
134137
Metropolis [Fritz Lang], 407, 415,
421, 422
The Million [Ren Clair], 411, 412,
413, 421
mode/modality, 2, 411, 2125, 29,
31, 33, 34, 4547, 5153, 62,
7375, 80, 81, 91, 95, 96,
100, 102, 103, 106, 108, 110
112, 119, 120, 137, 139, 147,
148, 168, 169, 174, 175, 177,
180, 181, 188, 190192, 199,
207, 213, 233, 234, 251253,
257261, 265, 266, 283, 290,
297, 299, 300, 302, 307, 310,
312, 313, 316, 318, 319, 321,
322, 329, 330, 331, 333337,
339, 341, 342, 344346, 348,
349, 361, 363, 375, 376, 383,
385, 387, 395, 404, 405, 413,
420422, 430, 431, 444
sub ~, 95, 100, 103, 104, 106,
110, 112
types of ~, 5, 6, 22, 23, 4648,
53, 80, 97, 300, 404, 405
models
mental ~, 46, 48, 60, 112
abstract ~, 50
Idealized Conceptual Models
(ICM, folk models), 225, 243,
244, 246, 248250, 256, 258,
260, 261
Modern Times [Charlie Chaplin],
413415, 421, 422
module, 100, 148, 149, 153, 167
language ~, 149, 153, 169
perceptual ~, 154, 162
specialized ~, 148, 167
motivation, 13, 95, 102, 134, 151,
170, 265, 266, 272, 278, 279,
288, 289, 396, 397, 419
experiential ~, 127, 130, 208,
283, 289
physiological ~, 271, 289
music, 4, 5, 6, 10, 13, 19, 2123,
35, 47, 51, 53, 57, 123, 126,
131, 133, 136, 139, 261, 300,
331, 359378, 383385, 389
392, 395398, 403405, 411
413, 415, 421, 430, 431
text painting in ~, 359, 364370
458 Subject index
diegetic ~, 393, 396, 397
nondiegetic ~, 393, 396, 413
multimodal metaphor
definition of ~, 4, 5, 46, 74, 82,
97, 120, 148, 174, 191, 198
199, 233, 235, 266, 297, 299,
336, 339, 383, 404, 429, 430,
See also metaphor, multimo-
dal
explicitness of ~, 13, 55, 56, 100,
397
types of ~, 6, 9, 25, 28, 33, 47,
53, 91, 404, 405, 430
~ versus monomodal metaphor,
9, 2225, 300, 302, 313, 413,
431

narrative, 11, 33, 111, 178, 185,
281, 290, 394, 397, 405, 415,
430, 434, 446
~ meaning, 178
~ and action/events, 11, 178
~ and time/chronology, 185, 190,
36
~ image, 187
~ styles, 198
~ context, 290, 446
film ~, 409

October [Sergei Eisenstein], 411,
422
Old and New [Sergei Eisenstein],
411, 416, 417, 422

personification (see also anthropo-
morphism), 13, 28, 51, 62,
80, 83, 84, 100, 106, 112,
113, 120, 224, 386
pragmatic adjustment, 151

Relevance Theory, 8, 24, 147, 148
166, 168, 199
representation, 23, 24, 29, 54, 100,
102, 149, 178, 199, 204, 207,
248, 256, 259, 301, 346, 360,
376, 377, 397, 405, 412, 430
artistic ~, 33
cinematic / film ~, 404, 411, 415
dynamic ~, 341, 347, 373
gestural ~, 330, 336
medium specific ~, 260
metaphorical ~, 9, 111, 243, 245,
260, 261, 339
multimodal ~, 6, 21, 29, 34, 88,
244, 283, 329, 330, 331, 341,
342, 347, 348
surface ~, 97
visual ~ (pictorial ~), 31, 54, 56,
105, 106, 109, 154, 198, 207,
245, 250, 251, 256, 258, 259,
261, 266, 285
resemblance, perceptual, 31
rhythm, 100, 310, 332, 367, 372,
373, 378, 386, 393, 395
runes, pictorial, 23, 59, 245, 246,
247, 250, 254, 255, 272, 275,
276, 277, 280, 281, 283, 289

scenario, 11, 48, 154, 176, 187, 202,
215, 223228, 234, 235, 254,
276, 280, 283, 297, 305, 429,
430, 445, 447
Semiotic(s), 5, 10, 329, 334
Jakobsonian ~, 331, 332, 334,
335, 348
Peircean ~, 283, 331, 348
~ process, 304, 306
sensory
~ domain, 80
~ experience, 74, 79, 80, 89, 91
~ motor abilities, 342
~ perception, 73, 80
~ schema, 167
~ information, 167
Subject index 459
simile, 24, 25, 55, 74, 88, 186, 197,
198, 214, 411
sign
iconic ~, 162, 256, 337
indexical ~, 59, 245, 246, 251,
255, 256, 260, 265, 272, 274,
276, 281, 283, 291
sonic analogs, 10, 363367, 369,
370, 376, 377
song, 123, 136, 359, 367, 369375,
384, 385, 390, 396, 398
sound, 36, 10, 19, 21, 22, 23, 25,
31, 47, 53, 57, 80, 96, 97,
103106, 214, 233, 234, 250,
251, 255, 261, 298, 299, 321,
359368, 377, 383397, 404,
405, 412, 413, 419421, 430
diegetic versus nondiegetic ~,
393, 396, 397, 405, 412, 413
Strike [Sergei Eisenstein], 406, 407,
409, 411, 422
Surrealist art, 24
The Swindle [Federico Fellini], 418,
419, 421, 422
synecdoche, 24, 331, 335, 336, 342,
347, See also internal meton-
ymy
syntactic structure, visual, 333, 342

The Texas Chain Saw Massacre
[Tobe Hooper], 432, 441,
442, 444446
thinking for speaking, 297, 320, 321
touch, 5, 22, 23, 47, 75, 80, 86, 261,
300
trope, 8, 12, 23, 34, 334, 405
television commercials, 7, 9597,
110, 111, 119

upload, conceptual, 148, 152, 158
160, 162, 165, 166
innovative ~, 148, 152, 160, 165
stable ~, 152, 159, 162
universality vs. individuality, 174,
181, 190, 289

verbal image, 137, 405, 408410,
416, 417
visual-conceptual interface, 147,
156158, 162, 163, 165

winespeak, 6, 7374, 7980, 85, 86,
92, 93
wine tasting note, 7, 74, 75, 91
Wolf Creek [Greg McLean], 434
436, 441, 444446

Author index

Agawu, Victor Kofi, 364
Ajzen, Icek, 232
Alexandrov, Grigori, 411
Allanbrook, Wye Jamison, 364
Altman, Rick, 32, 446
Amoraritei, Loredana, 74, 93,
Arnheim, Rudolf, 404, 405, 412,
413
Askegaard, Sren, 50, 63

Balzs, Bla, 405, 406, 410, 411,
416, 420
Baldry, Anthony, 5, 176
Balmer, John T.M., 50, 52
Barcelona, Antonio, 12, 46, 95, 98,
120, 139, 330, 340, 341, 349
Bartsch, Renate, 26, 29, 30, 394
Baruch, Yehuda, 60
Batra, Rajeev, 96, 214,
Bazin, Andr, 405, 406, 418, 419,
420
Becker, Jonathan A., 216
Bell, Steve, 176
Bengtsson, Anders, 50, 57, 60, 63,
Black, Max 12, 13, 24, 28, 244,
405, 429
Blair, Preston, 248, 249, 258,
Bohle, Ulrike, 323
Boiles, Charles L., 363
Bolter, Jay David, 21
Bordwell, David, 256, 391, 397,
405, 406, 409, 415, 420,
Bouvet, Danielle, 299, 331, 336,
348
Bressem, Jana, 304,
Brooke-Rose, Christine, 31
Bruce, Nigel, 74
Brugman, Claudia, 207
Brunn, Stanley D., 216
Buckland, Warren, 404
Caballero, Rosario, 6, 12, 14, 79,
93, 112, 397
Calbris, Genevive, 299, 320, 331,
332, 336
Cameron, Lynne, 112, 321
Cappetta, Rossella, 51
Carroll, Noel, 24, 33, 46, 177, 244,
266, 330, 390, 405, 406, 408,
423, 429
Carston, Robyn, 150, 152
Cavanagh, Patrick, 169
Chafe, Wallace, 302, 322
Charteris-Black, Jonathan, 32, 397
Cheney, George, 58, 64
Christensen, Lars Thger, 50, 58,
64
Chun, Rosa, 65
Ciccolella, Francesco, 50, 51, 64,
Cienki, Alan, 9, 13, 21, 25, 34, 214,
297, 299, 302, 304, 311, 317,
318, 319, 320, 321, 323, 330,
331, 333, 336, 348, 364, 430
Clair, Ren, 410, 411, 412
Clark, Herbert, 298
Cohen, Ted, 405, 423
Collins, James C., 51
Colston, Herbert, 21
Cook, Guy, 96
Cook, Nicholas, 370
Coulson, Seana, 20, 122, 149
Crisp, Peter, 27
Croft, William, 98, 330, 340, 363
Csaba, Fabian, 50, 57, 60, 63
Culpeper, Jonathan, 12
Cumming, Naomi, 377
Cunliffe, Ann L., 51
Cupchik, Gerry D., 34
Czerniawska, Fiona, 52

Damasio, Antonio R., 363,
Danaher, David, 331
462 Author index
Danto, Arthur C., 403, 420, 423
Dastjerdi, H. Vahid, 181
Davies, Gray, 65
Day, Sean, 80
de Chernatony, Leslie, 46
Deacon, Terrence W., 377
Deutscher, Guy, 214
Dez Velasco, Olga Isabel, 7, 95, 97,
99, 110, 158
Dines-Levy, Gail, 176
Dirven, Ren 12, 24, 95, 111, 112,
120, 139, 330, 334
Duncan, Susan, 316, 320
Durand, Jacques, 23

Edwards, Janis L., 173, 174, 176,
179, 216
Eerden, Bart, 9, 13, 29, 244, 249,
254, 257, 260, 272
Efron, David, 298, 320
Eggertsson, Gunnar, 11, 14, 395,
444
Eisenstein, Sergei, 405411, 416
418
Ejxenbaum, Boris, 405, 408, 410
El Refaie, Elizabeth, 8, 13, 34, 110,
149, 150, 155, 176, 177, 192,
193, 207, 216, 219, 233, 348,
390, 405, 410
Elliot, Richard, 51
Entzenberg, Claes, 423
Epstein, Jean, 424
Eves, Anmarie, 214

Fauconnier, Gilles, 7, 8, 10, 20, 46,
122, 169, 235, 330, 370, 378,
446
Fein, Ofer, 244
Feldman, Ofer, 216
Fischer, Roger A., 182
Fishbein, Martin, 232
Fodor, Jerry, 8, 149, 153, 167
Forceville, Charles, 411, 13, 14,
20, 24, 25, 28, 29, 3235, 46,
47, 49, 51, 55, 60, 61, 74, 81,
82, 86, 88, 91, 92, 9598, 112,
120, 122, 123, 137, 140, 147
149, 155, 174, 175, 177, 180,
181, 191, 197199, 202, 207,
214, 216, 217, 220, 233235,
243, 244249, 251254, 257,
259, 260, 261, 262, 265, 266,
271, 272, 274277, 279, 280,
283, 288, 291, 299, 300, 310,
313, 314, 330, 336, 339, 348,
349, 378, 383, 384, 387, 389,
390, 394, 395, 397, 404, 413,
422, 423, 429, 430, 431, 444
Forns, Johan, 363
Fox, John, 52
Fox, Renata, 52
Franke, George R., 214
Frick, Wolfgang, 65
Fricke, Ellen, 331, 333, 343
Friedman, Vanessa, 56
Furuyama, Nobuhiro, 349

Geeraerts, Dirk, 291, 341
Gentner, Dedre, 13, 33, 214, 225,
226, 236
Gevaert, Caroline, 27, 291
Gibbs, Raymond, 14, 20, 21, 28, 34,
121, 139, 149, 151, 170, 214,
235, 248, 261, 321, 330, 331,
336, 348, 395, 421, 445
Gilmartin, Patricia, 216
Gilot, Franoise, 313
Gineste, Marie-Dominique, 8, 24,
152
Gioia, Dennis, 51
Gluck, Malcolm, 74
Glucksberg, Sam, 214, 228
Goatly, Andrew, 31
Goldin-Meadow, Susan, 332, 364
Gombrich, Ernst H., 193
Goodwin, Charles, 332
Goossens, Louis, 98, 330, 341
Goscinny, Ren, 245, 247, 250, 261
Grady, Joseph, 27, 65, 121, 122,
208, 256, 265
Green, Christopher D., 59, 194
Groarke, Leo, 216, 220
Author index 463
Grondelaers, Stefan, 291
Grusin, Richard, 21

Haferland, Harald, 304
Hake, Sabine, 403, 408
Harquail, Celia V., 51
Haser, Verena, 6, 14
Hatch, Mary Jo, 50, 51, 64, 65
Hayward, Malcolm, 32
Heckler, Susan E., 57, 58, 65
Heinke, Dietmar, 169
Herold, Debora, 317
Hiraga, Masako, 331
Hirschberg, Julia, 316
Hoeken, Hans, 214
Holm, Lisbeth S., 53
Hols, Edith, 23
Holt, Douglas B., 46
Holyoak, Keith J., 369
Horn, Robert E., 178
Hrschelmann, Kathrin, 183, 184,
192
Hubbard, Edward, 80
Huhmann, Bruce A., 214
Humphreys, Glyn, 169
Humphries, Reynald, 432

Indurkhya, Bipin, 24, 152, 393,
421, 422

Jakobson, Roman, 97, 111, 329
332, 334, 335, 341, 348, 349,
415
Jeulink, Marloes, 29, 444
Johansson, Ulla, 53
Johnson, Christopher, 208
Johnson, Mark, 3, 4, 1921, 35, 46,
64, 7980, 98, 106, 121, 132,
139, 163, 166, 175176, 179,
181, 198, 207208, 213214,
225, 232, 243, 244, 250, 260,
265266, 267, 313, 317, 319,
330, 331, 336338, 340, 344,
345, 349, 422423, 429, 430,
445447
Johnson, Michael, 421
Johnston, Ollie, 248, 249, 258

Kapferer, Jean-Nol, 49, 50
Kaplan, Stuart J., 24, 175
Kasher, Asha, 244
Keller, Hans, 370
Kendon, Adam, 298, 299, 302, 304,
309, 318, 322, 332, 364
Kennedy, John M., 23, 24, 34, 59,
175, 244, 245, 246, 250, 272,
276, 330
Khordoc, Catherine, 244
Killgren, Lucy, 57
Kittay, Eva, Feder, 423
Klein, Naomi, 63
Knop, Sabine de, 23
Koller, Veronika, 6, 14, 49, 52, 60,
105
Kller, Wilhelm, 337
Kotze, Zacharias, 291
Kvecses, Zoltn, 9, 11, 20, 28, 29,
99, 104, 121, 122, 139, 174,
175, 181, 243, 244, 247, 248,
249, 250, 254, 256, 257, 258,
259, 260, 261, 265, 267, 268,
269, 280, 291, 330, 422, 445,
446
Kress, Gunther, 5, 34, 61, 65, 96,
177, 178, 348, 349
Kriegel, Uriah, 154
Kristiansen, Gitte, 12, 50
Kunda, Ziva, 50

Ladewig, Silva, 304,
Lake, Carlton, 313
Lakoff, George, 34, 13, 1921, 25,
26, 27, 28, 35, 46, 61, 64, 79
80, 98, 106, 121, 128129, 132,
139, 163, 166167,175176,
179, 181, 198, 207208, 213
214, 225, 232, 235, 243244,
250, 260, 265267, 276, 280,
313, 317, 319, 330331, 336
338, 340, 344345, 361, 373,
377, 407, 410, 419, 422423,
429, 430, 445447
464 Author index
Langacker, Ronald W., 208, 340
Larson, Steve, 35
Lassen, Inger, 49
Laudenbach, Peter, 62
Lehrer, Adrienne, 74, 77
Lidov, David, 365
Lindsay, Greg, 59
Lindsay, Vachel, 420, 424
Lodge, David, 331, 341
Loewenstein, Jerry, 13, 33
Low, Graham, 321

Maalej, Zouhair, 29
Maes, Alfons, 8, 14, 149, 231, 348,
390
Maestri, George, 248, 249, 258
Mandler, Jean, 340
Matsuki, Keiko, 268
Matsunaka, Yoshihiro, 9, 13, 29,
248, 259, 261, 268270, 273,
284, 291
Mautner, Gerlinde, 65
McCarthy, Michael S., 214
McCloud, Scott, 245, 291
McLuhan, Marshall, 21
McMahon, Jennifer A., 154
McNeill, David, 9, 25, 34, 299, 302,
316, 320, 322, 330332, 336,
337, 364
McQuarrie, Edward, 34, 49, 96, 97,
113, 156, 165, 214, 220
Messaris, Paul, 64, 100, 175, 177
Mick, David, Glen, 34, 214
Miller, Christine M., 34
Mnguez, Norberto, 154
Mittelberg, Irene, 10, 13, 95, 98,
299, 304, 313, 319, 330, 331,
333, 336, 346, 348, 349, 430
Moingeon, Bertrand, 50
Morford, Jill, 331
Morris, Desmond, 304
Morrison, Susan, 173, 216
Mothersbaugh, David L., 214
Mller, Cornelia, 9, 13, 25, 34, 214,
299, 301, 302, 304306, 308,
310, 311, 313, 314, 316, 318
323, 330333, 336, 339, 348,
364, 430
Mumford, Alan, 182
Muniz, Albert M., 63
Musolff, Andreas, 11, 48, 429, 430,
447

Neale, Steve, 446
Neumann, Ragnhild, 304
Nusbaum, Howard C., 365

OGuinn, Thomas C., 63
OHalloran, Kay, 5
Oakley, Todd, 122, 235
Ochs, Eleanor, 332, 349
Okrent, Arika, 365
Ortony, Andrew, 3, 19, 228

Pankhurst, Anne, 111
Panther, Klaus-Uwe, 95, 98, 120,
139, 330, 339, 342
Papasolomou, Ioanna, 174
Pascual, Esther, 20
Peirce, Charles Sanders, 256, 262,
272, 283, 289, 331, 332, 334,
337, 343, 346, 348, 377
Peamarn, Cristina, 149
Perlman, Marcus, 14
Petrakis, Stefanos, 20
Peynaud, mile, 74, 77
Philippe, Robert, 173
Phillips, Barbara, 34, 49, 96, 97,
113, 156, 165, 175, 214
Pierrehumbert, Janet, 316
Pilkington, Adrian, 150
Pimentel, Ronald W., 57, 58, 65
Plumb, Steve, 215, 216,
Polzenhagen, Frank, 112
Prings, Ralf, 12, 24, 95, 120, 330,
334
Porras, Jerry I., 51
Pragglejaz Group, 14, 301
Pratt, Michael G., 53
Proctor, Stella, 174
Proctor, Tony, 174
Pudovkin, Vsevolod, 411, 416, 417
Author index 465
Radden, Gnther, 95, 98, 99, 111,
120, 139, 330, 340, 341
Rafaeli, Anat, 53
Ramachandran, Vilayanur, 80
Ratner, Leonard G., 364
Rayson, Paul, 52
Richards, Ivor Armstrong, 23, 423
Ricoeur, Paul, 11
Ritchie, David, 174, 181, 182, 187,
191, 192
Rogers, Priscilla, 52
Rohdin, Mats, 10, 11, 13, 14, 34,
404, 411, 423, 429
Rompay, Thomas, van, 34
Rothbart, Myron, 63
Rozik, Eli, 24
Ruiz de Mendoza, Francisco, Jos,
7, 95, 97, 99, 110, 149, 151,
158, 163, 170, 208

Salt, Barry, 405
Scart, Vronique, 152
Schilperoord, Joost, 8, 14, 149, 231,
348, 390
Schirato, Tony, 178
Schultz, Majken, 50, 51, 64, 65
Schwartz, Jonathan, 48
Scott, Linda, 96, 175, 214
Seitz, Jay A., 175
Semino, Elena, 12
Shapiro, Michael, 334
Shen, Yeshayahu, 33, 213, 225,
226, 228, 235, 236
Shibles, Warren, 23
Shinohara, Kazuko, 9, 13, 29, 248,
259, 261, 268270, 273, 284,
291
Shintel, Hadas, 365, 377
Shore, Bradd, 28, 319
Shotter, John, 51
Simons, Jan, 29
Singer, Peter, 447
Slobin, Dan I., 320
Smith, Gregory, W. H., 176
Smith, Ken, 23
Soenen, Guillaume, 50
Speckmann, Gerald, 304, 305, 306
Sperber, Dan, 8, 24, 97, 147, 148
Spitzer, Michael, 35
Stanford, W(illiam) B(edell), 404
Steen, Gerard J., 28, 32, 139, 248,
261, 321, 421, 445
Stckl, Hartmut, 100, 103
Stockwell, Peter, 27
Streeck, Jrgen, 304, 318, 332
Strunck, Jeanne, 49
Sun, Sewen, 34, 61, 96, 197, 214,
220, 222
Swales, John, 52
Sweetser, Eve E., 20, 129, 139, 299,
330, 331, 336, 337, 338, 345

Talebinejad, M. Reza, 181
Tanaka, Keiko, 96, 148
Taub, Sarah, 322, 331, 336
Taylor, John, 98
Taylor, Marjorie, 63
Templin, Charlotte, 173
Tendahl, Markus, 149, 151, 170
Teng, Norman, 8, 13, 34, 61, 96,
197, 198, 214, 220, 222
Terkourafi, Marina, 20
Teendorf, Sedinha, 304, 305, 306,
323
Thagard, Paul, 369
Thibault, Paul J., 5, 176
Thomas, Frank, 248, 249, 258
Thompson, Kristin, 256, 391, 397,
405, 406, 415, 420
Thorau, Christian, 35
Thornburg, Linda L., 95, 98, 120,
139, 330, 339, 342
Tillyard, E.M.W., 13, 27
Tom, Gail, 214
Tomasello, Michael, 363
Turner, Mark, 7, 8, 10, 13, 20, 25
28, 35, 46, 61, 98, 106, 122,
139, 169, 181, 207, 235, 248,
330, 373, 377, 378, 407, 410,
419, 446
Tversky, Barbara, 23
Tynjanov, Jurij, 406, 408
466 Author index
Uderzo, Albert, 245, 247, 250, 261
Ungerer, Fiedrich, 55
Urios-Aparisi, Eduardo, 7, 13, 14,
112

Van Dijk, Teun A., 49
Van Enschot-van, Dijk, Renske, 214
van Leeuwen, Theo, 5, 34, 47, 48,
61, 65, 96, 178, 348
Van Mulken, Margot, 214
Van Noppen, Jean-Pierre, 23
Vega-Moreno, Rosa, 150, 151, 152,
161, 168
Ventola, Eija, 5, 34
Versante, Laura, 309
Vervaeke, John, 59, 175
Vestergaard, Torben, 49
Villafae, Justo, 154

Wales, Katie, 335
Ward, Janet, 408
Warren, Beatrice, 98
Wattanasuwan, Kritsadarat, 51
Waugh, Linda, 10, 13, 95, 98, 299,
304, 319, 331, 332, 335, 346,
365, 430
Webb, Jen, 178
Webb, Rebecca, 299, 336
Wee, Thomas Tan Tsu, 50
Whittock, Trevor, 24, 28, 34, 244,
334, 348, 422, 429
Wiggin, Amy A., 34
Wilcox, Phyllis P., 299, 330, 331,
339
Wilcox, Sherman, 331
Willenbrock, Harald, 62
Williams, Richard, 248, 249, 258
Wilson, Deirdre, 8, 24, 97, 147,
148, 152
Wittgenstein, Ludwig, 359, 363,
376
Wodak, Ruth, 48
Wolf, HansGeorg, 112
Wolff, Philip, 214
Worth, Sol, 175, 404

Yamanaka, Keiichi, 270
Yu, Ning, 7, 8, 13, 28, 80, 96, 98,
126, 129, 132, 139, 248, 291
Yus, Francisco, 8, 12, 13, 100, 152,
155, 169, 348, 390

Zakia, Richard D., 177
Zbikowski, Lawrence, 10, 13, 35,
214, 365, 370, 376, 377
Zwaan, Rolf, 32
Metaphor and metonymy index
This index contains the verbalizations used by authors to designate conceptual
metaphors and *metonyms

A PURPOSEFUL LIFE IS A JOURNEY,
121, 122, 140
ABORTING IS UNKITTING A BABY
BOOTIE, 107, 109
ABSTRACT A IS CONCRETE B, 422
ABSTRACT AS CONCRETE, 313
*ACTION FOR OBJECT INVOLVED IN
ACTION, 339
ACTIONS ARE (SELFPROPELLED)
MOTIONS, 122, 128, 130, 131
AFFECTION IS WARMTH, 104, 110
AMERICAN NEWS IS LIKE HORROR
NOVEL, 198
ANGER IS A BURDEN, 259
ANGER IS A CAPTIVE ANIMAL, 244,
259, 238, 441
ANGER IS A DANGEROUS ANIMAL,
256, 258
ANGER IS A HOT FLUID IN A
(PRESSURIZED) CONTAINER, 247,
248, 256, 258, 259, 265, 267,
271, 273, 277, 278, 280, 283,
290, 291
ANGER IS AN OPPONENT (IN A
STRUGGLE), 258, 259
ANGER IS FIRE, 243, 279
ANGER IS GASTRIC CONTENTS IN THE
DIGESTIVE SYSTEM, 248
ANGER IS HEAT, 27
ANGER IS OBJECTS IN THE HEART, 181
ANGER IS SWELLING, 27
ANGER IS THE HEAT OF A FLUID IN A
CONTAINER, 280
ANIMAL IS HUMAN BEING, 446
AUTHORITIES HIDING BEHIND
BUREAUCRACY IS CLERK BEHIND
BOOK, 417, 422
BAD IS DOWN, 317
THE BALLOT BOX IS A DICE, 163, 164
BEING SUBJECT TO CONTROL IS
DOWN, 176, 345
BICYCLE IS DRESSAGE HORSE, 31
BOOK IS MEDICINE, 225
THE BOOK IS A BALL, 161
BRANDS ARE LIVING ORGANISMS, 6,
46, 47, 53, 56, 6062, 64
BRANDS ARE PEOPLE, 5153, 57, 61,
65
BUILDINGS ARE RUBBISH, 157, 159

CAR IS A (LONGJUMP) ATHLETE, 100,
110
CAR IS PERSON, 102, 106, 107, 110
CARS ARE FISH, 386, 388
CATEGORIES ARE CONTAINERS, 338
COFFEE MACHINE IS BUTLER, 30
COFFEE MACHINE IS SERVANT, 30
COFFEE MAKER IS MOTORCYCLE, 389
COLIN IS A CHILD, 431
COMPANIES ARE LIVING ORGANISMS,
48
CONCEPTUAL STRUCTURE IS
GEOMETRIC PHYSICAL
STRUCTURE, 337, 338, 345
CONCRETE A IS CONCRETE B, 422
CONCRETE IS CONCRETE, 27
CONTENTS ARE CONTAINERS ,338
COOKIE IS PERSON DOING FITNESS
EXERCISES, 385, 386, 396
CORNCOB AND FRENCH BEAN ARE
BRIDE AND GROOM, 388, 396
COUNTRY IS ANGRY TEACHER, 223,
COUNTRY IS LITTLE BAD BOY, 223
COUNTRY IS PATIENT, 225, 229
468 Metaphor and metonymy index
CULTURE IS A PAPER SERVIETTE, 157

DEATH IS A DEVOURER, 407
*DEGREE OF MOTIVATION FOR
SUCCESS IS SIZE OF HEART, 126
THE DEPLOYMENT OF ARMED FORCES
AGAINST TERRORISM IS AN
INFINITY DEPLOYMENT IN A VAST
GEOGRAPHICAL AREA, 203
THE DIFFERENCE BETWEEN NORTH
AND SOUTH IS AN ABYSS, 167
THE DIRECTION OF THE ECONOMIC
TREND IS THE DIRECTION OF THE
ARROW AS IT IS VIEWED FROM
EITHER THE BUSINESS PERSONS
POSITION OR THE LABORERS
POSITION IN THE
UNCONVENTIONAL SPATIAL
FRAME, 205

THE EARTH IS A SAUCEPAN, 162, 163,
*EFFECT FOR CAUSE [Relaxation for
Drinking Tea], 102, 103
*EMBLEM FOR PRODUCT [Tea Tag for
Tea], 102, 103
EMOTION IS EXTERNAL METEORO-
LOGICAL/NATURAL PHENOME-
NON THAT SURROUNDS THE SELF,
270, 271, 273, 283, 290
EMOTION IS FORCE, 268
EMOTIONS ARE FORCES, 9
EYES ARE CONTAINERS FOR THE
EMOTIONS, 250, 258

FINANCIAL SUCCESS IS NATURAL
GROWTH, 54, 55
FOOTBALL IS SUN, 220, 221

*GARMENT FOR PERSON [Baby bootie
for baby], 107, 109, 110
GOOD IS UP, 65, 313, 317
GREAT CHAIN OF BEING, 13, 27
HAPPY IS UP, 289
HAVING CONTROL IS UP, 176, 345
HIGH STATUS IS UP/LOW STATUS IS
DOWN, 176, 345
HUMAN (BEING) IS ANIMAL, 433, 437,
441, 446
HUMAN (BEING) IS DOG, 394
HUMAN VICTIM IS ANIMAL, 11, 429
431, 433, 444446
HUMAN VICTIM IS ANIMAL RESOURCE,
437, 441
HUMAN VICTIMS ARE CATTLE, 442
HUMAN VICTIMS ARE KANGAROOS,
436, 437
HUMAN VICTIMS IN CAR ARE DEAD
ARMADILLO IN STREET, 441

IDEAS ARE (PHYSICAL) OBJECTS, 235,
337, 338, 340, 342
(THE INSTITUTION OF) CINEMA IS
MOLOCH, 408, 421
INTENSITY IS HEAT, 181, 256,
INTENSITY IS QUANTITY, 257
INTENSITY IS SPEED, 257
INTRODUCING AN IDEA IS PRESENTING
AN OBJECT, 318

KNOWN IS DOWN, 317

LIFE IS A BURDEN, 26,
LIFE IS A JOURNEY, 7, 8, 20, 26, 29,
35, 119, 120, 122, 125, 127
129, 136, 138, 139, 179, 419
LIFE IS A PLAY, 26,
LIFE IS A STAGE, 7, 26, 126, 130,
131, 133, 134, 136, 139
LIFE IS BONDAGE, 26,
*LOCATION FOR OBJECT, 339
LOUDER IS BIGGER, 279
LOVE IS A JOURNEY, 98, 300
LOW STATUS IS DOWN, 176, 345

MACHINE IS MONSTER, 406
MACHINE IS MOLOCH, 406, 408
Metaphor and metonymy index 469
MACHINES ARE PEOPLE, 410, 422,
MAKING A DECISION IS PULLING A
THREAD, 107110
MAN IS AN ANIMAL, 406, 422, See
HUMAN IS ANIMAL and PERSONS
ARE ANIMALS
MASSACRE IS RELIGIOUS RITUAL, 392
MEASURE IS COOKING EQUIPMENT,
226
MEASURE IS INGREDIENT, 226
MEASURE/POLICY IS COOKING POT,
234
MEN CHASING COAT ARE MEN
PLAYING RUGBY, 412
MENTAL STATE IS FAST TRAIN, 392,
393
THE MIND IS A MACHINE, 161
MIND IS BODY, 20
MINISTER IS COOK, 226
MISGUIDED BELIEF IS AN OPPOSITE
BODY ORIENTATION, 201, 202
MOBILE PHONE IS PIANO, 384, 385
MOTHER RUSSIA IS HEADLESS PERSON,
416, 422
MULTINATIONALS ARE DRIP, 233

NEGATIVE IS DOWN, 317
NEW VOTING SYSTEM IS COOKING
POT, 233235

OBJECT A IS OBJECT B, 28
OIL IS INTRAVENOUS DRIP, 216, 217
ONES MENTAL CAPACITY IS SIZE OF
ONES HEART, 127

*PART FOR WHOLE [Car front for
Car], 99, 105, 107, 109, 128,
130, 228, 341
*PLACE FOR INSTITUTION, 99
PEACE IN THE MIDDLE EAST IS AN
ENDANGERED SPECIES, 200, 201
PERSON IS CAPTIVE ANIMAL, 438, 441
PERSONS ARE ANIMALS, 106
PITCH RELATIONSHIPS ARE
RELATIONSHIPS IN PHYSICAL
SPACE, 361, 362
POLICY IS DISH, 226,
POLITICAL DOMAIN/PUBLIC AREA IS
HOSPITAL ROOM, 225
POLITICAL MOVEMENT IS MEDICINE,
225
POLITICIAN IS TODDLER, 177, 180,
188, 189, 191
POLITICIAN IS DOCTOR, 225, 229, 230
POSITIVE IS UP, 317
POWER/HIGH STATUS IS UP, 344, 345
*PROTOTYPICAL ITEMS OF A CULTURE
STAND FOR THAT CULTURE, 137
PUBLIC SCHOOL IS ARMY, 391
PURPOSES ARE DESTINATIONS, 122,
128

REAL CANNON IS CIRCUS CANNON,
390, 391
*REPRESENTATION FOR
REPRESENTED, 339

*SALIENT FEATURES OF A THING
STAND FOR THAT THING, 137
SIMILARITY IS ALIGNMENT, 8, 197,
208, 209
SOFT COLOR IS WARMTH, 103, 104,
110
SPACE-IN-FRONT IS FUTURE, 180
STATE OF BEING IS ORIENTATION IN
VERTICAL SPACE, 362
STATES ARE LOCATIONS, 128130,
377
STREETS RUNNING WITH BLOOD ARE
MAP SPILLED WITH INK, 409
SUCCESSFUL IS BIG, 133, 134, 140
SUCCESSFUL IS UP, 7, 129, 130, 140
SUMMER TV PROGRAMMING IS (LIKE)
LEFT BEHIND PET, 222
SWIRLING WINE IS SWIRLING BALLET
DANCER, 86
470 Metaphor and metonymy index
SYNTACTIC STRUCTURES ARE
GEOMETRIC PHYSICAL
STRUCTURE, 345

TEA TAG IS A HYPNOTIZERS WATCH,
102104
TENT IS TRAIN, 394, 495
THE TENNIS RACKET IS A SHARKS
JAW, 165
TIME IS A DEVOURER, 26
TIME IS A MOVER, 26
TIME IS A MOVING OBJECT, 179
TIME IS A THIEF, 26
TIME IS MOTION THROUGH SPACE,
185
TIME PASSING IS MOTION, 179
TSUNAMI IS SHARK, 464
TV PROGRAMS ARE GARBAGE, 199

UNKNOWN IS UP, 317

WINE IS A BABY, 84
WINE IS A BALLERINA, 87
WINE IS A TEXTILE
WINE IS A THREEDIMENSIONAL
ARTEFACT, 77
WINES ARE BUILDINGS, 77
WINES ARE GEOMETRICAL ENTITIES,
77
WINES ARE JEANS, 88
WINE IS LIVING ORGANISMS, 77
WINES ARE PEOPLE, 83
WINES ARE PIECES OF CLOTHING, 76
WOMEN ARE ANGELS, 418, 419, 421
WORKERS ARE SHEEP, 414
WORKERS ARE SLAUGHTERED OXEN,
411

Multimodal Metaphor

Uploaded by

Copyright:

Available Formats

Multimodal Metaphor

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Multimodal Metaphor

Uploaded by

Copyright:

Available Formats

What is the book about based on the index entries?

What publishing information is provided about the book?

Multimodal Metaphor

Applications of Cognitive Linguistics

You might also like