Hand gestures are a natural, ubiquitous and meaningful part of spoken
language – so much so, researchers have claimed that gesture and speech
form a tightly integrated system during language production and comprehension (McNeill 1992, 2005; Clark 1996; Goldin-Meadow 2003; Kita
and Özyürek 2003; Kendon 2004; Özyürek and Kelly 2007). David
McNeill, focusing on language production, was the first to argue that
gesture and speech make up a single, integrated system of meaning expression
(1992). He posited that because gesture and speech temporally overlap but
convey information in two very different ways – speech is conventionalized
and arbitrary, whereas gesture is idiosyncratic and imagistic – the two
modalities capture and reflect different aspects of a unitary underlying
cognitive process. Thus, according to McNeill, gesture and speech
combine to reveal meaning that is not fully captured in one modality
alone. For example, imagine explaining to a child how to tie a shoe
without actually physically touching the laces. It is surprisingly hard to
2 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
explain this simple and common action without moving one’s hands
(try it), but adding gesture to speech makes it very easy and understandable.
There are four main types of co-speech gesture (McNeill 1992). The
above example illustrates an iconic gesture – these are gestures that imagistically represent object attributes, actions and spatial relationships. A
second type of gesture is a deictic, or pointing, gesture. These gestures
index, or connect, some aspect of speech to some other idea, object,
location or action. For example, imagine you are hiking through dense
vegetation and your more experienced travel companion says, ‘Watch out
for that poison ivy’. Without a deictic gesture to the plant, you might be
in for a long and unpleasant camping trip! A third type of gesture is a
metaphoric, which conveys an abstract idea in a concrete form. For example,
during a brainstorming session on a new advertising campaign, a marketing
executive may say, ‘Here’s what I am thinking’, while holding her fingers
at her forehead and temples, and then suddenly thrusting them outward
as if holding an object out to the group. This is metaphoric because it is
impossible to physically remove an idea from one’s head and literally
present it to others, but the gesture in this case conveys that meaning
nonetheless. Finally, beat gestures are hand movements that keep the
rhythm of speech. These gestures are not thought to convey any semantic
content, but they do connect portions of discourse over large spans. For
example, if a chef were to explain the sequence of adding ingredients to
a stir-fry, he might say, ‘You add the lemongrass, the ginger and then the
basil’ while making beat gestures (right hand repeatedly flipping outward
every time he mentioned an ingredient) throughout the sequence to
create a sense of rhythm and cohesive structure. These four types of
gestures are different from a fifth prominent gesture called an emblem.
Emblems, such as the ‘thumbs up’ or ‘peace sign’, are hand configurations
that have a culturally specified meaning, and unlike co-speech gesture,
they convey information independently of speech. In fact, emblems often
stand alone from speech – think of how you might communicate with
only a gesture (and there is a range of options here) to an aggressive driver
who just cut you off!
There are two elements of the speech–gesture relationship that are
particularly interesting and require further explanation. A crucial aspect of
co-speech gestures is the tight temporal synchrony with the accompanying
speech. According to McNeill (1992), co-speech gestures do not make
sense without the accompanying speech, and so it is very important to
study gestures in the context of the accompanying speech – that is, to study
them as a combined system, not as two separate things. Related to this,
the second key feature is that gesture and speech combine to reveal
meaning that goes beyond the sum of the two individual parts. Consider
an example. Suppose a friend describes to you how he got into an auto
accident by saying, ‘I didn’t see it coming’. In gesture, your friend might
represent how the cars collided by making two, perpendicular flat-handed
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
Gesture Gives a Hand 3
gestures that move toward one another (making a T shape). The addition
of this iconic gesture would provide a much clearer and more elaborate
representation of what happened: the reason your friend ‘didn’t see it
coming’ was that the other car blindsided him from the passenger side. In
this way, the simultaneous coupling of gesture and speech conveys information that is much more complete than the information conveyed
through speech or gesture alone.
The present review adopts the theoretical stance put forth by McNeill
and others that gesture and speech indeed do comprise an integrated
system. Although historically there has been some debate about the issue
– see Krauss 1998, and Krauss et al. 1991, for the view that gesture and
speech are two separate systems,1
and that gesture plays a role merely in
accessing words during language production – the majority of recent
research on the topic supports the integrated system account of gesture
and speech during language production and comprehension.
Specifically, the present review adopts an evolutionary and embodied
perspective on the relationship between the two modalities in language
use (for a more detailed account of this theoretical perspective, see Kelly
et al. 2002a). Researchers have argued that human language has evolved
from nonverbal communication systems in our evolutionary past (Rizzolatti
and Arbib 1998; Armstrong 1999; Bates and Dick 2002; Corballis 2003).
Although it is very difficult to prove this claim with archeological fossil
records, the present review assumes that we may gain insights into this
integrated relationship by focusing on present-day behavioral and neurocognitive
fossils (Povinelli 1993). That is, if spoken language systems emerged from
gestural communication systems in our evolutionary past, one would
hypothesize that this integrated relationship is carried through and
maintained in present-day language use and development. And if spoken
language is grounded in nonverbal communication systems, it suggests
that the neural mechanisms connecting gesture and speech may not be
solely linguistic in nature, but imagistic, motoric, and even emotional
as well.
So what is the empirical evidence to support this integrated view? We
begin by covering recent research in cognitive neuroscience that explores
the neural mechanisms for gesture–speech processing. We then examine
the role that gesture plays with speech in typically and atypically developing
children. Next, we highlight the importance of considering gesture in the
context of teaching and learning. Finally, we finish by outlining some
future directions in studying the integrated nature of gesture and speech
in communication.
The Cognitive Neuroscience of Gesture
It is now well established in behavioral studies in psychology that gesture
and speech have an integrated relationship in language production (Alibali
4 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
et al. 2000; Goldin-Meadow 2003; Kita and Özyürek 2003) and language
comprehension (Goldin-Meadow et al. 1992; Kelly and Church 1997,
1998; Beattie and Shovelton 1999; Cassell et al. 1999; Kelly et al. 1999).
To grossly sum up, these studies have shown that producing and comprehending speech is significantly influenced by the presence of co-speech
gestures. However, it is difficult to determine from this research the extent
of this integrated relationship – that is, are speech and gesture also linked
at a level beneath the overt behavior? Recently, researchers in the field of
cognitive neuroscience have addressed this question by investigating the
neural mechanisms that underlie cognitive and social processes and behaviors.
This line of work is very informative because it highlights neural processes
and structures that reveal why gesture and speech form an integrated
system on the psychological and social levels of analysis. Understanding
the relationship between gesture and speech on multiple levels allows for
more robust claims about this integrated system.
A glimpse of the neural relationship between speech and gesture was
first seen with the discovery of ‘mirror neurons’ in area F5 of monkeys,
the rostral part of the ventral premotor cortex and purported homologue
to Broca’s area (an important language region in the left hemisphere) in
the human brain (di Pellegrino et al. 1992; Gallese et al. 1996; Rizzolatti
and Arbib 1998). These neurons discharge when a monkey both executes
a specific manual action and when he observes another primate executing
the same action. Since their discovery, several papers have demonstrated
that the human brain, specifically Broca’s area, also has similar ‘mirror
properties’ (for a review, see Nishitani et al. 2005). This suggests a link
between neural areas responsible for hand actions and language.
This linkage between language and action areas of the brain has been
fleshed out by a number of recent experiments with humans using different
types of cognitive neuroscience methods (for a nice recent review, see
Willems and Hagoort 2007). Indeed, several studies have found that brain
regions that process speech also process actions made with the hand
(Bonda et al. 1996; Puce and Perrett 2003; Gallese et al. 2004; Nishitani
et al. 2005). For example, the superior temporal region (STS) in the left
hemisphere is implicated not only in processing sound-based representations of speech (Hickok and Poeppel 2000) but also goal-directed hand
movements (Bonda et al. 1996). In addition, evidence from research using
transcranial magnetic stimulation (which interferes with or enhances the
neural processing of stimuli) demonstrates that when there are disruptions
to parts of the brain that control hand movements, speech comprehension
also suffers (Flöel et al. 2003).
In another suggestive line of work, researchers have demonstrated that
the neural processing of action verbs is somatotopically organized in the
premotor and motor cortices (Pulvermuller et al. 2005; Tettamanti et al.
2005). For example, Tettamanti et al. (2005) used functional magnetic
resonance imaging (fMRI, which measures blood flow in the brain in
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
Gesture Gives a Hand 5
response to stimuli) to investigate the neural processing of sentences
containing verbs using different body parts (e.g. ‘catch’ vs. ‘kick’). They
found that sentences with the different verbs activated corresponding parts
of the premotor cortex (a part of the mirror neuron system in humans);
that is, ‘catch’ activated hand areas and ‘kick’ activated foot areas. As a
test of whether this neural activation plays a functional role in action verb
processing, Pulvermuller et al. (2005) applied transcranial magnetic stimulation
(in this case, enhancing neural processing) while participants listened to
different types of actions words. They found that stimulating hand regions
sped reaction times to hand words but not foot words and stimulating foot
regions sped reaction times to foot words but not hands words. This is
strong evidence that activating brain areas involved in action execution
plays a causal role in comprehending (and perhaps producing) language
about actions, and provides suggestive evidence that communicative hand
movements, such as gesture, may share a special link with speech
during language use.
Indeed, very recent work has begun to flesh out this neural link.
Montgomery et al. (2007), using fMRI,2
found that the inferior parietal
lobule, another area purported to be part of the human mirror neuron
system, was equally active when adults viewed, imitated and produced
communicative hand gestures (emblems such as the ‘thumbs up’ gesture)
and actions towards objects (iconic gestures such as pounding a nail). This
similar activation suggests that the human mirror neuron system may be
involved in producing and comprehending hand gestures in the absence
of speech.
Focusing on gestures that do accompany speech, researchers have started
to identify mechanisms for how the two modalities may be integrated in the
brain (Hubbard et al. 2007; Skipper et al. 2007; Willems et al. 2007;
Holle et al. 2008; Wilson et al. 2008). For example, Willems et al. (2007)
used fMRI to show that Broca’s area integrates gestural and spoken information in a similar fashion during sentence comprehension, suggesting a
common neural mechanism for processing the two types of communication. In addition, Skipper et al. (2007) used fMRI to show that Broca’s area
may be partly responsible for integrating gesture and speech. Specifically,
they had participants watch videos of different stories comprised of speech
and gesture and found that connections among Broca’s area and other
cortical areas (e.g. motor areas involved in gesture processing) were weaker
for stories containing congruent speech and gesture compared to stories
containing irrelevant self-grooming movements. This weaker connection,
according to the authors, reflected gesture’s ability to reduce the workload of Broca’s area in processing the meaning of the accompanying
speech. Moreover, recent research has shown that in addition to traditional
language areas, other brain regions may be recruited for comprehending
verbal and gestural utterances, such as multimodal integration sites (STS),
other parts of the mirror neuron system (inferior parietal lobule and
6 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
premotor regions) and emotional centers such as the cingulate cortex
(Calvert and Thesen 2004; Holle et al. 2008; Wilson et al. 2008). These
new findings suggest that gestures may interact with speech not just in a
linguistic way, but imagistic, motoric and affective fashions as well.
As a different test of whether gesture and speech form an integrated
system, researchers have used event-related potentials (ERPs, measuring
the brain’s electrical response to stimuli) to explore the online processing
(i.e. the immediate integration) of gesture and speech during language
comprehension (Kelly et al. 2004, 2007; Holle and Gunter 2007; Özyürek
et al. 2007; Wu and Coulson 2007a,b). For example, Wu and Coulson
(2007b) presented gesture–speech utterances followed by pictures that
were either related to gesture and speech or just the speech alone. When
pictures were related to gesture and speech, participants produced a
smaller N400 effect (the traditional semantic integration effect that occurs
400 ms after stimulus onset; see Kutas and Hillyard 1980) and N300 effect
(image-based semantic integration that occurs 300 ms after stimulus onset;
see McPherson and Holcomb 1999) than when the pictures were related
to just the speech. Because small N300 and N400 effects typically reflect
ease of integrating linguistic information, this finding suggests that visuospatial aspects of gestures facilitated subsequent processing of the speech.
In this way, gesture may have combined with speech to build stronger and
more vivid expectations of the pictures than just speech alone.
Researchers have also demonstrated that the semantic content of gesture
influences the processing of accompanying speech. For example, Kelly
et al. (2004) presented gesture–speech pairs that were congruent (saying
tall while gesturing the ‘tallness’ of a tall container) versus incongruent
(saying tall while gesturing the ‘shortness’ of a short container), and found
that incongruent gestures produced a larger N400 effect to speech than
congruent gestures. More recently, Özyürek et al. (2007) demonstrated
that the semantic content of gesture and speech are processed similarly
(i.e. identical N400s to words and gestures that were incongruent with a
previous sentence context), providing a nice complement to their fMRI
work (Willems et al. 2007) demonstrating a shared mechanism for the
semantic processing of the two modalities in the brain. Moreover, the
semantic integration of gesture and speech is affected by contextual factors
(Holle and Gunter 2007; Kelly et al. 2007). For example, Holle and
Gunter (2007) showed that the presence of non-meaningful gestures (e.g.
grooming behaviors) influences the extent to which meaningful gestures
are integrated with speech – that is, the presence of non-meaningful
gestures reduces the size of the N400 effect to incongruent meaningful
gestures – during sentence processing. This effect of context is exciting
because it suggests that the integration of gesture and speech may not be
an entirely automatic and obligatory neurocognitive process.
Together, these studies from the field of cognitive neuroscience
complement the work from psychology – gesture influences the behavioral
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
Gesture Gives a Hand 7
processing of speech during language production and comprehension, and
one explanation for this behavioral finding is that gesture and speech are
integrated in space and time in the brain’s processing of this information.
We next turn to the implications of this neural relationship in the domain
of language and cognitive development.
Gesture and Development
Another way to explore the relationship between gesture and speech is to
study children. Indeed, researchers have theorized that if gesture and
speech form an integrated system, gestures should play an important role in
language and cognitive development (Bates and Dick 2002; Kelly et al. 2002b).
Deictic and iconic gestures are pervasive in children’s speech. Children
produce deictic gestures before they begin to talk (Bates 1976; Butcher and
Goldin-Meadow 2000), and shortly thereafter (usually by 18 months), they
produce iconic gestures along with their speech (Bates et al. 1979; Masur
1983; Morford and Goldin-Meadow 1992; Iverson et al. 1994; Butcher
and Goldin-Meadow 2000). Throughout childhood, deictic and iconic
gestures become more complex and frequent (Jancovic et al. 1975;
McNeill 1992), and children produce them in a number of different
contexts: with friends (Azmitia and Perlmutter 1989; Church and AymanNolley 1995), family (Bates 1976), and teachers (Fernandez et al. 1996).
They also use gestures while talking about a number of different topics:
telling stories (McNeill 1992), giving directions (Iverson and GoldinMeadow 1997), and explaining concepts (Church and Goldin-Meadow
1986; Perry et al. 1988, 1992).
Several studies suggest that the gestures children produce while speaking
reveal much more about what they are thinking than does their speech
alone (Church and Goldin-Meadow 1986; Perry et al. 1988; 1992; Alibali
and Goldin-Meadow 1993; Goldin-Meadow et al. 1993; Church et al.
1995; Garber 1997; Alibali 1999; Church 1999; Goldin-Meadow 2000).
For example, in a study investigating the role of gesture in children’s
explanations of Piagetian (1967) conservation problems, Church and
Goldin-Meadow (1986) discovered that children frequently produced
iconic gestures in their explanations and that those gestures conveyed
different information than the spoken component of the explanations
(gesture–speech mismatches). For example, one child explained that two
containers of water (a tall, thin glass and a short, wide dish) were different
because, ‘One was short and one was tall’, while simultaneously gesturing
that the two containers had different widths. This suggests that the child
knew more about the problem than his speech let on. The ‘mismatch’
phenomenon generalizes to more traditional educational domains as well,
such as mathematics. For example, Perry et al. (1988, 1992) found that
when 10-year-old children solve math problems (e.g. 3 + 4 + 5 = __ + 5),
their deictic gestures often reflect different strategies than does their speech.
8 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
For example, a child producing ‘speech–gesture mismatch’ might say that
he added up all the numbers on the left side to get his answer, but he may
simultaneously point to all of the numbers in the equation, including ‘5’
on the right-hand side of the problem.
One of the most interesting findings in these studies (Church and
Goldin-Meadow 1986; Perry et al. 1988) was that the children who
produced many gesture–speech mismatches in their explanations were
precisely the ones who later benefited most from instruction on those
problems. This finding suggests that speech and gesture can serve as
an index of transitional, implicit knowledge in a specific domain and may
be a way of determining a child’s ‘readiness to learn’. The claim that
gestures reflect implicit knowledge and readiness to learn fits nicely with
educational research arguing that teachers can better interpret a student’s
work by being aware of that student’s underlying or implicit understanding
of a topic (Ball 1993; Carpenter et al. 1996, 1998). For example, Carpenter
et al. (1996) argued that awareness of children’s implicit understanding of
mathematical concepts could allow teachers to better assess and instruct
children in that domain. If hand gestures are a window into this implicit
knowledge, attention to this information may make this task easier. We will
return to the educational implications of gesture in a later session.
Gesture and Atypical Development
Researchers have also investigated the role of gesture in atypical development. With regard to language production, one interesting finding is that
congenitally blind children gesture when they speak (Iverson and GoldinMeadow 1998). Moreover, these children gesture even when speaking to
other blind children. The fact that language learners produce gestures
without ever having seen them is some of the best evidence that speech
and gesture form a tightly integrated system indeed.
This integrated relationship can also be seen in children with developmental delays and disorders. For example, deficits in gesture production
positively correlate with deficits in cognition and language (Thal and Bates
1988; Thal et al. 1999; Charman et al. 2003). Charman et al. (2003) found
that preschool-aged children with autism spectrum disorder, a disorder that
compromises communication and social cognition, were delayed compared
to typically developing children in their ability to produce deictic gestures
that involved shared reference (e.g. pointing to an object that can be seen
by two people). Moreover, research focusing on children with Specific
Language Impairment (SLI), a disorder that causes delays in language
but generally spares other cognitive abilities, has also uncovered gesture
production deficits. Toddlers with SLI, or who are at risk of SLI,
demonstrate deficits in their ability to produce and imitate symbolic
gestures (e.g. flapping hands to represent a birds) compared to typically
developing peers (Thal and Bates 1988; Thal et al. 1999).
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
Gesture Gives a Hand 9
Although gesture is often impaired in children with developmental
delays and disorders, there is evidence that gestures may at times compensate
for cognitive and language deficits as well (Harris et al. 1997; Iverson et al.
2003; Stefanini et al. 2007). For example, children with Down syndrome
produce more gestures than children with Williams syndrome (Harris et al.
1997) and also typically developing children (Caselli et al. 1998). This
finding suggests that Down syndrome children, more than Williams
syndrome and typically developing children, may use hand gestures to
‘make up’ for language problems (Stefanini et al. 2007; but see Iverson
et al. 2003 for a different view). The compensatory use of gesture apparently
also occurs in children with SLI. For example, Evans et al. (2001) found that
children with SLI often presented information uniquely in their gesture
compared to their speech. Moreover, this gestural information was often
more sophisticated than their speech. The authors concluded that because
SLI children have phonological deficits in their speech, gesture allows them
to reveal knowledge in an embodied format that is easier for them to
express, and in this way, they may be able to overcome their linguistic
difficulties by simply moving their hands.
Gesture may also be used to predict developmental delays (Mundy et al.
1995; Brady et al. 2004; Mitchell et al. 2006; Luyster et al. 2007; Smith
et al. 2007). Brady et al. (2004) found that the children with general developmental delays who produced pointing gestures showed greater increase
in expressive language abilities than children who did not point. This
suggests that deictic gestures may be indicative of future language growth
in developmentally delayed children. Gestures are also useful for predicting
language outcomes in children with autism. For instance, Smith et al.
(2007) discovered that in children with autism, the number of gestures used
to initiate joint attention – the ability to direct and share attention with
another – was linked to greater vocabulary growth after two years of
intervention. Moreover, Mitchell et al. (2006) demonstrated that gestures
might be better than speech in the early identification of the disorder.
Using the MacArthur Communicative Development Inventory, the
researchers found that the gestural repertoires (e.g. the production of
deictic and emblematic gestures) of 12-month-old infants differentiated
children who were and were not later diagnosed with autism spectrum
disorder at 24 months of age. Impressively, these gestural indicators were
evident a full 6 months before any verbal patterns differentiated the two
groups of children.
If gestures are helpful in early identification of developmental disorders,
they may be useful in early interventions as well (McCathren 2000; Whittaker
and Reynolds 2000; Yoder and Warren 2001a,b; Calculator 2002; Brady
et al. 2004). Brady et al. (2004) found that high parental responsiveness to
the gestures of children with language delays predicted higher expressive
and receptive language scores in those children. Given the positive effects
of parental responsiveness, it makes sense to explicitly instruct parents
10 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
to attend to gesture. Indeed, Calculator (2002) trained parents of children
with Angelman syndrome – a disorder involving disfluent speech and
general motor deficits – to recognize enhanced natural gestures, which are
intentional behaviors in a child’s motor repertoire (e.g. teaching the child
to use a ‘drink’ gesture to request a glass of milk). Parents reported that
this training greatly helped them recognize more of their children’s gestures
and communicative attempts. With regard to parental gestures, McCathren
(2000) showed that training parents to imitate certain hand and body
gestures of developmentally delayed children was effective at enhancing
intentional communication between parent and child.
Another approach is to train developmentally delayed children to
compensate for delays by using their own gestures more effectively
(Buffington et al. 1998; McCathren 2000; Whittaker and Reynolds 2000;
Yoder and Warren 2001b; Calculator 2002). Buffington et al. (1998)
attempted to teach autistic children appropriate gestural responses to stimuli.
As a pretest, they presented verbal and nonverbal stimuli to children and
observed their gestural and verbal responses. Then there was a gestural
training phase that included physically modeling, prompting, and reinforcing
correct responses. After training, the researchers found that the children
increased their appropriate gestural responses to novel stimuli as they
progressed through treatment. The authors concluded that if children with
autism can learn early on to produce socially appropriate gestures, they
may have more successful social interactions later in life.
It is important to note that many of the gestures identified in this section
are different from the standard ‘co-speech’ gestures discussed in this paper.
Indeed, the majority of gestures in the above studies are emblematic
or deictic gestures that occur in isolation from speech. Given the importance
of understanding the relationship between speech and gesture, an exciting
direction for future research is to focus on the combination of the two
modalities in order to provide an additional layer of clinical insight into the
various linguistic, cognitive and social disorders that develop in childhood.
Gesture and Education
Even a casual observation of teachers and students interacting in the
classroom will reveal that gestures are as pervasive as blackboards, desks
and lesson plans. Because gesture is so prevalent in this environment, it is
important to consider what role these hand movements play in educational
situations involving teaching and learning.
One powerful role is that gesture production may help children (and
adults) free up cognitive capacity when communicating about conceptual
problems (Goldin-Meadow et al. 2001; Cook and Goldin-Meadow 2006).
For example, Goldin-Meadow et al. (2001) had 10-year-old children
and adults explain their understanding of difficult math problems (e.g.
3 + 4 + 5 = __ + 5 for children; X + 2X + X/2 = 21 for adults) with and
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
Gesture Gives a Hand 11
without deictic gestures. Simultaneously, they were asked to perform
a cognitive load task requiring them to remember short lists of words. The
main finding was that participants remembered the most words when they
gestured than when they did not gesture. This effect generalized even
when the cognitive load was more spatial in nature, but the effect disappeared
when the semantic content of the gesture was not congruent with the
accompanying speech (Wagner et al. 2004). In addition, children who
produce gestures in these sorts of math problems are more likely to sustain
learning over long periods of time (Cook et al. 2008). Finally, explicitly
requiring children to move their hands while explaining their (incorrect)
understanding of these types of math problems causes them to produce
new strategies in gesture not previously conveyed in their speech, and makes
them more receptive to subsequent instruction on how to correctly solve
the problems (Broaders et al. 2007).
Gestures also influence how information is exchanged between teachers
and students during learning sessions. For example, Goldin-Meadow and
Sandhofer (1999) observed natural adult–child interactions and discovered
that adults often incorporated children’s deictic and iconic gestures into
what they thought that children had verbally explained (about Piagetian
conservation problems) in their speech. This has obvious educational
implications. For example, if a child verbally described the different heights
of two objects but gesturally represented the different widths, adults may
make an assessment of that child’s knowledge that incorporates height and
width. This sensitivity to gesture is, not surprisingly, evident in teachers as
well. Alibali et al. (1997) demonstrated that teachers often incorporate
gestured information into their assessments of children’s knowledge of
mathematical equivalence (i.e. problems of this form, 3 + 4 + 5 = __ + 5).
For example, if a child points to a number that he did not mention in his
speech, teachers often incorporate that gestured number into an assessment
of what that child knows about the problem. However, there is room for
improvement. Kelly et al. (2002b) found that training adults to pay attention
to gesture helped them to better make assessments of children’s knowledge
of mathematical problems. For example, if regular adults – not teachers – are
given a 5-minute instructional session on how children’s gestures can add
to speech, and are then shown videos of children producing mismatches
(e.g. a child saying that he added up numbers only on the left, but pointing
to all numbers in a problem), the adults incorporate gesture into their
assessments of the children’s knowledge more than adults without training.
Children are also sensitive to gesture in contexts of mathematical
reasoning and learning (Kelly and Church 1997, 1998; Church et al.
2001). For example, Church et al. found that 10-year-old children showed
a better understanding of mathematical concepts when nonverbal gestures
accompanied verbal instruction on the concepts. This effect generalizes
from the laboratory to actual educational interactions between children and
teachers: Singer and Goldin-Meadow (2005) found that when teachers use
12 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
gestures during math instruction, 10-year-old children learn the concepts
better than when teachers use speech alone.
Given that gestures play a role in teaching and learning, teachers should
be able to take advantage of gesture – their own and children’s gestures – in
the classroom. For example, Pozzer-Ardenghi and Roth (2007) have recently
studied teacher–student interactions during high school biology lessons and
found that for many concepts, hand gestures provided additional clarifying
input for students. They reasoned that hand gestures and other visual aids
might help students who are struggling with advanced concepts that are
not easily represented and taught through speech alone. Indeed, when
teaching first-grade children about basic mathematical concepts (i.e. counting
numbers of objects), teachers frequently use nonverbal behaviors such
as pointing, counting on fingers, circling objects with the finger, etc.
(Flevares and Perry 2001). Interestingly, this visual clarification occurs more
frequently when students appear confused. Moreover, this increased visual
instruction often occurs in the absence of increased verbal clarification. In
other words, teachers specifically use things like gestures to target students
who struggle with mathematical concepts.
Teachers can also use gestures to help struggling learners in other
domains as well. For example, when second language learners grapple with
aspects of a new language, teachers can use gesture to help with these
problems. In fact, in a recent literature review on the role of gesture in
second language learning, Gullberg (2006) outlined several reasons why
hand gestures may be a crucial tool in helping struggling learners master a
new language. For example, she argues that visually rich gestures, such as
iconic gestures, serve as ideal input to beginning learners of a second
language. Although researchers have long claimed that teachers’ gestures
enhance second language instruction for learners (Moskowitz 1976), there
is a surprising paucity of experimental work investigating this issue (but see
Allen 1995; Sueyoshi and Hardison 2005). In one recent study on this
topic, Kelly et al. (2007) found that iconic gestures helped English-speaking
adults to learn and remember novel Japanese words. For example, producing
a drink gesture while saying, ‘Nomu means drink’, helped people remember
the meaning of the Japanese word. Interestingly, the gestures did not help
simply because they captured visual attention – in fact, the gestures
facilitated learning only when they conveyed congruent information to
speech, but they disrupted learning when they conveyed incongruent
information. So gesture does not promote learning merely because of
hand waving – content matters.
In perhaps the most extreme demonstration that gestures help confused
learners in second language contexts, Church et al. (2004) studied how
gestures aid first-grade Spanish speakers (with English as a second
language) learn novel mathematical concepts (like the Piagetian conservation
problems described above) in their second language. The children – who
did not speak any English – were shown instructional videos in English
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
Gesture Gives a Hand 13
that contained or did not contain gesture. Remarkably, the children
improved twice as much in their understanding of the mathematical
concepts when verbal instruction included gesture. The authors argued
that although the Spanish speakers did not understand the verbal portion
of the training, the gesture represented universal aspects of the mathematical
concepts, and these aspects are accessible even when someone does not
speak a language.
Future Directions
Although there has been a flurry of research on gesture within the last
decade, there are many avenues yet to be explored. In addition to delving
deeper within disciplines, there are new and stimulating opportunities to
make connections between research areas. One exciting direction is to
bring cognitive neuroscience techniques into traditional developmental,
clinical and educational domains. For example, Sheehan et al. (2007) have
recently completed the first developmental study to investigate the neural
processing of gesture and speech in young children. Using ERPs, they
found that speech and gesture appear to share a similar neural system in
early stages of language acquisition (18 months), but the two systems began
to diverge shortly thereafter at 2 years of age. Given what we know about
the integrated neural relationship of gesture and speech in adults, it would
be interesting to explore when and how the two systems come back
together at later stages of development.
Connecting to clinical psychology, cognitive neuroscientists have
recently advanced theories that autism may be a disorder caused by a
breakdown in the mirror neuron system (Williams et al. 2001), and new
research has just recently started to use fMRI to investigate neural correlates
of autistic children’s inability to successfully imitate nonverbal behaviors
(Dapretto et al. 2006). In the work by Dapretto et al., the researchers
showed that the higher an autistic children’s score on a test that involved
imitating faces, the more their inferior frontal gyrus was active. Because the
inferior frontal gyrus is also implicated in the integration of hand gesture
and speech (Skipper et al. 2007; Willems et al. 2007), it would be interesting
to extend this research with autistic children into the realm of imitating
language and hand gestures.
With regard to education, researchers are just beginning to explore the
neural role that gesture plays when people master and retain new information
in learning/teaching contexts. For example, Kelly et al. (2007) recently
used ERPs to investigate whether gestures play a role in second language
learning. In an experiment that taught adults novel Japanese verbs with and
without iconic hand gestures, they demonstrated that words learned with
gesture produced deeper and stronger neural memory traces, as measured
by the Late Positive Complex, which indexes strength of memory encoding
(Rugg and Curran 2007). This is an exciting development, because it
14 Spencer D. Kelly et al.
© 2008 The Authors Language and Linguistics Compass 2 (2008): 10.1111/j.1749-818x.2008.00067.x
Journal Compilation © 2008 Blackwell Publishing Ltd
suggests that gesture and speech not only have an integrated neural
relationship during language comprehension, but this relationship may
be preserved in lasting memories formed as a product of learning.
Another interesting direction in gesture research connects to a very
different domain – computer science. With people increasingly interacting
with computing technologies, computer scientists have recognized the
importance of creating more life-like ‘embodied conversational agents’ that
serve as the interface between humans and machines (Wachsmuth 2002;
Kopp and Wachsmuth 2004; Cassell 2007; Cassell et al. 2007). This work
has demonstrated how the integrated systems view of gesture and speech
has helped programmers design more natural conversational agents – indeed,
people like interacting with gesturing conversational agents better than
non-gesturing ones (Cassell et al. 2001). On the flip side, computer models
and simulations have in turn informed theories of gesture and speech. For
example, Cassell et al. (2007) has recently created an embodied conversational agent that spontaneously produces novel gestures along with speech,
providing a computational model of how the form of gesture and the
content of speech interface to capture a common representation and
pragmatic function. This sort of research is interesting because it represents
early steps to computationally test theories of how gesture and speech
interact in real human face-to-face communication.
