Members of different
cultures may behave quite differently from one another when interacting
face-to-face. Culture-specific aspects of speech and nonverbal behavior are
signals that enable members of a culture to establish and maintain a sense of
rapport with one another over intervals of interaction. Rapport and the means
by which conversation partners achieve it, is important to study
systematically, because rapport is known to increase the likelihood of success
of goal-directed interaction, and also to promote knowledge sharing and
learning.
Subtle
cues signal engagement, endorsement, or appreciation. In the verbal channel,
these include mirroring of word choices and of grammatical structures as well
as vocal feedback. Similarly, many dimensions of nonverbal behavior such as
posture, gaze, nods, and gesticulation, signal - both to the conversation
participants and to observers of them -the extent to which the participants feel
a sense of affiliation. A multidisciplinary team of psychologists,
anthropologists, linguists, and computer scientists, will examine and compare
such indices of rapport in natural interactions across members of three diverse
language/cultural groups: Gulf/Iraqi Arabic-, Mexican Spanish-, and American
English-speaking cultures.
An
integral part of this project is to further develop technologies that enable
micro-analyses of synchronized gesture and speech, both within individuals and
across conversation partners. These technologies include computer interfaces
for visualization, annotation, and analysis of complex, multimodal behaviors.
The project will develop techniques to automatically recognize states of
rapport and the verbal and nonverbal signals that lead to its disruption.
The
results of these examinations of human-human interaction will then be used to
program behavioral repertoires for "Embodied Conversational Agents" (ECAs).
These are computer-generated, two-dimensional figures, human in appearance and
capable of a range of verbal and nonverbal behaviors characteristic of listeners
in interactions. The team will study human research participants in interaction
with the ECAs, manipulating, in the ECAs, aspects of behavior identified as
related to establishment, maintenance, and disruption of rapport, so as to
observe the effects on human participants. The cross-cultural dimension of this
comparative study will make it possible to identify what aspects of behavior
are crucial for scaffolding successful intercultural interaction and
communication among people of different cultures.