When words fail: Collaborative gestures during clarification dialogues

TY - CHAP

T1 - When words fail

T2 - 2015 AAAI Spring Symposium

AU - Healey, Patrick G T

AU - Plant, Nicol

AU - Howes, Christine

AU - Lavelle, Mary

PY - 2015

Y1 - 2015

N2 - Research on co-speech gestures has primarily focussed on speakers. However, in conversation non-speaking addressees can also gesture. Often this is to provide concurrent feedback such as backchannels but sometimes it involves gestures that relate to the specific content of the speaker's turn. We hypothesise that non-speakers should contribute most actively during clarification sequences i.e. at the moments when mutual-understanding is threatened. We test this hypothesis using a corpus of story-telling dialogues, captured using video and motion capture. The results show that during clarification sequences speaker and non-speaker behaviours tend to merge. Non-speakers in particular move their hands faster and produce more than twice as many content-specific specific gestures in overlap with speaker turns. These results underline the collaborative nature of conversation, the strategic importance of nonverbal resources for sustaining mutual-understanding and the critical role of clarification and repair in successful communication.

AB - Research on co-speech gestures has primarily focussed on speakers. However, in conversation non-speaking addressees can also gesture. Often this is to provide concurrent feedback such as backchannels but sometimes it involves gestures that relate to the specific content of the speaker's turn. We hypothesise that non-speakers should contribute most actively during clarification sequences i.e. at the moments when mutual-understanding is threatened. We test this hypothesis using a corpus of story-telling dialogues, captured using video and motion capture. The results show that during clarification sequences speaker and non-speaker behaviours tend to merge. Non-speakers in particular move their hands faster and produce more than twice as many content-specific specific gestures in overlap with speaker turns. These results underline the collaborative nature of conversation, the strategic importance of nonverbal resources for sustaining mutual-understanding and the critical role of clarification and repair in successful communication.

UR - http://www.scopus.com/inward/record.url?scp=84987624142&partnerID=8YFLogxK

M3 - Chapter

AN - SCOPUS:84987624142

SN - 9781577357117

VL - SS-15-07

SP - 23

EP - 29

BT - AAAI Spring Symposium - Technical Report

PB - AI Access Foundation

Y2 - 23 March 2015 through 25 March 2015

ER -