Do you speak to a human or a virtual agent? automatic analysis of user’s social cues during mediated communication

Abstract : While several research works have shown that virtual agents are able to generate natural and social behaviors from users, few of them have compared these social reactions to those expressed during a human-human mediated communication. In this paper, we propose to explore the social cues expressed by a user during a mediated communication either with an embodied conversational agent or with another human. For this purpose, we have exploited a machine learning method to identify the facial and head social cues characteristics in each interaction type and to construct a model to automatically determine if the user is interacting with a virtual agent or another human. The results show that, in fact, the users do not express the same facial and head movements during a communication with a virtual agent or another user. Based on these results, we propose to use such a machine learning model to automatically measure the social capability of a virtual agent to generate a social behavior in the user comparable to a human-human interaction. The resulting model can detect automatically if the user is communicating with a virtual or real interlocutor, looking only at the user's face and head during one second.
Complete list of metadatas

Cited literature [33 references]  Display  Hide  Download
Contributor : Magalie Ochs <>
Submitted on : Tuesday, June 12, 2018 - 9:01:15 AM
Last modification on : Sunday, March 17, 2019 - 10:26:08 PM
Long-term archiving on : Thursday, September 13, 2018 - 12:51:13 PM


Files produced by the author(s)



Magalie Ochs, Nathan Libermann, Axel Boidin, Thierry Chaminade. Do you speak to a human or a virtual agent? automatic analysis of user’s social cues during mediated communication. ICMI 2017 - 19th ACM International Conference on Multimodal Interaction, Nov 2017, Glasgow, United Kingdom. pp.197-205, ⟨10.1145/3136755.3136807⟩. ⟨hal-01793356⟩



Record views


Files downloads