MULTIMEDIAT: MULTImodal MEdiation of Debates with conversatIonal AgenTs

Project Summary

This project explores how multimodality and tools can enhance socially intelligent agents, particularly focusing on understanding and modeling non-verbal phenomena to mediate a political debate. The first objective is to develop large language model (LLM)-based mediator agents that analyze arguments, detect stances and values, and guide discussions toward consensus using argument-mining and conflict- resolution strategies. The second objective extends this approach to multimodal data, integrating speech, facial, and gestural cues into Embodied Conversational Agents (ECAs). Using transformer-based and information-theoretic fusion methods, these agents will both interpret and generate socially coherent non-verbal behavior. The expected outcome is a new generation of socially aware ECAs capable of facilitating empathic and constructive human–AI interactions for debate mediation.

In France

• Chloé Clavel, researcher, MULTIMEDIAT coordinator, ALMAnaCH project-team, Inria Paris Centre, Inria

• Djame Seddah, researcher, ALMAnaCH project-team, Inria Paris Centre, Inria

• Celia Nouri, PhD student, ALMAnaCH project-team, Inria Paris Centre, Inria

• Yi Yu, PhD student, ALMAnaCH project-team, Inria Paris Centre, Inria

• Yannis Karmim, postdoctoral researcher, ALMAnaCH project-team, Inria Paris Centre, Inria, Inria Chile 

In Chile

• Valentin Barrière, researcher, MULTIMEDIAT coordinator, Universidad de Chile

• Jorge Silva, researcher, Universidad de Chile

• Benjamín Farías, PhD student, Universidad de Chile

• Mitchell Bosley, postdoctoral researcher, Centro Nacional de Inteligencia Artificial (CENIA)

• José Guillén, R&D engineer, Centro Nacional de Inteligencia Artificial (CENIA)