PANORAMA
User AdaPtive Artificial INtelligence fOR HumAn CoMputer InterAction
Overview:
The key concept of this project is “user adaptive AI in the context of human-computer interaction”.
This project addresses two aspects for this concept.
First, we will conduct research on user adaptivity of artificial intelligence embodied as a conversational agent.
When people talk to other people, they change their verbal and nonverbal communication behaviors according to those of the partner. Therefore, user adaptivity is an essential issue in improving human-agent interaction. Communication style is also different depending on the culture, and adapting the agent behaviors to a target culture is useful in system localization.
We will tackle these problems by employing a machine learning approach.
However, a bottleneck of this approach is that annotating users’ nonverbal behaviors to create training data is time consuming. We will solve this problem by exploiting Explainable Artificial Intelligence (XAI) technique, through which labels predicted by the system is adapted based on the interaction with the user as an annotator. Thus, the concept for user adaptive AI is used to support users in creating multimodal corpus as well as improve the human-agent interaction.
Moreover, the concept of user adaptivity is also focused on the psychological studies in this project, in which user motivation will be investigated in one relevant use case (personalised motivational coaching for physical activity).
Therefore, this project envisions a new research methodology for machine-learning-based conversational agents by focusing on the concept of user adaptivity.
Scientific and Technical Goals
Propose a user adaptive multimodal annotation tool based on XAI techniques,
Exploit this tool to collect annotated multimodal corpora in four countries (Japan, France, Germany and India)
Propose models and methods for developing conversational agents with multi-level adaptation functionality, where nonverbal signals of the agent as well as the content of the dialogue are adapted to the user.
Provide multitask learning and transfer learning techniques to learn models using the multi-cultural corpus obtained in (2) and adapt the conversational agent to each culture.
Propose the design basis of adaptive AI systems grounded in psychological theories and evaluation studies.
Keywords: Adaptive HCI, eXplainable AI, Motivation theory, ECA, Dialogue system, Reinforcement learning, Transfer learning
Synergy of international collaboration:
This project team covers different aspects of research on adaptive AI: theoretical/psychological foundation, XAI, Embodied Conversational Agent (ECA), and machine learning. The work packages are designed to emerge new research directions in AI interfaces. Therefore, this collaboration provides good opportunities for young researchers to broaden their skills and research interests.
Impact of the proposed innovation
Adaptive AI interface impacts the economy and the future society. First, user adaptive technology enhances the quality of human-computer interaction and this contributes to improve task performance and productivity of the users in industries. Moreover, user adaptive AI will effectively motivate the users to adopt a healthy lifestyle in the long term, and this may change people’s lives in the future society.
OBJECTIVES
Overall objective: This project provides technological and psychological foundations for user adaptive and culture-sensitive AI systems by designing advanced Embodied Conversational Agents (ECA).
Scientific objectives
provide the foundations for cross-cultural corpora creation and learning
Provide methods for creating personal and cultural adaptivity in ECAs
Provide methods for adapting ECA’s behavior to the interaction context and partners
Provide the psychological theories of interindividual differences as a basis for designing and evaluating user adaptive agents (consistent consideration of personality and culture)
Technical working objectives
Reduce the effort required for data collection and annotation by leveraging
Cooperative machine learning
Transfer learning (across cultures) unlike earlier work not just language, but also nonverbal signals (gestures, postures etc.)
Data augmentation
Models for user adaptive ECA
Models for adaptiving nonverbal behaviors of ECA
Models for controlling dialogues in user adaptive ECAs based on multimodal information
Funding
France: Agence Nationale de la Recherche https://anr.fr/Projet-ANR-20-IADJ-0008
Japan : https://www.jst.go.jp/kisoken/aip/en/program/research/trilateral2020.html (Grant Number : JST AIP Trilateral AI Research (grant no. JPMJCR20G6), Japan.)
Germany: Deutsche Forschungsgemeinschaft (DFG) https://gepris.dfg.de/gepris/projekt/442607480