MAUI avatars: Mirroring the user's sensed emotions via expressive multi-ethnic facial avatars

In this paper we describe the multimodal affective user interface (MAUI) we created to capture its users’ emotional physiological signals via wearable computers and visualize the categorized signals in terms of recognized emotion. MAUI aims at (1) giving feedback to the users about their emotional states via various modalities (e.g. mirroring the users’ facial expressions and describing verbally the emotional state via an anthropomorphic avatar) and (2) animating the avatar's facial expressions based on the users’ captured signals. We first describe a version of MAUI which we developed as an in-house research tool for developing and testing affective computing research. We also discuss applications for which building intelligent user interfaces similar to MAUI can be useful and we suggest ways of adapting the MAUI approach to fit those specific applications.