Efficient ocular expression analysis for synthetic reproduction

This paper presents an original framework that analyzes the complete ocular (eye + eyebrow) expression on video sequences to reproduce it later on synthetic three-dimensional (3-D) models in real-time. We propose a two step process to develop robust techniques for facial feature analysis aimed at working without special illumination conditions or physical constraints on the user (markers, fixed frontal pose, etc.). First, simple and efficient image-processing methods based on motion models are designed over a frontal point of view of the face. Natural and realistic intra-feature and inter-feature constraints are applied to improve the analysis results. Then, the usability of these algorithms is extended to enable the analysis regardless of the person's pose in front of the camera. This is achieved by redefining the motion models involved over the speaker's highly realistic synthetic representation (clone), by using a suitable observation model, and by predicting the head pose in 3-D, frame by frame.