An analysis/synthesis cooperation for head tracking and video face cloning

In the context of a face cloning system, we present a head tracking algorithm based on an enhanced analysis/synthesis feedback loop which is able to handle very large rotations out of the image plane. The key idea is to make the feedback loop synthesize search patterns for the head facial features by taking into account the current face position, rotation and lighting. As pointed out by the work of A. Gagalowicz, analysis/synthesis cooperations are very promissing, but require a high level of realism from the synthesis module. Consequently, we present geometric and photometric head modeling techniques that are realistic and computationally efficient. Finally, we reformulate a classic differential block{matching algorithm to integrate real and synthesized facial features. In addition, the feature tracking will be shown to be robust to the speaker's background, and the system performance are reported and discussed.