Separation of identity and expression information in 3D scans of human faces

Our work is motivated by the problem of automatic face recognition, a difficult task, still missing a general solution. Its complexity lies in the wide range of variations presents in the input data, due to different lightings, background scenes and head positions. Moreover, the face appearance is affected by internal sources of variations: on a long temporal scale aging and weight gain, and on a short scale the action of the facial muscles. An effective recognition algorithm should be insensitive to all these sources of variations. During the last decade, good results to the recognition problem have been obtained using 3D Morphable Models (3DMMs). Their use allowed to separate the data variations due to the identity from the ones due to external sources like the lighting conditions. However, other internal sources were not considered. Our goal is to include expressions as an additional source of internal variation in 3DMMs, enabling us to recognize faces not only under different illuminations and pose conditions, but also with different expressions. In general, the construction of a 3DMM requires a corpus of training data; for our task we need a training set including examples of both identity and expression variations. Unfortunately, their acquisition alone is not sufficient, since they have to be previously registered with a reference 3D head model. The registration of 3D scans of expressions is a difficult problem, which could not be solved with the registration algorithm previously used. The main contribution of our work is a new registration algorithm which can cope with arbitrary expressions in the 3D data. Our algorithm is also capable of registering data with missing values, an important property since virtually no 3D acquisition devices is immune to holes and artifacts in the output. Given the training set of registered 3D examples, we construct a 3DMM where identity and expression variations are represented with two separate linear Gaussian models. The two models are then linearly combined, yielding an expression-identity 3DMM which we apply to the problem of 3D face recognition. Although this modeling approach does not take into account the interdependency between expressions and identity, the recognition performance is not negatively affected.