Abstract
While the concept of ‘face space’ (Valentine, 1991) has been highly influential in representing identity, it has seen fewer applications in representing within-person variation such as changes in appearance with lighting, expression, speech and pose. Recently, Burton and colleagues (2016) proposed that in addition to an identity-general space that captures between-individual variation, we may also have a separate, identity-specific space for each familiar individual that captures their within-person variation. We addressed the problem of viewpoint invariance using expression-based PCA spaces. As there is currently little evidence for a 3D representation in humans, we assessed the possibility of identity-specific, multi-view face spaces, based on a few prototypical 2D views, that can be used to represent and reconstruct facial motion across changes in viewpoint. When we concatenate video from five views, captured simultaneously, together into multi-view vectors and create a single multi-view PCA space, we are able to reconstruct facial motion across views remarkably well from a single input view. However, this model lacked biologically plausibility due to the need to ‘see’ multiple views simultaneously, so we then assessed more biologically plausible models. If we make a separate space for each viewpoint, we are able to reconstruction motion across views well, but only once the association between the components across views is learned. We also explored a model in which neighbouring views were concatenated in time, creating a single space with overlapping viewpoints. This avoided the requirement for separate, view-specific spaces, whilst still also allowing us to reconstruct motion across views well. These models, although a work in progress, show that a 3D representation might not be necessary for certain face-related tasks such as processing motion and speech across views, and that a few 2D representations might be sufficient.