Race-specific norms for coding face identity and a functional role for norms

Abstract

Models of face perception often adopt a framework in which faces are represented as points or vectors in a multidimensional space, relative to the average face that serves as a norm for encoding. Since faces are very similar in their configuration and share many visual properties, they could be encoded in one common space against one norm. However, certain face properties may result in grouping and “subclassification” of similar faces. We studied the processing of faces of different races, using high-level aftereffects, where exposure to one face systematically distorts the perception of a subsequently viewed face toward the “opposite” identity in face space. We measured identity aftereffects for adapt–test pairs that were opposite relative to race-specific (Asian and Caucasian) averages and pairs that were opposite relative to a “generic” average (both races morphed together). Aftereffects were larger for race-specific compared to mixed-race adapt–test pairs. These results suggest that race-specific norms are used to code identity because aftereffects are generally larger for adapt–test pairs drawn from trajectories passing through the norm (opposite pairs) than for those that do not. We also found that identification thresholds were lower when targets were distributed around race-specific averages than around the mixed-race average, suggesting that norm-based face encoding may play a functional role in facilitating identity discrimination.

Introduction

Humans recognize faces with remarkable accuracy, despite the fact that faces all share the same basic structure and differ only in subtle aspects of featural and configural information. This ability is thought to rely on adaptive face-coding mechanisms that are dynamically updated in response to changes in the statistics of faces encountered over time (for recent reviews, see Clifford & Rhodes, 2005; Rhodes & Leopold, 2011; Webster & MacLeod, 2011). By representing every face relative to a stored average or prototype that functions as a norm, such “norm-based coding” may allow the visual system to see past the shared, highly redundant, structure of faces and to focus on what is distinctive about each individual. 

Support for a norm-based representational framework for faces comes from different lines of research. First, it seems that people spontaneously abstract averages or prototypes from sets of seen faces (Bruce, Doyle, Dench, & Burton, 1991; Inn, Walden, & Solso, 1993; Walton & Bower, 1993), i.e., the face corresponding to the central tendency of a series of faces seems familiar, even when it has not been seen. Furthermore, distinctive faces that lie further from the average in a face-space framework are recognized better than typical ones (Valentine, 1991, 1992, 2001), and caricaturing a face, i.e., exaggerating how it differs from the average, can facilitate its recognition (Benson & Perrett, 1994; Byatt & Rhodes, 1998; Calder, Young, Benson, & Perret, 1996; Lee, Byatt, & Rhodes, 2000; Rhodes, 1996; Rhodes, Brennan, & Carey, 1987). Recent neuroimaging and neurophysiological findings are also consistent with norm-based coding of faces: Faces that lie further from the average elicit stronger fMRI activation in the human fusiform face area (Loffler, Yourganov, Wilkinson, & Wilson, 2005), and increased firing rates of face-selective neurons in monkey anterior inferotemporal cortex (Leopold, Bondar, & Giese, 2006), compared with more average faces. 

Recently, studies of face identity aftereffects have provided more direct evidence that identity is coded relative to a norm. Adapting to a face biases perception toward the “opposite” identity in face space relative to the average face (Leopold, O'Toole, Vetter, & Blanz, 2001; Leopold, Rhodes, Müller, & Jeffery, 2005; Rhodes & Jeffery, 2006). The fact that perception is biased in a selective way toward the opposite identity relative to the average, not just non-selectively away from the adapting face, strongly supports the notion that faces are coded as deviation vectors from a norm (Rhodes & Jeffery, 2006; Tsao & Freiwald, 2006). Norm-based coding can be implemented by a two-pool opponent coding mechanism at the neural level, where all possible values along a single dimension in face space are coded by the relative output of only two oppositely tuned pools of neurons (for reviews, see Rhodes & Leopold, 2011; Webster & MacLeod, 2011). Several findings support such a model. First, adaptation to the average face itself does not shift the perception of non-average faces (Leopold et al., 2001; Webster & MacLin, 1999), in accordance with an opponent coding model where the average is perceived when both pools of neurons produce equal output strength. In addition, aftereffects become larger for adaptors with more extreme distortion levels relative to the norm (Robbins, McKone, & Edwards, 2007), as predicted by opponent coding: Here, because adaptation reduces responses in proportion to the initial unadapted firing rate, adaptors furthest from the norm produce the maximum change in the response ratio of the two pools (e.g., Maddess, McCourt, Blakeslee, & Cunningham, 1988; Movshon & Lennie, 1979). Moreover, test faces that lie further from the average than the adaptors are perceived as less extreme than before (Robbins et al., 2007). 

The concept of a norm-based face space raises the question of whether there is one central norm representing the mean values on all dimensions on which faces can differ or whether there are multiple norms corresponding to a variety of visually distinct face categories. This question has been addressed by asking whether opposite aftereffects can be generated simultaneously for faces from different categories. Such “category-contingent aftereffects” should not be possible if there is only one generic face norm, because that norm would be pulled in opposite directions by opposite adapting distortions, resulting in no net aftereffect. “Category-contingent” figural (distortion) aftereffects have been reported for faces from several different categories. For example, adaptation to “contracted” male and “expanded” female faces produces opposite aftereffects, with undistorted male faces looking slightly expanded and undistorted female faces looking slightly contracted (Bestelmeyer, Jones, & DeBruine, 2008; Jaquet & Rhodes, 2008; Little, DeBruine, & Jones, 2005). Category-contingent aftereffects have also been reported for faces of different races (Bestelmayer, Jones, DeBruine, Little, & Welling, 2010; Jaquet, Rhodes, & Hayward, 2007, 2008; Little, DeBruine, & Jones, 2008). These findings point to the existence of dissociable norms for faces of different races and sexes. Notwithstanding these category-contingent effects, there can also be some transfer of aftereffects between categories (Jaquet & Rhodes, 2008; Jaquet et al., 2008). Transfer could result from adaptation of general shape dimensions (Susilo, McKone, & Edwards, 2010) and/or face-selective dimensions on which faces from the two categories have similar distributions. 

Category-contingent figural (distortion) aftereffects indicate that there are dissociable norms for visually distinct face categories. However, figural aftereffects do not directly tap identity coding and these norms might be used to categorize faces (on race, sex, etc.) or to assess facial characteristics, like attractiveness, that vary with distance from the average (Rhodes, 2006; Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003), while the identity of each face could be coded against a common general norm (Rhodes et al., 2011). Distortion aftereffects that, for example, bias the perception of highly familiar faces toward previously seen figurally altered versions of the same identity (as in Carbon & Leder, 2006; Carbon et al., 2007) are generally taken as evidence of the flexibility and adaptability of the visual system in a changing environment. They do not demonstrate that identity-specific norms are used to encode faces in face space. The goal of the present study was thus to directly test whether dissociable norms are used to code the identity of faces in different categories. Specifically, we asked whether individual faces from different races, which form socially important and visually distinct categories (Meissner & Brigham, 2001; Michel, Caldara, & Rossion, 2006; Michel, Corneille, & Rossion, 2007), are coded relative to a norm of their own race rather than relative to a general norm. 

To do so, we compared the size of face identity aftereffects (e.g., Leopold et al., 2001; Rhodes & Jeffery, 2006) for adapt–test pairs from trajectories that span (pass through) a race-specific average (Asian or Caucasian) and from trajectories that span a generic (mixed race) average. The relative size of the aftereffects produced on different adapt–test trajectories can be used to infer whether the trajectory passes through the true norm in face space. Adapt–test pairs that lie opposite each other in face space, relative to the norm, generate larger identity aftereffects than non-opposite pairs (Rhodes & Jeffery, 2006). Therefore, if faces are coded using race-specific norms, then we should see larger aftereffects for pairs that span a race-specific average than a generic average. This approach has been used recently to demonstrate that sex-specific norms are used to code face identity (Rhodes et al., 2011). 

On each trial, participants see an adapting face, followed by a test face (one of four learned target identities, shown at various identity strengths), which they have to identify. We used different reference faces to create three kinds of adapting antifaces and their corresponding adapt–test trajectories (Figure 1). The reference faces were: (1) a race-specific, “same-race” average, created by morphing together either Asian or Caucasian faces, as appropriate; (2) a “generic” or “mixed-race” average, created by morphing both Asian and Caucasian faces; or (3) a “non-central” reference face (another identity of the same race as the target face). This non-central reference face was used because adapt and test faces from same-race-average trajectories appear to be the same race, whereas adapt and test faces on the generic-average trajectories do not (Figure 2). We, therefore, included the non-central trajectories, on which adapt–test pairs also appear to be the same race, to rule out the possibility that reduced aftereffects on the generic-average trajectories could somehow result from this mismatch in perceived race of the adapt and test faces. The non-central face identities were chosen to be as similar to the target faces (i.e., had the same perceptual distance to the target faces) as the same-race average (of the same race) was to the target faces (see Supplementary materials for details), thus controlling for differences in test trajectory length (see also below). The reference faces were used to create trajectories as shown in Figure 1B. We compared the size of identity aftereffects for pairs of adapt and test faces that were taken from the different types of trajectories. 

Figure 1

 

(A) Two identity trajectories through the same-race (here, Caucasian) average face, in a simplified two-dimensional face space. Antifaces were made by morphing the target identity toward and beyond the average face. On match trials, participants adapted to matching antifaces (e.g., adapt Anti-Dave, test Dave). On mismatch trials, they adapted to non-matching antifaces from another identity (e.g., adapt Anti-Dave, test Seth). Numbers indicate percent identity strength (morph level). (B) The three average faces and the two “non-central” face identities used to create morph trajectories. The “non-central” and same-race averages for each race were matched on perceived similarity to the target faces.

Figure 2

 

(A) Trial structure and conditions. The targets and antifaces shown come from a trajectory through the mixed-race average. (B) Example of matching and mismatching antifaces for each of the three trajectories for target face “Jack.”

Aftereffects were measured, on each trajectory, as the difference between “match trials,” where an antiface of the same identity as the test face was shown, and “mismatch trials,” where an antiface of another identity was shown (cf., Pellicano, Jeffery, Burr, & Rhodes, 2007; Rhodes et al., 2011). Larger aftereffects for adapt–test pairs made using the race-specific average than for those made using the generic average would indicate that Asian and Caucasian faces are coded using race-specific norms. In this case, aftereffects should also be larger on race-specific than non-central trajectories. If, instead, a single generic race norm is used to code face identity, then larger aftereffects should be found for the mixed-race than the race-specific and the non-central adapt–test pairs. 

All aftereffects were measured as proportional shifts of identification thresholds along trajectories with an arbitrary “length” of one. However, Asian and Caucasian target faces are likely more similar to their respective race-specific Asian and Caucasian averages than to a generic (mixed-race) average (whose race clearly differs from that of the target faces), so that the true perceptual length of the trajectories in face space would likely be shorter for the same-race than the generic trajectories. In this case, larger aftereffects on same-race trajectories might not necessarily represent larger shifts in absolute perceptual distance. If, for example, same-race trajectories were only half the perceptual length of mixed trajectories, then an aftereffect of 0.4 on a same-race trajectory would represent the same perceptual shift in face space as an aftereffect of 0.2 on a mixed-race trajectory. Therefore, following Rhodes et al. (2011), we obtained ratings of perceived similarity of the target faces to all three reference faces (same-race, mixed, non-central), in order to assess, and potentially correct for, any differences in perceptual trajectory length. In addition, the non-central reference face for each race was selected to ensure that the perceptual length of the non-central and the corresponding same-race trajectories were similar. We also sought to rule out an alternative account based on differences in perceptual contrast between adapt faces and test faces on the three different trajectories. Lower levels of perceptual contrast can produce smaller aftereffects (Clifford, 2002; Robbins et al., 2007). Following Rhodes et al. (2011) we obtained ratings of perceived similarity of each adapting antiface to all three reference faces (same-race, mixed, non-central) in order to estimate, and potentially correct for, possible differences in perceptual adapt–test contrast (see Methods section below and Supplementary materials for more details). 

Another important goal of the study was to test whether discrimination is enhanced around the norm. To do so, we measured identification thresholds in the absence of adaptation for target faces from the three different trajectories. If the trajectory that yields the largest aftereffects also yields the best identification performance, then we would conclude that discrimination is enhanced around the “true” norm. Many have proposed that face discrimination should be enhanced around the norm face, be it the “natural” one or the shifted norm after adaptation (e.g., Rhodes, Watson, Jeffery, & Clifford, 2010; Wilson, Loffler, & Wilkinson, 2002), as sometimes found in lower level vision (Kohn, 2007). However, the evidence is equivocal so far, with several studies finding no evidence that adaptation to either the average face (i.e., the “natural” norm) or another face identity (to shift the natural norm) improves performance around the old or new norm (Jaquet, Rhodes, & Clifford, 2005; Ng, Boynton, & Fine, 2008). In contrast, others have found enhanced discrimination around an adapted state. For example, discrimination of synthetic radial frequency faces was better around the average than a non-central location of their face space (Wilson et al., 2002). Adaptation to the average of a natural-looking face population enhances identification of faces from the adapted, relative to an unadapted population (in this case Asian and Caucasian faces; Rhodes et al., 2010). Adaptation to a male or female face selectively enhances gender discrimination for faces from the respective category (at least for faces from the same identity continuum, see Yang, Shen, Chen, & Fang, 2011). Adaptation can enhance discrimination of face views around an adapted viewpoint (Chen, Yang, Wang, & Fang, 2010). Finally, adapting to an individual face can enhance discrimination around that face (Oruç & Barton, 2011). In the current study, we asked whether participants are better at discriminating between realistic face identities that vary around the true psychological norm rather than a non-central location in face space. Better discrimination around the norm of a face category would be direct evidence of a functional benefit of norm-based coding in face perception. 

Methods

Participants

Twenty-nine Caucasian adults (eighteen females, eleven males) between 18 and 35 years of age were recruited from the University of Western Australia and received either credit points or cash reimbursement for their participation. 

Stimuli and apparatus

All faces were derived from scanned 3D heads from the Max Planck face database in Tübingen (http://faces.kyb.tuebingen.mpg.de/index.php). The “Morphable Model” algorithm developed by Blanz and Vetter (Blanz, 2000; Blanz & Vetter, 1999) was used to generate stimuli. Four East Asian and four Caucasian male faces were chosen as target faces, based on the ease with which they could be discriminated, as determined by the experimenters. 

Asian and Caucasian averages were created by morphing together 20 Asian and 20 Caucasian male faces (including the four target faces), respectively. A generic average was created from all 40 faces of both races. The non-central reference face was another face identity of the same race as the target faces (i.e., Asian for the Asian target faces and Caucasian for the Caucasian targets). The two non-central faces were chosen on the basis of similarity ratings, to match the perceptual distance between each non-central face and the target faces of the corresponding race to the perceptual distance between the same-race averages and the targets (see Supplementary materials for details). 

We created three different antifaces and thus three different trajectories for each target face by morphing their texture and shape toward and beyond each of the three reference faces (see Figure 1): (1) a same-race trajectory, where the target faces were morphed through their same-race average (i.e., Asian for Asian target faces and Caucasian for Caucasian targets); (2) a mixed-race trajectory, where the average made from both Asian and Caucasian faces was used; and (3) a non-central trajectory, using a non-central same-race face instead of an average face. Morphing was done from each target face (100% identity strength) through one of the reference faces (0%) to a −50% antiface (no further, since stronger antifaces showed distortions). Each test trajectory consisted of six versions of the target face, varying in identity strength: −20%, 0%, 20%, 40%, 60%, and 80% (negative values represent antifaces). 

All face stimuli were devoid of secondary cues such as hair and makeup. They were presented in color, surrounded by a black oval mask that hid the inner hair line and part of the ears (see Figure 2) but not the face outline, on a black background. A gray oval of the same size and overall luminance as the antifaces was used on no-adaptation trials. The adapt faces measured approximately 125% of the size (8.6° × 7.0°) of the test faces (7.2° × 5.2°), at a viewing distance of about 57 cm. 

Experiments were run on a 20-inch LCD screen (1680 by 1050 pixel resolution) iMac OS X, version 10.5.6, using SuperLab 4.0.6 software. Participants always responded using a standard computer keyboard. 

Similarity ratings

Similarity ratings were obtained from additional participants to assess (and adjust aftereffects for) any differences in (a) perceptual length and (b) perceived contrast between adapt and test faces, for the three kinds of trajectories (cf. Rhodes et al., 2011). Participants rated the similarity of target–reference pairs (80% vs. 0%) to assess perceptual length. They rated the similarity of antiface–reference pairs (−50% vs. 0%) to assess adapt–test contrast. Although test trajectories go from −20% to 80% (in 20% steps) identity strengths, we used only the 0% test level to keep the task brief. However, it seems reasonable to assume that any trajectory differences on this measure would generalize to the full trajectories. Face pairs were presented sequentially on the computer screen, and participants answered using a 7-point scale with 1 labeled “not at all similar” and 7 labeled “very similar.” See Supplementary materials for details of these rating experiments. 

Procedure

Each session began with a training phase, in which participants learned four target face identities, either four Asian male faces or four Caucasian male faces. In the subsequent identification phase, participants had to identify the same four target faces at various identity strengths, after adapting to either a matching or mismatching antiface. Trials for the three trajectories were intermixed. Identification was also tested in the absence of adaptation. Adaptation and no-adaptation trials were randomly intermixed and the whole identification phase was repeated three times, in separate sessions, each lasting 45–60 min. Target race was the only between-participants factor; all other factors were varied within participants. 

Training. At the beginning of each session, participants were given a printout of the four target faces (at 100% identity strength) and their corresponding names and were asked to assign characteristic adjectives (e.g., likable, trustworthy, arrogant) from a list to each face identity. They were told to take as much time as necessary and when they felt confident they could tell the faces apart and identify each by name, they moved on to practice identifying the faces on the computer. This practice consisted of three blocks of trials in which the same four target faces (also at 100% identity strength) were presented for an unlimited amount of time (in the first block), 500 ms (second block), and 200 ms (third block). Participants answered using labeled keyboard keys and received visual accuracy feedback at the end of each trial. Each face was shown four times in random order in each practice block (i.e., 16 trials per block), and blocks were repeated if necessary until participants could correctly identify all faces with 100% accuracy. The printout of the faces was kept in view during the first block, but participants were advised that it would be removed during the experiment so they should feel certain that they could identify the faces without it. Finally, in two additional blocks of trials, participants practiced identifying the four test faces at weaker identity strengths (80% and 60%), taken from all three trajectories (same-race, mixed-race, non-central). Exposure duration was unlimited in the first block and 200 ms in the second block. Each face was shown once at each identity strength for each trajectory, i.e., 4 identities × 3 trajectories × 2 identity strength levels = 24 trials per block. 

Identification. For each trajectory (same-race, mixed-race, non-central), there were two adapting conditions (see Figure 2A): (a) match adapt, in which the identity of the antiface used as the adapting stimulus corresponded to the target face, and (b) mismatch adapt, in which a randomly assigned antiface of one of the other three target faces was used as the adapting stimulus. The non-matching antifaces were preassigned to each of the three identities across trajectories (i.e., anti-Dan was used as mismatching adaptor for Jack in one trajectory, for Seth in another, and for Troy in the third one) and held constant within each trajectory (see Figure 2B). On no-adaptation identification trials, a gray oval stimulus replaced the adapting antifaces to maintain the trial sequence. It was the same average size and average luminance as the antifaces (see Figure 2A for trials of all conditions). For each testing session, there were 216 trials, consisting of 4 target identities × 3 testing conditions (match, mismatch, no-adapt) × 6 identity strengths (−20%, 0%, 20%, 40%, 60%, 80%) × 3 trajectories (same-race, mixed-race, non-central). Trials were presented in random order. Each trial consisted of 5000-ms exposure to an antiface adaptor or a gray oval, followed by a 150-ms ISI, followed by a test face for 200 ms, followed by a 150-ms blank ISI, followed by a response screen asking the participant to indicate the identity of the target. Participants initiated each trial by pressing the space bar when they were ready and used the same labeled keyboard keys to answer as during training. Participants were told to be as accurate as possible even though some of the faces would be difficult to identify and that they should make their best guess when uncertain. There were self-timed rest breaks after every 36 trials. 

Results

Participants' identification responses were scored correct if they corresponded to the identity from which the test face was made. Following Leopold et al. (2001), a correct response was arbitrarily assigned to each presentation of a 0% identity strength face, with each of the four target identities assigned equally often, to measure “performance” on these trials. The data of two (female) participants were removed from the analysis, one because her recognition performance was below the group mean by more than 3 standard deviations in six out of nine conditions (no other participant was that far from the group mean in any condition) and the other one because she did not complete all three experimental sessions. 

Aftereffects

For each participant, the mean proportion correct was calculated and plotted as a function of identity strength for each trajectory and adaptation condition (match, mismatch). Cumulative Gaussians were fitted to each identification curve using GraphPad Prism version 5.00 for Windows (GraphPad Software, San Diego California USA, www.graphpad.com). The fits were good (mean R ² = 0.931, SD = 0.062, range = 0.668 to 1.000, N = 27 participants × 3 trajectories × 3 adapt conditions). Mean identification curves for match and mismatch trials (averaged across participants) for all 3 trajectories (same, mixed, non-central) are shown in Figure 3. Since race of target face had no impact on aftereffects (see below), the results are collapsed across Caucasian and Asian faces. On all trajectories, performance for match trials was better than for mismatch trials. 

Figure 3

 

Mean identification performance as a function of identity strength after adapting to matching and mismatching antifaces for each trajectory (same-race, mixed, non-central). Fitted cumulative Gaussians are shown.

For each participant, we calculated an adaptation aftereffect for each trajectory by subtracting identification thresholds on match trials from identification thresholds on mismatch trials. We used the means of the fitted Gaussians as the identification thresholds (see Figure 4A and Table 1). The aftereffect (Figure 4B) was bigger for the same trajectory than for the other two trajectories, while the aftereffects for the mixed and non-central trajectories did not differ. This pattern was confirmed by a repeated measures Analysis of Variance (ANOVA), with target race (Asian, Caucasian) as a between-subject factor and trajectory (same-race, mixed, non-central) as a within-subject factor. A main effect of trajectory, F(2,50) = 7.480, p = 0.001, partial η ² = 0.230, indicated that the size of the aftereffect differed between the three conditions. There was no significant main effect of target race, F(1,25) = 1.633, p = 0.213, partial η ² = 0.061, and no interaction, F(2,50) = 0.695, p = 0.504, partial η ² = 0.027. Planned pairwise t-tests confirmed that the aftereffect was significantly larger on the same-race trajectory than on each of the other trajectories [same vs. mixed: t(26) = 3.307, p = 0.003; same vs. non-central: t(26) = 3.592, p = 0.001], suggesting that the antifaces on the same-race trajectories are the ones that lie truly opposite to the targets in face space. The aftereffects for the mixed and non-central trajectories were not significantly different from each other [t(26) = 0.268, p = 0.791]. 

Figure 4

 

(A) Mean identification thresholds (means of fitted Gaussians) after adapting to matching and mismatching antifaces for each trajectory. (B) Size of after effects for each trajectory.

Table 1

 

Aftereffects as a function of face race and trajectory: original and adjusted for differences in trajectory length and adapt–test contrast. See Supplementary materials for details.

Adjustment		Size of aftereffect
		Original		Trajectory length		Adapt–test contrast
Face race	Trajectory	Mean	SE	Mean	SE	Mean	SE
Caucasian	Same	0.082	0.016	0.082	0.016	0.085	0.016
	Mixed	0.028	0.018	0.036	0.024	0.036	0.024
	Non-central	0.044	0.024	0.046	0.025	0.053	0.029
Asian	Same	0.121	0.018	0.121	0.018	0.126	0.019
	Mixed	0.045	0.016	0.059	0.021	0.059	0.021
	Non-central	0.039	0.016	0.041	0.016	0.048	0.019

Correction for differences in trajectory length. Mean similarity ratings for target–reference pairs (80% vs. 0%; see Supplementary materials) were 30% lower for mixed (M = 2.52, SE = 0.24) and 3% lower for non-central (M = 3.52, SE = 0.22) trajectories compared to same-race trajectories (M = 3.61, SE = 0.21), indicating that the mixed and non-central test trajectories were longer than the race-specific ones. We therefore scaled the mixed (multiplying by 1.30) and non-central (multiplying by 1.03) trajectory aftereffects and repeated the two-way ANOVA. Aftereffects remained larger for same-race (M = 0.102, SE = 0.012) than for mixed (M = 0.048, SE = 0.015) and non-central (M = 0.043, SE = 0.014) trajectories (Table 1). There was a significant effect of trajectory, F(2,50) = 4.844, p = 0.012, partial η ² = 0.162, but no main effect of face race, F(1,25) = 1.678, p = 0.207, partial η ² = 0.063, and no interaction, F(2,50) = 0.570, p = 0.569, partial η ² = 0.022. Pairwise t-tests showed that aftereffects remained significantly larger for same-race than for mixed and non-central trajectories [same vs. mixed: t(26) = 2.385, p = 0.003; same vs. non-central: t(26) = 3.457, p = 0.002], with no significant difference between mixed and non-central trajectories [t(26) = 0.219, p = 0.829]. 

Correction for differences in adaptor–test contrast. Mean similarity ratings for adaptor–reference pairs were comparable for same-race (M = 5.33, SE = 0.20) and mixed trajectories (M = 5.13, SE = 0.17) but significantly higher for non-central trajectories (M = 6.21, SE = 0.17; see Supplementary materials). Smaller aftereffects on the non-central trajectory could therefore potentially be attributed to reduced perceptual contrast between adapt and test pairs (or less extreme adaptors) on that trajectory. However, when we rescaled the previously scaled aftereffects (see above) again, according to these ratings, and repeated the ANOVA, the results remained exactly the same (see Supplementary materials for more details and Table 1 for means and SEs). There was a significant main effect of trajectory, F(2,50) = 4.323, p = 0.019, partial η ² = 0.147, but no main effect of face race, F(1,25) = 1.443, p = 0.241, partial η ² = 0.055, and no interaction, F(2,50) = 0.557, p = 0.577, partial η ² = 0.022. Pairwise t-tests showed that aftereffects remained larger for same-race than for mixed and non-central trajectories [same vs. mixed: t(26) = 2.523, p = 0.018; same vs. non-central: t(26) = 2.919, p = 0.007], with no significant difference between aftereffects for mixed and non-central trajectories [t(26) = 0.110, p = 0.913]. Note that rescaling of the aftereffects for differences in adaptor–test contrast only (i.e., without correcting for trajectory length at the same time) yields exactly the same results. 

Identification performance. To assess whether identification performance was best around the “true norm,” we plotted performance as a function of identity strength for each trajectory, for each participant. Mean identification curves for the three trajectories are plotted in Figure 5A. Overall identification accuracy was highest (i.e., identification thresholds were smallest) on the same-race trajectory, followed by the mixed trajectory, and poorest on the non-central trajectory (also see Table 2). The same two-way ANOVA as above showed a significant effect of trajectory, F(2,50) = 27.579, p < 0.001, partial η ² = 0.525, but no main effect of target race, F(1,25) = 2.361, p > 0.135, partial η ² = 0.086, and no interaction, F(2,50) = 3.633, p > 0.065, partial η ² = 0.127. Pairwise t-tests showed that all three pairwise differences were significant [same vs. mixed: t(26) = 2.79, p = 0.010; same vs. non-central: t(26) = 6.75, p < 0.001; mixed vs. non-central: t(26) = 4.08, p < 0.001]. 

Figure 5

 

(A) Mean identification performance as a function of identity strength for each trajectory (same-race, mixed, non-central). Fitted cumulative Gaussians are shown (B) Mean identification thresholds (means of fitted Gaussians) for each trajectory (same-race, mixed, non-central).

Table 2

 

No-adapt identification thresholds as a function of face race and trajectory: original and adjusted for differences in trajectory length. See Supplementary materials for details.

Adjustment		Identification threshold
		Original		Trajectory length
Face race	Trajectory	Mean	SE	Mean	SE
Caucasian	Same	0.132	0.016	0.132	0.016
	Mixed	0.164	0.011	0.214	0.014
	Non-central	0.263	0.021	0.271	0.216
Asian	Same	0.131	0.008	0.131	0.008
	Mixed	0.168	0.016	0.219	0.021
	Non-central	0.200	0.012	0.206	0.013

Correction for differences in trajectory length. When we adjusted identification thresholds in the same way as aftereffects, to correct for differences in length of the three trajectories (see above), the pattern remained the same: Thresholds were still smallest for the same-race trajectory, followed by the mixed trajectory and non-central trajectory (see Table 2). The ANOVA showed a significant effect of trajectory, F(2,50) = 27.414, p < 0.001, partial η ² = 0.523, but no main effect of target race, F(1,25) = 1.971, p > 0.170, partial η ² = 0.073, and no interaction, F(2,50) = 3.158, p > 0.085, partial η ² = 0.112. Pairwise t-tests showed that differences between the same-race and both other trajectories were significant [same vs. mixed: t(26) = −5.77, p < 0.001; same vs. non-central: t(26) = 7.07, p < 0.001] but not between the mixed and non-central trajectories [t(26) = 1.185, p = 0.247]. 

Discussion

Judging the race of an individual in everyday life is not merely about classifying faces into visually distinct categories based on physical facial differences. Faces of different races also represent meaningful biological and social categories, and belonging to one or the other might have important social implications. In this study, we therefore investigated how faces of different races are represented in the brain, that is, whether they are coded relative to race-specific norms or relative to a common, generic face norm. We found larger face identity aftereffects for adapt–test pairs that lie opposite a race-specific average than opposite either a generic mixed-race average or another identity from a non-central location in face space. These results strongly suggest that the race of a face plays a role for encoding its identity and that faces of different races are coded using race-specific norms because aftereffects are largest when adapt–test pairs lie opposite the true “norm” in face space (Leopold et al., 2001; Rhodes & Jeffery, 2006). 

Race-contingent figural face aftereffects, where adaptation to oppositely distorted Chinese and Caucasian faces induces opposite changes in the perceived normality of Chinese and Caucasian test faces suggest that different norms are maintained for faces of different races (e.g., Jaquet et al., 2007, 2008). The present results explicitly implicate race-specific norms in coding the identity of faces. A similar adaptation paradigm has also been used to show that identity is coded relative to norms selective for the sex of each face (Rhodes et al., 2011). Taken together, these findings suggest that face coding mechanisms are tuned to capture variation within visually distinct and socially important categories (see also Little et al., 2008). 

The question remains how category-specific norms are implemented in face space. Since faces of different races and sexes share many visual properties, a functional and neural architecture in which these categories are represented in distinct spaces and coded by completely distinct neural populations is implausible. Instead, for male and female faces, a “dissociable coding model” has been proposed, in which faces are represented in a single face space containing common dimensions that are represented in all faces, as well as category-selective dimensions that are most useful for differentiating faces of one particular category (see Rhodes & Jaquet, 2010; Rhodes et al., 2010). We propose a similar system for faces of different races, with some common dimensions that are shared by faces of all races, and some race-selective dimensions. Race-specific norms within such a common space would then represent “subcenters,” in the sense that they, instead of the central average face, are taken as a reference for encoding each face's identity, using the most useful dimensions of the respective race category. 

It is possible that our mixed-race average face might not represent the true generic norm for face race in general. This average was created by morphing together only Asian and Caucasian faces that may not represent a realistic generic race norm, since there are more than two race categories. It is also possible that even our Caucasian/Asian average lies somewhere between Asian and Caucasian faces but not exactly in the middle as we assumed by morphing an equal number of faces of each race. Nevertheless, an average face made from two races morphed together should still be closer to the central tendency of all race categories (in the absence of any subcategorization) than is a Caucasian or an Asian average alone. Therefore, if a generic norm is used as a perceptual reference, we should still have found a bigger aftereffect for the mixed-race trajectory than for race-specific ones. However, instead we found the reverse effect, providing evidence that faces of different races form distinct (although overlapping) perceptual categories and that individual identity is coded using different norms for different race categories. 

Could it be problematic that our race-specific averages consisted of only twenty faces, while the generic average face was created by morphing together all forty faces? We suggest not, because increasing the number of faces included in the generic average should produce a face that is even closer to the central tendency of all faces, making the mixed-race average even “more average.” Moreover, given that a generic norm that is built up over experience will naturally be based on more faces than category-specific norms of any kind, the difference in the number of faces used to create our averages in fact reflects the reality of such norm faces. Thus, again, if faces were not categorized into Asian and Caucasian faces, the mixed average, as it is closer to the center of the face space, should represent the norm that is used to code individual identities, and we should have found larger aftereffects for mixed-race trajectories. However, we found larger aftereffects for same-race trajectories, suggesting that the mixed-race average is not the true psychological norm. 

To more precisely control for differences in perceptual distances on the three trajectories, we obtained similarity ratings between reference and target faces on all trajectories and adjusted the aftereffects accordingly. To control for differences in adaptor–test similarity and thereby for perceptual contrast between the two, which could lead to differences in the size of the aftereffect (Clifford, 2002; Robbins et al., 2007), we also adjusted the aftereffects based on similarity ratings of the perceptual distance between reference faces and antifaces on each trajectory. Aftereffects remained larger on race-specific trajectories after correction for differences in trajectory length and differences in perceptual contrast between adapt and test faces. 

Our results also provide direct evidence for a functional role of norms in face perception. Identification was better around the true perceptual (i.e., race-specific) norm than around a mixed-race average or another non-central face identity (see Figure 5), and the advantage remained after correcting for differences in trajectory length. This result extends Wilson et al.'s (2002) earlier finding of better discrimination around the center of face space, using synthetic radial frequency faces. It suggests that norm-based coding of face identity may facilitate successful recognition, by highlighting what is distinctive about each individual face and minimizing the processing of redundant facial information. The use of distinct (although overlapping) norms for visually distinct face categories, as indicated by our results, might be even more economical, since neural responses increase with distance from the norm, and faces will generally lie further from a generic than a category-specific average. 

Another advantage of category-specific norms might be that they can be selectively updated so that changes in visual experience with one face population can selectively retune coding of faces within that population. Experience can also shift category boundaries closer to the more familiar category (Webster, Kaping, Mizokami, & Duhamel, 2004), which might serve to increase discriminability within that category. One might ask then why, since all our participants were Caucasian, we did not find better performance for Caucasian than Asian face identities. We suggest that it is because our participants extensively learned and were tested on only four individual faces of each race, so that they were highly familiar with the targets. When only a few highly familiar faces must be discriminated, other-race effects are not generally observed (Jaquet et al., 2008; Rhodes, Watson, Jaquet, Winkler, & Clifford, 2004). 

It remains to be determined just how norms for faces of distinct race categories differ between races and between individuals. Depending on the environment we grow up in, we all have very different experience with only one or sometimes several races so our long-term race norms might also develop very specifically. Correlating personal experience with different races and performance in high-level face adaptation experiments might shed some light on the question of how norms are derived over time and how this process influences perception of and identification performance for these visually and socially different groups of faces. 

In summary, our data add to a growing literature showing that exposure to faces biases subsequent perception of novel faces (e.g., Leopold et al., 2001; Rhodes et al., 2003, 2004; Webster et al., 2004), presumably by shifting a norm that is used to encode faces in the brain. Faces as a category can be divided into subcategories based on appearance, and here, we show that faces of different races are coded against their respective category-specific norm face rather than a generic single norm. The face space model may, thus, be best conceptualized as a space containing category-selective, as well as generic, face dimensions. In addition, our findings suggest a functional role for long-term face norms abstracted from adaptation to populations of faces over long time periods. 

Supplementary Materials

Supplementary PDF - Supplementary PDF 

Acknowledgments

This research was supported by a project grant from the German Academic Exchange Service awarded to Regine Armann, by the Max Planck Society, by ARC fellowships to Gillian Rhodes and Linda Jeffery, and by the Australian Research Council Centre of Excellence in Cognition and its Disorders (Project Number CE110001021). We wish to thank Isabelle Bülthoff, Christian Wallraven, and Johannes Schultz for help with stimulus collection and the face database and Libby Taylor for assistance with testing participants and data processing. 

Commercial relationships: none. 

Corresponding author: Regine Armann. 

Email: [e-mail]. 

Address: Max Planck Institute for Biological Cybernetics, Spemannstr. 38, 72076 Tübingen, Germany.