Face format at encoding affects the other-race effect in face memory

Abstract

Abstract

Abstract:

Abstract  Memory of own-race faces is generally better than memory of other-races faces. This other-race effect (ORE) in face memory has been attributed to differences in contact, holistic processing, and motivation to individuate faces. Since most studies demonstrate the ORE with participants learning and recognizing static, single-view faces, it remains unclear whether the ORE can be generalized to different face learning conditions. Using an old/new recognition task, we tested whether face format at encoding modulates the ORE. The results showed a significant ORE when participants learned static, single-view faces (Experiment 1). In contrast, the ORE disappeared when participants learned rigidly moving faces (Experiment 2). Moreover, learning faces displayed from four discrete views produced the same results as learning rigidly moving faces (Experiment 3). Contact with other-race faces was correlated with the magnitude of the ORE. Nonetheless, the absence of the ORE in Experiments 2 and 3 cannot be readily explained by either more frequent contact with other-race faces or stronger motivation to individuate them. These results demonstrate that the ORE is sensitive to face format at encoding, supporting the hypothesis that relative involvement of holistic and featural processing at encoding mediates the ORE observed in face memory.

Introduction

People are experts in recognizing familiar faces but often find it difficult to discriminate between unfamiliar faces, especially those from a different race. Other-race faces seem to be more alike than own-race faces and are remembered less well than own-race faces (Hayward, Rhodes, & Schwaninger, 2008; Rhodes, Hayward, & Winkler, 2006; Rhodes, Tan, Brake, & Taylor, 1989). This other-race effect (ORE) in face memory has been well established with a variety of face memory tasks and with observers from different races (Meissner & Brigham, 2001). Nonetheless, most ORE findings are based on empirical studies using static, frontal-view faces, leaving it unclear whether the ORE persists when faces are learned more naturally, such as when movement occurs or when they are viewed from different perspectives. In the present study, we investigated whether face format at encoding modulates the ORE in face memory, and in particular whether encoding faces presented in rigid motion or from multiple viewpoints reduces the ORE. 

Theories of the ORE in face memory

Before discussing how face format may affect the ORE, we briefly review theories about what causes the ORE. One hypothesis proposes that the ORE is caused by different levels of holistic processing for own- and other-race faces. Holistic processing—the tendency to integrate facial information as a unified whole rather than a collection of independent face parts (Maurer, Le Grand, & Mondloch, 2002)— has been shown to be stronger for own- than for other-race faces (Michel, Caldara, & Rossion, 2006; Michel, Rossion, Han, Chung, & Caldara, 2006; Tanaka, Kiefer, & Bukach, 2004; but see Hayward, Crookes, & Rhodes, 2013). Given that face memory ability is closely linked to holistic processing (DeGutis, Wilmer, Mercado, & Cohan, 2013; Richler, Cheung, & Gauthier, 2011), stronger holistic processing for own- than other-race faces would predict an ORE in face memory (DeGutis, Mercado, Wilmer, & Rosenblatt, 2013). Nonetheless, few studies have investigated this holistic processing hypothesis by directly manipulating the engagement of holistic processing at encoding. A reduction of the ORE has been observed in previous face inversion studies (e.g., Rhodes et al., 1989), but there is an ongoing debate as to whether face inversion disrupts holistic processing or simply leads to an overall inefficiency in face processing (McKone & Yovel, 2009; Rossion, 2008; Sekuler, Gaspar, Gold, & Bennett, 2004). Therefore, direct evidence for a causal relation between holistic processing and ORE remains weak (Hayward et al., 2013). 

Other researchers have attributed the ORE to different ways of face processing for own-race and other-race faces. As elaborated by Hugenberg, Young, Bernstein, and Sacco (2010), people spontaneously attend to identity-diagnostic information for own-race faces (e.g., shape of face features and spatial relations). In contrast, for other-race faces, people automatically attend to the features (e.g., skin color) that signify the racial category of faces (i.e., other-race faces; Levin, 2000), but are less likely to encode individuating information. The categorization of other-race faces without individuating face identities may impair their later recognition, resulting in an ORE. This categorization-individuation hypothesis echoes Valentine's (1991) proposal that other-race faces are represented more densely in “face space” than own-race faces, making them perceptually homogeneous. Consistent with this hypothesis, the ORE can be reduced or eliminated by directly asking participants to individuate other-race faces (Hugenberg, Miller, & Claypool, 2007; Tanaka & Pierce, 2009). However, the hypothesis cannot readily account for findings of other studies (e.g., Rhodes, Lie, Ewing, Evangelista, & Tanaka, 2010; see also Rhodes, Locke, Ewing, & Evangelista, 2009). 

Different levels of contact with own- and other-race faces also contribute to the ORE. Inferior memory of other-race faces may be due to the lack of expertise in differentiating those faces (Brigham & Malpass, 1985; Hancock & Rhodes, 2008; Rhodes, Ewing, et al., 2009). Consistent with this contact hypothesis, enhanced contact or training with other-race faces can reduce or even reverse the ORE (McKone, Brewer, MacPherson, Rhodes, & Hayward, 2007; Sangrigoli, Pallier, Argenti, Ventureyra, & de Schonen, 2005). Rhodes and colleagues (Hancock & Rhodes, 2008; Rhodes, Ewing, et al., 2009) also demonstrate that people with more frequent other-race contact tend to exhibit a smaller ORE. Note that mere passive contact with other-race faces without actively discriminating them may not enhance the ability to recognize those faces (see also Yovel et al., 2012). That is, the quality, not the quantity, of contact with other-race faces modulates the ORE (Bukach, Cottle, Ubiwa, & Miller, 2012). 

These theoretical accounts of ORE are not mutually exclusive. Frequent other-race contact may modulate the motivation to individuate other-race faces. Higher motivation for face individuation may also lead to enhanced holistic processing. None of these theories claim that the ORE is due solely to a single process. Each hypothesis has difficulty in readily accounting for all existing empirical findings (Hayward et al., 2013; Meissner & Brigham, 2001; Rhodes et al., 2010), suggesting that the ORE might be mediated by a diversity of underlying mechanisms. 

ORE and face format at encoding

As pointed out by Meissner and Brigham (2001, p. 5), most of the ORE studies reviewed in the foregoing used static, single-view photographs. Therefore, little is known about whether learning faces shown in motion or from different viewpoints might change the ORE. Using staged person interactions in a real-life scenario (i.e., buying cigarettes from a clerk in a grocery store), earlier field studies observed an ORE in face identification (Platz & Hosch, 1988; Wright, Boyd, & Tredoux, 2001; but see Brigham, Maass, Snyder, & Spaulding, 1982). These results suggest that the ORE might persist for real-life faces. However, these studies cannot tell whether faces that are learned in real-life situations are different from those learned as static face images, or how this change of learning may modify the ORE. 

Butcher, Lander, Fang, and Costen (2011) recently examined the influence of elastic motion (i.e., speaking) on memory of both own-race and other-race faces. They showed participants either video sequences or single static face images during learning and then asked them to discriminate these faces from novel, unseen faces at test. Participants showed better memory when they learned moving faces than when they learned static faces. Moreover, there was an overall ORE across learning formats, suggesting that elastic facial motion does not eliminate the ORE. However, caution is needed in interpreting these data. The study only tested Caucasian participants and used face race as a between-participant factor, so the main effect of face race might be due to differences between participants or between face stimuli used in own-race and other-race conditions. 

Whether learning rigidly moving faces reduces or enlarges the ORE has not yet been directly investigated. Unlike elastic motion that provides mainly additional idiosyncratic movement information about face identity, rigid motion (e.g., rotation) conveys 3-D face shape information from motion. Previous studies have shown that elastic and rigid motion have different influences on face processing (de la Rosa, Giese, Bülthoff, & Curio, 2013; Lander & Bruce, 2003; O'Toole, Roark, & Abdi, 2002), so the influence of elastic and rigid motion on the ORE may differ too. 

How may learning rigidly moving faces modulate the ORE? Xiao, Quinn, Ge, and Lee (2012) have demonstrated that the presence of rigid facial motion enhances feature-based face processing. They found no composite face effect (Young, Hellawell, & Hay, 1987)—a hallmark effect of holistic processing—when faces were moving rigidly. This result suggests that rigidly moving faces are processed in a feature-based manner (i.e., less holistically). This feature-based face encoding, according to the holistic processing hypothesis, would impair memory of own-race faces more than other-race faces. First, own-race faces are processed more holistically than other-race faces, and this other-race difference in holistic processing is assumed to be the underlying mechanism of the ORE (DeGutis, Mercado, et al., 2013; Michel, Caldara, et al., 2006; Michel, Rossion, et al., 2006; Tanaka et al., 2004). Second, it has long been assumed that other-race faces tend to be processed in a more analytical way (Nakabayashi, Lloyd-Jones, Butcher, & Liu, 2012; Rhodes et al., 1989; Tanaka et al., 2004). Therefore, if rigid motion disrupts holistic processing or enhances featural processing, it should reduce the ORE by impairing own-race face recognition and/or improving other-race face recognition. 

Information available from faces that are presented from multiple views (hereafter labeled as multiview faces) is similar to that available from rigidly moving faces (Christie & Bruce, 1998; Lander & Bruce, 2003). Nevertheless, multiview faces contain no facial motion information, thereby providing a control condition to test whether rigid facial motion per se or multiple face views actually influence the ORE. If presenting faces from different viewpoints enhances feature processing and/or attenuates holistic processing, as has been demonstrated in rigidly moving faces, the ORE might be reduced in learning multiview faces. Xiao et al. (2012) suggest that feature-based face processing may be enhanced for multiview faces if multiple face views are presented with a short interstimulus interval (ISI; e.g., less than 300 ms) and in a coherent order (e.g., following natural frame sequences). Under these conditions, learning multiview faces may reduce the ORE in the same way as encoding rigidly moving faces. Therefore, according to the holistic processing hypothesis, the ORE following study of moving and multiview faces should differ from that observed with static, single-view faces. 

The present study

We investigated whether and, if so, how face format at encoding might modulate the ORE in face memory. Specifically, we asked whether learning faces shown in rigid motion or displayed from multiple viewpoints reduces the ORE. To lay a baseline, in Experiment 1 we tested whether learning single-view faces leads to an ORE. To encourage participants to use face identification and avoid pattern matching during the task, face views were changed between learning and test. We used a standard old/new recognition task, in which participants learned a number of faces and then discriminated the learned (“old”) faces from unlearned (“new”) faces. Participants performed the task for own-race faces and for other-race faces separately. Both male and female faces were included to ascertain the generality of the ORE across face gender, which was often overlooked in prior studies. 

Experiment 2 examined whether learning rigidly moving faces reduces the ORE. Participants encoded the same set of faces as that in Experiment 1, but the faces were shown in rigid motion (i.e., yaw rotation from left to right 45° view back and forth). Experiment 3 tested whether learning multiview faces has the same effect on ORE as learning rigidly moving faces. We showed four discrete face views for each face (i.e., left 45°, left 15°, right 15°, and right 45° views), with each view displayed for 2 s followed by a 250-ms blank screen. We used the natural order of these views (e.g., from left to right) rather than a randomized order. This manipulation was aimed to prevent participants from perceiving apparent motion, while feature-based face processing was still possible (Xiao et al., 2012). The total presentation time for each face was 8 s across encoding formats. Comparison of the magnitude of the ORE across different encoding formats should reveal whether and how they modulate the ORE. 

Establishing the effect of encoding format on ORE will give insights into the underlying mechanism of the ORE. As participants' experience with other-race persons was not manipulated, the contact hypothesis would predict the same ORE whether faces were presented from a single view, from multiple views, or in rigid motion. Alternatively, if the ORE is completely determined by social categorization or motivation to individuate faces, we would still expect the same pattern of ORE across experiments, because face formats at encoding should not alter these sociocognitive factors. The holistic processing hypothesis, however, predicts a reduction of the ORE when participants learn rigidly moving faces or multiple face views, since these learning conditions are assumed to enhance featural encoding and/or reduce holistic processing. 

General methods

Participants

A total of 144 participants took part in the three experiments. Each experiment had a different group of 24 Caucasian and 24 Asian participants. Caucasian participants (39 female and 33 male, mean age = 28, SD = 8) were tested at the Max Planck Institute for Biological Cybernetics, and Asian participants (36 each female and male, mean age = 23, SD = 5) were tested at the University of Hong Kong. Most of the Caucasian participants had lived most of their lives in Germany, and all Asian participants lived in Hong Kong or mainland China. All participants were naïve to the purpose of the study. The study was approved by the local institutional review board, and signed consent forms were obtained from each participant before the experiment. 

Stimuli

Face stimuli

Stimuli were created using the face database of the Max Planck Institute for Biological Cybernetics (http://faces.kyb.tuebingen.mpg.de; Blanz & Vetter, 1999). We used 24 faces in each of the following four categories: female Asian, male Asian, female Caucasian, and male Caucasian. The faces in the database were 3-D face laser scans of real people's faces. By changing the position of a virtual camera in the 3-D space accordingly, we could render these faces from different viewpoints. For each of the 96 faces, we created images of 19 face views, ranging from left 45° to right 45° view in steps of 5° (Figure 1). All faces images showed a neutral expression and showed the face and part of the neck below the chin. The face stimuli were rendered in gray scale and were presented on a neutral gray background (450 × 450 pixels). The average size of individual face images was 260 × 376 pixels and subtended 6.7° × 9.7° in visual angle. 

Figure 1

 

Examples of different views of one sample face (front view is not shown here).

Other-race contact questionnaire

The questionnaire was modified from the one used by Walker and Hewstone (2006). It consists of five statements that describe how often one meets and interacts with other-race people (“I often see East Asian/Caucasian people,” “I spend a lot of my free time doing things with East Asian/Caucasian people,” “I have many friends that come from East Asian/Western European countries,” “I often go round to the houses of East Asian/Caucasian people,” and “I often meet with East Asian/Caucasian people at my house”). Participants indicated their agreement with each statement on a 6-point scale ranging from 1 (very strongly disagree) to 6 (very strongly agree). 

Procedure

All stimuli were displayed on a 22-in. LCD screen and participants were seated about 60 cm away. Each participant performed four blocks of the old/new recognition task, one for each of the four face categories (e.g., male Asian faces). The order of these four blocks was counterbalanced across participants, with the constraint that faces of the same race were tested successively. In each block, participants first learned 12 faces and were instructed to memorize those faces for a later recognition test. For each participant, the 12 target faces were randomly selected from the 24 faces for that block (e.g., male Asian faces). When participants learned all 12 target faces, their memory for those faces was immediately tested with 24 faces (12 old and 12 new). At test, each face was displayed until a response was made. Participants were asked to press one button if they learned the face during the learning stage (i.e., old) and to press another button if the face was completely new. They were instructed to respond as accurately as possible without spending too much time on a single trial. 

This procedure was identical for all three experiments except that faces were shown in different formats during learning. In Experiment 1, participants learned static, single-view faces (either left 15° or right 15° view, see Figure 1). Each face was displayed for 8 s, with an intertrial interval of 1 s. In Experiment 2, faces were rotating back and forth from left to right 45° views for 8 s. The apparent rigid motion was induced by a rapid sequential presentation of 18 face views from left to right 45° view in steps of 5° (excluding the front, 0° view) and then in reverse order (i.e., rotating from right to left 45° view). These image sequences were presented twice. Each face view appeared for 112 ms before being replaced immediately by the next view, resulting in a total presentation time of about 8 s (112 ms × 18 views × 2 orders × 2 repetitions = 8.064 s). In Experiment 3, only four discrete viewpoints were presented during learning (in order: left 45°, left 15°, right 15°, and right 45°), each displayed for 2 s with an ISI of 250 ms of blank screen. The test faces were always shown in front view, which had not been shown during learning, so participants never saw identical face images between learning and test. At the end of the experiment, participants completed the other-race contact questionnaire. 

Results and discussion

Recognition performance was measured as d′, which was calculated using hit and false alarm rates in each condition (Snodgrass & Corwin, 1988). The d′ data were submitted to a 2 (observer: Asian vs. Caucasian) × 2 (face race: own race vs. other race) × 2 (face gender: male vs. female) mixed repeated-measures ANOVA, with face race and gender as within-participant factors and observer as a between-participant factor. We report all results that reached statistical significance (α = 0.05) and report mean data with standard errors (i.e., M ± SE). We report partial eta-square (η_p²) as an index of effect size, with values of 0.01, 0.06, and 0.14 representing small, medium, and large effect sizes, respectively (Cohen, 1988). 

Experiment 1: Encoding single-view faces elicited the ORE

Experiment 1 tested whether learning static, single-view faces elicits the ORE. Participants learned faces that depicted either a left or right 15° view and then made old/new judgments for these learned faces as well as novel, unlearned faces. All test faces were shown in front view (i.e., 0°). The viewpoint change between encoding and test faces was aimed at minimizing image matching strategies for both own- and other-race faces. 

Figure 2 shows mean d′ values as a function of face race, face gender, and observer. There was a significant main effect of face race, F(1, 46) = 15.17, MSE = 0.21, p < 0.001, η_p² = 0.25, indicating consistent better recognition performance for own-race faces than other-race faces (1.29 ± 0.08 vs. 1.03 ± 0.07). The interactions between face race and face gender, F(1, 46) = 0.19, MSE = 0.41, p = 0.67, η_p² < 0.01, and between face race and observer, F(1, 46) = 1.21, MSE = 0.21, p = 0.28, η_p² = 0.03, were not significant. Separate ANOVAs (face race × face gender) were conducted for data from Caucasian and Asian participants. We found a significant ORE for Caucasian participants (1.43 ± 0.11 vs. 1.10 ± 0.10), F(1, 23) = 13.17, MSE = 0.20, p = 0.001, η_p² = 0.36, and a marginally significant ORE for Asian participants (1.14 ± 0.11 vs. 0.96 ± 0.10), F(1, 23) = 3.71, MSE = 0.22, p = 0.07, η_p² = 0.14. Planned contrasts confirmed the significant ORE for all conditions except recognition of male faces by Asian participants, t(23) = 0.9, one-tailed p = 0.19, Cohen's d = 0.15; all other contrasts, ts(23) > 1.85, ps < 0.05, all Cohen's ds > 0.28. The smaller ORE in Asian than in Caucasian participants may be due to the fact that they had relatively more other-race contact than did Caucasian participants (see later, under “Contact with other-race faces was correlated with the ORE”). 

Figure 2

 

Mean d′ values in recognition after encoding single-view faces in Experiment 1. Error bars are standard errors estimated from the ANOVA.

Experiment 1 showed an ORE after learning single-view faces, consistent with many previous studies that used the same type of faces (Meissner & Brigham, 2001). The results also directly demonstrate that the ORE survives viewpoint change at recognition. Although in one condition (Asian participants recognizing male faces) the ORE was not significant, it showed the same pattern as in all other conditions. Furthermore, we found no significant interaction between face gender and face race or any other factors, suggesting that face gender does not significantly modulate the ORE. In the subsequent two experiments, we varied face format at encoding while keeping all other aspects identical to Experiment 1, so any change of the ORE should be attributed to encoding format. 

Experiment 2: Encoding rigidly moving faces eliminated the ORE

Experiment 2 examined whether the ORE observed in Experiment 1 persists when participants learned rigidly moving faces. Participants learned faces that were rotating back and forth between left and right 45° view (with front view skipped). No participants reported that they noticed the gap at the front view. As in Experiment 1, front-view faces were only shown at test to avoid image matching. 

Figure 3 plots the mean d′ values as a function of face race, face gender, and observer. In contrast to Experiment 1, we found neither a significant main effect of face race (1.03 ± 0.08 vs. 1.01 ± 0.09), F(1, 46) = 0.01, MSE = 0.27, p = 0.94, η_p² < 0.01, nor significant interactions between face race and face gender, observer, or higher order interactions, Fs < 2.60, ps > 0.11, all η_p²s ≤ 0.05. Planned contrasts revealed equivalent memory performances for own- and other-race faces in each condition [recognition of female faces by Caucasian participants: t(23) = 1.51, one-tailed p = 0.07, Cohen's d = 0.32; all other contrasts, ts < 0.82, ps > 0.21, all Cohen's ds < 0.20]. Thus, when participants learned rigidly moving faces, the own-race advantage observed in Experiment 1 disappeared. There was also a main effect of face gender, F(1, 46) = 4.63, MSE = 0.25, p = 0.04, η_p² = 0.09, with an overall better performance for female than for male faces (1.05 ± 0.09 vs. 0.89 ± 0.09). The main effect of observer was significant, F(1, 46) = 7.02, MSE = 1.30, p = 0.01, η_p² = 0.13. Caucasian participants performed better than Asian participants (1.19 ± 0.12 vs. 0.75 ± 0.12). 

Figure 3

 

Mean d′ values in recognition after encoding rigidly moving faces in Experiment 2.

To directly test whether encoding format modulates the ORE, we combined data from Experiments 1 and 2 and performed a four-way mixed-design ANOVA. Face race and face gender were within-participant factors, and observer and face format were between-participant factors. We found a main effect of face race (1.12 ± 0.06 vs. 1.00 ± 0.06), F(1, 92) = 6.31, MSE = 0.24, p = 0.01, η_p² = 0.06, and a marginally significant effect of encoding format (1.16 ± 0.08 vs. 0.97 ± 0.08), F(1, 92) = 3.18, MSE = 1.08, p = 0.08, η_p² = 0.03. More importantly, the interaction between these two factors was significant, F(1, 92) = 6.90, MSE = 0.24, p = 0.01, η_p² = 0.07. These results confirmed that an ORE was observed when participants learned single-view faces but was absent after learning rigidly moving faces. There was an overall trend of better memory for female than for male faces (1.12 ± 0.06 vs. 1.01 ± 0.06), F(1, 92) = 3.69, MSE = 0.32, p = 0.06, η_p² = 0.04. 

The results of Experiments 1 and 2 indicate that change of face format affects the processing of own-race and other-race faces differently. Nonetheless, it remains unclear which types of information embedded in rigid facial motion abolish the ORE. In comparison to single-view faces, rigidly moving faces not only incorporate facial motion but also contain additional face views. To tease apart these two factors, we had participants learn multiview faces in Experiment 3. 

Experiment 3: Encoding multiview faces eliminated the ORE

Experiment 3 investigated whether encoding multiview faces leads to the disappearance of the ORE. Instead of learning rigidly moving faces, participants in Experiment 3 learned faces that were displayed from four discrete viewpoints (in order: left 45°, left 15°, right 15°, and right 45° views). Each face view was displayed for 2 s, with a 250-ms blank screen inserted in between to prevent participants from perceiving apparent motion. Therefore, multiview and rigidly moving faces were similar in terms of multiple face views and natural presentation order, but they differed in term of facial motion. 

Figure 4 plots mean d′ values as a function of face race, face gender, and observer. Consistent with Experiment 2, neither the main effect of face race (0.97 ± 0.09 vs. 0.97 ± 0.10), F(1, 46) = 0.06, MSE = 0.41, p = 0.80, η_p² < 0.01, nor its interactions with face gender or observer were significant, Fs < 1.30, ps > 0.26, all η_p²s < 0.03. Planned contrasts showed no significant ORE in all individual conditions, ts < 0.64, ps > 0.26, all Cohen's ds < 0.16. These results suggest that presentation of multiple face views in natural order is sufficient to eliminate the ORE. 

Figure 4

 

Mean d′ values in recognition after encoding multiview faces in Experiment 3.

As in Experiment 2, to statistically confirm that the ORE is sensitive to face format at encoding, we performed the same four-way ANOVA on the combined data from Experiment 1 (encoding single-view faces) and Experiment 3 (encoding multiview faces). We found a significant main effect of face race (1.16 ± 0.05 vs. 1.02 ± 0.06), F(1, 92) = 6.11, MSE = 0.31, p = 0.02, η_p² = 0.06, and a significant interaction between face race and face encoding format, F(1, 92) = 4.26, MSE = 0.31, p = 0.04, η_p² = 0.04. These results confirmed that the significant ORE observed in Experiment 1 was eliminated when participants learned multiview faces. Caucasian participants exhibited overall better performance than Asian participants (1.19 ± 0.07 vs. 0.99 ± 0.07), F(1, 92) = 4.61, MSE = 0.90, p = 0.03, η_p² = 0.05. 

To check whether encoding multiview faces and encoding moving faces are functionally equivalent, we performed the same four-way ANOVA for combined data from Experiments 2 and 3. The results showed only a significant main effect of observer (1.16 ± 0.08 vs. 0.84 ± 0.08), F(1, 92) = 8.72, MSE = 1.12, p < 0.005, η_p² = 0.09, and a marginally significant interaction between face gender and encoding format, F(1, 92) = 3.88, MSE = 0.28, p = 0.05, η_p² = 0.04. Thus, encoding multiview faces and rigidly moving faces produced equivalent performance for memory of own- and other-race faces. This result suggests that the encoding process that is shared by learning rigidly moving and multiview faces plays a critical role in eliminating the ORE. 

Encoding format affected own-race and other-race faces differently

To examine how the encoding format affects memory of own-race and other-race faces differently, we combined data across the three experiments and performed separate three-way ANOVAs (face gender × encoding format × observer) on memory performance for own-race and other-race faces. As illustrated in Figure 5, encoding format affected only memory of own-race faces, F(2, 138) = 4.50, MSE = 0.61, p = 0.01, η_p² = 0.06, not of other-race faces, F(2, 138) = 0.11, MSE = 0.72, p = 0.89, η_p² < 0.01. 

Figure 5

 

Mean d′ values in recognition of own- and other-race faces as a function of encoding format.

For own-race faces, encoding single-view faces resulted in higher memory performance than encoding moving faces, F(1, 92) = 7.86, MSE = 0.63, p < 0.01, η_p² = 0.08, or encoding multiview faces, F(1, 92) = 5.55, MSE = 0.57, p = 0.02, η_p² = 0.06. There was no difference in recognition performance after learning rigidly moving or multiview faces, F(1, 92) = 0.32, MSE = 0.65, p = 0.58, η_p² < 0.01. In contrast, memories of other-race faces were equivalent across different encoding formats, Fs < 0.23, ps > 0.63, all η_p²s < 0.01. Caucasian participants showed better performance than Asian participants for both own-race and other-race faces, Fs > 5.70, ps < 0.02, all η_p²s > 0.04, but none of the interactions involving the observer factor ere significant, Fs < 1.15, ps > 0.24, all η_p²s ≤ 0.02. 

We further examined whether encoding format similarly affects hit and false alarm rates in recognizing own- and other-race faces. For hit rates (Figure 6, upper panel), a main effect of encoding format was observed for both own-race faces, F(2, 138) = 8.53, MSE = 0.04, p = 0.0003, η_p² = 0.11, and other-race faces, F(2, 138) = 7.16, MSE = 0.05, p = 0.001, η_p² = 0.09. For faces of both races, hit rates were higher following study of single-view faces than following study of either rigidly moving faces, Fs > 14.77, ps ≤ 0.0002, both η_p²s ≥ 0. 14, or multiview faces, Fs > 6.58, ps ≤ 0.01, all η_p²s ≥ 0. 07. For false alarm rates (Figure 6, lower panel), influence of encoding format was only observed for other-race faces, F(2, 138) = 4.36, MSE = 0.04, p = 0.01, η_p² = 0.06, with higher false alarm rates following study of single-view faces than following study of moving or multiview faces, Fs > 4.94, ps ≤ 0.03, all η_p²s ≥ 0. 05. In contrast, false alarm rates in recognizing own-race faces were the same across different encoding formats, F(2, 138) = 0.11, MSE = 0.04, p = 0.89, η_p² < 0.01. Taken together, in comparison to learning single-view faces, study of rigidly moving or multiview faces decreased hit but not false alarm rates in recognizing own-race faces, resulting in a lower sensitivity (d′). In contrast, for recognition of other-race faces, both hit and false alarm rates were reduced after learning rigidly moving or multiview faces, leading to an unchanged sensitivity across encoding format. Therefore, the elimination of ORE was due to different influences of encoding format on hit and false alarm rates in recognizing own- and other-race faces. 

Figure 6

 

Mean hit (upper) and false alarm rates (lower) in recognition of own- and other-race faces as a function of encoding format.

Contact with other-race faces was correlated with the ORE

To examine whether the ORE is associated with contact with other-race faces, we correlated the magnitude of ORE with the scores of interracial contact acquired from the questionnaire. As shown in Figure 7, there was a significant correlation between the two measures—Pearson's r(144) = 0.23, p = 0.005, 95% confidence interval = [−0.38, −0.07]—indicating that higher interracial contact tends to produce a smaller deficit in other-race face recognition (Hancock & Rhodes, 2008; Rhodes, Ewing, et al., 2009). The self-reported experience with other-race faces accounted for approximately 5% of the individual differences, consistent with earlier findings that interracial contact has a small, albeit reliable, contribution to the ORE (Meissner & Brigham, 2001). 

Figure 7

 

Correlation between other-race effect in recognition and scores of contact with other-race faces. CI = 95% confidence interval.

To test whether contact with other-race faces underlay the presence and absence of the ORE in the present study, we conducted a univariate ANOVA on the scores of interracial contact questionnaire, with observer (Caucasian vs. Asian) and encoding format (single-view vs. rigid motion vs. multiview) as fixed factors. There was no difference in contact scores between participants tested with different encoding formats (2.29 ± 0.12 vs. 2.33 ± 0.16 vs. 2.45 ± 0.14), F(2, 138) = 0.32, MSE = 0.97, p = 0.73, η_p² < 0.01. Hence, different patterns of ORE across experiments cannot be attributed to different levels of contact with other-race faces. Asian participants showed relatively more frequent contact with other-race people than Caucasian participants (2.48 ± 0.10 vs. 2.23 ± 0.13), but the difference was not statistically significant, F(1, 138) = 2.22, MSE = 0.97, p = 0.14, η_p² = 0.02. 

General discussion

The present study demonstrates that the ORE in face memory is sensitive to face format at encoding. When participants learned static, single-view faces, memory of own-race faces was consistently better than that for other-race faces. However, when the same set of faces was learned as moving faces rotated back and forth, the own-race advantage in face memory disappeared. This elimination of ORE cannot be attributed to the motion dynamics, as learning multiview faces also eliminated the ORE. These findings indicate that whether memory of own-race faces is better than other-race faces is dependent on how faces are presented during learning. 

Our results show both the robustness and the malleability of the ORE. On the one hand, the ORE could be reliably replicated when single-view faces were used, consistent with previous studies (Michel, Caldara, et al., 2006; Michel, Rossion et al., 2006; Rhodes et al., 2010; Tanaka et al., 2004). On the other hand, the ORE disappeared when encoding faces moved rigidly or were displayed from multiple views. It has been shown that the ORE can be reduced or eliminated by manipulation of perceptual and sociocognitive factors, such as attention (Hills & Lewis, 2006), facial emotion (Ackerman et al., 2006), training (McKone et al., 2007), and motivation for face individuation (Hugenberg et al., 2007). The present study indicates that the ORE can be eliminated without altering interracial contact or manipulating motivation to individuate other-race faces. In addition, most studies that have reported the elimination of ORE only tested Caucasian participants, which diminishes the generalizability of their findings. We demonstrate the malleability of the ORE for both Asian and Caucasian observers. 

What is the underlying mechanism for the influence of encoding format on the ORE? Differences in contact, in motivation for individuating other-race faces, and in holistic face processing have been assumed to underlie the ORE. The present data could not be readily accounted for by the first two hypotheses. The level of interracial contact was correlated with the magnitude of ORE for individual participants, consistent with prior studies (Hancock & Rhodes, 2008; Meissner & Brigham, 2001). However, we found no difference in interracial contact between participants who exhibited an ORE (after learning single-face faces) and those who did not (after learning moving or multiview faces). This result rules out the possibility that more experience with other-race faces, rather than encoding format, led to the disappearance of the ORE. 

The absence of an ORE cannot be attributed to increased motivation to individuate other-race faces either. In comparison to the single-view condition, neither the moving nor the multiview condition can be expected to shift participants' attention to either category- or identity-diagnostic face information. Furthermore, even if these conditions encouraged participants to attend more to identity-diagnostic facial aspects, the categorization-individuation hypothesis should predict an improvement in face memory. This was not the case in our study. Alternatively, it may be argued that categorization processes dominated the encoding of moving or multiview faces, which would remove the own-race advantage in memory. But there is no reason to expect such a categorization strategy for moving or multiview faces but not for single-view faces. Note that even when participants are explicitly asked to categorize the race of faces, they still show an ORE (Rhodes, Locke, et al., 2009). Therefore, we argue that the effect of encoding format on the ORE is not mediated by face individuation/categorization. 

The present study is consistent with the holistic processing hypothesis for the ORE. According to this hypothesis, the own-race advantage is supported by stronger holistic processing for own-race than other-race faces (DeGutis, Mercado, et al., 2013; Michel, Caldara, et al., 2006; Michel, Rossion, et al., 2006; Tanaka et al., 2004). Therefore, when faces are processed holistically, as has been demonstrated for single-view faces (Maurer et al., 2002; McKone, 2008), an ORE is expected. However, when faces are processed in a more analytic manner, as observed for rigidly moving faces (Xiao et al., 2012), the ORE should be reduced or eliminated. In addition, if own-race faces are processed more holistically (Michel, Caldara, et al., 2006; Tanaka et al., 2004), whereas other-race faces are processed more analytically (Nakabayashi et al., 2012; Rhodes et al., 1989; Tanaka et al., 2004), feature-based face processing should impair memory of own-race faces more than other-race faces. Our data are in good accordance with this prediction. 

One concern is that the presence and absence of the ORE may be dependent on whether encoding and test faces are shown in the same format (Experiment 1) or in different formats (Experiments 2 and 3). Although we did not manipulate face format at test, the mismatch between encoding and test formats cannot readily account for our data. If consistency between encoding and test affects face recognition, it should affect recognition of own- and other-race faces similarly, which was not the case in our study. Moreover, neither empirical studies nor theoretical accounts of the ORE are consistent with the idea that a mismatch between encoding and test face formats would eliminate the ORE. Butcher et al. (2011) have shown that following study of single-view faces, recognition of moving faces was not different from recognition of static faces, for both own- and other-race faces. Other studies have also showed no influence of test face format on the ORE (Hayward et al., 2008; Nakabayashi et al., 2012). It is therefore unlikely that testing moving faces in our study would lead to an ORE whereas testing single-view faces would not. In line with these empirical findings, most if not all theories of ORE assume that the ORE occurs at the encoding stage rather than the test stage (Hayward et al., 2013; Hugenberg et al., 2010; Levin, 2000; Michel, Rossion, et al., 2006; Rhodes et al., 2006; Rhodes et al., 1989; Sporer, 2001; Tanaka et al., 2004; Valentine, 1991; Walker & Tanaka, 2003). Our results are consistent with these theories. Therefore, in our view the elimination of the ORE seen here was unlikely due to the mismatch between encoding and test face format. 

Viewpoint change between learning and test faces cannot account for our data either. Schwaninger and Yang (2011) found an ORE in Caucasian participants even when learning and test viewpoints differed by 90° (i.e., learning front view, test profile view), indicating that viewpoint change cannot eliminate the ORE. Moreover, the presence/absence of an ORE was independent of viewpoint change (i.e., between test view and the nearest learning view), which was 15°, 5°, and 15° for our Experiments 1–3 respectively. 

Multiview and rigidly moving faces showed equivalent face memory performance (see Figure 5), suggesting that these face formats affect face encoding similarly. To form a consistent face representation for moving faces, participants may pay attention to the correspondence of facial features across different face views. Xiao et al. (2012) have proposed that such feature-based face encoding may be also possible when multiple face views are presented in coherent order with short a ISI (e.g., less than 300 ms). Under these circumstances, one face view would still be a strong cue to predict and therefore facilitate attention to the corresponding face features in the subsequent view. We followed these suggestions to induce feature-based encoding in the multiview condition. Therefore, the elimination of the ORE is probably due to feature-based encoding for rigidly moving and multiview faces. 

Similar to learning rigidly moving or multiview faces, learning inverted faces also reduces the ORE. The ORE is usually observed after encoding upright faces but not after encoding inverted faces (Crookes, Favelle, & Hayward, 2013; Hancock & Rhodes, 2008; Rhodes et al., 2006; Rhodes et al., 1989). Face inversion also impairs own-race face recognition more than other-race face recognition (Hancock & Rhodes, 2008; Rhodes et al., 1989). Nonetheless, whether face inversion disrupts holistic processing or decreases the efficiency of all aspects of face processing remains in debate (McKone & Yovel, 2009; Rhodes, Brake, & Atkinson, 1993; Rossion, 2008; Sekuler et al., 2004). Therefore, although face inversion also reduces the ORE as observed in our study, whether these influences are mediated by the same underlying processes remains to be elucidated. 

Rigid and elastic facial motion seem to have different influences on other-race face memory. We found no ORE for rigidly moving faces, whereas Butcher et al. (2011) report an ORE for elastically moving faces (i.e., speaking). These seemingly discrepant results might be reconciled according to the holistic processing account of ORE. Whereas rigid motion abolishes the composite face effect, elastic motion only reduces, but does not eliminate, this holistic processing effect (Xiao et al., 2012; Xiao, Quinn, Ge, & Lee, 2013). Assuming that holistic processing plays a key role in eliciting the ORE (DeGutis, Mercado, et al., 2013; Michel, Rossion, et al., 2006; Tanaka et al., 2004), the spared holistic processing in encoding elastically moving faces might still be able to elicit the ORE. In contrast, the stronger feature-based encoding for rigidly moving faces removes the advantage in processing own-race faces, eliminating the ORE. This speculation might also account for the ORE observed in field studies mentioned earlier (Brigham et al., 1982; Platz & Hosch, 1988; Wright et al., 2001), as facial motion in real-life situations is predominantly elastic (e.g., talking, smiling, etc.). 

Do holistic and featural processing also underlie the influence of contact or face individualization on the ORE? Recent studies suggest that it is possible. McKone et al. (2007) have shown that familiarization with other-race faces via training diminishes the ORE. More importantly, they found that holistic processing for observers trained with other-race faces was similar to that for own-race faces. This result suggests that the improvement of holistic processing may underlie the reduction of ORE via contact with other-race faces. The categorization-individualization hypothesis also proposes that face individuation may be implemented by selectively attending to identity-diagnostic characteristics such as configural information (Hugenberg et al., 2010, p. 1170). In this sense, face individuation is conceptually equivalent to holistic (configural) processing (Maurer et al., 2002). Bukach et al. (2012) have recently demonstrated that more individuating experiences with other-race faces lead to a smaller difference between holistic processing for own- and other-race faces, providing further evidence to support the view that more attention to face individuation may be associated with stronger holistic face processing. Future research is needed to directly address whether face individuation at encoding promotes holistic face processing, which would help specify what kind of perceptual process underlies the influence of face individuation on the ORE. 

In conclusion, the ORE observed after encoding single-view faces disappeared after learning faces in rigid motion or displayed from multiple viewpoints. The influence of encoding format is consistent with the hypothesis that the ORE is grounded upon the relative involvement of holistic face processing and cannot be attributed to either contact with or motivation to individuate other-race faces. Thus, whereas previous studies using single-view, static faces have emphasized the robustness of the ORE in face memory, the present study highlights its malleability in generalization to different face learning conditions. Relative engagement of holistic and featural processing at encoding probably underlies this malleability of ORE in face memory. 

Acknowledgments

This study was supported by the Max Planck Society. Part of this study was also supported by grants from the Hong Kong Research Grants Council to WGH (HKU 7440/08H and HKU 7449/11H). We would like to thank Matt Oxner for assistance with data collection. Part of this study was presented at the 2013 annual workshop on Object Perception, Attention, and Memory (OPAM 2013), Toronto, Canada. 

Commercial relationships: None 

Corresponding author: Mintao Zhao; Isabelle Bülthoff. 

Email: [e-mail]; [e-mail]. 

Address: Department of Human Perception, Cognition, and Action, Max Planck Institute for Biological Cybernetics, Tübingen, Germany.