Face aftereffects involve local repulsion, not renormalization

Abstract

After looking at a photograph of someone for a protracted period (adaptation), a previously neutral-looking face can take on an opposite appearance in terms of gender, identity, and other attributes—but what happens to the appearance of other faces? Face aftereffects have repeatedly been ascribed to perceptual renormalization. Renormalization predicts that the adapting face and more extreme versions of it should appear more neutral after adaptation (e.g., if the adaptor was male, it and hyper-masculine faces should look more feminine). Other aftereffects, such as tilt and spatial frequency, are locally repulsive, exaggerating differences between adapting and test stimuli. This predicts that the adapting face should be little changed in appearance after adaptation, while more extreme versions of it should look even more extreme (e.g., if the adaptor was male, it should look unchanged, while hyper-masculine faces should look even more masculine). Existing reports do not provide clear evidence for either pattern. We overcame this by using a spatial comparison task to measure the appearance of stimuli presented in differently adapted retinal locations. In behaviorally matched experiments we compared aftereffect patterns after adapting to tilt, facial identity, and facial gender. In all three experiments data matched the predictions of a locally repulsive, but not a renormalizing, aftereffect. These data are consistent with the existence of similar encoding strategies for tilt, facial identity, and facial gender.

Introduction

After looking at a photograph of someone, a previously neutral-looking face can take on an opposite appearance in terms of its gender (Rhodes et al., 2004; Webster, Kaping, Mizokami, & Duhamel, 2004), identity (Leopold, O'Toole, Vetter, & Blanz, 2001), emotional expression (Hsu & Young, 2004), ethnicity (Webster et al., 2004), age (Schweinberger et al., 2010), eye gaze, and head direction (Calder, Jenkins, Cassel, & Clifford, 2008; Fang & He, 2005; Lawson, Clifford, & Calder, 2011). It is less clear how nonneutral faces are affected. For example, does the appearance of the adapting face change during adaptation? Would a caricatured version of the original identity look more or less caricatured after adaptation? 

The answer to these questions may yield insights into how faces are represented neurally. Face aftereffects have repeatedly been attributed to a form of perceptual renormalization (Burton, Jeffery, Calder, & Rhodes, 2015; Jeffery et al., 2010; Jeffery et al., 2011; Leopold et al., 2001; McKone, Jeffery, Boeing, Clifford, & Rhodes, 2014; Pond et al., 2013; Rhodes & Jeffery, 2006; Rhodes et al., 2005; Robbins, McKone, & Edwards, 2007; Susilo, McKone, & Edwards, 2010a; Webster & MacLin, 1999). According to this account, faces are represented in terms of how they differ from a perceptual norm. The perceptual norm is constantly updated according to recent experience to minimize differences between the norm and the prevailing average stimulus in the environment (Rhodes & Jeffery, 2006; Rhodes et al., 2005; Webster et al., 2004). This process can change the appearance of a given face by increasing or decreasing its distance from the norm, explaining the face aftereffect. 

The perceptual renormalization hypothesis predicts that as the norm is updated to lie closer to the adaptor, the adapting face should appear more neutral. More extreme versions of the adaptor should also look more neutral after adaptation, as the distance between these and the updated norm, relative to the unadapted norm, is also decreased. Previously neutral-looking faces should take on an appearance opposite to the adaptor, as the distance between these and the updated norm will be increased relative to the unadapted norm (Robbins et al., 2007; Webster & MacLeod, 2011; Webster & MacLin, 1999). This pattern constitutes a unidirectional aftereffect, in which the appearance of all stimuli is shifted in the same direction to recenter around an updated norm (see Figure 1b). 

Figure 1

 

A depiction of how a renormalizing and a locally repulsive aftereffect might manifest along a dimension of facial gender. (a) A physical stimulus continuum, which ranges from caricatured male faces, through the gender-average face, to caricatured female faces. Prior to adaptation, the observer perceives a different strength of masculinity (blue) or femininity (red) in each face. (b) According to the renormalization account, facial gender is represented in terms of how each face differs from a perceptually gender-neutral norm (indicated by white circle). After adapting to a male face, the gender-neutral norm is recalibrated to lie closer to the adapting face, and all other faces in the encoded range change in appearance in the same direction. (c) If the facial gender aftereffect follows a locally repulsive pattern, then after adapting to a male face, there should be no change in the appearance of the adapting face, but the differences between adapting and test faces should be exaggerated. Faces that had previously appeared more masculine than the adaptor should be exaggerated in their masculinity, while faces that had appeared less masculine than the adaptor should appear more neutral or feminine than they had previously. Note that the two proposals predict similar perceptual changes for faces more neutral than the adapting face, but make different predictions for the adapting face itself and for more extreme versions of the adapting face.

Adaptation to simple spatial patterns, like lines and shapes, instead results in a bidirectional locally repulsive pattern of aftereffects (see Figure 1c). In the tilt aftereffect (Gibson & Radner, 1937; Vernon, 1934), adaptation to any oriented stimulus can exaggerate differences in orientation between adapting and test stimuli in both directions (Mitchell & Muir, 1976), with little change in the appearance of the adapted orientation itself (Köhler & Wallach, 1944; Mitchell & Muir, 1976; Storrs & Arnold, 2015). Locally repulsive aftereffects have also been reported following adaptation to spatial frequency (Blakemore, Nachmias, & Sutton, 1970) and aspect ratio (Badcock, Morgan, & Dickinson, 2014). If face aftereffects follow a similar locally repulsive pattern, the appearance of the adapting face should be unchanged by adaptation, while differences between the adaptor and subsequent test faces should be exaggerated. Taking facial gender as an example, a locally repulsive aftereffect uniquely predicts that adapting to a masculine face should make less masculine faces seem more feminine and more masculine faces seem more masculine (Robbins et al., 2007; Storrs & Arnold, 2012; Webster & MacLeod, 2011; Zhao, Seriès, Hancock, & Bednar, 2011). As yet, it is unclear whether face aftereffects follow the pattern predicted by renormalization or that predicted by local repulsion. 

Evidence that face aftereffects result from renormalization is ambiguous

The method most often used to quantify face aftereffects—a single stimulus binary-classification task—has limitations that make it difficult to determine what precise pattern a given aftereffect follows (see Morgan, 2014). On each trial in such an experiment, a participant is shown a single test stimulus, usually selected from a continuum (e.g., a morph of facial genders, from masculine to feminine). The participant is then asked to classify the test as belonging to one of two categories (e.g., male or female). An estimate of the point along the continuum at which classifications switch from being predominantly of one category to predominantly the other is taken as an estimate of the observer's category boundary. A shift in this boundary after adaptation is taken as a measure of the perceptual aftereffect. 

This method has two weaknesses. The first is that a shift in the category boundary is equally consistent with either a change in the appearance of stimuli, or with a change in criteria used to apply category labels (Gescheider, Herman, & Phillips, 1970; Green & Swets, 1966; Morgan, Dillenburger, Raphael, & Solomon, 2011; Morgan, Melmoth, & Solomon, 2013; Morgan, 2014; Storrs, 2015; Yarrow, Jahn, Durant, & Arnold, 2011). A second weakness is that binary classifications tend only to be sensitive to changes in the appearance of stimuli near a category boundary. Here, a small change in appearance can produce a large change in categorization behavior. For stimuli far away from the boundary, however, even a large perceptual change might be insufficient to change how stimuli are categorized. A very masculine-looking face, for instance, might be made to look less masculine, but still look sufficiently male to be so categorized. 

Unfortunately the subjective appearance of stimuli near a category boundary, as measured in a standard binary-classification task, is usually unhelpful in differentiating renormalizing from locally repulsive aftereffects. After adapting to a nonneutral stimulus, both proposals predict a shift in the appearance of neutral stimuli away from the adapted value (see Figure 1). Also problematic is that after adapting to a neutral stimulus, neither proposal predicts a shift in the appearance of that stimulus. 

It has been suggested that renormalizing and locally repulsive aftereffects can be dissociated by determining aftereffect magnitudes at the category boundary as a function of increasingly nonneutral adapting stimuli (Jeffery et al., 2010; Jeffery et al., 2011; McKone et al., 2014; Pond et al., 2013; C. Zhao et al., 2011). The rationale is that if an aftereffect entails a local repulsion, then the aftereffect for stimuli near the category boundary might initially increase, but should then diminish as the distance between adapting and test stimuli exceeds the limited range of the local repulsion. On the other hand, if an aftereffect involves renormalization of the whole perceptual dimension toward the adapting stimulus, the extent of recalibration should only increase for more extreme adaptors. There is, however, an important caveat that prevents this protocol from being diagnostic. Facial dimensions are thought to be finite, so increasingly nonneutral stimuli will eventually escape the confines of the dimension, and begin to look unnatural. An increasingly male-looking face, for instance, might begin to look nonhuman—ogre-ish. A gradual increase, and eventual reduction, in aftereffect magnitude for increasingly nonneutral adaptors (now repeatedly reported, see McKone et al., 2014; C. Zhao et al., 2011), is therefore equally consistent with either a locally repulsive aftereffect, or with a renormalization process that is not strongly activated by unrealistic faces (Pond et al., 2013). 

A novel spatial-comparison task

To test the renormalization and local-repulsion hypotheses, one needs a task for which these aftereffect patterns make distinct predictions. In a previous report (Storrs & Arnold, 2012), we showed that by using a three-category classification task (e.g., male, androgynous, or female), and adapting to one of the two category boundaries, one can measure an aftereffect at a particularly diagnostic point: the adapted value itself. A renormalizing aftereffect will result in a previously nonneutral adaptor looking more neutral after adaptation, while a locally repulsive aftereffect predicts no change in the adaptor's appearance. For facial gender adaptation, we found no change in how the adapted value was categorized, consistent with a locally repulsive aftereffect. After adapting to a configurally distorted face, however, there was an increased tendency to place the adaptor in the “undistorted” category, consistent with renormalization. While this suggested multiple facial encoding strategies, caution must be exercised when interpreting these data, due to the subjective nature of the categorical tasks on which the protocol depended. In the following experiments we adopt another protocol that allows one to measure aftereffects at any point along a test continuum, and to avoid relying on subjective categorical decisions. 

Our protocol makes use of two characteristics of face aftereffects. First, face aftereffects are strongest when adapting and test stimuli are matched in retinal position (A. Afraz & Cavanagh, 2009; S.-R. Afraz & Cavanagh, 2008; although some tolerance is displayed to positional variance, see Yamashita, Hardy, De Valois, & Webster, 2005; L. Zhao & Chubb, 2001). Second, when different adaptors are shown simultaneously in different retinal locations, distinct spatially contingent aftereffects can ensue (S.-R. Afraz & Cavanagh, 2008). This allows for a powerful psychophysical method: a spatial comparison task. One can present a standard test stimulus in one location and find which stimulus value in a second location appears to match it (e.g., Elliott, Georgeson, & Webster, 2011; Farell & Pelli, 1999; Jäkel & Wichmann, 2006; Kompaniez, Abbey, Boone, & Webster, 2013). At baseline, perceptually matched stimuli are likely also to be physically matched (for an exception, see A. Afraz, Pashkam, & Cavanagh, 2010). If adaptation impacts perception, a standard stimulus presented in an adapted location should appear to match a physically different stimulus presented in an unadapted location. Aftereffect magnitudes can therefore be calculated as the difference between matches to the same standard stimulus pre- and postadaptation. 

In the experiments reported here, we use a forced-choice double-pair task (Kaplan, Macmillan, & Creelman, 1978; Rousseau & Ennis, 2001). Two pairs of peripheral stimuli were shown at the test. One contained an identical pair of standard stimuli, whereas the other contained one standard and a test that differed from the standard by a variable amount. Participants were asked to indicate which interval contained the identical pair. If the proportion of times that a participant selected the incorrect interval is plotted as a function of the test value, one should find a bell-shaped distribution peaking at the value that perceptually matches the standard. The magnitude of any aftereffect is signaled by the shift of this peak, from pre- to postadaptation. 

Rather than compare appearance between an adapted and an unadapted location, we instead adopt a double-adaptor paradigm, comparing appearance between two differently adapted locations. We do this for two reasons. First, previous research suggests adaptation to a single peripheral facial image can affect classifications of spatially distant tests, so it may be difficult to find a measurable difference between adapted and unadapted locations (S.-R. Afraz & Cavanagh, 2008). In the same study, however, spatially contingent aftereffects were induced when multiple adaptors were presented, highlighting the possibility of differentially adapting different locations. Second, using two adaptors minimizes the potential that asymmetric spatial attention might bias subsequent perceptual decisions. If there were just one adaptor, people might attend more to stimuli in that location (attention has been shown to modulate face aftereffects measured via a binary classification task; Rhodes et al., 2011), or be tempted to compare stimuli to the single adaptor. With two adaptors, spatial attention is more likely balanced, and it becomes harder to compare the four test images to the two different adaptors. 

If face aftereffects reflect a renormalization of all stimuli about a new neutral point (Rhodes et al., 2005; Webster & MacLin, 1999), aftereffects measured at different positions along a test continuum will tend to be constant (see Figure 2d through f). As we will be measuring the relative aftereffect induced between two differently adapted locations at different standard test values, renormalization predicts a constant magnitude of relative aftereffect for all test values, equal to the difference between the aftereffects induced in each location. If, on the other hand, face aftereffects involve local repulsion, we should find that the relative aftereffect is largest for standard test values midway between the two adaptors (see Figure 2a through c). Such tests should be pushed in opposite directions by opposite local repulsions, whereas aftereffects measured at either of the adapted values should be smaller, as they will reflect the influence of just one adaptor. 

Figure 2

 

Patterns of results predicted by a locally repulsive (left) and a renormalizing aftereffect (right) in our spatial-comparison task. (a) After adapting in two different retinal locations to stimulus values of −200 (left; arbitrary units) and −50 (right), an aftereffect is induced in each location which manifests (b) as a local repulsion of nearby values away from the adapted value, with no change in the appearance of the adapted value. (c) The predicted mismatch in appearance between stimuli presented at the two differently adapted locations is given by the difference between the two aftereffect functions (black). The shape of the difference function varies depending on the shape and separation of the aftereffects in each location, but the qualitative pattern remains constant across a wide range of possible values: Maximal aftereffects are predicted for test values in between the two adaptors, and lesser aftereffects are predicted for tests at either of the two adapted values. (d) If, after adapting to these same stimulus values, aftereffects manifest as (e) a uniform renormalization of all stimulus values, then the difference in appearance between two differently adapted locations (f) will also be uniform across all encoded test values.

In the experiments that follow, we test these predictions for tilt adaptation (Experiment 1; to validate our protocol in a context repeatedly linked to locally repulsive aftereffects), for facial identity adaptation (Experiment 2), and for facial gender adaptation (Experiment 3). We find that data in all three contexts are consistent with locally repulsive, but not renormalizing, patterns of aftereffect. 

Method

Experiment 1: Tilt aftereffects

Participants

Ten observers participated in Experiment 1, including the first author, three experienced psychophysical observers naive to the research hypotheses, and six inexperienced observers recruited from the University College London (UCL) psychology participation pool, who were compensated with £6 for their time. The Experimental Psychology Ethics Committee at UCL approved all experiments. 

Stimuli and apparatus

Stimuli were presented on a 22-in. Mitsubishi Diamond Plus 230SB monitor (Mitsubishi Electric, Hatfield, UK; resolution 1280 × 1024 pixels; refresh rate 85 Hz; not gamma corrected), using the Psychophysics Toolbox for Matlab (Brainard, 1997; Pelli, 1997). Participants viewed stimuli from a distance of 57 cm using a chinrest. During both a preliminary sensitivity measure and the main experimental task, pairs of stimuli were presented simultaneously, centered 4.5 degrees of visual angle (dva) to either side of a central white fixation point (which had a diameter subtending 0.24 dva). Participants were instructed to fixate the central fixation point throughout all experiments. 

Test and adapting stimuli were pairs of Gabors with a Michelson contrast of 1 and a spatial frequency of 2.5 cycles/dva, presented within Gaussian spatial envelopes with standard deviations subtending 1.1 dva. The phase of each Gabor waveform was randomized on a trial-by-trial basis. The display background was gray and matched the average luminance of test stimuli. 

Procedure

In the main experiment, we measured the relative aftereffect between two differently adapted retinal locations at three different standard test values. The three standard test values were determined independently for all participants via a preliminary procedure that measured orientation discrimination sensitivity. 

Preliminary procedure

Stimuli were presented in a dual-pair task (Rousseau & Ennis, 2001; also known as a “four-interval AX” task, e.g., Kaplan et al., 1978; see Figure 3). Two pairs of static Gabors were presented sequentially for 300 ms, separated by a 300-ms blank interstimulus interval (ISI). On each trial, three of the test stimuli were set to a standard value of −45° (which will serve as the central of the three standard test values in the main experiment). The orientation of the fourth varied via an adaptive procedure described below. The interval and position (left or right of fixation) of the variable test was randomized on a trial-by-trial basis. The participant indicated which of the two intervals had contained identical stimuli by clicking one of two mouse buttons. Feedback was given, with the fixation dot turning green for correct decisions and red for incorrect decisions for 500 ms prior to the next trial. 

Figure 3

 

Trial structure for Experiment 1. Two pairs of test stimuli were shown for 300 ms each, sequentially with a 300-ms ISI. Three of the four tests were set to an identical standard test value (−45°), while the orientation of the fourth was varied according to an adaptive procedure (see main text for description). Tests were either shown in isolation (during the preliminary sensitivity measure, and during baseline trials of the main experiment) or after exposure to a pair of adapting stimuli for 5 s. Participants were asked to indicate which of the two test intervals had contained identical stimuli.

The orientation difference between the variable test and the standard test value was varied on a trial-by-trial basis according to adaptive staircase procedures (Cornsweet, 1962). Four staircases were interleaved, with two sampling values rotated counterclockwise from the standard and two sampling values rotated clockwise from the standard. For both counterclockwise and clockwise sampling procedures, one of the two staircases was initiated at an orientation difference of 0° (i.e., an initial orientation of −45°), and the other was initiated at an orientation difference of 10° (i.e., initial orientations of −55° for clockwise-sampling staircases, and −35° for counterclockwise-sampling staircases). 

Test orientation differences were adjusted according to a three-down, one-up decision rule, wherein three consecutive correct responses resulted in a decrease in the magnitude of orientation differences, and any incorrect response resulted in an increase in orientation difference magnitude. This decision rule converges on 79% correct performance (Levitt, 1970). For each staircase, a reversal was recorded when the direction of adjustment (increasing versus decreasing the orientation difference between the −45° standard and the variable test) differed from the direction of the last adjustment for that staircase. Test orientation differences were adjusted in steps of 2° until the first three reversals for that staircase, after which adjustments were made in steps of 0.5°. Minimum orientation differences of 0° were enforced by repeatedly sampling this value if repeated correct judgments made this necessary. Each staircase terminated after six reversals. Orientation difference values corresponding with the last three reversals in each staircase were averaged across the four staircases to produce a single just-noticeable difference (JND) estimate from the standard test value. The average JND was 9.8° (minimum = 6.1°, maximum = 14.7°). 

Main experiment

The central standard value for the main experiment was set to −45° for all participants, and lower and upper standard test values were set to multiples of the participant's JND from the central standard (see Figure 4a). Lower standard test values were −45° minus two JNDs, and upper standard test values were −45° plus two JNDs. In adaptation trials, the adaptor shown to the left of fixation was of the lower standard value, and the adaptor shown to the right of fixation was of the upper standard value. The average lower standard test value was −64.5° (±2.0° SEM) and the average upper standard test value was −25.5° (± 2.0°). 

Figure 4

 

Selection of standard test stimuli for (a) tilt, (b) facial identity, and (c–d) facial gender experiments. The selection procedure was identical in all cases. A central standard test value was chosen (−45° for tilt adaptation; Image 66 for facial identity adaptation; Image 300 for female-centered facial gender adaptation, and Image 200 for male-centered facial gender adaptation). Estimates of the participant's JND from central standard test values were determined in a preliminary sensitivity measure (see main text for details). Lower and upper standard test values were then set respectively to −2 JNDs and to +2 JNDs from central standard test values. Arrows indicate approximate average lower, central and upper standard test values in each experiment.

All participants first completed a baseline run of trials during which they viewed sequential tests in the absence of adaptation. They then completed an adaptation run of trials, in which they viewed sequential tests after either 30 s (first trial) or 5 s (all subsequent trials) of exposure to the adapting stimuli (see Figure 3). 

On each trial, during both baseline and adaptation runs of trials, three of the four test stimuli were set to a standard test value, pseudorandomly selected on each trial from either the lower, central or upper standard test values. The value of the fourth variable test stimulus was determined according to an adaptive procedure described below. The test stimulus was always shown on the right hand side of fixation in a random interval. Observers indicated which of the two intervals had contained identical stimuli. No performance feedback was given. 

The value of the variable test stimulus was determined on each trial by a Pólya urn adaptive sampling procedure (Rosenberger & Grill, 1997; Yarrow, Sverdrup-Stueland, Roseboom, & Arnold, 2013). The range of possible test values was −135° to +45°, sampled in increments of 0.25° (i.e., 360 possible test values). Three independent sampling procedures were maintained, one for each of the three standard test values. 

On each trial, a standard test value was pseudorandomly selected, and a variable test value was selected according to the relevant sampling procedure. The initial probability distribution for each of the three sampling procedures was uniform, with a nominal sampling probability value of 1 assigned to each test value falling within a range ±3 JNDs from the relevant standard, and zero to values falling outside this range. The actual probability of sampling any given test value on a trial was the current nominal probability of sampling that test value, divided by the sum of the current nominal probability values across all test values. On each trial, a particular value was chosen for the variable test via the generation of a random number used to index ordered sampling probability values. Whenever a participant incorrectly selected the interval containing the variable test as having contained identical stimuli, nominal sampling probability values associated with test values ±0.5° and ±2° from the test were increased by 2 and 1, respectively. These adaptive procedures ensured disproportionate sampling of tests perceptually similar to each standard stimulus. Each of the three adaptive procedures was sampled for 50 trials, yielding a total run of 150 trials (taking approximately 10 min in baseline runs of trials, and 20 min in adaptation runs of trials). 

Results

A run of trials provided three distributions, one for each standard stimulus, describing the proportion of times each variable test value had been presented on which it was mistaken as being identical to the relevant standard stimulus (see Figure 5a). Gaussian distributions were fit to these data, using a least-squares regression weighted by the number of observations at each variable test value. Peaks of the fitted Gaussians were taken as estimates of the participant's point of subjective equality (PSE) between tests presented in the right retinal location and each standard stimulus presented in the left retinal location. Relative aftereffects between the two differently adapted locations were calculated as the postadaptation PSE for a given standard minus the corresponding baseline PSE. 

Figure 5

 

(a) Example data from one participant in the tilt adaptation experiment. Open blue dots represent the proportion of incorrect responses for each variable test value during baseline trials on which the upper (top panel), central (middle), or lower (bottom) standard test stimulus was shown; closed red dots represent the same during adaptation trials. Data are binned into 5° bins for illustration purposes. Dotted gray lines indicate the physical value of each of the three standard test stimuli. Curves show Gaussians fit to response data before (blue) and after (red) adaptation. The peaks of the fitted Gaussians indicate the participant's PSE between the two retinal locations for each standard stimulus. The relative aftereffect between the two locations is calculated independently for each standard stimulus by subtracting the baseline PSE from the postadaptation PSE. (b) Relative aftereffects at each of the three standard test values in the tilt adaptation experiment. Error bars indicate ± 1 SEM between individual estimates.

Renormalization predicts that the magnitude and direction of relative aftereffects should be constant for our three standards (see Figure 2f). Local repulsion predicts that relative aftereffects will be greater for the central standard than for lower and upper standards (see Figure 2c). We confirmed that our pattern of results conformed to the latter prediction by conducting a repeated-measures ANOVA. This revealed a significant quadratic trend for data relating to our three standards, F(1, 9) = 10.16, p = 0.011, with the greatest aftereffect magnitude at the central standard—as predicted by a locally repulsive aftereffect (see Figure 5b). 

Experiment 2: Facial identity aftereffects

Having validated our protocol for tilt adaptation, we then applied it to facial identity adaptation. Details were as for Experiment 1, with the following exceptions. 

Participants

Fifteen observers participated in the facial identity experiment, including the first author, three experienced psychophysical observers who were naive to the experimental hypotheses, and 11 naive paid volunteers. 

Stimuli and apparatus

Test and adapting stimuli were pairs of face images. Test images subtended 5.8 dva in width and 7.7 dva in height, whereas adaptors subtended 7.3 by 9.6 dva. The size difference between adaptors and tests was intended to mitigate local retinotopic adaptation, a precaution widely used in face aftereffect research. 

Face stimuli were color images generated from the Basel Face Model (BFM; Paysan et al., 2009),¹ a generative model created by performing principal components analysis (PCA) on the three-dimensional head scans of 100 male and 100 female, predominantly Caucasian, individuals. Images generated by the model are cropped around the ears, necks, and forehead to remove hair (see Figure 4b). 

In the BFM two sets of 199 principal components have been derived independently for facial structure and facial texture (capturing variations due to skin color, eye color, lighting, etc.). This allowed us to minimize color and luminance artefacts by generating faces that varied structurally but not texturally. Data accompanying the BFM specify directions in face-structure and face-texture spaces along which variance is greatest for individuals of different genders, heights, weights, and ages. This allows these attributes to be manipulated in a naturalistic manner. 

We generated a unique continuum of face images for each participant to ensure our data were not dependent on idiosyncrasies of any single stimulus set. Each continuum consisted of 201 images drawn from a trajectory in face-structure space linking two different same-gender identities, centered on an average male or female face (seven male continua, eight female). For each continuum, Image 1 was a novel male (or female) identity chosen by adding a random proportion and direction of the height, weight, and age vectors to an average male (or female) face, along with a small random amount of each principal component. Image 101 was the average male (or female) face, and Image 201 was an “anti-identity” face that differed from the male (or female) average in an opposite manner relative to Image 1. Other images were then generated from the BFM, traversing in equal steps the distance between Images 1 and 201. Selected images from one stimulus continuum are shown in Figure 4b, and from all 15 continua in Supplementary Figure S1. 

Procedure

Sequential pairs of face images were presented for 500 ms each, separated by 300-ms ISIs. The central standard test value was set to Image 66 of each individual face continuum. 

Preliminary procedure:

Four independent, randomly interleaved staircase procedures were used to estimate each participant's JND from the central standard face. Two procedures sampled test faces with lower image numbers than the central standard, and two sampled higher image numbers. Within each pair of sampling procedures, one staircase was initiated at an image number difference of zero (i.e., Image 66), while the other was initiated at either Image 26 (for procedures sampling lower image numbers) or Image 106 (for procedures sampling higher image numbers). The average JND estimate was 26 image units (minimum = 14, maximum = 36). 

Main experiment:

Two participants had JNDs that would have led to the lower standard (central standard minus 2 JNDs) being outside the range of available test faces (JNDs = 34 and 36). For these observers, an arbitrary JND of 32 was used to select standard test images. Whenever a participant incorrectly selected the test interval as having contained identical faces, nominal sampling probability values associated with test images numbered ±10 and ±20 from the test were increased by 2 and 1, respectively. 

Results

Analyses of data were as for Experiment 1, unless specified. To assess whether aftereffects were greatest for the central standard, relative to lower and upper standards, we conducted a repeated-measures ANOVA. This revealed a significant quadratic trend for data relating to our three standards, F(1, 14) = 7.14, p = 0.018 (see Figure 6), with a maximal aftereffect for the central standard—as predicted by a locally repulsive aftereffect. 

Figure 6

 

Relative aftereffect at each of the three standard test values in the facial identity aftereffect experiment. Details as for Figure 5.

Experiment 3: Facial gender aftereffects

Details for Experiment 3 were as for Experiment 2, with the following exceptions. 

Participants

Fifteen observers participated in Experiment 3a (male-face–centered experiment), including the first author, five experienced psychophysical observers naive to the experimental hypotheses, and nine naive paid volunteers. In Experiment 3b (female-face–centered experiment), participants included the first author, six experienced observers, and eight paid volunteers. Note that some people participated in both experiments, with a total of 19 unique observers (five male, 14 female). When the same person participated in both experiments, these were conducted on separate days. 

Stimuli and apparatus

Test images subtended 5.9 dva in width and 6.5 dva in height. Adaptors subtended 7.8 by 8.7 dva. Face stimuli were 500 color images previously used in C. Zhao et al. (2011) and Pond et al. (2013; the “twenty” continuum).² These were generated by morphing between and beyond an average male face (−100% gender; Image 200) and an average female face (100% gender; Image 300), to create a continuum spanning from a masculine caricature (−500% gender; Image 1) through androgynous (0% gender; Image 250) to a feminine caricature (500% gender; Image 500. For further details see C. Zhao et al., 2011). Face images were presented against a gray background, inside a gray oval frame of the same luminance, which occluded the ears and side of the head. All participants saw the same stimulus continuum. 

Procedure

Experiment 3a had a central standard stimulus of 200; Experiment 3b a central standard stimulus of 300. 

Preliminary procedure

In Experiments 3a and b, four staircase procedures were used to locate each participant's JND for differences from the central standard, two sampling lower image numbers and two sampling higher image numbers. Within each pair of staircase procedures, one was initiated at an image number difference of 0 (i.e., initiated at Image 200 and 300, respectively, for Experiments 3a and 3b), while the other was initiated at an image number difference of 100 (i.e., lower sampling procedures were initiated at Image 100 in Experiment 3a and 200 in Experiment 3b; higher sampling procedures were initiated at Image 300 in Experiment 3a and 400 in Experiment 3b). Average JNDs were 54 image units (minimum = 28, maximum = 92) in Experiment 3a, and 57 image units (minimum = 35, maximum = 97) in Experiment 3b. 

Main experiments

In Experiment 3a, variable test stimuli were shown on the right-hand side of fixation, in the same location as the upper adaptor, as in the previous experiments. In Experiment 3b, variable test stimuli were shown to the left of fixation, in the same location as the lower adaptor. This meant that aftereffects should predominantly manifest as participants matching standard stimuli to lower variable test values after adaptation, rather than higher values as in the previous experiments. We took this precaution to avoid a possible ceiling effect when estimating the aftereffect at the upper standard, which was for some participants near the end of the image continuum (see Figure 4c). 

In both experiments, whenever a participant incorrectly selected the test interval as having contained identical faces, nominal sampling probability values associated with test images numbered ±10, ±25, and ±50 from the variable test value were increased by 3, 2, and 1, respectively. 

Results

For both experiments, data for each standard were grouped into 10-image bins, then expressed in terms of the proportion of times, for variable test values within each bin, that the participant had responded incorrectly. Gaussian functions were fit to these data, and their peaks were taken as PSE estimates for each standard stimulus, before and after adaptation. 

To assess whether aftereffects were greater for central standards than lower and upper standards, we conducted two repeated-measures ANOVAs. For Experiment 3b, this revealed a significant quadratic trend for data relating to our three standards, F(1, 14) = 6.33, p = 0.025 (see Figure 7), with a maximal aftereffect at the central standard—as predicted by a locally repulsive aftereffect. In Experiment 3a, aftereffects were evident (a two-tailed one-sample t test performed on the average aftereffect across all three standards for each participant revealed a mean relative aftereffect of 42 image units, t₁₄ = 4.67, p < 0.001), but there was no significant quadratic trend between data relating to the three standards, F(1, 14) = 1.82, p = 0.200. The reader should note, however, that the pattern of results in this experiment conformed with the pattern of results in all other experiments we have reported here, in that the largest aftereffect observed was for the central standard, whereas aftereffects for the lower and upper standards were smaller, albeit not statistically different in this case. 

Figure 7

 

Relative aftereffect at each of the three standard test values in (left) the male-face–centered and (right) the female-face–centered facial gender adaptation experiments. Because the variable test was shown in the same location as the lower adaptor for Experiment 3b (but that of the upper adaptor for other experiments), the aftereffects manifested as a negative rather than a positive shift, relative to baseline. For ease of comparison, aftereffects are shown here with their sign flipped. Other details as for Figures 5 and 6.

General discussion

We measured aftereffects following tilt, facial identity, and facial gender adaptation, and found strikingly similar patterns of results, which were in excellent agreement with the predictions of a locally repulsive aftereffect. These data were, however, inconsistent with the predictions of a renormalizing aftereffect. 

Locally repulsive aftereffects provide evidence for multichannel encoding

Perceptual aftereffects are often thought to result from neural adaptation—changes in the responsiveness of neurons after prolonged stimulation (see, e.g., Kohn, 2007; Webster, 2011). Two broad classes of encoding scheme have been proposed. Locally repulsive aftereffect patterns have been associated with encoding schemes in which multiple channels are narrowly tuned for different input values, with no particular stimulus playing a special role (e.g., Blakemore et al., 1970; Clifford, Wenderoth, & Spehar, 2000; Pouget, Dayan, & Zemel, 2000; Seriès, Stocker, & Simoncelli, 2009). In Figure 8a we illustrate a simulated multichannel code before adaptation (blue curves), consisting of Gaussian channels with peaks uniformly spaced along the stimulus dimension (see Supplementary Materials for Matlab code implementing this model). 

Figure 8

 

Simulations of channel structure and adaptation in (left) a multichannel and (right) an opponent code. (a) The response of each channel in a multichannel code to each stimulus value, before (blue) and after (red) adapting to a stimulus value of −100 (arbitrary units), shown by vertical dashed red line. MLE can be used to decode (b) the most probable value of each stimulus, given the channel activity it elicits, before (blue) and after (red) adaptation. The aftereffect (b, inset) is calculated as the postadaptation decoded stimulus minus the pre-adaptation decoded stimulus. The aftereffect manifests as a local repulsion of test stimuli away from the adapted value, with no change in the appearance of the adapted value. (c) The relative aftereffect (black) predicted at each test value is given by the difference between the two location-specific aftereffects (green curve minus orange curve). (d) Channel responses in an opponent code before and after adaptation. (e) MLE performed on the upper and lower channel activity veridically decodes stimulus values before adaptation (blue), and predicts a constant bias at all test values after adaptation (red). (e, inset) The aftereffect manifests as a simple renormalizing effect, in which the appearance of all test stimuli is shifted in the same direction by the same amount. (f) The predicted relative aftereffect between two differently adapted locations (black) is also constant across a wide range of test values.

Adaptation to a given stimulus is proposed to cause a temporary reduction in the excitability of a subset of channels, in proportion to their response to the adaptor (red curves in Figure 8a). Perceptual experience is presumably determined by the pattern of activity across the population of channels. When this decoding process is simulated via a technique such as maximum-likelihood estimation (MLE; Ma & Pouget, 2009; Pouget et al., 2000), systematic differences can arise between estimates based on pre- and postadaptation activity. These take the form of a locally repulsive pattern of biases (red curve and inset in Figure 8b), in which the decoded value of the adaptor is unchanged, but differences in decoded values between the adaptor and nearby stimuli are exaggerated. Crucially, a multichannel encoding scheme accurately predicts the patterns of perceptual bias seen after adapting to orientation or spatial frequency (e.g., Blakemore et al., 1970; Clifford et al., 2000; Goris, Putzeys, Wagemans, & Wichmann, 2013; Seriès et al., 2009). 

Renormalizing aftereffect patterns have been associated with opponent-channel encoding schemes, in which two channels respond increasingly as stimulus values deviate from a perceptual norm—one preferring low values along the encoded dimension and the other high values (Giese & Leopold, 2005; Regan & Hamstra, 1992; Rhodes et al., 2005; Susilo et al., 2010). In Figure 8d we illustrate a one-dimensional simulation of such an encoding scheme, in which two monotonic channels intersect at stimulus value zero (blue curves; see Supplementary Materials for accompanying Matlab code). The stimulus corresponding to the norm value (here, zero) activates both channels equally, while any other stimulus elicits an imbalanced response from the two channels. Again, the perceived stimulus is determined by the particular combination of channel activities. 

As in the multichannel scheme, adaptation is thought to bring about a temporary reduction in the excitability of one or both channels, in proportion to their response to the adaptor (red curves, Figure 8d). When one channel is disproportionately suppressed, the input that elicits a balanced response across the two channels shifts toward the adapted value, thereby updating the norm to more closely resemble recent experience. This recalibration can produce systematic changes in the perception of the same physical stimulus before and after adaptation. The opponent-channel code predicts a uniform renormalization, in which the adaptor appears more neutral after adaptation, and the appearance of all other stimuli is altered in the same direction by the same amount (red line and inset in Figure 8e). The magnitude and direction of this constant shift depends on the adapting value, with adaptors further from neutral producing larger shifts, and adaptation to zero uniquely producing no aftereffect. 

A uniform shift in appearance of all test stimuli is perhaps the simplest possible prediction that can be derived from an opponent-channel code. One can also plausibly predict nonuniform changes, with reduced aftereffects for extreme test values near the limits of the encoded range. We do not think this possibility could account for our findings for two reasons. First, upper and lower standard values were not more extreme than central standards in our experiments. In both the facial identity and gender experiments, adapting and test stimuli were centered about a point offset from the computational average along the stimulus dimension. In the facial identity experiment, the upper standard stimulus was closest to the computational average, yet aftereffects were smaller for this standard than for the central standard (see Figure 4b). 

Second, in our two facial gender experiments we centered adapting and test stimuli about two different points along the same stimulus continuum, neither of which was the computationally average androgynous face (see Figure 4c and d). In both cases we found that aftereffects were greatest for the test value lying midway between the two adapted values (although this pattern only produced a significant quadratic trend for female-face–centered data). This suggests that aftereffect magnitudes were tied to the positions of test stimuli relative to adapting stimuli, rather than being determined by the absolute positioning of test stimuli along the stimulus dimension. This is more consistent with a locally repulsive aftereffect than with a renormalizing aftereffect that is reduced for extreme test stimuli. 

The locally repulsive pattern of aftereffects we observe is consistent with a model in which face aftereffects arise from local interactions, such as those that govern the tilt aftereffect (Dickinson, Almeida, Bell, & Badcock, 2010; Dickinson et al., 2012). However, as yet it is unclear how this model can account for the transfer of aftereffects between differently sized adapting and test stimuli. An alternate possibility is that our data are driven by adaptation within a multichannel-structured representation of facial attributes. 

Rather than committing to a particular instantiation of a neural encoding scheme (any form of which would almost certainly be an oversimplification), the key message of our data is that facial identity and gender aftereffects are better described as locally repulsive than as renormalizing aftereffects. If face adaptation involves a recentering of all stimuli around a new neutral point, as has often been proposed (e.g., Burton et al., 2015; Jeffery et al., 2010; Jeffery et al., 2011; D. Leopold et al., 2001; McKone et al., 2014; Pond et al., 2013; Rhodes & Jeffery, 2006; Rhodes et al., 2005; Robbins et al., 2007; Susilo et al., 2010; Webster & MacLin, 1999), then an aftereffect measured over a wide range of test values should be unidirectional and approximately equal in magnitude. If this were the case, we should have found approximately equal relative aftereffects for different standard test values in our spatial-comparison task. This is not what we found. Instead, aftereffects were greatest when tests differed from, and therefore could be influenced by, both adaptors. Aftereffects were smaller when tests were identical to one of the two adaptors and, according to the local repulsion hypothesis, should therefore not be affected by that adaptor. 

Isn't there lots of other evidence in favor of opponent coding?

Although there are several lines of evidence hinting that face aftereffects involve renormalization, all of these are currently equivocal. In the Introduction, we described the adaptor strength paradigm (Jeffery et al., 2010; Jeffery et al., 2011; McKone et al., 2014; Pond et al., 2013; C. Zhao et al., 2011), and explained why the results of this paradigm are inconclusive. We also described a paradigm previously proposed by us (Storrs & Arnold, 2012), in which participants classify a range of stimuli in a three-category classification task. When tested in this paradigm, the facial gender aftereffect matched the predictions of a locally repulsive aftereffect, as in the present report. Facial distortion aftereffects, however, better matched a renormalizing pattern (see below for further discussion). 

Two other paradigms designed to dissociate locally repulsive from renormalizing aftereffects deserve mention. In the first, identity aftereffects are compared after adapting and testing along facial identity trajectories that pass through the center of a hypothetical face space, as compared to facial identity trajectories that do not (Leopold et al., 2001; Rhodes & Jeffery, 2006). Aftereffects were found to be larger along the former than the latter type of trajectory, which was thought to demonstrate that the norm plays a special role in facial encoding. However, there is some recent evidence that both multichannel and norm-based models may predict this pattern of results (Ross, Deroche, & Palmeri, 2013). 

In a second paradigm, changes in a three-category classification task are compared after adapting either to a neutral stimulus value, or to alternating positive and negative stimulus values. For instance, participants might classify faces as “looking left,” “looking straight ahead,” or “looking right” before and after adapting either to faces that look straight ahead or to alternating faces that look left and right (Calder et al., 2008). A multichannel code can predict that after consistent neutral adaptation, the range of stimuli placed in the central category should narrow, and that after alternating negative and positive adaptation, the range of central stimuli should broaden. A simple opponent-channel scheme predicts no change in response pattern. Eye gaze direction (Calder et al., 2008) and head direction aftereffects (Lawson et al., 2011) both followed the pattern of changes predicted by a multichannel code when tested in this paradigm. Facial expression aftereffects, however, produced a narrowing of the central range in both conditions (Burton et al., in press), which was argued to be more consistent with an opponent than multichannel code. 

There are important caveats to conclusions based on data from the above paradigm. First, the predictions of an opponent-channel scheme are unclear, as additional assumptions must be made (e.g., that the response functions of adapted channels steepen or flatten) in order to predict a change in categorization decisions after either neutral or alternating adaptation (Burton et al., 2015; Calder et al., 2008; Lawson et al., 2011). Second, repeated exposures to an adapting stimulus might alter the range of stimuli placed in the middle category, not because such exposure has changed the appearance of test stimuli, but because participants adopt the repeatedly seen adaptors as exemplars against which other inputs can be judged when applying the middle category label. Both factors combine to dictate that any result from this paradigm cannot be regarded as conclusive evidence either of multichannel or opponent-channel encoding. 

Perhaps the strongest evidence for normalization in face aftereffects comes from experiments involving geometrically distorted images. Webster and MacLin (1999) had participants remember a distorted face and then adjust a test to match this remembered face after adapting to either a distorted or to an undistorted face. They found that adapting to an undistorted face had no effect on participants' matches to distorted faces. They inferred from this that the undistorted face played a special role in perception. In general support of this conclusion is the observation, given anecdotally by Webster and MacLin (1999) and quantified by us and others since (Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003; Robbins et al., 2007; Storrs & Arnold, 2012), that during adaptation a distorted face may come to appear more normal. We, however, found that this did not generalize to facial gender adaptation (Storrs & Arnold, 2012). One possible interpretation of this is that distortion adaptation does not tap facial mechanisms specifically, but instead is akin to adapting to a geometric distortion, such as during prism adaptation (Helmholtz, 1909; Redding, Rosetti, & Wallace, 2005). 

Caveats

While our data speak to the computational processes underlying face adaptation, they do not provide insight into where these computations might take place. It is worth noting that our spatial-comparison paradigm measures differences in perception between two differently adapted regions. It would therefore be insensitive to any adaptation that uniformly affected all locations in the visual field. Unfortunately psychophysics lacks methods to measure global biases that do not confound perceptual with decision-level effects (Morgan, Melmoth, & Solomon, 2013; Morgan, 2014; Storrs, 2015). On the basis of our data, we cannot therefore preclude the possibility that there is a global face adaptation effect that impacts all face perceptions equally, no matter where in the visual field a face might be presented. However, since facial identity and gender aftereffects as measured by a standard binary-classification task are retinotopically localized (A. Afraz & Cavanagh, 2009; S.-R. Afraz & Cavanagh, 2008), we believe our data speak to the same effects as reported in previous face adaptation studies, and that our data and these previous reports are driven by a common computational process. 

We would like to stress one final caveat in relation to our testing paradigm. Stimuli presented in the first test interval might have induced adaptation, affecting the appearance of stimuli presented in the second, as facial aftereffects can be induced by adapting for as little as 200 ms (Fang & He, 2005). This influence should, however, be smaller than that induced by our more prolonged adaptors, as facial identity aftereffects reportedly increase logarithmically with adaptation duration (Leopold, Rhodes, Muller, & Jeffery, 2005). Moreover, any interactions between sequential test stimuli would also have been present during baseline trials, and so would not systematically impact our aftereffect data. In the future, variance arising due to interactions between test stimuli could be mitigated by introducing a second adaptation period on each trial in between the two test intervals. 

Conclusion

Our data add to the growing body of evidence (Ross et al., 2013; Storrs & Arnold, 2012; C. Zhao et al., 2011) that an opponent-channel model predicting renormalizing aftereffects does not well describe the perceptual shifts following face adaptation. Instead, facial identity and gender aftereffects are better described as locally repulsive aftereffects, in which differences between adapting and test stimuli are exaggerated in all directions, with little or no change in the appearance of the adaptor. This is consistent with an encoding model in which multiple channels are tuned for preferred values, with no value having a special role. Face aftereffects therefore behave similarly to other figural aftereffects (Badcock et al., 2014; Blakemore et al., 1970; Köhler & Wallach, 1944; Mitchell & Muir, 1976), perhaps suggesting common processes underlie the encoding of spatial patterns throughout the visual hierarchy. 

Acknowledgments

This research was supported by an Australian Research Council Discovery project grant (DP0878140) to DHA. The authors have no competing financial interests. 

Commercial relationships: none. 

Corresponding author: Katherine R. Storrs. 

Email: [e-mail]. 

Address: School of Psychology, The University of Queensland, St. Lucia, Queensland, Australia. 

Footnotes

¹  Available from http://faces.cs.unibas.ch/bfm/

Footnotes

²  Available from http://homepages.inf.ed.ac.uk/jbednar/stimuli/