The timecourse of expression aftereffects

Abstract

Adaptation to facial expressions produces aftereffects that bias perception of subsequent expressions away from the adaptor. Studying the temporal dynamics of an aftereffect can help us to understand the neural processes that underlie perception, and how they change with experience. Little is known about the temporal dynamics of the expression aftereffect. We conducted two experiments to measure the timecourse of this aftereffect. In Experiment 1 we examined how the size of the aftereffect varies with changes in the duration of the adaptor and test stimuli. We found that the expression aftereffect follows the classic timecourse pattern of logarithmic build-up and exponential decay that has been demonstrated for many lower level aftereffects, as well as for facial identity and figural face aftereffects. This classic timecourse pattern suggests that the adaptive calibration mechanisms of facial expression are similar to those of lower level visual stimuli, and is consistent with a perceptual locus for the adaptation aftereffect. We also found that aftereffects could be generated by as little as 1 s of adaptation, and in some conditions lasted for as long as 3200 ms. We extended this last finding in Experiment 2, exploring the longevity of the expression aftereffect by adding a stimulus-free gap of varying duration between adaptation and test. We found that significant expression aftereffects were still present 32 s after adaptation. The persistence of the expression aftereffect suggests that they may have a considerable impact on day-to-day expression perception.

Introduction

A fascinating property of the visual system is that many aspects of visual perception are adaptive, such that the appearance of a given stimulus may be affected by what has been seen before. For instance, after viewing a waterfall for a period of time, the visual system adapts to that downward motion and a stationary surface viewed immediately afterwards will temporarily appear to be moving upwards (Barlow & Hill, 1963; Wohlgemuth, 1911). This perceptual bias is known as an aftereffect. 

Although many of the documented adaptation aftereffects concern lower level visual properties like motion, adaptation also occurs for higher level stimuli, such as faces (Leopold, O'Toole, Vetter, & Blanz, 2001). In the facial expression aftereffect, adaptation to a face with a particular expression will bias participants' judgments of subsequent faces towards the “opposite” expression: The expression with visual characteristics opposite those of the adaptor, relative to the central tendency of expressions (Burton, Jeffery, Skinner, Benton, & Rhodes, 2013; Skinner & Benton, 2010, 2012). For example, Figure 1 shows fear and its ‘opposite' expression, anti-fear. Where fear has raised eyebrows and an open mouth, anti-fear has lowered eyebrows and a closed mouth, and so on. Adapting to anti-fear produces a selective perceptual bias towards fear (Burton et al., 2013; Skinner & Benton, 2010, 2012). Adaptation aftereffects can also be observed for other facial attributes, including identity (Leopold et al., 2001), gender and race (Webster, Kaping, Mizokami, & Duhamel, 2004), and gaze direction (Jenkins, Beaver, & Calder, 2006). 

Figure 1

 

Fear (left) and anti-fear (right). The feature positions of these two expressions are opposite one another relative to the central tendency of facial expressions.

Aftereffects are particularly useful to researchers because they reflect changes in the responsiveness of the neurons that code the adapted property, and so can be used to reveal the nature of the coding mechanisms that underlie visual perception. In a recent review, Strobach and Carbon (2013) described three key dimensions of face aftereffect research that can provide information about different aspects of these coding mechanisms: the type of information that is adapted, the temporal dynamics of adaptation, and the transfer of the adaptation aftereffects between different kinds of stimuli. Here we focus on the temporal dynamics of adaptation. By studying the timecourse of an aftereffect, researchers can describe how quickly the visual system adapts to current stimuli, how the length of adaptation affects the strength of the aftereffect, and how long the adapted state persists. There has been some exploration of the timecourses of face-related adaptation (Carbon & Ditye, 2011; distortion of familiar faces: Carbon & Ditye, 2012; gaze direction: Kloth & Schweinberger, 2008; e.g., facial identity: Leopold, Rhodes, Müller, & Jeffery, 2005; Rhodes, Jeffery, Clifford, & Leopold, 2007; Strobach, Ditye, & Carbon, 2011). However, little is known about the timecourse of the expression aftereffect. 

Although several previous studies have investigated the timecourse of face aftereffects, these findings do not necessarily generalize to facial expression. Not all face attributes are coded by the same types of mechanisms: For instance, facial expression is opponent coded (Burton, Jeffery, Calder, & Rhodes, 2015; Burton et al., 2013; Cook, Matei, & Johnston, 2011; Skinner & Benton, 2010, 2012), while gaze direction is multichannel coded (Calder, Jenkins, Cassel, & Clifford, 2008). Different facial attributes also have different temporal properties: For instance, identity is a stable aspect of the face, while facial expressions are dynamic and can change many times per second. If adaptation aftereffects serve some functional purpose in face perception (e.g., calibrating perceptual resources) then the optimal adaptation timecourse for facial identity perception may not be optimal for expression perception. 

We conducted two experiments to measure the temporal dynamics of expression adaptation. Our first aim was to determine whether the expression aftereffect follows the “classic” timecourse pattern of logarithmic build-up and exponential decay found for lower level visual aftereffects such as tilt (Wolfe, 1984), motion (Sekuler, 1975), and shape (Krauskopf, 1954). Face aftereffects are different to lower level visual aftereffects in many respects. The stimuli themselves are more complex, and the aftereffects persist over changes in the size, position, and angle of stimuli between adaptation and test (Leopold et al., 2001; Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003; Watson & Clifford, 2003; Zhao & Chubb, 2001), and over the changes in lighting, etc., found between different images of the same individual (Carbon & Leder, 2005). However, this same classic timecourse pattern has been shown for both facial identity and figural face aftereffects (Leopold et al., 2005; Rhodes et al., 2007), and is not simply inherited from lower level adaptation because it remains even with a size change between adaptation and test (Rhodes et al., 2007). This classic timecourse pattern makes it unlikely that these face aftereffects could be explained simply by changes in participants' response strategies or demand characteristics of the adaptation task. The similarity in the timecourse of these face aftereffects and lower level perceptual aftereffects is consistent with the identity and figural face aftereffects being perceptual in nature. We aimed here to determine whether the same is true for expression aftereffects. 

In our first experiment, participants adapted to anti-expressions (faces with opposite visual characteristics to familiar “target” expressions) for varying amounts of time, and then viewed an ambiguous test expression for varying amounts of time. Adaptation should bias perception of the test face towards the opposing target expression. Participants then rated the strength of the expression they perceived on the test face. This design allowed us to measure how aftereffect strength varied with adaptation duration (the build-up of the aftereffect) and with test duration (the decay of the aftereffect). 

The design of Experiment 1 also allowed us to observe which of our conditions produced expression aftereffects—whether our shortest adaptation durations were long enough to result in an aftereffect, and whether the aftereffects lasted long enough to be measured at the end of our longest test durations. To foreshadow our results from this first experiment, we found that expression aftereffects were still present after our longest test duration (3200 ms). We extended this finding in Experiment 2 by measuring how long the expression aftereffect persists. An interesting feature of visual adaptation is that it can persist for a surprisingly long time. Following 15 min of adaptation, a motion aftereffect can in some conditions be observed 24 hr later. Fifteen minutes of adaptation can produce a McCollough effect that lasts as long as 3 months (Jones & Holding, 1975). Considerable longevity has also been found for some face aftereffects: For instance, adapting to a distorted famous face for 25 min can result in altered perception of that identity as much as a week later (Carbon & Ditye, 2011, 2012). A dynamic face stimulus like facial expression might be less likely to produce such long-lasting aftereffects. However, for gaze direction, another rapidly changing dynamic face cue, 84 s of adaptation can produce aftereffects that last for 7 min (Kloth & Schweinberger, 2008); certainly long enough to cover many changes in gaze direction during conversation. We were interested in the longevity of the expression aftereffects created by the relatively brief adaptation durations (up to 16 s) used in Experiment 1. These seconds-long adaptation durations are likely to be representative of the duration of expression adaptation that might be experienced in a naturalistic setting. 

Experiment 1

The aim of Experiment 1 was to map the effects of both adaptation and test durations on aftereffect size. In particular, we aimed to determine whether expression aftereffects show the classic timecourse found for both lower level visual aftereffects and facial identity and figural face aftereffects. On each trial, participants adapted to an anti-expression and then rated the strength of the opposing expression in an ambiguous test face. We used static expressions as adaptors and test faces, which allowed us to straightforwardly vary the duration of the adaptation and test exposures. We expected that, following the classic timecourse, aftereffect magnitude would increase logarithmically with increased adaptation time and decrease exponentially with increased test time. 

Method

Participants

Twelve adults (five male) participated. Mean age was 23.8 years, SD = 6.1 years. Participants were either first-year psychology students participating for course credit, or were compensated $15 for their travel costs. 

Stimuli

Stimuli were adapted from Skinner and Benton's (2010) gender-neutral expressive faces created from images of 50 Caucasian individuals (25 male and 25 female) posing various expressions. For each expression the 50 images were morphed together using Psychomorph (Tiddeman, Burt, & Perrett, 2001) to create a single prototypical image that captured the key characteristics of that expression while minimizing any individual idiosyncrasies. The test face was an average expression created by morphing together seven of these prototypical expressions (anger, disgust, fear, happiness, sadness, surprise, and neutral; Figure 2). This average expression has an ambiguous appearance that is readily affected by adaptation (Burton et al., 2013; Skinner & Benton, 2010, 2012). 

Figure 2

 

The average expression, created by morphing together seven expressions (anger, disgust, fear, happiness, sadness, surprise, and neutral) derived from 20 identities.

The adaptors were anti-expressions, each created by morphing along a trajectory that ran from one of the identity-neutral expressions, through the average expression and beyond it to a point that differed from the average to the same extent as the original expression. Six anti-expression adaptors (anti-anger, anti-disgust, anti-fear, anti-happiness, anti-surprise and anti-sadness) were created in this way (Figure 3). Points on each trajectory are labeled as percentages: 100% for the original expression, 0% for the average expression, and −100% for the anti-expression. During training, we used weaker versions of the original expressions taken from these trajectories (e.g., 50% fear, which lies halfway between the average expression and fear). 

Figure 3

 

Anti-expression adaptors.

Procedure

Participants were first familiarized with strong (90%) depictions of the six target expressions (anger, disgust, fear, happiness, sadness, and surprise) and then practiced identifying weaker versions of these expressions. Adaptation to an anti-expression biases perception towards the opposite target expression. However, the resulting impression tends to be a weak version of that target expression. Participants might have difficulty identifying these weak percepts, leading to an underestimation of the size of the aftereffect. By training participants to identify the target expressions we hoped to maximize their ability to categorize weak version expressions, which would improve our ability to measure small aftereffects such as those that might be created by very brief adaptation. The same procedures have been used when studying the timecourse of other face aftereffects (Leopold et al., 2005; Rhodes et al., 2007). 

Each expression was shown for 400 ms, after which participants identified the expression that they had just seen using marked keys. Stimuli subtended a visual angle of approximately 7.6° high by 10.9° wide. Participants identified the expressions in two stages, first at 90% strength and then at 50% strength. At each stage they were only able to move on once they had correctly identified a sequence of all six expressions twice consecutively. No explicit feedback was given, but participants were told that the task would repeat if they were incorrect. Participants required M = 3.8 (SD = 2.7) repetitions of the six-expression sequence to meet criterion for the 90% strength expressions, and M = 4.9 (SD = 4.7) repetitions to meet criterion for the 50% expressions. 

Participants then learned to rate the strength of expressions. Following the method of Leopold et al. (2005) and Rhodes et al. (2007), each rating trial began with a cue—the name of the expression that participants would be rating in that trial (e.g., “Fear”). Participants then saw a face for 400 ms. Finally, participants were asked “How strong was your impression of [Fear]?” which they answered on a 7-point scale from 1, No [Fear], to 7, Strong [Fear], using labeled keyboard keys. Participants rated each of the six target expressions at six strengths: 90%, 70%, 50%, 30%, 10%, and 0% (average). These expressions were presented in random order. Before making their ratings, participants saw a sequence of faces that demonstrated the range of variation in the set (19 images: 10%, 50%, and 90% versions of each expression, plus the average). Experimenters checked that participants used higher ratings for more intense expressions, and that participants attempted to use the full range of the scale. Participants repeated the rating training if these criteria were not met. Only one participant was required to repeat the rating training, and only repeated it once. 

The adaptation procedure followed Leopold et al. (2005) and Rhodes et al. (2007). Each trial began with a cue to the expression that participants would be rating, as above. Participants then viewed an anti-expression adaptor, followed by a test face (always the average expression), and rated the strength of their impression of the cued expression using the same scale that was used in the training task. Because impressions could be dynamic, participants were asked to rate their impression at the offset of the test face. 

Timings were as follows: a fixation cross (200 ms), then the expression cue (1000 ms), an adaptor (variable duration), an interstimulus interval (ISI) (150 ms), the test face (variable duration), and then the response screen with rating scale. A beep sounded 250 ms before the end of the adapting face to warn participants that the test face was about to appear. Adaptor and test stimuli were presented in the center of the screen. Adaptors subtended a visual angle of approximately 14.5° high by 10.2° wide when viewed from 50 cm. Test stimuli were presented at 75% of the size of adaptors to reduce the contribution of retinotopic adaptation (approximately 10.9° high by 7.6° wide). The fixation cross and cue word were centered at approximately the level of the eyes of the face stimuli. Following Leopold et al. (2005) and Rhodes et al. (2007), there were five adaptation durations (1000, 2000, 4000, 8000, and 16,000 ms) and five test durations (200, 400, 800, 1600, and 3200 ms). Each combination of adaptation and test duration was used in 12 trials, two with each of the six expressions (300 trials total). Trials were divided into 10 blocks of 30, containing an equal number of trials with each expression and each adaptation and test duration (not fully crossed). Trials were randomized within blocks. All trials were completed in a single session, with participant-timed breaks between blocks. The rating task began with 12 practice trials; in these trials the adaptor was the average expression (shown for 1000, 2000, or 4000 ms to introduce participants to the variable nature of the adaptor durations) and the test face was a 30% version of one of the six target expressions, shown once for 200 ms and once for 1600 ms. The entire session, including training and adaptation, took around 90 min to complete. 

Results and discussion

The strength of the aftereffect was measured as the strength of the impression of a target expression following adaptation to its anti-expression. For each participant an average aftereffect size was found for each combination of adapting duration and test duration, collapsed across the six target expressions (Figure 4). 

Figure 4

 

Mean size of expression aftereffects as a function of adaptation duration and test duration (N = 12). Size of aftereffect was measured as the strength of impression of the cued expression.

We began by examining what effect adaptation duration and test duration had on the size of the aftereffect. Ratings of expression intensity increased with increasing adapting duration (1000 ms: M = 3.55, SD = 0.89; 2000 ms: M = 3.72, SD = 0.98; 4000 ms: M = 3.82, SD = 0.99; 8000 ms: M = 3.99, SD = 1.03; 16,000 ms: M = 4.27, SD = 1.0) and decreased with increasing test duration (200 ms: M = 4.34, SD = 1.05; 400 ms: M = 3.96, SD = 0.98; 800 ms: M = 3.81, SD = 0.99; 1600 ms: M = 3.68, SD = 0.96; 3200 ms: M = 3.57, SD = 1.01). We used a two-way repeated-measures analysis of variance with a Greenhouse–Geisser correction to test these effects. There was a significant main effect of adaptation duration, F(1.478, 16.254) = 23.30, p < 0.001, η²_p = 0.679 and a significant main effect of test duration, F(1.487, 16.352) = 18.60, p < 0.001, η²_p = 0.628, with no significant interaction, F(6.527, 71.796) = 1.49, p = 0.189, η²_p = 0.119. These results confirm that both adaptation duration and test duration had significant effects on the size of the expression aftereffect. 

To examine whether the data showed the expected pattern of logarithmic build-up and exponential decay, we plotted the ratings at each adaptation duration (collapsed across test duration) and at each test duration (collapsed across adaptation duration) on semilog coordinates (Figure 5c, d; also shown on untransformed coordinates, Figure 5a, b). We used relative ratings, calculated by subtracting each participant's grand mean from their ratings (Leopold et al., 2005; Rhodes et al., 2007). This adjustment accounts for any overall biases in participants' responses (e.g., a tendency to rate all expressions lower on the scale than other participants), allowing us to more clearly see any patterns across participants' responses. If the data follow the expected pattern the points should form straight lines when plotted on semilog coordinates. Straight line fits to the group data (Figure 5c, d) were excellent, with R² = 0.97 for the adaptation duration function (slope = 0.57) and R² = 0.92 for the test duration function (slope = −0.61). Wald–Wolfowitz runs tests indicated no significant nonlinearities (ps > 0.50). These results confirm that expression aftereffects follow the classic timecourse pattern of logarithmic build-up and exponential decay found for lower level visual aftereffects. This pattern is also consistent with the timecourses found for other face aftereffects (Leopold et al., 2005; Rhodes et al., 2007). 

Figure 5

 

Relative ratings (M ± SE) as a function of adapt and test duration. Relative ratings calculated by subtracting each participant's grand mean from their ratings. (a) Relative ratings as a function of adapting time. (b) Relative ratings as a function of test duration. (c) Relative ratings plotted on semilog coordinates, as a function of adapting time, with line of best fit. (d) Relative ratings plotted on semilog coordinates, as a function of test duration, with line of best fit.

To further examine the shape of these build-up and decay functions, we fit functions to individual participants' data, and compared the fit of the log-transformed functions described above to linear functions fit on untransformed axes. This analysis allowed us to both examine whether the functions found above fit well at the individual level, and to compare their goodness of fit to an alternative, linear function. We found each participant's mean rating at each adaptation duration (collapsed across test duration) and at each test duration (collapsed across adaptation duration). For each participant we fit a linear trend to these mean ratings on semilog coordinates. We also fit a linear trend to the data on untransformed coordinates. The mean R² for each type of fit is given in Table 1. For test duration, R² values were significantly higher for the functions fit on log-transformed axes than for the functions fit on untransformed axes, t(11) = 3.33, p = 0.007, indicating that the pattern of the data is better described by an exponential function. For adaptation duration there was no significant difference in R² values between the two types of fit. Examination of Table 1 suggests that this lack of a significant difference is driven by a strong linear trend in the data (high R² for the untransformed fit), rather than a poor fit for the logarithmic function. Overall we have evidence consistent with the classic timecourse of logarithmic build-up and exponential decay for the expression aftereffect in the group data, with a significantly better fit for an exponential decay function than a linear decay function at the individual level. 

Table 1

 

Means and standard deviations of R² values for functions fit to individual participants' ratings of expression intensity at each adaptation duration and each test duration. Notes: Functions were linear trends fit on either a log-transformed or untransformed x-axis.

 

A secondary aim was to determine which adaptation durations produced significant aftereffects, and whether aftereffects were still present after our longest test durations. Single-sample t tests confirmed that ratings were significantly higher than zero in all conditions (all ps < 0.001). However, it is unlikely that expression ratings would fall to zero in the absence of an aftereffect because the average expression contains each of the target expressions and may resemble each of them to some extent. We can however compare the size of ratings between conditions—if ratings are significantly higher in some conditions than others, we can infer that adaptation resulted in an aftereffect that elevated those ratings. 

Inspection of Figure 4 suggests that aftereffects were produced after as little as 1 s of adaptation. At the shortest test duration (Adapt1Test200), ratings were higher than at the longest test duration (Adapt1Test3200), suggesting that an aftereffect was present at the offset of the 200-ms test face that had decayed by the offset of the 3200-ms test face. A paired samples t test confirmed that ratings were significantly higher for Adapt1Test200 than for Adapt1Test3200, t(11) = 4.14, p = 0.002, d = 0.63 (Bonferroni-corrected α= .01). This result indicates that the expression aftereffect can be generated by as little as 1 s of adaptation, at least when the test duration is brief. 

It also appears that aftereffects remained for as long as 3200 ms of test exposure, at least in the longer adaptation conditions. To determine what length of adaptation was required to produce a significant aftereffect that remained after 3200 ms of test exposure, we used paired samples t tests to compare ratings of expression intensity at Adapt1Test3200 (the shortest adaptation duration, when aftereffects should be smallest) to ratings at Adapt2Test3200, Adapt4Test3200, Adapt8Test3200, and Adapt16Test3200. There was a significant aftereffect after 16 s of adaptation, t(11) = −5.03, p < 0.001, d = 0.54, and after 8 s of adaptation, t(11) = -3.12, p = 0.010, d = 0.40. There was no significant aftereffect after 4 s of adaptation, t(11) = −2.33, p = 0.040, d = 0.29, or 2 s of adaptation, t(11) = -1.23, p = 0.245, d = 0.16 (with Bonferroni-corrected α= .01). These results show that adaptation of a sufficiently long duration (at most 8 s, and possibly 4 s) can produce an aftereffect that remains after 3200 s of exposure to the test face. 

Experiment 2

In Experiment 1 we saw significant aftereffects that lasted for at least 3200 ms. Here we tested how much longer the aftereffect might persist. We modified the method of Experiment 1, using only the 200-ms test duration and inserting a stimulus-free gap of varying duration between adaptation and test. In Experiment 1, the duration of the test face varied, and participants rated the expression they saw immediately before offset of the test face. In this experiment, we used a fixed test duration and a gap between adaptation and test to more explicitly test the persistence of the aftereffect. We were concerned that if we just extended the duration of the test face, as in Experiment 1, participants might anchor their ratings to the stronger aftereffects experienced earlier in the test exposure (see Figure 4), causing us to overestimate the persistence of the aftereffect. We chose an adaptation duration of 16 s because it produced the largest aftereffects in Experiment 1. We also included an 8 s adaptation condition. This duration also produced large aftereffects in Experiment 1 (including a significant aftereffect after 3200 ms of exposure to the test face). This shorter adaptation duration might more closely resemble the durations of expressions seen in typical interactions than the longer 16-s exposures. For instance, spontaneous expressions elicited by film clips have been found to range from an average duration of 3 s (Frank, Ekman, & Friesen, 1993) to 13 s (Pfister, Li, Zhao, & Pietikainen, 2011). Adaptation-test gap durations were chosen from both within and beyond the range of test periods from Experiment 1 (500, 1000, 4000, and 32,000 ms). 

In Experiment 1, adaptation durations varied sufficiently to hold participant attention throughout the adaptation period (as the test stimulus could appear at any time). In this experiment both adaptation durations were relatively long. Therefore, to maintain participant attention during adaptation, we included a change detection task in which participants identified changes in the brightness of the eyes or lips of the adaptors (Burton et al., 2013). In this experiment we also included a baseline rating measure for each expression in the absence of adaptation. As discussed above, even if no adaptation occurs, participants are unlikely to rate the intensity of any given target expression as zero, because the test face is made up of a combination of the target expressions. Comparison of postadaptation ratings to baseline ratings provides an explicit test for the aftereffect. 

Method

Participants

Twenty-three adults (11 male) participated in the study (24 participants were tested, but one was excluded from analysis; see Results). Mean age was 19.6 years, SD = 2.7 years. Participants were first-year psychology students participating for course credit. Our sample was larger than in Experiment 1 because we expected that aftereffects would be reduced given the longer delays between adaptation and test images. 

Stimuli

Stimuli were the same as those used in Experiment 1. For the change detection task that took place during adaptation, versions of the adaptors were created which had slightly brightened irises or lips. These were made by overlaying a white mask onto these features at 10% opacity (Figure 6). 

Figure 6

 

Brightness changes used to maintain participant attention during adaptation: brightened irises (left) and lips (right).

Procedure

Participants began with the same training tasks as in Experiment 1. Next, we found each participant's baseline expression intensity rating for the average test face, for each expression. Participants saw a fixation cross (200 ms), then an expression cue (1000 ms), the test face (always the average expression, 200 ms), then the response screen with rating scale. Participants rated the perceived intensity of each of the six target expressions 10 times, giving 60 trials total. 

Each trial of the adaptation task began with a cue to the expression that participants would be rating, as in Experiment 1. Participants then viewed an anti-expression adaptor. There was then a delay period, followed by a test face (always the average expression). Participants rated the strength of their impression of the cued expression. 

Timings were as follows: a fixation cross (200 ms), then the expression cue 1000 ms), an adaptor (either 8000 or 16,000 ms), a delay period (500, 1000, 4000, or 32,000 ms), the test face (200 ms) and then the response screen with rating scale. A 200-ms beep sounded 400 ms before the end of the delay period to warn participants that the test face was about to appear. Participants were given no special instructions about what to do during the delay. The fixation cross and cue word were centered at approximately the level of the eyes of the face stimuli. To ensure that they were not exposed to any other faces during this time, testing was conducted in an otherwise empty room, with the experimenter seated out of view behind the participant. An anti-glare film was applied to the monitor to prevent participants from seeing their own reflection. 

To maintain attention during adaptation, participants were asked to detect temporary changes in the brightness of the irises or lips of the adaptors. To facilitate this change detection task, adaptors were shown over repeated 1000-ms exposures, separated by 150-ms ISIs. During each adaptation period, one of these 1000-ms exposures included either an eye or a lip change. Participants responded using marked keyboard keys as soon as they saw a change. 

Each combination of target expression, adaptation duration and delay duration (six expressions by two adaptation durations by four delay durations = 48 combinations) was used in six trials, for a total of 288 trials. Trials were presented in two sessions, each containing an identical set of 144 trials. Trials were presented in a pseudorandom order, in which each run of six trials must contain each of the six expressions, and no two consecutive trials could contain the same expression. Participants took breaks every 24 trials. 

The baseline block was repeated at the end of each testing session to confirm that baseline ratings had not shifted during the session. In total, each session took approximately 90 min for a total of 3 hr of testing. 

Results and discussion

We examined participants' eye and lip change performance during adaptation to check that they had been attending during the task. One participant (from the original sample of 24) was removed from analysis because of poor change detection performance in Session 2 (27% eye changes and 14% lip changes correctly detected). The remaining participants correctly detected a mean of 92.8% of eye changes (SD = 7.7%) and 87.0% of lip changes (SD = 12.3%) in Session 1, and 90.4% of eye changes (SD = 10.3%) and 88.5% of lip changes (SD = 11.9%) in Session 2. 

There was no significant difference between baseline ratings at the start and end of the adaptation task, or between sessions (see Supplementary Materials). We therefore took an average of the ratings from all baseline blocks for each subject to give the final baseline rating for each expression. 

To calculate each participant's aftereffects, we calculated their average rating for each combination of expression, adaptation duration, and gap duration across the two sessions, and subtracted their average baseline rating for that expression. We then took an average across all expressions to calculate each participant's aftereffect for each combination of adaptation duration and gap duration (Figure 7). 

Figure 7

 

Mean aftereffect size (postadaptation ratings minus baseline ratings) for each adaptation and gap duration, ± 1 SEM.

To determine whether all conditions produced significant aftereffects, we compared each aftereffect score to zero using one-sample t tests. All aftereffects were significantly larger than zero (Table 2). 

Table 2

 

One-sample t tests comparing aftereffect score to zero in each condition.

 

We tested the effects of adaptation duration and gap duration on aftereffect size using a within-subjects analysis of variance. We found a significant main effect of adaptation duration, F (1, 22) = 49.87, p < 0.001, η²_p = 0.694, a significant main effect of gap duration, F(3, 66) = 27.75, p < 0.001, η²_p = 0.558, and no significant interaction, F(3, 66) = 1.94, p = 0.131, η²_p = 0.081. As expected, aftereffects were larger for the 16000-ms adaptation than for the 8000-ms adaptation, and decreased in size as gap duration increased. However, as noted above, aftereffects were still present after an adaptation-test gap of 32000 ms for both 16000 and 8000 ms of adaptation. This finding shows that expression aftereffects that could reasonably be generated during typical social interactions can persist for a surprisingly long time, given the fast-changing nature of expressions. 

General discussion

In Experiment 1, we found that the strength of the expression aftereffect depended on both adaptation duration and test duration. The aftereffect decayed exponentially with increasing test duration, following the classic timecourse pattern found for lower level visual aftereffects. We also found evidence consistent with the classic logarithmic build-up of the aftereffect. Our results further support a shared timecourse for face aftereffects and lower level perceptual aftereffects (Leopold et al., 2005; Rhodes et al., 2007), and extend the pattern to a new face attribute, namely expression. 

Face aftereffects are often used to examine the visual representation of faces, with the assumption that these aftereffects tap the mechanisms of visual perception in the same way as lower level visual aftereffects. It is therefore important to consider whether face aftereffects are indeed perceptual in origin. The similarity between the timecourse of the expression aftereffect and the timecourses of lower level perceptual aftereffects is consistent with a perceptual locus for the expression aftereffect. Can we rule out the possibility of a postperceptual locus (some kind of contrastive version of semantic priming, for instance)? There is certainly evidence that some cognitive effects can be time dependent (e.g., semantic priming: Becker, Moscovitch, Berhmann, & Joordens, 1997; Lee, Rayner, & Pollatsek, 1999). However, these effects differ in many ways from face aftereffects (e.g., priming generally produces assimilative rather than contrastive biases) and there is no evidence that they show the classic timecourse observed here. Therefore, it seems more reasonable to interpret our results as evidence for a perceptual locus. 

It also seems unlikely that the expression aftereffect observed here reflects a decisional bias. An expression aftereffect might potentially be explained by participants implementing a conscious or unconscious strategy to respond with the expression opposite the adaptor. It has been claimed that this sort of bias might also follow a logarithmic timecourse (Storrs, 2015), but there is no current evidence that a decision bias would be affected by stimulus duration in this way, and it is not clear how this might occur. Importantly, there is no obvious semantic relationship between the adaptors and the related target expressions in this study: The anti-expressions are not familiar expressions, and there is no clear semantic opposition between an expression and its anti-expression (e.g., anti-sadness does not appear happy; see Figure 3). It therefore seems more likely that the aftereffects observed here reflect changes in the way that the test stimuli appeared to participants, rather than changes in response strategy. 

Anecdotal evidence also suggests that the expression aftereffect affects the appearance of test stimuli. Following pilot testing and informal demonstrations of the effect, several of our colleagues have reported experiencing the test stimulus visibly changing as the aftereffect decays. During the debriefings for the present study, several participants expressed surprise when told that the test face was always the same image, and described seeing the test faces as varying in expression. These phenomena suggest that the measured aftereffect reflects a change in the way that expressions are experienced following adaptation. 

In Experiment 1 we found significant expression aftereffects after just 1 s of adaptation. Only one previous study has shown expression aftereffects with such a brief adaptation duration (Xu, Liu, Dayan, & Qian, 2012), and the stimuli in that study were simple schematic faces that varied only in mouth curvature, unlike the naturalistic face adaptors used in the present experiment. We also found that the expression aftereffect was still present after 3200 ms of continuous exposure to the test face, at least for the longer adaptation durations (8000 and 16,000 ms, possibly 4000 ms). In Experiment 2 we found that expression aftereffects can persist over long gaps between adaptation and test (at least 32 s in duration), indicating that these aftereffects could potentially be retained over breaks in a social interaction. It is even possible that that an expression aftereffect produced during one interaction could affect subsequent interactions with other individuals, especially given that expression aftereffects show (partial) transfer across identity (Skinner & Benton, 2012). We did not include any adaptation-test gaps longer than 32 s, so we cannot be sure how much longer the aftereffect might persist. By fitting exponential functions to the data from Experiment 2, we can estimate that extinction might occur at somewhere around 9.6 hr following 8 s of adaptation and 10 hr following 16 s of adaptation. However, these estimates are extrapolations based on very few data points, and should be used only as a rough guide for a future systematic examination of the extinction of the aftereffect. 

Given the long duration of the aftereffects found in Experiment 2, the expression aftereffect could also potentially persist over several changes of expression. However, we cannot be sure exactly how long the aftereffect would last when other faces or expressions appear after adaptation. In Experiment 2, we used a stimulus-free gap to vary the duration between adaptation and test. Kiani, Davies-Thompson, and Barton (2014) found that facial identity aftereffects decay faster when other faces are seen between the adaptor and the test face. If the same is true for expression, the aftereffect may not last for as long as we have observed here if faces are present between adaptation and test. Given that there is both an identity-specific and identity-independent component of the expression aftereffect (Skinner & Benton, 2012), another interesting question is how the duration of the aftereffect is affected by intervening faces of the same identity as the adaptor (as in a one-on-one conversation) as compared to intervening faces of a different identity to the adaptor (as in a conversation with multiple individuals). 

Liberman, Fischer, and Whitney (2014) recently reported another effect of recent perceptual experience on the perception of faces. They found a serial dependence effect for facial identity, such that the perceived identity of a face is pulled towards the identity of faces seen in the several seconds previous. This effect operates in the opposite direction to the repulsive adaptation aftereffects reported here. Interestingly, this attractive effect of facial identity could be induced by viewing previous faces for as long as 1 s. Liberman et al. suggested that serial dependence helps to maintain visual stability that contributes to our experience of the continuity of objects. Repulsive aftereffects, on the other hand, may improve discrimination of subtle changes from a persistent state. We should therefore expect that serial dependence would occur over shorter adaptation durations, giving way to repulsive aftereffects at longer adaptation durations. However, we found here that 1 s of viewing a naturalistic facial expression is enough to produce a repulsive aftereffect, and Xu et al. (2012) reported a repulsive aftereffect from only 35 ms of adaptation to a schematic expression. Given the dynamic nature of facial expressions compared to the more stable facial identity, it would be interesting to determine whether serial dependence operates at all for facial expressions, and if so, whether the continuity field—that is, the length of time over which expressions have an attractive effect on the perception of subsequent expressions—is shorter for facial expressions than for facial identity. 

We measured expression aftereffects here using a rating of perceived expression intensity, following Leopold et al. (2005) and Rhodes et al. (2007). In contrast, previous expression aftereffect studies have generally used a forced-choice expression-labeling task, and measured the aftereffect as a change in response thresholds or increase in proportion of responses opposite the adaptor (e.g., Burton et al., 2013; Fox & Barton, 2007; Hsu & Young, 2004; Pell & Richards, 2011; Skinner & Benton, 2010). The benefit of the present rating method is that participants are making a judgment about the intensity of a single expression, rather than having to make a choice between several expressions, some of which may be readily confused with one another (e.g., fear and surprise; Gao & Maurer, 2010). Moreover, as expression intensity increases from very weak intensities, participants are able to identify that an expression is present before they are able to accurately identify what that expression is (X. Gao, personal communication, July 4, 2015). The forced choice method may therefore underestimate weak aftereffects, as participants may not be able to correctly identify what is nevertheless a change in their perception of the test face. The rating method may therefore be more useful in designs such as this one where very brief adaptation durations lead to weak aftereffects. 

In conclusion, we have demonstrated that expression aftereffects follow the classic timecourse pattern seen for other face aftereffects and in lower level vision. This finding provides further evidence of similarities in adaptive perceptual mechanisms between higher and lower level vision, and is consistent with a perceptual locus for the expression aftereffect. We were able to produce expression aftereffects after only 1 s of adaptation, and longer adaptation periods produced aftereffects that were still present after more than 3 s of exposure to the test stimulus, and that remained after a gap between adaptation and test faces of more than 30 s. These findings suggest that expression aftereffects have the potential to affect our day-to-day social experiences. 

Acknowledgments

This research was supported by the Australian Research Council (ARC) Centre of Excellence in Cognition and its Disorders (CE110001021), an ARC Professorial Fellowship to Rhodes (DP0877379), and an ARC Discovery Outstanding Researcher Award to Rhodes (DP130102300). We thank Andrew Skinner and Chris Benton for providing the stimuli. Ethical approval was granted by the Human Research Ethics Committee of the University of Western Australia. 

Commercial relationships: none. 

Corresponding author: Nichola Burton. 

Email: [e-mail]. 

Address: School of Psychology, The University of Western Australia, WA, Australia.