Is the early modulation of brain activity by fearful facial expressions primarily mediated by coarse low spatial frequency information?

Abstract

Rapidly decoding the emotional content of a face is an important skill for successful social behavior. Several Event Related brain Potential (ERP) have indicated that emotional expressions already influence brain activity as early as 100 ms. Some studies hypothesized that this early brain response to fear depends on coarse-magnocellular inputs, which are primarily driven by Low Spatial Frequency (LSF) cues. Until now however, evidence is inconclusive probably due to the divergent methods used to match luminance and contrast across spatial frequencies and emotional stimuli. In the present study, we measured ERPs to LSF and HSF faces with fearful or neutral expressions when contrast and luminance was matched across SF or not. Our findings clearly show that fearful facial expressions increases the amplitude of P1 (only for contrast–luminance equated images) and N170 in comparison to neutral faces but only in LSF faces, irrespective of contrast or luminance equalization, further suggesting that LSF information plays a crucial role in the early brain responses to fear. Furthermore, we found that, irrespective of luminance or contrast equalization, N170 occurred faster when perceiving LSF faces than HSF faces, again emphasizing the primacy of LSF processing in early face perception.

Introduction

Rapidly decoding the emotional content of a face is an important skill for successful social behavior, since it helps to evaluate the states and intentions of others and to adapt future behavior. The time course of emotional face processing has been explored in several EEG and MEG studies. These studies report that particularly negative expressions affect the amplitude and/or latency of various early ERP and MEG components related to face processing. The two most prominently studied components are P1 (e.g., Ashley, Vuilleumier, & Swick, 2004; Batty & Taylor, 2003; Pizzagalli, Regard, & Lehmann, 1999; Pizzagalli et al., 2002) and N170 (e.g., Batty & Taylor, 2003; Campanella, Quinet, Bruyer, Crommelinck, & Guerit, 2002; Righart & de Gelder, 2005; Stekelenburg & de Gelder, 2004). P1 is a fast exogenous response to visual stimulation, which reflects striate as well as extrastriate visual processing (Gomez Gonzales, Clark, Luck, Fan, & Hillyard, 1994; Heinze et al., 1994; Rossion et al., 1999). It occurs about 100 ms after stimulus onset and is located over lateral occipital regions of the scalp. N170 occurs at around 170 ms and is maximally recorded at occipitotemporal electrodes (Bentin, Allison, Puce, Perez, & McCarthy, 1996). N170 is the earliest ERP component to consistently show larger amplitude for faces than other non-face object categories (e.g., Jacques & Rossion, 2004). N170 originates from a network of regions, probably including the fusiform gyrus, inferior occipital cortex, superior temporal sulcus, and the inferior, middle, and superior temporal gyri (Henson et al., 2003). N170 reflects not only the detection of a face but also the encoding of the structure or configuration of the face, based on which individual faces can be discriminated from each other (Jacques & Rossion, 2006). 

A faster P1 for fearful expressions as compared to neutral expressions has been noted in several studies and is consistent with the idea that negative emotions tend to capture attention in an involuntary reflexive manner and, as a consequence, tend to be processed faster (Eimer & Holmes, 2002; Lidell, Williams, Rathjen, Shevrin, & Gordon, 2004; Pourtois, Dan, Grandjean, Sander, & Vuilleumier, 2005; Pourtois, Grandjean, Sander, & Vuilleumier, 2004; Williams et al., 2004). Other studies report effects of emotional expressions only somewhat later, at the level of the face-specific N170 (Batty & Taylor, 2003; Blau, Maurer, Tottenham, & McCandliss, 2007; Campanella et al., 2002; Stekelenburg & de Gelder, 2004). 

Overall, the above-mentioned ERP studies underline the rapid processing of emotional expressions. However, facial expressions are complex stimuli and it is still not clear what information the visual system extracts in order to decode emotional expressions at such an early stage. Any input to the visual system consists of luminance variations occurring at various frequencies across space (e.g., De Valois & De Valois, 1988; Goldstein, 1999). Low spatial frequencies (LSF) of an image capture large-scale luminance variations (i.e., coarse information) whereas high spatial frequencies (HSF) represent small-scale luminance variations of the image (i.e., fine information; De Valois & De Valois, 1988; Goldstein, 1999). The spatial frequency content of a stimulus is generally expressed in cycles per degree of visual angle (c/deg). The present experiment addresses the visual input properties of facial expression processing by means of spatial frequency filtering. 

Two fMRI studies have now provided evidence for the importance of LSF information in emotional expression processing. One study showed that LSF information in a face is crucial to produce an increase in activation to fearful relative to neutral faces in the amygdala (Vuilleumier, Armony, Driver, & Dolan, 2003), a key subcortical structure in emotional processing (Morris, Ohman, & Dolan, 1999; Whalen et al., 1998). In contrast, high spatial-frequency (HSF) information in faces did not evoke a differential response to fearful compared to neutral expressions in the amygdala. A similar pattern was found in the fusiform cortex (Vuilleumier et al., 2003; Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004; Winston, Vuilleumier, & Dolan, 2003). Given the modulatory role of the amygdala (Morris et al., 1998; Rotshtein, Malach, Hadar, Graif, & Hendler, 2001), the hypothesis was raised that the enhanced processing of fear in the visual areas (including fusiform gyrus) is primarily mediated by rapid LSF cues, possibly via feedback from the amygdala, which gets input from a rapid magnocellular tecto-pulvinar pathway, preferentially tuned to LSF (Pourtois et al., 2005; Vuilleumier et al., 2003; Winston et al., 2003). However, since fMRI has a low temporal resolution, the observed activation patterns may alternatively mirror SF differences occurring at late decisional stages rather than at the early visual analyses. 

In contrast to fMRI, the technique of ERP has a high temporal resolution and is therefore well suited to give insight into the stages at which LSF and HSF inputs become important for facial expression processing in visual brain areas. However, only two ERP studies investigated the respective contribution of LSF and HSF to early facial expression processing (Holmes, Winston, & Eimer, 2005; Pourtois et al., 2005). An increased P1 for LSF fearful expressions relative to LSF neutral expression was found at occipitotemporal electrodes by Pourtois et al. (2005). In the other study, by Holmes, Winston et al. (2005), the occipitotemporal P1 was not analyzed. At the level of the N170, neither study found an interaction between spatial frequency and emotional expression. 

However, methodological aspects may account for the reported absence of SF influence on the early processing of facial expressions. A problem is that, when investigating SF processing, outputs of LSF and HSF filtering differ not only at the level of the spatial scale of information they convey but also in terms of luminance and contrast. This is related to the fact that the frequency power in natural stimuli is maximal at low SF and almost exponentially decays at higher SF (see for review Loftus & Harley, 2005). In the study of Pourtois et al. (2005), the overall difference in luminance and contrast between LSF and HSF images was matched by using hybrid stimuli. Pourtois et al. (2005) combined the LSF content of a given face (shown upright) with the HSF content of the same face presented upside down, or vice versa. However, since contrast/luminance was not equated between LSF and HSF components within a given hybrid, superimposing an inverted LSF image on an HSF image may have disrupted perception of the expression carried by HSF more strongly, than superimposing an inverted HSF image on an LSF image (see Pourtois et al., 2005, Figure 1). This might have prevented the detection of an effect of emotional expression at the level of P1 and/or N170 for HSF faces. 

The aim of the present study was to further explore the contribution of LSF and HSF to early emotional processing of faces, as reflected by P1 and N170 ERP components. To evaluate whether SF differences previously reported for emotion processing can be accounted by contrast and luminance, we directly investigated emotional processing when luminance and contrast were equated or not across LSF and HSF, within one study. 

Differential sensitivity to HSF and LSF contents of emotional expression is also apparent in some behavioral tasks. Whereas HSF seem to be relevant for the elaborate rating of emotional expressiveness as well as emotion discrimination (Schyns & Oliva, 1999; Vuilleumier et al., 2003; but see Goren & Wilson (2006) for different effects using synthetic faces), LSFs are important for rapid attentional responses to fear (Holmes, Green, & Vuilleumier, 2005) as well as rapid categorization of happiness, sadness, and anger (Schyns & Oliva, 1999), although the latter has not been investigated for fear. Complementing Experiment 1, we will investigate in Experiment 2 whether an LSF advantage for the processing of fear is reflected in reaction times (RTs) when subjects have to rapidly categorize neutral and fearful faces. Because RTs reflect the final stage of information processing, at which information about the facial expression of a face might be available in both LSF and HSF, RT may reveal emotion effects for HSF faces as well. We also evaluated whether differential effects in SF and emotion in RT are related to differences in luminance and contrast between the images. 

Experiment 1

Methods

Participants

Twenty students (10 females and 10 males; mean age: 21.7 ( SD= 2.7)) from Maastricht University participated in this study. Four participants were left-handed, the others were right-handed. All participants had normal or corrected-to-normal vision. The experiment was approved by the ethical committee and participants gave written informed consent before participation. 

Stimuli

Face stimuli consisted of 16 grayscale images (8 males; 8 females), one half depicting a neutral expression, the other half depicting a fearful expression. Different identities displayed different emotions. The photographs were taken from the NimStim Face Set (http://www.macbrain.org/faces/index.htm, Tottenham, Borscheid, Ellertsen, Marcus, & Nelson, 2002) and have shown to evoke emotional effects at the level of N170 (Blau et al., 2007). Face images included European-American and African-American models. Face pictures were trimmed to remove external features (neck, ears, and hairline). All pictures were fitted in a gray frame of 500 × 700 pixels. Each face subtended 6.3 degrees of visual angle (at a 113-cm viewing distance). The HSF images were created by filtering the original photographs, using a high-pass cutoff that was ≥6 cycles/deg of visual angle (≥36 cycles per object). The LSF images were created using a low-pass filter that was ≤2 cycles/deg of visual angle (≤12 cycles per object). These cutoffs (≤2 cycles/deg; ≥6 cycles/deg) were based on previous literature (e.g., Boeschoten, Kenemans, van Engeland, & Kemner, 2007; Costen, Parker, & Craw, 1994; Deruelle & Fagot, 2005; Deruelle, Rondan, Gepner, & Tardif, 2004; Deruelle, Rondan, Salle-Collemiche, Bastard-Rosset, & Da Fonséca, 2008; Goffaux, Gauthier, & Rossion, 2003; Schyns & Oliva, 1999). Filtering was performed in Matlab (The Mathworks, Natick, MA) using a set of Gaussian filters. After the filtering, HSF and LSF stimuli largely differed in terms of luminance and Root Mean Square (RMS) contrast (LSF: Mean luminance: 147; RMS contrast: 40; HSF: Mean Luminance: 131; RMS Contrast: 12.5). RMS contrast has been shown to be the best index for perceived contrast in natural images (Bex & Makous, 2002). Finally, global contrast and luminance were equated across scales by assigning both HSF and LSF images the mean luminance and RMS contrast of the 16 original broadband photographs (Mean Luminance: 141; RMS contrast: 29, see Figure 1). Since original contrast and luminance of LSF filtered faces were naturally close to full spectrum values, the equalization of these parameters to full spectrum values mostly affected HSF faces. 

Figure 1

 

Example of stimuli used in the Non-EQ task in which HSF and LSF images differ in luminance and contrast, and the EQ task in which HSF and LSF stimuli were matched on contrast and luminance.

Procedure

Participants were seated in a dimly and quiet room and were presented with two series (in a counterbalanced order) of 4 experimental blocks of 79 trials each. In one series, HSF and LSF faces were equated for luminance and contrast; in the other series, luminance and contrast differed between HSF and LSF images. Each block contained 32 neutral face trials (HSF and LSF) and 32 fearful face trials (HSF and LSF), each presented for 500 ms, in randomized order with an interstimulus interval that varied randomly between 1600 and 1800 ms. To maintain attention to the task, each block contained 15 animation figures (for example Disney figures), each with a duration of 2 s. Participants had to press a response button as soon as they saw an animation figure on the screen and had to refrain from responding to all other images. Animation figures were included because we want to conduct this experiment with children in the future. 

ERP recording and data analysis

During task performance, the EEG (0.1–200 Hz, sampling rate of 500 Hz) was recorded with a 31-channel Quickcap (Neuromedical supplies of Neurosoft) covering frontal, central, temporal, and parietal scalp areas. An electrode attached at the left mastoid served as a reference. Afz was used as ground electrode. Blink and vertical eye movements were monitored with electrodes placed at the sub- and supra-orbital ridge of the right eye. Lateral eye movements were monitored with two electrodes placed on the right and left external canthi. All electrode impedances (EEG and EOG) were kept below 10 kΩ. 

The EEG data were analyzed offline using “Vision Analyser” software (Brain Products, Munich, Germany). A common average reference was recomputed for all electrodes. EEG epochs were extracted beginning 200 ms before and ending 400 ms after each stimulus . The 200 ms prior to stimulus onset was used as baseline. The epochs were band-pass filtered with a 30 Hz, 24 dB/octave low pass filter. Artifacts from vertical eye movements and blinks were reduced with the algorithm of Gratton, Coles, and Donchin (1983). Thereafter, all epochs containing artifacts, amplitudes larger than 75 μV, were removed. Separate ERP averages were computed for the four stimulus conditions (SF (HSF/LSF) × Emotion (Fear/Neutral)). For each condition, P1 and N170 latencies and amplitudes were automatically extracted at peak-maximum occipitotemporal electrodes PO7/PO8 and P7/P8 (time windows: P1: 70–140 ms; N170: 100–200 ms). Both P1 and N170 were largest at these electrodes. All peaks were confirmed by visual inspection. Latency and amplitude values were subjected to a repeated-measure ANOVA with SF (HSF versus LSF), Emotion (Fear versus Neutral), Equalization (luminance/contrast equalized, i.e., EQ versus luminance/contrast non-equalized, i.e., Non-EQ), Electrode Position (posterior versus posterior occipital), and Hemisphere (Left versus Right) as within-subject factors. Based on a priori hypothesis conditions were further compared using paired t-tests. 

Results

Figure 2 shows the grand averages for neutral and fearful HSF and LSF faces at PO7/PO8 and P7/P8, separately for EQ and Non-EQ stimuli. 

Figure 2

 

Grand averages for HSF (black) and LSF (gray) fearful (dashed) and neutral (continuous) faces at channel P7/P8 and PO7/PO8. Fearful faces elicit larger amplitudes than neutral faces in the LSF condition only. Furthermore, the topographical distribution of the difference (Neutral minus Fear (160–220 ms)) in ERP activity between fearful and neutral faces is shown for N170. Note that the difference covers the ventro-temporal areas.

P1 amplitude

Statistical analysis of P1 amplitudes revealed a significant interaction between SF, Emotion, Equalization, and Hemisphere ( F(1,19) = 7.62, p < 0.05). Separate analysis for left and right hemispheres showed that the SF × Emotion × Equalization interaction was only significant for the right hemisphere ( F(1,19) = 6.22, p < 0.05). Further analysis indicated that there was a significant interaction between Emotion and Equalization at P8/PO8 solely in LSF ( F(1,19) = 4.56, p < 0.05). Hence, pairwise comparisons computed on P8/PO8 electrodes showed that LSF fearful faces elicited larger P1 amplitudes than LSF neutral faces only in EQ condition ( t(19) = −2.39, p < 0.05). 

P1 latency

The overall four-way ANOVA indicated a significant interaction between SF and Emotion ( F(1,19) = 5.65, p < 0.05). Further analysis of this interaction indicated that faster latencies for LSF vs. HSF faces occurred irrespective of facial expression (neutral: t(19) = 5.48, p < 0.001; fear: t(19) = 5.24, p < 0.001). Yet, there was a trend toward a significant effect of emotion for LSF faces only ( t(19) = −1.788, p = 0.090). SF × Equalization interaction was also significant ( F(1,19) = 8.15, p < 0.05) indicating significantly longer latencies for HSF non-EQ faces compared to HSF EQ faces ( t(19) = −2.45, p ≤ 0.05), whereas in the LSF condition, there was no latency difference between EQ and non-EQ conditions. In addition, faster latencies for LSF vs. HSF faces were found for both EQ (mean difference: 6 ms; t(19) = 4.11, p = 0.001) and non-EQ faces (mean difference 13 ms; t(19) = 5.11, p ≤ 0.001). 

N170 amplitude

Statistical analysis of N170 amplitude indicated a significant main effect of Hemisphere ( F(1,19) = 7.46, p < 0.05). Larger N170 amplitudes were found for the right compared to the left hemisphere. Furthermore, there was a significant interaction between SF, Emotion, and Electrode ( F(1,19) = 4.99, p < 0.05). ANOVA separately conducted in LSF and HSF indicated a significant interaction between Emotion and Electrode for LSF faces only ( F(1,19) = 22.09, p < 0.01). Although LSF fearful faces elicited larger N170 amplitudes compared to neutral faces at both electrode positions (PO7/PO8: t(19) = 8.23, p < 0.001, P7/P8: t(19) = 9.17, p < 0.001), this effect was stronger on PO7/PO8 (mean difference: 1.43 μV) as compared to P7/P8 electrodes (mean difference: 1.03 μV). 

N170 latency

The overall four-way ANOVA indicated a significant SF × Equalization interaction ( F(1,19) = 82.52, p < 0.001). Further analysis of this interaction indicated a significant effect of SF in both EQ and non-EQ conditions (Non-EQ: F(1,19) = 12.07, p < 0.001; EQ: F(1,19) = 7.31, p < 0.001) with shorter latencies for LSF compared to HSF faces. Yet, N170 latency differences between HSF and LSF were larger in non-EQ (mean difference: 23 ms) than EQ conditions (mean difference: 8 ms). Furthermore, HSF faces in the Non-EQ task showed significantly longer latencies than HSF faces in the Non-EQ task ( t(19) = −4.93, p < 0.001), whereas there was no difference between EQ and non-EQ images for LSF faces. 

Summary

Consistent with our hypotheses, the early influences of emotion as observed at the level of P1 and N170 amplitudes were only present in the LSF condition. For P1, the effect of emotion in the LSF condition was only significant in right hemisphere and only in the equalized condition. In contrast, for N170 this effect was found irrespective of contrast equalization. In addition, a main effect of SF was found for P1 and N170 latencies, indicating faster latencies for LSF compared to HSF faces. This effect was found irrespective of contrast equalization, although it was smaller in the equalized condition due faster latencies to HSF images. 

Experiment 2

In this experiment, we investigated whether the LSF advantage for the processing of facial expressions found in ERPs in Experiment 1, using a passive paradigm, is also reflected in reaction times when subjects have to perform an active categorization task on emotional expression. Although an LSF bias has been shown for the rapid categorization of emotional stimuli including happiness, sadness, and anger (Schyns & Oliva, 1999), this has not been investigated for fear. There is however evidence that LSF are important for rapid attentional responses to fear (Holmes, Green et al., 2005). Based on our ERP findings and literature, we predict increased sensitivity to fear in LSF and faster categorization of LSF fearful compared to LSF neutral faces. Alternatively, in contrast to ERP observations, emotion effects on RT may be obvious on both scales. Contrary to P1 and N170, behavioral RT reflects the final stage of visual processing. At this final stage, information about the facial expression of a face might be available in both LSF and HSF and RT may reveal emotion effects for HSF faces as well. 

Participants

Twenty adult subjects (10 women and 10 men; mean age: 22.21 years ( SD: 2.97)) participated in this experiment. The data of one participant were not included in the analysis because performance was below chance in one of the conditions. One participant was left-handed, the others were right-handed. Participants had normal or corrected-to-normal vision. 

Stimuli, task, and data analysis

Experimental conditions were identical to Experiment 1 (same stimuli, same duration, etc.), except that no animation figures occurred between face stimuli and that participants had to decide whether the presented stimulus was a fearful or a neutral face by pressing the appropriate keyboard button (left/right arrow). Key assignments were counterbalanced across subjects. Instructions emphasized speeded and accurate decisions. Trials with reaction times shorter than 150 ms and expanding 1500 ms were discarded. Reaction times were subjected to an SF (HSF/LSF) × Emotion (Neutral/Fear) × Equalization (Non-EQ/EQ) repeated-measure ANOVA. Furthermore, bias-free sensitivity indexes ( D′) were computed for each subject in all conditions (following Stanislaw & Todorov, 1999). 

Results

Reaction times

Repeated-measures ANOVA revealed a main effect of emotion ( F(1,18) = 21.51, p < 0.001) on RTs indicating shorter reaction times to fearful (mean: 544 ms) than neutral faces (mean: 566 ms; see Figure 3). Furthermore, a significant Spatial Frequency × Equalization interaction was found ( F(1,18) = 4.96, p < 0.05). Further analysis of this interaction indicated that participants were overall faster at categorizing LSF faces than HSF faces in both non-EQ ( F(1,18) = 50.09, p < 0.001) and EQ conditions ( F(1,18) = 15.28, p < 0.001) but that this effect was stronger in the Non-EQ task (see Figure 3). 

D′

SF significantly influenced D′ ( F(1,18) = 5.83, p < 0.05), indicating that participants were more sensitive to fear in the LSF condition than in the HSF condition, irrespective of equalization (see Figure 3). There were no other significant effects or interactions on D′. 

Figure 3

 

Mean reaction times (RT) and D′ (+ SE) for all stimulus conditions in the Non-Equalized (Non-EQ) and Equalized (EQ) task.

Discussion

In the present study, we investigated the role and time course of LSF and HSF information in the decoding of facial expressions. Based on previous literature, we expected to find that an early modulation of P1 and/or N170 by facial expression is primarily mediated by LSF. The second goal of our study was to investigate whether SF influences on emotional processing genuinely reflect the differential involvement of LSF and HSF or whether these are merely due to contrast and luminance differences across SF. We used emotional stimuli that were known to affect ERPs in broadband viewing conditions (REF) and filtered them to preserve either HSF or LSF. HSF and LSF faces were either matched (i.e., EQ condition) or not matched (i.e., non-EQ condition) on luminance and contrast. As expected, emotional expression modulated the amplitude of early visual ERPs, but only when presented in LSF. The increased P1 amplitude for LSF fear was observed in the right hemisphere and only when contrast/luminance was adjusted to fit full spectrum values (i.e., EQ condition). The N170 amplification related to LSF fear was observed in both EQ and non-EQ conditions. In contrast, HSF fear did not affect P1 or N170 amplitude. 

Several studies have reported increased P1 amplitude for negative emotional compared to neutral broadband face stimuli (Batty & Taylor, 2003; Pizzagalli et al., 1999, 2002). Increased P1 amplitude may reflect the allocation of attentional resources to emotionally significant stimuli (Eimer & Holmes, 2002; Lidell et al., 2004; Pourtois et al., 2005, 2004; Williams et al., 2004). The present findings indicate that P1 modulation by facial expression is primarily driven by LSF cues. This effect was only significant in the right hemisphere, which is consistent with several studies that show a right hemisphere advantage for the processing of faces and emotional expressions (e.g., Halgren, Raij, Marinkovic, Jousmaki, & Hari, 2000; Kawasaki et al., 2001; Pizzagalli et al., 1999, 2002; Rossion, Joyce, Corell, & Tarr, 2003; Streit, Wöwler, Brinkmeyer, Ihl, & Gaebel, 2000; Williams, Palmers, Lidell, Song, & Gordon, 2006). 

The present P1 findings are also in agreement with the study of Pourtois et al. (2005). These authors also showed that the early modulation of P1 by facial expression was primarily driven by LSF. These authors controlled for luminance and contrast differences across scales by using hybrid stimuli. Hybrid stimuli are built by superimposing an LSF-filtered image with an image filtered to preserve HSF. However, in the study of Pourtois et al. (2005), contrast/luminance was not equated between LSF and HSF within a given hybrid. In the present study, no hybrid stimuli were used and since the contribution of luminance and contrast was systematically addressed, we provide unequivocal evidence that P1 effects are primarily driven by LSF, using HSF and LSF stimuli that are matched on contrast and luminance. Emotion effects were absent for both HSF and LSF stimuli in the non-equalized condition, possibly because early visual evoked potentials are very sensitive to luminance and contrast alternations (Blau et al., 2007; Ellemberg, Hammarrenger, Lepore, Roy, & Guillemot, 2001). Contrast and luminance differences across HSF and LSF trials in non-equalized blocks might have impeded early P1 differences related to LSF emotion. 

A recent ERP study found evidence for enhanced early brain responses in extrastriate areas to threatful non-facial stimuli presented in LSF at occipital electrodes at a similar time range as the P1 (Carrétie, Hinojosa, López-Martín, & Tapia, 2007). This suggests that the early response to emotion in visual areas in the time range of P1 is not face specific but possibly results from threat signals in general, irrespective of the object category, as was also suggested by Pourtois et al. (2005). 

In addition to the findings on P1, we also found an enhanced N170 to fearful expressions in LSF only. In contrast, Holmes, Winston et al. (2005) and Pourtois et al. (2005) did not report any effect of emotion or interaction between emotion and SF for N170 amplitude. However, Holmes, Winston et al. (2005) and Pourtois et al. (2005) also did not report emotion effects on N170 in broadband viewing conditions in contrast to previous literature (Batty & Taylor, 2003; Blau et al., 2007; Campanella et al., 2002; Eger, Jedynak, Iwaki, & Skrandies, 2003; Krombholz, Schaefer, & Boucsein, 2007; Stekelenburg & de Gelder, 2004). N170 is thought to be a face-specific component that reflects encoding of the structure of a face in such a way that it can be differentiated from others (Jacques & Rossion, 2006) and is influenced by emotional expressions (Batty & Taylor, 2003; Blau et al., 2007; Campanella et al., 2002; Stekelenburg & de Gelder, 2004). An absence of emotional processing at N170 could be related to the mixed presentation of face stimuli (66%) and objects (33%) of similar size and color in the study of Holmes, Winston et al. (2005) and the type of task (gender discrimination) in the study of Pourtois et al. (2005), which might take attention away from the emotional signals. To our knowledge, all previous studies using broadband stimuli also failed to report any N170 emotional effects when they used a mixed presentation of objects and face stimuli (Ashley et al., 2004; Holmes, Vuilleumier, & Eimer, 2003). Furthermore, Krombholz et al. (2007) report that tasks in which facial emotion is irrelevant might impede emotional processing at the level of N170. However, these suggestions are subject for future study and are beyond the scope of the present study. Nonetheless, the present results on N170 show that early (<170 ms) encoding of fearful facial expressions is based on LSF input. Importantly, the interaction between SF and Emotion was significant for both contrast/luminance EQ and Non-EQ images, indicating that the absence of an effect of emotion at N170 is not merely due to HSF images being less luminant and containing less contrast than HSF images. 

Schyns, Petro, and Smith (2007) have provided interesting insights on how N170 reflects face processing over time. More specifically, subjects categorized stimuli consisting of samples of information, randomly sampled in x, y and SF dimensions of face images (see Figure 1 in Schyns et al., 2007). On each trial, a face was presented, which was partly revealed by a mid-gray mask punctured by a number of randomly located Gaussian windows (called “bubbles”) revealing information from five non-overlapping SF bands. By using classification image techniques, Schyns et al. (2007) investigated which combination of SF bands and image features was diagnostic for the categorization of each expression and how this related to N170. Interestingly, the ERP results of Schyns et al. (2007) showed that the initial processing of a fearful face starts by processing of the eyes at around 120-ms post-stimulus onset and is followed by the processing of mouth information. Their data further suggest that LSF information located around the mouth and HSF information located around the eyes may be particularly diagnostic for processing facial fear (see Figure 2 in Schyns et al., 2007); nevertheless, the authors did not explicitly address or quantify the HSF or LSF contribution to facial expression categorization. 

In contrast to the study of Schyns et al. (2007), the contribution of the various facial features to the perception of fear cannot be disentangled in the present study, as only SF content was manipulated. However, it has been shown that facial expression processing as well as many other aspects of face processing is not the mere outcome of purely local feature analyses but rather relies on the integration of features into a so-called holistic representation (Calder, Young, Keane, & Dean, 2000; Maurer, Le Grand, & Mondloch, 2002; see also for review Goffaux & Rossion, 2006). Since it decomposes face stimuli into parts, the use of bubbles may artificially induce a local bias in face emotional processing (Goffaux & Rossion, 2006; Rossion, 2008). In agreement with this suggestion is the larger N170 amplitude in left hemisphere as well as the faster information integration, as reflected by shorter N170 peak in this hemisphere in the study of Schyns et al. (2007). Left hemisphere advantages for face processing (and other types of stimuli) have namely been specifically linked to local, as opposed to, global processing, which has been found to be dominated by the right hemisphere (see Iidaka, Yamashita, Kashikura, & Yonekura, 2004). 

In our ERP study, SF not only affected P1 and N170 amplitudes but also largely modulated their latencies. LSF images were processed faster than HSF images irrespective of luminance or contrast equalization, as indicated by the shorter P1 and N170 latencies for LSF as compared to HSF images. Contrast/luminance equalization in the present study only influenced the processing of HSF faces, irrespective of emotional content. HSF images were more luminant and more contrasted in the EQ task than in the Non-EQ task. In contrast, luminance and contrast manipulation only slightly modulated LSF images across studies since their luminance/contrast was naturally close to broadband values to which they were normalized. Consequently, HSF EQ faces were processed faster than HSF non-EQ faces, as indicated by faster RT as well as shorter P1 and N170 latencies. The higher contrast transmitted in HSF EQ images also induced larger P1 amplitudes. 

SF influences on ERP latency are consistent with electrophysiological face studies that reported shorter ERP latencies to LSF than HSF facial stimuli (Halit, de Haan, Schyns, & Johnson, 2006; Hsiao, Hsieh, Lin, & Chan, 2005; McCarthy, Puce, Belger, & Allision, 1999) as well as visual evoked potential studies that reported SF effects on ERP latency for non-facial stimuli (Mihaylova, Stomonyakov, & Vassilev, 1999; Musselwhite & Jeffreys, 1985). This temporal precedence of LSF compared to HSF is consistent with previous findings that the neuronal pathways sensitive to LSF and HSF have dissociable time scales with faster cortical arrivals of information processed in the parvocellular (mainly sensitive to HSF) compared to magnocellular (mainly sensitive to LSF) system (Bullier, Schall, & Morel, 1996; Klistorner, Crewther, & Crewther, 1997; Maunsell et al., 1999; Schroeder, Tenke, Arezzo, & Vaughan, 1989; see for review Laycock, Crewther, & Crewther, 2007). 

In Experiment 2, we investigated whether the LSF advantage for the processing of facial expressions found in ERPs in Experiment 1 would be reflected in RT when subjects have to complete an active categorization task. In contrast to the ERP findings, we found an effect of facial expression for both HSF and LSF faces in the EQ as well as the Non-EQ task. Participants decided more quickly that a face was fearful than that it was neutral, irrespective of SF content. RTs to HSF stimuli were overall slower than to LSF stimuli. This is consistent with several studies that indicate that stimuli signaling threat receive preferential attention over neutral stimuli (see for review Holmes, Green et al., 2005) and behavioral studies indicating faster processing of LSF over HSF stimuli (Coin, Versace, & Tiberghien, 1992; Parker, Lishman, & Hughes, 1992, 1997). The lack of interaction between SF and emotion in RT could be related to the fact that RT reflects the final stage of information processing, when the facial expression of a face is available for both LSF and HSF. Our results suggest that HSF participate to fear processing but at a later stage (>170 ms). In addition to these reaction time effects and consistent with our ERP findings, analysis of SDT measures showed that subjects were more sensitive to fear in LSF faces than HSF faces, irrespective of contrast/luminance equalization. This matches a recent study showing that a connectionist model better classified fearful faces based on LSF than HSF input (Mermillod, Guyader, Vuilleumier, Alleysson, & Marendaz, 2005). 

As a final point, we would like to discuss some limitations to the present study. It specifically aimed at investigating the role of HSF and LSF in facial expression processing. Like previous studies, the present study did not include intermediate SF bands, which are known to best carry face identity (Costen et al., 1994; Costen, Parker, & Craw, 1996; Parker & Costen, 1999) and could also play a role in the rapid processing of facial expressions. To our knowledge, the role of intermediate SF to facial expression processing had not been studied and should be addressed by measuring behavioral and electrophysiological responses to fear while SF content of face images is parametrically shifted from low to high spatial frequencies (see Tanskanen, Näsänen, Montez, Päällysaho, & Hari, 2005 for a similar approach using non-emotional faces). 

Based on the observation that amygdala activity to fearful faces is related to the processing of LSF information, Vuilleumier et al. (2003) argued that amygdala has a rapid access to LSF components of visual information, either through a direct subcortical tecto-pulvinar route preferentially tuned to LSF information, or through the initial feedforward sweep within the visual system. Here, we provide high-temporal resolution evidence that LSF input is processed fast and drives the early detection of fearful expressions as indicated by an enhanced P1 and N170 across ventro-temporal areas for LSF only. Since the amygdala modulates activity in visual areas via feedback signals (Morris et al., 1998; Rotshtein et al., 2001), the amygdala may rapidly enhance ventral temporal visual responses to faces at P1 and N170 latencies. This hypothesis needs further investigations, combining EEG and fMRI methodology and connectivity analyses. 

In sum, our behavioral as well psychophysiological findings indicate that LSF are important for the rapid extraction of facial expression information. For the first time, we have shown that a modulation of the face-specific N170 by facial expression is primarily mediated by LSF information and that this effect cannot be explained by differences in luminance and contrast between HSF and LSF faces. Furthermore, in a behavioral study we found that in addition to LSF, HSF also contain important information about the facial expression of a face, which might be important at a later stage of information processing. The present ERP findings might be important for all studies investigating the role of SF in facial expression recognition. Especially, for children with psychiatric disorders like autism, for whom deficits in LSF processing have been suggested (Johnson, 2005). For these populations, the passive ERP technique, as used in the current study, is an excellent method. 

Acknowledgments

The authors gratefully thank Pia Jansen and Sabine Koenraads for helping with the data collection and Judith Peters for previous comments on the manuscript. 

Commercial relationships: none. 

Corresponding author: Petra Vlamings. 

Email: [e-mail]. 

Address: Biological Developmental Psychology Section, Faculty of Psychology, Universiteit Maastricht, P.O. Box 616, 6200 MD Maastricht, The Netherlands.