Eye coding mechanisms in early human face event-related potentials

Abstract

In humans, the N170 event-related potential (ERP) is an integrated measure of cortical activity that varies in amplitude and latency across trials. Researchers often conjecture that N170 variations reflect cortical mechanisms of stimulus coding for recognition. Here, to settle the conjecture and understand cortical information processing mechanisms, we unraveled the coding function of N170 latency and amplitude variations in possibly the simplest socially important natural visual task: face detection. On each experimental trial, 16 observers saw face and noise pictures sparsely sampled with small Gaussian apertures. Reverse-correlation methods coupled with information theory revealed that the presence of the eye specifically covaries with behavioral and neural measurements: the left eye strongly modulates reaction times and lateral electrodes represent mainly the presence of the contralateral eye during the rising part of the N170, with maximum sensitivity before the N170 peak. Furthermore, single-trial N170 latencies code more about the presence of the contralateral eye than N170 amplitudes and early latencies are associated with faster reaction times. The absence of these effects in control images that did not contain a face refutes alternative accounts based on retinal biases or allocation of attention to the eye location on the face. We conclude that the rising part of the N170, roughly 120–170 ms post-stimulus, is a critical time-window in human face processing mechanisms, reflecting predominantly, in a face detection task, the encoding of a single feature: the contralateral eye.

Introduction

Electrophysiological studies of human recognition mechanisms have identified face preferential responses within 200 ms following stimulus onset. The most important of these are the N170 event-related potential (ERP) measured from scalp recordings (Rossion & Jacques, 2008) and the N200 ERP measured from intracranial recordings (Allison, Puce, Spencer, & McCarthy, 1999; Rosburg et al., 2010; Rossion & Jacques, 2008). Irrespective of recognition task, preferential responses typically correspond to larger ERP amplitudes to face pictures compared with other object categories (Rousselet, Gaspar, Wieczorek, & Pernet, 2011; Rousselet, Husk, Bennett, & Sekuler, 2008). However, such coarse characterization under-specifies a genuine mechanistic account of the N170, leading to considerable debate about the functional role and timing of the underlying mechanisms. Simply stated, we do not fully understand the specific information processing mechanisms that the N170 reflects. With integrated ERP measurements, these mechanisms can reflect processing of a succession of specific information contents, i.e., a succession of image features that modulate N170 ERP amplitudes, to a different degree and at different time points. Understanding such a detailed relationship between information content and ERP modulations is a necessary prerequisite to understand the transformations of visual information that the visual system operates en route to recognition—i.e., the information processing mechanisms (Schyns, Gosselin, & Smith, 2009). 

To understand these mechanisms, researchers have sought to specify the visual information associated with single-trial N170 responses. Classification images can depict these associations because they represent the covariations between pixel visibility and the subjects' brain and behavioral responses (Murray, 2011, 2012). For instance, bubbles is one of several techniques (Haig, 1985) that produce classification images via reverse correlation (Gosselin & Schyns, 2001). With bubbles, contiguous face image pixels are randomly sampled with Gaussian apertures. Bubbles revealed that the presence of the contralateral eye initially modulates N170 amplitudes at left and right occipital-temporal electrodes, followed by the features diagnostic of the task—e.g., the mouth to categorize a “happy” face (Schyns, Jentzsch, Johnson, Schweinberger, & Gosselin, 2003; Schyns, Petro, & Smith, 2007; Smith, Gosselin, & Schyns, 2004, 2007; van Rijsbergen & Schyns, 2009). Precedence of the contralateral eye suggests that cortical face mechanisms could use the eye as an anchor from which the face is scanned downwards, until diagnostic information is reached and integrated (Schyns et al., 2007). In other ERP studies, in which face features were defined a priori as regions of interest, the eyes alone elicited large N170s (Bentin, Allison, Puce, Perez, & McCarthy, 1996; Itier, Alain, Sedore, & McIntosh, 2007; Nemrodov & Itier, 2011). Computational models have also highlighted the importance of the eyes in face detection (Tsao & Livingstone, 2008; Ullman, Vidal-Naquet, & Sali, 2002). Finally, recordings in monkey face patches suggested the prominence of the eye area in driving early neuronal responses (Freiwald, Tsao, & Livingstone, 2009; Issa & DiCarlo, 2012; Ohayon, Freiwald, & Tsao, 2012). A recent bubbles study in monkeys revealed that early responses from posterior and middle face patches are tuned to the contralateral eye (Issa & DiCarlo, 2012), in a size- and translation-invariant manner, suggesting that the early response is a candidate for local processing of features in the eye region. 

The reviewed computational models and human and monkey electrophysiology evidence lead to a straightforward prediction: Early neuronal activity in humans should reflect early coding mechanisms of the contralateral eye in a face detection task, with consequences for behavioral decisions (e.g., faster and more accurate responses). To test this prediction and understand coding mechanisms over the parameters of early human brain responses (i.e., N170 amplitude and latency), we designed an experiment in which observers discriminated faces from noise textures, both sparsely masked by Gaussian apertures. We contrasted noise textures lacking sharp edges with faces to ensure that any face feature was sufficient to perform the task. Sparse sampling of the images with Gaussian apertures revealed contiguous pixel areas without introducing sharp discontinuities. To measure the association between randomly sampled image pixels and behavior and brain activity, we used mutual information, a robust measure of association (Cover & Thomas, 2006). As predicted, reverse correlation coupled with information theoretic analyses revealed that presence of the contralateral eye led to larger and earlier N170s, as well as to shorter reaction times and more accurate responses. 

Methods

Subjects

The study comprised 16 subjects: nine females, 15 right handed, median age = 23 (min 20, max 36). Prior to the experiment, all subjects read a study information sheet and signed an informed consent form. The experiment was approved by the local ethics committee with approval no. CSE00740. Subjects did not report any eye condition, history of mental illness, taking psychotropic medications, or suffering from any neurological condition. Subjects' visual acuity and contrast sensitivity were assessed in the lab on the day of the first session using a Colenbrander mixed contrast card set and a Pelli-Robson chart. All subjects had normal or corrected-to-normal vision and contrast sensitivity of 1.95 and above (normal score). Subjects were compensated £6/hr for their participation. 

Stimuli

Stimuli were gray-scale pictures of faces and textures (Figure 1). Faces from 10 identities were front view photographs, oval cropped to remove hair, and pasted on a uniform gray background (Gold, Bennett, & Sekuler, 1999). A unique image was presented on each trial by introducing noise (70% phase coherence) into the face images (Rousselet, Pernet, Bennett, & Sekuler, 2008). Textures were face images with random phase (0% phase coherence). As a result of phase randomization, these textures lacked local edges characteristics of faces, so that all face features were diagnostic, i.e., they were all sufficient to detect faces. All stimuli had an amplitude spectrum set to the mean amplitude of all faces. All stimuli also had the same mean pixel intensity, 0.2 contrast variance, spanned 9.3° × 9.3° of visual angle. The face oval was 4.9° × 7.0° of visual angle. 

Figure 1

 

Examples of stimuli. Row 1 shows the 10 face identities used throughout the experiment. Row 2 shows examples of textures. Rows 3 and 4 show the same faces and textures with examples of masks with Gaussian apertures. The fixation cross fell around the midline, on the nose, thus coinciding with natural saccade landing positions (Bindemann, Scheepers, & Burton, 2009; Hsiao & Cottrell, 2008).

Face and noise pictures were revealed through 10 two-dimensional Gaussian apertures (sigma = 0.36°) randomly positioned with the constraint that the center of each aperture remained in the face oval and was at a unique position. In the rest of this article, we refer to these masks with Gaussian apertures as bubble masks. Information sampling was dense enough to reveal face features, but sparse enough to prevent the task from being trivial. 

Experimental procedure

During the experiment, subjects sat in a sound attenuated booth and rested their head on a chin rest. Viewing distance measured from the chin rest to the monitor screen was 80 cm. At the beginning of each of two experimental sessions, subjects were fitted with head cap and EEG electrodes. Stimuli were displayed on a Samsung SyncMaster 1100Mb monitor (600 × 800 pixels; height and width 22° × 28° of visual angle; 85-Hz refresh rate). Subjects were given experimental instructions including a request to minimize blinking and movement. Subjects were asked to categorize images of faces and textures as fast and accurately as possible: they pressed one for face and two for texture, on the numerical pad of a keyboard, using the index and middle fingers of their dominant hand. At the end of every block they received feedback on their overall performance and, after Block 1, on their performance in the previous block: Median reaction times and percent correct remained on the screen until subjects press a key to move on to the next block. Before the main experiment, subjects performed a practice block with images without bubble masks, to minimize spatial uncertainty. After the practice block, subjects performed 11 blocks of the images with bubble masks. All 12 blocks had the same structure: they consisted of 100 trials, with 10 face identities, each repeated five times, each time with a unique noise field, and 50 unique noise textures. Subjects could take a break at the end of each block. The whole session consisted of 1,200 trials, including 100 practice trials. All subjects participated in two experimental sessions, bringing their total of bubble trial number to 2,200. Each session lasted about 60 to 75 min, including breaks, excluding the time required to apply the EEG electrodes prior to actual testing. 

Each trial began with a small black fixation cross (0.4° × 0.4° of visual angle) displayed at the center of the monitor screen for a random time interval of about 500–1000 ms, followed by an image of a face or a texture presented for seven frames (∼82 ms). A blank gray screen followed stimulus presentation until subject's response. The fixation cross, the stimulus, and the blank response screen were all displayed on a uniform gray background with mean luminance ∼ 43 cd/m². 

EEG recording and preprocessing

EEG data were recorded at 512 Hz using an active electrode amplifier system (BIOSEMI) with 128 electrodes mounted on an elastic cap. Four additional electrodes were placed at the outer canthi and below the eyes. 

EEG data were preprocessed using Matlab 2013a, RRID:nlx_153,890, and the open-source toolbox EEGLAB version 11, RRID:nif-0000-00076 (Delorme et al., 2011). Data were first re-referenced off-line to an average reference. Data were then band-pass filtered between 1 Hz and 30 Hz using a noncausal fourth order Butterworth filter. This high-pass noncausal filtering improves independent component analysis (ICA) but can potentially distort onsets (Acunzo, Mackenzie, & van Rossum, 2012; Rousselet, 2012; Widmann & Schroger, 2012). Therefore, we checked for timing distortions by creating a second dataset in which data were preprocessed with fourth order Butterworth filters: high-pass causal filter at 2 Hz and low-pass noncausal filter at 30 Hz. All our results were virtually identical for the two filter settings, except for onset timings, as expected. Data from the two datasets were then downsampled to 500 Hz, and epoched between −300 and 1000 ms around stimulus onset. Baseline correction was performed using the average activity between time 0 and −300 ms only for the high-pass causal filtered data set; for the noncausal filtered dataset, the channel mean was removed from each channel instead (Groppe, Makeig, & Kutas, 2009). Noisy electrodes and trials were detected by visual inspection of the noncausal dataset and rejected from the two datasets on a subject-by-subject basis. The reduction of blink and eye-movement artifacts was performed using ICA, as implemented in the infomax algorithm from EEGLAB. ICA was performed on the noncausal filtered dataset and the ICA weights were then applied to the causal filtered dataset (on a subject by subject basis) in order to ensure removal of the same components from both datasets (Ullsperger & Debener, 2010). Components representing blinks and eye movements were identified by visual inspection of their topographies, time-courses and amplitude spectra. After rejection of artifactual components (median = 4; min = 1; max = 10), baseline correction was performed again, and data epochs were removed based on an absolute threshold value larger than 100 μV and the presence of a linear trend with an absolute slope larger than 75 μV per epoch and R² larger than 0.3. The median number of bubble trials accepted for analysis was, out of 1,100: face trials = 1,090 [min: 1,074, max: 1,098]; noise trials = 1,090 [min: 1,057, max: 1,100]. Finally, we computed single-trial spherical spline current source density (CSD) waveforms using the CSD toolbox (Kayser & Tenke, 2006; Tenke & Kayser, 2012). CSD waveforms were computed using parameters 50 iterations, m = 4, lambda = 1.0e-5. The head radius was arbitrarily set to 10 cm, so that the ERP units in all figures are μV/cm². The CSD transformation is a spatial high-pass filtering of the data, which sharpens ERP topographies and reduces the influence of volume-conducted activity. CSD waveforms also have the advantage of being reference free. 

Electrode selection

For each subject, we proceeded in two steps. First, we computed classic ERPs by averaging EEG responses separately for all face bubble trials and all noise bubble trials and then subtracting the resulting ERPs. From these ERP differences we selected posterior-lateral left hemisphere (LE), and right hemisphere (RE) electrodes with the constraint that the electrodes displayed the largest N170—i.e., local minimum ERP difference around 150–220 ms. Thus the electrode selection was different across subjects (Foxe & Simpson, 2002; Rousselet & Pernet, 2011). However, not surprisingly, all the electrodes were A10/B7 (PO7/PO8) or their immediate neighbors, electrode locations typically associated with the N170. To this electrode selection, we added four posterior midline electrodes: from top to bottom CPz, Pz, POz, Oz. These electrodes have been associated with task-relevant feature sensitivity, possibly reflecting perceptual decisions (Schyns, Thut, & Gross, 2011; Smith et al., 2004; Smith, Gosselin, & Schyns, 2006). 

Statistical analyses

Analysis framework

Here, we applied mutual information (henceforth, MI) in two categories of analyses (green and red links in Figure 2). First, we used MI to reveal the image pixels statistically associated with behavioral (green box) and EEG modulations (red box); we refer to this approach as the forward analysis. It is equivalent to the reverse-correlation technique used in previous EEG studies (Schyns et al., 2003; Schyns et al., 2007; Smith et al., 2004) because it reveals the image locations statistically associated with fluctuations in behavioral or EEG response distributions. Second (not illustrated in Figure 2), we used MI to reveal the EEG components associated with specific image parts; we refer to this approach as the reverse analysis (Smith et al., 2004). Statistical analyses were performed in Matlab 2013a and 2013b. For group analyses, the mean was used as a measure of central tendency in most cases, except for some descriptive statistics and for the N170 latency, for which we used the Harrell-Davis estimate of the median (Wilcox, 2012). Throughout this paper, square brackets indicate 95% confidence intervals computed using the percentile bootstrap technique, with 1,000 bootstrap samples. 

Figure 2

 

Framework to analyze information processing mechanisims. Colored links represent the different information theoretic quantities computed between information samples (obtained with bubbles from face pictures), behavioral measurements (reaction times and accuracy), and brain measurements (single trial EEG measured at three sensors, left hemisphere electrode [LE], and right hemisphere electrode [RE] and Oz). Colored boxes represent three outcomes of our computations: in green, information values between samples and categorization behavior; in red, schematic information values between samples and single-trial distributions of brain activity at different sensors and time points; in blue, schematic information values between brain activity and behavior (reaction times and face identification accuracy). In the red box, the right eye is shown as a green disk, the left eye an orange disk, and the mouth as a yellow oval.

Mutual information

In single subjects, we used mutual information (MI), to quantify the relationship existing between the variables of interest: sampled image pixels and behavioral and EEG responses to the pixels—we detail the specific measurements below and illustrate the overall framework in Figure 2. We used MI because it is a nonparametric (i.e., model free) measurement that precisely quantifies (in bits) the dependence, whether linear or nonlinear, between pairs of variables (Fairhall, Shea-Brown, & Barreiro, 2012; Ince, Petersen, Swan, & Panzeri, 2009; Magri, Whittingstall, Singh, Logothetis, & Panzeri, 2009; Panzeri, Brunel, Logothetis, & Kayser, 2010; Schyns et al., 2011). Direct estimation of mutual information between discrete variables from limited amounts of data results in a systematic upward bias. Whereas several techniques exist to correct for this bias we do not employ them here (Panzeri, Senatore, Montemurro, & Petersen, 2007). Instead, we use MI as the effect size of a statistical test of independence (Ince, Mazzoni, Bartels, Logothetis, & Panzeri, 2012). 

To compute MI, we first grouped the data from each behavioral, EEG, and image pixel variable into bins. Here, we used three equiprobable bins—using four, five, or six bins did not change the results but increased MI values (due to increased bias). At first glance, the use of such a small number of bins may seem a crude approximation of the signal, but the use of equiprobable bins effectively makes the resulting mutual information value a robust rank statistic. We have found that this makes it ideally suited to the properties of the EEG signal given its relatively low signal-to-noise ratio, large amplitude differences across subjects, and so forth. We calculated MI directly from the standard definition (Ince, Mazzoni, Petersen, & Panzeri, 2010), using the following formula:  

 

where B_i represents the bubble mask value (pixel visibility) at pixel i and R represents the response of interest (either behavioral or EEG recording). P(b) is the probability of pixel i having bubble mask falling inside bin b (of the three equiprobable bins); P(r) is the probability of the considered response falling inside bin r, and P(b,r) is the joint probability of the coincidence of both events. I(B_i; R) quantifies the reduction of uncertainty about the neural/behavioral response that can be gained from knowledge of the visibility of pixel i. To evaluate MI values, it is useful to consider an upper bound on the measure. The maximum MI value in MI(X, Y) is the smallest entropy of the two variables—i.e., the maximum uncertainty that could be removed from X by perfect knowledge of Y, or vice versa, if their association was noiseless. When we represent Y with two bins, e.g., as in MI(pixels; correct vs. incorrect responses), the upper bound of MI is log2(2) = 1 bit, assuming equally likely possibilities. Based on the average proportion correct of 93% (see Results), the upper bound of MI is given by the entropy −0.93log2(0.93) − 0.07log2(0.07) = 0.3659 bit. When we represent Y with three bins, e.g., as in MI(pixels; reaction times, and MI(pixels; ERPs), the upper bound is log2(3) = 1.585 bit. Intuitively, 1 bit of MI means that on average, two equally likely possibilities are perfectly discriminated from a single observation. That is, observation of variable X (e.g., a particular pixel intensity) allows correct prediction of variable Y (e.g., a correct response). When the association between pixel intensities and correct versus incorrect responses is weaker, more trials need to be observed to determine if the response was correct or incorrect; for example a value of 0.04 bit suggests on average 25 observations would be required to determine which of the response classes was sampled.  

Other measures of association could be used instead of MI, for instance correlation techniques and general(ized) linear models. However, in our view MI has a number of advantages that justify its selection here. First, MI has the potential to detect association of any order, making it an excellent choice when the nature of the association is unknown. Second, as described above the binning procedure results in a statistic with robust statistical properties, whereas alternative correlation techniques can be strongly affected by deviation from ideal conditions (Pernet, Wilcox, & Rousselet, 2012; Rousselet & Pernet, 2012). Finally, we believe that the MI provides substantially greater statistical power for investigations such as this. To confirm this, we performed a comparison between MI, Pearson's correlation, which estimates linear association, and Spearman's rank correlation, which estimates monotonic association. We restricted this analysis to EEG responses at 160 ms (latency of the strongest MI between ERP amplitudes and image pixel intensities), at the six electrodes of interest. All methods identified very similar spatial regions affecting the EEG responses (results not shown). However, the MI measure appeared to demonstrate a much higher statistical power—for the strongest effects the MI value was 24/26 standard deviations further away from the permutation null distribution than that for Spearman/Pearson correlation, respectively. A more rigorous systematic comparison between such measures over a range of different experimental conditions would be an interesting topic for future methodological studies. Given that the MI is closely related (via a scale factor) to the log-likelihood ratio significance test statistic, which by the Neyman-Pearson lemma is the most powerful test for a given α, we would expect the above findings to hold over any situation in which there is no fine-grained relationships that would be lost in the binning procedure. Finally, for all the analyses, we included all the trials, regardless of whether subjects were correct or not (VanRullen, 2011). 

Statistical significance of mutual information

To establish statistical significance (and control for multiple comparisons), we used a permutation test coupled with threshold-free cluster enhancement or TFCE (Smith & Nichols, 2009). TFCE scores transform pixel MI values to represent how they locally support clustering. Clustered pixels (e.g., corresponding to spatially contiguous modulating regions) will get higher TFCE scores, whereas isolated individual pixels won't. This enhances the statistical power of the maximum statistics multiple comparison approach, based on the assumption that a real effect is unlikely to be restricted to a single pixel. For permutation, we shuffled subjects' responses 500 times while keeping the bubble masks constant. For each repetition, we derived an MI map, TFCE-scored it, and extracted the maximum TFCE score across pixels. The 500 TFCE maxima produced a distribution, the 95th percentile of which was used as threshold for statistical comparison with the original TFCE scores. This procedure achieved a family-wise error rate across pixels of 0.05. The main advantage of TFCE is to allow inferences at the pixel level, in contrast to cluster-based statistics that allow inferences about clusters, but not their component pixels (Smith & Nichols, 2009). Here, TFCE parameters were E = 1, H = 2 and dh = 0.1, the default for 2-D fields. A Matlab implementation of TFCE is available in the LIMO EEG toolbox, RRID:nlx_155,782 (Pernet, Chauveau, Gaspar, & Rousselet, 2011; Pernet, Latinus, Nichols, & Rousselet, 2014). We computed the centroids of the significant clusters with Matlab's regionprops function (with option WeightedCentroid). 

Results

Behavior (green link and box in Figure 2)

Subjects were fast and accurate: median of the median reaction times = 383 ms [351, 409]; mean percent correct = 0.93 [0.92, 0.94]. To determine image features significantly associated with reaction times (RT) and accuracy, in the forward analysis we applied MI measures between pairs of variables: 

• 1.  
  MI(PIX, RT) measures in bits the association strength between bubble masks and subject's RTs, computed independently for face and noise trials.
• 2.  
  MI(PIX, RESP) measures the association between bubble masks and subject's face versus noise behavioral responses.
• 3.  
  MI(PIX, CORRECT) measures the association between bubble masks and subject's correct versus incorrect responses.

MI(PIX, RT) revealed for all subjects a strong association between presence of the left eye and RT, as revealed by significantly high MI values (Figure 3, Row 1). We also found significant MI for the right eye in a few subjects. Subjects were faster on trials that revealed the left, the right eye, or both. On noise trials, MI values were low, mainly nonsignificant, and not clearly clustered (Figure 3, Row 2). Thus, the relationship between presence of the left eye and RT seemed to be due to the presence of structured, high contrast elements in the eye area. 

Figure 3

 

Behavioral results. The first column shows the mean MI across 16 subjects. MI was much larger for face trial RT than for noise trial RT or responses, and independent scales had to be used to visualize the results. The second column shows the number of subjects showing significant effects at any pixel. In that column, in each plot the top left corner indicates the maximum number of subjects showing a significant effect at the same pixel. In the third column, the green plus signs indicate the centroids of the significant clusters. Each plus sign was contributed by one subject, and each subject could contribute more than one. In each centroid plot, the top left corner indicates the number of subjects showing significant effects at any pixel.

MI(PIX, CORRECT) and MI(PIX, RESP) were significant in a few subjects and clustered primarily around the left eye (Figure 3, Rows 3–5), indicating in these subjects stronger associations between the left eye and correct face detection responses. 

Forward ERP analysis (red link and box in Figure 2)

Full scalp analysis

We removed one subject from analysis due to poor EEG signal, for a total of 15 subjects for EEG analyses. First, we consider the maximum across electrodes of the absolute differences between mean face bubble ERPs and mean noise bubble ERPs (Figure 4, Row 1). We computed the peak amplitudes and latencies of ERPs in a 200 ms time window following stimulus onset, after applying a 20 Hz low-pass fourth order Butterworth noncausal filter. ERP differences reveal a typical P1 ERP, with a peak shortly after 100 ms, followed by a typical N170 ERP, with a larger and more sustained peak at 176 ms [170, 186]. ERP differences in practice trials without bubbles peaked at 137 ms [134, 142], and in the bubble trials about 38 ms later (we consider the possible cause of this delay in the Discussion). 

Figure 4

 

Maximum mean ERP differences and mutual information across all electrodes. In all panels, thick lines show the mean across subjects and the shaded areas portray 95% confidence intervals. To summarize the results, for every subject and every time point, we computed the maximum MI across electrodes, leading to one virtual electrode per subject. Waveforms at these virtual electrodes were then averaged across subjects. Results were baseline corrected to facilitate comparison. The head maps show the normalized average across subjects of the topographic distributions at the latency of the mean peaks occurring under 200 ms. The maps range from zero in blue to the matching mean peak value in red. Not surprisingly, MI was maximum at posterior lateral electrodes. Row 1 shows the absolute differences between face and noise mean ERPs. Two differences are shown, one for practice trials without bubbles (magenta), one for regular trials with bubbles (black). The vertical black line marks the latency of the maximum mean ERP difference for the bubble trials, corresponding to the N170. This line is displayed in the other panels to facilitate timing comparisons. Row 2 shows, for face trials, the mutual information between ERPs and subjects' RT (green) and between ERPs and image pixels (orange). MI(PIX, ERP) was summarized by taking the maximum across pixels. Row 3 shows, for noise trials, the same mutual information as in Row 2.

Having established classic ERPs in face detection as reported in the literature, our aim is now to understand the information processing function of the ERP signals. To this aim, we computed, for each electrode and time point, MI between single-trial ERP distributions and three variables, leading to three quantities (Figure 4). 

(1) MI(PIX, ERP) measures in bits the association strength between bubble masks and single-trial ERPs measured at six electrodes of interest and across all time points, separately for face and noise trials. For each subject, we derived a 3-D volume per electrode (i.e., two image coordinates by time points). We applied similar statistical thresholding methods as with behavior, swapping behavioral response with single-trial ERPs. To remove small clusters, we applied TFCE to EEG MI images filtered by a 3-D Gaussian kernel (5 × 5 Pixels × 5 Time Points). Figure 4, Row 2, shows that single-trial face ERPs strongly associate with face information (as sampled by image pixels): MI(PIX, ERP) shows a sharp increase shortly after 100 ms and peaked about 15 ms [11, 20] before the N170. 

(2) MI(ERP, RT) measures the association between single-trial ERPs and behavioral reaction times, separately for face and noise trials. Figure 4, Row 2 shows that the measure started to increase slightly after MI(PIX, ERP), peaking slightly before the N170, followed by a large sustained period around 300–800 ms, corresponding to the time window of subjects' motor responses. 

(3) MI(ERP, CORRECT) measures the association between single-trial ERPs and correct versus incorrect responses. It is not shown because it was flat for all subjects, indicating that brain activity was not statistically associated with correct responses. 

The early ERP differences and MI around 100 ms were partly due to filter distortions, as demonstrated by their strong attenuation in causal filtered data (Figure 4, insets). All later effects were virtually identical with the noncausal filtered data, and were thus not due to filtering distortions. 

In noise trials, no clear peak could be identified for MI(PIX, ERP) and MI(ERP, RT) in the N170 period. There was a small MI(PIX, ERP) peak shortly after 100 ms, but with a medial topography, as opposed to the lateral effects observed in face trials. Finally, consistent with the involvement of motor related brain activity, we did observe the same long sustained period of MI(ERP, RT) as in face trials, beyond 200 ms. 

In sum, full scalp ERP analyses revealed an association between single-trial ERP distributions at lateral electrodes and image pixels as well as RT. Associations between ERPs and pixels were maximum around 160 ms, before the peak of the N170, and were absent in noise trials. The ERP results appeared to be specific to face trials, similar to results for behavior, and thus rule out a spatial attention bias as an explanation of the effects. 

Information content

Next, we look at the information content at six electrodes of interest (see Methods). These electrodes capture the lateral and midline activity portrayed in Figure 4. First, we consider the maximum MI across time points, to reveal any information sensitivity occurring from stimulus onset (Figure 5). For left and right lateral electrodes, MI was maximum for the contralateral eye, and overall stronger for the left eye, contralateral to the right hemisphere. There was also weak sensitivity to the ipsilateral eye in a few subjects. Midline electrodes showed attenuated versions of this pattern. In addition, Oz, the lowest of the midline electrodes, revealed ERP sensitivity to both eyes, the nose and mouth/chin. However, nose and mouth/chin sensitivity appeared earlier and was weaker than eye sensitivity and was circumscribed to midline electrodes; nose and mouth/chin sensitivity was also observed in noise trials (Figure 5, bottom right), and across subjects did not differ significantly between face and noise trials: It might thus be a low-level response to stimulation at these locations rather than reflecting an explicit feature integration process. 

Figure 5

 

Maximum MI(PIX, ERP) across frames. The rows correspond to different electrodes; the first pair of columns shows mean MI across subjects, and the second pair of columns shows frequency of effects, separately for face and noise trials. In every frequency plot, the top left corner indicates the maximum number of subjects showing a significant effect at the same pixel, whereas the top right corner indicates the number of subjects showing significant effects at any pixel.

Second, we investigate how the contralateral eye sensitivity unfolds over time (Figure 6). Lateral electrodes showed strongest sensitivity to the contralateral eye around 140–180 ms post-stimulus onset, with a peak around 160 ms. In the same period, midline electrodes were sensitive to the two eyes. In addition, POz and Oz showed sensitivity to the nose and mouth/chin, with maximum sensitivity at least 20 ms before the LE and RE showed maximum eye sensitivity. 

Figure 6

 

MI(PIX, ERP) time course at six electrodes. The top panel shows the MI time courses time-locked to stimulus onset, from 130 to 190 ms. Each oval shows the mutual information between face pixels and ERP amplitudes, with the first two rows illustrating sensitivity to the contralateral eye: right eye for LE and left eye for RE. The bottom panel shows the MI time courses time locked to the peak of the N170, marked as time zero. The realignment of time courses was performed in every subject individually, and then the new time courses were averaged across subjects. The color scale in the lower panel was used to illustrate results at LE and RE in both panels, whereas results at midline electrodes are illustrated using the color scale in the upper panel. To put the MI results in perspective, we compare them to Spearman's correlation estimates at 160 ms post-stimulus. At LE, the median across subjects of the maximum MI across pixels was 0.06 [0.03, 0.08], min = 0.01, max = 0.20; Spearman's correlation was 0.28 [0.19, 0.34], min = 0.08, max = 0.49. At RE median maximum MI was 0.08 [0.05, 0.11], min = 0.01, max = 0.13; Spearman's correlation was 0.34 [0.29, 0.41], min = 0.11, max = 0.45.

Figure 4 showed that MI(PIX, ERP) peaked before the N170, and therefore we also considered the time course of the information content relative to the N170 peak (Figure 6, bottom panel). With N170 peak latency as time zero, sensitivity to the contralateral eye started and peaked at negative times, before the N170 peak. Contralateral eye sensitivity stopped shortly after the peak of the N170, and resumed, in a much more attenuated form and mostly for the left eye, about 20 ms later. 

So far, the results suggest that in our detection task, early brain activity is mostly sensitive to the contralateral eye. However, it is possible that sensitivity to other features, and their integration, is hidden somewhere in different temporal frequency bands of the ERPs (Smith, Gosselin, & Schyns, 2007). To check for this possibility, we computed MI between 0 and 400 ms on distributions of single-trial phase and amplitude, MI(PIX, PHA) and MI(PIX, AMP). Single-trial ERPs were decomposed every Hz between 4 and 20 Hz, using three-cycle Morlet wavelets. Results from these time-frequency analyses revealed that all time-frequency bands showed the same sensitivity to the contralateral eye at lateral electrodes (Figure 7). Midline electrodes, and more prominently Oz, showed sensitivity to the two eyes, the nose and the mouth/chin. Sensitivity to nose and mouth/chin was also observed in noise trials, suggesting again that this sensitivity in face trials could be a low-level effect. 

Figure 7

 

Maximum MI(PIX, PHA) and MI(PIX, AMP) across time frames. Results are shown for temporal frequencies between 4 and 20 Hz, at six electrodes, for phase (left column) and amplitude (right column), for face trials. The black rectangle shows Oz phase results for noise trials, with nose and mouth/chin sensitivity similar to that observed for face trials. The feature sensitivity at all electrodes was maximum around 100–200 ms after stimulus onset. The left hand side color scale was used to illustrate results at LE and RE, whereas midline electrode results are illustrated using the right hand side color scale.

The time-frequency analyses also suggest that phase contained more information about the contralateral eye than amplitude, an observation that was confirmed by the analysis of the N170 peak latency and amplitude presented below. This phase advantage is likely due to the more robust single-trial measure of the transients forming the ERP provided by phase. Indeed, phase is sensitive to local maxima and minima that might be smaller in amplitude than other nontask related fluctuations in the single-trial signal. 

In sum, the forward analysis has demonstrated that in a face detection task, brain activity is mostly associated with the presence of the eyes. This association is maximum at lateral electrodes and shortly before the peak of the N170. 

Reverse analysis

In the forward analysis presented in the previous section, we used the single-trial ERP distributions to reveal the image features statistically associated with their variability. However, it is possible that this approach could mask sensitivity to other features (Smith et al., 2004). So we also used a reverse approach, in which we start from the image features, to infer how their presence modulates single-trial ERP distributions. To avoid having to define facial features of interest, we employed a systematic, unbiased approach in which the face stimuli were split in 16 nonoverlapping horizontal bands that covered the whole face (Figures 8–9). Then we used the single-trial bubble masks and correlated each mask with each face band—where high correlation means presence of face information from this face band on that trial. To confirm that single-trial ERP distributions carry mostly information about the eyes, for each band, we then computed MI between these correlation values and the single-trial ERP distributions, separately for face and noise trials—MI(ERP, BAND). This enabled a fine breakdown of the time course of single-trial ERP variability as a function of location of information in the face—i.e., in the 16 face bands. MI(ERP, BAND) was maximum in bands around the eyes and eyebrows (Figure 8, Column 1, Band 7 in black, and Band 6 in gray). Analyses performed using two, four, and eight bands also showed a systematic bias for the band(s) containing the eyes. 

Figure 8

 

MI(ERP, BAND) and ERP modulations. The six rows correspond to the six electrodes of interest. Columns 1 and 2 show the face trials, and Columns 3 and 4 the noise trials. MI, in Columns 1 and 3, is color coded with one color per band, as shown in the oval inset. A larger version of the color code is available in Figure 9. The ERP modulation associated with Band 7 is shown in Columns 2 and 4. For face trials, but not for noise trials, the N170 becomes larger and leftward shifted as band activity increased from minimal in cyan (Bin 1), to maximum in magenta (Bin 10). In Column 2, top two plots, the dashed vertical lines mark the transition between the P1 and the N170, the critical time window in which we observed sensitivity to the eyes.

Figure 9

 

N170 and behavior band modulations. The two top plots show, at left and right lateral electrodes, the latency and amplitude differences between Bin 10 ERPs and Bin 1 ERPs, for every band. Amplitude differences are expressed in proportion of the Bin 1 ERP amplitudes: for instance, a difference of −0.7 means that on average Bin 10 ERPs were 170% the size of Bin 1 ERPs. The two lower plots show, for face trials (left) and noise trials (right), the reaction time and percent correct differences between Bin 10 and Bin 1, for every band. In all plots, circles indicate the mean amplitude and median latency across subjects, and the bars mark the 95% confidence intervals.

Having confirmed that bands around the eyes are the strongest modulators of single-trial ERPs, we applied a reverse analysis to reconstruct the mean ERPs associated with presence of face information in every band (Figure 8, Column 2). To compute these ERPs, we split the correlations between bubble masks and bands into 10 bins, resulting in a 16 (Face Bands) × 10 (Correlation Bins) matrix. Each cell of the matrix comprises the ERP single trials that we average to create mean ERPs. For each face band, we can examine how visibility covaries with these mean ERPs. 

Figure 8 illustrates how the presence of Band 7 was associated with earlier and larger N170 at the two lateral electrodes. Weaker effects were observed at midline electrodes, and also corresponded to changes in latency and amplitude of their predominant ERP peak. Importantly, these results were not observed for noise trials, except for a similar sensitivity to bands around the mouth/chin at the two lowest midline electrodes—suggesting again that this is a low-level effect. 

To go beyond the qualitative description in Figure 8, we confirmed that the eye bands triggered earlier and larger N170s by measuring the peak amplitude and latency of the mean N170 in every subject and every band. We measured the N170 at LE and RE for lowest (Bin 1 of the bubble mask by band correlations) and highest (Bin 10) ERPs. We defined N170 as the first local minimum after 120 ms in ERPs low-pass filtered at 20 Hz using a fourth order Butterworth noncausal filter. This analysis revealed significantly earlier N170 latencies and larger amplitudes in trials in which Bands 6 and 7 were strongly active, compared to trials of weak activation (Figure 9). At Band 7, which showed the strongest effects, the N170 at LE was 11 ms [6, 19] earlier in trials of highest band activation, compared to trials of lowest activation. At RE the difference was 14 ms [5, 21]. Also, amplitudes increased by about 169% [149, 195] at LE and 177% [157, 206] at RE. Bands 5 and 8 were associated with a much weaker version of this pattern. This weaker pattern could be due to the overlap between the Gaussian apertures in those bands and the neighboring eye bands. The other bands showed opposite and much weaker effects to those observed at Bands 6 and 7: The N170 peaked later and with lower amplitude in trials in which they were more strongly active. Because of the fixed and small number of bubbles, this opposite effect for the bands distant from eye bands reflects the absence of the eye area. 

We also performed the face band analyses on behavioral responses. For face trials only, the presence of the different bands affected RT and percent correct, following a pattern very similar to that observed for the N170: the four bands around the eyes were associated with faster and more accurate responses—for Bands 6 and 7 around 60 ms faster and 20% more accurate, whereas the other bands had the opposite effect. The RT effect confirms the direction of the dependence between eye information and RT, depicted as MI(PIX, RT) in Figure 3. The effect on percent correct is somewhat unexpected because the forward analysis suggested effects in only a few subjects. This discrepancy demonstrates the importance of the reverse analysis to get a full description of the effects. 

In sum, the reverse analysis complements the forward analysis and demonstrates that the presence of pixels around the eyes is associated with earlier and larger N170s in both hemispheres, as well as faster and more accurate behavioral responses. 

Forward analysis: Single-trial N170s

Finally, based on the reverse analysis, we predicted that the latency and amplitude of the N170 might code for the presence of the contralateral eye. To test this prediction, we perform a new forward analysis in which we estimated the pixel information content of the distributions of single-trial N170 latencies and amplitudes, MI(PIX, N170 LAT) and MI(PIX, N170 AMP). We used a cross-correlation approach to estimate the single-trial N170 latencies and amplitudes. First, we used the average ERP across all bubble masks as a template. Second, we looked for the minimum template value in the search window 110 ms to 230 ms. We then cross-correlated each single-trial with the ERP template, in the time-window ranging from 100 ms before to 100 ms after the latency of the template local minimum. The latency of the maximum cross-correlation defined the N170 latency, and its amplitude was measured as the value at that latency. Both latency and amplitude were modulated by the presence of the contralateral eye (Figure 10). The N170 latency carried significantly more information than the N170 amplitude, particularly at RE. 

Figure 10

 

MI(PIX, N170 LAT) & MI(PIX, N170 AMP). Results are shown averaged across subjects at the left lateral electrode (LE, Column 1) and the right lateral electrode (RE, Column 2). The matching frequencies of significant effects are shown in Columns 3 and 4. Row 1 shows MI(PIX, N170 LAT), Row 2 shows MI(PIX, N170 AMP), and Row 3 shows the difference between Row 1 and Row 2. In every frequency plot in Columns 3 and 4, the top left corner indicates the maximum number of subjects showing a significant effect at the same pixel, whereas the top right corner indicates the number of subjects showing significant effects at any pixel.

Finally, N170 latency and amplitude were associated with RT, with significant MI in most subjects at LE and RE (Table 1). There was also an association between the N170 and percent correct in a few subjects. 

Table 1

 

MI(N170, RT) and MI(N170, CORRECT). Results are presented separately for N170 latency (LAT) and amplitude (AMP). Confidence intervals (CI) are indicated in square brackets. Frequency refers to the number of subjects showing a significant effect.

	LE		RE
	Mean + CI	Frequency	Mean + CI	Frequency
MI(RT, LAT)	0.0361 [0.0168, 0.0629]	12	0.0447 [0.0270, 0.0692]	15
MI(RT, AMP)	0.0216 [0.0089, 0.0362]	11	0.0265 [0.0146, 0.0398]	14
MI(CORRECT, LAT)	0.0022 [0.0010, 0.0040]	3	0.0022 [0.0013, 0.0033]	2
MI(CORRECT, AMP)	0.0028 [0.0015, 0.0044]	5	0.0027 [0.0018, 0.0038]	4

In sum, N170 latency and amplitude are both associated with the presence of the contralateral eye, with a stronger association for latency. This result suggests a mechanism in which, in a face detection task, the N170 latency encodes mostly the presence of the contralateral eye. Availability of diagnostic information (presence of the left eye) was associated with earlier and larger N170s, which in turn were associated with faster RT. However, the link between N170 and RT is only correlational here, and both variables are correlated with the presence of the eyes, a triangular relationship difficult to interpret. 

Discussion

In a face detection task, we found that both behavior and neuronal activity were mostly modulated by the presence of the eyes: Behavioral responses were faster and more accurate when the left eye was present; the N170 ERP component, recorded at lateral electrodes, was both larger and peaked earlier in response to the contralateral eye. Earlier and larger single-trial N170s were also associated with faster reaction times. The N170 was also modulated, in a few subjects and to a lesser extent, by the ipsilateral eye. Other face features had virtually no effect on behavior or the N170. Overall, our analyses point to the N170, and in particular the transition between the P1 and the N170, as a critical time window in which the brain extracts task-relevant information: in this case the presence of the contralateral eye. Although both eyes were processed contralaterally, their presence or absence was not equally weighted: The presence (or absence) of the left eye largely dominated modulations of the reaction time distributions. The right eye, while containing identical potential information, only modulated reaction time distributions in a few subjects. This suggests that the left eye is the preferred stimulus for face detection, which is consistent with the reported right hemisphere dominance for face processing (Sergent, Ohta, & MacDonald, 1992), as well as for attention in general (Driver & Vuilleumier, 2001). 

The independent processing of the contralateral eye by each hemisphere, observed here and in previous studies, echoes findings of a division of labor between the two hemispheres in human and nonhuman primates (Kravitz, Vinson, & Baker, 2008; Rousselet, Thorpe, & Fabre-Thorpe, 2004). This result is also consistent with the observation of independent variability of single-trial N170 amplitudes in the two hemispheres (Nguyen & Cunnington, 2014). The apparent independence of the two hemispheres, and their sensitivity to mostly one feature—the contralateral eye, is not due a lack of sensitivity of our technique, or the masking of other features by the predominant eyes. First, we used two unbiased and systematic approaches, forward and reverse analyses to establish feature sensitivity. Second, as in a previous reverse-correlation experiment (Schyns et al., 2011), we found sensitivity to the two eyes and the mouth at midline electrodes. However, in the present experiment, the mouth sensitivity might be due to low-level factors, because it was also present for noise trials, and occurred earlier and was weaker than contralateral eye sensitivity. Sensitivity to the two eyes could reflect explicit feature integration, or passive summation of eye-related activity from the two hemispheres at the scalp level. Reverse correlation techniques explicitly testing feature integration would be necessary to resolve this question. For instance, EEG frequency tagging could be used to provide objective signatures of integration of specific features (Boremanse, Norcia, & Rossion, 2013). This would require testing a large number of combinations of features, parts or aspects of a face, which could be done by combining EEG frequency tagging with the sort of systematic sparse sampling we used in the current experiment. 

Sensitivity to the contralateral eye tended to be larger for the N170 latency than its amplitude. Similarly, after time-frequency decomposition of the single-trial ERPs, we found that phase contributed more information than amplitude, confirming results from a recent report (Schyns et al., 2011). Because the presence of the contralateral eye is associated with a leftward shift of the N170, it means that ERP amplitudes start to vary strongly before the peak of the N170, and more precisely at the transition between the preceding P1 and the N170. Indeed, we found that contralateral eye sensitivity was maximum before the N170 peak, which speaks against analyses limited to ERP peaks (Rousselet & Pernet, 2011; Rousselet, Pernet, Caldara, & Schyns, 2011; Schyns et al., 2007). 

Many studies using categorical designs have reported early face responses or adaptation of these responses to eyes or the eye area (Bentin et al., 1996; Eimer, Kiss, & Nicholas, 2010; Harris & Nakayama, 2008; Itier et al., 2007; McCarthy, Puce, Belger, & Allison, 1999; Nemrodov & Itier, 2011). Reverse-correlation studies in humans go a step further, by showing that irrespective of the task, early face responses seem to first encode the presence of the contralateral eye, followed by sensitivity to task-relevant features, and their combinations (Schyns et al., 2003; Schyns et al., 2007; Schyns et al., 2011; Smith et al., 2004, 2007; van Rijsbergen & Schyns, 2009). This two-step process has led to the suggestion that face processing could start with the eyes, used as anchors to localize and integrate information from the rest of the face (Schyns et al., 2007). This is akin to the idea of using pre-alignment to help template-matching routines (Karlinsky, Dinerstein, Harari, & Ullman, 2010; Ullman, 1996). In our case, the eyes are sufficient to detect a face, and hence the integration process stops there. A link between eye processing and detection is consistent with a recent reverse correlation study in monkeys in which single-unit and multi-unit recordings from the left posterior and middle lateral face patches revealed strong sensitivity to the right eye (Issa & DiCarlo, 2012). This modulation by the contralateral eye was present in most neurons during their first period of activity and is consistent with the idea that diagnostic features are first encoded in posterior IT cortex (Nielsen, Logothetis, & Rainer, 2006b). It remains to be determined if this early coding of the contralateral eye is task dependent, and what these neurons process over longer periods. In particular, the actual computational goal of the system responding strongly to eyes remains unclear: Instead of eye detection per se, the response of the putative eye detection neurons could reflect an early bias towards information present in the eyes, such as emotion or gaze. Diagnostic information is always task relative: We cannot assume that the information processing goals will not adapt to task demands and the availability of stimulus information (Schyns, 1998). 

Is it possible that the N170 coding of the contralateral eye is equivalent to the neuronal responses from posterior and middle lateral patches in macaque monkeys? Even though the equivalence between monkey and human face areas is largely unknown, the early activation and location of the monkey lateral patches suggest that they could reflect activity around the STS or OFA in our subjects (Janssens, Zhu, Popivanov, & Vanduffel, 2014; Yovel & Freiwald, 2013). In our data, the topographic maps of the contralateral eye sensitivity suggest the involvement of posterior-lateral sources. Furthermore, studies using source analyses or correlations between BOLD and ERP amplitudes suggest N170 sources around the STS (Itier & Taylor, 2004; Nguyen & Cunnington, 2014; Sato, Kochiyama, Uono, & Yoshikawa, 2008; Watanabe, Kakigi, & Puce, 2003), the fusiform gyrus (Horovitz, Rossion, Skudlarski, & Gore, 2004), or both (Dalrymple et al., 2011; Prieto, Caharel, Henson, & Rossion, 2011; Sadeh, Podlipsky, Zhdanov, & Yovel, 2010). However, none of these studies has linked BOLD information content to ERP information content. A single-trial fMRI-EEG experiment using bubbles would help solve this problem. More directly, intracranial data also support the involvement of occipital and temporal lateral areas, such as the right inferior occipital gyrus, in generating the scalp N1/N170 (Engell & McCarthy, 2011; Jonas et al., 2012; Jonas et al., 2014; Rosburg et al., 2010; Sehatpour et al., 2008). In addition, an MEG reverse-correlation study revealed sensitivity to face features, including the eyes, in the time window of the M170 in lateral cortical areas (Smith, Fries, Gosselin, Goebel, & Schyns, 2009). So, it seems plausible that lateral sources are involved in the generation of the N170 contralateral eye sensitivity. But the timing of this sensitivity differs dramatically between monkeys and humans: It started around 60 ms and peaked around 100 ms in monkeys (Issa & DiCarlo, 2012), whereas in humans the effects started around 100 ms and peaked at about 160 ms. The delay between human and monkey eye sensitivity could be explained by differences in brain sizes. For instance, if one follows the 3/5 rule (Kelly, Vanegas, Schroeder, & Lalor, 2013), the timings agree very well between species. Also, we observed later ERPs in bubble trials compared to trials without bubbles, which could be due lower stimulus energy in the bubble trials, and would fit with recent observations of strong N170 latency sensitivity to luminance (Bieniek, Frei, & Rousselet, 2013). Alternatively, delayed ERPs in bubble trials might reflect extra processing required for occluded stimuli (Harris & Aguirre, 2008; Tang et al., 2014) and suggest the possibility that stimulus occlusion, necessary to infer information content through reverse correlation, could alter how faces are processed. An alternative to the 2-D sampling used in the current experiment would be to use 3-D bubbles, in which the sampling is performed independently in different spatial frequency bands (Schyns et al., 2007; Schyns, Petro, & Smith, 2009). This technique allows the presentation of extended face areas at lower spatial scales, and hence preserves a constant face context, without complete occlusion of face features. Nevertheless, whatever the origin of the delay, it cannot explain inter-species differences, unless bubbles have a stronger effect on scalp ERPs than single-unit recordings. But this is rather speculative at this point, especially given that Issa and DiCarlo used a single large Gaussian aperture per trial, compared to 10 smaller apertures in our experiment. The effect of these differences in sampling strategies on neuronal timings remains to be tested. Alternatively, the N170 coding of the contralateral eye could reflect a stage at which this information is available in areas that do not contribute to eye detection per se, but rather process task-related features, possibly involving re-entrant connections from other face areas (Rossion et al., 2003; Tang et al., 2014). 

It would be necessary to carry out studies using the same stimuli and the same tasks in both monkeys and humans (Nielsen, Logothetis, & Rainer, 2006a, 2008) to establish the equivalence of the N170 eye response to the response from monkeys' lateral patches, and more generally to firmly establish differences in timing, location, and information content of visual brain activity. This work could also be extended to other species (Alemi-Neissi, Rosselli, & Zoccolan, 2013; Gibson, Lazareva, Gosselin, Schyns, & Wasserman, 2007; Gibson, Wasserman, Gosselin, & Schyns, 2005). Nevertheless, in keeping with results from Issa and DiCarlo (2012), our results support a bottom-up, data-driven model of face processing, in which the presence of a face is first inferred by detecting the contralateral eye, before integrating other task-relevant features. Following Issa and DiCarlo, next we will need to determine if the encoding of the contralateral eye is tolerant to changes in contrast, size, position, and orientation. A full account of face detection mechanisms should also consider the impact of cortical magnification and fixation location (de Lissa et al., 2014; Nemrodov, Anderson, Preston, & Itier, 2014; Rousselet, Husk, Bennett, & Sekuler, 2005; Zerouali, Lina, & Jemel, 2013). Also, by testing the same subjects using the same stimuli in various tasks (e.g., detection, gender, expression, age discrimination tasks) we will be able to test a clear prediction of systematic contralateral eye sensitivity followed by task specific sensitivity to diagnostic information (Schyns et al., 2007). Finally, it might be possible to establish a causal relationship between feature sensitivity and brain activity: TMS has been used to determine the contribution of lateral visual areas to behavior and the N170 to different categories (Pitcher, Charles, Devlin, Walsh, & Duchaine, 2009; Sadeh et al., 2011). Using reverse correlation, it would be possible to go one step further, by providing a mechanistic account of how TMS affects the shape and the information content of the N170, and its relationship to behavior. 

Acknowledgments

This research was supported by the Leverhulme Trust grant F/00 179/BD to GAR and the BBSRC grant BB/J018929/1 to GAR, PGS, and NJV. We thank Hanna Isolatus, Sean Henderson, Fei Li, and Magdalena M. Bieniek for their help with data collection. 

Commercial relationships: none. 

Corresponding author: Guillaume A. Rousselet. 

Email: [e-mail]. 

Address: Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, UK.