Event-related potentials reveal an early advantage for luminance contours in the processing of objects

Abstract

Detection and identification of objects are the most crucial goals of visual perception. We studied the role of luminance and chromatic information for object processing by comparing performance of familiar, meaningful object contours with those of novel, non-object contours. Comparisons were made between full-color and reduced-color object (or non-object) contours. Full-color stimuli contained both chromatic and luminance information, whereas luminance information was absent in the reduced-color stimuli. All stimuli were made equally salient by fixing them at multiples of discrimination threshold contrast. In a subsequent electroencephalographic experiment observers were asked to classify contours as objects or non-objects. An advantage in accuracy was found for full-color stimuli over the reduced-color stimuli but only if the contours depicted objects as opposed to non-objects. Event-related potentials revealed the neural correlate of this object-specific luminance advantage. The amplitude of the centro-occipital N1 component was modulated by stimulus class with the effect being driven by the presence of luminance information. We conclude that high-level discrimination processes in the cortex start relatively early and exhibit object-selective effects only in the presence of luminance information. This is consistent with the superiority of luminance in subserving object identification processes.

Introduction

Detection and identification of objects is the most crucial goal of human visual perception. Vision operates through continuous, distributed processes involving multiple parallel streams which are integrated at several hierarchical stages in order to achieve a stable representation of the environment (see Schroeder, Mehta, & Givre, 1998). Low-level vision analyses elementary features such as local color, luminance, motion or binocular disparity while mid-level processes establish a layering of these features into surfaces and perform figure-background organization. High-level processes such as object or face recognition are largely dependent on semantic input (Palmer, 1999). 

Object identification is an extremely efficient and rapid process. The feed-forward stream of visual processing takes around 100 ms to reach high-level representational areas in the temporal cortex and this is sufficient for coarse and rapid categorization (Liu, Agam, Madsen, & Kreiman, 2009). In humans, such classification processes have been demonstrated for both objects (Rousselet, Fabre-Thorpe, & Thorpe, 2002; VanRullen & Thorpe, 2001; but also see Rousselet, Mace, Thorpe, & Fabre-Thorpe, 2007) and faces (Mouchetant-Rostaing, Giard, Bentin, Aguera, & Pernier, 2000). The speed of object recognition seems to be largely driven by luminance information. The magnocellular pathway rapidly projects initial shape information obtained from luminance detectors into the prefrontal cortex which subsequently constrains processing in posterior representational areas (Bar, 2003; Bar et al., 2006). Luminance contours serve another important role: they allow object representations to enact an early influence on figure-ground organization which ensures that objects are preferentially assigned figural status (Peterson & Gibson, 1993, 1994b). 

In low-level vision, separate visual channels process chromatic and achromatic information. Broadly speaking, three different channels are distinguished at the level of the lateral geniculate nucleus: the magnocellular pathway processes luminance information only while parvo- and koniocellular pathways predominantly subserve color processing (for a review, see Kulikowski, 2003). The parvocellular pathway processes information from L and M cones, and is sensitive to chromatic but also to luminance information, depending on the spatial scale (Reid & Shapley, 2002). Recent physiological studies have revealed subdivisions within the koniocellular pathway itself, with its middle layers involved in S-cone information processing (Hendry & Reid, 2000; Tailby, Solomon, & Lennie, 2008). 

While it is clear that both chromatic and achromatic information can in principle be used for low-level spatial tasks such as orientation discrimination (Webster, DeValois, & Switkes, 1990; Wuerger & Morgan, 1999), the contributions of chromatic and achromatic information in everyday visual processing differ, with luminance being more dominant for rapid processing of lines, edges, shape and motion and color being more relevant for segmentation of visual scenes (Gegenfurtner & Rieger, 2000) or for recognizing color diagnostic objects (Martinovic, Gruber, & Müller, 2008; Naor-Raz, Tarr, & Kersten, 2003). The role of these pathways in the processing of object form has not been extensively studied as yet. 

To our knowledge there is only one functional magnetic resonance imaging (fMRI) study which has examined processing of chromatic and achromatic object contours—thus looking at the role of different parallel pathways in high-level vision (Kveraga, Boshyan, & Bar, 2007). This study compared responses to line drawings of objects designed to bias processing either towards magnocellular or towards parvocellular pathway by low luminance contrast or isoluminant red–green stimuli. Using dynamic causal modeling, the study demonstrated that top-down facilitation in object recognition is triggered through a projection of magnocellular information from the occipital cortex to the orbitofrontal cortex and then back to the fusiform cortex. Parvocellular-biasing stimuli on the other hand elicited activity that spread from the occipital cortex directly into the fusiform cortex and thus seemed to be more related to a feed-forward bottom-up representational process. 

In our own study, the aim was to identify the timecourse of luminance's special role in object representation, demonstrated by Kveraga et al. (2007). EEG studies on the role of different visual pathways in high-level vision (i.e. object representation) would have the advantage over fMRI as they can directly assess the time-course of cortical activity driven by information along different visual pathways. Therefore, we employed EEG to identify the early markers of neural processing elicited by luminance and chromatic information and investigate their differential roles in high-level vision. We expect that the relatively early perception-related event-related potential (ERP) component N1 will occur at an earlier latency and show object-sensitive amplitude effects when luminance is included. N1 is generated in a broad network of cortical areas and perception-related effects during its window can be localized to extrastriate areas of occipito-temporal cortices (Plomp, Michel, & Herzog, 2010). It is related to visual discrimination processes within the focus of attention (Vogel & Luck, 2000) as well as to the detection of object features and their classification into certain expert categories (e.g., faces; Allison, Puce, Spencer, & McCarthy, 1999; Schendan, Ganis, & Kutas, 1998). Selective modulations of the N1 by the inclusion of luminance information would indicate that the presence of luminance in the image can lead to benefits in representational processing in a relatively early time window and would provide further support for models that posit its preferential role in object identification. 

Methods

Participants

Twelve healthy subjects (6 male; age 21–40 years) with normal or corrected-to-normal vision gave written informed consent to take part in this study. All participants had normal color vision as assessed with the Cambridge Colour Test (Regan, Reffin, & Mollon, 1994). The participants received a small honorarium to compensate for their time. The study conformed to the ethical guidelines of the University of Liverpool. 

Apparatus

The experiments were run on a DELL PC equipped with a dedicated visual stimulus generator (ViSaGe, Cambridge Research Systems, Ltd., Kent, UK). Stimulus presentation was controlled using Matlab (Mathworks, Natick, Massachusetts) and the stimuli were presented on a Mitsubishi 2040u 21 inch CRT monitor. The chromatic and luminance output of the monitor were calibrated using the CRS calibration system (ColourCAL, Cambridge Research Systems, Ltd., Kent, UK); the accuracy of the calibration was verified with a spectroradiometer (Photo Research PR650, Glen Spectra Ltd., Middlesex, UK). The monitor had been switched on for at least 30 minutes before any experiment. Participants responded via a button box (Cedrus RB-530, Cedrus Corporation, San Pedro, USA) and were seated 57 cm from the screen in an otherwise dark electrically shielded chamber. 

Color space

To describe the chromatic properties of our stimuli, we use the DKL-color space (Brainard, 1996; Derrington, Krauskopf, & Lennie, 1984) which is an extension of the MacLeod–Boynton chromaticity diagram (MacLeod & Boynton, 1979). In this space, any color is defined by modulations along three different ‘cardinal’ axes: along the achromatic axis, all three cone classes (L, M and S) are modulated such that the contrast is identical, that is, ΔL/ΔL _BG = ΔM/ΔM _BG = ΔS/ΔS _BG , where ΔL, ΔM, and ΔS denote the incremental cone excitations in three cone classes, respectively. L _BG , M _BG and S _BG indicate the L-, M-, and S-cone excitations of the background. The second direction refers to a modulation along a red–green axis; modulations in this direction leave the excitation of the S cones constant (i.e. ΔS = 0), and the excitation of the L and M cones covaries as to keep their sum constant. Therefore, this axis if referred to as a “constant S-cone axis” (Kaiser & Boyton, 1996), or a “red–green isoluminant” axis (Brainard, 1996). Along the third axis, only the S cones are modulated, and ΔL = ΔM = 0. Therefore, this axis is often referred to as a “constant L & M cone” axis (Kaiser & Boyton, 1996), or as an “S-cone isoluminant” axis (Brainard, 1996) or as a “tritanopic confusion line”. 

Instead of defining the chromatic properties of a stimulus by their respective L-, M-, and S-cone modulations, the stimulus can be defined in terms of the responses of a set of hypothesized post-receptoral mechanisms that are isolated by these cardinal color modulations (Brainard, 1996; Derrington et al., 1984; Eskew, McLellan, & Guilianini, 1999; Ruppertsberg, Wuerger, & Bertamini, 2003; Wuerger, Watson, & Ahumada, 2002). The three corresponding mechanisms are two cone-opponent color mechanisms and a luminance mechanism (see Figure 1a). One of the two cone-opponent mechanisms is a red–green mechanism that takes the weighted difference between the differential L- and the M-cone excitations. The second cone-opponent mechanism is a yellowish-violet mechanism that takes the weighted difference between the differential S-cone and the summed differential L- and M-cone excitations. The luminance mechanism sums the weighted differential L- and M-cone signals. These orthogonal mechanisms are often referred to as “L + M”, “L − M”, “S − (L + M)” (Derrington et al., 1984). 

Figure 1

 

(a) The chromaticities of stimuli in the DKL color space. Along the achromatic axis, cone contrasts in all three cone classes vary (L + M + S). Along the L − M axis, only the difference between L- and M-cone varies, keeping L + M constant. Along the S-axis, only the S cones vary. Colors along the S-cone-isolating line range from violet to yellowish; intermediate isoluminant colors range from magenta to greenish; addition of an achromatic component results in stimuli ranging from bright magenta to dark greenish. (b) Examples of stimuli: objects and non-objects, represented in colors that excite different directions.

In this paper we will define the chromatic properties of the stimuli in terms of their modulations in the L, M and S cones (i.e. as ‘S’; ‘L − M’; ‘L + M + S’), as described in the first paragraph. 

The CIE coordinates of the gray background are as follows: x = 0.296, y = 0.309 and Lum = 46.3 cd/m²; the corresponding cone coordinates are: L = 30.08, M = 16.54 and S = 0.95. Along the “L − M” direction, red is indicated by 0 deg and at a radius of 1 (cf. Figures 1 and 3) its absolute cone coordinates are: L = 28.32, M = 18.28 and S = 0.98; 180 deg refers to green (at r = 1, L = 31.88, M = 14.32 and S = 0.97). Along the “S” direction, 90 deg represents violet (at r = 1, L = 30.46, M = 16.54, and S = 1.67), and 270 deg indicates yellowish (at r = 1, L = 30.71, M = 16.49, and S = 0.28). Therefore, for a radius of 1, the resulting L-, M- and S-cone contrasts along the L–M axis are as follows; for modulations from gray to red: 0.06 (L), −0.12 (M), 0.0 (S); for modulations from gray to green: −0.06 (L), 0.13 (M), 0.0 (S). For positive S-cone modulations (between gray and violet) the L-, M- and S-cone contrasts corresponding to a radius of 1 are: 0.0 (L), 0.0 (M), 0.7 (S); negative S-cone modulations (between gray and yellow-green) of radius 1 yield a negative S-cone contrast of −0.7 and zero L- and M- contrasts. Contrast thresholds were expressed in terms of radius (see Results). 

Stimuli

The stimuli were taken from existing stimulus sets that contain line drawings of common objects (International Picture Naming Project with 525 pictures—Bates et al., 2003; 400 pictures from a French-language naming study—Alario & Ferrand, 1999; and 152 images used in object recognition studies—Hamm, 1997). A set of 225 objects was selected for use in the baseline experiment while a set of 168 objects was selected for use in the main experiment. The images represented common, familiar and easily nameable objects from various semantic categories: ship, stapler, harmonica, grasshopper, etc. (for a full list, see Supplementary Material A). Non-objects were produced by manipulating images of objects so that they became unrecognizable but still maintained approximately the same aspect ratio and closed line structure that is characteristic of objects. Images were scrambled using the image distorting functions of the GIMP software. One of the authors scrambled the images. After scrambling, the image was assessed for closed line structure and aspect ratio and if the scrambling procedure resulted in a violation of these rules it was edited by hand in order to make it approximately similar in aspect ratio and with a closed line structure. Afterwards the images were all transformed to JPEGs and their sizes were compared. JPEG file size provides an objective estimate of visual complexity for line drawings that has been used in picture naming studies (Szekely & Bates, 2000), including the normative set provided by Bates et al. (2003). Where there were discrepancies in size present, the non-object images were edited by hand to reduce the number of inner contours while maintaining an object-like structure. In the final stimulus set, there were no differences in visual complexity between objects and non-objects (t(167) = 1.63, n.s.). It was also necessary to assess the low-level differences in object and non-object images by running a permutation analysis of their Fourier spectra. This analysis, using 1000 permutations, revealed that although images of objects contained more cardinally oriented lines than images of non-objects, these differences were not significant. 

In the experiments, object and non-object contours were defined along three directions in DKL color space: 1) S-cone-isolating (S), or 2) intermediate isoluminant (S and L − M), or 3) a full-color stimulus with an additional achromatic component (S; L − M; L + M + S), providing a luminance signal (see Figure 1). For each direction, both increments and decrements were used in order to obtain a signal that is representative for the whole direction (see Figure 1; data were collapsed across increments and decrements in the final analysis). Thus, the stimuli either involved processing predominantly in the koniocellular pathway (S-cone defined contours), or in both chromatic pathways (konio- and parvocellular), or in all three visual pathways (full color images including chromatic and achromatic information: konio-, parvo- and magnocellular). 

The majority of the stimuli subtended a visual angle of approx. 5° × 2° (smallest stimulus was around 3° × 1°, biggest stimulus was around 9° × 3.5°) and where shown on a gray background. Stimulus onset was synchronized to the vertical retrace of the monitor. Stimulus presentation was balanced across the sample to control for item-specific effects: thus, across the sample, each item was presented equally often with edges defined along each of the three directions of the DKL color space. 

Static random luminance noise was superimposed over the stimulus display area in the form of 3 × 3 pixel elements modulated at an RMS noise contrast of 19.5% (Ruppertsberg et al., 2003). The noise was added to each trial starting with the fixation cross preceding the stimulus presentation—this avoided luminance-onset related components in the ERP which would have been elicited by the noise had it coincided with the stimulus onset. The purpose of the noise was to diminish luminance-related activity due to chromatic aberration, which would be inevitable for isoluminant stimuli with high-frequency edges. 

Observer isoluminance

There are significant individual differences in the luminous efficiency (V(λ) Wyszecki & Stiles, 2000) which may result in a small luminance signal being present in the nominally isoluminant chromatic signals. In order to adjust for observers' individual point of isoluminance, we used heterochromatic flicker photometry (Walsh, 1958) for each individual observer, following the same procedure as described in Ruppertsberg et al. (2003) and Wuerger, Ruppertsberg, Malek, Bertamini, and Martinovic (2011). 

The display altered between a red and a green stimulus at a 20 Hz frequency. By varying the luminance of red and green in opposite directions, it is possible to find a setting for which the perception of flicker is minimal. HCFP utilizes the fact that the chromatic system is too slow to follow fast temporal changes but the luminance system is able to detect the fast changing luminance differences between red and green. Therefore, if the perception of flicker is minimized the luminance difference is minimized as well. Since temporal and spatial factors may affect the individual isoluminance point, we used randomly chosen objects from the 225 items in the preliminary psychophysical experiment set. All observers reported that they could find a setting where the flickering stimulus became the least visible. Each observer repeated this selection ten times reliably, the lowest and highest values were eliminated and the average of the remaining eight selections was taken. 

Procedure

Initially a two hour experimental session was conducted with each participant, consisting of control measurements (Cambridge color vision test and Heterochromatic Flicker Photometry) and the baseline psychophysical experiment. The purpose of the baseline threshold experiment was to equate stimulus salience between different directions in the color space based on obtained measurements of individual color contrast discrimination thresholds. Stimulus contrast in the main experiment was adjusted upwards towards monitor's gamut relative to these individual thresholds in order to provide adequate EEG signal. This was done by setting the contrast in the direction with the highest measured threshold to the value of r = 1.2 (maximum of monitor's gamut for S-cone decrement, which was the lowest limit out of all the used directions) and then adjusting all other contrasts upward from threshold using the scale factor provided by dividing 1.2 with the highest measured threshold. This procedure was intended to allow for an adequate signal-to-noise ratio in the EEG while maintaining equal salience along different color directions. 

In the baseline two-interval forced choice (2IFC) experiment, a trial started with a fixation cross for 500 ms presented in the center of the screen (see Figure 2a). Then the first item was displayed for 700 ms, followed by another fixation cross for 500 ms, and finally the second item appeared for 700 ms. Participants were required to respond after the second item by pressing a button to indicate the interval in which they thought the object had appeared. The next trial started after the participant had responded. Participants were instructed to give a correct answer, not a fast answer, and they were provided with acoustic feedback. 

Figure 2

 

(a) Trial outlook for the baseline experiment. Participants responded if the object was located in the first or the second interval. (b) Trial outlook of the EEG experiment. Participants responded whether the presented item was an object or a non-object.

The participant's responses guided an adaptive QUEST procedure of the signal contrast (Watson & Pelli, 1983). To estimate the color contrast threshold from the relative frequency of a correct response a Weibull function was fitted. Threshold was defined as the 81% correct point on the psychometric function. Thresholds in each of the tested color directions were measured five times for every participant. The mean threshold data from all participants is shown in Figure 3; the colors on the polar plots correspond to those depicted in Figure 1a; for polar plots that contain threshold contrasts and scaled contrast values for each observer separately see Supplementary Material B. For the full-color stimulus (bright magenta; dark green) the graph only shows the projection of the thresholds onto the isoluminant plane. Differences between increment and decrement thresholds were analyzed using paired t-tests. 

Figure 3

 

Results from the baseline threshold experiment for all 12 observers. (a) Threshold plot: the Xs indicate the contrasts at which the observers could reliably discriminate line-drawings of objects from non-objects. Only components in the isoluminant plane (S vs. L − M) are shown; the achromatic component (L + M + S) of the thresholds is not shown in this figure, but is used for scaling purposes. (b) Scaled threshold plot: suprathreshold contrast values that were used in the EEG experiment.

On a subsequent day, each participant took part in an EEG experiment lasting approximately one hour. An established paradigm that requires discrimination of line-drawings of familiar, real-life objects from scrambled images of such objects, so-called non-objects was employed (Gruber & Müller, 2005). Participants were presented a single item and made a judgement whether the shown image represented an object or a non-object (see Figure 2b). Prior to the commencement of the recording, the participants first performed a practice block of 20 trials that contained a subset of stimuli not used in the experimental trials in order to become familiarized with the task. A total of 336 trials were distributed in four 84 trial blocks each lasting approximately four minutes each. A trial started with a variable baseline period (550–750 ms) of fixation. Afterwards the stimulus was displayed for 700 ms, followed by a fixation cross displayed for 1000 ms. Participants responded with a button press, indicating if the presented stimulus was an object or a non-object. Button-to-response allocation was balanced across participants. Participants were instructed to minimize eye movements and blinking during the display of a stimulus or the fixation cross and to try and remain relaxed and refrain from body or head movements throughout the experiment as these could have an adverse effect on EEG signal. At the end of each trial, an ‘X’ would replace the fixation cross for 900 ms. Participants were instructed that they should refrain from blinking except during the presentation of the ‘X’. 

In the EEG experiment accuracies and RTs were also collected and analyzed. For trials with correct responses RTs between 300 and 1700 ms (the maximum time allowed for responses) were taken into further analysis. Median RTs for correct items were computed for each participant. Means across participants were then computed to obtain a measure of central tendency known as a mean of median RT. On the basis of the preliminary experiment data was collapsed across increments and decrements within each color direction. Differences in accuracies and RTs between the conditions were analyzed with a 3 × 2 repeated measures ANOVA with factors direction in color space (S-cone isolating, intermediate isoluminant, full color) and objecthood (object, non-object). Greenhouse–Geiser correction was used when necessary. Post-hoc tests were performed using paired t-tests, with Bonferroni correction for multiple comparisons. 

EEG data acquisition and analysis

Continuous EEG was recorded from 32 locations using active Ag–AgCl electrodes (Biosemi ActiveTwo amplifier system, Biosemi, Amsterdam, Netherlands) placed in an elastic cap. Standard locations of the international 10–20 system (Jasper, 1958) were used. In the Biosemi system the typically used ‘ground’ electrodes in other EEG amplifiers are replaced through the use of two additional active electrodes. In the 32-electrode montage these electrodes are positioned in close proximity to the electrode Cz of the international 10–20 system: Common Mode Sense (CMS) acts as a recording reference and Driven Right Leg (DRL) serves as ground (Metting Van Rijn, Peper, & Grimbergen, 1990, 1991). Vertical and horizontal electrooculograms were recorded in order to exclude trials with large eye movements and blinks. EEG data processing was performed using the EEGlab toolbox (Delorme & Makeig, 2004) combined with self-written procedures running under Matlab (The Mathworks, Inc, Natick, Massachusetts). EEG signal was sampled at a rate of 512 Hz and epochs lasting 1200 ms were extracted, starting from 500 ms before stimulus onset and incorporating the 700 ms of stimulus presentation. Artifact removal was performed on the basis of visual inspection, which is a standard procedure in studies that use a limited number of electrodes. When artifact rejection was completed, the average rejection rate was 18.11%. All trials with incorrect responses were also excluded from the final analysis. This left an average of 43 trials per condition. Further analyses were performed using the average reference. A 40 Hz low-pass filter was applied to the data before ERP waveform analyses. 

As ERPs were averaged for purely chromatic and achromatic stimuli matched for salience, the full-color stimuli which contained an achromatic component had a relatively low contrast in order to provide equal discrimination performance to isoluminant stimuli. To verify that signal-to-noise ratio (SNR) was adequate for analyses in the N1 window, we used the approach recommended by Koenig and Melie-Garcia (2010). Permutation tests were performed on global field power (GFP) for every condition in each participant. The time points when signal-to-noise ratio became satisfactory were noted down and analyzed statistically using a repeated measures ANOVA with factors direction in color space (S-cone isolating, intermediate isoluminant, full color) and objecthood (object, non-object). GFP is equivalent to the standard deviation of the electrode values at each of the time points and can thus be taken as an indicator of signal quality. 

The midline occipital N1 component of the ERP was the main focus of the analysis, as it was expected to be the earliest clearly observable component in all waveforms on the basis of previous research on ERPs elicited by low contrast luminance and isoluminant contrasts (e.g. Crognale, 2002; Gerth, Delahunt, Crognale, & Werner, 2003). Baseline-to-peak N1 amplitude and N1 peak latency were analyzed statistically with a repeated measurement ANOVA in order to assess if there are any differences between conditions. For the analysis of the late positive wave at 300–450 ms, often labeled L1 (e.g. Gruber & Müller, 2005; Martinovic, Gruber, Ohla, & Muller, 2009), regional means were assigned based on which electrodes exhibited maximal activity when data was collapsed across conditions and the mean amplitude during the period 100 ms prior to stimulus onset (baseline) was subtracted. This component is known to reflect a later stage of object processing, related to visual memory and semantics. For the topographical display of the N1 and L1 components across all conditions, average amplitudes were computed in the time windows of maximal activity (90–230 and 300–450 ms), as shown in Figure 6b (Results). Differences in amplitude and latency of the ERPs were analyzed with a 3 × 2 repeated measures ANOVA with factors direction in color space (S-cone isolating, intermediate isoluminant, full-color) and objecthood (object, non-object). Correlations between N1 amplitude and individual thresholds were calculated using Pearson's correlation coefficients. Post-hoc tests were performed using paired t-tests and Bonferroni correction for multiple comparisons was used. 

ERPs are a well-known and widely accepted method for analyzing differences in the time course of brain activity. However, in multi-channel recordings that evenly sample from the scalp it is also possible to assess the spatial configuration of brain activity over time. It has been found that EEG activity is characterized by a finite set of alternating spatial activation patterns: so-called microstates (for an overview, see Murray, Brunet, & Michel, 2008). Potential differences in these topographic patterns between the conditions were assessed using the Cartool software, programmed by Denis Brunet (http://sites.google.com/site/cartoolcommunity/). Significance of topographical differences was first determined through a non-parametric randomization test colloquially known as TANOVA. This is a test of temporal differences in global dissimilarity, which is an index of distinctness in spatial configuration between two electric fields, independent of their strength (Lehmann & Skrandies, 1980). However, as TANOVA can also be sensitive to the difference in the temporal alternation of topographic maps (i.e., amplitude differences across time), it is necessary to further assess the microstates that are characteristic for each condition. In order to obtain a decomposition of data into microstates, topographical Atomize and Agglomerate Hierarchical Clustering (AAHC) was performed on grand-mean waveforms of activity 0–500 ms after stimulus onset. Topographical AAHC is a clustering approach which operates in a bottom-up manner: the number of clusters is initially set to a user-selected value and progressively diminishes, by iteratively removing the clusters with the lowest global explained variance and assigning their maps to those surviving clusters with which they have the highest spatial correlation. The procedure is completed when only one cluster remains. The optimal solution will involve a set of clusters that provides the best explanation for variance in data topographies. To determine which number of clusters provides an optimal solution, two criteria are jointly used: the cross validation criterion, whose absolute minimum gives the optimal number of segments, and the Krzanovski–Lai criterion, whose maximum shows the point at which additional clusters have led to the highest global quality. To assess the significance of observed microstate differences in grand-mean waveforms, the chosen optimal segmentation was subsequently fitted to single participant data and its correlation with this data was statistically tested using a repeated measurement ANOVA with factors direction in color space (S-cone isolating, intermediate isoluminant, full-color) and objecthood (object, non-object). For more detail on the correct way to perform topographical analyses using Cartool, see methodological papers by Brunet, Murray, and Michel (2011) and Murray et al. (2008). 

Results

Baseline experiment

Contrast thresholds for all 12 observers are presented in Figure 3a while scaled contrasts that were used in the main (EEG) experiment are presented in Figure 3b. For individual polar plots of contrasts see Supplementary Material B. In terms of Weber contrasts, the absolute values of scaled contrasts used in the main experiment had the following ranges: from 0.54 to 0.89 for S-cone contrast of S-isolating stimuli; for intermediate isoluminant stimuli (S & L − M) the S-cone contrast went from 0.36 to 0.64 while the L − M contrast went from 0.09 to 0.15; and finally, for the full-color stimulus (S & L − M & L + M + S), the S-cone contrast ranged from 0.15 to 0.51, L − M contrast was approximately 0.01–0.02 and luminance Weber contrast went from 0.10 to 0.26. There were no significant differences between the threshold radiuses for increments and decrements. 

Main experiment

The key behavioral finding is that object/non-object classification is more accurate when the stimuli contain, in addition to chromatic information (S & L − M), an achromatic component (L + M + S; Figure 4a; leftmost panel). Accuracy differed for the three color directions (F(2, 22) = 9.76; p < 0.005) and this difference in accuracy was driven by the inclusion of luminance information (full-color vs. S: t(11) = −3.51, Bonferroni corrected p < 0.05; full-color vs. S & L − M: t(11) = −3.24, Bonferroni corrected p < 0.05). Adding an L − M component to the S-cone isolating object contours did not yield a significant increase in performance. There was also an interaction between color direction and the factor ‘object’/‘non-object’ (F(2, 22) = 7.28): differences between color directions in accuracy were only prominent for objects; there were no significant differences in accuracies for the non-object conditions (Figure 4a; right panel). 

Figure 4

 

Behavioral data from the EEG experiment: (a) accuracy; (b) mean of median response times.

Reaction times also differed both for stimuli defined along different color directions (F(2, 22) = 16.42; p < 0.001) and for objects and non-objects (F(1, 11) = 10.62; p < 0.01), as can be seen in Figure 4b. There was no interaction between these factors (F(2, 22) = 2.35, n.s.). For objects, responses were faster for stimuli with an achromatic component than both intermediate isoluminant (t(11) = 2.72, p < 0.05) and S-cone isolating contours (t(11) = 4.67, p < 0.01). Intermediate isoluminant direction-defined contours also led to faster responses than S-cone isolating contours (t(11) = 4.37, p < 0.01) for objects. 

We assessed SNR of the obtained waveforms using the global field power permutation test recommended by Koenig and Melie-Garcia (2010). This is depicted in Figure 5. On average, adequate SNR was reached at the following times (means and standard errors): S-cone objects 64 ± 25 ms, intermediate isoluminant objects 39 ± 37 ms, full-color objects 41 ± 39 ms; S-cone non-objects 67 ± 36 ms, intermediate isoluminant non-objects 30 ± 39 ms, full-color non-objects 6 ± 38 ms. A repeated measures ANOVA showed that there were no significant differences in SNR between objects and non-objects (F(1, 11) = 1.05, n.s.), but the differences in SNR between contours defined along different directions in color space were significant (F(2, 22) = 3.73, p < 0.05). As can be seen from Figure 5, it took somewhat longer until the SNR stabilized in the S-cone isolating condition. 

Figure 5

 

Box plots of the time points when stable signal strength was achieved for all the conditions in the experiment. Midlines indicate medians, ends of boxes indicate 25th and 75th percentiles, ends of lines indicate 10th and 90th percentiles and dots indicate observations falling in the outlying 10 percentiles.

A prominent central-occipital N1 component dominated the early part of the ERP waveform, being the first component to reliably and consistently emerge in the grand-mean waveforms during the 90–230 ms time window (Figures 6a and 6b). At electrode Oz, N1 occurred with an earlier latency for stimuli containing a luminance signal (F(2, 22) = 25.67, p < 0.001), with no differences for objects and non-objects (F(1, 11) = 0.10, n.s.). The N1 baseline-to-peak amplitude was modulated by object identity (F(1, 11) = 6.05; p < 0.05) being enhanced for non-objects. This effect was driven by the introduction of luminance information (t(11) = −2.27, p < 0.05), with no significant effects revealed for isoluminant stimuli (S-cone: t(11) = 0.19, n.s.; intermediate isoluminant: t(11) = −1.73, n.s.). Finally, a late parietal positivity which ensued in the ERP waveform at 300–450 ms was maximal at parietal sites (roughly central, around P_z; see Figure 6b) and also enhanced for non-objects (F(1, 11) = 52.21, p < 0.001). 

Figure 6

 

(a) Event-related potential waveform at electrode Oz. N1 and L1 components are indicated by gray boxes. (b) The topographies of the N1 and L1 components, calculated for the time-window indicated in Figure 6a collapsed across all conditions. (c) Linear fits of N1 amplitudes to stimulus contrasts, expressed in multiples of threshold.

Correlations between the N1 amplitude and the stimulus contrast demonstrate that the N1 component is stimulus-driven (Figure 6c): baseline-to-peak N1 amplitude correlated with contrast expressed in terms of multiples of discrimination thresholds (S-cone isolating: r = 0.57, p = 0.05; S & L − M: r = 0.66, p < 0.05; S & L − M & L + M + S: r = 0.64, p < 0.05). 

To assess the differences in topographic patterns between the object and non-object conditions we conducted a TANOVA and topographical Atomize and Agglomerate Hierarchical Clustering (T-AAHC) segmentation analysis (see Figure 7). For S-cone isolating contours, differences in topography between objects and non-objects only emerged after 234 ms while for intermediate isoluminant contours differences appeared after 226 ms. When full-color objects and non-objects were contrasted, a TANOVA indicated that differences started already at 164 milliseconds after stimulus onset (Figure 7a). This approximately coincides with the N1 peak in the grand mean waveform (compare with Figure 6a). However, since a TANOVA is performed sample-by-sample, it can detect significant differences between two conditions even if the modulation concerns a temporal shift in the activations of the very same generators in the brain. Therefore it is important to compare its findings with a segmentation of microstates, in order to confirm that it is indeed a difference in the type of the active generator, instead of being simply a difference in the level of activation of the same generators (Brunet et al., 2011). Topographic clustering of grand-mean activity for all conditions provided an adequate 4-cluster solution with 11 maps for objects, which is depicted in Figures 7c (segmentation of the timecourse) and 7b (a selection of topographical maps). There were several notable differences in topographical patterns. Firstly, in full-color conditions, a map reminiscent of the P1 component was present (map 4) prior to the emergence of the N1 (maps 6 and 7). Map 4 was also very briefly present in the grand-mean intermediate isoluminant non-object condition—this could be due to chromatic aberration which cannot be fully eliminated for stimuli with high-frequency edges. However, map 4 explained more than 1% of global variance in only 3 out of 12 participants during the 70–120 time window. Secondly, object and non-object maps differed in the post-N1 period for all conditions. Non-object processing was characterized by a map that involved a more pronounced anterior negativity (map 9) while in the same period object processing exhibited a central anterior positivity (map 8). However, as can be seen from Figure 7b, the differences in maps 8 and 9 were subtle, mostly to do with the amplitude around a subset of anterior electrodes, and were thus not confirmed by statistical tests of correlations of the microstate maps with single-participant data. 

Figure 7

 

Topographical analyses. (a) Time points of significant differences between full-color objects and non-objects as indicated by a TANOVA analysis, depicting 1 minus p-value across time. (b) A selection of topographic template maps from the T-AAHC segmentation analysis. These are the maps that are characteristic for the period around 100–300 ms after stimulus onset. The templates are normalized GFP-weighted averages of all maps belonging to a particular segment. (c) The full timecourse of microstates determined by the T-AAHC segmentation analysis of grand-mean data for all experimental conditions. This segmentation was characterized by 11 maps. The y axis depicts global field power, an indicator of response strength. Colors represent the sequence of different amplitude maps obtained by the T-AAHC procedure. Each subsequent map is presented in a different color and marked with a different number.

Discussion

We measured performance for classifying line drawings of familiar, meaningful objects and non-objects (scrambled objects) and obtained ERPs elicited by these two classes of stimuli. The novel manipulation in our study was that the stimulus contours were matched for salience and defined along three directions in cardinal color space: 1) S-cone-isolating (S), or 2) intermediate isoluminant direction (S & L − M), or 3) a full-color direction with an additional achromatic (L + M + S) component, providing a luminance signal in addition to chromatic information. The main finding of the study is the object-specific increase in classification performance when a luminance signal is present. This behavioral improvement is reflected in the ERPs—the amplitude of the earliest component to reliably and consistently emerge in the grand-mean waveforms (centro-occipital N1, peaking around 170 ms) was modulated by stimulus identity, being enhanced for non-objects as opposed to objects; this effect was driven by the presence of luminance information. This provides further support to the models of object representation that posit an early and distinct input into posterior representational processes which relies on luminance signals (e.g., Bar, 2003). 

Bar (2003) argues that the contribution of luminance to object classification occurs through magnocellular signals. The luminance contrasts of our stimuli, however, cannot in themselves exclude the presence of a parvocellular contribution to luminance processing, being above 4% Michelson contrast (equivalent to about 8% Weber contrast) which is usually taken as the cut-off for magnocellular-biased stimuli (e.g. Kveraga et al., 2007). Studies relying on stimuli with spatial parameters that make them clearly visible even at very low contrasts can obtain adequate task performance; however in our study, Weber contrasts below 8% would have led to an unacceptable loss of signal-to-noise ratio in the ERP waveforms. Furthermore, high-spatial frequency stimuli such as line drawings have a preferential tendency to excite the parvocellular pathway. Still, by comparing our ERP waveforms to those observed by Foxe et al. (2008) in their study of magnocellular-biasing (M), parvocellular-biasing (P) and combined (M & P) stimuli it can be seen that the ERP elicited by the full-color stimulus in this study (Figure 6a) bears a greater similarity to the M-elicited waveform than to the waveform elicited by the high-contrast, M & P stimulus (Figures 3 and 4 from Foxe et al., 2008). Therefore we conclude that our results are consistent with Bar's (2003) model, but that the benefits to object classification in this study are likely to involve both a significant contribution of the magnocellular, luminance-specialized system as well as the parvocellularly driven luminance input providing information of high spatial resolution (Kingdom & Mullen, 1995; Reid & Shapley, 2002; Shapley, 1990; Victor, Blessing, Forte, Buzas, & Martin, 2007). 

The contrast of the stimuli along different color directions (S; S & L − M; S & L − M & L + M + S) was individually equated in terms of discrimination thresholds (objects/non-objects) to ensure iso-salience across the reduced and full-color conditions. Using multiples of threshold units as a way to establish suprathreshold stimuli of equal salience rests on the assumption that the different color directions exhibit the same contrast dependence. This is a problematic assumption since it has been demonstrated that although the coding of simple visual features within each parallel pathway seems to be independent at threshold, it is subject to highly non-linear interactions at suprathreshold levels (Kulikowski, 2003). The results in our main experiment, however, vindicate our choice to use multiples of discrimination thresholds as a way of equating the stimuli across different color directions. For non-object contours, the classification accuracy is the same for all three color directions (Figure 4a, right panel). Differences in accuracy only emerge for object contours (Figure 4a, left panel). If the performance increase for contours containing a luminance signal was due to a different effective signal strength across the three different color directions (S; S & L − M; S & L − M & S + L + M), we would expect to observe the same differential luminance effect for both kinds of stimuli (object and non-object contours), which is not the case. 

The amplitude of the N1 component is correlated with the contrast of the stimulus contours (Figure 6c). Correlation between contrast and amplitude is in accordance with typical chromatic-onset ERP characteristics (e.g., Berninger, Arden, Hogg, & Frumkes, 1989) and provides further proof that the observed N1 was driven by the properties of the stimulus. According to the normative study by Porciatti and Sartucci (1999) visual evoked potentials elicited by onsets of isoluminant chromatic stimuli do not saturate; luminance contrasts contained in the full-color stimulus are also below the limits of saturation, which is taken to occur at around 16% Michelson contrast. Moreover, while the effect of saturation in M- and P- pathways results in different shifts in component latency, with achromatic P1 latency increasing and chromatic N1 latency decreasing, the amplitudes are generally suppressed for both pathways (Brigell, Strafella, Parmeggiani, Demarco, & Celesia, 1996). Thus the interpretation of observed ERP effects as being due to selective saturation is not likely. An interpretation that the findings are due to a difference in signal-to-noise ratio also does not hold, as permutation tests of global field power showed that the signal quality was comparable between objects and non-objects. 

Another alternative interpretation would be that observed ERP effects were due to some uncontrolled low-level differences for which the luminance system was selective and which impacted upon N1 amplitude. Early visual evoked potentials are indeed highly susceptible to modulations by low-level image features. In this study, we tried to control for as many low-level image properties as possible. Although vertical and horizontal lines were somewhat more prominent in object images, these differences were not significant. Furthermore, the N1 component of the visual evoked potential is higher for gratings containing cardinally oriented lines (Song et al., 2010; but also see Ito, Sugata, & Kuwabara, 1997), in line with the ‘oblique effect’ which states that cardinal orientations are more strongly represented in the visual cortex. Thus, a line orientation account of our findings is not likely. 

N1 is generally considered to be a marker of visual discriminative processing and is largely generated in extra-striate areas of the visual cortex (Vogel & Luck, 2000). First object-specific effects in the ERP waveform usually emerge during the N1 time window: activity related to target objects begins to diverge from activity for distractor objects at around 150 ms, characterized by negativity at posterior and positivity at anterior sites (VanRullen & Thorpe, 2001). Vogel and Luck (2000) concluded on the basis of their own findings into visual discrimination that the brain begins to perform controlled, discriminative processing within 100–125 ms of stimulus onset, acquiring further information about abstract stimulus identity within an additional 50 ms. Due to using individually adjusted contrast levels which were lower than what is usually used in EEG studies on object classification, the present study did not have adequate signal-to-noise ratio to observe a P1 component for full-color stimuli in all of the participants and thus detect differences that could emerge even before the N1 window (see Figure 5). However, previous research with adequate signal-to-noise ratio in the P1 time window has shown that object-specific differences are not likely and that ERPs prior to the N1 window are largely driven by low-level image content and reliable object category differences only emerge around 150 ms (VanRullen & Thorpe, 2001). Our study is the first to demonstrate that object-specific information at such a relatively early time window can be reflected in the ERP waveforms only if luminance information is provided. Similarly, cognitive effects of the classification process are revealed in the time window of late positivity (L1; 300–450 ms), with non-objects eliciting a more positive wave for all contour types which is assumed to reflect enhanced semantic processing afforded to non-objects (e.g. Gruber & Müller, 2005). On the contrary, in the N1 time-window the differential processing of objects is determined solely by the presence or absence of luminance information. 

When luminance information was included, non-objects elicited an N1 that was higher in amplitude than the N1 for objects. Some previous studies have also compared activity evoked by objects and non-objects. Both in Gruber and Müller (2005) and in Busch, Herrmann, Müller, Lenz, and Gruber (2006) initial non-object presentations elicited an enhanced N1 at posterior sites, although this difference did not reach significance. N1 modulations are often related to differences in a “limited-capacity discrimination process”, as proposed by Hillyard, Vogel, and Luck (1998). In this study, non-objects have been created so that they are similar to objects in as many relevant low- and mid-level characteristics as possible, while in Busch et al. (2006) and partly in Gruber and Müller (2005) they sometimes resembled nonsense patterns, without a closed line structure. This would make them stand out against real objects more clearly. However, in our study, while an object would get facilitated in its recognition by a rapid match of its line structure to an internal representation, for non-objects this discrimination process would ultimately fail and warrant longer processing. The behavioral data also supports this account. The well-controlled stimulus material, with non-objects being highly matched to the real objects, may have made the task harder and thus necessitated a further involvement of early discrimination processes. In our experiment, reaction times for non-objects are slower than reaction times for objects, with overall reaction time around 650 ms. In Gruber and Müller (2005) response times for objects and non-objects are approximately the same and very rapid (around 520 ms). As early discrimination processes seem to be reflected by the N1 component of the ERP, this difference in difficulty could have also led to the emergence of object vs. non-object distinction. Finally, it is possible that in this study a significant object vs. non-object difference at N1 level was obtained due to the fact that the relatively low luminance contrasts that were used did not saturate the N1 response, leaving room for the effect to emerge. 

The topographical comparison of full-color objects and non-objects revealed a similar evolution of amplitude maps for objects and non-objects. Differences in amplitude maps as indicated by a TANOVA analysis appeared at 164 ms for full-color object and non-object contours and around 50 ms later for isoluminant stimuli. However, it appears that these differences were caused by a different timing of activation of the brain generators involved in processing the stimuli, rather than a different configuration of maps and thus different brain generators characteristic for object and non-object processing. The emergence of significant differences roundabout the peak of the N1 (see Figures 6a and 7b–7c) indicates that high-level processes already start occurring in this window. This is in line with VanRullen's and Thorpe's (2001) findings on target versus non-target differences in natural image classification appearing after approximately 150 ms (see also Rousselet et al., 2002). 

On the basis of this, we conclude that high-level discrimination processes in the cortex start relatively early and exhibit object-selective effects only in the presence of luminance information, which is consistent with the superiority of luminance in subserving object identification processes (Bar, 2003; Peterson & Gibson, 1993, 1994a, 1994b). Although isoluminant objects are also classified successfully, this classification is slower and less accurate, in particular for stimuli with S-cone isolating contours. More importantly, the processing of isoluminant contours is not characterized by an early difference in brain activity driven by stimulus class. Due to the selectivity of the behavioral effects for object stimuli, our findings cannot be explained by low-level differences in stimulus salience and are most likely to be due to a unique mode of processing that occurs for object form that contains luminance information, providing further experimental support to the model proposed by Bar (2003). 

Supplementary Materials

Supplementary PDF - Supplementary PDF 

Supplementary PDF - Supplementary PDF 

Acknowledgments

The research was funded by an ESRC postdoctoral fellowship and a new investigator BBSRC grant to JM. The Cartool software (http://sites.google.com/site/cartoolcommunity/) is developed by Denis Brunet, from the Functional Brain Mapping Laboratory, Geneva, supported by the Center for Biomedical Imaging, Geneva and Lausanne, Switzerland. 

Commercial relationships: none. 

Corresponding author: Dr. Jasna Martinovic. 

Email: [e-mail]. 

Address: School of Psychology, University of Aberdeen, William Guild Building, Aberdeen, AB24 2UB, UK.