Chromatic contrast in luminance-defined images affects performance and neural activity during a shape classification task

Abstract

Models of object recognition generally emphasize the importance of luminance-defined shape. However, it is still not fully understood how color signals combine with luminance signals to affect object-related form processing. This electroencephalographic study aimed to examine the contribution of chromatic contrast by assessing its effects on the time course of shape-related processing. Participants classified Gaborized images of object shapes, nonobject shapes, and patches of pseudorandomly scattered Gabors. Stimuli excited (a) the luminance (L+M) channel alone, (b) luminance and L-M channels, or (c) luminance, L-M, and S-(L+M) channels and were presented either at mean discrimination threshold or at twice this mean threshold. As expected, classification accuracy was comparable at threshold, as were the attributes of the early, perceptual first negative (N1) component of the event-related potential (ERP). Differences emerged at suprathreshold: Objects defined by the full combination of channels were associated with the poorest performance and the lowest N1 amplitude. Shape sensitivity was not consistently observed in the N1 but was more evident in the late positive potential (LPP), a cognitive ERP component. Both the N1 and the LPP were affected by the amount and type of contrast in the image. While the effects of luminance and L-M contrast were similar, affecting the ERP selectively during the N1 and LPP period, S-(L+M) contrast elicited a sustained shift in amplitude. Our results demonstrate, for the first time using a combination of behavioral as well as early and late electrophysiological effects, that shape classification is determined by both the chromatic and the luminance content of the image.

Introduction

Acquiring knowledge about objects is essential for adaptive behavior in everyday environments. Both achromatic and chromatic information are relevant for everyday vision, but their contributions to object processing have traditionally been perceived as different: Luminance is seen as more relevant for shape processing, and color is seen as more relevant for segmenting objects from their backgrounds (Tanaka, Weiskopf, & Williams, 2001). This is reflected in models of object recognition. For example, low-level inputs that drive object processing in the model of Sowden and Schyns (2006) stem from luminance-driven spatial frequency channels. Further, in Bar's (2003) model of object recognition, the fast, top-down input essential for constraining the processing in posterior representational areas is driven by rapid projections of low-spatial-frequency luminance information. At the neuronal level, the tuning of luminance-driven spatial frequency channels is affected by lateral inhibition between neurons with spatially overlapping receptive fields, which are tuned to different spatial frequency and orientation bands (Greenlee & Magnussen, 1988; Tolhurst, 1972). Lateral interactions also exist between spatial frequency channels sensitive to different spatial locations: Polat and Sagi (1993) found that foveal target detection is affected by a narrow inhibitory surround and a further, much larger facilitatory area. In this way, neuronal sensitivity is fine tuned to spatial variations of luminance contrast that define shape across orientation and size. However, there is emerging evidence that color signals can and do contribute to the processing of object form. To a degree, color mechanisms are also able to provide low-level information that sustains object recognition, with spatial frequency (Mullen & Losada, 1994, 1999) and orientation (Webster, DeValois, & Switkes, 1990; Wuerger, Morgan, Westland, & Owens, 2000) channels that are not vastly dissimilar to those driven by luminance information. Anatomical and physiological investigations found that a substantial amount of neurons in areas V1 and V2 of the cortex receives inputs from different visual streams, indicating that the segregation of luminance and color signals is not as normative as had been previously thought (Levitt, Yoshioka, & Lund, 1994; Vidyasagar, Kulikowski, Lipnicki, & Dreher, 2002; for models, see Lund, Wu, Hadingham, & Levitt, 1995; Zhaoping, 2014; for comprehensive reviews, see Kulikowski, 2003; Solomon & Lennie, 2007). Benefits brought about by the availability of spatial information from both luminance and color might be expected from considerations of the complexities of our everyday visual environments. Contributions of chromatic signals to form processing might be particularly salient due to their independence from shadows and shading, which are defined through changes in luminance only (for a review, see Shevell & Kingdom, 2008). Indeed, edge extraction from luminance and chromatic spatially superimposed components within a set of natural scene images showed that these signals provided mutually independent information (Hansen & Gegenfurtner, 2009). Jennings and Martinovic (2014) described facilitatory interactions between L-M chromatic and luminance signals in a task that required discriminating familiar, nameable shapes (objects) from novel, unnameable shapes (nonobjects). Chromatic contrast benefitted discrimination by combining with colocalized luminance contrast in a facilitatory fashion, leading to reduced object–nonobject discrimination thresholds. 

The brief literature overview presented above raises one important question: If chromatic signals do combine with colocalized luminance signals to contribute to form perception, at which stage of neural processing does this occur? With its millisecond resolution, electroencephalography (EEG) is a very useful method for studying the time course of visual processing. A specific sequence of event-related potential (ERP) components is typically observed in EEG experiments that require classification of visual stimuli. Some of the earlier components, such as the first positive (P1) and first negative (N1) components, are more perceptual in nature, while the components that develop later in the time course reflect progressively more cognitive processing. Traditionally, these components are taken as dependent variables, and predictions are then made about modulations that should occur due to an early, perceptual, or late cognitive contribution. P1 and N1 components are considered to be early components, reflecting perceptual processes; they are both contrast- and spatial frequency–dependent and relatable to psychophysical threshold (Boon, Suttle, & Dain, 2007; Souza, Gomes, Saito, da Silva, & Silveira, 2007). Isoluminant stimuli do not elicit the earliest P1 component of the visual ERP, but they do elicit a prominent negative deflection that corresponds in timing to the N1 component (Berninger, Arden, Hogg, & Frumkes, 1989; Murray, Parry, Carden, & Kulikowski, 1986). The shape of the ERP waveform is determined not only by the spatial frequency and chromoluminance content of the stimulus but also by the regularity and duration of the stimulus presentation (Kulikowski, 1977; Rabin, Switkes, Crognale, Schneck, & Adams, 1994). In a study that used relatively long stimulus presentations and variable intertrial intervals, typical of object recognition ERP experiments, Martinovic, Mordal, and Wuerger (2011) found that the amplitude of the N1 component correlated with stimulus contrast. The N1 component is thus the earliest locus of possible contributions of both color and luminance to the ERP. Martinovic et al. (2011) also observed object-sensitive modulations of the N1 only for images that contained luminance contrast in addition to chromatic contrast. But the N1 is not always sensitive to the presence of objects (e.g., Gruber & Müller, 2005), implying that object-sensitive N1 effects are likely to be reliant on stimulus and task characteristics. Object sensitivity is found much more reliably in the late positive potential (LPP) component of the ERP, known to be robustly modulated by semantic content of stimuli (e.g., their familiarity and nameability; Gruber & Müller, 2005; Martinovic, Gruber, Ohla, & Muller, 2009). 

In order to establish the way in which the time course of object-related shape processing is influenced by the presence of different contrast types in addition to luminance, we conducted an ERP study. As in Jennings and Martinovic (2014), our stimuli consisted of Gaborized images of objects, nonobjects, and pseudorandom patches. We used stimuli defined by luminance alone as well as luminance colocalized with an L-M chromatic signal and luminance colocalized with both an L-M and an S-(L+M) chromatic signal. Thus, all of our stimuli contained luminance contrast, either on its own or in combination with chromatic contrast. Comparisons between conditions that excite different chromoluminant channels are complicated by the necessity to establish a common contrast metric, which is far from straightforward (for a discussion, see Shevell & Kingdom, 2008). Most often, contrasts in different channels are matched through multiples of threshold. We opted to set our contrast levels on the basis of object–nonobject discrimination thresholds from Jennings and Martinovic (2014) since we intended to use the same stimulus set. Contrasts were set at threshold or suprathreshold (defined as twice threshold). We intended to perform two types of analysis on the EEG data: (a) a traditional ERP analysis, focused on N1 and LPP components, to indicate the level at which differences emerge between our object, nonobject, and random patch stimuli and to assess whether these differences are affected by the contrast content of the stimuli, and (b) linear modeling of the EEG waveforms in order to identify how three types of contrast—luminance, L-M, and S-(L+M)—affect the stages of processing reflected in the N1 and LPP components. As mentioned earlier, Jennings and Martinovic (2014) found that less luminance contrast was required to reach threshold when it was combined with L-M chromatic contrast. Therefore, our conditions significantly differed in the amount of luminance, L-M, and S-(L+M) contrast they contained, enabling the modeling approach. 

We expected to find performance and early, perceptual ERP components to be matched at threshold. At suprathreshold, we predicted that gains in performance should be matched by increases in both the N1 and LPP amplitudes. We tested whether the N1 and the LPP were sensitive to differences between the three classes of stimulus images: (a) familiar, nameable shapes (objects); (b) shapes that lack familiarity and nameability (nonobjects); and (c) stimuli that lack familiarity, nameability, and any clear shape (pseudorandom patches). We expected to find such sensitivity, assuming on the basis of Martinovic et al. (2011) that it was driven mainly by the information derived from luminance contrast. Models that assume that luminance is more relevant for object representation processes would predict that any object-sensitive ERP markers should be more pronounced for stimuli that contain significantly more luminance. However, if this is not the case, it would necessitate models of object recognition to include a shape-processing stage at which chromatic contrast combines with luminance contrast (for a similar line of research with naturalistic and natural images, see Groen, Ghebreab, Lamme, & Scholte, 2012; Groen, Ghebreab, Prins, Lamme, & Scholte, 2013). Finally, the linear modeling of the EEG using contrast metrics would allow us to directly examine the degree to which the ERP waveforms are sensitive to each type of contrast: luminance, L-M, or S-(L+M). In order for chromatic contrast to contribute to perceptual and cognitive processing that is marked by N1 and LPP components, it needs to have a modulatory effect that is circumscribed to the time windows of these components. 

Materials and method

Participants

A total of 22 participants were recruited for the study. Each participant reported normal or corrected-to-normal visual acuity and had normal color vision as assessed with the Cambridge Color Test (Regan, Reffin, & Mollon, 1994). Three participants were excluded due to inadequate behavioral performance, defined as below-chance accuracy on any single condition, and one participant was rejected due to more than 40% trials with artifacts. Excluded participants were replaced with new participants in order to maintain counterbalancing of button-to-response allocation (see Procedure). The final sample of 18 participants (12 females, six males) had a mean age of 25 ± 3.9 years (M ± SD; range: 19–35 years), and 16 were right handed. Participants were reimbursed for their time. The study was approved by the ethics committee of the School of Psychology, University of Aberdeen and is in accordance with the ethical principles stated in the Declaration of Helsinki. 

Derrington Krauskopf Lennie color space

The Derrington Krauskopf Lennie (DKL) color space (Derrington, Krauskopf, & Lennie, 1984) was used to describe the chromatic properties of the stimuli. Figure 1 shows a representation of the DKL color space indicating the two chromatic mechanisms—L-M and S-(L+M)—and the luminance mechanism—L+M—along with a vector (P) defining a particular chromaticity and luminance defined with a radius r, chromatic angle ϕ, and luminance elevation θ. The DKL space was implemented in the Color Toolbox (CRS, Rochester, Kent, UK; Westland, Ripamonti, & Cheung, 2012) using measurements of monitor phosphors' spectral power distributions obtained with a SpectroCAL (CRS) and cone fundamentals (Stockman & Sharpe, 2000; Stockman, Sharpe, & Fach, 1999). A uniform midgray background located at the adaptation point DKL(r, ϕ, θ) = (0, 0, 0) was used throughout the experiments; this corresponded to CIE 1931 (x, y, Y) = (0.30, 0.32, 46.4), where Y is in units of cd m⁻². 

Figure 1

 

The DKL color space with three perpendicular axes corresponding to the L-M, S-(L+M), and L+M mechanisms was used to specify the chromatic and luminance conditions used in this experiment and further defined in Figure 2. The chromaticity and luminance at point P is described by DKL (r, ϕ_DKL, θ_DKL)_polar, where r is the three-dimensional Euclidean distance from the center of the space located at (0, 0, 0), ϕ_DKL is the chromatic angle, and θ_DKL is the luminance elevation. The figure also provides an example Gabor patch on a gray background for each of the three cardinal directions in DKL space.

Figure 2

 

Contrasts for the three threshold and three suprathreshold conditions as used in the experiment. Contrasts are based on mean data of the main experiment in Jennings and Martinovic (2014).

Stimulus contrast settings

Three conditions were used in the study: The first isolated the luminance contrast (L+M); the second combined luminance and L-M contrasts; and the third combined luminance, L-M, and S-(L+M) contrasts. As explained in the Introduction, the choice of conditions was based on object–nonobject discrimination results of Jennings and Martinovic (2014). We selected those combined conditions in which an interaction between luminance and color was observed, such that less luminance contrast was needed in the combined condition to achieve threshold. On the other hand, the two conditions that combined color and luminance did not differ significantly from each other in terms of L-M and L+M signals at threshold, reflecting the fact that S-(L+M) signals did not affect performance. Stimuli in our study were presented either at mean object–nonobject discrimination threshold or at twice the threshold. This provides a range of luminance and chromatic contrasts, allowing us to use linear modeling of single-trial activity by contrast in L+M, L-M, and S-(L+M) mechanisms. 

Figure 2 summarizes the contrasts along with the DKL parameters. Mechanism contrasts shown in Figure 2 were derived from Michelson cone contrasts. These were calculated according to Equation 1, where I_max and I_min are the maxima and minima cone excitations of the Gabors. Mechanism contrasts were then computed for L-M, S-(L+M), and L+M.  

 

 

As mentioned earlier, it can be seen from Figure 2 that stimuli at threshold do not contain exactly the same amount of luminance contrast. If the stimuli did include the same amount of luminance, on the basis of Jennings and Martinovic's (2014) findings of facilitations between L-M and L+M signals it would be reasonable to expect improved performance for conditions combining luminance with a nonnegligible amount of L-M information. This would create a problem for interpreting the results unequivocally in relation to contrast type, as differences in ERPs could also be ascribed to mismatched performance. An alternative that would ensure matched performance would have been to fix the luminance contrast at threshold and to add chromatic contrast that is small enough to not affect performance. This approach would be suitable if our objective was to study contrast summation without attempting to relate it to performance on a shape classification task, as these chromatic contrasts would not be contributing to performance in any way. Differences in contrast-response functions between luminance alone and luminance with color would warrant a separate contrast-additivity study with a much simpler stimulus and task (for some previous work with EEG, see Rabin et al., 1994; Rudvin, 2005; Rudvin & Valberg, 2005). Our shape discrimination task would not be suited for this purpose, as L-M and S-(L+M) isolating conditions require relatively high levels of contrast at threshold (see figure 3 in Jennings & Martinovic, 2014), making it impossible to stay within the cathode ray tube (CRT) monitors gamut if they were to be combined with any significant levels of other contrast types. 

Last, in order to understand the way in which we matched stimulus contrast to account for performance, it is important to note that thresholds in Jennings and Martinovic (2014) were obtained using a two-interval forced-choice task in which participants had to select the interval that contained the object, with the other interval containing the nonobject. Therefore, when we say that stimuli were presented at object–nonobject discrimination threshold, this implies that performance in discriminating these two categories of stimuli should be matched at this level of contrast. However, it does not necessarily mean that in a one-interval forced-choice task similar accuracy rates will be obtained for object and nonobject images since one-interval forced-choice tasks are additionally prone to response biases. For example, if there is an overall bias to classify an image as a nonobject, this will lead to higher error rates for objects than nonobjects and higher hit rates for nonobjects than objects. In that case, if the hit rate for objects is 41%, with 10% of nonobjects misclassified as objects, the corresponding performance matches a d′ of approximately 1 (thus 75% correct overall). With the hit rate for nonobjects at 85% and 48% of objects misclassified as objects, the corresponding d′ is again approximately 1 (matching 75% correct overall). This shows that discriminability can indeed be matched, although hit rates and error rates for individual stimulus classes differ. A similar approach to stimulus contrast matching was used successfully in Martinovic et al. (2011) and Kosilo et al. (2013). To further quantify the relations between the three types of stimuli, we performed an analysis of response patterns (see Supplementary Material S2). These percentages can be used to approximately assess the discriminability between the different classes of stimuli, although when performing these calculations it is important to account for the fact that there are three possible responses (object, nonobject, random). Considering that the stimuli were matched in performance using the two-interval forced-choice thresholds from Jennings and Martinovic (2014) but that this does not necessarily imply that the resulting performance will be 75% for each of the three stimulus classes (especially the random patches, which were added as a control stimulus with no explicit contours), one could alternatively apply the labels lower contrast match and higher contrast match for our threshold and suprathreshold conditions, respectively. We opt to use the terms threshold and suprathreshold, as this reflects that the contrasts were chosen not provisionally but rather on the basis of experimental threshold data from Jennings and Martinovic (2014). 

Stimuli

The stimulus set from Jennings and Martinovic (2014) was used (available for download at http://homepages.abdn.ac.uk/j.martinovic/pages/dept/project.htm). This is a set of 377 Gaborized nameable, familiar objects and their unnameable, unfamiliar nonobject counterparts, similar to the image library provided by Sassi and colleagues (Sassi, Machilsen, & Wagemans, 2012; Sassi, Vancleef, Machilsen, Panis, & Wagemans, 2010). This stimulus set was supplemented by 377 images with pseudorandomly scattered Gabor patches, which unlike the nonobjects did not consist of iso-oriented contours. All stimuli comprised a series of center-symmetric 3-cpd Gabor patches. This spatial frequency was chosen so that roughly equal contrast dependence of orientation sensitivity across the mechanisms would be maintained based on available data for L-M and luminance mechanisms (Wuerger & Morgan, 1999). An additional benefit is that amplitudes and latencies of S- and L-M–elicited visual evoked potentials (VEPs) are roughly similar around 3 cpd (see figure 9 in Rabin et al., 1994). The creation of the object–nonobject stimuli started by selection of suitable line images of objects from various stimulus sets (Alario & Ferrand, 1999; Bates et al., 2003; Hamm & McMullen, 1998) and by the manual digital drawing of additional line images of objects that were not represented in those sets. The lines of these images were replaced with a series of Gabor patches, with the position of each Gabor patch predefined by hand in order to ensure that shape-defining lines were maintained in the images. (For an algorithmic approach to the same problem, see the Grouping Elements Rendering Toolbox for Matlab [The Mathworks, Inc., Natick, MA; Demeyer & Machilsen, 2012].) The corresponding nonobject images were created by using image-editing software to distort the line images of the objects until they became unrecognizable. The lines were then replaced by Gabor patches, similarly to the procedure described above. Figure 3 shows examples of an object (a zebra; Figure 3a), a nonobject (Figure 3b), and a random patch (Figure 3c). The process of scrambling the object images into nonobject images attempted to preserve some important attributes of the initial object images, including the visual complexity of the images as reflected in .jpeg file size (Szekely & Bates, 2000) and their aspect ratio. In the process of transforming line drawings into Gaborized images, care was taken to have some of the lines defined by Gabor patches located near the fixation point (no farther than approximately 1° away) in order to preclude the need for eye movements to outer object edges in low-contrast conditions close to threshold. Finally, the nonobjects were also constrained to have a closed outer contour in order to be consistent with that property of objecthood and to prevent them from appearing as random clusters of Gabor patches (which was the added third stimulus class). These pseudorandom clusters were created by scattering the same number of elements that formed the matching object and nonobject pair over the approximate area that they occupied as defined by an ellipse. The patches are pseudorandom as the Gabors were not allowed to overlap. A pilot naming test was conducted on the stimuli in which participants had to decide whether a presented shape was an object or a nonobject and then, if they classified the image as an object, provide a name (see Jennings & Martinovic, 2014). The final piloted set of object stimuli subtended a height and width of 2.9° ± 1.0° and 6.7° ± 1.1° (M ± SD), respectively, while the nonobject stimuli subtended a height and width of 2.8° ± 0.8° and 7.6° ± 0.9° (M ± SD), respectively. For more details on the Gabor properties and the attributes of the stimulus set, see Jennings and Martinovic (2014). 

Figure 3

 

Examples of stimulus types: (a) an object (a zebra), (b) a nonobject, and (c) a random patch. (d) A luminance-defined Gabor patch. (e) A luminance- and L-M–modulated Gabor patch. (f) A luminance-, L-M–, and S-(L+M) (i.e., the full condition)–modulated Gabor patch.

Procedure

Participants were informed that their task was to discriminate objects, nonobjects, and random images. They were shown some examples of images and then performed a practice block of 51 trials that contained a subset of stimuli not used in the main experiment (17 stimuli per image class). The intention was to familiarize them with the task. Participants repeated the practice block if their performance was below 70%. Usually, that criterion was reached after one repetition; sometimes no repetitions were needed, and rarely participants repeated the practice twice. The main experiment consisted of a total of 1,080 trials distributed over ten 108-trial blocks. A trial started with a variable period (500–700 ms) during which only the fixation cross was displayed. After that the stimulus was displayed for 1200 ms, followed by the fixation cross only for a further 1000 ms. Participants responded with a button press indicating whether the presented stimulus was an object, a nonobject, or a pseudorandom patch. Button-to-response allocation was counterbalanced across participants. Participants were instructed not to make eye movements or blink during the display of a stimulus or the fixation cross and to try to remain relaxed and refrain from making body or head movements throughout the experiment. At the end of each trial the fixation cross was replaced with an “X” for 1000 ms; participants were instructed to blink during this period if required. Figure 4 shows the sequence of one trial. 

Figure 4

 

Trial outlook starting with a variable period of fixation that preceded stimulus onset, followed by the stimulus presentation, and ending in an additional fixation-only period during which observers could still respond. Finally, the fixation “+” changed to an “X” to indicate to the participant that they could blink if required.

Behavioral data analysis

Accuracies and reaction times (RT) between 300 and 2200 ms were analyzed. Percentage of correct responses was computed for all conditions and subjected to statistical analysis, but incorrect responses were also taken into consideration in an additional analysis of potential biases in response patterns. Median RTs for correct items were computed for each participant. Differences in accuracies and median RTs between the conditions were analyzed with a repeated measures analysis of variance (ANOVA) with the factors contrast level (threshold, suprathreshold), contrast combination (L+M isolating, L+M combined with L-M, L+M combined with both L-M and S-[L+M]), and stimulus type (object, nonobject, random patch). Greenhouse–Geiser correction was used when necessary. Post hoc tests were performed using Tukey's honestly significant difference (HSD) to follow up on interactions and Bonferroni-corrected paired t tests to further assess sources of main effects. The suprathreshold data are presented in the Results, and a comparison of these data with the threshold data is presented as Supplementary Material S1. Biases in response patterns are presented as Supplementary Material S2. 

EEG data acquisition and analysis

Continuous EEG was recorded from 128 locations using active Ag–AgCl electrodes (Biosemi ActiveTwo amplifier system, Biosemi, Amsterdam, the Netherlands). In the Biosemi system, the typically used ground electrodes are replaced with two additional active electrodes. These are the common mode sense (CMS) electrode, which acts as a recording reference and the driven right leg (DRL) electrode, which serves as the ground. In the 128-electrode montage these two electrodes are positioned in close proximity to the electrode Cz (Metting Van Rijn, Peper, & Grimbergen, 1990, 1991). Vertical and horizontal electro-oculograms were recorded in order to exclude trials with large eye movements and blinks. 

EEG data processing was performed using the EEGlab toolbox (Delorme & Makeig, 2004) combined with self-written procedures running under Matlab. The EEG signal was sampled at a rate of 512 Hz, and epochs lasting 2000 ms were extracted, starting from 500 ms before stimulus onset and incorporating 1500 ms after stimulus onset. All trials with incorrect responses were excluded from the ERP analysis. Artifact removal was performed using the FASTER toolbox (Nolan, Whelan, & Reilly, 2010), followed up with a visual inspection method. This left an average of 36 ± 11 (M ± SD) trials per condition. Further analyses were performed using the average reference. A 40-Hz low-pass filter was applied to the data before ERP waveform analyses. Signal-to-noise ratio (SNR) analysis was performed using the approach recommended by Koenig and Melie-Garcia (2010). This was done to assess whether adequate SNR was reached in our experimental conditions, as ERPs at thresholds may suffer from SNR problems due to the low number of trials remaining in the analysis and the relatively low amplitude of evoked responses at relatively low contrast levels (Campbell & Maffei, 1970). This may in turn impact the latencies and amplitudes of ERP components. The latencies and amplitudes of the N1 component and the amplitude of the LPP component at suprathreshold contrast were analyzed with a repeated measures ANOVA with factors the contrast combination (L+M isolating, L+M combined with L-M, L+M combined with both L-M and S-[L+M]) and stimulus type (object, nonobject, random patch). As with the behavioral data, an analysis with the additional factor contrast level (threshold, suprathreshold) is presented in Supplementary Material S1; the main purpose of this analysis was to confirm that there are no differences between the three contrast combinations at threshold. The components were defined based on the visual inspection of grand-mean waveforms separately for the threshold and suprathreshold components as they were expected to differ in latency (for a normative study, see Porciatti & Sartucci, 1999). N1 at threshold extended from 180 to 380 ms, while the suprathreshold N1 extended from 150 to 270 ms. LPP at threshold was analyzed in the range between 550 and 800 ms, while suprathreshold LPP was analyzed between 500 and 750 ms. In line with previous literature, the N1 for a visual evoked potential with a strong chromatic component was expected to occur at central occipital sites (Porciatti & Sartucci, 1999), while the LPP was expected to be maximal at midline parietal sites (Gruber & Müller, 2005). Similarly to the timing of components, their topographical locations were verified using grand-mean plots. Greenhouse–Geiser correction was used when necessary. Post hoc tests were performed using Tukey's HSD for more complex interactions, which involved nine variables, and Bonferroni-corrected paired t tests for main effects and less complex interactions, which involved six variables. Ratios of suprathreshold:threshold amplitudes were calculated using only those data points with sufficient SNR. Linear modeling of the first second of the EEG single-trial data was performed using the LIMO EEG toolbox for Matlab (Pernet, Chauveau, Gaspar, & Rousselet, 2011) in order to establish more precisely the effect of contrast on the waveforms. For this analysis, all artifact-free trials were included as per the recommendations made by VanRullen (2011), allowing us to encompass more broadly how the waveforms were affected by contrast content. Linear regression analysis was performed at each time point and for each electrode based on three continuous predictors: the amount of L+M, L-M, and S-(L+M) contrast present in the stimulus on each trial. Following the approach from Kovalenko, Chaumon, and Busch (2012), we orthogonalized sequentially the three parameters (Gram-Schmidt orthogonalization method, SPM8; http://www.fil.ion.ucl.ac.uk/spm/). The outcome of sequential orthogonalization is that the variance that is explained by a parameter is discarded from subsequent ones. This decorrelates the three types of contrast and allows us to attribute effects that can be explained by more than one contrast type to just one of them. 

Results

The experiment was conducted for stimuli presented at threshold and suprathreshold (i.e., two times threshold), as outlined in the Stimulus contrast settings section and Figure 2. It was found that accuracies and ERP amplitudes measured at threshold presentation levels showed no differences between the different contrast combinations employed; that is, at threshold the stimuli were matched. We hence present here only the suprathreshold data. The main differences between ERPs at threshold and suprathreshold were (a) in terms of their latency, which was faster at suprathreshold, and (b) in terms of the increases in amplitude with increased contrast, which were present for luminance alone and L-M with luminance but absent for the full-contrast combination. Detailed analyses and comparison of threshold control data and suprathreshold data (both behavioral and ERPs) are presented as Supplementary Material S1, and an analysis of behavioral response biases is presented as Supplementary Material S2. 

Behavioral data

Due to the complexity of the behavioral data analysis, which covers accuracies, RTs, and response patterns, we first give an overview of the results and then go into statistical detail. Figure 5 illustrates the main findings; differences in accuracy and RTs exist between the three contrast conditions. Overall, accuracy for a given stimulus type was equal for luminance and L-M with luminance conditions and better than for the full-information contrast combination. Responses were fastest for random patches, followed by objects; nonobjects elicited the slowest responses, with correct responses being lowest for object stimuli. A correlation analysis of accuracy–RT combinations revealed no speed–accuracy tradeoffs (all ps > 0.05). 

Figure 5

 

Correct responses (top row) and corresponding RTs (bottom row) for each chromoluminance condition at both threshold and suprathreshold. The stimulus types are color coded: gray = objects; green = nonobjects; purple = random patches. Error bars are 2 SE. The y-axis does not start at 0; the dotted gray line in the top row indicates chance level (33%).

A 3 × 3 (contrast combination by stimulus type) repeated measures ANOVA analysis of accuracies on suprathreshold data revealed significant main effects of contrast combination, F(2, 34) = 12.90, p < 0.001, η_p² = 0.43, and stimulus type, F(2, 34) = 41.06, p < 0.001, η_p² = 0.71. There was also a significant interaction, F(4, 68) = 3.63, p = 0.01, η_p² = 0.18. 

Post hoc (Tukey's HSD) tests revealed that luminance-only objects were identified with the same performance as L-M with luminance objects, while both were more accurately identified than full-information contrast condition objects. For nonobjects, there were no significant differences between the three contrast combinations. Finally, for random patch stimuli, again both had equal performance when defined with luminance only or both L-M and luminance, and both were more accurately identified than full-information contrast condition random patches. Within each contrast combination, objects were associated with poorer performance than the other two stimulus types, whereas performance between nonobjects and random patches did not differ significantly. 

In terms of RTs, there was a main effect of contrast combination, F(2, 34) = 69.09, p < 0.001, η_p² = 0.80, and stimulus type, F(2, 34) = 39.25, p = 0.001, η_p² = 0.70. No significant interaction existed between these levels, F(4, 68) = 1.71, p = 0.092. 

Bonferroni-corrected t tests informed us that performance was fastest for random patches, followed by objects; nonobjects were responded to most slowly overall (all ps < 0.001). Performance was equally fast for luminance-defined and L-M with luminance–defined objects (p = 0.18) and significantly slower for the full-contrast combination (both ps < 0.001). 

Signal-to-noise ratios

The SNRs of the ERP waveforms at suprathreshold level were assessed using the global field power permutation test recommended by Koenig and Melie-Garcia (2010). A repeated measures ANOVA revealed no significant differences in time point of SNR stabilization between the different stimulus types (object, nonobject, random patch), F(2, 26) = 1.52, p = 0.24. Also, no significant differences were found over the three luminance and color conditions; F(2, 26) = 0.62, p = 0.55; interaction: F(4, 52) = 0.59, p = 0.67. On average, an adequate SNR was reached at the following times (median ± SE): objects, 124 ± 26 ms; nonobjects, 125 ± 21 ms; and random patches; 129 ± 28 ms (these values are collapsed over contrast combinations). For completeness the noncollapsed data are depicted in Figure 6. 

Figure 6

 

Box plots indicating the time at which the SNR stabilized for all of the conditions at suprathreshold contrast levels. The stimulus types are color coded: gray = objects; green = nonobjects; purple = random patches. The first, second, and third subcolumns represent luminance isolated, luminance combined with L-M, and luminance combined with both L-M and S-(L+M) signals, respectively. The lines represent the median, the edges of boxes represent the 75th percentile, the ends of lines represent the 95th percentile, and red crosses represent outliers.

Threshold and suprathreshold differences in ERP amplitudes and latencies

Detailed differences between threshold and suprathreshold ERPs, in terms of both their amplitudes and their latencies, are presented in Supplementary Material S1. Here we give a broad overview of the main contrast-related differences, which are depicted in Figure 7. This figure collapses the data across different stimulus types (object, nonobject, random) in order to more clearly depict changes that arise due to the twofold increase in contrast, from threshold to suprathreshold, for each stimulated combination of contrasts. 

Figure 7

 

ERP at posterior sites (see N1 topography inset) depicting data collapsed across stimulus class. The full lines depict the three contrast combinations at threshold, while the dotted lines depict them at suprathreshold. Topographies were calculated after data in the N1 window (see Figure 8) were collapsed across all conditions for threshold and suprathreshold contrast levels. The electrodes that were used for data analysis are indicated with thick black circles on the topography plots.

Figure 8

 

(a) N1 component of the ERP. Waveforms at occipital sites and topographies during the N1 window (indicated by the gray box) are depicted for suprathreshold stimuli. (b) LPP component of the ERP. Waveforms at parietal sites and topographies during the LLP window (again indicated by the gray box) are depicted for suprathreshold stimuli. In both cases, topographies were calculated after data were collapsed across all conditions. The electrodes that were used for data analysis are indicated with black circles on the topography plots.

Latencies are slower for threshold stimuli and somewhat slower for the full combination of contrasts. Ratios of suprathreshold:threshold amplitudes within the N1 analysis windows were the following: luminance only, 1.64 ± 0.30; luminance and L-M, 1.69 ± 0.14; luminance, L-M, and S-(L+M), 1.13 ± 0.07 (M ± SE). Ratios were calculated only from data points with adequate SNR in order to reduce the noisiness of the calculation. Although there is an increase in amplitude for luminance alone and luminance with L-M, this does not occur for the full-contrast combination. 

Event-related potentials: N1

The suprathreshold N1 waveform and topography are depicted in Figure 8a, while the bar plot of its amplitudes is presented in the top panel of Figure 9. There was a main effect of contrast combination, F(2, 34) = 30.09, p < 0.001, η_p² = 0.64, and a trend toward a main effect of stimulus type, F(2, 34) = 3.11, p = 0.058, η_p² = 0.16. A significant interaction existed between these levels, F(4, 68), p = 0.02, η_p² = 0.15. 

Figure 9

 

Bar plot of ERP amplitudes. N1 is depicted in the top panel, and LPP is depicted in the bottom panel. The stimulus types are color coded: gray = objects; green = nonobjects; purple = random patches. The contrast combinations are as follows: left three bars = luminance only; middle three bars = luminance and L-M; right three bars = luminance, L-M, and S-(L+M). Error bars depict ± 2 SE.

Post hoc tests for the main effects indicated that the full-contrast combination elicited the least activity compared with both luminance only and L-M with luminance (p < 0.05), with luminance only in turn eliciting even less activity than the combined L-M with luminance condition (p < 0.05). Post hoc tests for the interaction (Tukey's HSD) considered all possible combinations, revealing a variety of differences. This was to be expected given the large main effect of contrast combination. But importantly, considering differences between objects, nonobjects, and random patches within each of the three contrast combinations, it was found that the only significant difference between stimulus types existed in the full-contrast combination. Here objects were found to be significantly different from nonobjects and random images (both ps < 0.05), which in turn did not differ among each other (p > 0.05). 

Event-related potentials: LPP

The LPP can be seen in Figure 8b, while the bar plot of its amplitudes is presented in the bottom panel of Figure 9. A 3 × 3 ANOVA (contrast combination by stimulus type) was performed and revealed main effects of both contrast combination, F(2, 34) = 12.77, p < 0.001, η_p² = 0.43, and stimulus type, F(1.41, 23.93) = 4.37, p = 0.036, η_p² = 0.20. No significant interaction was discovered, F(2.68, 45.58) = 0.88, p = 0.45. 

Considering the contrast combination, post hoc tests revealed that the LPP had a lower amplitude for the full-contrast combination compared with both the luminance-only and the luminance combined with L-M combinations (p = 0.002 and p < 0.001, respectively), which did not differ from each other (p = 0.85). Considering stimulus type, the LPP is higher for random patches than for nonobjects (p = 0.015), but there are no significant differences between objects and random patches (p = 0.14) or objects and nonobjects (p = 0.1). 

Linear modeling of single-trial EEG by contrast parameters

To assess effects of each contrast type—L+M, L-M, or S-(L+M)—on the ERP waveforms, we also conducted a single-trial linear regression analysis using the approach described in Pernet et al. (2011). We recursively orthogonalized the three contrast levels in order to decorrelate them and then entered them simultaneously into the general linear model. The results of the analysis are presented in Figure 10 for electrodes Oz, exemplifying the N1 component, and Pz, exemplifying the LPP component. It can be seen that all three types of contrast affect the waveforms. While the effects of L+M and L-M contrasts are temporally constrained to the windows of the N1 and LPP components, the effects of S-(L+M) contrast are much broader and less specific, and although the onset of contrast modulation occurs at approximately the same time as in the case of L+M and L-M, its influence on amplitude extends in a sustained fashion throughout the analyzed window and is not constrained to the period of any specific component. 

Figure 10

 

Linear modeling of ERP waveforms by mechanism contrasts. The left panel depicts the modeling at electrode Oz, representative of the N1 component, while the right panel depicts the modeling at electrode Pz, representative of the LPP component. The blue lines reflect the effects of the model on the averaged waveform for each contrast type, with bootstrapped confidence intervals shown in magenta lines. Straight red lines underneath each waveform indicate the period in which the modeled effect was significant.

Discussion

This EEG study examined whether the presence of chromatic contrast in luminance-defined images alters both performance and neural activity during a shape classification task. Participants classified Gaborized images of objects, nonobjects, or random patch textures defined by different combinations of luminance and chromatic signals and set to mean threshold or suprathreshold contrast levels. The stimuli excited the luminance channel in isolation, the luminance and L-M channels, or the luminance and both the L-M and S-(L+M) channels simultaneously. The goal was to assess the effect of the presence of chromatic contrast through behavioral data and EEG markers of perceptual and cognitive object-related processing (N1, LPP). Classification accuracy for the three types of stimuli was comparable across channel combinations at threshold, confirming that the contrasts were at the level that elicits matched performance. However, a mismatch appeared at suprathreshold: Increases in performance were less pronounced for objects defined by the full combination of signals, resulting in their poorer classification. The first ERP component reliably observed in the waveforms was an N1 peaking 200 to 300 ms after stimulus onset. It occurred earlier and had larger amplitude at suprathreshold for both luminance-only and luminance combined with L-M conditions. The full combination at suprathreshold elicited only a shift in latency but produced the same amplitude as at threshold. Some sensitivity to stimulus class was found in both N1 and LPP, but it was driven mainly by different processing of random patch stimuli, which lacked contour-defined shape, with LPP showing a stronger effect than the N1. Linear modeling of the EEG revealed that while luminance and L-M contrasts modulated EEG specifically within the time windows of the perceptual and cognitive processing markers N1 and LPP, the S-(L+M) contrast had a more sustained, temporally noncircumscribed effect on amplitude. The transition to suprathreshold creates differences in performance for the full-information stimuli, which correspond to ERP findings of less amplitude gain for the full combination of contrasts. We did not find any significant differences between luminance only and luminance with L-M, although the combination with L-M contained much less luminance contrast. In fact, luminance and L-M contrast contributed to the amplitudes of the N1 and LPP in a roughly similar fashion. Based on this, we conclude that L-M chromatic contrast contributes to shape processing when joined with luminance contrast. Meanwhile, S-(L+M) contrast does not provide a direct, facilitatory input into these processes. 

The waveforms we observed were characterized by an absence of a P1-like positive deflection, with the first component being a relatively large N1 akin to those found in studies on chromatic VEPs (Crognale, Switkes, & Adams, 1997; Murray et al., 1986; Rabin et al., 1994). The absence of the P1 is likely due to the relatively low level of luminance contrast in our stimuli (for a similar finding, see Mathes & Fahle, 2007), reflected in the late stabilization of SNR (on average between 120 and 130 ms), which is after the standard P1 window. Further, we observed differences between contrast combinations when contrast level was doubled for suprathreshold stimuli. While luminance alone and combined with L-M signals produces relatively uniform contrast-related N1 amplitude increases and performance benefits at suprathreshold, the full-channel stimulus that also contained S-(L+M) information was not associated with an amplitude increase or an equivalent performance benefit. Meanwhile, the latency benefit from contrast increase was uniformly present across all contrast combinations, although the N1 elicited by a full combination of contrasts did lag behind the other two combinations. Linear modeling demonstrated a more general effect of S-(L+M) contrast on amplitude, which was not restricted to the time window of the N1 and the LPP. We did not test S-(L+M) and luminance combined or S-(L+M) and L-M combined, so we cannot conclude whether the addition of S-(L+M) signals selectively suppresses the gain of the luminance mechanism, the gain of the L-M mechanism, or whether it interacts with both L+M and L-M signals in this fashion. An investigation of detection thresholds for S-cone increments and decrements in the presence of different types of noise masks found that while luminance masks had a similar and weak effect on S increments and decrements, chromatic masks revealed asymmetries between them by exhibiting a stronger masking effect on S increments, most likely due to greater contrast gain control in the unipolar S increment mechanism (Wang, Richters, & Eskew, 2014). Parametric mapping of contrast-response functions for different combinations of luminance and chromatic contrasts conducted across a range of spatial frequencies would extend our understanding of chromatic mechanisms themselves as well as the way in which they interact with luminance. Such experiments should also attempt to model for possible contributions of chromatic aberrations to these neural signals, as Forte, Blessing, Buzas, and Martin (2006) demonstrated that chromatic aberrations can produce neural responses comparable in magnitude with those driven by high-frequency luminance isolating stimuli. 

Our study also aimed to assess whether object sensitivity would be found. Martinovic et al. (2011) used full-information or isoluminant stimuli in an object discrimination task and concluded that object sensitivity of the N1 is brought about by the addition of an achromatic signal. However, the current experiment did not find highly reliable and consistent differences between objects and nonobjects in the ERPs. The only object-sensitive effect in the N1 was found for the full combination of contrasts. This is surprising as luminance information is considered to be the most relevant for object processing (e.g., Bar, 2003; Peterson & Gibson, 1994). The most parsimonious explanation is that the full combination of signals does not scale equally with the increase in contrast for different stimulus classes (see Zele, Cao, & Pokorny, 2007). If object stimuli scaled least favorably of all, this would result in reduced classification performance for objects, while the larger N1 for objects could perhaps be explained through increased difficulty for these stimuli. Still, it is difficult to fathom that the addition of a relatively small amount of S-(L+M) contrast can have such dramatic effects both on performance and on the ERP markers of visual processing, especially as the S-(L+M) signals added to a mixture of L-M and L+M signals at threshold were not found to influence performance in the psychophysical study of Jennings and Martinovic (2014). 

Another difference in our findings compared with those of Martinovic et al. (2011) is that they found differences in both the N1 and the LPP amplitudes elicited by line drawings of objects as opposed to nonobjects, while in this study the most consistent, general effect of shape-specific processing is driven by a differential response for random patches (see Figure 8b). This is most likely due to differences in stimulus material and the associated difficulties of item classification. Line drawings and Gaborized images are likely to engage different perceptual processes to different degrees. For example, studies that compare evoked potentials elicited by grayscale photographic-quality images of objects and their phase-scrambled versions find larger N1s for object images, arguing that this is due to the fact that they engage figure–background processes (Schendan & Lucia, 2010). Gaborized stimuli engage midlevel processes to a much higher level than line drawings as they require some perceptual organization in order to be correctly perceived. N1 seems to be particularly sensitive to perceptual context in midlevel vision tasks (e.g., Machilsen, Novitskiy, Vancleef, & Wagemans, 2011). While N1 showed a series of interactions between the perceptual effects of contrast level, contrast combination, and stimulus type, LPP showed independent effects of these factors (for more detail, see Supplementary Material S1). We failed to replicate previous findings of more positive late potentials for nonobjects than for objects, which were again obtained with line-drawing stimuli (e.g., Gruber & Müller, 2005; Martinovic et al., 2009, 2011). However, we did find increased positivity for random patches—the stimulus class that lacked contour-defined shape. It is likely that the lack of differences between Gaborized objects and nonobjects was due to the fact that they were very closely matched. This is supported by relatively high error rates between these two stimulus classes in this study (see Supplementary Material S2), which are much higher than in any of the previous studies. The LPP was also increased for suprathreshold stimuli compared with threshold stimuli and lower for a full combination of channels, confirming its relation to successful discrimination of contour-defined shapes from contourless patches. 

In conclusion, our study provides further evidence that signals from different channels interact in the visual cortex during shape classification. L-M signals are effectively combined with luminance signals at both perceptual and cognitive stages of processing, while S-(L+M) signals seem to play a different role. Their presence results in a reduced performance benefit at suprathreshold relative to other conditions, and their effects on EEG amplitude are not circumscribed to the time windows of the perceptual N1 or cognitive LPP components. These findings extend psychophysical evidence that L-M contrast contributes to object shape processing provided by Jennings and Martinovic (2014), demonstrating that these contributions occur early in processing, in line with contrast pooling studies by Groen et al. (2012, 2013). The model of Sowden and Schyns (2006) would be able to accommodate for these findings by including signals derived from chromatic spatial frequency channels (for a mathematical definition of these channels, see Zhaoping, 2014). It is generally thought that S-(L+M) contrast contributes largely to color appearance and much less to spatial vision (e.g., Mollon, 1989), but we do find adverse effects on object performance and ERP response amplitudes for suprathreshold stimuli that contain it. Future studies will need to establish whether this is simply due to the fact that their presence alters the slopes of related psychometric functions or whether they play another more general role in spatial vision, which would be a very intriguing prospect. 

Acknowledgments

We thank Karol Puch for his assistance with data collection. This work was supported by the Biotechnology and Biological Sciences Research Council (BB/H019731/1 to J. M.). 

Commercial relationships: none. 

Corresponding author: Jasna Martinovic. 

Email: [e-mail]. 

Address: School of Psychology, University of Aberdeen, Aberdeen, UK.