Symmetry perception for patterns defined by color and luminance

Abstract

Perception of visual symmetry is fast and efficient and relies on both early low-level and late mid- and high-level neural mechanisms. To test for potential influences of early low-level mechanisms on symmetry perception, we used isoluminant, achromatic, and combined (color + luminance) patterns in a psychophysical and an event-related-potential (ERP) experiment. In the psychophysical experiment, pattern contrast was fixed at individual symmetry-discrimination threshold. Participants then judged whether a pattern was symmetric or random. Stimuli at isoluminance were associated with a large bias toward symmetry, achromatic stimuli introduced the opposite bias, and stimuli containing a balance of both color and luminance were perceived without bias. These findings are in line with distinct contrast sensitivity functions for color and luminance, with color providing low-frequency information useful for symmetry detection and luminance providing high-frequency information useful for detection of detail. The subsequent ERP experiment was run at high contrasts to assess processing of symmetry in suprathreshold conditions. Sustained posterior negativity, a symmetry-sensitive ERP component, was observed in all conditions and showed the expected dependence on symmetry. However, interactions between symmetry and contrast type were not observed. In conclusion, while our findings at threshold support models that propose an important contribution of low-level mechanisms to symmetry perception, at suprathreshold these low-level contributions do not persist. Therefore, under everyday viewing conditions, symmetry perception engages a relatively broad cortical network that is not constrained by low-level inputs.

Introduction

Symmetry is pervasive in our environment. Many natural and created objects are symmetric along at least one axis. Fluent perception of symmetry is thus important for fast, accurate perceptual organization, particularly for figure–ground segmentation. From a geometrical standpoint, symmetry is defined as a form of regularity possessed by a mathematical object and is characterized by the operations that leave the object invariant. The main rigid transformations that generate symmetry are reflection (which creates what is also sometimes referred to as mirror or bilateral symmetry), rotation, and translation (i.e. repetition). Patterns can also contain combinations of these transformations, and classifications of these exist for friezes and wallpaper patterns (for an overview, see Wagemans, 1997). Although the human visual system can detect all types of symmetries, reflectional symmetry has been shown to be the most salient (Julesz, 1971; Palmer & Hemenway, 1978; Royer, 1981), particularly if the axis is vertical (Friedenberg & Bertamini, 2000; although see Wenderoth, 1994). 

Symmetry detection is fast and noise resistant (Barlow & Reeves, 1979; Carmody, Nodine, & Locher, 1977; Wagemans, Vangool, & Dydewalle, 1991). It has been demonstrated that symmetry detection is possible after brief presentation times, of 100 ms or less (Carmody et al., 1977; Julesz, 1971; Niimi, Watanabe, & Yokosawa, 2005). Taken together, the speed of detection and resistance to noise may suggest that symmetry processing is supported by early, low-level visual mechanisms, especially those tuned to low spatial frequencies (Dakin & Watt, 1994; Julesz & Chang, 1979). In line with that, most theories of symmetry perception distinguish between two stages of processing: preattentive versus attentive, low-level versus high-level, global versus local. For example, the bootstrap model (Locher & Wagemans, 1993; Wagemans, 1997; Wagemans, Vangool, Swinnen, & Vanhorebeek, 1993) distinguishes between lower order regularities (i.e., point-by-point correlations) and higher order regularities. It suggests that detection is fastest when point-by-point correspondences are also supported by regular higher order structures and slowest when only lower order regularity is available. 

Previous research on putative early, low-level mechanisms of symmetry detection used luminance-defined grayscale stimuli—but what about the contribution of chromatic signals to symmetry perception? Human vision relies on three basic retinogeniculate mechanisms to process contrast: a luminance channel that combines information from medium and long-wavelength cones (L+M) and two chromatic channels that compare information from different cone types (L−M, reddish-greenish; and S−(L+M), bluish-yellowish). Contrast sensitivity functions (CSFs) for chromatic mechanisms are low-pass (Mullen, 1985), offering increased sensitivity to global structure as opposed to fine spatial detail. Meanwhile, the luminance CSF is band-pass, and compared to color it is more sensitive to contrast and orientation at higher spatial frequencies (Wuerger & Morgan, 1999). 

Parraga, Brelstaff, Troscianko, and Moorehead (1998) discuss the mismatch between the CSFs and spatial-frequency content of natural images in terms of luminance and chrominance. The visual system discards high-spatial-frequency chrominance and low-spatial-frequency luminance which are present in the images, presumably because sensitivity to global rather than local color changes is much more useful for the range of detection and identification tasks assumed to lie behind the evolution of M cones in primates (Parraga, Troscianko, & Tolhurst, 2002). Processing of chromatic contrast from an object would thus act to boost the sensitivity to its global structure, while the sensitivity to local detail is already provided by the luminance content. In terms of symmetry perception, chromatic signals would thus be more reliable in signaling global regularities, while luminance signals would provide a reliable source of information on local regularities. 

Bridging the gap between chromatic and luminance mechanisms and symmetry detection, Troscianko (1987) found that while movement detection was not possible for isoluminant stimuli, symmetry detection remained at around 70% correct. Color sensitivity of symmetry detection has recently been reported by studies that relied on isoluminant stimuli (Gheorghiu, Kingdom, Remkes, Li, & Rainville, 2016; Wu & Chen, 2014). Furthermore, spatial integration mechanisms have been found to respond equally effectively to chromatic and luminance signals (Kingdom, Moulden, & Collyer, 1992). In fact, if spatial groupings can be based on either color or luminance information, global structure judgments seem to be more influenced by the chromatic content (Hernandez-Lloreda & Janez, 2001). This may be taken to imply that color signals are as effective as luminance signals for symmetry detection—however, Troscianko (1987) has shown that symmetry perception does benefit from the addition of luminance signals, rising to 80%–85% correct once 8% of luminance contrast is added to the previously isoluminant pattern. Taken together, this implies that chromatic signals may provide an important input into the early, low-level mechanisms that sustain symmetry perception, and that they may be even more effective if combined with luminance signals. 

Studies on the involvement of early, low-level mechanisms in symmetry perception have mainly relied on psychophysical methods. At the same time, neuroimaging studies have complemented these findings by providing a wealth of information on the characteristics of late, extrastriate mechanisms of symmetry perception, while failing to obtain any symmetry-selective involvement of early areas V1 or V2. In a study using functional magnetic resonance imaging (fMRI), Tyler et al. (2005) found symmetry-related activation in extrastriate areas including the lateral occipital complex (LOC). They concluded that high activation in this region might represent a subset of a class of computations that require the integration of information across large parts of the visual field. Another study has found robust fMRI activity in extracortical areas V3A, V4, V7 and the LOC, supporting and extending previous findings (Sasaki, Vanduffel, Knutsen, Tyler, & Tootell, 2005). Importantly, this neural activation was highly correlated with psychophysical perceptions of symmetry and was present irrespective of attention, stimulus type (i.e., dots or lines), configuration, or size. The magnitude of the effect was higher for fourfold symmetry compared to two- or onefold symmetry, for vertical compared to horizontal symmetry, and for perfect compared to noisy symmetry. Furthermore, there was little activation outside the visual cortex, suggesting that symmetry perception does not require modality-independent top-down mechanisms. 

Repetitive transcranial magnetic stimulation over both the left and right LOC has been shown to impair symmetry judgments (Bona, Herbert, Toneatto, Silvanto, & Cattaneo, 2014). However, the disruption was higher following stimulation over the right LOC than over the left LOC, suggesting a degree of right-hemisphere lateralization for symmetry processing. Conversely, disruption of V1 had no effect on symmetry processing. A combined fMRI/electroencephalography (EEG) study on rotational-symmetry perception found that responses in V3 and V4 (∼75 ms) lead to responses in LOC (∼110 ms), consistent with a feed-forward mechanism in which earlier stages of extrastriate processing provide crucial information to later stages (Kohler, Clarke, Yakovleva, Liu, & Norcia, 2016). There was no symmetry response in V1. That study design used steady-state visual evoked potentials, which perhaps explains the early latencies in the extrastriate response. However, Kohler, Cottereau, and Norcia (2018) used an event-related design and found a rotational-symmetry response beginning at ∼150 ms in several extrastriate regions of interest (although not in subregions of LOC1 and LOC2). Similarly, Makin, Wilton, Pecchinenda, and Bertamini (2012) have found surprisingly early P1 modulation for rotational symmetry but not for reflection. 

In line with the evidence suggesting that primary visual areas do not determine symmetry detection, perception of symmetry is generally apparent only in later components of event-related potentials (ERPs). Jacobsen and Höfel (2003) have found that symmetry is associated with a sustained posterior negativity (SPN), largest at parieto-occipital sites and present from 600 to 1,100 ms after stimulus onset. More recently, Makin et al. (2012) recorded EEG while participants performed simple discrimination tasks. First, their results replicated the presence of a symmetry-sensitive SPN in the ERPs recorded from parieto-occipital sites between 300 and 1,000 ms after stimulus onset, irrespective of the variations in the discrimination task. Second, source-localization analysis revealed that the source of this modulation was attributable to activity in the lateralized extrastriate visual cortex, consistent with neuroimaging work (Tyler et al., 2005; Sasaki et al., 2005). Third, the SPN was more pronounced for reflectional symmetry than for rotational symmetry, as would be expected if it reflected a neural marker of symmetry processing. Even though reflectional, rotational, and translational symmetry are all isomeric transformations, behavioral data and the SPN were modulated by the distinct types of symmetry, with the largest effect being elicited by reflection (Makin, Rampone, Pecchinenda, & Bertamini, 2013; for reviews, see Bertamini & Makin, 2014; Bertamini, Silvanto, Norcia, Makin, & Wagemans, in press). Moreover, the same pattern of SPN modulation was present in an explicit discrimination task and in an oddball-detection task that required no symmetry analysis. Fourth, the SPN modulation by symmetry is not confined to central vision: Lateralized responses have been observed when stimuli were presented exclusively in one hemifield (Wright, Makin, & Bertamini, 2017). Finally, further studies have shown that the amplitude of SPN varies with perceptual goodness, as measured by a quantitative model (van der Helm & Leeuwenberg, 1996), particularly in its early part at approximately 350–450 ms (Makin et al., 2016). 

Combined evidence from psychophysics and neuroscience research thus indicates that symmetry perception relies on both early, low-level and late, mid- and high-level visual mechanisms, with an initial retinal frame of reference and subsequent crucial contributions from extrastriate areas V3, V4, and LOC. Imaging studies did not manipulate low-level stimulus properties that could provide differential inputs into early symmetry-sensitive mechanisms (e.g., contrast or spatial frequency). Rather, they controlled image properties—texture granularity and power spectrum for texture patterns (Kohler et al., 2016), or dot size, number, and density for random-dot patterns (e.g., Sasaki et al., 2005). Thus, while these studies provided a wealth of information on later, extrastriate contributions to symmetry perception, the neural signature of early, low-level mechanisms remained elusive. Meanwhile, psychophysical studies provided evidence for early contributions of luminance-driven spatial-frequency filters but did not directly address the potential role of early retinogeniculate mechanisms that provide chromatic information in addition to luminance information. 

To fill this gap in knowledge, we used stimuli defined by different contrast types (chromatic alone, luminance alone, or combined), systematically varying processing within the three retinogeniculate mechanisms. We ran a psychophysical experiment at threshold and an ERP experiment at suprathreshold to assess the contribution of different cone-opponent mechanisms to symmetry perception, in isolation or in combination with luminance. The ERP experiment, using the amplitude of the SPN component as its dependent variable, enabled us to investigate symmetry processing in situations where suprathreshold contrast prevents behavioral effects from emerging. If isoluminant signals contribute to symmetry perception in a comparable way to luminance signals, we should observe the same SPN amplitudes for all our stimuli. In addition, if chromatic and luminance signals provide separate early sources of information for symmetry judgments, with different spatial scales, we should observe enhanced symmetry perception for images that combine color and luminance. By using stimulus contrasts matched at threshold (psychophysical experiment) and suprathreshold (EEG experiment), we can also assess whether the low-level contributions to symmetry perception are independent of contrast. For example, combining luminance and chromatic content might affect symmetry judgments at threshold without altering the SPN symmetry sensitivity at suprathreshold. While cue combination may be beneficial for weak, at-threshold signals, it may not be obligatory, allowing suprathreshold signals to be free of low-level constraints. Such a result would provide evidence that symmetry perception engages a broad cortical network that is not fully constrained by low-level inputs, contextualizing the lack of V1/V2 contributions reported by previous neuroimaging work. 

General methods

Apparatus

All stimuli were centrally presented on a ViewSonic P227f monitor under the control of a Dell PC equipped with a dedicated visual-stimulus generator (ViSaGe; Cambridge Research Systems [CRS], Ltd., Kent, UK). Stimulus presentation was controlled using MATLAB (MathWorks, Natick, MA). The chromatic and luminance output of the monitor were calibrated prior to testing using a ColorCal2 (CRS). Measurements of monitor phosphors by a SpectroCAL (CRS) were used in combination with cone fundamentals (Stockman & Sharpe, 2000; Stockman, Sharpe, & Fach, 1999) to ensure accurate color representation. The background was set to midgray (CIE x = 0.3011, y = 0.3215) with a luminance of 43.24 cd/m². The monitor was switched on at least 30 min before the start of the experiment. Participants sat in an acoustically and electrically shielded chamber, at a viewing distance of 90 cm from the screen, and responded via a button box (Cedrus RB-530, Cedrus Corporation, San Pedro, CA). Stimuli were generated using the CRS toolbox and CRS Colour Toolbox for MATLAB (Westland, Ripamonti, & Cheung, 2012). 

Color space

The DKL color space (Derrington, Krauskopf, & Lennie, 1984) was used to define the chromoluminant properties of the stimuli. Any color can be defined in this space by a chromatic radius r, chromatic angle φ, and luminance elevation θ. Figure 1 shows a representation of the DKL color space with the conditions tested in these experiments. The stimuli were designed to excite the S−(L+M) mechanism (yellowish-bluish), the L−M mechanism (reddish-greenish), or the L+M pathway (luminance; light and dark gray) either in isolation or in controlled combinations. 

Figure 1

 

DKL color space, with its three perpendicular axes corresponding to the L−M, S−(L+M), and L+M mechanisms, was used to specify the chromatic and luminance conditions. The center of the DKL space is also known as the white point, and its CIE 1931 xyY coordinates are set to be equivalent to the background. The DKL space thus represents stimulus contrast within each of the three mechanisms. Any location (for example, point P) in DKL space is described by three parameters: r, φ, and θ, where r is the location's Euclidean distance in three-dimensional space from the center, φ is its angle of azimuth, and θ is its angle of elevation. The angle of azimuth defines the chromaticity (e.g., φ = 0° is reddish, φ = 180° is greenish, φ = 90° is bluish, and φ = 270° is yellowish). The angle of elevation defines the relative amount of luminance. For the same r, larger elevation angles correspond to more luminance contrast relative to chromatic contrast, and vice versa. In (a) and (b), a point is described by its r, φ, and θ values. In (b), points P1–P4 all share the same chromaticity (φ = 270°) and radius, but the contrasts at each point differ. Projecting the lines from P2 and P3 onto the S−(L+M) and L+M axes reveals that at P3, there is more chromatic relative to luminance contrast, while at P2 there is more luminance relative to chromatic contrast. Meanwhile, P1 and P4 have exclusively one type of contrast and are isolating either the luminance or the S−(L+M) mechanism.

Stimuli

The stimulus set consisted of 1,000 symmetric and 1,000 randomly configured geometric patterns (see Makin et al. 2012). Examples are presented in Figure 2. The patterns were generated by producing one triangular quadrant. In that quadrant, there were nine square elements: five different from and four equivalent to the background in brightness (codes used to generate the stimuli are available on Open Science Framework: https://osf.io/UVDXA). The circular and diamond background was used for consistency with previous SPN research (Höfel & Jacobsen, 2007; Jacobsen & Höfel, 2003; Makin et al., 2012). This background highlighted the potential axes of symmetry. The size of the elements varied randomly from 20 to 80 pixels, and the orientation was randomly set to either 0° (vertical and horizontal edges) or 45° (diagonal). The position of the squares was shuffled randomly, and they occluded each other, making continuous, angular substructures. For symmetric patterns, the other three quadrants were mirror images of the first quadrant. For random patterns, three new quadrants were generated according to the same algorithm. The stimulus-generation procedure ensured that symmetric and random patterns could not be distinguished based on the information in a single quadrant. Because of this, the symmetric images were defined by twofold mirror symmetry along the horizontal and vertical axes and varied in color depending on the condition tested (see Figure 2). We chose twofold mirror symmetry because it produces faster reaction times than onefold symmetry (Julesz, 1971; Palmer & Hemenway, 1978; Royer, 1981) and larger SPN components (Makin et al., 2012; Makin et al., 2016; Makin, Rampone, & Bertamini, 2015). The diameter of the circle within which patterns were fitted was 15.66° of visual angle—that is, the outer border of the stimulus could extend up to 7.83° away from fixation. 

Figure 2

 

Stimuli and procedure. In the upper row, symmetric and random patterns are presented in relatively low-contrast versions in each of the five possible contrast combinations—each retinogeniculate mechanism isolated: L+M, L−M, and S−(L+M); and color and luminance mechanisms combined: L−M and L+M, and S−(L+M) and L+M. Note: Appearance of the stimuli in this figure is only an approximation of how they looked in the psychophysical experiment. True depiction is not possible on an uncalibrated device. In the lower row is the experimental procedure for the two-interval forced-choice discrimination-threshold task (psychophysical experiment) and the one-interval forced-choice task (EEG experiment). Note that the stimulus in the EEG experiment has contrast set to 5 times threshold. Also note that the symmetric and random examples for each mechanism are shown in different color in the top row, to give an impression of both poles of that direction in color space.

Protocol

The Cambridge Colour Test (Regan, Reffin, & Mollon, 1994), heterochromatic flicker photometry (Walsh, 1958), and symmetry-discrimination thresholds were conducted in the first session, which lasted around 1.5 hr. The second session included the psychophysical experiment (approximately 1 hr) or the EEG experiment (approximately 2 hr; the actual recording lasted around 50 min in total, with eight blocks of ∼5 min each and short breaks offered after each block). 

Heterochromatic flicker photometry

Individual differences in the luminous efficiency function (Wyszecki & Stiles, 2000) may result in a small luminance signal being present within nominally isoluminant chromatic signals. Therefore, we used heterochromatic flicker photometry (Walsh, 1958) at 20 Hz to measure and afterward adjust each participant's individual point of isoluminance. This technique utilizes the difference in temporal resolution of the luminance channel compared to the chromatic channels, the luminance channel being capable of higher temporal resolution and hence able to process higher frequency flicker. As the luminous efficiency function is mainly driven by L- and M-cone signals, the biggest variations between observers in flicker-photometry settings are always observed for colors from the L−M axis (for an example, see supplementary material from Jennings & Martinovic, 2014). Our procedure required participants to use the button box to adjust the luminance of a chromatically flickering image selected randomly from the stimulus set, until the flicker was reduced to a minimum. One button increased the luminance of the image, while the other button decreased it. The task was repeated eight times, with each measurement starting from a randomized initial luminance value. A basic outlier-rejection method was performed: The highest and the lowest values were eliminated, and the mean for the remaining six values calculated. The individual means were then used as isoluminance settings for each participant. 

Symmetry-discrimination thresholds

Discrimination-threshold measurements employed a two-interval forced-choice (2IFC) paradigm. Two staircases were interleaved, each one testing opposite colors in DKL color space; for example, if one staircase was reddish (φ = 0°), the other would be greenish (φ = 180°). The participant's task was to identify which of the two serially presented images was symmetric. Participants were instructed to guess if they were unsure. Within each trial, a fixation cross was centrally presented for 700 ms, followed by a 2,000-ms display of the first image, followed by a 700-ms fixation and the 2,000-ms presentation of the second image, as presented in Figure 2. While 2,000 ms is a relatively long display time for psychophysics, we nevertheless opted for this time to ensure similar presentation length for psychophysical and EEG experiments. Images were randomly selected from the stimulus set, with no repetitions within the same block. The participants' responses controlled the contrast of the stimuli in an adaptive fashion, using the Weibull function as implemented in the Palamedes toolbox (Prins & Kingdom, 2009) and thus providing the 81% correct threshold. Staircases terminated after eight reversals. On average, this took 32 trials per staircase. No feedback on accuracy was given during the experiment. The order of conditions was balanced across participants to control for order effects. 

Contrast data processing and statistical analysis

The measured thresholds expressed in terms of DKL radius (r), chromatic angle (φ), and luminance elevation (θ) were converted into L−M, S−(L+M), and L+M mechanism contrasts. This was achieved by measuring the CIE 1931 xyY coordinates of each condition's threshold with a spectroradiometer (SpectroCAL, CRS); these xyY values were converted into CIE XYZ tristimulus values and subsequently into L-, M-, and S-cone excitations. The isoluminant conditions were measured at nominal isoluminance (i.e., θ = 0°). The conversion of CIE XYZ values into L-, M-, and S-cone excitations was achieved using a 3 × 3 transformation matrix, derived according to the method outlined by Golz and MacLeod (2003); the Stockman and Sharpe cone fundamentals (Stockman et al., 1999; Stockman & Sharpe, 2000), along with the measured red, green, and blue spectral power distributions from the Viewsonic P227f CRT monitor guns, were used as inputs. Weber cone contrasts for each cone type were calculated from the obtained cone-excitation values. Mechanism contrasts were then computed for L−M, S−(L+M), and L+M. 

We then analyzed the contrasts statistically, as they can provide information on ways in which color and luminance mechanisms sustain symmetry perception. First, to assess if there are any differences between poles of the cardinal axes (e.g., bluish and yellowish, or positive and negative luminance polarity), we conducted paired t tests between increments and decrements in DKL space. We did not expect to see any differences, which would justify collapsing the data across the two poles. Second, to assess if there were differences in the amount of contrast required to reach threshold for stimuli that isolated or combined color and luminance, we conducted paired t tests on such collapsed contrast data. This allowed us to understand if performance in the combined-stimulus condition was driven by color, luminance, or a combination of the two. For example, if the combined stimulus at threshold had significantly less luminance content than the luminance-isolating stimulus at threshold, but the same amount of color signals as the color-isolating stimulus, then performance in that condition was determined purely by the chromatic content (see Jennings & Martinovic, 2014; Jennings, Tsattalios, Chakravarthi, & Martinovic, 2016). Bonferroni corrections for multiple comparisons were used throughout. 

Psychophysical experiment: Symmetry perception at threshold

In this experiment, we fixed the stimulus for each contrast type (color alone, luminance alone, or color and luminance combined) at the threshold required to discriminate symmetric from random patterns measured with a 2IFC task. We then presented participants with a single pattern, which they had to judge as symmetric or random. 

Although average performance on the 1IFC task should remain at threshold (i.e., d′ to discriminate symmetric and random images should be ∼1), differences in accuracy may emerge between symmetric and random patterns, as they are qualitatively different stimuli. A difference in performance without a shift in sensitivity is conceptualized in signal-detection theory as a change in response criterion (Macmillan & Creelman, 2004). For example, if low-pass chromatic signals provide stronger evidence for symmetry, then the presence of such signals may lead to a permissive criterion, in which most shapes are seen as symmetric; in turn, this enhances accuracy for symmetric patterns but reduces it for random patterns. 

The main aim of this experiment was to analyze response biases and establish how different types of contrasts (chromatic and luminance) contribute to symmetry perception. However, the contrast thresholds themselves were also informative. To measure combined thresholds, we set a direction in DKL color space along which chromatic and luminance contrasts covaried by fixing the elevation angle (see Figure 1b); participant responses adaptively controlled the DKL radius along this direction until threshold performance was reached. We could then decompose the DKL radius into constituent chromatic and luminance contrasts. By comparing the mechanism contrast at threshold from single-channel conditions and combined conditions, we could determine which signal type (if any) was driving performance when color and luminance were combined. Naturally, performance was driven by luminance in the luminance-only condition, but in combined contrast conditions, it is possible to establish whether threshold was influenced predominantly by a single channel or a combination of channels (for the same approach, see Jennings et al., 2016; Jennings & Martinovic, 2014). 

Methods

Participants

Twenty-two participants took part in the study, but one was excluded for not being able to perform the task. Twenty-one participants (12 female, nine male; 18 right-handed, three left-handed) remained in the sample, with the mean age of 22 years (range: 18 to 38). They received class credit or a small reimbursement for their time and effort. All participants reported normal or corrected-to-normal visual acuity and had standard color vision as assessed with the Cambridge Colour Test. Individual written informed consent was obtained. The ethics committee of the School of Psychology, University of Aberdeen, approved the study, conducted in accordance with the Declaration of Helsinki. 

Stimuli and procedure

Stimulus contrasts were defined to either isolate each of the three retinogeniculate mechanisms, L−M, S−(L+M), and L+M, or combine chromatic and luminance mechanisms, L−M and L+M; S−(L+M) and L+M. Contrasts were fixed at individual symmetry-discrimination threshold. Participants fixated a cross for 500–700 ms before being shown a single stimulus image for 2,000 ms. They indicated with a button press whether the image was symmetric or random. After the presentation of the stimulus, an X was shown in the center of the screen to notify the participants that they should get ready for the next trial. Button-to-stimulus allocation was balanced so that odd-number participants pushed the right button for symmetry while even-number participants pushed the left button. There were 80 trials per condition, interleaved in eight blocks of 100 trials. To familiarize themselves with the task, participants completed a practice block of 20 trials. 

Statistical analysis of behavioral data

Median reaction times (RTs) were subjected to repeated-measures ANOVAs with factors of contrast type and symmetry (symmetric or random). The contrast types were L−M isolating, S−(L+M) isolating, L+M isolating, combined L−M and L+M, and combined S−(L+M) and L+M. We also computed signal-detection-theory measures of sensitivity (d′) and criterion, to shed more light on parameters underlying behavioral performance. As d′ was computed using symmetric and random responses as signal and noise, these data were analyzed with repeated-measures ANOVAs that had only the factor of contrast type. Since stimuli were fixed at threshold, we expected d′ values to be equivalent. RTs and criteria could still vary between conditions, revealing any differences in the response to symmetry. Greenhouse–Geisser correction for nonsphericity was used, as well as Bonferroni correction for multiple comparisons. 

Results

Contrasts at threshold

Figure 3 depicts contrasts for each of the color/luminance conditions at threshold, obtained using the 2IFC task. Mechanism contrast is depicted on the y axis—with higher contrast, the stimulus has more contrast and thus appears more saturated. Some stimuli should consist of only a single contrast type (achromatic and isoluminant), while others combine multiple contrast types. Inspection of plots makes it clear that isoluminant stimuli were indeed defined purely by chromatic contrast, as their luminance contrasts are equivalent to zero. 

Figure 3

 

Box plot of symmetry contrast thresholds from the psychophysical experiment. To represent the distribution of our data more comprehensively, we chose a box plot. The dark line in the middle of the boxes is the median. The bottom of the box indicates the 25th percentile, while the top of the box indicates the 75th percentile. The T-bars that extend from the boxes (also known as inner fences or whiskers) extend to 1.5 times the height of the box or, if no case has a value in that range, to the minimum and maximum values. The empty circles are outliers, defined as values that do not fall in the inner fences. The asterisks are extreme outliers, defined as cases that have values more than 3 times the height of the boxes. At the bottom of the plot, we also present means and standard deviations for chromatic and/or luminance contrast at threshold in each condition.

We analyzed the contrasts statistically to understand how color and luminance mechanisms sustain symmetry perception. First, to assess if there were any differences between poles of the cardinal axes, we conducted three paired t tests, with a Bonferroni-corrected criterion p value of 0.017. We rectified and then compared negative with positive luminance thresholds—M_neg = 0.042, SD_neg = 0.0096; M_pos = 0.041, SD_pos = 0.0085; t(20) = 0.76, p = 0.46; reddish with greenish—M_red = 0.012, SD_red = 0.0045; M_gr = 0.013, SD_gr = 0.0056; t(20) = −1.99, p = 0.061; and bluish with yellowish—M_blu = 0.075, SD_blu = 0.043; M_yel = 0.067, SD_yel = 0.025; t(20) = 1.03, p = 0.32. As predicted, we found no significant differences. Therefore, we collapsed the data across the poles, obtaining contrast within each of the three retinogeniculate mechanisms: S−(L+M), L−M, and L+M. 

Second, to assess differences in the amount of contrast required to reach threshold for stimuli that isolated or combined color and luminance, we conducted five t tests on such collapsed contrast data, with a Bonferroni-corrected criterion p value of 0.01. The combination of S−(L+M) and luminance were similar in chromatic content to the stimulus isolating S−(L+M), t(20) = 2.71, p = 0.014, d = 0.79, but significantly lower in luminance information than the achromatic stimulus, t(20) = 7.45, p < 0.001, d = 1.65. Likewise, the combination of L−M and luminance had the same amount of color as the stimulus isolating L−M, t(20) = −0.98, p = 0.34, but significantly less luminance than the achromatic stimulus, t(20) = 14.75, p < 0.001, d = 3.23. Contrasting the two combined conditions in terms of luminance content revealed that the combination with S−(L+M) at threshold had more luminance signals than the combination with L−M, t(20) = 10.28, p < 0.001, d = 2.25. Thus, the two combined stimuli were equal to isolating stimuli in terms of chromatic content needed to achieve threshold performance, but they differed in luminance content. While both are lower than achromatic stimuli at threshold, stimuli that combined S−(L+M) contrast with luminance had more luminance content than the stimuli that combined L−M contrast with luminance. This is an important distinction that needs to be accounted for when interpreting the data from the subsequent symmetric/random judgments from single trials. 

Behavioral performance in a single-trial symmetric/random judgment

Figure 4 depicts accuracy and RTs, as well as signal-detection-theory measures of sensitivity (d′) and criterion. We performed a series of repeated-measures ANOVAs on these data to assess whether symmetry perception operated similarly on stimuli defined by color alone, luminance alone, or a combination of the two. 

Figure 4

 

Data from the psychophysical experiment. No differences in d′ (top right panel) are present, but the criterion shift is evident (bottom right panel). The criterion shift can be better understood through consideration of accuracy data (top left panel): There is a response bias to respond “symmetric” for stimuli with more color contrast/less luminance contrast, and a bias to respond “random” when no color contrast is present. For example, there are lower false-alarm rates (i.e., fewer random images identified as symmetric) when the stimulus is defined by luminance alone, compared to higher false-alarm rates when the stimulus is defined by color alone, with hit rates being affected in the opposite direction. In this way, although overall sensitivity is matched between different types of color/luminance combinations (see d′, top right panel), they would fall on different section of a receiver operating characteristic curve, reflecting different response criteria. In terms of reaction times (bottom left panel), responses to symmetry are faster and thus more fluent only for stimuli which are defined mainly by their chromatic content; these are stimuli isolating S−(L+M), isolating L−M, and combining L−M with luminance (for symmetry contrast thresholds, see Figure 3). Error bars are ±2 standard errors.

Measures of d′ and criterion incorporated responses to both symmetric and random patterns, and therefore a single-factor repeated-measures ANOVA was performed on these data. No overall effect of color/luminance combination was observed—F(4, 80) = 1.11, p = .36), with d′ values close to 1 (all ts < 1.87, ps > 0.08)—confirming that our stimuli were equated at threshold across the different conditions. This confirms that thresholds from the 2IFC task were correctly measured and justifies our approach of examining differences in RT and criterion between symmetric and random pictures while ensuring that the contrast at which they were presented enabled similar discrimination between them for each color and/or luminance condition. 

There were indeed significant differences in criterion values, F(2.02, 40.39) = 42.75, p < 0.001, η_p² = 0.68, reflecting the different patterns of responses for symmetric and random figures across the color/luminance conditions. The only conditions that did not differ from each other were the two isoluminant conditions, t(20) = 1.72, p = 0.10 (all other ts > 3.58, ps < 0.002). As can be seen from Figure 4, a negative criterion was observed for all the isoluminant images and the images that combined a low amount of luminance signals with L−M chromatic signals (all ts > 5.01, ps < 0.001), indicating a bias toward perceiving images as symmetric. Conversely, a positive criterion was seen for the achromatic stimuli, t(20) = 4.54, p < 0.001, indicating a bias toward perceiving images as random. This bias was absent for the condition that combined S−(L+M) signals and luminance, as evidenced by the criterion that was approximately zero, t(20) = 0.92, p = 0.37. As shown in Figure 3, these stimuli contained as much color information as did stimuli isolating S−(L+M) at threshold, but more luminance information than the stimuli that combined L−M contrast with luminance. 

RTs during symmetry perception differed across color/luminance combinations, F(2.47, 49.49) = 10.52, p < 0.001, η_p² = 0.35, and even though there was no overarching effect of symmetry, F(1, 20) = 3.59, p = 0.073, a significant interaction indicated that performance for symmetric and random patterns differed depending on the low-level mechanisms that were stimulated, F(2.80, 56.01) = 4.57, p = 0.007, η_p² = 0.19. We performed paired t tests to assess this interaction further. Responses were faster for symmetric compared to random patterns when they were defined by stimuli isolating L−M, t(20) = 4.56, p < 0.001), isolating S−(L+M), t(20) = 5.07, p < 0.001, and combining L−M with luminance contrast, t(20) = 6.12, p < 0.001. They did not differ when S−(L+M) contrast was combined with luminance, t(20) = 0.93, p = 0.37, or when luminance contrast alone was used to define the stimulus, t(20) = 1.99, p = 0.06. 

In summary, the criterion results complemented the RT results, and when contrasts shown in Figure 3 are also considered, color and luminance at threshold combined to facilitate efficient discrimination of symmetry. In other words, to achieve optimal, unbiased performance at threshold, a sufficient quantity of both color and luminance signals was required. 

Spatial-frequency analysis of stimulus images

The observed pattern of results was most probably the outcome of differences in the CSFs of the achromatic and chromatic systems. Chromatic mechanisms are more sensitive to low frequencies (Mullen, 1985; Mullen & Kingdom, 1991) and thus respond more strongly to global regularities while failing to detect local irregularities. Meanwhile, achromatic mechanisms are more sensitive to high-frequency information and are thus able to provide information on local irregularities more than on global regularities. Because the stimuli were at threshold, the low-pass CSF for color was more prone to lead to errors in detecting irregular local details on opposite sides of the symmetry axis in an isoluminant image. Consequently, a random image got misclassified as symmetric. On the contrary, the luminance channel's sensitivity to higher spatial frequencies allowed for local detail to be processed in every image, but unreliable at-threshold local signals on either side of the symmetry axis led to an increased likelihood that a symmetric image was erroneously classified as random. However, it is also possible that systematic differences in the spatial-frequency content present in each symmetric and noise pattern interacted with stimulus contrast to influence symmetry detection. To account for this possibility, we conducted a post hoc spatial-frequency analysis of stimulus images. 

Based on all participants' responses, stimuli that were correctly identified at least 70% of the time (high accuracy) and stimuli that elicited performances of 50% or below (low accuracy) were extracted from the stimulus database. This was done for stimuli presented in both the achromatic and the S-cone-isolating conditions (the two conditions with the most extreme criterion differences) for both pattern types—that is, symmetric and noise. 

Power-density spectra were calculated for each of the eight conditions: achromatic–low accuracy (symmetric); achromatic–high accuracy (symmetric); achromatic–low accuracy (noise); achromatic–high accuracy (noise); S-isolating–low accuracy (symmetric); S-isolating–high accuracy (symmetric); S-isolating–low accuracy (noise); and S-isolating–high accuracy (noise). We then performed a 2-D fast Fourier transform on each stimulus image and shifted the zero-frequency component of the transformed images to the central coordinates. To generate 2-D plots of the power spectrum as a function of spatial frequency, the shifted Fourier transforms of the input images were radially averaged; this was performed over half the transformed spectrum due to the inherent symmetric nature of the transformed spectra. 

Figure 5 shows the power spectra of the stimuli as a function of spatial-frequency content plotted in a log-log space. The left panel shows the high- (red) and low-accuracy (black) symmetric (solid curves) and noise (dashed curves) stimuli presented under S−(L+M) conditions. The right panel shows the same conditions presented achromatically. Each condition was compared to each other via a series of two-tailed paired t tests, with all ps > 0.14. Thus, no differences in the spatial-frequency content exist between stimuli that produced better (correct identification of symmetric patterns or rejection of noise patterns) and worse performance for any condition, presented under either S-isolating conditions or achromatically. So differences in symmetry detection cannot be explained by differences in the stimuli spatial-frequency content, and are more likely to be attributable to the CSF properties of the chromatic and achromatic mechanisms themselves. 

Figure 5

 

Spatial-frequency content of stimulus images. We computed spatial-frequency content for images that led to good and poor performance in the symmetry-detection task at threshold. This was done separately for S−(L+M) and achromatic conditions, as they showed the most prominent differences in judgment criterion. Red lines depict high accuracy and black lines depict low accuracy; full lines depict symmetric patterns, while dashed lines depict random patterns. Note that the results are plotted in log-log space. Typical ranges for standard deviations are indicated within the insert of the left panel.

EEG Experiment: Symmetry perception at suprathreshold

We conducted two separate EEG experiments: Experiment 2a tested stimuli that isolated or combined S−(L+M) and luminance signals, while Experiment 2b tested stimuli that isolated or combined L−M and luminance signals. Within each experiment, we had four contrast-type conditions. Three of them were identical to those used in the psychophysical experiment: isoluminant, achromatic, and a combination with luminance at 30° of DKL elevation. We also added a combination with luminance at 60° of DKL elevation, in which the threshold was expected to have more luminance contrast than in the 30° elevation condition. In this way, we attempted to test a broader range of contrast combinations, to see if our findings generalize across them. The aim was to assess how chromoluminant content affects symmetry perception at suprathreshold, using symmetry-sensitive posterior activation in the SPN window. A further aim was to characterize the spatiotemporal dynamics of this network in more detail, using state-of-the-art mass-univariate-analysis approaches (Pernet, Chauveau, Gaspar, & Rousselet, 2011), as all previous SPN studies have used only traditional ERP methods, loosely distinguishing between “early” and “late” SPN intervals (Makin et al., 2016; Wright et al., 2017). For instance, Makin et al. (2016) have found that SPN amplitude was highest, and most tightly correlated with quantitative models of perceptual goodness, at around 300–400 ms after stimulus presentation. We were interested to see if the patterns from this data set would also emerge as two distinctive windows in EEG-derived spatiotemporal network dynamics. 

Methods

Participants

Experiment 2a's sample consisted of 21 participants, but one was removed because more than 40% of trials were contaminated by artifacts in the EEG and another was removed due to a technical problem with the recording (the stimulus onsets failed to be recorded together with the EEG data). The 19 participants in the final sample were between 19 and 67 years old (M = 27; SD = 12). Experiment 2b's sample consisted of 23 participants, but four were removed due to excessive numbers of trials contaminated by artifacts. The 19 participants in the final sample were between 20 and 38 years old (M = 27; SD = 6). Participants were given a small reimbursement for their time and effort. All participants reported normal or corrected-to-normal visual acuity and had adequate color vision, assessed with the Cambridge Colour Test. Individual written informed consent was obtained. The study was in accordance with the Declaration of Helsinki and was approved by the ethics committee of the School of Psychology, University of Aberdeen. 

Stimuli and procedure

In Experiment 2a, stimuli isolated or combined S−(L+M) and luminance signals, while in Experiment 2b they isolated or combined L−M and luminance signals. 

Participants sat in an EEG recording chamber, with the only light source coming from the monitor. First they were given a practice block of 16 trials to familiarize themselves with the task. The experiment had 70 trials per condition, randomly intermixed and run in eight blocks. After 600–800 ms of fixation, participants were shown a stimulus image for 2,000 ms at five times above threshold contrast, individually measured in a preliminary session (Figures 2, 6, and 10). Stimuli were randomly selected from our stimulus set of 1,000 symmetric and 1,000 random images. No image was shown twice. After stimulus offset, the words “symmetric” and “random” appeared on the left and right sides of the screen to announce which button on the button box corresponded to which response (i.e., if “symmetric” was on the left, then the left button was to be pressed for symmetry and the right for random, and vice versa). Since behavioral judgments were entered after stimulus offset, and response mapping was unpredictable, none of our ERPs could be generated by motor responses or motor preparation. There was no time limit for the response, and the next trial began only after a response was made. Participants were instructed to blink in this period if necessary. 

Figure 6

 

Symmetry contrast thresholds from the EEG experiment with S−(L+M) and luminance contrasts. Plots are constructed as in Figure 3. Comparison with Figure 3 (psychophysical experiment) reveals that the data are highly similar.

EEG recording and statistical analysis

Continuous EEG was recorded from 64 locations using active Ag–AgCl electrodes (BioSemi Active-Two amplifier system, Biosemi, Amsterdam, Netherlands) placed in an elastic cap. In addition, the Common Mode Sense (CMS) electrode was used as a recording reference and the Driven Right Leg (DRL) served as the ground (Metting Van Rijn, Peper, & Grimbergen, 1990, 1991). The EEG signal was sampled at a rate of 256 Hz and epochs lasting 1,500 ms were extracted (starting from 500 ms before stimulus onset to 1,000 ms following stimulus onset). The subsequent EEG data processing was performed using the EEGLAB toolbox (Delorme & Makeig, 2004), as well as self-written procedures running in MATLAB. All trials with incorrect answers were excluded prior to the analysis. Artifact removal was performed by using the FASTER toolbox (Nolan, Whelan, & Reilly, 2010), the ADJUST toolbox (Mognon, Jovicich, Bruzzone, & Buiatti, 2010), and self-written procedures in MATLAB. 

FASTER is an automated procedure that detects contaminated trials and noisy channels that need interpolation (either in the entire EEG recording or on any single trials) by calculating statistical parameters of the data and using a Z score of ±3 as the metric that defined contaminated data. FASTER required data to be referenced to Fz. After conducting FASTER, we re-referenced the data to average reference. ADJUST is an automated procedure that operates on maps resulting from independent component analysis of EEG data, using properties of these components to label them as eye blinks, vertical or horizontal eye movements, or channel discontinuities so that they can be subtracted from the recording. We first rejected trials with global artifacts using FASTER, then ran an independent component analysis and applied ADJUST to the obtained decompositions, then finally conducted channel interpolation with FASTER. In addition, any trials with eye movements were rejected based on ±25-μV deviations from the horizontal electrooculogram in the uncorrected data. Blinks were rejected using a thresholding procedure similar to FASTER (Junghoefer et al., 2000). In Experiment 2a the average rejection rate was 20.46%, and in Experiment 2b it was 14.85%. 

We analyzed the data in the first 1,000 ms after stimulus onset using the mass-univariate approach implemented in the LIMO EEG toolbox, to assess the spatiotemporal development of the symmetry-sensitive response (Pernet et al., 2011). This approach conducts a repeated-measures ANOVA on single-trial data across all electrodes and time points. Our design required a 2 × 4 repeated-measures ANOVA, with factors being image type (symmetric or random) and color (color alone, color with 30° luminance elevation, color with 60° luminance elevation, and luminance alone). In this way, the presence of an interaction would imply that symmetry perception differed per contrast type. We used the method of threshold-free cluster enhancement to correct for multiple comparisons (Pernet, Latinus, Nichols, & Rousselet, 2015), with 1,000 iterations and criterion p value of 0.01. 

The ERP data were also analyzed in a more traditional fashion, investigating SPN amplitudes over the electrodes and time intervals identified through the mass-univariate approach. We expected to obtain congruent findings with the two approaches, with the mass-univariate approach being able to provide more detailed analysis of the dynamics of the symmetry-processing network than that provided by previous ERP work. We also expected to replicate previous observations on isoluminant and luminance-defined waveforms (Jennings & Martinovic, 2015; Martinovic, Mordal, & Wuerger, 2011; Rabin, Switkes, Crognale, Schneck, & Adams, 1994), with the amplitude of the luminance-driven P1 depending on achromatic contrast, and the amplitude of the subsequent N1 depending on chromatic contrast. 

Greenhouse–Geisser correction was used when necessary. Post hoc tests were performed using the Bonferroni–Holm method. 

Results of Experiment 2a: S−(L+M) and luminance

Threshold measurements

Independent t tests confirmed that thresholds for isoluminant, achromatic, and combined stimuli (30° elevation) did not differ from those measured in the psychophysical experiment (all ts < 1, all ps > 0.40). Therefore, our samples were comparable in terms of symmetry discrimination. 

To assess whether there were differences in the amount of contrast required to reach threshold for stimuli that isolated or combined color and luminance, we conducted six paired t tests with a Bonferroni-corrected criterion p value of 0.0083. The combination of S−(L+M) and luminance at 30° was similar in chromatic content to the stimulus isolating S−(L+M), t(18) = 2.80, p = 0.012, d = 0.77, but significantly lower in luminance information than the achromatic stimulus, t(18) = 7.84, p < 0.001, d = 1.65. On the other hand, the combination of S−(L+M) and luminance at 60° was lower in chromatic content than the stimulus isolating S−(L+M), t(18) = 5.95, p < 0.001, d = 1.83, but roughly the same in luminance information as the achromatic stimulus, t(18) = 2.55, p = 0.02, d = 0.60. The two combined stimuli differed significantly from each other, with the 30° combination having more chromatic signal, t(18) = 14.09, p < 0.001, d = 3.86, but less luminance signal, t(18) = 7.84, p < 0.001, d = 1.99. 

Behavioral data

As expected, all participants performed the symmetry-detection task with high accuracy (M = 99% correct; SD = 1%). 

SPN

First, we performed the mass-univariate analyses on the whole-scalp EEG data. Figure 7 depicts the main effect of symmetry, while Figure 8 depicts the main effect of contrast type. As expected, the highest levels of symmetry-related activity were found in two lateralized clusters around electrodes PO7 (left) and PO8 (right). This activity was sustained from around 220 ms until at least 1,000 ms after stimulus presentation, peaking at ∼400 ms in the left parieto-occipital cluster and ∼600 ms in the right parieto-occipital cluster. Significant differences between symmetric and random images were observed not only at posterior but also at multiple anterior sites, especially in the early window of differential activity. On the other hand, the effect of contrast type was first observed at ∼100 ms, reappearing and peaking at ∼200–300 ms, with subsequent sustained activity observed at central sites until ∼700 ms. Maximal differences were found centrally (electrode Iz). Some of the right posterior electrodes that showed large symmetry effects were also found to show large and sustained contrast-type effects. However, no interactions between the two factors were observed. 

Figure 7

 

Main effect of symmetry from the EEG experiment that manipulated S−(L+M) contrast (Experiment 2a). The mass-univariate approach was used to compute the F statistic across all electrodes and time points, to represent the dynamics of the symmetry-sensitive network across time (1,000 ms after stimulus presentation). Cluster-based computational methods were used to correct for multiple comparisons, with a criterion p < 0.01. Electrodes are coded by their number on the Biosemi 64 electrode montage, with six external electrodes also plotted. We used the spatiotemporal dynamics to identify the electrodes of interest, enclosed in red boxes/ovals. The largest symmetry effects were observed at PO7 (electrode 25) and PO8 (electrode 62; the time courses of F values are plotted above and below the representation of the scalp with electrode clusters). There are two time windows of interest, represented by pink boxes: early (300–500 ms) and late (500–800 ms; see also Figure 10). The first window is characterized by activity in a larger network, incorporating both anterior and posterior sites.

Figure 8

 

Main effect of contrast type for the EEG experiment that manipulated S−(L+M) contrast (Experiment 2a). The mass-univariate approach was used to compute the F statistic across all electrodes and time points, to represent the dynamics of the contrast-sensitive network across time (1,000 ms after stimulus presentation). Cluster-based computational methods were used to correct for multiple comparisons, with a criterion p < 0.01. Electrodes are coded by their number on the Biosemi 64 electrode montage, with six external electrodes also plotted. We used the spatiotemporal dynamics to identify the electrodes of interest, enclosed in blue boxes/ovals. For comparison purposes, we also enclose (in red) the electrodes with maximal symmetry sensitivity (see Figure 7). The largest contrast-related effect was observed at Iz (electrode 28). It can be seen that the contrast-sensitive cluster occupies central posterior sites, with some overlap to the left (electrode 27) and right (electrodes 62–64).

The waveforms and difference waves that relate to the more traditional analysis of ERPs are depicted in Figure 9. The figure shows that the ERP waveform at parieto-occipital sites was in accordance with typical contrast-related effects: The P1 component was not present for the condition isolating S−(L+M), which was instead characterized by a delayed and pronounced N1. The presence of S−(L+M) contrast was associated with a more negative-going waveform in the later time window. 

Figure 9

 

Event-related potentials from electrodes in the region of interest from Experiment 2a: the left and right clusters surrounding PO7 and PO8. On the left, waveforms elicited by symmetric (full lines) and random (dashed lines) patterns are shown. On the right, difference waves between symmetric and random patterns are shown for each color and/or luminance condition. The amount of luminance on the stimulus is represented with the thickness and color of the line, so that the red line represents the isoluminant condition, the pink line represents the combination of color and luminance with relatively less luminance content, the gray line represents the combination of color and luminance with relatively more luminance contrast, and the black line represents the achromatic stimulus.

We analyzed the SPN at the left and right sites in the early (300–500 ms) and late (500–800 ms) windows. In the early time window, there were main effects of symmetry, F(1, 18) = 78.19, p < 0.001, η_p² = 0.81, and of contrast type, F(1.53, 27.45) = 28.36, p < 0.001, η_p² = 0.61. No other effects or interactions were significant—electrode site: F(1, 18) = 0.24, p = 0.63; Symmetry × Contrast type: F(3, 54) = 1.14, p = 0.34; Symmetry × Electrode site: F(1, 18) = 0.00, p = 0.95; Contrast type × Electrode site: F(1.41, 25.44) = 0.21, p = 0.73; three-way interaction: F(2.18, 39.18) = 1.67, p = 0.20). In the late time window, we again observed main effects of symmetry, F(1, 18) = 89.85, p < 0.001, η_p² = 0.83, and contrast type, F(1.53, 27.63) = 16.22, p < 0.001, η_p² = 0.47. Other effects and interactions were again not significant—electrode site: F(1, 18) = 2.04, p = 0.17; Symmetry × Contrast type: F(3, 54) = 1.53, p = 0.22; Symmetry × Electrode site: F(1, 18) = 0.50, p = 0.49; Contrast type × Electrode site: F(1.85, 33.23) = 0.74, p = 0.49; three-way interaction: F(3, 54) = 2.46, p = 0.073, η_p² = 0.12. The addition of chromatic content increased the negativity of waveforms, with the most negative SPN for the condition isolating S−(L+M), while the luminance condition had the least negative waveforms. Symmetry had a similar effect, making the waveforms more negative, but there were no robust interactions between the two. 

Results of Experiment 2b: L−M and luminance

Threshold measurements

Figure 10 shows the contrasts needed to achieve symmetry-discrimination threshold. As in the previous experiments, we collapsed the data across the poles of each mechanism. Independent t tests confirmed that thresholds for isoluminant, achromatic, and combined stimuli (30° elevation) did not differ from those measured in the psychophysical experiment—achromatic: t(38) = 1.98, p = 0.055; all other ts < 1, ps > 0.38. Therefore, our samples were comparable in terms of symmetry discrimination. 

Figure 10

 

Symmetry contrast thresholds from the EEG experiment with L−M and luminance contrasts. Plots are constructed as in Figure 3. Comparison of the conditions that are also present in Figure 3 (psychophysical experiment) reveals that the data are highly similar.

To assess whether there were any differences in the amount of contrast required to reach threshold for stimuli that isolated or combined color and luminance, we conducted six paired t tests with a Bonferroni-corrected criterion p value of 0.0083. The combination of L−M and luminance at 30° elevation was the same in chromatic content as the stimulus isolating L−M, t(18) = 0.08, p = 0.94, but significantly lower in luminance information than the achromatic stimulus, t(18) = 31.61, p < 0.001, d = 8.85. Similarly, the combination of L−M and luminance at 60° elevation was also the same in chromatic content as the stimulus isolating L−M, t(18) = 0.99, p = 0.33, but lower in luminance information than the achromatic stimulus, t(18) = 18.95, p < 0.001, d = 4.56. The two combined stimuli did not differ significantly in terms of chromatic content, t(18) = 1.25, p = 0.23, but the 30° combination had less luminance signal than the 60° combination, t(18) = 21.95, p < 0.001, d = 5.76. Thus, in both combined conditions performance was driven by L−M signals, although the amount of luminance differed between them. 

Behavioral data

All participants performed the symmetry-detection task with high accuracy (M = 98% correct; SD = 2%). 

SPN

Figure 11 depicts the main effect of symmetry, while Figure 12 depicts the main effect of contrast type from mass-univariate analyses of whole-scalp data. The elicited activity was similar to that obtained in Experiment 2a, which manipulated S−(L+M) and luminance contrast. The highest levels of symmetry-related activity were found in two lateralized clusters around electrodes PO7 (left) and PO8 (right). This symmetry-related activity was sustained through the whole 1,000-ms period after stimulus presentation, with two peaks: at ∼300–400 ms and ∼600 ms. The most significant effects were present at electrode PO7. As before, significant differences between symmetric and random images were observed at multiple anterior sites, especially in the early window of differential activity. On the other hand, the effect of contrast type was more limited, first observed at ∼100 ms and then reappearing and peaking at ∼200–300 ms, with subsequent sustained activity observed at central sites until ∼700 ms. Maximal differences were found on the right (electrode P6), and it was also at right-side sites that some of the same electrodes that showed large symmetry effects were found to show large and sustained contrast type effects. No interactions between the two factors were observed. 

Figure 11

 

Main effect of symmetry from the EEG experiment that manipulated L−M contrast (Experiment 2b). Method and conventions as in Figure 7. As in Experiment 2a (Figures 6 and 7), the largest symmetry effects were observed at PO7 (electrode 25) and PO8 (electrode 62; the time courses of F values are plotted above and below the representation of the scalp with electrode clusters). There are two time windows of interest, represented by pink boxes: early (300–500 ms) and late (500–800 ms; see also Figure 10). The first window is characterized by activity in a larger network, incorporating both anterior and posterior sites.

Figure 12

 

Main effect of contrast type for the EEG experiment that manipulated L−M contrast (Experiment 2b). Method and conventions as in Figure 8. For comparison purposes, we also enclose (in red) the electrodes with maximal symmetry sensitivity (see Figure 10). The largest contrast-related effect was observed at P6 (electrode 59), at ∼200 ms, but this electrode did not show a sustained activation. The contrast-sensitive cluster again occupies central posterior sites, with some overlap to the left (electrode 27) and right (electrodes 62–64; compare with Figure 7).

We then analyzed the SPN at the left and right sites in the early (300–500 ms) and late (500–800 ms) windows. The ERPs that relate to this more traditional analysis are depicted in Figure 13. In the early time window there were main effects of symmetry, F(1, 18) = 90.69, p < 0.001, η_p² = 0.83, and of contrast type, F(1.99, 35.78) = 35.96, p < 0.001, η_p² = 0.67. No other effects or interactions were significant—electrode site: F(1, 18) = 0.37, p = 0.55; Symmetry × Contrast type: F(3, 54) = 0.80, p = 0.50; Symmetry × Electrode site: F(1, 18) = 0.81, p = 0.38; Contrast type × Electrode site: F(2.32, 41.70) = 1.91, p = 0.16; three-way interaction: F(3, 54) = 0.04, p = 0.99. Similarly, in the late time window we observed main effects of symmetry, F(1, 18) = 71.73, p < 0.001, η_p² = 0.80, and contrast type, F(3, 54) = 17.71, p < 0.001, η_p² = 0.50. Other effects and interactions were again not significant—electrode site: F(1, 18) = 0.28, p = 0.60; Symmetry × Contrast type: F(3, 54) = 0.37, p = 0.78; Symmetry × Electrode site: F(1, 18) = 0.20, p = 0.66; Contrast type × Electrode site: F(3, 54) = 0.52, p = 0.67; three-way interaction: F(3, 54) = 0.39, p = 0.76. The addition of chromatic content tended to make the waveforms more negative, with the most negative response for the condition isolating L−M, while the luminance condition had the least negative waveforms. Symmetry had a similar effect to contrast, making the waveforms more negative, but there were no interactions between the two. 

Figure 13

 

Event-related potentials from electrodes in the region of interest from Experiment 2b: the left and right clusters surrounding PO7 and PO8. Conventions as in Figure 9.

General discussion

We investigated the role of early, low-level neural mechanisms in the perception of symmetry in a combined psychophysical and EEG study. This is an outstanding question, as neuroimaging and neurophysiological studies generally focus on manipulations that affect late, extrastriate contributions to symmetry. We designed our stimuli to differentially excite low-level, retinogeniculate mechanisms that process chromatic and luminance information. Combinations of color and luminance in the stimulus may be beneficial for weak, at-threshold signals, but they may not be obligatory, allowing suprathreshold signals to be free of such low-level constraints. Isoluminant, achromatic, and combined (color- and luminance-defined) patterns were individually adjusted relative to symmetry-discrimination contrast thresholds, which is a standard procedure used to equate the different mechanisms and make them comparable (for a review, see Shevell & Kingdom, 2008). Participants then judged whether a single presented pattern was symmetric or random at threshold (psychophysical experiment) or suprathreshold (EEG experiment). This study is the first to investigate symmetry sensitivity in the EEG across the whole scalp using a mass-univariate approach introduced by Pernet et al. (2011; for a different approach using microstates analysis, see Wright, Mitchell, Dering, & Gheorghiu, 2018). Our results convincingly show that symmetry perception engages a broad cortical network that is not fully constrained by low-level inputs, with broad implications for the field. 

In the psychophysical experiment (Experiment 1), contrasts were fixed to threshold measured using a 2IFC task. This ensured that symmetric and random patches were equally discriminable from each other in each color and/or luminance condition. The d′ values of approximately 1 confirmed that such discriminability was indeed controlled for. Therefore, any distinctive patterns of performance for symmetric as opposed to random images for a color/luminance condition would reveal that such contrast is more suitable for or more fluent in extracting symmetry. Indeed, stimuli at isoluminance were associated with the largest bias toward perceiving a pattern as symmetric, while stimuli that isolated the luminance channel introduced a bias toward perceiving a pattern as asymmetric. Participants did not show a bias when judging the stimulus which contained a mixture of S−(L+M) and luminance signals. This was a stimulus that contained a considerable quantity (although still subthreshold in terms of driving performance on its own) of the luminance signal combined with a chromatic signal that was equal to that in the condition isolating S−(L+M). The result extends the findings of Troscianko (1987) by demonstrating that early mechanisms can combine low-resolution chromatic and high-resolution achromatic contrast to sustain fluent and efficient symmetry perception. 

The subsequent ERP experiments were run at high contrasts to assess whether these early, low-level mechanisms also contribute to neural processing of symmetry in everyday, suprathreshold conditions. Sustained posterior negativity, a symmetry-sensitive ERP component, was observed in all conditions. For the first time, symmetry-sensitive EEG activity was investigated across the whole scalp with a mass-univariate approach introduced by Pernet et al. (2011), revealing dynamics of a network that peaked at parieto-occipital sites and exhibited broadly distributed symmetry sensitivity already in an early time window (∼300–400 ms). Although right posterior sites featured prominently both in the symmetry-sensitive network and in the contrast-sensitive network, our EEG analyses did not reveal interactions between symmetry and contrast type. This suggests that while our findings at threshold support models proposing an important contribution of large-scale, low-resolution information to symmetry perception, at suprathreshold these low-level contributions are not robust. Therefore, under everyday viewing conditions, symmetry perception probably engages a relatively broad cortical network that is not fully constrained by low-level inputs. 

Based on previous EEG evidence, symmetry perception makes use of temporally sustained neural mechanisms that pool information from relatively large receptive fields (Oka, Victor, Conte, & Yanagida, 2007). In line with this, our EEG study shows that color signals are as efficient as luminance signals in driving symmetry processing in the brain. Jennings and Martinovic (2014) report that S−(L+M) signals on their own are quite poor for mid- and high-level visual tasks. However, these tasks required processing of fine spatial detail. When it comes to symmetry judgments on relatively large stimuli, SPN difference waves for stimuli defined by S−(L+M) were similar to those driven by signals from other channels. In fact, our psychophysical experiment showed that in combination with subthreshold—although not insubstantial—luminance signals, S−(L+M) contrast enabled efficient performance by eliminating biases in symmetry perception. This implies that summation of low-resolution chromatic and high-resolution luminance signals provides most reliable information. This finding extends previous reports that low-spatial-frequency information is useful for symmetry perception, provided that the images also contain fine-scale structure (Csatho, van der Vloed, & van der Helm, 2004). 

The S−(L+M) mechanism, which has relatively large receptive fields, might provide a particularly suitable input into low-pass symmetry detectors. Long-range interactions of luminance and S-cone signals have been previously reported for surround suppression (Wade, 2009) and extraction of shapes from backgrounds (Jennings et al., 2016). The current results seem to also point in this direction. However, in our experiment, the combination of L−M with luminance had less achromatic contrast than the combination of S−(L+M) with luminance. Therefore, it is not conclusive that the S−(L+M) mechanism alone is particularly effective in driving symmetry detection. Stimuli isolating L−M in the psychophysical experiment led to similar biases towards symmetry as stimuli isolating S−(L+M). We thus conclude that L−M signals should also be able to combine with a sufficiently large luminance signal to drive efficient performance, implying a general role of color signals in symmetry perception. Indeed, contrast-type-sensitive networks in the EEG experiments (Figures 8 and 12) appear similar for the L−M and S−(L+M) experiments, with a central occipital peak that extends to the right into some of the areas that also belong to the symmetry-sensitive network. 

Our mass-univariate analysis of EEG data provided the first insight into precise spatiotemporal dynamics of the network sensitive to reflection symmetry. It would not be possible to obtain such data using fMRI: Due to its low temporal resolution, it would likely have sufficient signal-to-noise ratio only from the regions with sustained activity over the entire stimulation window, which are the same areas that SPN has been source localized to. However, in the early window of symmetry-sensitive activation (300–400 ms), there is a high degree of symmetry sensitivity that can be observed across the scalp, at anterior and posterior sites. This is the window in which correlations with the holographic model's measure of “perceptual goodness” are most pronounced (Makin et al., 2016), and may thus be the most prominent part of the symmetry-sensitive brain response. Future source-localization studies need to focus on this early window to assess which source areas play a direct role in reflection-symmetry perception. A beamforming reanalysis of the data from Makin and colleagues (e.g., Makin et al., 2016) would be a fruitful route toward that aim. This would extend findings on source space dynamics which are currently available for rotational symmetry in wallpaper patterns (Kohler et al., 2016) to reflection symmetry from geometric or random-dot patterns. 

Future work could also examine presence of SPN under conditions where contrast is near symmetry-detection threshold. The symmetry threshold would need to be sufficiently above the basic detection threshold (i.e., threshold for reliably detecting that a stimulus is indeed present) to allow for a good signal-to-noise ratio in the EEG, as reliable visual evoked potentials are only elicited by stimuli above detection threshold (Campbell & Maffei, 1970). It could then be investigated whether the biases found in our psychophysical experiment are discernible in the neural response. That is, there might be no SPN for achromatic at-threshold symmetries which are classified as asymmetric, while an SPN would appear for isoluminant at-threshold asymmetries which are classified as symmetric. Further, SPN could be sensitive to the amount of chromatic contrast or low-spatial-frequency contrast in an at-threshold stimulus, so that the SPN amplitude is larger for a stimulus with more chromatic/less luminance content than for a stimulus with less chromatic/more luminance content even when they are matched for equal performance. This at-threshold EEG approach would combine excellently with the analysis of stimulus-locked and response-locked activity to assess perceptual- and decision-related stages in symmetry perception (as in Kohler et al., 2018). 

Overall, our results provide new information on early, low-level contributions to symmetry perception and integrate chromatic channels into models of symmetry perception. Contributions of lower and higher spatial-frequency channels, which have already been explored psychophysically, could be contrasted in an EEG study using a similar approach to ours. The mass-univariate analysis of EEG activity is a method that provides insight into the neural dynamics of symmetry processing. The highest symmetry-sensitive responses were found to be temporally subscribed to the window of activity previously shown to be particularly sensitive to perceptual goodness. In conclusion, we demonstrate that symmetry perception engages a relatively broad extrastriate network that is not fully constrained by low-level inputs once contrasts are set to high, suprathreshold levels, contextualizing the lack of V1/V2 contributions reported by previous neuroimaging work. 

Acknowledgments

We would like to thank Sanni Ahonen, Alexander Donald, Tomohawk McGinn, Clara Mutschler, Lilja-Maaria Kurppa, and Sophia Thompson for assistance with data collection. MB was supported in part by a grant from the Economic and Social Research Council (ES/K000187/1). JM was supported by a grant from the Biotechnology and Biological Sciences Research Council (BB/H019731/1). 

Commercial relationships: none. 

Corresponding author: Marco Bertamini. 

Email: . 

Address: Department of Psychological Sciences, University of Liverpool, Liverpool, UK.