Perceptual resolution of ambiguous neural representations for form and chromaticity

Abstract

A coherent percept of our visual world is important for functioning. Ambiguities, however, are implicit in visual neural representations and must be resolved for stable perception of objects and scenes. Grouping processes can link multiple neurally ambiguous fragments across the visual field. Experiments here determined how multiple visual features of each fragment contribute to perceptual resolution of ambiguity by grouping. Chromatic interocular-switch rivalry, a technique for presenting competing dichoptic images, was used to induce ambiguous neural representations for equiluminant chromatic discs and gratings. Two dichoptic stimuli were presented simultaneously to measure the amount of time they both appeared the same in at least one feature domain. The two stimuli were grouped when they appeared to share ambiguous features such as color, orientation, and spatial frequency more often than chance. Experiments here tested whether unshared and unambiguous features impeded grouping of the ambiguous components. Overall, the results show that grouping can be driven by neural ambiguity that is common for fragments across the visual field, even when the fragments also have other unshared, unambiguous features.

Introduction

Neural ambiguity is implicit in representations of visual scenes because of limitations in translating physical light on the retina into biological representations (Barlow, 1981). This ambiguity is resolved by neural processes that lead to conscious perception, though much remains unknown about the neural mechanisms that determine the perceptual resolution of ambiguous neural responses. Moreover, resolution of ambiguity goes beyond disambiguating the features of an object in view (for example, its color or shape). Disambiguation requires also grouping together distinct fragments of a retinal image that are perceived to be parts of a single object. This process of grouping has been described as the visual system detecting “suspicious coincidences” among neural representations as cues to link regions of the visual field for coherent perception (Barlow, 1986). 

Grouping these regions or fragments is critical for creating a coherent interpretation of the external world. While grouping of fragments can depend on their features in common (Wertheimer & Riezler, 1944; Alais & Blake, 1999; Silver & Logothetis, 2004; Wu, Kanai, & Shimojo, 2004; Papathomas, Kovács, & Conway, 2005; Suzuki, Wolfe, Horowitz, & Noguchi, 2013; Wang & Shevell, 2014; Shevell & Wang, 2016), the specific relation between common features and neural disambiguation by grouping remains a thorny problem. Here, specific hypotheses about the resolution of neural ambiguity were tested by creating neural ambiguity for various features of fragments. The aim was to determine how ambiguous and unambiguous features interact to alter perceptual resolution of neural ambiguity. 

Previous research has shown that multiple features in common across ambiguous representations can lead to increased viewing time with all fragments appearing the same (Papathomas et al., 2005; Kim & Blake, 2007). Specifically, chromatic features in common may cause the ambiguous fragments to be resolved to the same form more often than when fragments do not share a chromaticity in common. Thus, resolution of ambiguous neural representations for each feature may occur separately, but when multiple features can be resolved together they may interact with each other to resolve their ambiguity. 

Further, ambiguity in the same feature dimension across the fragments (e.g., ambiguous neural representations for chromaticity) may be a sufficient cue for grouping to occur, regardless of other, unambiguous features in the fragments (e.g., all fragments perceived as the same color no matter their unambiguous forms). Color is used here as a model system to determine how similarity in one feature domain (form) affects perceptual resolution of ambiguity in another domain (chromaticity). Similarly, perceived form is also measured to determine how similarity of chromatic neural representations affects perceptual resolution of ambiguity for form. 

While grouping of ambiguous visual representations (of chromaticity or form) is well known (e.g., Kovács, Papathomas, Yang, & Fehér, 1996; Ngo, Miller, Liu, & Pettigrew, 2000; Slezak & Shevell, 2018), fundamental questions remain about grouping with multiple stimulus features. Must neural representations have all features matching in order to be grouped? Can binocularly integrated yet ambiguous neural representations lead to grouping? The following experiments test the hypothesis that grouping is acting on binocularly integrated representations that share ambiguous features (i.e., have the same kinds of competing neural representations) but do not necessarily share unambiguous features. 

Methods

Stimuli that create neural ambiguity

The resolution of neural ambiguity in vision can be studied by presenting conflicting visual stimuli to the two eyes. Rivalrous dichoptic stimuli traditionally have been presented steadily at corresponding locations on each retina, a method referred to here as standard binocular rivalry (SBR). SBR usually causes observers to perceive slow alternations between two percepts (Wheatstone, 1838; Levelt, 1965). These alternating percepts have been accounted for by both monocularly and binocularly driven neurons (Kulikowski, 1992; Kovács et al., 1996; Lee & Blake, 1999; Ngo et al., 2000; Blake, 2001). A newer variant of SBR is interocular-switch rivalry (ISR), which also presents dichoptic stimuli to corresponding retinal locations but swaps the stimuli between the two eyes several times a second (typical swap rate between 1.5 and 6 Hz, implying three to 12 swaps each second; Logothetis, Leopold, & Sheinberg, 1996; Christiansen, D'Antona, & Shevell, 2017). This method, when used with equiluminant rivalrous chromaticities, produces slowly alternating color percepts lasting about 2 s on average (that is, longer than six or more chromaticity swaps in each eye). These sustained color percepts can be accounted for by competition between binocularly driven neurons without any monocular rivalry mechanism (Christiansen et al., 2017; Slezak & Shevell, 2018). Equiluminant chromatic ISR causes slow alternating color percepts without adding higher frequency on-off flicker, as required for luminance-defined switch rivalry (Logothetis et al, 1996, Lee & Blake, 1999; Denison & Silver, 2012). All conditions in the following experiments used equiluminant chromatic stimuli, whether rivalrous in chromaticity or form or both, so all experiments here used ISR without any on-off flicker. 

Apparatus

Stimuli for all experiments were presented on a Sony CRT monitor (Model CPD-G520P) driven by an iMac computer. Observers viewed the display through an eight-mirror haploscope so that the image from the left half of the display was seen by only the left eye, and the right half by only the right eye (Figure 1a). A chin rest ensured a stable head position. The path length was 115 cm from the screen to the observer's eyes through the mirrors. The pair of mirrors closest to the observer was mounted on a saddle secured to a triangular rail so it was able to move toward or away from the observer to adjust for fusion of the left- and right-eye images. The stimuli presented to each eye were surrounded by a thin white rectangular line with Nonius lines to aid fusion (Figure 1a). The top and left Nonius lines were presented to the left eye, and the bottom and right Nonius lines to the right eye. Correct fusion was achieved when the top and bottom Nonius lines were perceived to align vertically and the left and right lines horizontally. 

Figure 1

 

Apparatus and stimulus arrangements. (a) A mirror haploscope display (as in Slezak & Shevell, 2018). The black rectangle represents the CRT monitor, the solid gray lines represent front-surface mirrors, and the dashed lines indicate the light paths. (b) A single disc presented below fixation in each eye (single discs could also appear above fixation). (c) Two discs presented to each eye, one above and one below fixation. Each disc was 1.5° in diameter and centered 1.5° from the fixation cross.

Stimuli

The stimuli for each experiment were 1.5°-diameter discs. Three stimulus arrangements were used: a single disc presented with its center either 1.5° above or below a white fixation cross (with fixation centered in the rectangle and Nonius lines; Figure 1b) or two discs presented, one 1.5° above fixation and one 1.5° below (Figure 1c). Conditions with only one stimulus on the top or bottom were used to create independence predictions that quantified how often an observer should perceive both the top and bottom discs to be the same color if the neural representations for top and bottom were resolved independently of each other. 

When both a top and bottom disc were displayed, they could be presented either conventionally or as a patchwork (Kovács et al., 1996). Conventional stimuli had the identical features in the discs above and below fixation in a single eye at any given moment (Figure 2, first row); patchwork stimuli had different features presented to a single eye at any given moment so that the top and bottom discs in each eye did not match (Figure 2, second row). Conventional stimuli could result in seeing the same features above and below fixation due to monocular dominance, but patchwork stimuli could result in seeing the same features above and below fixation due to only a binocular representation. 

Figure 2

 

Stimuli (top two rows) and percepts measured (bottom row) in Experiment 1A. Conventional stimuli (top row) had the same chromaticity for each stimulus in a given eye, while patchwork stimuli (middle row) presented two different chromaticities to each eye. The measured percepts (bottom row) were for those periods when identical colors were seen above and below fixation. The figure shows one possible set of stimuli; grating orientation was counterbalanced across trials.

The discs themselves could be one solid chromaticity (Figure 2, left column) or square-wave gratings of various orientations and spatial frequencies (Figure 2, right three columns). The orientations used were 45° and 135°, and the spatial frequencies were 1.0, 3.3, and 5.0 cycles per degree (cpd). Discs were presented in chromatic ISR with each disc swapping between the eyes at a frequency of 3.75 Hz (that is, 133-ms presentation duration before each swap or, equivalently, 7.5 chromaticity swaps each second). 

Stimulus chromaticities were defined in MacLeod–Boynton color space (MacLeod & Boynton, 1979) with the unit of S/(L + M) arbitrarily scaled to 1.0 for equal-energy-spectrum “white.” The chromaticities were [L/(L + M), S/(L + M)] of [0.72, 0.3], called red; [0.61, 0.3], called green; and [0.665, 1.0], called gray. All chromaticities were at 5.0 cd/m² on a dark background (about 0.3 cd/m² measured at the face of the CRT). Stimuli with color and/or form rivalry were used in the experiments. In instances with color rivalry, one of the two rivalrous solid discs in one retinotopic location was “red” and the other “green”, or one was a “red/gray” grating while the other was a “green/gray” grating. In instances without color rivalry, the rivalrous gratings at each retinotopic location were both “red/gray” or both “green/gray” at orthogonal orientations. In all instances, form was defined by equiluminant chromaticity differences. 

Procedure

All experiments took place in a dark room using the haploscope (Figure 1a). Observers were instructed to press and hold buttons on a game pad for the entire duration of a particular percept. For each experiment, specific percepts were assigned to one of three possible response buttons, as described on an instruction screen prior to starting each trial. Specific button assignments are discussed for each experiment.¹ A 70-s trial followed the instruction screen. The first 10 s of each trial were ignored due to possible differential adaptation in the two eyes from the initial phase of ISR and to avoid any possible onset bias (Carter & Cavanagh, 2007; Stanley, Forte, Cavanagh, & Carter, 2011). 

The first day of the experiment was considered practice, and its results were not analyzed. Three, six, or 12 days of experimental sessions followed, depending on the number of conditions and counterbalanced trials required for each experiment. 

Observers

Each observer provided written informed consent before participating in the experiments, as required by the University of Chicago Institutional Review Board. Observers were screened for normal stereoscopic vision using the Titmus Stereo Test and for normal color vision using Ishihara plates and Rayleigh matches conducted with a Neitz anomaloscope. After completion of this screening, heterochromatic flicker photometry, repeated on three days, was used to determine equiluminance for each observer. Each individual's equiluminance measurements were applied during their experimental sessions. One observer never experienced any sustained percepts from ISR and so was excluded from the study. 

The null hypothesis: Determining the probability of independence

The probability that both of two independently resolved ambiguous neural representations will yield the same percept was calculated for each measured two-disc percept. This represented the chance probability of two discs appearing identical. The proportion of time that the top disc when presented by itself was perceived as a given stable percept was multiplied by the proportion of time the bottom disc presented by itself was perceived as the same stable percept. This gives the predicted proportion of time both discs will be seen as the same given percept if each disc's percept is resolved independently of the other. Independence predictions were compared to measurements with both discs presented simultaneously. If the two discs presented simultaneously appeared the same significantly more often than the independence prediction, then independence was rejected in favor of a grouping mechanism that linked the appearance of the two perceived discs. 

Results

Experiment 1A: Grouping chromatically rivalrous representations

Previous work has shown that multiple chromatically rivalrous discs viewed simultaneously and presented with ISR all appear the same color far more often than chance (Slezak & Shevell, 2018). This implies grouping among the discs; that is, the color of each rivalrous disc in view is not determined independently of the others. Grouping was tested here with two circular stimuli in the fused percept (Figure 1c), first replicating the earlier results with only two solid discs and then testing whether grouping was found also with chromatic gratings at spatial frequencies of 1.0, 3.3, or 5.0 cpd (Figure 2). In the first experiment here, all discs on each trial were identical spatially with ambiguous chromaticity (either a uniform disc or a grating of some spatial frequency and orientation; Figure 2). In separate trials, grating orientation was counterbalanced at 45° or 135°. Both conventional and patchwork binocular presentation were tested (Figure 2, rows 1 and 2, respectively). The fused percept was always two circular fields, one above fixation and one below it (Figure 2, row 3), except in one-field stimulus conditions included to test the independence hypothesis. 

Specific procedures and observers

Observers were instructed to fixate on the central cross and hold down a particular button on a game pad to indicate whether they perceived either a single disc (for stimuli in Figure 1b) or both discs to be a particular sustained color (red or green for stimuli in Figure 2). Separate buttons were used for each color. If the percept was a patchy mixture, an inhomogeneous disc, or rapidly changing hues several times a second (the stimulus swap rate), observers were instructed not to press any button. 

Eight observers participated in the study. Four of the observers completed all four spatial-frequency conditions shown in Figure 2, while the other four completed only the 3.3-cpd conditions. Observers who participated in all spatial-frequency conditions completed both conventional and patchwork runs, while those tested at only 3.3 cpd did only the patchwork condition. Observers who completed all spatial frequencies required 12 sessions on separate days, and those who completed only the 3.3-cpd condition needed three days. Repetitions were split up in sets of four per day. 

Measurements

The proportions of time out of 60 s that observers perceived discs as either both red or both green were added together to give the total proportion of time when both discs were seen to have identical color. An arcsine transformation was applied prior to statistical analyses in order to better approximate the assumption of normality implicit in statistical tests of proportional data (Kirk, 2013, p. 107). Measurements at each spatial frequency were compared to the (arcsine-transformed) independence prediction, calculated separately for each condition and each observer. For every spatial frequency tested, planned contrasts were performed to determine if the conventional and patchwork conditions were significantly different, as predicted if a monocular neural representation can influence perception of the two discs. No observer ever showed a significant difference between patchwork and conventional stimulus presentation for any of the conditions tested (a total of 16 separate tests). Results from one observer are shown in Figure 3a. Further analyses averaged over the patchwork and conventional conditions at each spatial frequency. 

Figure 3

 

Results from Experiment 1A. (a) Results for one observer comparing the proportion of time that the two regions appeared red (red portion of stacked bar) or green (green portion of stacked bar) for the conventional and patchwork stimuli (Figure 2). The different groups on the horizontal axis are different spatial frequencies of gratings. (b) Results for all eight observers in the 3.3-cpd condition. Saturated colors represent two-disc measurements; pale colors represent independence predictions derived from top and bottom discs presented alone. (c) As in (b), except results are for four observers at every spatial frequency tested. The horizontal axis shows the different spatial frequencies. Error bars show +1 standard error of the mean for red + green. ns = p > 0.05, *p < 0.05, **p < 0.01, ***p < 0.001.

Planned orthogonal contrasts compared the sum prediction (both discs red + both discs green) for independence to the sum of the two-disc measurements to test for significant grouping. For seven of the eight observers who viewed the 3.3-cpd condition (each observer analyzed separately), the two-disc measurements were significantly greater than the independence prediction, clearly rejecting the independence hypothesis (Figure 3b). For the four observers who viewed all three spatial-frequency gratings and the whole discs, the two-disc measurements were significantly greater than the independence predictions in all 16 separate tests (4 observers × 4 conditions; Figure 3c). 

Separate analyses were conducted on the two-disc measurements alone to test for an effect of spatial frequency on the resolution of ambiguity from gratings with rivalrous chromaticities. Three of the four observers who viewed the four different spatial-frequency conditions (all but observer EL) showed a significant effect of the spatial frequency of the gratings, which were always matched in orientation. The three observers with significant effects, however, did not show a consistent trend as a function of spatial frequency (Figure 3c): The largest proportion of time could be at the highest or lowest spatial frequency tested, depending on the observer. Also worth noting is the wide ranges of proportions of viewing time in which observers perceived any stable color: from observers BS, RH, and RR with less than 25% of the total time with stable percepts to observers EL, ES, and AG with sometimes over 75% of the total time with stable percepts (Figure 3b and 3c). Despite these individual differences, all observers showed significant perceptual grouping in every spatial-frequency condition, as discussed in the previous paragraph. Thus, these individual differences strengthen the generality of the grouping conclusion. 

Experiment 1B: Grouping chromatic- and orientation-rivalrous representations

Experiment 1A showed that two chromatically rivalrous representations are not resolved independently of each other over a wide range of spatial frequencies (1.0–5.0 cpd). In all of those cases, the equiluminant chromatically defined form was consistent across the two eyes' stimuli, so there was chromatic rivalry but no form rivalry. In this experiment, dichoptic gratings (spatial frequencies ranging from 1.0 to 5.0 cpd) differed in orientation and, sometimes, also in chromaticity (Figure 4). This allows for grouping of objects with a common chromaticity, spatial frequency, or orientation. The experiment also introduced measurements of plaids, which are considered further in Experiment 2 and followed up more thoroughly in Experiment 3. Perceptual superposition of two eyes' representations, as with a plaid, has been reported (Anderson, Bechtoldt, & Dunlap, 1978; Wolfe, 1983; Liu, Tyler, & Schor, 1992; Burke, Alais, & Wenderoth, 1999) but often not examined; instead, plaids have been lumped together with inhomogeneous percepts indicating incomplete dominance of one neural representation over another (e.g., Logothetis et al., 1996; Lee & Blake, 1999; Papathomas et al., 2005). Plaid is measured explicitly here to determine if grouping occurs for an integrated binocular percept (plaid) rather than only for individual component stimulus features. 

Figure 4

 

Stimuli and percepts measured in Experiment 1B. (a) Stimuli (above) that rival only in orientation (one of the two possible nonrivalrous chromaticities is shown) and the percepts measured (below): both discs perceived at 45°, both 135°, or both plaid. (b) Stimuli that rival in both orientation and chromaticity (top two rows) and the percepts measured (bottom row). Conventional stimuli (top row) had the same chromaticity and orientation for each stimulus in a given eye, while patchwork stimuli (middle row) had two different chromaticities and orientations in each eye. The measured percepts (bottom row) are for the identical perceived color and orientation (or plaid) both above and below fixation. The figure shows one possible set of stimuli; orientation/chromaticity pairing was counterbalanced across trials.

As in Experiment 1A, grating stimuli were tested at 1.0, 3.3, and 5.0 cpd, and all gratings in a given trial had the same spatial frequency (Figure 4). Unlike in Experiment 1A, orthogonal gratings were presented dichoptically, so grating orientation was always rivalrous (the gratings in each retinal location were counterbalanced at 45° or 135°). In some conditions, stimuli were rivalrous in only orientation (thus all gratings either were red or green; Figure 4a), while in others the stimuli had both chromatic and orientation rivalry (Figure 4b). As in Experiment 1A, additional conditions with the top or bottom discs alone were run in order to test the independence hypothesis. 

Specific procedures and observers

Observers were instructed to fixate on the central cross and hold down buttons on a game pad to indicate whether they perceived either a single grating (as in Figure 1b) or both gratings to be of a particular sustained orientation (45° or 135°, as in bottom row of Figure 4a), a combination of orientation and color (e.g., red and 135°, or green and 45°), or a plaid (bottom row of Figure 4b). Separate buttons were used for each percept. If mixed or patchy percepts or rapidly changing hue were perceived, observers were instructed not to press any button. The same eight observers participated in Experiment 1B as in 1A. Four of the observers completed all four spatial-frequency conditions shown in Figure 4b, while the other four completed only the 3.3-cpd conditions with only orientation rivalry (Figure 4a) and both chromatic and orientation rivalry (Figure 4b, middle column). The observers who participated in all spatial-frequency conditions completed both conventional and patchwork runs (Figure 4b, rows 1 and 2), while those who completed only the 3.3-cpd tests did only patchwork conditions. Observers who completed all spatial frequencies required 12 sessions on separate days, and those who completed only the 3.3-cpd conditions needed three days. Repetitions were split up in sets of four per day. 

Measurements

Similar to Experiment 1A, measured and predicted proportions for 45° and 135° percepts (or red and green percepts) were measured separately and then added together. An arcsine transformation was applied prior to statistical analyses. The plaid results were also arcsine transformed and analyzed separately. As previously found in Experiment 1A, there was never a significant difference between the conventional and patchwork stimuli, so they were averaged at each spatial frequency. Planned orthogonal contrasts were performed comparing each two-disc measurement (average of patchwork and conventional conditions when both configurations were tested) to its independence prediction. Conditions in which an observer had a stable percept for a cumulative time less than 5% of the total viewing time were excluded from analysis to avoid a floor effect. 

Measurements from conditions with nonrivalrous chromaticity and rivalrous orientations (Figure 4a) showed significantly greater proportions of time when the two gratings were seen to be of the same orientation than predicted by independence for only two of five observers (Figure 5a). The other three observers had insufficient data (<5% stable grating percepts), primarily because the dominating stable percept for them during the 60-s measurement period was plaid (analyzed separately later; Figure 6). 

Figure 5

 

Results from Experiment 1B (grating percepts). (a) Results from gratings with only orientation rivalry, showing the proportion of time (vertical axis) that both gratings appeared with the same orientation. Darker bars are the two-disc measurements and lighter bars are the independence predictions (the darker of the two stacked bars, where visible, represents 135°, and the lighter 45°). (b) Results from grouping two 3.3-cpd gratings by color and orientation for all eight observers (horizontal axis). The vertical axis represents the proportion of time (out of 60 s) when observers saw both discs as red in its presented orientation (135°, for instance) and both as green in its presented orientation (45°, for instance; stacked bars). Saturated colors represent two-disc measurements and pale colors the independence predictions. (c) Results for four observers for all spatial frequencies tested. The vertical axis represents the proportion of time (out of 60 s) that observers perceived both discs as having the same color and orientation (stacked bars for red and green percepts). Saturated colors represent the two-disc measurements, and pale colors the independence predictions. The horizontal axis shows the different spatial frequencies tested. Error bars show +1 standard error of the mean for red + green or 45° + 135°. ns = p > 0.05, *p < 0.05, **p < 0.01, ***p < 0.001. Bars without asterisks were not analyzed because the proportions did not reach the 5% floor criterion.

Figure 6

 

Results from Experiment 1B (plaid percepts). (a) Measurements for five observers comparing (i) each 3.3-cpd plaid measurement to its independence-hypothesis prediction and (ii) single-colored plaid percepts (i.e., nonrivalrous chromaticities) to two-colored plaid percepts (horizontal axis). The vertical axis represents the proportion of time (out of 60 s) that observers perceived both discs as plaid. Saturated yellows represent the two-disc measurements and pale yellows represent independence predictions. (b) Results for four observers for all spatial frequencies tested. As in (a) except that the horizontal axis shows the different spatial frequencies of the stimuli. Error bars are +1 standard error of the mean. ns = p > 0.05, *p < 0.05, **p < 0.01, ***p < 0.001. Bars with no asterisks were not analyzed because the proportions did not reach the 5% floor criterion.

Results from the eight observers who viewed the 3.3-cpd gratings with both chromatic and orientation rivalry show significant grouping for five of six observers (the remaining two observers' sums of red and green were less than 5% of the total viewing time and therefore excluded from analysis; Figure 5b). Results from the four observers who viewed all spatial-frequency conditions showed that the two-disc measurements were significantly higher than the independence predictions for 1.0-cpd gratings (all observers, p < 0.05) and for 3.3- and 5.0-cpd gratings (two of two observers, p < 0.05; the remaining two observers did not reach 5% of the total viewing time; Figure 5c). 

Measurements of plaid percepts were analyzed separately to test whether grouping can occur with percepts requiring superimposed images from both eyes. Planned orthogonal contrasts compared the two-disc plaid measurements to their independence predictions. For the single-colored plaids (Figure 4a), two of the five observers perceived both discs to be plaid more often than the independence prediction (Figure 6a). One of the three observers who did not reject the independence hypothesis (observer ES) had a ceiling effect (measurements above 95% of total time). All five observers who viewed both the conditions with single-colored (Figure 4a) and two-colored 3.3-cpd orthogonal grating stimuli (Figure 4b, middle column) perceived significantly more plaid when the component gratings were of the same chromaticity (Figure 6a; p < 0.05 for each observer). Results from the four observers who viewed all the spatial-frequency conditions were mixed (Figure 6b). Three of four observers showed significant grouping with 1.0-cpd stimuli, and two of three showed significant grouping for 3.3- and 5.0-cpd stimuli (the fourth observer did not reach the 5% floor criterion). 

Separate analyses of variance were performed on the two-disc measurements for both individual orientations and plaid percepts to test for an effect of spatial frequency on the resolution of ambiguity. Two of the four observers who viewed all the spatial frequencies showed a significant effect for spatial frequency on the percepts of gratings tilted in the same direction and of the same color. The results may be driven by the 1.0-cpd condition (Figure 5c). Three of the four observers showed a significant effect of spatial frequency on plaid percepts, but the effect was not consistent across observers (Figure 6b). Most importantly, despite any effect of spatial frequency, grouping for plaids was often found across the spatial frequencies. 

Experiment 2: Grouping chromatic- and form-rivalrous representations in the presence of feature differences

Experiment 1 showed that two separate chromatically rivalrous representations with shared form are grouped, meaning they appeared to be the same color more often than chance. Experiment 2 tested whether two chromatically rivalrous representations with different forms are also grouped, thus appearing to be the same color more often than chance. If grouping of color requires a common form, then both areas should appear the same color at the level of chance (that is, no grouping). On the other hand, if grouping of two areas for color is not dependent on common form, then grouping should occur despite their different spatial properties. 

Stimuli here were gratings or discs that always rivaled in chromaticity. Experiment 1 laid the groundwork by showing grouping with shared form at each of three spatial frequencies. Experiment 2 used mismatched stimuli such that the stimulus above fixation was not identical in form to the stimulus below fixation (Figure 7, top row; conditions with identical forms above and below fixation were also tested for comparison). All stimuli were presented using only a patchwork display—that is, the chromaticities of the shapes above and below fixation within each eye were different at any given moment (Figure 7, top row). Measured percepts were identical in color above and below fixation, sometimes having different forms (Figure 7, bottom row) and sometimes, in other conditions, having the same form (not shown in the figure). As in previous experiments, measurements were made also with stimuli only above or below fixation (not shown in the figure), in order to derive the independence prediction for each condition. 

Figure 7

 

Stimuli (top row) and measured percepts (bottom row) in Experiment 2. Rivalrous stimuli for all conditions (top row) had chromatic rivalry and sometimes also orientation rivalry. Condition A: 3.3-cpd grating on top, full disc on bottom. Condition B: 3.3-cpd grating on top and 5.0-cpd grating on bottom. Condition C: 3.3-cpd grating on top and 5.0-cpd grating on bottom with rivalry in both chromaticity and orientation. The figure shows one possible set of stimuli; all mismatched combinations were tested with chromaticity and chromaticity/orientation pairings counterbalanced across trials.

Specific procedure and observers

Observers were instructed to fixate on the central cross and hold down buttons on a game pad to indicate when they perceived either the chromatic regions within a single disc (as in Figure 1b) or both discs to be a particular color or when they perceived a plaid (single disc alone) or both plaid (both discs). Separate buttons were used for each percept, and, as before, observers were instructed not to press any button if they perceived mixed or patchy percepts or rapid flicker. Three observers participated in the study. Repeated measurements were made over 12 days. 

Measurements

For all conditions, the proportions of time out of 60 s that an observer perceived both discs as red or both as green were added together. In conditions with rivalrous orientation, the proportion of time observers perceived both discs as plaid was also measured (analyzed separately). These proportions were arcsine transformed prior to statistical analyses. For both the summed red and green results and the plaid results, planned orthogonal contrasts compared each condition to its independence prediction. 

Two of the three observers showed grouping in every condition that paired gratings of one spatial frequency with a different form (observers SH and SK; Figure 8, top and middle rows). Observer KO (Figure 8, bottom row) showed significant grouping for only the condition with both at 3.3 cpd. 

Figure 8

 

Results from Experiment 2 for conditions without orientation rivalry. Each panel shows results from a separate observer. Proportion of time (vertical axis) that an observer perceived both stimuli to be the same color. More saturated bars are the two-disc measurements and paler bars are the independence predictions. Results labeled “same” (horizontal axis) indicate conditions with the same form above and below fixation; results labeled “mix” (horizontal axis) indicate conditions with different forms above and below fixation. Error bars show +1 standard error of the mean. ns = p > 0.05, *p < 0.05, **p < 0.01, ***p < 0.001.

Additional nonorthogonal Bonferroni-corrected contrasts were performed separately for each observer to test for a significant influence of shared form on grouping. One contrast compared the average of the conditions presenting the same form both above and below fixation (labeled “same” in Figure 8) to the average of conditions presenting different forms above and below fixation (“mix” in Figure 8). Observers SH and KO showed significantly higher proportions of time perceiving the same color above and below fixation in the same compared to the mix conditions (p < 0.05), but observer SK did not. 

Overall, the mix results indicate that common spatial frequency is not a necessary prerequisite for grouping two objects of common color, although a common spatial pattern may influence the proportion of time both discs appear the same color. 

Conditions with rivalrous chromaticity and orientation (Figure 7, condition C) also showed significant grouping for two of three observers (SH and SK). Their two-disc measurements were significantly larger than independence predictions (p < 0.001 for the mixed-spatial-frequency condition for percepts of both oriented gratings—sum of both red and both green; Figure 9, top—as well as plaids; Figure 9, bottom). No observer showed a significant difference between conditions with the same patterns on top and bottom (Figure 9, “same”) compared to different patterns (Figure 9, “mix”), for percepts of oriented gratings or plaids. This shows that the grouping process can act on representations with mismatched spatial frequencies for percepts of gratings and also for plaids. 

Figure 9

 

Results from Experiment 2 (conditions with chromatic and orientation rivalry). Proportion of time (vertical axis) that observers perceived both stimuli as the same color and orientation (top graph) or plaid (bottom graph). More saturated bars are the two-disc measurements and paler bars are the independence predictions. The horizontal axis represents each observer as well as the stimulus condition. Top: Sum of red and green percept proportions measured for three observers. Bottom: Measurements of plaid percepts for three observers. Error bars show +1 standard error of the mean of red + green (top) or plaid (bottom). ns = p > 0.05, *p < 0.05, **p < 0.01, ***p < 0.001.

Experiment 3: Grouping representations of form, including plaids, in the presence of chromatic differences

Experiments 1 and 2 showed that observers can experience plaid percepts from dichoptically presented orthogonal gratings, and that grouping is found with plaids. Experiment 1B showed that two identical plaids appear above and below fixation simultaneously more often than chance, and Experiment 2 found that this occurs even when the two plaids are composed of different spatial frequencies. Experiment 1B showed also that observers perceive single-colored plaids more often than two-colored plaids, perhaps due to the larger chromatic contrast between the two rivalrous stimuli in the two-colored-plaid case leading to more single-grating percepts. Experiment 3 tested whether orthogonal, dichoptically presented gratings that give the percept of plaid of different colors above and below fixation can be grouped; that is, does the simultaneous percept of a plaid both above and below fixation occur more often than chance? Conditions include a two-colored plaid seen above fixation and a single-colored plaid below, and also two single-colored plaids of different colors above and below fixation. If simultaneous plaids occur more often than chance, this would indicate that grouping occurs for perception of plaids despite chromatic differences between them. 

Experiment 2 showed also that representations of two different forms could be grouped to appear to be the same color more often than chance. Experiment 3 also tested the opposite question: Can two representations of gratings with different chromaticities be grouped so they appear to be of the same form more often than chance? If so, this would indicate that the grouping mechanism examined in Experiments 1 and 2 can act on representations without shared chromaticity as well as without shared form. To test this hypothesis, the percepts measured above and below fixation here could always match in form but not necessarily chromaticity. 

Stimuli here were 3.3-cpd gratings that always rivaled in orientation. Only patchwork stimulus presentation was used. Measured percepts could be either identical in color and form (Figure 10, conditions A and B) or mismatched in color but still identical in form (Figure 10, conditions C and D). As in previous experiments, measurements were also made with the top disc alone and bottom disc alone (not shown in the figure) to derive the independence prediction for each condition. 

Figure 10

 

Stimuli (top row) and measured percepts (bottom row) for Experiment 3. Rivalrous gratings for all conditions (top row) had orientation rivalry and some also had chromatic rivalry. The first two conditions (A and B, left two columns) measured identical percepts above and below fixation with stimuli having identical rivalry above and below fixation. The other two conditions (C and D, right two columns) measured percepts from stimuli that differed in chromaticity above and below fixation. The figure shows one possible set of stimuli, but chromaticity and chromaticity/orientation pairings were counterbalanced across trials.

Specific procedure and observers

Observers were instructed to fixate on the central cross and hold down buttons on a game pad to indicate when they perceived either the single disc (as in Figure 1b) or both discs to be of a particular form (45°, 135°, or plaid; Figure 10, bottom row). Separate buttons were used for each percept. Observers were instructed not to press any button if they perceived mixed or patchy percepts or rapid flicker. 

Six observers participated in the study. All observers completed three sessions on separate days, with repetitions split up in sets of four per day. 

Measurements

The proportions of time out of 60 s that observers perceived both discs as 45° or 135° gratings were added together and then analyzed separately from the proportion of time observers perceived both discs as plaid. These proportions were arcsine transformed prior to statistical analyses. For both plaid and individual-orientation results, planned orthogonal contrasts compared each condition to its independence prediction. Additionally, two nonorthogonal, Bonferroni-corrected contrasts were conducted. One compared the condition with both discs in chromatic and orientation rivalry (Figure 10, condition A) to the condition with one disc in only orientation rivalry and the other in chromatic and orientation rivalry (Figure 10, condition C). The other compared the condition with all four discs at the same chromaticity (Figure 10, condition B) to the condition with different chromaticities above and below fixation (Figure 10, condition D). 

In general, observers often perceived both gratings to be oriented in the same direction above chance, regardless of their chromaticity. Conditions A and B measured grouping when neural representations above and below fixation were from identical stimuli. Five of six observers showed significant grouping for gratings with chromatic rivalry (Figure 11a, condition A), and four of five observers showed significant grouping for gratings without chromatic rivalry (Figure 11a, condition B; the sixth observer did not reach the 5% floor criterion). The other two conditions (C and D) tested for grouping with chromatic stimulus properties above and below fixation that were not the same (i.e., gratings above and below fixation shared orientation rivalry but had different chromaticities). Four of six observers showed significant grouping with rivalrous chromaticities in only one of the positions (Figure 11a, condition C), and five of six showed significant grouping for gratings with identical rivalrous orientations but nonrivalrous (though different) chromaticities (Figure 11a, condition D). 

Figure 11

 

Percepts from Experiment 3. (a) Results for oriented grating percepts showing the proportion of time (vertical axis) that the two gratings above and below fixation appeared to tilt in the same direction. Darker bars are the two-disc measurements and lighter bars are the independence predictions. The darker of the two stacked bars represents 135°, and the lighter 45°. The letters along the horizontal axis indicate the condition as labeled in the legend (and Figure 10). (b) Results for plaid percepts show the proportion of time (vertical axis) that a plaid was perceived both above and below fixation. More saturated yellows represent the two-disc measurements and paler yellows the independence predictions. The letters along the horizontal axis indicate the condition as shown in the legend. Error bars show +1 standard error of the mean for 135° + 45° or plaid. ns = p > 0.05, *p < 0.05, **p < 0.01, ***p < 0.001. Bars without asterisks did not reach the 5% floor criterion (see Methods section).

Only two of the six observers showed a significant difference in the direction of a higher proportion of time the two discs appeared tilted in the same orientation in the condition with both chromatic and orientation rivalry in both discs (Figure 11a, condition A > condition C). The contrast comparing the condition with all gratings at one chromaticity to the condition with gratings of different chromaticities above and below fixation (conditions B and D) did not yield a significant difference for any of the five observers who reached the 5% floor criterion for both plaids (p > 0.1). While there may be second-order differences among conditions A through D, the main point is that neural representations with one nonshared feature, here chromaticity, are still grouped perceptually. 

Observers generally perceived plaids together above and below fixation more often than chance. Two of three observers showed significant grouping of two-colored plaids (Figure 11b, condition A; the other three did not reach the 5% floor criterion), and four of six showed significant grouping for identical single-colored plaids (Figure 11b, condition B). The other two conditions measured grouping of different plaids. Three of five observers showed significant grouping of two-colored with single-colored plaids (Figure 11b, condition C; the remaining observer did not reach the 5% floor criterion, and one observer showed a significant difference in the opposite direction). Four of five observers showed significant grouping of single-colored plaids that had different chromaticities (Figure 11b, condition D; the remaining observer did not reach the 5% criterion). 

The nonorthogonal contrast comparing the condition with both two-colored plaids to the condition with one two-colored plaid and one single-colored plaid (A vs. C) was not significant for two of the three observers who reached the 5% criterion for both conditions (the remaining observer had a significant result, perceiving significantly more grouped plaids in the mixed condition C; Figure 11b). The nonorthogonal contrast comparing the two conditions with single-colored plaids (B vs. D) failed to reach significance for any of the five observers who reached the 5% criterion for both plaids (p > 0.05). While we remain mindful not to accept the null hypothesis, this is consistent with unequal but nonrivalrous chromaticities not reducing grouping of the two plaids. Overall, these results indicate that perceived plaids can group despite differences in chromaticity. 

Discussion

Grouping links two or more objects, resulting in both having the same appearance in one or more feature dimensions. In this article, grouping was inferred if the percepts of two spatially separated regions matched in color, form, or both more often than predicted by independence for the two regions. The stimulus features in each region could either match in all features above and below fixation (Experiment 1) or not (Experiments 2 and 3). By manipulating the experimental stimuli, we addressed the following aims. 

Experimental aims

The overarching aim was to determine how a grouping process acts on representations of equiluminant, chromatically defined gratings, in order to better understand how complete objects are perceived. Experiment 1 was a baseline study that determined whether grouping occurred for chromatic gratings matched in spatial frequency and orientation, as well as whether spatial frequency up to 5.0 cpd alters perceptual resolution. Experiments 2 and 3 used the results from Experiment 1 to test whether a grouping process can act on neural representations that differ in one feature dimension. Specifically, Experiment 2 tested whether grouping can act on neural representations of ambiguous chromaticity despite differences in form (specifically the spatial frequency of the gratings); Experiment 3 tested whether it can act on neural representations of ambiguous form (specifically orientation or plaid) despite differences in chromaticity. 

Basic findings

Experiment 1 showed that neural representations from two rivalrous gratings that matched in spatial frequency were grouped, meaning that the two perceived gratings (above and below fixation) appeared to be the same color significantly more often than chance. This was true for sets of gratings that rivaled in chromaticity alone or in chromaticity and also orientation. Further, with the addition of rivalrous orientation, there was evidence for grouping of the superimposed combination of the two gratings that perceptually formed a plaid. The results did not reveal any consistent effect of spatial frequency on the resolution of ambiguity for equiluminant gratings during interocular-switch rivalry (for contrary evidence for luminance-defined gratings, see Lee & Blake, 1999; Denison & Silver, 2012). While luminance-defined stimuli of different spatial frequencies may preferentially stimulate the magnocellular or the parvocellular pathway (Denison & Silver, 2012), equiluminant stimuli are thought to minimize magnocellular activity, which may explain the differences between the current results and those from previous studies. 

These results were groundwork for Experiments 2 and 3, which presented rivalrous gratings with different, unambiguous features in the stimuli above and below fixation. Grouping was still found for one or more ambiguous features in common. The nonshared features between the two stimuli above and below fixation in Experiments 2 and 3 did not eliminate grouping for the ambiguous feature or features, even though the two rivalrous regions did not appear completely identical. Further, this held whether the perceptual difference was in spatial frequency (Experiment 2) or in color (Experiment 3). 

Specifically, Experiment 2 showed that the spatial frequency need not be identical between two stimuli for their ambiguous color, and sometimes orientation, to be resolved by grouping. This result can be interpreted in concert with the results from Experiment 3, which found the same pattern of results for grouping form despite differences in chromaticity. Experiment 3 showed that the chromaticity need not be identical in two gratings for their ambiguous form (here, a 45° grating, 135° grating, or plaid) above and below fixation to be resolved by grouping. This supports the hypothesis that grouping is a general process that acts when multiple regions with ambiguous neural representations induce binocular competition of the same kind—that is, competing neural representations of chromaticity or of patterns (45°, 135°, and plaid), or both simultaneously. 

Neural processes of grouping

The results indicate that a grouping process links two distinct areas when they share a common neural ambiguity. This process is not stopped by unshared, unambiguous features—that is, it still links together fragments of the visual field based on commonalities so that an observer perceives coherent objects and scenes. The current results show that common ambiguous color with different unambiguous forms (Experiment 2, spatial frequency) and common ambiguous form with different unambiguous colors (Experiment 3) produce grouped percepts. Thus, grouping links fragments based on competing representations even in the presence of nonidentical but noncompeting (i.e., unambiguous) representations. 

Could the results here be explained by independent resolution of ambiguity for form and color? If so, observers could have experienced feature-misbound percepts (independent grouping by form and by color could lead to perceptual resolution of two features that never were presented together in any stimulus). This possibility was examined in a pilot study with two ambiguous features—color and orientation—and no feature misbinding was found (see footnote 1 in Methods). Although this cannot rule out the possibility that misbinding could occur only when two perceived discs are grouped, misbinding was never observed with interocular-switch rivalry for a single perceived object. 

Plaids

Plaid percepts are a combination of both “rivalrous” gratings, resulting from perceptual dominance of a new representation different from the representation of each component grating alone. Other examples of this kind of new whole from rivalrous parts are well known. Dichoptic stimuli with binocular disparity can create a percept of a single (fused) object in depth that alternates in perceived color (Treisman, 1962; Hong & Shevell, 2008). Steady dichoptic gratings with rivalrous orientation and chromaticity can form percepts that alternate between two stable orientations but contain colors from both of the two rivalrous stimuli (Hong & Shevell, 2006, 2009). These percepts include components from both of the two dichoptic stimuli, so they cannot follow from either eye's stimulus representation alone. 

The current study found grouping of plaids, suggesting that a grouping process acts on neural representations following integration of the component parts. Further, these component neural representations need not have the same chromaticity in order for grouping of plaids to occur. In fact, plaid grouping occurs even when the component gratings forming a plaid above and below fixation are not rivalrous in the same way (e.g., only grating orientation rivalry above fixation but chromatic as well as orientation rivalry below fixation). This shows that grouping occurs for competing representations of form even when perceptual resolution includes both the 45° and 135° gratings that seen together form a plaid. Moreover, this is found with either ambiguous or nonambiguous color, indicating that binocularly integrated form (the plaids) is not disrupted by simultaneous perceptual dominance of only one color. 

Perceiving two plaids together, however, might be explained instead by a lack of inhibition for both of the rivalrous stimuli in view. Normally, two gratings presented in binocular rivalry will inhibit one another, thus leading to alternating percepts of one grating and then the other. Suppression of this inhibition could lead to the percept of both gratings being perceived simultaneously (i.e., a plaid). This lack of inhibition may occur over the entire visual field, or a lack of inhibition in one local area of the visual field may drive it in another local area (e.g., two plaids appear together due to the absence of dominance in one region failing to reinforce dominance in another). Either way, this might explain why observers tend to see two plaids together more often than chance without invoking a neural grouping process. This suppression-of-inhibition hypothesis is being addressed in an ongoing study. 

No evidence for monocular influence on perceptual resolution of chromatic ISR

With stimuli having luminance-defined form or equiluminant chromaticities, monocular effects have been found using standard binocular rivalry (Papathomas et al., 2005; Stuit, Paffen, van der Smagt, & Verstraten, 2011). In ISR, luminance-defined form also has been found to produce percepts consistent with dominance of monocularly driven neural representations (Lee & Blake, 1999; Denison & Silver, 2012; Brascamp, Sohn, Lee, & Blake, 2013). Equiluminant chromatic stimuli presented in ISR, however, give percepts accounted for by competition between binocularly driven neurons and minimize or eliminate the monocular dominance found with SBR (Slezak & Shevell, 2018). The current study did not find evidence for monocular cues influencing grouping during chromatic ISR, as there never was a significant difference between conventional and patchwork rivalrous stimuli (Experiment 1). The results here, therefore, are in accord with the absence of a monocular influence on chromatic ISR, including grouping from ambiguous form defined by equiluminant gratings. 

Acknowledgments

Support was provided by National Institutes of Health Grant RO1 EY-026618 to SKS. 

Commercial relationships: none. 

Corresponding author: Emily Slezak. 

Email: [email protected]. 

Address: University of Chicago, Department of Psychology, Chicago, IL, USA. 

Footnotes

¹  Prior to running the experiments, a pilot study tested whether feature misbinding occurred while viewing these equiluminant chromatic gratings in ISR (cf. Hong & Shevell, 2009). All observers always perceived rivalrous gratings with correctly bound colors and orientations (as presented in the stimulus). For instance, when presented with a rivalrous green/gray 45° grating in one eye and a red/gray 135° grating in the other eye, observers never perceived the 45° grating to be red or the 135° grating to be green. Thus, only correctly bound color/orientation percepts were measured in the experiments.