Binocularly-driven competing neural responses and the perceptual resolution of color

Abstract

Competing rivalrous neural representations can be resolved at several levels of the visual system. Sustained percepts during interocular-switch rivalry (ISR), in which rivalrous left- and right-eye stimuli swap between eyes several times a second, often are attributed to competing binocularly driven neural representations of each rivalrous stimulus. An alternative view posits monocular neural competition together with a switch in eye dominance at the moment of each stimulus swap between eyes. Here, a range of experimental conditions was tested that would change the colors seen if mediated by eye dominance but not if by competition between binocularly driven responses. Observers viewed multiple chromatically rivalrous discs in various temporal and spatial patterns, and reported when all discs in view appeared the same color. Unlike typical ISR paradigms that swap the complete stimulus in each eye, some of the rivalrous discs were swapped at a different time, or faster frequency, than other discs. Monocular dominance of one eye at a time implies that all discs will rarely be seen as identical in color when some discs swap at a different frequency than others. On the other hand, competing binocularly driven neural responses are not affected by asynchronous swap timing among the individual discs. Results for every observer are in accord with competing responses at the level of binocularly driven, chromatically tuned neurons. Although an account based on eye dominance can be constructed using many small retinotopic zones that have independent timing for the moment of switching the dominant eye, competing binocularly driven responses are a more parsimonious explanation.

Introduction

Visual awareness is the result of neural processes in the eyes and brain. At any given moment, the human visual system has numerous neural representations that ultimately are compiled into percepts of coherent objects and scenes. These neural representations, however, often are ambiguous because many possible objects in the physical world can give rise to identical representations (Von Helmholtz, 1867; Brascamp & Shevell, 2021). The ambiguity is resolved by the visual system in the process of generating conscious percepts of the external world. 

Resolution of neural ambiguity normally is effortless and unnoticed so viewers perceive only the outcome of the perceptual resolution process. In the laboratory, perceptual resolution of neural ambiguity can be studied using dichoptic stimuli that are presented to the same retinotopic location in each eye. Dichoptic presentation of dissimilar stimuli leads to competition between the conflicting signals from the two eyes’ images. Consider, for instance, a single disc that appears red presented to only the left eye and a disc that appears green to only the right eye at the same retinotopic location. The discs are too different in chromaticity to fuse so instead they establish chromatic ambiguity because both colors cannot be seen at the same location at the same time (Wolfe, 1983). The visual system must resolve the ambiguity; in this case, the percept may be a red disc or green disc. Typically, the percept alternates over time between red and green if the dichoptic stimuli are presented steadily. Perceptual resolution of the ambiguity created by two competing representations has been studied extensively (Levelt, 1965; Blake, 1989; Kovács, Papathomas, Yang & Fehér, 1996; Logothetis, Leopold, & Sheinberg, 1996; Carlson & He, 2004), but the underlying neural mechanisms remain controversial. 

Neural competition can be resolved at several levels of the visual system (Tong, Meng, & Blake, 2006). For example, a classic theory, eye rivalry, posits competition between signals from the two eyes. This approach assumes that each eye's stimulus is represented by monocularly driven neurons that compete for perceptual dominance, resulting in conscious perception from one eye's stimulus while the other eye's response is suppressed (Blake, 1989; Lee & Blake, 1999; Blake, 2001; Lee & Blake, 2004; Brascamp, Sohn, Lee, & Blake, 2013). In this view, the resolution of a red or green disc is mediated by competition between signals driven by only the left-eye stimulus and signals from only the right-eye stimulus. 

An alternative theory, stimulus rivalry, argues that neural competition is between representations of the two stimuli. These representations can be at the level of binocularly driven neurons (Figure 1) so stimulus rivalry may be resolved by competition between binocularly integrated neural responses with each response selectively tuned to one dichoptic stimulus (Logothetis et al., 1996; Wilson, 2003; Christiansen, D’Antona, & Shevell, 2017; Slezak & Shevell, 2018; Slezak, Coia, & Shevell, 2019). Applying this approach to rivalrous chromaticities implies competition between separate binocularly driven neural representations that evoke either a red or a green percept. Macaque cortical areas V1 and V2 include neurons that have specific chromatic selectivity and respond to the appropriate chromaticity presented to either eye (Peirce, Solomon, Forte, & Lennie, 2008). Note that competing binocular responses do not rely on eye-of-origin information, whereas competing representations resolved by eye rivalry implicitly depend on segregating the signals from each eye. 

Figure 1.

 

(Below) A dichoptic stimulus with rivalrous chromaticities in the two eyes. During chromatic interocular switch rivalry, the two chromaticities swap between the two eyes several times each second (that is, the eyes’ stimuli alternate rapidly between “SWAP-PHASE 1” and “SWAP-PHASE 2”). (Above) Two chromatically selective, binocularly driven neurons respond to different stimulus chromaticities that fall in the same retinotopic area of either eye. The swapping dichoptic stimuli (below) create continuous, competing responses for both chromaticities (solid and dashed lines with arrows) despite the rapid chromaticity swaps between the eyes.

Resolution of neural ambiguity for a rivalrous chromatic disc can be explained by either theory because both predict alternating red and green percepts. Moreover, eye rivalry and stimulus rivalry are not mutually exclusive processes; each one can affect perception under certain circumstances, and hybrid models combine aspects of both theories (Díaz-Caneja, 1928; Ngo, Miller, Liu, & Pettigrew, 2000; Lee & Blake, 2004; Tong, Meng, & Blake, 2006). Understanding how different levels of the visual system resolve neural ambiguity can be advanced by isolating one level of neural competition from another. One approach to distinguishing eye rivalry from stimulus rivalry uses multiple dichoptically presented discs viewed simultaneously. With two or more discs in each eye, the stimuli can be presented in either conventional arrays or patchwork arrays (Kovács et al., 1996; Slezak & Shevell, 2018). In conventional arrays, all discs presented to one eye have the same chromaticity, whereas in patchwork arrays some discs within each eye have one chromaticity and other discs a different chromaticity (Figures 2A,B). Both conventional and patchwork arrays have chromatic rivalry at every retinotopic location so either type of array has neural ambiguity for each of the multiple discs in view. 

Figure 2.

 

(A) Sixteen chromatic discs in conventional arrays. All discs presented to one eye at any given time have the same chromaticity. (B) Patchwork arrays. Discs of two different chromaticities are presented to each eye at any given time, always with chromatic rivalry at each retinotopic location (after Slezak & Shevell, 2018).

Monocular eye-of-origin bias is well known to affect perceptual resolution when multiple rivalrous stimuli are presented to each eye (Kovács et al., 1996; Stuit, Paffen, van der Smagt, & Verstraten, 2011): with conventional arrays (see Figure 2A), an eye-of-origin cue increases the frequency of a coherent color percept, defined as seeing all discs as the same color, compared to patchwork arrays (see Figure 2B) for which dominance of one eye cannot give a coherent color percept. Although both patchwork and conventional arrays sometimes produce coherent color percepts, this difference reveals a contribution from eye dominance, although it does not exclude binocularly driven stimulus rivalry because the two are not mutually exclusive. 

Another approach to isolating one level of neural competition uses interocular-switch rivalry, which is designed to bypass eye rivalry by swapping the dichoptic left-eye and right-eye stimuli several times a second (Logothetis, Leopold, & Sheinberg, 1996; Denison & Silver, 2012; Christiansen, D'Antona, & Shevell, 2017). Despite the stimulus change in each eye many times each second, a sustained percept from one of the dichoptic stimuli often is seen for two seconds or longer. This duration of seeing one percept can be accounted for by a chromatically selective binocular neuron that responds to the same stimulus regardless of which eye views it (see Figure 1). Alternatively, however, this might be explained by left-eye and right-eye dominance that switches in synchrony with the interocular stimulus swaps (Brascamp, Sohn, Lee, & Blake, 2013). 

Perceptual grouping as a measurement tool

When multiple rivalrous chromatic discs are presented to each eye in patchwork arrays (see Figure 2B), the percept is sometimes all discs of the same color. This might (rarely) happen by chance if perceptual resolution of neural competition were determined independently for each disc in view. Empirically, however, 16 discs at different retinotopic positions are seen as the identical color far more often than predicted by independence (Slezak & Shevell, 2018). This reflects perceptual grouping (also called binocular or interocular grouping), which depends on shared traits among the objects, such as the same chromaticities (Kovács et al., 1996) or orientations (Alais & Blake, 1999; Ngo, Miller, Liu, & Pettigrew, 2000). Thus, the visual system links the appearance of multiple objects in view, each with competing rivalrous neural representations and seen simultaneously. 

Perceptual grouping and neural models that can account for it are important topics, but they are not the focus here. Instead, the questions here concern the individual competing neural representations for each object in view. The experiments measure the proportion of viewing time during which all discs appear to be the same color, and this proportion depends on grouping. These measurements, however, are used only as an indicator of the underlying competing neural representations for each disc. Specifically, the measurements serve to test whether varying the temporal properties of dichoptic stimuli alters these competing neural representations. If not, so that the neural competition for each object in view is invariant with respect to changes in the temporal stimulus properties, then the perceptual grouping duration (that is, the time seeing all discs as the same color) should be constant. This approach assesses a strong prediction of binocularly driven stimulus rivalry: varying the temporal interocular swap parameters should not alter the proportion of time that all dichoptic discs appear the same color. The reason is that a binocular neuron, driven by either eye, will continue to respond so long as the stimulus falls in a given retinotopic area of either the left eye or right eye. On the other hand, a contribution from monocular eye dominance would alter the measured duration with changes in swap parameters, in accord with the amount of time when all discs presented to one eye have the same chromaticity. 

Varying interocular swap temporal phases and frequencies

Does the duration of perceiving all discs of the same color vary with different phases and frequencies of temporal interocular swapping? This is tested here in three experiments. An initial experiment uses the minimal stimuli that can distinguish conventional from patchwork arrays of rivalrous discs: two dichoptic discs, presented here with one above fixation and the other below. If neural competition is between binocularly driven representations, then the representations should remain the same for either type of array because a neuron tuned to a chromaticity that appears red will always be excited by a stimulus in one eye or the other. Simultaneously, a binocular neuron tuned to a chromaticity that appears green also will respond continuously, thus setting up neural competition for color at a binocular level for each of the two perceived discs. Note a general principle for these binocular representations: a neural response in one retinotopic area is unaffected by the temporal phase or frequency of swapping in another area. Further, this experiment generalizes beyond conventional arrays (0 degrees temporal phase difference in stimulus swaps above and below fixation, so both discs in each eye are always the same chromaticity) and patchwork arrays (180 degrees phase difference, so the 2 discs in one eye are never the same chromaticity). Intermediate phases at 90 or 270 degrees have a full swap cycle that includes conventional arrays for two quarter-cycles and patchwork arrays for two quarter-cycles (thus both discs in one eye have the same chromaticity for half of the time). Binocularly integrated competing responses should be insensitive to any of these temporal phase differences. 

Additional experiments extend the stimuli presented with ISR from two dichoptic discs to 30, all presented within an aperture above fixation (or as a counterbalanced control within an aperture below fixation). Even with 30 discs in view, observers often see all of them to be the same color; again, the proportion of viewing time when this occurs is the critical measurement. In separate conditions, the 30 discs could swap between eyes either in phase at the same frequency or, instead, with 10 of the discs swapped between eyes at one temporal frequency, 10 other discs at a second frequency, and the last 10 discs at a third frequency. A stimulus with different swap frequencies for subsets of the 30 discs rarely has all discs in one eye at the same chromaticity, but competition between binocularly driven neural responses at each retinotopic location should be insensitive to the unequal swap frequencies, implying that the measurements of viewing time with all discs having the same color should be the same in all the conditions with 30 dichoptic stimuli. 

Methods

Apparatus

Experimental stimuli were presented on a cathode ray tube (CRT) display in an otherwise dark room (Sony GDM-F520 in experiment 1 and NEC MultiSync FP2141SB in experiments 2 and 3). In both cases, the monitor resolution was 1280 × 1024 and the refresh rate was 75 Hz. The CRT was driven by an Apple iMac desktop computer and viewed through a haploscope (Figure 3). A chin rest ensured a viewing distance of 115 cm from each eye to the monitor. The mirrors closest to the chin rest could be adjusted to achieve fusion of the two eyes’ images. Each eye's stimulus contained two Nonius lines oriented at 3 and 6 o'clock (right eye) or 9 and 12 o'clock (left eye) on a white rectangular frame. Observers saw one rectangular frame surrounding fused stimuli with the Nonius lines aligned vertically and horizontally (see “Fused Percept” in Figure 3). 

Figure 3.

 

The haploscope. The black rectangle represents the CRT display (not to scale). The thin solid black lines represent mirrors and the dashed lines show light paths. The solid thick horizontal line represents an opaque panel that blocks stray light reflection. The left half of the screen is seen by only the left eye and the right half by only the right eye. The fused percept at the bottom includes all four Nonius lines.

Observers

All observers gave written informed consent as required by the University of Chicago Institutional Review Board. All had normal color vision as assessed by the Ishihara Plates and Rayleigh matching using a Neitz anomaloscope. Stereoscopic vision was normal as assessed using the Titmus Stereo Test. 

Stimuli

Neural ambiguity was created using chromatic interocular-switch rivalry (CISR), a technique with dichoptic chromatic stimuli swapped between the eyes several times a second (Christiansen et al., 2017; Slezak & Shevell, 2018). The temporal eye-swapping frequency and relative phase of the dichoptic stimuli in view were manipulated to assess predictions from the monocular eye-dominance and the binocularly driven stimulus-rivalry theories. All stimuli were equiluminant to eliminate the need to embed them in 18 Hz flicker (cf. Logothetis et al., 1996; Denison & Silver, 2012). 

Procedure

Prior to the experiments, all observers performed heterochromatic flicker photometry (HFP) to establish equiluminance of the chromatic stimuli for each individual subject. HFP was done on 4 separate days to ensure reliability, including a practice day for which data were not included in the equiluminant determination. Observers were introduced to the CISR task after the last measurement of HFP as CISR relies on equiluminance. Each observer practiced the experiment for 2 days; data from practice sessions were excluded from analysis. 

During the task, each trial was initiated by a button press after the observer achieved binocular fusion. Observers were instructed to press and hold a button on a gamepad for the duration that all discs in view were perceived to be the same color. They were instructed not to press any button if they experienced rapid flickering (in time with the stimulus swaps in a single eye) or if all discs did not appear to be the same color. The discs were presented in CISR for 70 seconds with responses from only the last 60 seconds recorded in order to minimize any effects of stimulus onset (Carter & Cavanagh, 2007; Stanley, Forte, Cavanagh, & Carter, 2011). The dependent variable was the total dominance time, reported as the proportion of time out of the total 60 seconds that all discs were perceived to be the same color. Procedures specific to each experiment are described in the following sections. 

Experiment 1: Two dichoptic discs varied in eye-swap temporal frequency and phase

This experiment tests whether resolution of ambiguity during CISR is altered by varying the temporal swap asynchrony for two spatially separated dichoptic discs in view. These temporal variations of the standard CISR technique fill a gap between conventional and patchwork stimulus arrays (see Figure 2). Altering the phase asynchrony of the two discs creates a challenge for a monocularly driven explanation of grouped percepts resulting from either conventional or patchwork arrays because some phase asynchronies cause monocular stimuli to alternate rapidly between conventional and patchwork arrays every quarter of a swap cycle (that is, more than 10 times per second). 

Binocularly driven stimulus rivalry, on the other hand, is insensitive to the asynchronous phase so the explanation for coherent color percepts remains unchanged: a binocularly driven representation of each chromaticity is constantly active at each retinotopic location because binocularly driven neurons respond equally well to a stimulus in either eye. The competing representations, therefore, remain active regardless of the degree of temporal phase asynchrony between the two discs, leading to a percept of both discs of the same color equally often with either synchronous or any asynchronous swapping. 

Stimuli

Stimuli in this experiment were two vertically aligned 1.5-degree diameter dichoptic discs with one located above the fixation cross and the other below it (Figure 4A). The distance from each disc's center to fixation was 1 degree. The luminance of the discs was 7.85 cd/m² and the background was dark (<0.1 cd/m²). Two different pairs of chromaticities were swapped in separate trials, defined in Macleod and Boynton (1979) color space: an [L/(L+M), S/(L+M)] swap-pair of (1) [0.715, 1.00] and [0.615, 1.00], as shown schematically in Figure 4, or (2) [0.665, 1.8] and [0.665, 0.2]. 

Figure 4.

 

(A) Stimuli used in a condition with two dichoptic discs (180 degrees phase difference shown). (B) Two possible coherent color percepts (same perceived color above and below fixation). (C) Schematic of the CISR paradigm with the top and bottom discs swapping chromaticity (only the left-eye stimuli are shown). The bottom disc could oscillate at 0 degrees, 90 degrees, 180 degrees, or 270 degrees temporal phase difference relative to the top disc. Vertical dashed red lines illustrate the phase difference between the top and bottom discs. Note that at 0 degrees the stimuli were in conventional arrays and at 180 degrees in patchwork arrays. At 90 and 270 degrees phases, discs in each eye were in conventional arrays for half of each swap cycle and patchwork arrays for half of each swap cycle.

The disc below fixation swapped between the two eyes at 0, 90, 180, or 270 degrees temporal phase offset relative to the top disc (see Figure 4C). A phase of 0 degrees (in phase) meant that the top and bottom disc in one eye always had the same chromaticity so both discs swapped at the same moment (that is, conventional arrays). A 180-degree phase difference (out of phase) meant that the top and bottom discs in each eye always had different chromaticities and, again, each disc's chromaticity was swapped at the same moment (patchwork arrays, as shown in Figure 4A). The 90- and 270-degree phase differences eliminated the synchrony of the two disc's chromaticity swaps. A 90-degree phase difference meant that the top disc swapped chromaticities, whereas the bottom disc remained steady for one-quarter of a cycle longer (a cycle is the length of time to present a disc once at each chromaticity). The temporal frequency of the top and bottom discs always was the same at all temporal phases. The two discs in one eye oscillated with the same temporal phase difference as in the other eye but with opposite chromaticity, thus maintaining chromatic rivalry in both retinotopic areas at all times. 

The discs were swapped at one of three temporal frequencies in a given session: 3.13, 3.75, or 4.69 Hz (swap cycle durations of 320, 267, or 213 milliseconds, respectively). 

Observers

Three observers participated in this experiment. Observer 1, author W.W., had prior experience with binocular rivalry. Observers 2 and 3 were naïve as to the purpose of the experiment. 

Procedure

Each session in this experiment consisted of eight randomly ordered trials. One of the two pairs of chromaticities was tested on each trial. Observers were instructed to report when the color of the top and bottom discs was identical (see Figure 4B) by pressing a button on a gamepad (separate buttons for each of the 2 rivalrous color percepts). Observers completed five sessions at each of the three temporal frequencies so there was a total of 15 sessions over 5 days. 

Results

Results from each observer were analyzed separately. An analysis of variance (ANOVA) using an arcsine transformation of proportions to stabilize variance revealed there was never a significant difference among the four different phase conditions, for any of the three temporal swap frequencies (Figure 5; each F(3,16) < 1, p > 0.4). Thus, there was no significant effect of phase asynchrony in nine separate tests (3 observers × 3 temporal swap frequencies). Further, the values plotted in Figure 5 show the measured duration at every phase was virtually the same. This is in accord with the binocular stimulus-rivalry model for resolution of neural competition. 

Figure 5.

 

Average proportion of total dominance time when two discs appeared the same color when swapped with a 0 degree (conventional arrays), 90 degrees, 180 degrees (patchwork arrays), or 270 degrees temporal phase difference. For each observer, results are shown separately for three different swap frequencies: 3.13, 3.75, and 4.69 Hz (horizontal axis). Each bar indicates the proportion of total dominance time averaged over the two pairs of chromaticities tested.

Experiment 2: Thirty dichoptic discs varied in eye-swap temporal frequency

Preliminary conditions

The previous experiment used two dichoptic rivalrous discs and found no significant effect of temporal swap phase or frequency within the range of 3.13 to 4.69 Hz. The next experiment used a similar set of temporal frequencies and expanded the number of dichoptic rivalrous discs to 30. Preliminary conditions addressed two questions: (1) are coherent color percepts (all discs seen as the same color) observed with 30 discs in view simultaneously and, if so, (2) does coherence depend on the particular temporal swapping frequency? 

Stimuli and observers

The dichoptic stimuli were 1.5 degrees diameter circular apertures, each containing 30 of the 0.1-degree diameter discs randomly located within the aperture (schematically shown in Figure 6). The small discs were at 20 cd/m² and the background within the aperture was 5 cd/m². The remaining background area was dark (approximately 0.3 cd/m²). The chromaticities of the small discs were defined in MacLeod-Boynton color space with “red” at [L/(L+M), S/(L+M)] of [0.715, 1.00] and “green” at [0.615, 1.00]; the achromatic background within the aperture was at the chromatic average [0.665, 1.00], which is metameric to equal-energy white (MacLeod & Boynton, 1979). 

Figure 6.

 

Apertures with 30 discs in conventional arrays used in the preliminary conditions. (A) Example stimuli presented above fixation. (B) The two coherent color percepts. The duration of seeing these percepts was measured and converted to a proportion of the 60-second viewing time.

In separate runs, the experiment was conducted at three different temporal frequencies and in two different retinotopic areas. The stimuli in the two areas were (1) a dichoptic stimulus containing 30 discs presented with its center 1.5 degrees above the central fixation cross (see Figure 6A) or (2) a similar dichoptic stimulus below the central fixation cross (not shown). Each of the 30 small discs in each eye was either “red” or “green” and arranged so that where a small disc in one eye was “red” the retinotopically corresponding disc in the other eye was “green.” The exact retinotopic locations of the small discs were randomized at the beginning of each trial but always matched between the two eyes to establish chromatic rivalry. The 30 discs were presented in conventional arrays, which should maximize the duration of perceiving all 30 discs of the same color according to an eye-dominance model. 

Three temporal eye-swap frequencies, 3.8 Hz, 4.4 Hz, and 5.0 Hz, were tested in separate runs (slightly higher frequencies than in the previous experiment due to a change in display apparatus). All 30 small discs were swapped simultaneously and at the same frequency. 

Four observers participated in this experiment (1 woman and 3 men, mean age = 23.5 years), all of whom were naïve as to the purpose of the experiment. 

Procedure

All conditions of the experiment were run four times across 8 total days, such that all conditions were run once in each 2 days in random order. The first run of each condition was considered practice so results from 6 days (3 runs) are reported here. Observers were instructed to press and hold down different buttons on a gamepad when they perceived all 30 small discs of a single color, one button for all discs appearing red and a second button for all discs appearing green (see Figure 6B). 

Results

Results from runs with the stimuli above or below fixation were combined prior to analysis to increase statistical power as there was no systematic difference between measurements above or below fixation (3 of 4 observers had no significant difference with F(1,16) < 1.0, and one observer was significant at p < 0.04, although the actual p value is larger as there is no correction for considering the largest value from 4 F tests). Observers reported seeing coherent color percepts (that is, all 30 discs the same color; Figure 6B) at every swap frequency (Figure 7). A 1-way ANOVA, run separately for each observer and using the arcsine transformation of proportions, tested for a difference across the three temporal eye-swap frequencies. There was never a significant difference (values of F(2,6) < 1.0, p > 0.5), in accord with the above two-disc experiment as well as previous results using a larger central chromatic disc (Christiansen et al., 2017). Moreover, the values shown in Figure 7 are very similar at all frequencies for each observer. These measurements from preliminary conditions were groundwork for the following main experiment in which subsets of the 30 discs were eye-swapped at unequal frequencies. 

Figure 7.

 

Proportion of time with all 30 discs appearing red or all 30 appearing green during 60 seconds of viewing time (vertical axis), for each of four observers. For each observer, the three bars starting from the left are measurements from the preliminary conditions run with conventional arrays at different eye-swap temporal frequencies (3.8, 4.4, or 5.0 Hz; horizontal axis). The rightmost bar for each observer shows measurements from the multiple-frequency-swap “main experiment” (labeled “Mixed” on horizontal axis). The dashed horizontal line shows the chance level assuming independence (see text).

Independence prediction

Theoretically, the amount of time seeing coherent color percepts with 30 discs could be due to chance. The measured durations, however, were always far longer than chance, which reveals grouping the discs in the process of perceptually resolving their 30 competing neural responses (Shevell, 2019). 

The proportion of time expected by chance is determined by assuming that each of the 30 discs in view is independently resolved in color. Let the probability of seeing color #1 (for example, red) for an individual disk be p, in which case the probability of seeing color #2 (green) is at most 1-p (in practice, the value will be less than 1-p because sometimes no unambiguous color is seen for a portion of the viewing time). Then, the predicted proportion of time to see all 30 discs with the same color is at most [p³⁰ + (1-p)³⁰]. With p = 0.5, this chance proportion is less than 0.001 (less than 0.1 second seeing all 30 discs as the same color in 60 seconds of viewing, or less than 0.2% of the viewing time). Even with bias for one color of 90% (that is, p = 0.9), the chance proportion is less than 0.05 (less than 3 seconds seeing all 30 discs as the same color, or less than 5% of the viewing time). For a conservative assessment, the measurements are compared to a chance proportion of 5% (horizontal dashed line, Figure 7). Every one of the 12 measurements (3 temporal frequencies × 4 observers) was at least six times greater than the chance proportion of time. 

Main experiment

The initial experiment revealed no significant effect of temporal phase differences using two discs in well separated retinotopic regions. The preliminary conditions immediately above demonstrated perceptual color coherence for 30 discs within a single 1.5 degrees diameter aperture and also corroborated no significant differences among the temporal swap frequencies tested. To consider whether the competing representations established by chromatic interocular-switch rivalry are at the level of monocularly driven or binocularly driven neurons, the next experiment introduced a multifrequency condition in which all three frequencies tested in the preliminary runs were used simultaneously with 10 of the 30 small discs at each swap frequency (i.e. 10 discs swapped at 3.8 Hz, 10 at 4.4 Hz, and 10 at 5.0 Hz). In this case, the 30 discs presented to each eye were at two different chromaticities for most of the viewing time, as shown schematically in Figure 8. 

Figure 8.

 

A schematic multiple-frequency dichoptic stimulus presented above the fixation point (+). Discs presented to one eye usually were a mix of two different chromaticities. The opposite eye always had the other chromaticity at each of the 30 locations to maintain continuous chromatic rivalry for all of the discs. This experiment was repeated with the 30 discs presented within a similar aperture below fixation.

If perceptual resolution relies on monocular eye dominance, then an observer's percept should reflect stimuli in the then-dominant eye and therefore include discs of two different colors most of the time, which would reduce coherent viewing time compared to the preliminary conditions that always used conventional arrays containing discs swapped at a single frequency. On the other hand, neural competition at the level of binocularly driven responses should not be affected by the three different swap frequencies for separate subsets of discs, implying that all 30 discs should appear as the identical color equally often with the conventional, single-frequency arrays (the preliminary conditions) and the new multiple-frequency condition. The reason, again, is that the competing representations at each disc's retinotopic location would have a continuous response for both chromaticities simultaneously, despite swapping the chromaticities of many of the 30 discs at different temporal frequencies. Thus, regardless of the temporal swapping features, an identical proportion of measured time seeing all discs at the same color is expected from binocularly driven competing representations for each small disc. 

Stimuli and observers

The multiple-frequency condition had 10 discs at each of the three different temporal swap frequencies. For each trial, the 30 discs were randomly divided into three sets of 10 discs with each set at a different temporal frequency (i.e. one set of 10 swapping at 3.8 Hz, another set at 4.4 Hz, and the last set at 5.0 Hz). Other stimulus features were unchanged from the preliminary conditions. 

The four observers from the preliminary conditions also participated in this experiment. 

Procedure

The experimental procedures were the same as for the preliminary conditions except as noted above. 

Results

As in the preliminary conditions, for all observers the proportion of time the 30 multiple-frequency discs appeared the same color (rightmost bar labeled “MIXED” for each observer in Figure 7) was far longer than chance, even assuming conservative bias of p = 0.9 (dashed horizontal line, Figure 7). Thus, every observer saw all 30 discs as the same color well above chance. 

Further, a planned contrast compared the measured proportion of time that all discs appeared the same color in this experiment (with subsets of discs swapped at three different temporal frequencies) to the proportions with all discs at a single temporal-swap frequency and presented with conventional arrays (the preliminary conditions). There never was a significant difference between the multiple-frequency condition and the three single-frequency conditions for any observer (t(8) absolute values always less than 1.26, p > 0.24 for each observer). This is as expected for a binocularly driven stimulus-rivalry mechanism. Because the binocularly driven mechanism predicts no difference between the multiple-frequency and single-frequency conditions, and therefore a statistically nonsignificant difference (as was found), the results were used to calculate also an estimate of Hayes omega-squared (Kirk, 2013, p.134), a measure of strength-of-association that gives an estimate of the proportion of variance accounted for by the various temporal-frequency conditions. The average omega-squared value for the four observers was 0.014 (1.4% of the variance), a small value that actually may be inflated because the separate analyses for each observer remove between-observer variation from the total variance. 

Experiment 3: Sixty dichoptic discs: 30 discs within each of two separate retinotopic regions

The next experiment extended the last one from 30 to 60 discs, with 30 in each of two distinct apertures located in separate parts of the visual field. As before, some of the 30 discs within each aperture could swap between eyes at a different temporal frequency than other discs. One aperture of 30 discs was presented above fixation and, simultaneously, the other one below fixation. The question in this experiment is whether resolution of the discs’ color within one aperture depends on the discs’ color seen in the other aperture and, if so, whether monocular eye-dominance or binocularly driven stimulus rivalry can account for that result. 

As in the previous experiment, there were four different temporal swap conditions: 60 discs all swapped at 3.8 Hz, all at 4.4 Hz, all at 5.0 Hz, or these three temporal frequencies used simultaneously with 10 of the 30 discs within an aperture swapped at each frequency. In the single-frequency conditions, the 30 discs within each aperture were in conventional arrays; however, discs within the top aperture always were presented in patchwork arrays with respect to the discs within the bottom aperture (i.e. the 30 top discs in each eye always had a different chromaticity than the 30 bottom discs in the same eye; Figure 9A, top row). 

Figure 9.

 

(A) Schematic of a stimulus configuration with 60 discs presented at a single temporal swap frequency and with patchwork arrays with respect to the discs above and below fixation (above), and the same configuration but with multiple temporal swap frequencies used simultaneously (below; one example of a rapidly changing stimulus is shown). (B) The two coherent percepts (all 60 discs perceived to have the same color) for the measurements in this experiment. The locations of the small discs were randomized on every trial so the ones shown are only examples.

If chromatic competition in this experiment is resolved at the level of monocularly driven eye dominance, then seeing all 60 discs of the same color in the single-frequency conditions would require opposite eye-dominance for the two retinotopic zones of the apertures (e.g. left-eye dominance above fixation and simultaneously right-eye dominance below fixation). In the multiple-frequency condition, an eye-dominance account would require that left- or right-eye dominance be determined separately in local retinotopic areas at the spatial scale of each individual disc (diameter 0.1 degrees) and that eye dominance in each of these areas be updated many times every second. 

Binocularly driven neurons, on the other hand, would not be affected by any of the temporal variations in the four conditions. Continuous neural responses from chromatically selective binocularly driven neurons would occur for both of the swapped chromaticities, regardless of each disc's spatial location and temporal-swap timing. Thus, the simple prediction for binocularly driven neurons is that all 60 discs of the same color will be seen equally often in every temporal frequency condition, including the multiple-frequency condition. 

Stimuli

This experiment had two dichoptic apertures, one 1.5 degrees above and one 1.5 degrees below the fixation cross presented simultaneously. Each aperture was as in the last experiment with 30 small discs within it, so 60 fused small discs were perceived in total (see Figure 9B). Each trial used a different set of randomized locations for the small discs within each aperture, and, as before, the stimuli were co-located retinotopically across eyes to maintain constant chromatic rivalry at each disc location. The 60 discs were presented in interocular-switch rivalry with each disc swapped at one of the three temporal frequencies. 

Observers

The same four observers as in the last experiment participated in this one. 

Procedure

The experiment was run four times across 8 total days such that all conditions were tested once in each 2-day period in a random order. Observers were instructed to press and hold down different buttons on a gamepad when they perceived all 60 small discs to be a single color — one button for all discs appearing red and another for all discs appearing green (see Figure 9B). The dependent measure was the total proportion of time each button was held down during a 60-second viewing period, as in previous experiments. 

A different independence prediction

The probability that all 60 discs would resolve to the same color independently is negligible: below 0.002 or equivalently less than 0.2% of the 60-second viewing time, even assuming bias for perceiving one color of p = 0.90. Pilot experiments revealed all 60 discs often were seen to have the identical color so the 60 discs were not resolved in color independently (confirming data to follow below), an unsurprising result given the measurements in the last experiment. 

A different independence question here is whether resolution of the color of the discs within the top aperture is independent of resolution in the bottom aperture. Results from the last experiment using the top aperture alone and, separately, the bottom aperture alone provided values to predict the experimental measurements here assuming independent perceptual resolution in the top and bottom apertures. Independence predictions were calculated by multiplying together the proportion of viewing time that the top aperture tested alone had all 30 discs of a given color by the proportion of time that the bottom tested alone had all 30 discs of that same color (Figure 10). This prediction was compared to the measurements for seeing all 60 discs in both apertures as the identical color. If the two-aperture measurements in this experiment are significantly higher than expected assuming independence, then independent resolution of color in the two apertures would be rejected and therefore grouping of color across the two apertures would be supported. This was tested separately for each observer and for every temporal-frequency condition. 

Figure 10.

 

Independence prediction for seeing all 60 discs as red: the measured probability of seeing all 30 discs above fixation as red multiplied by the measured probability of seeing all 30 discs below fixation as red. A similar independence prediction was determined for seeing all 60 discs as green.

Results

Planned orthogonal contrasts tested for significant grouping by comparing each two-aperture measurement to its independence prediction (see Figure 10). The contrasts were significant for every swap-frequency condition for each of the four observers (t(16) > 1.75 so p < 0.05 for every one of the 16 tests; see asterisks in Figure 11), implying that observers perceived all 60 discs as red or all as green for significantly longer durations than predicted by independent resolution of color within the apertures above and below fixation. The 60 discs above and below fixation, therefore, were grouped to appear the same color more often than expected from the grouping determined within each aperture alone. 

Figure 11.

 

Proportion of time all 60 discs appeared the same color, for each of four observers. For each temporal frequency condition (horizontal axis), the left bar in each adjacent pair represents the two-aperture measurement and the right bar, shown in paler colors, the independence prediction (see text). The measurement was always significantly larger than predicted by independence. A separate contrast for each observer compared the multiple-frequency (“Mixed”) measurement to the single-frequency measurements; there was never a significant difference for any observer. *p < 0.05, **p < 0.01, ***p < 0.001.

Another contrast compared the two-aperture multiple-frequency-condition measurement to the average of the three two-aperture single-frequency measurements, separately for each observer. None of the observers was close to a significant difference between the proportion of time all 60 discs appeared the same color in the multiple-frequency condition compared to the single-frequency conditions (|t(16)| always less than 1.75 for each observer, ranging from p > 0.16 to p > 0.80 among the 4 observers; see Figure 11). For two of the observers, the multiple-frequency measurement was larger (although nonsignificantly) than for the single-frequency conditions, which is in the opposite direction from the monocular eye-dominance prediction. As in the previous experiment, the tests of statistical significance were complemented by calculating the average strength of association, which was small (Hayes omega-squared of 0.015 or 1.5% of the variance). Overall, the measurements are consistent with competition between binocularly driven neural representations for each rivalrous chromaticity. 

Overall, the findings from all three experiments are in accord with perceptual resolution of chromatic competition between binocularly driven stimulus-rivalry responses (see Figure 1). 

Discussion

The aim of this study was to assess the level of neural competition at which chromatic ambiguity is resolved during interocular-switch rivalry. This has remained an open question (Brascamp, Sohn, Lee, & Blake, 2013) despite the “widespread belief that ... stimulus rivalry is governed by eye-independent neural processes at a purely binocular stage of cortical processing (p. 1).” A similar point was raised by van Boxtel, Knapen, Erkelens, and van Ee (2008), who noted that stimulus rivalry may involve competition between “not generally acknowledged (p. 2)” monocular responses. 

The measurements here determined the amount of time when all discs in view appeared the same color, and how this was affected by a variety of changes in temporal stimulus properties. The range of temporal properties was chosen (i) to alter the amount of time with all discs seen as the identical color if perception is determined by monocular dominance of one eye at a time but (ii) not to affect this amount of time if perception is mediated by competition between binocularly driven chromatically selective neural responses (see Figure 1). The first experiment used two uniform chromatic discs presented with interocular-switch rivalry at the same eye-swap frequency but with different temporal phases (see Figure 4C). The second experiment increased to 30 the number of chromatic discs viewed simultaneously in swap rivalry, with all discs in one eye always identical in chromaticity at each moment (that is, presented in conventional arrays). It showed that all 30 discs often appeared the same color using any of three different interocular swap frequencies (3.8, 4.4, or 5.0 Hz), an important preliminary result for the main condition with a similar set of 30 discs but with 10 of the discs swapped at 3.8 Hz, 10 other discs at 4.4 Hz and the last 10 discs at 5.0 Hz (the “multiple-frequency” condition). Although the 30 discs in each eye rarely had the same chromaticity in this multifrequency condition, these temporal stimulus-swap variations did not significantly affect the amount of time that all discs in view appeared the same color. This held for every observer analyzed separately. Given that the temporal variations were designed to alter perception driven by monocular eye dominance, but not by competition between binocularly driven neural responses, the results favored perception mediated by competing binocular neural representations (see Figure 1). 

A final experiment extended the results to two sets of 30 discs seen simultaneously, one set within a retinotopic area above fixation and one within an area below fixation (so 60 discs seen in all). The critical comparison was between conditions with a single interocular-swap frequency and phase for all 60 discs versus a multiple-frequency condition with 20 discs swapped at 3.8 Hz, 20 discs at 4.4 Hz, and 20 discs at 5.0 Hz. Again, there was never a significant difference between the single-frequency and the multiple-frequency conditions, as expected for competing chromatically selective binocularly driven neural responses of the kind found physiologically by Peirce et al. (2008). Such neurons constantly receive excitation regardless of the eye receiving the chromatic stimulus, thus generating continuous neural representations of both chromaticities simultaneously for the entire presentation duration. This implies coherent color percepts should be unaffected by localized stimulus differences in temporal swap frequency or phase. The experiment showed also that not only was the color of each small disc within a single aperture resolved non-independently but also the two sets of discs in separate apertures, one set above and the other below fixation, also were not resolved independently of each other. 

Eye dominance determined separately in localized retinotopic zones?

The first experiment never found a significant effect of varying temporal phase (see Figure 4), at any phase for any observer. Moreover, this held repeatedly at each of the three temporal-swap frequencies tested. This result is as predicted for competition between constantly active, binocularly driven representations of each rivalrous chromaticity (see Figure 1) at both retinotopic locations, as discussed above. 

This finding, however, does not completely exclude possible competition between monocularly driven signals if eye dominance can switch from one eye to the other at different times in each separate retinotopic zone (in this case, one zone above fixation and the other zone below it). This possibility, rooted in observations by Díaz-Caneja et al. 1928 (translated by Alais, O'Shea, Mesana-Alais, & Wilson, 2000), could compensate for the temporal phase difference between the top and bottom zones by allowing independent timing of the eye-dominance switches in each zone. 

The visual system can form coherent percepts that integrate signals from separate parts of the visual field with each part dominated by a different eye (Díaz-Caneja, 1928; Blake, O'Shea, & Mueller, 1992; Kovács et al., 1996; Ngo et al., 2000; Lee & Blake, 2004; see Figure 12). A coherent percept in the first experiment, composed of both discs seen as the same color, could be based on localized dominant neural representations, each one from a single eye, even though the full percept includes elements that do not originate from one eye alone (that is, “patches, or zones, of dominance collated within and between the two eyes (Lee & Blake, 2004, p. 990)”). For the CISR stimuli in the first experiment, such an eye-dominance model could lead to seeing both discs of the same color if further assumptions are included: (i) eye dominance for each retinotopic zone can switch rapidly (about 7 times a second) and (ii) eye-dominance switches within each zone are in synchrony with the swap of stimuli between eyes in that zone. The second assumption implies independent timing of eye-dominance switches for each of the two zones. A similar model could account also for seeing all 60 discs of the same color when presented using CISR at a single temporal frequency, as in the top panel of Figure 9A. 

Figure 12.

 

Dichoptic left- and right-eye stimuli (from Blake & Logothetis, 2002; after Díaz-Caneja, 1928; translated by Alais, O'Shea, Mesana-Alais, & Wilson, 2000). The fused percept is often a full circle filled completely with either red stripes or a green bullseye. Reprinted by permission from Springer Nature.

Theoretically, an extended version of this model could predict that measurements from the multifrequency conditions (see Figure 8 and lower panel of Figure 9A) would be no different than results from the single-frequency conditions if there were as many localized retinotopic zones as discs in view, so 60 separate retinotopic zones in the final experiment within 2.25 degrees of the center of the fovea. The multifrequency experiments here with 30 or 60 discs of diameter 0.1 degrees cannot exclude this possibility, although larger independent zones have been suggested this close to the fovea (cf. Lee & Blake 2004, although see also Blake, O'Shea, & Mueller, 1992). 

Individual differences

The experiments show that some observers had a consistent tendency to see one of the rivalrous colors more often than the other though there was not uniform bias toward one color overall. Instead, among those tested, there was one observer who predominantly saw green, one who predominantly saw red, and others who were closer to an equal balance (see Figures 7, 11). The intent here, of course, was not to study individual differences, although some were noted in the first report of chromatic interocular-switch rivalry that used a single dichoptic stimulus (see discussion in Christian, D'Antona, & Shevell, 2017). 

The experimental design here implicitly controls for individual differences by analyzing results from each observer separately. Moreover, consistent conclusions from each observer, despite the individual differences, support the robustness of the experiments’ implications. 

Color as a model system, but how general are the conclusions?

The experiments here used competing chromatic neural representations to study the perceptual resolution of color. Color vision is a valuable model system to study neural representations that correspond to conscious visual experience, as opposed to representations of light stimulating the retina (Kim, Hong, Shevell, & Shim, 2020). Note, however, that isolating neural representations of perceived color depends on experimental procedures that minimize differences in the luminance of the stimuli, so an open question here, and for any model system of vision, is how well the experimental results generalize to broader properties of the visual system. 

The main conclusion here is the robust success of binocularly driven competing neural responses to account for all of the broad-ranging results. As discussed earlier, mechanisms at many levels of the visual system may resolve neural competition, and, in order to study a mechanism at one level, an experimental paradigm may aim to isolate it from mechanisms at other levels. In chromatic interocular switch rivalry, all swapped stimuli are made equal in luminance for each observer using HFP. This effectively eliminates luminance changes that otherwise would accompany the interocular stimulus swaps in chromaticity; without HFP, swaps in chromaticity would be confounded with swaps in luminance level. 

Most studies of interocular-switch rivalry, however, swap stimuli that are unequal in luminance at many retinotopic locations (Logothetis, Leopold, & Sheinberg, 1996; Stuit, Paffen, Smagt, & Verstraten, 2011; Brascamp, Sohn, Lee, & Blake, 2013) and, moreover, may be chosen specifically to avoid color (van Boxtel, Knapen, Erkelens, & van He, 2008). The mechanisms that affect percepts resulting from interocularly swapped dichoptic stimuli of varying luminances (for example, achromatic gratings at rivalrous orientations) are known to have different properties than mechanisms driven by swapped equiluminant stimuli as used here. Competition between binocularly driven mechanisms, therefore, may be more prominent with chromatic equiluminant stimuli than with achromatic patterns presented with interocular-switch rivalry. 

One fundamental distinction with luminance variation in interocular-switch stimuli is the requirement to present the stimuli in a more complex temporal pattern in order to achieve sustained percepts of one of the rivalrous stimuli, in comparison to swapping equiluminant lights. Instead of simply swapping the stimuli interocularly, as done here with chromatic lights, the entire stimulus that has luminance variation may be presented with 19 Hz on/off flicker so there is a blank (“off”) period every 27 msec, with the rivaling dichoptic stimuli presented only during subsequent 27 msec “on” periods; interocular stimulus swapping of rivalrous patterns presented during the “on” phase occurs every 107 msec (that is, at 4.7 Hz). Or, instead of flicker, rivaling stimuli may be presented to the two eyes for 107 msec, followed by a 107 msec blank period, followed by the rivaling stimuli swapped to opposite eyes for 107 msec, followed by another 107 msec blank period, and so on (van Boxtel, Knapen, Erkelens, & van He, 2008). The critical point is that a blank period before swapping the rivaling stimuli has a strong influence on the sustained percepts from achromatic switch-rivalry stimuli. The specifics of temporal stimulus-presentation patterns have been leveraged fruitfully to infer potential mechanisms of rivalry at different levels of the visual system (van Boxtel, Knapen, Erkelens, & van He, 2008). 

On the other hand, chromatic interocular switch rivalry at constant luminance gives robust, sustained rivalrous color percepts without using flicker or blank periods. Equiluminant stimuli without luminance transients presumably have an effect similar to a temporal blank period: they sharply reduce monocular competition between the rivaling stimuli. Equiluminant chromatic stimuli achieve this while exchanging the rivalrous stimuli between eyes. No flicker or blank period is required. 

The temporal variations here were limited to a modest range of stimulus swap frequencies between 3 and 5 Hz. The range was introduced only in the service of creating stimuli with different discs viewed simultaneously that were swapped at modestly different frequencies in the multifrequency (“MIXED”) conditions (see Figures 7, 11). Previous work suggested this frequency range would cause no differences in the viewing time with sustained rivalrous color percepts (Christian, D'Antona, & Shevell, 2017), and this finding was repeatedly corroborated here for coherent color percepts evoked by the single-swap-frequency conditions. Swap frequencies significantly outside of this range, however, can affect additional mechanisms that could disrupt the experimental designs here: on the higher temporal-frequency side, perceived fusion of alternating chromatic stimuli would result in seeing all discs steadily at their (identical) time-average chromaticity (if faster than about 10 Hz, as exploited in HFP); and on the lower frequency side, the changes in perceived color found with standard (steadily presented) rivalrous chromatic stimuli could intrude before the interocular exchange of chromaticities. 

Finally, all experimental measurements here were based on what observers perceived during interocular-switch rivalry. Specifically, observers reported periods of seeing all discs in view of the same color. Neural competition between rivalrous stimuli, however, can also occur without causing discriminable percepts (Zou, He, & Zhang, 2016). Again, resolution of neural competition can occur at many levels of the visual system; the results here may not apply to competing representations that do not result in a visually distinguishable difference. 

Occam's razor and monocular dominance

The multiple-frequency conditions here cannot be explained by separate monocular dominance zones above and below fixation. Instead, an account based on monocular dominance would require a great many monocular-dominance zones that (1) are small (on the order of diameter 0.1 degrees; cf. Blake et al., 1992, for size estimates based on steadily presented orthogonal achromatic gratings), (2) switch eyes rapidly and in synchrony with the local interocular stimulus swaps, and (3) act independently of nearby zones with different temporal swap rates. Note too that the change in dominance between eyes in each zone would have to occur almost instantaneously. At least theoretically, however, a large number of independent retinotopic zones with localized monocular dominance could account for the results, though further assumptions would be needed to predict perception of all discs at the same color. 

On the other hand, resolution of ambiguity generated by chromatically selective, binocularly driven mechanism accords simply and parsimoniously with all of the temporal and spatial variations used here, which never had a significant effect on resolving color ambiguity for up to 60 small discs. Thus, competition between binocularly mediated neural representations of chromaticities provides a straightforward account for all of the experimental results, and corresponds well with physiological measurements of chromatically selective neurons in V1 and V2 (Peirce et al., 2008). 

Acknowledgments

Supported by National Eye Institute grant EY-026618. 

Commercial relationships: none. 

Corresponding author: Steven Shevell. 

Email: [e-mail]. 

Address: Department of Psychology, University of Chicago, 940 E. 57th Street, Chicago, IL 60637, USA.