The effect of spatial structure on binocular contrast perception

Abstract

To obtain a single percept of the world, the visual system must combine inputs from the two eyes. Understanding the principles that govern this binocular combination process has important real-world clinical and technological applications. However, most research examining binocular combination has relied on relatively simple visual stimuli and it is unclear how well the findings apply to real-world scenarios. For example, it is well-known that, when the two eyes view sine wave gratings with differing contrast (dichoptic stimuli), the binocular percept often matches the higher contrast grating. Does this winner-take-all property of binocular contrast combination apply to more naturalistic imagery, which include broadband structure and spatially varying contrast? To better understand binocular combination during naturalistic viewing, we conducted psychophysical experiments characterizing binocular contrast perception for a range of visual stimuli. In two experiments, we measured the binocular contrast perception of dichoptic sine wave gratings and naturalistic stimuli, and asked how the contrast of the surrounding context affected percepts. Binocular contrast percepts were close to winner-take-all across many of the stimuli when the surrounding context was the average contrast of the two eyes. However, we found that changing the surrounding context modulated the binocular percept of some patterns and not others. We show evidence that this contextual effect may be due to the spatial orientation structure of the stimuli. These findings provide a step toward understanding binocular combination in the natural world and highlight the importance of considering the effect of the spatial interactions in complex stimuli.

Introduction

Our binocular perception of the world is not just a simple average of what the two eyes see. For example, closing one eye does not decrease the apparent brightness of the world by one-half. A complex, hierarchical network of neural circuits is involved in performing binocular combination, and many properties of binocular interactions develop through our visual experience of the natural world (see Başgöze, Mackey, & Cooper, 2018, for a review). 

Laboratory studies in which the inputs to the two eyes are made to be binocularly discrepant (i.e., dichoptic) have been highly influential for characterizing both the general principles by which the visual system integrates information from the two eyes and the importance of experience with natural visual stimulation for normal binocular function (Le Vay, Wiesel, & Hubel, 1980; Wiesel & Hubel, 1963). At the same time, understanding the binocular combination of dichoptic imagery has important clinical implications for conditions such as amblyopia, in which there are disruptions in the balance of suppression and facilitation between the eyes (Ding, Klein, & Levi, 2013a). Beyond amblyopia, there is a range of real-world situations in which image quality is different in the two eyes. For example, a difference in refractive error, a unilateral cataract, or a unilateral scotoma, can result in different levels of apparent contrast between the two eyes. How do these clinical interocular image differences influence binocular percepts during daily life? Lastly, there are emerging stereoscopic display methods that intentionally introduce luminance and contrast differences between the two eyes to create novel graphical effects. However, assessments of the efficacy of these techniques have produced mixed results, suggesting that our existing understanding of binocular combination may be insufficient to account for binocular percepts of this natural dichoptic imagery (Wang & Cooper, 2021; Yang, Kanazawa, & Yamaguchi, 2012; Zhang, Hu, Liu, & Wong, 2018; Zhong et al., 2019). 

Common stimuli used to study binocular combination of dichoptic imagery include simple shapes or gratings with different luminance, colors, or contrast between the two eyes. Under most conditions, the existing literature tends to find a winner-take-all pattern, in which the binocular percept of the dichoptic stimulus is dominated by the eye seeing the stronger stimulus, that is, the brighter, more saturated, or higher contrast stimulus (Ding, Klein, & Levi, 2013b; Ding & Levi, 2017; Huang, Zhou, Zhou, & Lu, 2010; Kingdom & Libenson, 2015; Legge & Rubin, 1981; Levelt, 1965). However, there are situations in which the weaker stimulus contributes more to the binocular percept, resulting in a percept that is closer to the average of the two eyes or even loser-take-all. For example, binocular luminance combination depends on the luminance level being tested, and some luminance levels are associated with averaging (Baker, Wallis, Georgeson, & Meese, 2012; Ding & Levi, 2017). In addition, adding a contour to the lower luminance eye shifts the binocular percept toward that eye (Ding & Levi, 2017; Levelt, 1965). Studies of dichoptic color perception and dichoptic masking have shown that adding luminance contrast to one or both eyes can shift color perception toward averaging and can decrease masking effects (Jennings & Kingdom, 2019; Kingdom & Libenson, 2015; Kingdom & Wang, 2015). 

Although these studies have greatly advanced our understanding of binocular combination, laboratory stimuli often have limited visual complexity and do not capture the rich spectral and contextual information available in the natural environment. For example, natural images are a specific subset of all possible visual stimuli and contain statistical regularities that the brain exploits during visual encoding (Simoncelli & Olshausen, 2001). One common property of natural images is that they tend to have a broadband spatial frequency composition with amplitude spectra that fall off predictably as a function of frequency (Field, 1987). This property is important to consider for binocular combination of natural imagery because there is evidence that binocular interactions are different across different spatial frequencies (Alberti & Bex, 2018). However, it is unknown whether and how these spatial frequency regularities influence how the visual system combines naturalistic stimuli to form a coherent binocular percept. 

When compared with natural viewing, a second key element that is missing in many prior studies of binocular combination is the influence of surrounding context. Contextual effects are ubiquitous in perception and it is thus ultimately essential to consider them for understanding binocular percepts during natural vision. For example, the same patch of gray can be perceived to be a different shade depending on whether it is on a dark or light background (Adelson, 1993) and the same grating can appear to have different contrast depending on its surrounding context (Pamir & Boyaci, 2016; Xing & Heeger, 2001). An effect known as surround suppression, in which a pattern seems to have a higher or lower contrast depending on the visual similarity of the surrounding area, can be observed both in early cortical neurons and psychophysically (Xing & Heeger, 2001). Importantly, human behavioral data show that surround suppression can also occur when the target is in one eye while the context is shown to the other eye, suggesting a binocular interaction (Schallmo & Murray, 2016). It has recently been demonstrated that surrounding context can also decrease the winner-take-all effect in binocular contrast perception, as well as related phenomena such as dichoptic masking (Han, Huang, Su, He, & Ooi, 2020; Jennings & Kingdom, 2019). For example, if there is a monocular contour around a low-contrast sine wave grating (induced by a phase shift with its surround), the lower contrast sine wave grating was shown to dominate the dichoptic contrast percept in a loser-take-all manner (Han et al., 2020). However, our understanding of contextual effects in dichoptic perception remains incomplete. 

In this study, we investigated the effect of surrounding context on binocular combination with noise and natural texture patterns that have rich spatial content. We compared the results to those obtained with simple grating patterns. We present the findings from two experiments. In experiment 1, we examine how different surrounding contexts influence binocular contrast perception of gratings, 1/f noise, and natural textures. In experiment 2, we further investigate which properties of the stimulus patterns may contribute to differences in binocular contrast perception, and we find that contextual effects are strongest when the spatial orientation structure is continuous between the stimulus and its surrounding context. Our results suggest that binocular contrast combination during natural viewing depends on both the spatial structure of the imagery, as well as the visual similarity with the surrounding context. 

Methods

Participants

Experiment 1 had 10 participants (ages 22–26 years, all female), and experiment 2 had 34 participants (ages 19–32 years, 25 female). Two participants from experiment 2 were later excluded from the analysis (see outlier criteria in the Data analysis section). All participants had normal or corrected-to-normal visual acuity and stereo vision. The targeted participant sample size in experiment 2 was increased by a factor of 3 from experiment 1 to increase statistical power, but no formal power analysis was used. The experimental procedures were approved by the University of California, Berkeley Institutional Review Board and were consistent with the tenets of the Declaration of Helsinki. Participants were compensated for their time. 

Setup

All stimuli were presented on two linearized liquid crystal displays (LCDs) (LG 32UD99-W, 3840 × 2160 pixels, maximum luminance of 138cd/m²) and viewed through mirror haploscope as shown in Figure 1. The stimuli were generated using MATLAB (MathWorks, Inc., Natick, MA) and Psychtoolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). During the experiment, participants sat in a dark room and viewed the displays at a distance of 63 cm with their head stabilized by a chin rest. 

Figure 1.

 

Top–down view of the haploscope used to present stimuli independently to the two eyes. The red line indicates the line of sight.

Procedure

On each trial, two images were presented on each monitor (four total). Each image consisted of a 2° diameter circular region embedded in a surrounding 4° × 4° square region (Figure 2A). When participants viewed these stimuli binocularly, the four images were fused into two: one image positioned in the upper half of the screen and the other positioned in the lower half of the screen (Figure 2B). One pair of images comprised the reference stimulus and the other pair comprised the adjustable stimulus (i.e., the test stimulus). The positions of the reference and adjustable stimuli were swapped for half of the participants (i.e., half of the participants adjusted the lower stimulus and half adjusted the upper stimulus). 

Figure 2.

 

(A) Each image had a center-surround layout, in which the contrast of the center and surround could differ. (B) On each trial, two stimuli (four images) were shown. The dichoptic reference stimulus was fixed and had different contrasts between the two eyes except during catch trials. Participants increased or decreased the contrast of the central region of the nondichoptic adjustable stimulus to match the appearance of the reference stimulus. The surround contrast was always the same in both eyes and both stimuli.

The participant's task was to change the physical contrast of the adjustable stimulus's central region until it perceptually matched, as closely as possible, the reference stimulus's central region. There was no time limit for the matching, and participants were able to look back and forth between the two stimuli to compare. On each trial, the contrast of the reference shown to the left and right eyes were selected from a predetermined set. During most trials, these contrasts differed between the two eyes; that is, the reference stimulus was dichoptic. There were some trials in which the reference stimulus was nondichoptic. These primarily served as catch trials. The adjustable stimulus always had the same contrast in both eyes (nondichoptic). The reference stimulus and the adjustable stimulus always shared the same physical contrast in the surround region. 

Stimulus contrast in the central region was adjusted by applying a multiplicative scale factor to an original image with 100% contrast. Image contrasts were adjusted according to the following formula:  

\begin{eqnarray}\qquad\quad {\bf{N}} = c\left( {{\bf{M}} - u} \right) + u,\quad\end{eqnarray}

(1)

where N is the contrast adjusted image, c is the contrast adjustment scalar, M is the original full contrast image, and u is the mean pixel intensity of the original image. For grating stimuli, this adjustment was equivalent to adjusting the Michelson contrast. The initial contrast of the adjustable stimulus was randomly selected between levels 0 and 1, and the participants made adjustments in steps of 0.05. 

Contextual modulation

The surround contrasts were fixed on each trial and not adjustable by the participant. To examine the effects of surrounding context, we explored three different surround conditions (Figure 3). In experiment 1, three contextual modulations were considered: 1) mean surround, 2) low surround, and 3) high surround. In experiment 2, just the mean surround and high surround conditions were tested on a new set of stimulus patterns (see Stimulus types section). 

Figure 3.

 

A pair of dichoptic reference images shown to the left and right eye in the (A) mean surround condition (binocular edge), (B) low surround condition (monocular edge in the eye viewing higher contrast target), and (C) high surround condition (monocular edge around the lower contrast target).

In the mean surround condition, the surround contrast of all four images on a given trial was equal to the mean contrast of the reference's central regions (Figure 3A). In the low surround condition, the contrast of all images’ surround was equal to the lower contrast of the dichoptic reference images (Figure 3B). Finally, in the high surround condition, the contrast of the surround was equal to the higher contrast of the dichoptic reference images (Figure 3C). It is worth noting that, in both the low and high surround conditions, there is a visible edge around the circular target present in one eye but not the other. We hypothesized that this contrast edge serves a similar purpose to the monocular contours in prior dichoptic luminance combination studies, where the presence of the contour in one eye boosts the contribution of that eye's image in binocular combination (Ding & Levi, 2017; Han et al., 2020; Jennings & Kingdom, 2019; Levelt, 1965). In this sense, the mean surround condition is more similar to the existing literature in which there is a contour in both eyes’ images. On the other hand, a monocular stimulus commonly used in the literature (i.e., one eye sees uniform gray and the other eye sees a target with some contrast) would be most similar to a trial in the low surround condition with zero contrast in the surround (Ding et al., 2013b; Ding & Sperling, 2006; Huang et al., 2010). 

Stimulus types

Another goal of this project was to assess whether the rules of binocular combination for simple stimuli can be generalized to more complex stimuli across different contexts. In experiment 1, we examined four types of stimuli: a low spatial frequency sine wave grating (1 cpd), a higher spatial frequency sine wave grating (5 cpd), natural image patches, and 1/f two-dimensional noise (Figure 4). The grating stimuli were vertical gratings (Figure 4A). We selected three natural image patches from the McGill Calibrated Image Dataset as the natural image stimuli (Olmos & Kingdom, 2004). These images contained natural textures, such as foliage, without obvious recognizable objects or scenes (Figure 4B). The noise stimuli were generated by phase scrambling the natural image patches so that the amplitude spectra matched closely with the natural images. We also included a synthetic 1/f noise stimulus with an amplitude spectrum slope of –1 in log–log space and a Gaussian intensity distribution (Figure 4C). All stimuli were gray scale. In summary, we tested all three surround conditions (mean, low, high) on the nine stimulus patterns (two gratings, four noise images, three natural images). For each stimulus type and surround condition, the contrast level of the center region of the reference was set to five preselected contrast levels for each eye (0, 0.25, 0.5, 0.75, and 1), resulting in 25 contrast combinations between the two eyes. In total, there were 675 trials with no repeats. 

Figure 4.

 

Stimulus types used in experiment 1, each shown with center and surround both having contrast of 1. Stimuli included (A) sine wave gratings (1 cpd and 5 cpd), (B) natural textures (with amplitude spectra slopes of −0.9, −0.7, and −0.9 on a log–log scale (Olmos & Kingdom, 2004), and (C) 1/f noise. Three of the 1/f noise images were generated by phase scrambling the natural textures, the fourth (bottom right) was synthesized to have a slope of −1 in log-log space and a Gaussian intensity histogram.

In experiment 2, we aimed to replicate and extend the findings from experiment 1. We only included the mean surround and high surround conditions. We included the 5-cpd sine wave grating and the synthetic 1/f noise from experiment 1 as baselines in experiment 2. Based on the results of experiment 1, we also created a set of intermediate stimuli to match specific properties of the grating and noise so that we could examine what factors contributed to the differences between the results for these stimuli observed in experiment 1. 

Specifically, we adjusted the noise stimulus in three ways to make it more similar to the grating stimulus (Figures 5C–E). First, we created a noise pattern with its pixel histogram matched to the grating (Figure 5C). Second, we bandpass-filtered the noise image around 5 cpd to create a narrowband noise image (Figure 5D). Third, we took the first row of pixels in the noise image (one-dimensional noise) and repeated it across all rows to create a broadband grating that contained a single orientation as with the grating stimulus (Figure 5E). As a control, we also included a horizontal 5-cpd sine wave grating in addition to the vertical 5-cpd grating for comparison (not shown). 

Figure 5.

 

Example images of the stimuli used in experiment 2, with center and surround both having contrast of 1. Stimulus types included (A) the vertical 5 cpd grating from experiment 1, (B) the 1/f noise pattern from experiment 1, (C) the noise pattern with histogram adjusted to match the grating, (D) the noise pattern with bandpass filtering centered at 5 cpd, and (E) a broadband grating.

Last, to evaluate the role of edge contrast, we also included an edge-blurred version of the vertical grating and noise stimuli (Figure 6). The blurring was done by applying two different masks to the images of center and surround separately. A Gaussian filter was applied to the masks such that the mask for the center peaks at the middle of image and has a decreasing ramp toward the surround. The mask for the surround peaks at the surround and ramps in the opposite direction. Effectively, the masks crop the center and surround with blurred border and the masked center and surround images were then added together. The standard deviation (σ) of the Gaussian filter was 0.5°. Without edge blur, it is notable that the edge appears more visible in the grating than in the noise images. The edge blur seems to reduce the visibility of the edge for both stimuli (Figures 6B, D). 

Figure 6.

 

Example images of high surround stimuli (surround contrast = 1) for the low contrast center eye (center contrast = 0.5) used in experiment 2. (A, B) The original 5-cpd grating and edge-blurred 5-cpd grating. (C, D) The original noise and edge-blurred noise.

In summary, for experiment 2 we tested the mean surround and high surround conditions, with eight different stimulus patterns (2 baseline stimuli, 3 intermediate stimuli, 2 edge-blurred, and 1 horizontal grating) and four contrast levels for each eye's reference stimulus: 0, 0.25, 0.5, and 1, resulting in 15 combinations. We did not include the 0 and 0 combination since this combination is simply uniform gray for both eyes’ reference images, and results from experiment 1 showed that participants matched this condition close to 0 contrast as expected. In total, there were 240 trials in experiment 2. 

Data analysis

Outlier criteria

Outliers were determined by assessing the performance on catch trials in which both the reference and the adjusted stimuli were nondichoptic. On these trials, we expect the adjustment to be a very close match to the reference if the participants are motivated to perform the task correctly. We calculated the square root of the mean squared error (RMSE) for each participant on the catch trials to capture the average error between an individual's match and the theoretical perfect match. If a participant's RMSE exceeded 1.5 times the interquartile range above and below the 75th and 25th percentiles across all participants, they were considered to be an outlier. Based on this criterion, none of the participants were excluded from experiment 1. For experiment 2, two of the 34 participants were excluded from further analysis. 

Data transformation

There are three naive predictions we can make about the appearance of the dichoptic reference stimulus. If the perceived contrast of the reference matches the eye seeing the higher contrast, then the results would follow a winner-take-all pattern (Figure 7A). In this prediction plot, the rows and columns of the heatmap correspond with the contrast levels shown to the two eyes for the reference stimulus and the matches for the adjustable stimuli are indicated by the gray levels. For a winner-take-all pattern, the predicted contrast match in each row and column pair is always equal to the higher contrast between the left and right eye. On the other hand, if the dichoptic reference percept is matched with the average contrast of the two eyes’ images, the resulting data would be similar to those shown in Figure 7B. Finally, in some conditions the perceived contrast of the reference stimulus could be matched to the lower contrast image (Figure 7C). Note that these predictions are illustrative and there are known cases, such as the Fechner's paradox (Gilchrist & McIver, 1985; Levelt, 1965), that do not fall into these three simple categories across all contrast combinations. 

Figure 7.

 

Hypothetical data showing three different naive predictions about the perceived contrast of the reference stimulus. The grey color of each square corresponds to the matched adjustable stimulus’ contrast for a given left (x axis) and right eye contrast (y axis) of the reference stimulus. The patterns shown are: (A) adjustable stimulus matches the higher contrast reference image, (B) adjustable stimulus matches the average contrast of the two reference images, and (C) adjustable stimulus matches the lower contrast reference image.

To create a summary measure of binocular combination and facilitate comparisons across different surround conditions and stimulus types, we wanted a method to fit the data represented by each heatmap with a single summary parameter. In a similar experiment to ours, Legge and Rubin (1981) used a nonlinear equation to fit their data, where an exponent parameter was varied to capture the three types of predictions. However, one disadvantage of this method is that the possible parameter space is not constrained. The exponent is equal to 1 for averaging, but needs to be at infinity to fit winner-take-all and negative infinity to achieve loser-take-all. 

We decided instead to express our data as a multiplicative weight (w) on the higher contrast stimulus, with a more restricted parameter space, between 0 and 1. We model the matched nondichoptic contrast (b) as a weighted average of the two eyes’ contrasts (h and l), for high and low. For example, a weight (w) of 1 for the higher contrast stimulus (h) would mean that the perceived contrast is solely determined by the high-contrast image (winner-take-all), a weight of 0.5 means averaging, and a weight of 0 means loser-take-all:  

\begin{eqnarray} \qquad\qquad\quad {\rm{b}} = w{{h}} + \left( {1 - w} \right){{l}}.\quad\end{eqnarray}

(2)

 

To determine the best fitting weight, we performed a grid search with weights ranging from 0 to 1 in steps of 0.01, and minimized the RMSE for each participant and each unique stimulus combination. This model is meant to capture the data rather than suggest the underlying neural computation involved. Because the weights were very similar across different exemplars of noise and natural images, we averaged the fitted weights across these exemplars for each participant. We kept the two grating stimuli separate to examine potential differences between higher and lower spatial frequencies. We compared our linear combination method with the nonlinear method used by Legge and Rubin and confirmed that our simple weighted combination method achieves similar RMSE with the best fit parameters. A comparison of the fitting performance between the two methods is shown in Table 1. In general, the fits of both models to the data from experiment 2 were worse. In the Results section, we describe an exploratory analysis of eye dominance effects in experiment 2, which may explain the larger errors. 

Table 1.

 

Fitting performance of our method compared with the Legge and Rubin method. The mean RMSE for each method across all stimulus types and surround conditions for all participants is indicated, and the standard deviation of the RMSEs for the different best fit parameters is shown in parentheses.

 

Statistical tests

Planned analyses included two-way analyses of variance (ANOVAs) (surround condition by stimulus type). A preliminary analysis of the results suggested that the responses did not satisfy the assumptions of a conventional ANOVA, so two-way permutation ANOVAs were used to determine statistical significance (aovp function from lmPerm R package). We denote the P values from the permutation ANOVA as Pp. We also ran the regular ANOVA and report the F statistics and effect size (generalized eta squared). Follow-up pairwise comparisons were conducted using Wilcoxon sign rank tests (wilcox.exact from exactRankTests R package) and P values were corrected for multiple comparisons using Bonferroni correction. A threshold of a P value of less than 0.05 was used to determine statistical significance throughout. 

Results

Experiment 1

To examine whether different stimulus types produced different binocular combination rules, we first compare all stimulus types in the mean surround condition, which has similar contour information in the two eyes. In Figure 8, the average response across subjects is shown in the same format as the predictions in Figure 7. The fitted weights that best describe each subject's data are shown in Figure 9A. Qualitatively, all stimulus types produced percepts that closely followed the winner-take-all pattern with a high weight given to the eye viewing the higher contrast image. The 5-cpd grating had a slight decrease in weight compared with the other stimuli (but this decrease was not statistically significant, see below). 

Figure 8.

 

Experiment 1 (N = 10), mean surround condition data averaged across all participants for the different stimulus types: (A, B) gratings, (C) noise stimuli, and (D) natural textures.

Figure 9.

 

Experiment 1 results (N = 10) for the four stimulus types in the (A) mean surround, (B) low surround, and (C) high surround conditions. The box-and-whisker plots show the median weight of the higher contrast image across individuals, the 25th and 75th percentiles, and the nonoutlier range. The black dots indicate each participant's weight. The gray dash lines represent the weights for the three types of combination rules winner-take-all (1), averaging (0.5), and loser-take-all (0).

How do manipulations of the surrounding context influence this pattern of results? In the low surround condition (Figure 9B), the eye seeing a higher contrast center contains an edge contour and the other eye does not. We expected that the contour would bias the binocular percept even more toward the high-contrast eye, giving it more weight. Indeed, qualitatively we see a weight close to 1 for all stimuli with no clear difference among the stimulus types. Finally, we expected the high surround condition (Figure 9C) to be associated with a lower weight on the higher contrast eye (and more weight on the lower contrast eye) compared with the previous two surround conditions, owing to the monocular contour in the eye seeing the lower contrast center. Consistent with this prediction, all stimulus types were associated with a decreased weight on the higher contrast eye in this condition. Interestingly, however, the noise and natural stimuli seemed to be less affected by this manipulation than the gratings. 

The main effects of surround condition and stimulus type were both statistically significant (surround, F(2,18) = 23.28, Pp < 0.001,η² = 0.54; stimulus, F(3,27) = 23.40, Pp < 0.001, η² = 0.24), and the interaction between surround condition and stimulus type was also statistically significant (F(6,54) = 27.77, Pp < 0.001, η² = 0.31). We asked two questions in the follow-up tests. First, we asked how the weights associated with the different stimuli compared with each other within each surround condition (Bonferroni correction for 18 tests). In the mean surround condition, the weight associated with the 5-cpd grating was significantly lower than the weight associated with the 1cpd grating (P = 0.04), but not the other stimuli. There were no significant differences among the stimulus types in the low surround condition. In the high surround condition, the 5-cpd grating was associated with a significantly lower weight than all other stimuli (Ps = 0.04). Second, we asked how the surrounding context affected each different stimulus type (Bonferroni correction for 12 tests). For both grating stimuli, we found that the high surround condition was associated with significantly lower weight as compared with both the mean surround and low surround conditions (Ps = 0.02). No other stimuli were associated with statistically significant differences across the different surround conditions. These results suggest that the surround significantly affected perceived binocular contrast for the grating stimuli, but not for the more complex stimuli. 

Experiment 2

In experiment 2, we aimed to replicate the results of experiment 1 and to understand why the effects of the surround differed between stimulus types. As such, we tested several stimuli with visual properties that were intermediate between the gratings and naturalistic stimuli, focusing on the mean surround and high surround comparison. There are many differences between these stimuli, so we considered several hypotheses about what might contribute to the different results: 1) the gratings are narrowband in spatial frequency, whereas the noise and textures are broadband, 2) the gratings have a different pixel value histogram than noise and textures (more values close to blacks and whites), and 3) the gratings have only one orientation, whereas the noise and textures are broadband in orientation. 

Qualitatively, the results for the grating and noise stimuli in the mean surround condition were consistent with experiment 1, with a higher weight given to the eye viewing the high contrast stimulus (Figure 10A, magenta and green boxes). The intermediate stimuli were also all associated with a high weight for the high contrast eye in this condition (Figure 10A, blue boxes). However, in this experiment the 5-cpd grating stimulus was associated with even more individual variation and a more neutral weighting compared with the other stimulus types in the mean surround condition. In the high surround condition, the results were again qualitatively similar to experiment 1 for the grating and noise: we observed a shift in weight toward the eye viewing the lower contrast stimulus, but the shift was stronger for the grating stimulus (Figure 10B, magenta and green boxes). Matches for all the intermediate stimuli were also shifted toward the lower contrast eye, but only the broadband grating was associated with a substantial shift similar to the 5-cpd grating (Figure 10B, blue boxes). 

Figure 10.

 

Experiment 2 results (N = 32) across the two surround conditions for the five stimulus types: the narrowband 5-cpd grating and noise baseline from experiment 1 (grating and noise), histogram-matched noise (hist eq), bandpass noise (bandpass), and broadband grating (broadband).

An ANOVA conducted on the results of experiment 2 indicated a significant main effect of stimulus type (F(4,124) = 64.15, Pp < 0.001, η² = 0.28) and surround, F(1,31) = 115.06, Pp < 0.001, η² = 0.37. The interaction between stimulus type and surround was also significant (F(4, 124) = 17.29, Pp < 0.001, η² = 0.08). Follow-up tests were conducted again to examine how the results for different stimuli differed from each other in each surround condition (Bonferroni correction for 20 comparisons), and how the results for each stimulus type varied across different surround conditions (Bonferroni correction for five comparisons). When comparing the mean and high surround conditions, there was a significant decrease in the weights associated with the 5-cpd grating from the mean surround to the high surround condition (P < 0.001), as in experiment 1. However, there was also a significant weight decrease between the surround conditions for the noise stimulus and the three intermediate stimuli (Ps < 0.001). That is, the weights for all five stimulus types were decreased in the high surround condition. 

Unlike in experiment 1, the weight associated with the 5-cpd stimulus was different from the weight associated with the noise stimulus in the mean surround condition (P < 0.001; in experiment 1, this difference was visible but not statistically significant, P = 0.18). Indeed, the weight associated with the 5-cpd grating was significantly lower than all other stimuli in this condition (Ps < 0.001). The weight associated with the bandpass noise also differed from noise (P = 0.004), histogram equalized noise (P < 0.001), and broadband grating (P < 0.001). Turning to the high surround condition, we found that both the 5-cpd grating and broadband grating stimuli were associated with statistically lower weights than the other stimulus types (Ps < 0.001), but were not different from each other (P = 0.16). 

Taken together, these results suggest that the broadband grating was the most similar intermediate stimulus to the 5-cpd grating in the high surround condition. The luminance histogram and spatial frequency of the grating may not be key factors for generating strong surround modulation, but the orientation structure may be. Specifically, the broadband grating seemed to be more like the noise and other intermediate stimuli in the mean surround condition but seemed to be more like the 5-cpd grating stimulus in the high surround condition. We conclude that the singular orientation structure of the narrow and broadband gratings likely interacts with the edge contour to modulate contextual effects on binocular contrast perception. 

Stimulus orientation and edge blur

In experiment 2, we also tested a horizontal grating as a control, and edge blurred versions of the grating and noise stimuli to examine whether smoothing out the edge contrast between the center and the surround would weaken the bias toward the lower contrast eye in the high surround condition. Starting with the orientation manipulation (Figure 11A), the main effects of orientation (F(1,31) = 7.08, Pp = 0.01, η² = 0.01) and surround (F(1,31) = 75.32, Pp < 0.001, η² = 0.32) were significant, but there were no significant interactions between these two factors, and the effect size for orientation was small. Thus, we conclude that the surround modulation effect was not substantially disrupted by making the grating horizontal. 

Figure 11.

 

Additional stimuli tested in experiment 2. (A) Comparison between vertical (V) and horizontal (H) 5-cpd gratings. (B) The effect of edge blur on the vertical 5-cpd grating and noise. The red line indicates the line of sight.

For the edge blurred stimuli (Figure 11B), the main effects of blur (F(1,31) = 14.57, Pp = 0.001, η² = 0.01), stimulus type (F(1,31) = 116.60, Pp < 0.001, η² = 0.37), and surround type (F(1,31) = 119.77, Pp < 0.001, η² = 0.30) were each significant. Blur was associated with a higher weight, however, the effect size for blur was small. There were small but significant interactions between surround and stimulus type (F(1,31) = 7.26, Pp = 0.01, η² = 0.03), and between stimulus types and blur (F(1,31) = 5.40, Pp = 0.04, η² < 0.01). In follow-up tests, we looked at the effects of blur and surround for gratings and noise separately. For the grating stimuli, the main effect of edge blur (F(1,31) = 16.01, Pp < 0.001, η² = 0.02) and surround (F(1,31) = 74.05, Pp < 0.001, η² = 0.38) were significant, and there was no significant interaction between surround and edge blur. For the noise, the effect of blur was not significant (F(1,31) = 1.47, Pp = 0.28, η² < 0.01), whereas the surround effect was significant (F(1,31) = 30.33, Pp < 0.001, η² = 0.21). In summary, the effect of blur was significant for the 5-cpd grating, but minimal compared with the effect of surround modulation. 

Eye dominance

In experiment 2, we sought to replicate and extend the findings from the grating and noise stimuli in experiment 1. The effects of interest were generally replicated, however, we also found unexpected variation in the best-fit weights for the 5-cpd grating in the two experiments. We used a post hoc analysis to explore the possibility that the large individual variations observed in our data could be a result of interocular imbalance (i.e., different binocular percepts are observed depending on whether the dominant eye sees the higher contrast image or the lower contrast image). We can visualize eye dominance effects by looking at individual participant's response heatmap. In Figure 12, the responses from two different participants in experiment 2 are shown. Subject A shows strong asymmetry depending on which eye was presented with the higher contrast stimulus in many conditions (in this case, the participant seems to be left eye dominant), whereas subject B does not show such a pattern. Thus, in this post hoc analysis, we separately examined trials in which the high-contrast reference images were presented to the dominant eye versus the nondominant eye. 

Figure 12.

 

Heatmaps showing the nondichoptic contrast match for all stimuli in the mean surround and high surround conditions for two observers in experiment 2. Subject A shows winner-take-all when the left eye sees the higher contrast, but not when the right eye sees the higher contrast in most situations. Subject B generally has the same response regardless of which eye saw higher contrast.

To determine whether the left or right eye was the dominant eye for each participant, we used a criterion defined by Legge and Rubin (1981). We separated out the trials in the mean surround condition based on whether the left eye saw the higher contrast reference target or the right eye saw the higher contrast reference target. After separating the two sets of data, we assessed which eye's data was closer to a winner-take-all model by comparing the RMSE of both eyes’ data against a perfect winner-take-all weighting scheme. The eye that resulted in a lower RMSE was labeled as the dominant eye. We tried three approaches to separating the data. Because the 5-cpd stimulus is the one that we are interested in understanding, we first considered using only trials with the 5-cpd grating stimulus. We also tried using the noise stimulus and all stimuli. The results were similar for all three approaches, so here we report the results when eye dominance was determined from the 5-cpd stimulus (Figure 13). By this measure, 60% of participants in experiment 1 and 63% of participants in experiment 2 are left eye dominant. 

Figure 13.

 

Comparison between trials where the dominant eye (top) or nondominant eye (bottom) was presented with the higher contrast image. Eye dominance is determined by 5cpd mean surround responses. (A) Experiment 1. (B) Experiment 2.

Quantitatively, the trend for different surround and stimulus patterns that we described previously in the main analysis holds both when the dominant eye and nondominant eye were presented with the higher contrast stimulus. However, there are some differences between the dominant eye data and the nondominant eye data. The dominant eye data has less variation and higher weight for the 5-cpd grating compared with the nondominant eye, and when both eye's data are combined in the main analysis (Figure 10). Interestingly, for experiment 2, eye dominance changes the responses for the bandpass noise centered at 5 cpd in the mean surround condition as well, suggesting a spatial frequency dependence. 

Discussion

In the current study, we examined how the spatial complexity of binocular stimuli affects binocular contrast perception. We found that dichoptic contrast percepts for both grating and naturalistic stimuli follow a roughly winner-take-all model when the surrounding contrast matches in the two eyes (but the results from experiment 2 suggest that the interocular weighting is more variable for gratings). Across a range of stimulus types, we also found evidence that monocular edge information owing to a contrast difference with the surround biases dichoptic contrast perception toward the eye seeing the edge. 

Our results with respect to the effects of a monocular edge are directly in line with previous work that assessed the role of a monocular contour in dichoptic luminance perception (Levelt, 1965; Ding & Levi, 2017) and dichoptic contrast perception (Han et al., 2020). This set of results can be well-described in terms of interocular suppression, in which the strength of the suppression depends on the strength of the winning stimulus. We can think of the winner-take-all behavior as a complete suppression of the weaker stimulus by the stronger stimulus. Introducing a monocular feature, such as contour, to the eye with the weaker stimulus could lead to averaging between the two eyes if the added feature causes the weaker stimulus to be less suppressed by the stronger stimulus (i.e., the contour feature strengthens the stimulus). 

Evidence of similar reductions in interocular suppression has also been found in studies of dichoptic masking. Dichoptic masking refers to the observation that the detection threshold for a stimulus presented in one eye can be increased due to the presence of a masking stimulus presented in the other eye (Meese, Georgeson, & Baker, 2006). We can reason that winner-take-all dichoptic contrast perception is congruent with a complete masking of the weaker stimulus by the stronger stimulus. In dichoptic masking, it has been shown that the suppression of a masked target can be reduced to the level of no masking when a monocular ring surrounds the target (Jennings & Kingdom, 2019), similar to the monocular contour effects observed in dichoptic luminance and contrast perception. 

Together, these results support the theory that monocularly-visible contours, however induced, contribute to the perception of dichoptic stimuli. It is worth pointing out that previous studies have also found that adding binocularly matched features to both eyes can modulate suppression effects in dichoptic color masking (Kingdom & Wang, 2015), dichoptic color perception (Kingdom & Libenson, 2015), binocular luster (Wendt & Faul, 2020), and binocular rivalry (Blake & Boothroyd, 1985). Our results thus contribute to converging lines of evidence that both monocularly visible and binocularly matched features modulate suppression, further motivating the notion that contextual relationships must be considered as a whole for determining interocular suppression. 

We also found a surprising result that gratings showed a greater contextual change than more complex noise and natural stimuli. In experiment 2, we tested several properties that could contribute to the difference and found evidence that the spatial orientation of the grating pattern having a (single orientation) contributes to the difference. How and why does the spatial organization of the stimulus affect binocular contrast perception? Here, we discuss two related possibilities: low-level suppression and higher level segmentation. 

Surround suppression is a well-documented characteristic of the visual system. It has been demonstrated that both grating and nongrating stimuli can have different apparent contrast depending on the surrounding contrast, and that the similarity between the center and surround plays a key role in the amount of modulation (Chubb, Sperling, & Solomon, 1989; Snowden & Hammett, 1998; Xing & Heeger, 2001). It is possible that the visual similarity between the center and surround in our grating and noise stimuli differs, which contributes to a difference in the amount of surround modulation on perceived contrast. For example, the similar orientation between the center and surround regions of the grating stimuli could cause stronger surround suppression in monocular pathways, which would thus lead to a stronger monocular edge between the center and surround and cause the eye with the edge to dominate more. On the other hand, for the broad orientation stimuli, there may be less suppression because the center and surround differ more in their orientation properties. It is important to note that, in our study, the noise and natural textures do not have a dominant orientation. However, in real-world scenarios, one can imagine viewing a dichoptic scene with trees or buildings that have a strong orientation bias, similar to the grating stimuli. Quantifying surround suppression for stimuli with a combination of spatial frequencies, phases, orientations, contrast, and luminance is difficult but may be important for ultimately explaining the full pattern of binocular contrast perception with natural stimuli. In addition, there are likely other types of suppression, both within and between the monocular pathways. For example, previous work has modeled suppression effects within each eye (cross-orientation suppression and masking) and between the eyes (dichoptic masking) at different stages of visual processing (Baker, Meese, & Summers, 2007). It would be interesting to see if this framework could be adopted to reflect surround suppression and dichoptic contrast perception as well. In general, monocular suppression mechanisms are likely key for determining how monocular channels interact upon binocular combination. Existing binocular combination models can generate good predictions for binocular perception of simple stimuli (Baker, Meese, & Georgeson, 2007; Ding et al., 2013b; Ding & Levi, 2017; Jennings & Kingdom, 2019); however, a versatile model that can predict binocular perception for any dichoptic image pair would be very desirable and would likely need to consider all the aforementioned characteristics of each eye's input. A precise computational instantiation of stimulus strength and its role in suppression remains a topic of ongoing empirical and modeling research. 

Another potential explanation for the contextual effects observed in this study is a difference in higher order segmentation for the single orientation versus broad orientation stimuli. It may be that the single orientation and regularity of the narrow and broadband grating patterns made it possible for observers to visually segment the center and surround better. In other words, the local contrast variation in the center and surround regions of the orientation–broadband stimuli (e.g., noise) could make it harder to segment them, which in turn may make it more difficult for the visual system to detect contours. This hypothesis is similar to earlier work showing that orientation-induced boundary contours influence eye dominance dynamics during binocular rivalry (Ooi & He, 2006) and that the subjective strength of a phase shift–induced contour determines which eye dominates the binocular percept (Han et al., 2020). There is some evidence that the visual system first determines the contour information then processes the interior of the enclosed contour via a filling-in mechanism (Su, He, & Ooi, 2011). Whether this segmentation is performed in the brain by the same mechanism underlying surround suppression of simple stimuli is unclear (Poltoratski & Tong, 2020). 

From the perspective of real-world applications, these results provide a foundation for assessing how visual information beyond the target of interest can influence binocular contrast perception. In our study, the high surround condition approximates a partial monocular scotoma where one eye has a region of low contrast compared with the rest of the view, whereas the other eye may have no vision defects. We found that this condition biases the binocular percept toward the lower contrast stimulus in persons with normal vision. We hypothesized that blurring the edge would decrease this bias, but the blurring manipulation in our study ultimately had minimal effect. It is possible that greater amounts of edge blur may produce stronger results, though. However, it is unclear how closely this manipulation applies to real-life scotomas (Fletcher et al., 2012). Indeed, it is still an ongoing area of research to characterize the vision of people with scotomas and accurately simulate scotomas (Durgin, Tripathy, & Levi, 1995; Reich, Levi, & Frishman, 2000; Tripathy, Levi, & Ogmen, 1996; Tripathy & Levi, 1999; Verghese & Ghahghaei, 2020). It is possible that the high surround stimulus does not correlate well with what people with scotomas experience, particularly if patients engage adaptation mechanisms to correct for local contrast sensitivity loss. Nonetheless, the importance of surrounding context suggests that the way the two eyes combine information relies on not just the corresponding areas in the two eyes, but also what is outside of those areas. 

Finally, we explored the effect of eye dominance on dichoptic contrast perception and found evidence that eye dominance exerts a greater influence over the dichoptic percept for spatially narrowband stimuli than broadband stimuli (i.e., differences owing to eye imbalance were more noticeable for the 5-cpd gratings and bandpass noise). Assessing eye dominance can have important implications for training and rehabilitation of eye imbalance problems, for example in the extreme case of amblyopia. In the research literature, several methods have been proposed to measure sensory eye dominance (Ooi & He, 2020). One method involves assessing eye balance by adjusting the interocular contrast difference between the two eyes to null the effects of an interocular phase difference (Ding & Sperling, 2006), orientation difference (Yehezkel, Ding, Sterkin, Polat, & Levi, 2016), or masking (Barboni et al., 2020). In amblyopic vision, the binocular imbalance depends on both spatial frequency and stimulus contrast, with more imbalance at higher frequencies and higher contrast (Ding et al., 2013a; Ding & Levi, 2014). In addition, it has been shown that, in adults with normal vision, there is a spatial frequency dependency of eye dominance, such that there is a greater imbalance at higher spatial frequencies measured using phase and orientation nulling tasks (Wang et al., 2019). Our study used the dichoptic contrast appearance as the measure of eye dominance, but found a similar result: eye imbalance was spatial frequency dependent, having a stronger effect on the 5cpd grating than the broadband or lower frequency stimuli. It remains to be explored why there is a spatial frequency difference in eye imbalance in the normal population. 

Conclusion

Binocular contrast combination rules learned from simple stimuli such as sine wave gratings do not generalize fully to more naturalistic stimuli. Here, we uncovered one property that likely contributes to the difference between gratings and naturalistic stimuli: the restricted spatial orientation of gratings results in context modulations that differ from orientation-broadband stimuli. Future research will be necessary to understand the underlying mechanism of this effect. We hypothesize that surround suppression and/or image segmentation may be important factors. Going beyond simple stimuli to study binocular contrast combination of complex spatial patterns will inform the development of better models that can predict binocular perception during natural vision. 

Acknowledgments

Funded by NSF Award #2041726 to EAC, a CIVO fellowship to MW, NEI RO1EY030544 to JD. There are no commercial interests for this work with any of the authors. 

Commercial relationships: none. 

Corresponding author: Minqi Wang. 

Email: [e-mail]. 

Address: University of California, Berkeley, 391 Minor Hall, Berkeley, CA 94720, USA. 

Appendix

Cross-fusible stimuli from experiment 1 and 2 are shown below. The black lines are added to aid cross fusion. Experiment 1 stimuli includes two grating patterns, four noise patterns, and three natural textures (Figure 14). Experiment 2 stimuli include the baseline grating and noise pattern, along with noise histogram matched to grating, bandpass noise, and broadband grating (one-dimensional noise) (Figure 15). Here, we are also showing what experiment 2 stimuli would look like for the low surround condition, which was not tested in experiment 2. We encourage the reader to look at these stimuli and see how the contrast of the centers for the 1 cpd, 5 cpd, and broadband grating change between the high surround condition and the other two conditions. 

Figure 14.

 

Cross-fusion examples of experiment 1 stimuli.

Figure 15.

 

Cross-fusion examples of experiment 2 stimuli.