Segmentation decreases the magnitude of the tilt illusion

Abstract

Abstract

Abstract:

Abstract  In the tilt illusion, the perceived orientation of a target grating depends strongly on the orientation of a surround. When the orientations of the center and surround gratings differ by a small angle, the center grating appears to tilt away from the surround orientation (repulsion), whereas for a large difference in angle, the center appears to tilt toward the surround orientation (attraction). In order to understand how segmentation/perceptual grouping of the center and surround affect the magnitude of the tilt illusion, we conducted three psychophysical experiments in which we measured observers' perception of center orientation as a function of center-surround relative contrast, relative disparity depth, and geometric features such as occlusion and collinearity. All of these manipulations affected the strength of perceived orientation bias in the center. Our results suggest that if stronger segmentation/perceptual grouping is induced between the center and surround, the tilt repulsion bias decreases/increases. A grouping-dependent tilt illusion plays an important role in visual search and detection by enhancing the sensitivity of our visual system to feature discrepancies, especially in relatively homogenous environments.

Introduction

Many visual illusions are the result of contextual modulation: Influenced by contextual information, we often perceive things differently from their physical reality. In the case of orientation perception, it has been demonstrated that the orientation of the surround affects the perceived orientation of the center (Blakemore, Carpenter, & Georgeson, 1970; Gibson, 1937; Goddard, Clifford, & Solomon, 2008; Schwartz, Sejnowski, & Dayan, 2009). A central grating is perceived as tilted away from the orientation of a surround grating when the two orientations are similar; this is called the direct (repulsion) form of the tilt illusion. When the center and surround orientations differ considerably, the perceived orientation of the central grating is attracted toward the surround orientation, which is known as the indirect (attraction) tilt illusion. The relative orientation between the center and surround determines whether we perceive the repulsion or attraction effect. 

The neural basis for the tilt illusion can be modeled as changes in the tuning curves of individual orientation-selective units in the presence of the surround (Blakemore, Carpenter, & Georgeson, 1971; Blakemore & Tobin, 1972; Clifford, Wenderoth, & Spehar, 2000; Gilbert & Wiesel, 1990; Schwartz, Hsu, & Dayan, 2007), and with the perceived orientation of the center being determined by the vector average (Georgopoulos, Schwartz, & Kettner, 1986) of the units' responses. The effect can also be modeled by lateral interactions at the population level (Bednar & Miikkulainen, 2000; Solomon, Felisberti, & Morgan, 2004). Electrophysiological results have demonstrated that modulations of neural response by surrounding context include magnitude variation (Cavanaugh, Bair, & Movshon, 2002; Levitt & Lund, 1997; Li, Thier, & Wehrhahn, 2000; Muller, Metha, Krauskopf, & Lennie, 2002; Sengpiel, Sen, & Blakemore, 1997; van der Smagt, Wehrhahn, & Albright, 2005), broadening or sharpening of tuning widths (Gilbert & Wiesel, 1990), and repulsive or attractive shifts in preferred orientation (Felsen, Touryan, & Dan, 2005; Gilbert & Wiesel, 1990). The tuning curve changes may serve to optimize sensory coding (Clifford, Wenderoth, & Spehar, 2000; Schwartz, Hsu, & Dayan, 2007; Simoncelli, 2003). Using principles of efficient coding of the input signals, the extra constraint provided by the context allows the central detectors to remove statistical dependencies, which acts as a transform that reduces redundancies among inputs (Attneave, 1954; Barlow, 1961; Li & Atick, 1994; Olshausen & Field, 1996). A simple efficient coding transform is divisive gain control normalization (Albrecht & Geisler, 1991; Carandini & Heeger, 1994, 2012; Heeger, 1992; Lyu, 2010, 2011), which nicely explains nonlinear response properties of neurons in primary visual cortex (Carandini, Heeger, & Movshon, 1997; Schwartz & Simoncelli, 2001; Simoncelli & Schwartz, 1999). 

Further studies (Coen-Cagli, Dayan, & Schwartz, 2012; Schwartz, Sejnowski, & Dayan, 2006; Schwartz et al., 2009) have shown that this divisive normalization process may only apply when the center and context are perceptually assigned to the same object or segment in natural scenes. Based on statistical measurements in natural images, Schwartz and colleagues (2009) reported that across segmentation boundaries, the orientation dependence between the central and surround patches was greatly reduced. Therefore, they proposed to combine a segmentation factor with a divisive gain control model to account for natural image statistical dependence more accurately. This model provides a unified explanation for both repulsion and attraction in the tilt illusion. In their model, the segmentation factor is controlled by center-surround relative orientations. The closer the relative orientations, the more likely they share the same gain pool. In this study, we explored whether segmentation/perceptual grouping cues other than relative orientation could be used by the visual system in a similar way to manipulate the tilt effects. Specifically, we tested local image features of relative contrast and disparity depth (Experiment 1), and geometric features, such as occlusion (Experiment 2) and collinearity (Experiment 3), in influencing the perception of central orientations. 

For relative contrast, the greatest tilt repulsion occurs when the center and surround gratings have the same contrast (Durant & Clifford, 2006; Tolhurst & Thompson, 1975), suggesting that contrast differences might provide segmentation cues to reduce the magnitude of the tilt bias. The effect of contrast cues on the tilt attraction has not been studied thoroughly. Previous findings showed that manipulations of spatial separation or spatial frequency have no significant effect on the attraction (Wenderoth & Johnstone, 1988) even though they change the tilt repulsion (Georgeson, 1973; Tolhurst & Thompson, 1975). We asked whether this pattern of results extends to the contrast cues, and whether the contrast cues affect the tilt repulsion and attraction differently. 

Similarly, effects of depth disparity on the tilt illusion are unclear. Using line segments, Sakai & Hirai (2002) and Westheimer (1990) observed that apparent tilt did not depend on the stereo disparity cues between the target and contextual bars. However, Durant and Clifford (2006) obtained reduction of the tilt repulsion with stereo disparity cues between the center and surround gratings. Since relative depth cues, just like relative contrast cues, would influence perceptual segmentation between the center and surround, we expected that both could manipulate tilt biases (Experiment 1) in a manner that could be predicted by the Schwartz model (Schwartz et al., 2009). 

In addition to the relative contrast and disparity depth between the center and surround, geometric features could also be an important factor for segmentation/perceptual grouping. For example, in three-dimensional (3-D) space, an occluding ring in front of a border between target and context would make the border ambiguous and actually encourage the grouping of the center and surround. This would leave the filtering input unchanged in the Schwartz model, and more directly reveal the effect of segmentation/perceptual grouping on the tilt repulsion (Experiment 2). Spatial layout is another cue that influences perceptual organization. Based on the natural image statistics, the end position along the central elements follows the most frequent direction of edge co-occurrence (Geisler, Perry, Super, & Gallogly, 2001; Sigman, Cecchi, Gilbert, & Magnasco, 2001). This result indicates that surround patches that are collinear with a central grating would provide stronger evidence for contour grouping than patches flanking the center. Hence, we predicted that this collinear layout would show different effects on the central orientation perception (Experiment 3). 

In the three experiments described below we systematically measured the tilt illusion affected by different segmentation cues between the center and surround, and sought to understand how the visual system responds to central orientation given various combinations of segmentation/perceptual grouping cues. In the first experiment, we examined whether the tilt illusion could be influenced by contrast and depth differences between the center and surround. To account for our results, we expanded the segmentation model by Schwartz et al. (2009) to include the contrast and depth cues. We showed that the model could account for the decrease of the tilt effects and orientation-tuning shift of the tilt biases as a function of relative orientation. In the second experiment, we used an occluding ring to affect perceptual grouping while maintaining the filtering activation in the Schwartz model. As predicted, increase of perceptual grouping cues led to stronger tilt repulsion. In Experiment 3, we measured the tilt repulsion with different surround spatial layouts, which showed that the maximal repulsion bias occurred when gratings were along the end locations of the central stimulus. 

Experiment 1: Relative contrast and depth

Methods

General

Six observers (mean age: 29 years, three males) with normal or corrected-to-normal visual acuity were tested. All were trained for a short time (2–5 min) in order to get acquainted with the task and to obtain ranges of individual stimuli variation. Four observers participated in all experimental conditions, and two observers completed six out of eight conditions. 

Visual stimuli, sinusoidal gratings of the same mean luminance as the background, were generated using Matlab (Mathworks, Inc., Natick, MA) in conjunction with the Psychophysics toolbox (Brainard, 1997; Pelli, 1997). They were displayed on a high-resolution monitor (1600 × 1200 pixels, 60 Hz refresh rate, NEC MultiSync LCD 2190 uxi, NEC Corporation, Tokyo, Japan) connected to a Mac mini (Apple, Inc., Cupertino, CA). Observers were seated 60 cm away from the screen. A stereoscope and split screen were used in all conditions to maintain consistency across conditions, and at the beginning of each session, the stereoscope was adjusted by aligning two short nonius lines. 

Procedure

Observers were shown stimuli at the center of the visual field, and were required to make binary judgments about the orientation of the central grating as tilted clockwise or counterclockwise from vertical. Stimulus duration was 500 ms. The observer's keyboard response initiated the next trial. A fixation point was displayed at the center of the screen at all times. The central circular test grating was 1° of visual angle in diameter and the surrounding annular grating was 3° in diameter. Both central and surround grating had a spatial frequency of 2 cycles per degree. 

In order to obtain a psychophysical measure of subjective vertical, the orientation of the central grating was varied around the vertical based on a random, double-staircase method (Cornsweet, 1962). Subjective vertical in each condition corresponded to 50% of the clockwise (right-tilt) responses as estimated from the psychometric function, which was fit using the psignifit toolbox version 2.5.6 for Matlab (see http://bootstrap-software.org/psignifit/), implementing the maximum-likelihood method described by Wichmann and Hill (2001). For each center and surround condition, the tilt bias was defined as the subjective vertical difference between perceived orientation of the center with and without surround. Thus, the tilt bias eliminated any individual biases in orientation perception. 

Eight viewing conditions (Figure 1) were employed to investigate three factors on the tilt illusion: contrast of the center grating, relative contrast, and relative depth between the center and surround. In the conditions with depth difference, the surround appeared farther from the observer than the center, which appeared at the same depth through all conditions. Data from 16 relative center-surround orientations (ranging from 0° to 90°) were collected in each condition in order to characterize the tilt repulsion and attraction effect as a function of relative orientations. Measuring tilt bias across the entire range of relative orientations allows us to monitor the true features of repulsion and attraction, which might be missed by only recording observations from one relative orientation between the center and surround (such as 20° or 70°). 

Figure 1

 

Example stimuli from eight conditions in Experiment 1. The darkest red and blue indicate the condition without any extra segmentation cues, while the yellow and light green show the conditions with both relative contrast and disparity cues. The disparity cue is illustrated using shadows. The conditions with reddish color code all have a high-contrast (70%) center, and bluish ones have a low-contrast (10%) center. The legends only illustrate conditions with 20 deg relative orientation.

The subjective vertical under each condition for one of the 16 relative center-surround orientations was estimated in sessions of 80 trials. The sixteen estimates were fit to a natural cubic smoothing spline with seven effective degrees of freedom. Five features were extracted from the fitted curve to quantify the repulsion and attraction effects (Figure 2): maximum repulsion, relative orientation at maximum repulsion, maximum attraction, relative orientation at maximum attraction, and crossover point (where the repulsion switches to the attraction effect). In Figure 3A, we report the mean of these features in six observers for eight conditions and the standard errors (SE) of these means. Analysis of variance (ANOVA) was used to compare among these group means (Figure 3B through D). Those sixteen estimates for each condition were also fit by a modified Schwartz model (Figure 4) to explore the relationship between the tilt effects and the segmentation features in different conditions. 

Figure 2

 

Example of a smoothing spline fit to data from one observer in one of the experimental conditions. The x-axis indicates the relative orientation between the center and surround. Sixteen relative orientations were sampled for each condition. The y-axis indicates the tilt bias from the vertical. Positive biases indicate the repulsion effect, and negative biases indicate the attraction effect. Squares and error bars show tilt biases estimated based on individual psychometric functions. The solid black line shows the fitted curve. The red points represent features extracted from the fitted curve. In this example, the maximum repulsion is 1.88° when the center-surround relative orientation is 26.9°, the maximum attraction is 0.260° when the center-surround orientation is 72.4°, and at a relative orientation of 59.5°, the repulsion switches to the attraction effect (cross-over point).

Figure 3

 

Results from Experiment 1. (A) Points are peaks (maximum repulsions) and valleys (maximum attractions) of the smoothing spline curves from various conditions. Error bars are ±1 SE of maximum bias (vertical) or its corresponding relative orientation (horizontal). (B–D) Results plot based on factors. The bluish points represent conditions with low central contrast, and the reddish points represent conditions with high central contrast. Solid lines connect conditions with no stereo disparity cue, while dashed lines connect conditions with disparity cue. The x-axis shows center-surround relative contrast cues, either the same or different as shown. (B) The maximum repulsions from eight conditions. (C) The maximum attractions (the absolute values). (D) The cross-over orientations extracted from the smoothing spline curves.

Figure 4

 

Average tilt biases (“x” labels) from eight observers and the least squares fit (solid lines) of the computational model from eight conditions. When finding the best fit of this model to our data, we fixed four parameters based on the stimuli in each condition, and allowed eight free parameters to fit the 128 data points. When applied to the date with a least squares fit, the model explains 84% of the variance in the data.

Results

Figure 3A shows average repulsion and attraction peaks from eight experimental conditions. The points on the left show repulsion features, and the points on the right show attraction. Conditions with no contrast or depth segmentation cues tend to have stronger repulsion and attraction effects (dark red and blue points). When contrast of the center grating is high (reddish points), tilt biases (the repulsion and attraction) were reduced by either contrast cues, depth cues, or both. The attraction effect almost vanishes in the high-contrast center condition with both depth and contrast cues (yellow points). Conditions with a low-contrast center grating (bluish points) show great variation in terms of the attraction effect. The conditions with the low-contrast center and high-contrast surround (light green and blue points) show stronger attraction, even when presented with contrast or depth segmentation cues. 

A mixed-effects model was used in an ANOVA to clarify fixed effects of central contrast, center-surround relative contrast and depth, and their interactions, while considering subject as a random effect. This test was performed for the five extracted features separately (see Appendix, Table A1). The factors of contrast, F(1, 33) = 58.7, p < 0.001, and depth, F(1, 33) = 21.3, p < 0.001, are significant in manipulating the maximum repulsion (Figure 3B): Both segmentation cues reduce the repulsion effect, whereas perceptual grouping cues increase the effect. For maximum attraction (Figure 3C), there is a strong interaction between the central contrast and the relative contrast cue, F(1, 33) = 22.1, p < 0.001. The conditions with a low-contrast center but high-contrast surround (with relative contrast cue) show much stronger attraction. With regard to the crossover points (Figure 3D), when the contrast of the center grating is high but surround contrast is low, the range of repulsion is much greater, F(1, 33) = 23.2, p < 0.001. This is consistent with a much weaker attraction effect in these conditions. In addition, the presence of depth cues significantly enlarges the range of repulsion, F(1, 33) = 7.84, p = 0.008. 

Model

The results described above suggest that contextual cues, such as relative contrast and depth, can affect the perception of tilt. In order to better understand the psychophysics of the perceived orientation changes in our results, we used a computational model to relate the role of context in perceived orientation to neural activity. Specifically, a computational model proposed by Schwartz et al. (2009) treats relative orientation as a cue to probabilistically co-assign the center and surround in the gain pool, and then a divisive gain control process combined with this co-assignment probability could well explain the tilt repulsion and attraction. Here, we considered whether this model could be expanded to include additional segmentation cues and predict our results. 

The Schwartz model has two main components to describe center/surround interaction: divisive normalization and segmentation. Divisive normalization can serve to reduce redundant information, for example, the orientation dependence between the center and surround in natural scenes (Schwartz & Simoncelli, 2001; Simoncelli & Olshausen, 2001; Simoncelli & Schwartz, 1999; Valerio & Navarro, 2003). However, increased evidence for segmentation (e.g., large relative orientation) would decouple the coordination between the center and surround. An adaptive response to an increase in evidence for segmentation therefore would reduce the influence of the gain control pool on the central filter activation. From natural image statistics, we demonstrated that the relative contrast and depth cues maintain similar segmentation effects as relative orientation cues: With greater contrast or depth differences, the center and surround are less likely to belong to the same segments in natural scenes (see Appendix, A1). In addition, contrast of a center grating could also affect segmentation: It may be easier to distinguish features of a high-contrast stimulus as opposed to a low-contrast stimulus from background. In the condition of low-contrast center and high-contrast surround, the surround stimulus would set stronger influence on the center, even when their orientations are quite different. This may lead to more co-assignment of center and surround units. However, in the opposite condition of high-contrast center and low-contrast surround, the surround is less likely to be included in the same gain pool as the center because the segmentation here is even clearer, and surround stimulus has a weaker influence. These effects could also be expected to arise through inference or learning in natural scenes (Coen-Cagli, Dayan, & Schwartz, 2009; Coen-Cagli et al., 2012). To summarize, in our modification of the Schwartz model, the probability of including a surround stimulus within the gain pool of the central detector (co-assignment probability) depends on segmentation cues: relative orientation, contrast and depth between the center and surround, and contrast of center gratings (see Equation A2). 

We allowed eight free parameters to find the best possible fit of this adapted model to our data. Five were used to control divisive normalization, and three to determine co-assignment probabilities: described in detail in Appendix Experiment 1: Model. The model fit shown in Figure 4, explains 84% of variance in the data. We also compared several nested models, none of which performed as well as the complete model. For example, when we assumed no contribution of the segmentation cues in the co-assignment probability, the model was inferior, only explaining about 66% variance in the data. This indicates importance of relative contrast and depth in deciding co-assignment probability in the model. In all, the amount of contribution that divisive normalization by itself introduced into the central orientation perception would be insufficient to completely explain the data. 

Experiment 2: Two-dimensional/three-dimensional occluding ring between the center and surround

The segmentation cues in Experiment 1 changed both filtering activations from the center/surround stimuli and the segmentation factor between them, and these manipulations resulted in a reduced tilt repulsion effect. In Experiment 2 we sought to only manipulate the segmentation factor, but leave the initial filtering activation part unchanged, in order to directly examine the effect of segmentation on the tilt repulsion. When relative orientations between the center and surround are small, an annulus covering the boundary between the center and surround may introduce either perceptual segmentation or grouping between the center and surround while maintaining the initial filtering activation. When the annulus is in the same depth plane as the center and surround (two-dimensional [2-D]), it encourages a perceptual interpretation of independence between center and surround, whereas when the annulus is in front of the center and surround in a 3-D) space, it is more likely to encourage grouping of the center and surround as a common surface. Perceptual grouping through amodal completion has been shown to have effects on perceived transparency (Nakayama, Shimojo, & Ramachandran, 1990) and lightness (Boyaci, Fang, Murray, & Kersten, 2010). We expected a perceptual grouping cue to enhance coordination between the center and surround, thereby increasing the repulsion effect as in Experiment 1. In Experiment 2 we tested whether this perceptual grouping behind a 3-D occluding ring can affect central orientation perception. 

Method

Stimuli were as described in Experiment 1 with the center and surround contrast both at 70% and the relative orientations between them at 20° or −20°. In Experiment 2 we introduced a 0.2° annulus between the center and surround (see Figure 5). This annulus was centered on and covered the boundary between the center and surround patch. It was the same luminance as the background and either in the same plane as the center and surround (2-D ring) or in front of the center-surround plane in space (3-D ring). A stereoscope was used in both conditions. Stimulus duration was 200 ms. The boundaries of the annulus and a fixation point were always presented to help maintain fixation. Eight observers (mean age: 27, five males) participated in both 2-D and 3-D ring conditions, and each of their subjective verticals was measured in eight psi adaptive staircase (Kontsevich & Tyler, 1999) runs, in which four runs were with +20° relative orientation and another four with −20° (40 trials for each run). When the center-surround relative orientation was 20°, the tilt repulsion would be counterclockwise, whereas for −20°, the repulsion would be clockwise. The magnitude of the tilt repulsion bias in 2-D/3-D ring condition was taken as half the difference between subjective verticals for the 20° and −20° relative orientation runs to eliminate individual vertical biases. 

Figure 5

 

Effect of a 2-D or 3-D occluding ring on the tilt repulsion. The stimuli used in this experiment are shown below the x-axis: 2-D and 3-D occluding ring (see the stereo image pairs: left pair for crossed fusion, right one for uncrossed fusion). The y-axis shows the tilt repulsion biases, which are the means of results in 20° and −20° conditions. The gray points show average of eight observers, and error bars show ±1 SE. Data for individual observers are shown with smaller icons. **p < 0.01 and ***p < 0.001.

Results

We first fit a linear mixed model with the four experimental conditions (2-D ring with ±20° and 3-D ring with ±20° relative orientation) as fixed effects, and with different subjects as a random effect. We assigned each condition its own mean in the model and built contrasts of these means to test whether repulsion in the 3-D ring condition was stronger than in the 2-D condition. Figure 5 shows individual and average results in the 2-D/3-D ring conditions (for simplicity we only present the means of results in 20° and −20° conditions). Both 2-D (z = 12.2, p < 0.001) and 3-D (z = 16.7, p < 0.001) conditions showed a significant tilt repulsion effect. As expected, stronger tilt repulsion effects occurred in the condition with 3-D ring than with 2-D ring (z = 3.14, p = 0.002). 

Experiment 3: Spatial layout of the surround

In the previous two experiments we observed greater tilt biases in conditions with perceptual grouping cues. The spatial layout of surround patches relative to the center could also be an important factor of perceptual grouping. When surround patches are located collinearly with the central grating, it provides stronger evidence to co-assign the surround and center than when the surround patches flanking the center (Geisler et al., 2001; Sigman et al., 2001). Hence, a stronger repulsion effect was predicted when surround patches were presented at end locations than at flanking locations due to a stronger grouping cue in the former condition. In Experiment 3, we assessed the spatial layout of contextual effects on the perceived central orientation by using stimuli composed of three circular patches: Two surround patches were located along different directions to the central grating patch. 

Method

Six different layouts, shown in Figure 6, were tested: two surround patches positioned vertically (A, end position along the central orientation) or horizontally (B, flanker position), four surround patches as the sum of previous two conditions (C), two surround patches with orientation axes aligned either parallel (E) or perpendicular (F) to the local orientation of the surround, and the sum of the pairs of oblique patches (D). The central patch was 1° in diameter as before, and the diameter of the surround patches, which were adjacent to the center, was 1.5°. The contrast of center and surround patches was 70%. The boundaries of these patches were slightly blurred using a Gaussian lowpass filter with standard deviation 0.08°. In the main experiment (condition A to F), peripheral patches contained 2 cpd gratings; in the control experiment (condition A′ to F′), the patch layouts were maintained, but noise with the same spatial frequency was presented instead in order to measure how global orientation of the surround could influence the center perception (Morgan & Baldassi, 1997; Morgan, Mason, & Baldassi, 2000). Noise patches were generated by filtering white noise in the frequency domain with a Gaussian distribution that was isotropic in orientation, centered about the same spatial frequency as the central grating (2 cpd) with a bandwidth of 0.75 octaves. Stimulus duration was 200 ms. Five observers participated in both main and control experiments. Two additional observers only participated in the main experiment, and two others in the control experiment only. In the main experiment, the relative orientation between the center and surround gratings was 20° or −20° as in Experiment 2. The repulsion bias for individual observers in each condition was measured using eight adaptive staircase runs with either 20° or −20° (four runs for each) relative orientations between the center and surround. The 20° and −20° conditions should both induce repulsion, and running under these two conditions was to eliminate the individual-dependent subjective vertical offset. The magnitude of the bias plotted in Figure 6 is the mean of subjective vertical for the 20° and −20° surround orientations. In the control experiments, all patch layouts matched those in the main experiments. However, since there is no orientation information in the surround patches in the control conditions, we characterized bias as leftward (counterclockwise) rather than repulsive or attractive. The y-axis of Figure 6 therefore plots the magnitude of the bias associated with sample stimuli along the x-axis rather than the signed repulsion and attraction shown for previous experiments. 

Figure 6

 

Dependence of tilt illusion on spatial layout around the center. Left (A–F): example stimuli and results from the main experiment; right (A′–F′): example stimuli and results from the control experiment. The magnitude of leftward bias plotted along the y-axis is the mean of subjective vertical for 20° and −20° relative orientations. Example stimuli are shown along the x-axis. Biases are significantly different from 0 in all conditions except A′, B′, C′, and D′ (***p < 0.001). In condition E′ and F′, the positions of noise patches provide global orientation cues in the surround and induce the tilt bias in the center.

Results

As in Experiment 2, we first fit a linear mixed-effect model with each condition having its own mean in the model as a fixed effect, plus a random effect from the subjects. Then we built contrasts of these means to test differences of interest among conditions. The central perception of tilt was significantly biased by peripheral patches with gratings in all conditions (Figure 6). For the noise patches, only conditions with oblique patches (E′, F′) showed tilt biases. The positions of surround noise patches provide global orientation information and induce repulsion and attraction effects on the central gratings in condition E′ and F′, respectively. We also noticed that, in this case, the magnitude of attraction to the surround global orientation in condition F′ was stronger than the magnitude of repulsion in E′ (z = 8.05, p < 0.001). However, by adding these oblique surround patches together (as in condition D′), the global orientation information was disrupted, causing the perceptual bias to vanish. The surround area is another important factor that contributes to the magnitude of the contextual modulation, as suggested by Petrov and McKee (2006). This is also true on the central perception of tilt. After taking account the effects of global orientation measured by the control conditions (e.g., A-A′), the four-patch surround caused stronger tilt repulsion bias than the two-patch surround (e.g., C-C′ > A-A′, with z = 3.70, p < 0.001), and the net effects of two two-patch conditions were not significantly different from the condition with four patches (e.g., C ≈ A + B, with z = 1.34, p = 0.180). 

The original question in this experiment was to assess the spatial layout of the surround induced orientation bias. We used noise patches in the control experiment (same location of surround patches as in the main experiment, but presented with band-pass noise instead of gratings) to discount the influence of global orientation from patch positions in the main experiment. After subtracting the control effects (see Figure 7A), multiple comparison was performed among conditions A-A′, B-B′, E-E′, and F-F′. Figure 7B shows estimated biases in these conditions based on measurements from the nine observers: the biases are not equal across four conditions. In particular, the bias in the condition with horizontally presented patches (B-B′) is very weak compared to the others. Condition E-E′ shows an unexpectedly large repulsion bias, which may result from nonlinear effects that cannot be fully discounted by the control, or, since the SE in this condition is large, it may not bias center orientation perception in a consistent manner. Only the difference between condition A-A′ and B-B′, that is, end positions versus flanking positions along the central orientation, is significant: The adjusted p value is 0.0344 (with Bonferroni correction). Surround patches located along the end positions of a central grating induce a stronger repulsion bias than those on the flanking locations after the global orientation of patch location is accounted for. 

Figure 7

 

Tilt biases induced by different spatial layouts of surround patches. (A) Tilt biases in the main (red) and control (blue) experiments. Example stimuli are shown by the icons, the same as the condition A(′), B(′), E(′), and F(′) in Figure 6. The biases are plotted in polar coordinates. Polar angle indicates global orientation of these patches. Radius shows the strength of leftward bias. Radius of the gray circle shows the bias with a whole annulus surround grating estimated from results in Experiment 1 (1.85°). (B) The tilt repulsion biases from the same four conditions after subtracting the control effects (condition X-X′). Black dots show average results from four conditions we actually tested, and gray dots are projected for visualization. The biases are not equal across all four conditions. The gray curve is the best-fit ellipse (a = 0.980, b = 0.650) of these tilt biases across spatial layouts of surround patches. When surround patches were placed along the end positions of the central grating (A-A′), the bias differs significantly (p = 0.0344, corrected) from the condition with the horizontally presented surround patches (B-B′). Error bars show ±1 SE.

Discussion

In order to understand how scene segmentation cues affect the tilt illusion, we performed three experiments in which depth, contrast, or surround geometry were manipulated. We first measured the effect of two sources of segmentation information, center-surround relative contrast, and stereo disparity on the strength of the tilt illusion in human observers (Experiment 1). Both segmentation cues perceptually decouple the center and surround and reduce the tilt effect. Our results on the relative depth are consistent with Durant and Clifford (2006) but not Sakai and Hirai (2002) or Westheimer (1990). Sakai et al. used two bars forming an x-shape in their psychophysics and showed that the tilt effect was almost the same regardless of variations in stereo disparity between the target and contextual bars. Westheimer also used short lines. It is possible that the depth effect here is induced by the difference in surface assignment, not stereo disparity per se (Huang, Chen, & Tyler, 2012). Gratings used in our study and Durant and Clifford's study provide surface segmentation information, while stimuli from Sakai et al. and Westheimer rely more on local stereo disparity. 

We also observed that the conditions with a low-contrast center but high-contrast surround show much stronger attraction. The low contrast reduces visibility of the center, which may require increasingly large amount of information from the surround in order to get the central orientation (Mareschal, Morgan, & Solomon, 2010), which potentially increases assimilation of central features to the surround (i.e., attraction). Or in this scenario, surround effects are relatively stronger when the center is weakly driven (Carandini, 2004; Cavanaugh, 2000; Coen-Cagli et al., 2012), which may lead to more co-assignment of surround units to the gain pool even when the center and surround orientations are quite different, thus causing stronger attraction. However, in the case of high contrast in the center and low contrast in the surround, the surround is less likely to be grouped and the center is easier to be perceived, thus the orientation biases could be reduced. To summarize, the contrast of center grating also matters in the perceived central orientation. 

In Experiment 1 we adapted the Schwartz model (Schwartz et al., 2009), which combines both divisive normalization and segmentation factors to fit the psychophysical results from our eight experimental conditions. A key feature of the model is its consideration of perceptual segmentation cues that determine the co-assignment probability of surround stimuli within the gain pool of a central detector. Cues such as center-surround relative orientation, contrast, and stereo disparity influence this co-assignment probability, which is crucial in explaining the data we have. For example, stronger input in the surround than in the center (e.g., the condition with high-contrast surround but low-contrast center) can direct the tilt effect toward the attraction, including the repulsion decrease and the attraction increase. These results cannot be predicted well by a traditional divisive gain control model. 

As shown in Figure 8, a surround grating with greater contrast (light blue) induces a stronger effect on the gain pool than the condition with the same low-contrast center and surround (dark blue), which successfully predicts more reduction of the overall population response in the center of the former condition (Carandini, 2004; Carandini & Heeger, 2012; Cavanaugh, 2000). The stronger gain effect could also push the population codes of the perceived orientation farther away from the real center orientation (a stronger repulsion shown in Figure 8). However, this is inconsistent with our observation that the condition with a higher contrast surround shows much weaker repulsion than the condition with low-contrast center and surround (see the dark blue and light blue dots in Figure 3). Introducing the segmentation factor can better account for this effect: When the center and surround orientations are similar, contrast difference between the center and surround decreases the co-assignment probability, makes the visual system less likely to assign the center and surround into the same gain pool, and reduces the repulsive bias. We separately manipulated this segmentation factor in Experiment 2 by keeping the same center and surround contrast and orientation but changing the co-assignment probabilities, and we did see variations of tilt associated with this factor. 

Figure 8

 

Predicted perceptual tilt biases from a divisive gain control model (McDonald, Seymour, Schira, Spehar, & Clifford, 2009; Solomon, Felisberti, & Morgan, 2004). The gray dashed line shows hypothetical population neural response to a vertical central grating without any contextual stimulus. The bluish dashed lines show, when the surround grating is oriented 20° relative to the center, its effect on the gain of the neurons responding to the center. The dark blue represents the condition with a low-contrast center and low-contrast surround, whereas the light blue shows the condition with the same center but high-contrast surround. The solid lines are predicted neural population responses to the central gratings surrounded by 20° grating with different contrast as shown in icons. The predictions are calculated by multiplying the response to the center-only condition (gray dashed line) by the bluish dashed lines. The condition with higher surround contrast elicits stronger tilt repulsion (farther away from the vertical), which is inconsistent with our results from Experiment 1.

In Experiment 2, a 2-D occluding ring (as a gap) spatially separates the center and surround and reduces the tilt effect, which is also shown in Clifford, Spehar, Solomon, Martin, and Zaidi (2003), Durant and Clifford (2006), and Wenderoth & Johnstone (1988), while a 3-D occluding ring encourages center and surround to be grouped as the same surface, and thus increases the co-assignment probability, resulting in stronger repulsion. Functionally, with extra perceptual grouping cues, the visual system may tend to increase the importance of inferring a “hidden” orientation-texture mismatch, and it leads to a stronger bias in this case, whereas a clear 2-D gap would make it unnecessary to overemphasize the discrepancy between the center and surround (Durant & Clifford, 2006). 

In Experiment 3, we observed that the tilt repulsion effect is strongest along the ends of the stimulus as defined by the axis of central orientation (collinear), which may be because the high edge co-occurrence rate along that location makes observers more likely to group those surround gratings with the center. The statistics of natural scenes suggests greater orientation dependence between collinear elements than parallel elements, which reflects the predominance of elongated boundaries in the environment (Geisler & Perry, 2009; Geisler et al., 2001; Sigman et al., 2001). To adapt to these statistics in natural scenes, observers would show bigger co-assignment probability in the collinear condition, which follows the local grouping function proposed by Geisler and colleagues (2001). This collinear grouping could also be explained by a generalized form of divisive normalization model using learned or inferred covariance matrices from natural scenes (Coen-Cagli et al., 2009, 2012), which successfully predicts a higher co-assignment probability for the collinear condition. When the center and surround are more coordinated, the system tends to exaggerate mismatches of the target from its context, which is represented as a stronger repulsion bias. This process helps to emphasize the discrepancies of actual inputs from the prior belief of the system. 

On one hand, the visual system sets a higher co-assignment probability for inputs that are more likely to be the same, and this ensures coding efficiency. This is generally consistent with Cavanaugh et al. (2002) and Li (2002): Neurons respond less to uniform stimuli and more to targets that are distinct from their context. On the other hand, with a high co-assignment probability, the system tends to exaggerate mismatches between elements, and to be more sensitive to the potential mispredictions. Using a set of similar stimuli, Mareschal, Sceniak, and Shapley (2001) found that in the collinear condition, orientation discrimination thresholds were significantly bigger than in the flanking condition. It is possible that the exaggeration of orientation bias along the end positions sacrifices the system's sensitivity to actual angles, but makes it focus more on discrepancies between the center and surround. In all, our results demonstrate that the tilt illusion is affected by spatial layouts of the surround mask, and this spatial anisotropy of the contextual effects may be related to the statistical features of edge co-occurrence relative to the center. 

However, our results in Experiment 3 are inconsistent with Kapadia, Westheimer, and Gilbert (2000), in which they presented three small line segments (each about 0.13° in length) in the fovea with viewing distance of 6 m. They observed stronger repulsion effects with lateral flankers than with collinear flankers when the relative orientation between the target and flanks was 20°. Also, a recent paper by Mareschal and Clifford (2013) reported that surrounding locations equally contributed to contextual effects, which did not show any collinear structure. Different patterns in the results may be due to different stimuli used in these experiments, which induce different segmentation between the center and surround. 

To summarize, we observed that the tilt repulsion biases increase as it becomes more difficult to perceptually separate the center and surround. Similarity or co-assignment between the center and surround stimuli increases the repulsive shift between the perceived center and surround orientations, which is apparently against our intuition that if the center and surround stimuli become more similar, we would expect our perception of them to be more similar. However, the visual system amplifies the discrepancy among environmental cues that have other evidence of common coordination. This may actually play an important role in contour detection and figure-ground segmentation. For an example of breaking camouflage, multiple sources of information (luminance, contrast, or color) may seem to say that it is only a bunch of dead leaves or uninteresting bark, but subtle clues (e.g., differently oriented boundaries) tell us that a butterfly is embedded in the background. The visual system must search for and detect camouflaged objects, while at the same time striving for efficiency. Therefore, interactions between the center and surround should not only achieve coding efficiency, but also control the importance of inferring a potential feature mismatch. This high sensitivity to feature contrast between the target and its context, especially in situations that seem to have a common source, could essentially benefit our visual search performance. 

The effect of segmentation on the tilt illusion induced by relative orientations and other sources, such as relative contrast, disparity depth, and geometric features, may have different mechanisms. If we assume that the effect of adding relative contrast is the same as increasing the relative orientation, then the effect we see at 20° relative orientation with relative contrast should be equal to the condition, say, at 30° relative orientation without relative contrast. Therefore, the tilt bias curve as a function of relative orientation should shift toward the left, when relative contrast is introduced between the center and surround. However, in Experiment 1, with relative contrast or disparity depth, the tilt bias curves tend to be right-shifted instead. This suggests that these relative cues may influence the orientation perceptual bias through different mechanisms. 

Our results agree with former work by van der Smagt et al. (2005) in which contrast and orientation segmentation cues were used for investigating the role of V1 cells in surface segregation: Though a surround of either the same orientation or the same contrast has a suppressive effect on the response to the central stimulus, the authors found that combining the two cues had no greater effect than one on its own. Another similar finding is by Clifford et al. (2003) in which a segmentation cue, color, was used: They found that the tilt repulsion was greater when the center and surround were the same color. This pattern is also true in Durant and Clifford (2006): When the center and surround are perceptually segregated by asynchronous presentation or spatial cues other than orientation, the tilt repulsion effect on the center is reduced. Just as suggested by the authors, if the mechanism underlying the tilt illusion tends to segment surfaces by emphasizing the difference in orientation, when surfaces are already segmented by other cues, the exaggerate changes of orientation are not that crucial (Durant & Clifford, 2006). On the other hand, if those cues aid perceptual grouping between the center and surround, the tendency of emphasizing the orientation difference would be enlarged. 

In order to demonstrate different mechanisms or explore the level of segmentation information processing, a backward noise masking of the surround (Clifford & Harris, 2005) or a rapid reverse-correlation method (Mareschal & Clifford, 2012) could be useful. In a recent paper by Mareschal and Clifford (2012), the authors suggested that a single mechanism operating in the early stages of visual processing (before conscious perception of the surround) could account for both the tilt repulsion and attraction. They used a reverse-correlation technique, in which the surround orientation was changed every 12 ms making it invisible to observers. They found that both the tilt repulsion and attraction occurred over a similar time course, which suggested that it may not be necessary to invoke a separate, higher-level mechanism. It will be interesting to see whether the effect of the perceptual grouping/segmentation cues used in our experiments persists when the surround orientation is not consciously perceived. This reverse correlation paradigm may help to entangle the levels of processing involved with different center-surround perceptual grouping/segmentation cues. 

Conclusion

In conclusion, our results from three experiments demonstrate that center-surround relative contrast, relative disparity depth, and geometric features, such as occlusions and colinearity, can affect the strength of perceptual orientation bias in the center. In general, a stronger perceptual grouping cue between the center and surround enhances tilt repulsion biases, whereas a segmentation cue reduces the effect. Functionally, this may increase the sensitivity of our visual system to feature discrepancies, especially in an environment rich in similarities, and this may play an important role in visual search and detection. 

Acknowledgments

This work was funded by R21 NS075525. DK was supported by the World Class University program funded by the Ministry of Education, Science, and Technology through the National Research Foundation of Korea (R31-10008), NIH grant R01 EY015261, and ONR N000141210883. The authors would also like to thank Damien Mannion, Michael-Paul Schallmo, and the reviewers for their comments on the manuscript. 

Commercial relationships: none. 

Corresponding author: Cheng Qiu. 

Email: [e-mail]. 

Address: Department of Psychology, University of Minnesota, Minneapolis, MN, USA. 

Appendix

Experiment 1: Model

Influenced by the gain control pool, the estimate of the normalized neural response associated with the central detector is:  

 

where l_ci is the filtering response of the central detector tuned to a particular orientation ϕ_ci (preferred orientation), l_ci = Display Formula · exp (−(ϕ_ci – θ_c)²/ Display Formula ), when the orientation of center stimulus is θ_c, and the contrast is C_c; l_si is the response to the surround stimuli oriented at θ_s with contrast of C_s, l_si = Display Formula · exp (−(ϕ_si – θ_s)²/ Display Formula ), where C_exp controls the contribution of contrast in filter responses, and the gain control pool for detector i is set by center and surround filter activations with the same orientation preference, the divisive term is l = Display Formula , where n describes the strength of surround influence on the gain pool (one can think n is related to surround size relative to center), and k is an additive constant; B(·) stands for a modified Bessel function of the second kind. 

Another key component in the model is the segmentation factor. Modified from the Schwartz model, the probability of including a surround stimulus with the orientation θ_s within the gain pool of the central detector (co-assignment probability) depends on all possible segmentation cues (orientation, contrast and depth):  

 

where σ = Display Formula gives the steepness of the co-assignment selection, which is determined by central contrast C_c, relative contrast (C_c − C_s), and relative depth (D_c − D_s). When there is no contrast difference between the center and surround, we have Display Formula = 1, which shows no effect on co-assignment, and in those conditions with contrast difference, the relative contrast is positively weighted by C_weight. Based on results in Experiment 1, the center contrast could also affect co-assignment. C_c, therefore, is another component in determining the steepness. Display Formula is the same as Display Formula , where a = 1 + C_weight and C_weight is a positive number. The higher the center contrast is, the shallower the slope of co-assignment probability is, and the smaller the peak co-assignment is; whereas the higher the surround contrast is, the steeper the co-assignment slope is, and the greater the peak co-assignment is. This is consistent with our expectation: When the surround contrast is constant, a higher contrast center would decrease the maximum probability of including a surround stimulus within the gain pool of the central detector, while when the center contrast is constant, a higher contrast surround would increase the maximum co-assignment probability. As for relative depth, when there is no depth difference between the center and surround, we have Display Formula = 1, which has no effect on co-assignment, and in those conditions with depth difference (in our experimental conditions assuming D_c ≥ D_s), the relative depth is positively weighted by D_weight. The greater the relative depth is, the smaller the peak co-assignment is. In other words, depth difference would decrease the co-assignment probability. 

If the surround stimulus is not taken as being part of the same gain pool as the center detector, then the detector would take into account only the center stimulus E(g_ci|l_ci). The net response is weighted by the co-assignment probability and is given by:  

 

 

Then through standard population decoding (Georgopoulos, Schwartz, & Kettner, 1986),  

 

we obtain the perceived central orientation r. 

When finding the best fit of this model to our data, we fixed C_c, C_s, D_c, and D_s based on the stimuli in each condition (C_c = 0.1 or 0.7, C_s = 0.1 or 0.7, D_c = 0 and D_s = 0 or 2), and allowed eight free parameters: n, k, center tuning width ω_c, surround tuning width ω_s, and C_exp, respectively; plus C_weight, D_weight, and λ, respectively, when calculating co-assignment probability. Average data from all eight conditions are summarized in Figure 4. Sample size is 128 (16 points in each of eight conditions). When applied with the least squares fit to the model, we obtained optimal parameters:  

 

 

Fit results are shown as solid lines in Figure 4, and it explains 84% of the variance in the data. In an attempt at parsimony, we obtained fits to several nested models. The small-sample-size corrected Akaike Information Criterion (AICc) was used to evaluate these models. A smaller AICc indicates a more efficient fitting. In one of the nested models, we forced the probabilities not associated with the relative contrast and depth, that is, set C_weight = 0 and D_weight = 0. The model was inferior (AICc = −223) to the original model (AICc = −317), and only explained about 66% variance in the data (vs. 84%), indicating importance of relative contrast and depth in deciding co-assignment probability in the model. In another nested model, the effect of center contrast on the co-assignment was eliminated, that is, the condition with high-contrast center and low-contrast surround and the condition with low-contrast center but high-contrast surround had the same co-assignment probability. This fit explained 77% variance in the data with AICc = −268. 

Figure A1

 

Statistical dependencies in term of contrast (A–D) and depth (E–H) information in natural images within (blue) and across (red) boundaries. The statistics for each condition were collected over 30,000 random samples from each of nine images. (A) Example image from the Berkeley database (Martin, Fowlkes, Tal, & Malik, 2001) including hand-labeled segmentation boundaries (as shown in B). (C) The joint statistics between center and surround patches when they belong to different segments (left, red) or the same segment (right, blue). The plots show the joint conditional statistics of the contrast in the center, given the contrast in the surround. Contrast within a given patch was measured based on Michelson contrast. The bottom left shows the count of the center contrast being 0 when the surround contrast was 0. Intensity is proportional to the counts, but each column is independently rescaled to show a conditional distribution given a certain surround contrast. The solid lines show the conditional standard deviation. Within boundaries, the center and surround patches tend to have similar contrast. This similarity is greatly reduced across boundaries, suggesting that the center and surround are less likely to be grouped. (D) Probabilities of central and surround patches within boundaries as a function of relative contrast between the center and surround. When the center and surround have similar contrast, they tend to belong to the same segment in natural scenes. (E) Example image from a stereo depth image database (Scharstein & Pal, 2007), and segmentation boundaries were calculated based on the depth information (as shown in F). (G) Joint conditional statistics of the center patch depth, given the depth information of the surround patch. Disparity depth was the mean of the depth value for all pixels in the patch. The bottom left shows the count of the center depth as 0 when the surround depth was 0. The solid lines show the conditional standard deviation. The blue represents when the center and surround patches belonging to the same segment, and the red represents patches across boundaries. Within boundaries, the center and surround patches tend to have similar stereo depth, whereas this similarity drops across boundaries, implying that the across boundary surround does not provide as much information as it provides to the center when they are belong to the same segment. (H) Probabilities of central and surround patches within boundaries as a function of relative stereo depth between the center and surround.

Table A1

 

Experiment 1 statistics results. Note: A:B indicates interactions between factors A and B.

Factor	Maximum repulsion	Relative orientation at maximum repulsion	Maximum attraction	Relative orientation at maximum attraction	Crossover orientation
Contrast	F(1, 33) = 58.7, p < 0.001	F(1, 33) = 2.05, p = 0.162	F(1, 33) = 4.94, p = 0.033	F(1, 33) = 5.86, p = 0.21	F(1, 33) = 1.56, p = 0.221
Depth	F(1, 33) = 21.3, p < 0.001	F(1, 33) = 24.4, p < 0.001	F(1, 33) = 6.21, p = 0.018	F(1, 33) = 4.42, p = 0.43	F(1, 33) = 7.94, p = 0.008
Central contrast	F(1, 33) = 6.25, p = 0.018	F(1, 33) = 0.0766, p = 0.784	F(1, 33) = 31.0, p < 0.001	F(1, 33) = 12.1, p = 0.001	F(1, 33) = 9.16, p = 0.005
Contrast: Depth	F(1, 33) = 2.49, p = 0.124	F(1, 33) = 0.603, p = 0.443	F(1, 33) = 3.99, p = 0.054	F(1, 33) = 0.322, p = 0.575	F(1, 33) = 1.43, p = 0.241
Contrast: Central contrast	F(1, 33)= 1.89, p = 0.179	F(1, 33) = 0.687, p = 0.413	F(1, 33) = 22.2, p < 0.001	F(1, 33) = 15.6, p < 0.001	F(1, 33) = 23.2, p < 0.001
Depth: Central contrast	F(1, 33) = 0.419, p = 0.522	F(1, 33) = 0.245, p = 0.624	F(1, 33) = 0.272, p = 0.605	F(1, 33) = 0.178, p = 0.676	F(1, 33) = 1.90, p = 0.177

Experiment 1: Model—Natural image statistics

In order to demonstrate that contrast and depth cues maintain similar segmentation effects as the orientation cue, we measured the joint conditional distribution of the contrast or disparity depth in the center, given the contrast or disparity in the surround. The center was defined as a 9 × 9 pixel square patch, and the surround as one of four possible edge-adjacent neighboring 9 × 9 pixel patches. When a continuous contour longer than eight pixels was detected within a 6 × 9 pixel patch centered on the boundary between the center and surround patches, the patches were classified as across boundaries, otherwise, they were said to be within boundaries. Contrast within a given patch was measured based on Michelson contrast. Disparity depth was the mean of the depth value for all pixels in the patch. Figure A1 shows that the correlations of contrast and disparity depth between the center and surround patches are reduced across boundaries from pictures in the Berkeley database (Martin, Fowlkes, Tal, & Malik, 2001) and the stereo depth database (Scharstein & Pal, 2007) respectively, suggesting that the center and surround tend to be more separated due to contrast and depth cues. Figure A1D, H further show that the probability of central and surround patches belonging to the same surface or object (within boundaries) decreases as the center-surround contrast (or depth) difference increases.