Depth perception in the framework of General Object Constancy

Abstract

Abstract

Abstract:

Abstract  Size constancy is a well-known example of perceptual stabilization accounting for the effect of viewing distance on retinal image size. In a recent study (Qian & Petrov, 2012), we demonstrated a similar stabilization mechanism for contrast perception and suggested that the brain accounts for effects of perceived distance on various other object features in a similar way, a hypothesis that we called General Object Constancy. Here we report a new illusion of depth further supporting this hypothesis. Pairs of disks moved across the screen in a pattern of radial optic flow. A pair comprised a small black disk floating in front of a large white disk, creating the percept of a pencil tip viewed head on. As these “pencils” moved away, they appeared to grow in contrast, in diameter, and also appeared to be getting “sharper.” The contrast and size illusions replicated our previous findings, while the depth gradient (sharpness) illusion revealed a depth constancy phenomenon. We discovered that depth and size constancies were related, e.g., the two illusions were strongly correlated across observers. Whereas the illusory diameter increase could not be canceled by any degree of depth modulation, decreasing the diameter of the “pencils” during optic flow motion (thus increasing their disparity gradient) weakened the illusory depth gradient increase. This paradoxical result, as well as our other results, is explained by the General Object Constancy model: Besides using the same scaling factor to account for size, contrast, and depth variations with distance, the brain uses the apparent object size to additionally scale contrast and depth signals.

Introduction

Perceptual constancy is a crucial characteristic of vision. It allows us to see objects as having consistent features, e.g., size, shape, or lightness, and to identify them, no matter how drastically the angle of perspective, distance, or lighting changes. Lightness and color constancies have been studied extensively by varying lighting conditions (Adelson, 1999; Brainard, 1998; Brainard, Brunt, & Speigle, 1997; Gilchrist, 2006; Kraft & Brainard, 1999; MacEvoy & Paradiso, 2001; Rutherford & Brainard, 2002). However, since there is also evidence that surface lightness (Gilchrist, 1977; Logvinenko & Maloney, 2006; Pereverzeva & Murray, 2009) and contrast (Aslin, Battaglia, & Jacobs, 2004; Aslin & Li, 1993) depend on depth, one would expect that perceptual constancy mechanism is also used for maintaining the perceived lightness and contrast of objects as viewing distance changes. Indeed, recent findings (Qian & Petrov, 2012) showed that an illusory modulation of an object's contrast was observed synchronously with an illusory size modulation when the object appeared to move further away in depth, a result consistent with the expected contrast constancy phenomenon. 

Besides contrast, an object's size and shape, including its profile in depth, are likewise to be perceived as invariant across changes of viewing distance. Size constancy, for instance, can be observed under stereoscopic and monocular viewing conditions, e.g., in photographs and drawings (Boring, 1964; Carlson, 1960, 1962; Gregory, 1963; Murgia & Sharkey, 2009). It is suggested that in order to keep the apparent size of an object invariant with changing viewing distances, a compensating mechanism can be used to scale the retinal image size by an estimate of distance (Boring, 1940; Epstein, 1963; Kaufman, 1974; Kaufman et al., 2006; Kilpatrick & Ittelson, 1953). 

While size constancy phenomenon has attracted a great deal of attention, relatively few studies on perception of object's depth profile can be found. Object's depth profile, as encoded by various depth cues on the retina, changes with viewing distance. In particular, this change in object's depth profile may result from binocular disparity changing approximately as the inverse of the square of viewing distance (Foley, 1980, 1987; Richards, 1985; Wallach & Zuckerman, 1963). However, we do not see object's depth profile increasing or decreasing as viewing distance varies. The perception of an object having the same amount of depth regardless of viewing distance is called stereoscopic depth constancy (Wallach & Zuckerman, 1963). To calculate the true depth profile of an object based on its disparity, the brain must estimate the viewing distance and scale the disparity accordingly (Glennerster, Rogers, & Bradshaw, 1996; Ono & Comerford, 1977). Even if the depth profile is determined up to a constant affine transformation (Petrov & Glennerster, 2004, 2006), the affine profile needs to be corrected according to the viewing distance, i.e., some mechanism based on the estimated viewing distance is required to produce the perception of depth profile invariant to viewing distance. 

In this study, we investigated the purported depth constancy mechanism in the context of the General Object Constancy hypothesis which was proposed to explain a strong covariance of size and contrast illusions induced by radial optic flow in our previous study (the StarTrek illusion, Qian & Petrov, 2012). The hypothesis posits that the brain uses the same factor to scale, as a function of viewing distance, various retinal metrics, such as size, contrast, disparity, color, etc. Here we report yet another illusion induced by the optic flow stimulus, the illusion of depth profile. A strong covariance between the depth profile and size illusions found in the current study provides additional support to the General Object Constancy hypothesis. 

Methods

Stimuli

Most studies on depth constancy manipulated viewing distance physically, so depth cues like convergence, accommodation, texture gradients, familiar size (e.g., monitors), or others could be used by the visual system to adjust the depth percept in accordance with viewing distance. Here, we used radial optic flow as a depth cue to evoke perceived changes in viewing distance; depth constancy occurs even if there is no physical change in viewing distance. The optic flow stimulus was displayed on a gray background and viewed through a Wheatstone stereoscope on a pair of linearized 21” ViewSonic G225f monitors. The display resolution was set to 1600 × 1200 pixels; and for the typical viewing distance of 110 cm, a pixel subtended 1 arcmin. 

The target was a set of high-contrast, randomly located pairs of disks moving in a pattern of radial optic flow on a gray background. Peripheral random pairs of disks on a gray background formed a static stencil mask. The mask had a 10° circular aperture positioned in the center of the screen, through which the moving disks could be seen (Figure 1a). Their motion created an optic flow consistent with the disks being positioned on a fronto-parallel plane moving back and forth with constant speed, that is, in a triangle-wave fashion. The amplitude of the optic flow motion corresponded to the disks moving away to twice the viewing distance. As the disks moved inward, new disks filled in along the boundary of the aperture from behind the occluding stencil mask and moved consistently with the pattern of optic flow. From the point of view of the observer, the density of the disks was increasing as they moved away. We refer to this motion phase as “stimulus contraction” and refer to the motion phase when disks moved toward the observer as “stimulus expansion”; the same convention was used in our previous study (Qian & Petrov, 2012). A disk pair comprised a small .05° black disk overlaid on a larger .15° white disk. Binocular disparity was added between the paired disks to create a 3D percept of the black disk placed in front of the white disk. The “pencil” interpretation of the stimulus was suggested to observers: Each pair represented a pencil viewed head-on (Figure 1b), where the black disk was the lead and the white disk was the wooden shell. Thus, we refer to it as the “pencil” stimulus. 100 pairs of disks were displayed during each trial, which lasted for 2 s and included one contraction-expansion motion cycle of the optic flow. Observers carried out 300 trials for each condition. 

Figure 1

 

Experimental stimuli. (Left panel) small black disks floating in front of large white disks, interpreted as pencil tips viewed head-on. Pairs of disks moved across the screen in a pattern of radial optic flow. Dashed circle (not shown in the experiments) indicates the central aperture through which the moving disks were observed; (Right panel) illustration of the size, contrast and depth gradient illusions during the contraction phase. The size and contrast illusions are illustrated in the top panel (Actual View); the depth gradient illusion is illustrated in the bottom panel (Perceived Object) as seen from the side.

Subjects

Twenty-one observers with normal or corrected visual acuity were tested. Eighteen of the observers were naive to the purpose of the study; only three were experienced psychophysical observers. Observers were trained for a short time (2–5 min) to get acquainted with the stimuli and the task. 

Psychometric procedure

Observers indicated whether the depth profile of a pencil was changing in the course of the optic flow by clicking the left and right mouse buttons. They were instructed to press the right mouse button if they perceived the “pencil” getting sharper during the contraction phase and the left mouse button otherwise. The depth illusion was measured with a nulling paradigm, where the relative disparity for each pair varied in such a way as to stabilize the depth profile in the course of the optic flow. The following formula describes the applied disparity modulation:  

 

where d₀ stands for the actual viewing distance, Δd = d – d₀ stands for the modulation of the apparent viewing distance d, as simulated by the optic flow, and δ(d₀) stands for the relative disparity between the pairs of disks for d = d₀. As in our previous study, the nulling amplitude of the disparity modulation, A, was calculated by a modified version of the Bayesian adaptive algorithm, devised by Kontsevich and Tyler (1999). Note that because A was always positive, disparity always decreased as the simulated distance d increased. The illusion strength was measured as the percent change of δ necessary to null the illusion for the maximum distance, d = 2d₀. Uncertainties for the illusion strength were given by the standard deviation of its estimate calculated by the adaptive algorithm. 

Results

Experiment 1: Illusion of the depth gradient

In Experiment 1, we tested the “pencil” stimulus shown in Figure 1. Four observes participated in this experiment. They judged whether the perceived sharpness of the pencil increased during the contraction phase. All of them reported that the pencils appeared to grow larger in diameter and also sharper (the solid angle subtended by the pencil tip decreased) during the contraction phase and that the reverse happened during the expansion phase (see Figure 1b). The perceived change in the pencil's diameter replicated our previous findings for the size illusion induced by optic flow. The perceived sharpness increase revealed a new illusion of depth. Importantly, the illusion of depth had to be stronger than the size illusion, because the rate of the illusory depth change had to exceed the rate of the illusory size change in order for the perceived sharpness to increase. In other words, if both illusions were of the same magnitude, the perceived pencil sharpness would have been constant; only the pencil's overall scale would have varied. We termed this illusion the depth gradient illusion as the pencil's sharpness represents the depth gradient. 

Figure 2a shows the disparity decrease required to null the illusory sharpness increase of the pencil for the four observers. Figure 2b shows the observers' average for the depth gradient illusion (blue bar), the contrast illusion (green bar), and the size illusion (orange bar). The values for the latter two illusions were taken from our previous study (Qian & Petrov, 2012). The depth gradient illusion was the strongest: On average, it was about 43%, compared to 30% for the contrast illusion and 15% for the size illusion, F(2, 14) = 23.28, p < 0.005. Taking into account that the depth illusion and the size illusion were opposing each other in the depth gradient illusion, one might conclude that the depth illusion per se is much stronger than the size illusion. This speculation is in accord with the fact that disparity falls off approximately as the square of viewing distance (Wallach & Zuckerman, 1963), whereas size falls off as a linear function of viewing distance. 

Figure 2

 

The illusory increase in depth gradient. (Left panel) The dark blue bars show the disparity decrease required to null the illusory sharpness increase for individual observers. (Left panel) Comparison between the average nulling amplitudes for three types of illusion. The blue bar indicates the average illusory sharpness increase, which was 43%. The green bar and the orange bar show the average illusory contrast and size increase from the previous study (Qian & Petrov, 2012).

Experiment 2: Effect of an object's size on the depth gradient illusion

Since the size illusion is a confounding factor in the depth gradient illusion, we wanted to measure the magnitude of the depth gradient illusion without the illusory size change, i.e., keeping the perceived size constant during the optic flow. To this end, we first measured the size illusion for each observer using the “pencil” stimulus. The relative disparity between the white and black disks was set to 0 to avoid any possible distraction from the depth percept of the “pencil.” In this case, the white and black disks in a pair appeared to be flat concentric circles. The size-nulling paradigm was the same as in our previous study, given by formula (2) in Qian & Petrov (2012). To measure the depth gradient illusion while keeping the perceived size constant, the relative disparity between the disks was resumed, and the size illusion was canceled by modulating the diameter of the black and white disks given the size-nulling values. The task remained the same as in Experiment 1; the same four observers were asked to judge the sharpness change. 

The results are shown in Figure 3 with black dots. There was a strong correlation between the illusory size increase and the illusory sharpness increase. Those observers who perceived a strong size illusion also perceived a strong depth gradient illusion, and vice versa. This was similar to our previous study, in which we observed a strong correlation between the size and contrast illusions across observers. On average, the relative decrease of the disk size required to null the size illusion was about half of the disparity decrease required to null the depth gradient illusion. Data from Experiment 1 is shown Figure 3 with red dots for comparison. Strikingly, for all observers, the depth gradient illusion was weaker in Experiment 2 than in Experiment 1. This result is counterintuitive because in Experiment 2, unlike in Experiment 1, the diameter of the pencils was decreasing in the contraction phase, which, in turn, was increasing their disparity gradient. Hence, a stronger depth gradient illusion would be expected in Experiment 2. This result can be explained by the General Object Constancy model, as will be discussed later. 

Figure 3

 

Comparison between the illusory size (x-axis) and depth gradient (y-axis) change of the pencil, measured by adaptively varying the disk size and the relative disparity between the disks respectively. Each datum represents a different observer. Data from Experiment 2 are shown in black and data from Experiment 1 are shown in red. The black and red curves show parameter-free predictions of the General Object Constancy model, y + 1 = (x + 1)² and y + 1 = (x + 1)³ respectively (see the Appendix).

Experiment 3: Effect of disparity nulling on the size illusion

Experiment 2 showed that size perception affects the magnitude of the depth gradient illusion in a paradoxical fashion. In this experiment, we wanted to test whether, conversely, disparity manipulation affects the size illusion. For this purpose, we used the same “pencil” stimulus, but instead of judging the sharpness, the observers were asked to judge whether the diameter of the “pencils” increased or decreased during the contraction phase. As in Experiment 1, disparity between the white and black disks was modulated adaptively, but now in an attempt to null the size illusion. For seven out of eight observers tested in this experiment, the size illusion could not be nulled by disparity manipulations no matter how large the changes were. For the remaining observer, the disparity manipulation did null the size illusion, but the required disparity change was quite high, 49%. 

Experiment 4: Adding global motion in depth

In all the experiments described thus far we used optic flow to create the percept of viewing distance change. Normally, such optic flow would be accompanied by the corresponding global disparity change. Therefore, in this experiment we tested whether the depth gradient illusion can be made stronger by adding such global disparity modulation to all disks, consistent with the disks moving back and forth in depth. In our previous study (Qian & Petrov, 2012), the same manipulation applied to the size and the contrast illusion made no difference, and one might expect that the observation would also hold true for the depth gradient illusion. The same “pencil” stimuli were used, except for the addition of the global disparity modulation consistent with the optic flow. Eleven observers participated; the average strength of the illusion was about 38%. Hence, adding global disparity did not affect the strength of the depth gradient illusion significantly, t(13) = 1.29, p > 0.1 (Figure 4). This result indicates that optic flow alone is a strong enough depth cue to render the additional global binocular disparity cue insignificance. 

Figure 4

 

The illusory increase in depth gradient with added global disparity modulation. (Left panel) Blue bars show the disparity decrease required to null the illusory sharpness increase for each observer. (Right panel) Comparison between the average strength of the illusion with and without global disparity, shown by the pink and blue bars respectively.

Discussion

The StarTrek illusion (Qian & Petrov, 2012) was used to explore the phenomenon of depth constancy in the current study. Using the “pencil” stimulus, we demonstrated a new illusion of the depth gradient, where the depth gradient was perceived to vary during the optic flow. This was an even stronger illusion, 43% illusory variation on average, compared to the contrast illusion, 30%, and the size illusion, 15%, reported in our previous study (Qian & Petrov, 2012). Experiment 2 showed that the strengths of the depth and size illusions were correlated across observers and revealed a paradoxical effect of perceived size on the depth gradient illusion, wherein smaller sizes corresponding to larger disparity gradients produced weaker depth gradient percepts. No such effect was observed in the opposite direction, from depth to size, in Experiment 3. Experiment 4 showed that adding binocular disparity that varied in accordance with optic flow motion did not enhance the illusion. The outcome is consistent with the results of a similar manipulation in our previous study, and several other studies (O'leary & Wallach, 1980; Wallach & Zuckerman, 1963) which used linear perspective cues instead of optic flow. 

The depth illusion we observed may result from a depth constancy mechanism implemented in the brain. Under normal viewing conditions, when viewing distance increases, an object's depth profile (encoded by the binocular disparity) decreases. Nevertheless we do not see the object's depth profile getting flatter: It is relatively invariant to viewing distance change. We hypothesize that depth constancy may be implemented similarly to size constancy via scaling the binocular disparity by a function of viewing distance. Given that our optic flow stimulus created a strong percept of viewing distance change, this scaling transformation was applied to the (constant) disparity signal in the stimulus. As a result, the depth illusion was observed, such that the perceived depth gradient increased in the contraction phase and decreased in the expansion phase. 

In Experiment 1, in which the size illusion was observed along with the depth illusion, the illusory depth gradient increase was significantly stronger than in Experiment 2, in which the size illusion was cancelled. At the first glance, the result appears paradoxical, because increasing size (pencil's diameter) decreases the depth gradient, and, hence, should weaken the depth gradient illusion. In our previous study, we investigated contrast and size illusions in the same optic flow paradigm. In particular, we discovered that in order to explain the contrast illusion, another scaling factor, in addition to viewing distance, was required. This factor was proportional to the perceived size change in the course of optic flow and significantly increased the contrast illusion compared to the size illusion. Although counterintuitive, the paradoxical effect of the object's size on its perceived depth profile revealed by Experiments 1 and 2 is explained by the same size factor scaling the perceived depth gradient (see the Appendix for more details). 

In order to account for the depth gradient illusion, we supplemented the General Object Constancy model proposed in our previous study (Qian & Petrov, 2012) with depth as a new feature (Figure 5). The model posits that the brain uses the same scaling factor for size, contrast, and depth profile, i.e., it scales retinal size, retinal contrast, and retinal disparity by a factor k(d), which is a function of viewing distance d. Because, unlike size, disparity decreases as the square of d, the factor k is squared in this case to ensure a constant depth percept. This factor alone makes the depth illusion much stronger than the size illusion. In addition, the change in perceived size contributes another factor k′, which scales both the perceived contrast and depth profile (k′ is squared in the latter case). Because of k′, the depth illusion is significantly stronger in Experiment 1, in which the perceived size increased during the contraction phase, than in Experiment 2, in which the perceived size was constant. The model provides parameter-free predictions to the results of Experiments 1 and 2 shown with the red and black curves in Figure 3, which were given by y + 1 = (x + 1)³ and y + 1 = (x + 1)² relationships respectively. Mathematical details are discussed in the Appendix. The model also explains the results of Experiment 3, since, analogous to the contrast illusion in our previous study, the perceived depth does not factor into the perceived size calculation. 

Figure 5

 

General Object Constancy mechanism. The brain scales disparity, retinal size and retinal contrast by a factor k as a function of distance. Additionally, the perceived size change contributes another factor, k′, to the perceived contrast and the perceived depth. The contributions of both factors to depth perception are squared to ensure depth constancy.

Our model suggests that perceived size, depth, and contrast all depend on an estimate of viewing distance. There are neurophysiological evidences showing that size perception is modulated by both feedforward signals originating from retina to primary visual cortex and feedback from high visual areas, providing viewing distance information (Fang, Boyaci, Kersten, & Murray, 2008; Liu et al., 2009; Murray, Boyaci, & Kersten, 2006; Sperandio, Chouinard, & Goodale, 2012). Moreover, single cell recordings have demonstrated gain-modulated size tuning cells in V1, V2, and V4 (Dobbins, Jeo, Fiser, & Allman, 1998), and disparity tuning cells in V1, V2, and MT (Trotter, Celebrini, Stricanne, Thorpe, & Imbert, 1992, 1996), whose firing rates depend on viewing distance. In support of our model, these findings imply that the perceived depth calculation depends on viewing distance information. 

There were studies examining whether the judgments of size, shape, and distance are independent. Brenner & van Damme (1999) tested whether adding information that improves one judgment influences the others. They found that adding distance information improved the three judgments in a consistent manner, whereas providing information about shape improved the shape (width, height, and depth) judgment, but not judgment of the size or position. Although Brenner & van Damme (1999) did not test whether manipulation of the size could influence the accuracy of shape judgment, Collett, Schwarz, and Sobel (1991) investigated how angular size and oculomotor cues interact in the perception of size and depth at different distances. Observers viewed stereoscopically simulated 3D surfaces through a darkened tunnel; thus, oculomotor cues were principal cues to distance perception. They found that estimates of the simulated depth dropped with increasing viewing distance, when surfaces were of constant angular size. But with surfaces of constant physical size, estimates were nearly independent of viewing distance—a demonstration of depth constancy. Based on these results, Collett et al. suggested that there were two components to depth scaling. One was related to viewing distance, the other was related to angular size, and their effects grew with viewing distance. They concluded that angular size and viewing distance contributed in a similar way to determine perceived size and depth. The angular size described in their study is proportional to the perceived size factor S(d) in our model (see the Appendix). Thus, these studies suggest that size perception contributes to the depth perception, supporting our model. In addition, the model is in accordance with a proposed neural mechanism of depth constancy (Bishop, 1994), which regards size and depth constancies as the first and second stages of a linked two-stage process. 

Conclusions

The StarTrek illusion demonstrates several strong illusions across different feature dimensions. Size and contrast illusions were studied in our previous work, in which we correlated the illusions across observers and discovered specific relationships between the two. In a similar fashion, size and depth illusions were investigated in this study. Our results demonstrate that perceptions of size, depth, and contrast, apparently independent visual features, are interconnected in a nontrivial fashion. All three are calculated from the corresponding retinal measures scaled by a common function of viewing distance. In addition, the perceived size scales retinal contrast and depth signals, sometimes producing paradoxical effects. Taken together, these results support the General Object Constancy model uniting size constancy, depth constancy, and contrast constancy phenomena into a single framework. 

Acknowledgments

Research reported in this publication was supported by the Department of Defense grant W81XWH-09-TATRC-VRP. 

Commercial relationships: none. 

Corresponding author: Jiehui Qian. 

Email: [e-mail]. 

Address: Northeastern University, Boston, MA, USA. 

Appendix

General Object Constancy mechanism Let s be the retinal separation and δ the binocular disparity between two points in space. Corresponding perceptual measures are given by the General Object Constancy model as follows. Brain first scales s by a dimensionless factor k. k is a function of the relative depth d/d₀, where d₀ stands for the reference viewing distance, e.g., the distance wherefrom the perceived motion in depth started in our optic flow paradigm. Based on our previous experiments (Qian & Petrov, 2012), function k(d/d₀) is approximately linear for small motion amplitude factors. This formulation is in agreement with the retinal size decreasing as a linear function of the viewing distance d. Correspondingly, δ is scaled by the square of k, because binocular disparity decreases as a square of the viewing distance, and therefore requires the squared factor k to keep its percept invariant to the viewing distance:  

 

 

 

where S and D stand for the perceived size and depth respectively. In addition, Experiments 1 and 2 demonstrate that increasing perceived size (the size illusion) makes the perceived depth gradient illusion stronger. This is accounted by adding a factor k′ to the depth equation:  

 

 

 

where S(d₀) is the perceived size at the starting viewing distance d₀, and S(d) is the perceived size at the current viewing distance d. In other words, the perceived depth is additionally scaled by the relative perceived size: S(d)/S(d₀). This model is illustrated in Figure 5. Without a loss of generality we can assign k(1) = 1 and therefore S(d₀) = s(d₀) and D(d₀) = δ(d₀). If the retinal size s remains constant (Experiment 2, size illusion), we obtain the illusion of the perceived size S as  

 

Because the perceived depth gradient (pencil's sharpness) is defined as the length of the pencil tip (encoded as its perceived disparity) over its perceived size, D/S, the depth gradient, DG, is given by  

 

Hence, we obtain for the strength of the depth gradient illusion in Experiment 1:  

 

and, therefore,  

 

This relationship is plotted by the red curve in Figure 3. The red curve does not pass through all the data points, but given the large error bars, it is unclear whether the model needs revision. Since the prediction is parameter free, any revision would have to be principled, rather than just by adjusting parameters. If the perceived size S remain constant (Experiment 2, depth gradient illusion), we obtain for the depth gradient illusion,  

 

Therefore,  

 

This relationship shown by the black curve in Figure 3 fits the data very well given that the relationship is parameter free.