Dynamics of perceptual filling-in of visual phantoms revealed by binocular rivalry

Abstract

How do selective and constructive visual mechanisms interact to determine the outcome of conscious perception? Binocular rivalry involves selective perception of one of two competing monocular images, whereas visual phantoms involve perceptual filling-in between two low-contrast collinear gratings. Recently, we showed that visual phantoms lead to neural filling-in of activity in V1 and V2, which can be dynamically gated by rivalry suppression (M. Meng, D. A. Remus, & F. Tong, 2005). Here, we used psychophysical methods to study the temporal dynamics of filling-in, by applying rivalry or flash suppression to trigger the suppression or appearance of visual phantoms. Experiments revealed that phantom filling-in involves an active, time-dependent process that depends on the phenomenal visibility of the phantom-inducing gratings. Shortly after the inducing gratings became dominant during rivalry, the likelihood of perceiving phantoms in the intervening gap increased over time, with larger gaps requiring more time for filling-in. In contrast, suppression of the inducing gratings promptly led to the disappearance of visual phantoms, with response times independent of gap size. The fact that binocular rivalry can prevent the formation of visual phantoms rules out the possibility that rivalry suppression occurs after the site of phantom filling-in. This study provides novel evidence that visual phantoms result from a slow time-dependent filling-in mechanism; possible models to account for its time course are discussed.

Introduction

Subjective visual experience differs from the physical patterns of light that strike the retinae in two important ways. First, visual perception is highly selective; only a subset of the information processed by the retina ultimately reaches awareness. Second, perception is also constructive in nature, such that vivid impressions can occur in regions of the visual field that lack direct stimulation. To understand how such processes lead to the formation of stable visual representations, it is important to investigate the interaction between selective and constructive mechanisms in visual perception. Binocular rivalry and visual phantom perception provide two striking examples of the selective and constructive nature of vision, respectively. During binocular rivalry, discrepant images presented to the two eyes compete for perceptual dominance ( Figure 1A), such that only one monocular image is perceived at a time while the other is suppressed from awareness (Porta, 1593, as cited in Wade, 1996). Visual phantoms can be perceived when perceptual filling-in occurs in the blank gap region separating two collinear low-contrast gratings (Tynan & Sekuler, 1975), typically presented against a dark background. Under such conditions, a ghostly impression of the inducing gratings appears to extend across the blank gap (Figure 1B). 

Figure 1

 

(A) Example of a binocular rivalry display consisting of dichoptic orthogonal gratings. The reader can experience rivalry by cross-fusing the left and right pairs of images until they appear aligned. (B) Example of a visual phantom display, consisting of two collinear inducing gratings separated by a blank gap. An impression of faint dark stripes extending from the surrounding inducers through the uniform gap may be perceived. This illusion is more vivid when the surrounding inducing gratings move in unison. (C) Stimuli used in Experiment 1 to investigate the effects of rivalry suppression on visual phantom formation. The vertical phantom-inducing gratings rival with horizontal gratings.

Many studies have investigated binocular rivalry and visual phantoms separately, but little is known about how the mechanisms underlying these distinct phenomena might interact. Current evidence suggests that binocular rivalry involves neural competition at multiple levels of the visual hierarchy (Blake & Logothetis, 2002), with competition first emerging at early sites of visual processing, including V1, where eye-of-origin information is preserved (Tong, Meng, & Blake, 2006). Psychophysical studies indicate that binocular rivalry reduces local sensitivity in the eye undergoing suppression (Freeman & Nguyen, 2001). Rivalry can also attenuate low-level adaptation to orientation and motion (Blake, Tadin, Sobel, Raissian, & Chong, 2006) and weaken the formation of visual afterimages (Gilroy & Blake, 2005; Tsuchiya & Koch, 2005), implying that rivalry suppresses visual responses at early stages of processing. Although single-unit recordings in early visual areas of the monkey have found limited evidence of awareness-related fluctuations in neural activity during rivalry (Leopold & Logothetis, 1996), recent human neuroimaging studies have found strong awareness-related rivalry modulations in monocular regions of the primary visual cortex (Tong & Engel, 2001) and also the lateral geniculate nucleus (Haynes, Deichmann, & Rees, 2005; Wunderlich, Schneider, & Kastner, 2005). Thus, there is mounting evidence that rivalry suppression first emerges at early monocular stages of processing. Nonetheless, some information about the suppressed visual stimulus can reach higher areas of the visual system, including the posterior parietal cortex and inferior temporal cortex (Fang & He, 2005; Sheinberg & Logothetis, 1997), consistent with the notion of multiple levels of visual competition (Blake & Logothetis, 2002; Freeman, 2005; Tong et al., 2006). 

Psychophysical studies suggest that the formation of visual phantoms may also involve quite early stages of visual processing. Visual phantoms exemplify a robust form of perceptual filling-in that can occur in both foveal and peripheral regions of the visual field across gaps as large as 10 degrees (Tynan & Sekuler, 1975). Illusory phantoms closely resemble the pattern, color, texture, and motion of the surrounding inducers and typically appear as a somewhat dimmer version of the surrounding inducers extending across the physically blank gap. Moving inducers lead to more vivid phantoms than stationary inducers and can even elicit a local motion aftereffect in the unstimulated gap region, suggesting that phantom impressions are actively represented in the visual system (Weisstein, Maguire, & Berbaum, 1977). Interestingly, visual phantoms can still be perceived when the two inducers are presented to different eyes (Tynan & Sekuler, 1975); also, the vividness of visual phantoms can be modulated by changes in the stereoscopic depth of the intervening gap (Brown & Weisstein, 1991). These findings suggest that binocular neurons are involved in visual phantom perception. In comparison, a major component of binocular rivalry appears to involve the suppression of monocular signals, which would presumably occur at sites prior to the binocular integration of visual inputs in the primary visual cortex. Although these phenomena have been studied separately, there is indirect evidence to suggest rivalry suppression should occur at an earlier, monocular stage of visual processing when compared to phantom filling-in. 

To date, only one study has investigated the neural basis of visual phantom perception. In a functional magnetic resonance imaging (fMRI) study, we found that collinear phantom-inducing gratings lead to enhanced responses in regions of V1 and V2 corresponding to the blank gap (Meng, Remus, & Tong, 2005). Of particular interest, neural filling-in responses were strongly modulated by rivalry suppression when collinear phantom-inducing gratings were shown to one eye and rivaling orthogonal gratings shown to the other eye (Figure 1C). When viewing this display, observers informally reported perceiving visual phantoms when the vertical inducing gratings were dominant but not when the horizontal gratings were dominant. fMRI scanning revealed that filling-in responses in areas V1–V3 were closely correlated with the phenomenal visibility of the phantoms. This study provided direct evidence of the suppressive effects of rivalry on neural filling-in responses and informal psychophysical evidence to suggest that rivalry suppression can impair the phenomenal visibility of phantoms. 

The goal of the present study was to investigate the impact of rivalry suppression on visual phantom formation in a more systematic manner. Specifically, we focused on the temporal relationship between phantom filling-in and the phenomenal visibility of the inducing gratings during binocular rivalry and flash suppression. In Experiment 1, we determined whether phantom filling-in depends on the phenomenal visibility of the inducing gratings during spontaneous rivalry alternations, using displays such as those shown in Figure 1C. In Experiment 2, a flash suppression paradigm (Wolfe, 1984) was used to obtain more precise measures of the time course of filling-in. This allowed us to determine whether a larger gap between the inducing gratings required more time to be filled-in. Experiment 3 determined whether inexperienced observers show a similar time course for phantom filling-in, and whether this time course remains consistent across changes in the contrast of the inducing gratings. 

On the basis of our previous neuroimaging results and informal psychophysical observations (Meng et al., 2005), we hypothesized that binocular rivalry may take place at an earlier stage of visual processing than phantom filling-in. According to this notion, the suppressive effects of rivalry should be able to “gate” or prevent the flow of visual information to subsequent processing stages, including those responsible for the formation of visual phantoms. Rivalry suppression might also prevent phantom filling-in if both mechanisms operate at common stages of processing. However, if phantom filling-in occurs at an earlier stage of processing than rivalry suppression, then one would expect that phantoms should remain visible even when the surrounding inducers are suppressed locally by rivalry. This is because monocular rival targets lead to local suppression of input from the corresponding location of the other eye (Carlson & He, 2004; Fukuda & Blake, 1992; Lee & Blake, 2004), and the visual phantoms in our display were spared of any corresponding rival target. Although it may seem unusual to consider the possibility that suppressed inducers might still lead to filling-in, previous studies have shown that filling-in of artificial scotomas and subjective contours can occur when the inducing stimuli are physically removed or fade from phenomenal view (Morgan, McEwan, & Solomon, 2007; Ramachandran & Gregory, 1991; Shimojo, Kamitani, & Nishida, 2001). This indicates that the visibility of inducers and the filled-in region can be dissociated in certain situations. 

The time course of phantom filling-in was also of interest in this study. Whereas cognitive or passive theories of perceptual filling-in propose that the visual system simply ignores the absence of information in the blank region (Dennett, 1991; Durgin, Tripathy, & Levi, 1995), theories favoring active completion propose that perceptual filling-in depends on some type of neuronal filling-in of activity that propagates across retinotopic regions corresponding to the blank gap (Awater, Kerlin, Evans, & Tong, 2005; Fiorani Júnior, Rosa, Gattass, & Rocha-Miranda, 1992; Komatsu, 2006; Ramachandran & Gregory, 1991; Pessoa, Thompson, & Noë, 1998). A specific prediction of the active filling-in hypothesis is that larger gaps may require more time to be perceptually filled-in (De Weerd, Desimone, & Ungerleider, 1998; Paradiso & Nakayama, 1991). Experiments 2 and 3 were designed to test this hypothesis. 

Experiment 1: Visual phantom perception during spontaneous rivalry alternations

Experiment 1 investigated the relationship between spontaneous rivalry alternations and phantom filling-in by obtaining separate reports regarding the visibility of the phantoms and the visibility of the surrounding inducers. Observers viewed a dynamic rivalry display resembling that shown in Figure 1C and provided an online report indicating whether the vertical phantom-inducing gratings or horizontal gratings were perceptually dominant. At random intervals, a tone sounded which prompted observers to report whether or not they perceived visual phantoms in the blank gap at that particular moment. 

Methods

Observers

Three adults with normal visual acuity participated in the experiment. MM and EF are authors. DT is an experienced observer who was not informed of the purpose of this experiment. All observers showed intact binocular perception of random-dot stereograms of depth-defined letters and intact perception of visual phantoms when the same phantom-inducing gratings were viewed dioptically with both eyes. 

Stimuli and procedure

Observers viewed dichoptic rivalry stimuli through a mirror stereoscope, using a chin rest to maintain head stability at a viewing distance of 60 cm. Observers maintained fixation on a nonius fixation point, which consisted of the upper-left half of a cross presented to the left eye, the lower-right half of a cross presented to the right eye, and a central dot common to both half images. With the nonius fixation point, observers could rely on Cyclopean perception to monitor the alignment between the arms of the cross and thereby maintain stable vergence. 

Visual conditions were chosen to optimize phantom perception while still ensuring fairly coherent rivalry. Whereas rivalry suppression is more effective with small, static, high-contrast targets (Blake, 1989), visual phantoms are more salient with large, drifting, low-contrast gratings presented against a dark background (Gyoba, 1994; Tynan & Sekuler, 1975). The rivalry display consisted of two collinear vertical sine-wave gratings presented to the left eye paired against horizontal gratings presented to corresponding locations of the right eye (see Figure 1C, size of each grating 6° × 6°, mean luminance 16.0 cd/m², contrast 15%, spatial frequency 0.5 cycles/degree, size of gap 6° × 3°, luminance of background 12.8 cd/m²). Gratings oscillated within a stationary window at a rate of 0.5 cycles/s, reversing direction every cycle (i.e., every 2 s), which minimized the formation of local afterimages and discouraged Troxler's fading. Because large visual stimuli were required to produce salient visual phantoms, the gratings were presented in the periphery (6° to the left of fixation, 4.5° above/below fixation) to reduce the incidence of piecemeal rivalry (Blake, O'Shea, & Mueller, 1992). 

In each trial, the display was presented for a 120-s viewing period. Observers reported transitions in rivalry perception by pressing one of four keys with their right hand to indicate whether they perceived exclusive dominance of the vertical gratings, exclusive dominance of the horizontal gratings, gratings differing in orientation above and below the gap, or piecemeal rivalry in which the vertical and horizontal gratings appeared to be mixed together incoherently. Each trial began with an initial 6-s viewing period, after which an auditory tone cue was presented at random intervals that varied from 9 to 15 s (10 tone presentations/trial). Observers were instructed to indicate whether or not they perceived visual phantoms completing across the blank gap at the time the tone was heard, by pressing one of two keys with their left hand. After making this phantom perception response, observers resumed the rivalry task. 

Prior to the experiment, observers received as many practice trials as needed to ensure that they could perform both tasks competently and switch quickly from the rivalry task to the visual phantom task. Each observer performed a total of 48 experimental trials over a period of 4 half-hour test sessions to obtain a total of 480 phantom perception responses. 

Data analysis

We analyzed the time course of visual phantom perception to determine its dependence on prior perceptual states during rivalry, using a method similar to reverse correlation. Phantom perception responses, indicating whether or not observers perceived visual phantoms at the time of each tone cue, were classified according to whether vertical or horizontal gratings were exclusively dominant during a prior time period. Periods of piecemeal rivalry or mixed dominance, which comprised 26% of the total viewing duration, were excluded from the analysis. 

In the first set of analyses, phantom perception responses were binned according to the time elapsed since any previously reported transition in rivalry dominance. Data were sorted into 400-ms bins according to whether the tone cue occurred 0–400, 400–800, 800–1200, or so forth, after any reported rivalry switch ( Figure 2A); therefore, intervening rivalry switches were permitted in this analysis. The percentage of trials for which phantom perception occurred was calculated for each time bin and experimental condition, and the standard error of the mean was determined based on the known properties of the mean and variance of the binomial distribution. 

Figure 2

 

Time course of visual phantom perception during spontaneous rivalry in Experiment 1. (A) Percentage of trials for which phantom perception occurred after any rivalry switch to exclusive dominance of vertical gratings (red) or horizontal gratings (blue). (B) Percentage of positive phantom perception responses, binned according to the most recently reported rivalry switch. Red, vertical gratings currently dominant; blue, horizontal gratings currently dominant. Response data were pooled in 400-ms bins; fewer responses were available after 1600 ms so these responses were pooled into a single bin. Error bars represent ±1 SEM.

In a second set of analyses, we focused on the relationship between phantom filling-in and the currently dominant rivalry percept, by binning the data according to the last or most recently reported rivalry switch ( Figure 2B). This allowed us to determine whether phantom filling-in depended on the currently dominant perceptual state or also depended on prior perceptual states. We tested for statistical differences in the frequency of visual phantom perception during exclusive dominance of the vertical phantom-inducing gratings versus the horizontal gratings by applying a chi-square test to the data for each time bin of a given observer. 

Results and discussion

Perceptual filling-in of the visual phantoms was highly dependent on whether the vertical or horizontal surrounding gratings recently became dominant during binocular rivalry. Figure 2A shows the likelihood of visual phantom perception, relative to the time of any previously reported change in rivalry dominance. (Phantom perception responses are grouped into 400-ms bins; the first bin labeled “0” corresponds to 0–400 ms after any rivalry switch.) As can be seen, observers were most likely to perceive visual phantoms immediately after reporting exclusive dominance of the vertical phantom inducers (red curves). Shortly thereafter, the likelihood of phantom perception decreased over time, dropping to intermediate levels of about 50% after only 1–2 s, presumably because the vertical phantom-inducing gratings and horizontal gratings were about equally likely to appear dominant at this time. ¹ 

In comparison, observers failed to perceive visual phantoms immediately after reporting the dominance of the horizontal gratings (blue curves). Gradually, however, the likelihood of phantom perception increased over time, reaching its highest level about 1–3 s after the reported onset in dominance of the horizontal gratings. This increase in phantom perception over time appeared to reflect the increasing likelihood of the phantom-inducing gratings appearing dominant at this time. Individual differences in dominance durations were consistent with this notion: observer DT, who showed a delayed rise in phantom perception, also showed much longer mean dominance durations for the horizontal grating (2.8 s) than the other two observers (MM 1.3 s; EF 1.5 s). Results indicate that this reverse correlation method can reveal the temporal dynamics of rivalry. 

To address whether visual phantom formation depends on which of the two sets of gratings is currently dominant, and not on prior perceptual states, we conducted a second analysis by binning the data according to the most recently reported change in perceptual state. If filling-in of visual phantoms occurs only when the phantom-inducing gratings are currently dominant, then phantom perception should persist so long as those gratings remain dominant. In Figure 2B, it can be seen that this is indeed the case (red curves), whereas sustained perceptual dominance of horizontal gratings rarely led to reports of phantom perception, even well after the initial change in dominance (blue curves). Chi-square tests revealed highly reliable differences in the frequency of phantom filling-in for vertical gratings and horizontal gratings, in all time bins for all individual observers ( χ ² > 16.0, p < 0.0001 in all cases, except for time bin >1600 ms for MM, χ ² = 6.11, p < 0.05). Although these differences were dramatic, the frequency of phantom perception did not quite reach absolute levels of 100% and 0% occurrence for vertical and horizontal dominance conditions, respectively. Small deviations would be expected if observers sometimes found it difficult to isolate what was perceived at the time of each tone presentation. All three observers reported finding the task quite difficult, especially when the tone occurred near the time of a spontaneous rivalry reversal. Consistent with this notion, observer DT, who experienced the slowest rivalry alternations, also showed the strongest difference between conditions. 

The results suggest that rivalry suppression occurs at a prior or common stage of visual processing, relative to the site of phantom filling-in. However, it might be argued that filling-in could precede the site of rivalry suppression if visual phantoms resulted from monocular stages of visual processing and rivalry alternations involved global changes in eye dominance encompassing the entire visual field. According to this notion, monocularly generated phantoms would be visible only when a single eye's view of both phantom-inducing gratings (and the intervening filled-in region) is globally dominant. Previous psychophysical studies suggest that this is unlikely. Visual phantoms can be perceived even when the two inducers are shown to different eyes, indicating that phantom filling-in does not depend on monocular processes (Tynan & Sekuler, 1975). Also, rivalry involves competitive interactions in local regions of the visual field rather than the global spread of eye dominance (Carlson & He, 2004; Lee & Blake, 2004). Although global factors can serve to organize these local competitive interactions, the immediate suppressive influence of a rival target occurs over a limited spatial range (Fukuda & Blake, 1992). 

Nevertheless, we performed a control experiment to address this potential concern directly. Observers MM and DT performed the same visual task, except that the top and bottom phantom-inducing gratings were presented to different eyes and paired with rival horizontal gratings. Because the phantom inducers were processed by different eyes, perceptual filling-in of the intervening gap could not result from monocular processing. The results of this control experiment revealed the same pattern of results ( Figure 3). Therefore, visual phantom formation does not depend on monocular processes, and global fluctuations in eye dominance during rivalry cannot account for these modulations in phantom perception. 

Figure 3

 

Control experiment for Experiment 1. Top and bottom phantom-inducing gratings were presented to different eyes and paired with rival horizontal gratings. (A) Percentage of trials for which phantom perception occurred after any rivalry switch to exclusive dominance of vertical gratings (red) or horizontal gratings (blue). (B) Percentage of positive phantom perception responses, binned according to the most recently reported rivalry switch. Error bars represent ±1 SEM.

Experiment 2: Flash suppression and the temporal dynamics of phantom filling-in

In Experiment 2, we used flash suppression to characterize the effects of rivalry on visual phantom formation with greater temporal precision. Flash suppression can occur after briefly viewing a monocular stimulus; the sudden onset of a competing stimulus to the other eye can quickly lead to the suppression of the first monocular stimulus (Wolfe, 1984). Flash suppression is believed to depend on similar mechanisms of interocular suppression as binocular rivalry (Tsuchiya & Koch, 2005; but see also Tsuchiya, Koch, Gilroy, & Blake, 2006). However, unlike rivalry, in which fluctuations in perception occur spontaneously and are difficult to control, flash suppression allows for precise control over the timing of perceptual alternations. This allowed us to investigate the time course required for visual phantoms to fill-in gaps of varying size. Active completion theory proposes that perceptual filling-in results from the propagation of visual information from the surrounding inducing stimuli to the blank gap region (Komatsu, 2006; Meng et al., 2005; Paradiso & Nakayama, 1991; Pessoa et al., 1998; Ramachandran & Gregory, 1991). According to this theory, one might predict that after phantom inducers become dominant during rivalry, some time might be necessary for perceptual completion across the blank gap. Also, larger gaps could require more time for filling-in, reflecting the time required for neural activity to propagate across corresponding regions of retinotopic cortex. 

Perceptual alternations between the vertical and horizontal surrounding gratings during flash suppression ( Figure 4A) were compared to actual stimulus alternations between the two sets of stimuli ( Figure 4B). Unlike flash suppression, in which the first stimulus is rendered invisible by the second stimulus, in the stimulus alternation experiment the first stimulus was physically replaced by the second stimulus. If binocular rivalry can fully inhibit the perceptual filling-in of visual phantoms, then flash suppression should lead to similar perceptual consequences as the physical removal of one of the two sets of stimuli. 

Figure 4

 

Design and stimuli for Experiment 2. (A) Flash suppression condition. Left, Vertical phantom-inducing gratings shown to the left eye for 5–6 s were subsequently rendered invisible by the onset of high-contrast horizontal gratings presented to the right eye. Right, low-contrast horizontal gratings suppressed by vertical phantom inducers. (B) Stimulus alternation condition. Physical alternations between the two sets of gratings mimicked the perceptual consequences of flash suppression. The first display was replaced by the second display.

Methods

Observers

Four observers from the Vanderbilt University psychology department participated in this experiment. Observers were not informed of the purpose of the study. All observers had normal or corrected-to-normal visual acuity. Prior to the experiment, we confirmed that observers showed intact perception of dioptic visual phantom displays and intact binocular processing of random-dot stereograms containing depth-defined letters. 

Stimulus design and procedure

Observers maintained fixation on a nonius fixation point while pairs of gratings were monocularly presented via a mirror stereoscope at equal distances above and below fixation. Stimuli consisted of collinear vertical phantom-inducing gratings shown to the left eye and horizontal gratings shown to corresponding locations of the right eye (size 4° × 4°, mean luminance 23.4 cd/m ², spatial frequency 0.5 cycles/degree, luminance of background 18.7 cd/m ²). Gratings oscillated within a stationary window (temporal frequency 1.71 cycles/s, reversal of direction every cycle) and the gap between the gratings varied across trials (2°, 3°, 4°, or 5°). Black rectangles (5° × 15°) surrounded each monocular display to aid binocular fusion. A chin rest was used to maintain head stability at a viewing distance of 60 cm. 

There were four experimental conditions, involving either flash suppression or stimulus alternation to elicit switches from vertical-to-horizontal or horizontal-to-vertical gratings ( Figure 4). In pilot studies, we found that the large drifting gratings required to generate visual phantoms were quite difficult to suppress, so we modified the relative contrast of the horizontal gratings to levels that ensured reliable flash suppression in most observers. Phantom-inducing gratings were always presented at a fixed contrast of 15%. For flash suppression of the phantom-inducing gratings, these gratings were first presented to the left eye. Then after a variable 5–6 s delay, 75% contrast horizontal gratings were presented to the right eye to mask the inducing gratings ( Figure 4A, left). Both sets of gratings remained on the screen until the observer made a response, and the response time was recorded. Observers were instructed to press “1” as soon as they saw the phantoms disappear entirely. On rare occasions, if observers did not perceive phantoms during the first part of the trial or failed to perceive complete flash suppression, they were instructed to press “2” to abort the trial. 

In the other flash-suppression condition, 3% contrast horizontal gratings were first presented, and phantom-inducing gratings were added to the display 5–6 s later ( Figure 4A, right). Observers were instructed to press “1” as soon as they perceived visual phantoms extending completely across the blank gap. On rare occasions in which the vertical gratings failed to become fully dominant, observers were instructed to press “2” to abort the trial. It was necessary to omit trials that led to incomplete suppression (which were rare), since the goal of the study was to investigate the consequences of complete flash suppression on the visibility of the phantoms. Less than 12% of all trials were discarded as a result of this criterion. 

The tasks of reporting the appearance or disappearance of the visual phantoms were carried out in separate experimental blocks. Within each block, we compared the effects of flash suppression and stimulus alternation using a randomized mixed-trial design. For stimulus alternation, we mimicked the perceptual consequences of flash suppression by simply replacing the first display with the second display ( Figure 4B); the timing of stimulus events was otherwise identical. Within each block, the size of the gap between the gratings was randomly varied across trials to estimate the time required for phantom filling-in. Observers received a total of 32 trials in every condition. Data were collected over 4 experimental sessions. 

Data analysis

The time between the onset of the dichoptic mask and the observer's behavioral response was recorded for every trial that led to reliable flash suppression (88.3% of all trials). Each response time was treated as an independent observation. Analysis of variance (ANOVA) was used to test for differences between experimental conditions for each individual observer, with phantom appearance vs. disappearance, flash suppression vs. stimulus alternation, and gap size as separate factors. 

Results and discussion

In general, observers reported perceiving visual phantoms only when the vertical phantom-inducers were dominant, as was found in Experiment 1. Response times for changes in visual phantom perception were plotted as a function of gap size separately for each observer to depict the effects of flash suppression and stimulus alternation ( Figure 5). Although absolute response times varied somewhat across observers, presumably due to differences in response criteria and the time required to make and report decisions, the general pattern of results was remarkably consistent across the four observers. When horizontal gratings were masked by vertical phantom-inducing gratings, the time required for perceptual filling-in of the visual phantoms increased systematically as a function of gap size ( Figure 5A, red curves; t > 3.0 or more, p < 0.01 for linear contrasts applied to each subject). These results suggest that visual phantoms require more time to propagate across larger gaps in the visual field, consistent with theories of active filling-in. In contrast, when the phantom-inducing gratings were masked by horizontal gratings, the visual phantoms disappeared quite promptly (blue curves). Response times were unaffected by gap size for all observers ( t < 1) except for subject OC who showed an opposite pattern of faster RTs at larger gap sizes (flash suppression, t = −1.97, p = 0.06, stimulus alternation t = −2.32, p < 0.05). Thus, manipulations of gap size have asymmetric consequences on the time required for the perceptual completion and perceptual suppression of visual phantoms. These asymmetric effects may provide important clues about the mechanism underlying phantom filling-in (see General discussion). 

Figure 5

 

Time course of phantom perception as a function of gap size in Experiment 2. (A) Flash suppression results for 4 observers. Mean response times for reporting the perceptual completion of visual phantoms across the blank gap, after horizontal gratings were masked by vertical phantom inducers (red curves). Mean response times for reporting the perceptual suppression of visual phantoms, after phantom inducers were masked by horizontal gratings (blue curves). (B) Stimulus alternation results for reporting the perceptual completion (red) and disappearance (blue) of visual phantoms. Stimulus events matched those of panel A, except that the first display was physically replaced by the second display. Error bars represent ±1 SEM.

The time course of phantom perception was remarkably similar for flash suppression ( Figure 5A) and stimulus alternation ( Figure 5B). This was confirmed by analyses of variance, which revealed highly reliable effects of gap size for every observer ( p < 0.0001 in all cases) and a strong interaction between gap size and phantom appearance/disappearance ( p < 0.0001 in all cases), with no evidence of an interaction between flash suppression/stimulus alternation and gap size or between flash suppression/stimulus alternation and any other factor (all F's < 2.0, p > 0.05). Average response times were also very similar in the two conditions, with only one of the four observers (CK) showing reliably longer response times for flash suppression than for stimulus alternation (896 and 789 ms, respectively, F = 8.35, p < 0.005). Thus, it appears that the effects of flash suppression are quite powerful, leading to similar response times as the physical removal of the surrounding stimuli, despite their sustained presence on the retina. 

The results suggest that larger gaps require much more time to be filled-in. However, it is conceivable that larger gaps lead to weaker visual phantoms, which in turn, could lead to slower response times. To address this potential concern, we ran a control experiment in which observers adjusted the contrast of a physical grating to match the perceptual salience of visual phantoms that varied in gap size. Next, observers were presented with physical gratings of the corresponding size and contrast and had to perform a speeded detection task. Results revealed significant decreases in the perceived contrast of moving visual phantoms as a function of gap size (mean contrast matches of 4.9%, 4.2%, 3.9%, and 3.3% for gap sizes of 2, 3, 4, and 5 degrees, respectively, t > 4.9 or more for all observers, p < 0.00001). However, variations in stimulus contrast over this limited range had no effect on detection times (mean RTs of 424, 448, 428, and 444 ms for gap sizes of 2, 3, 4, and 5 degrees, respectively, no significant effect of gap size for any subject). Therefore, larger gaps do lead to less vivid phantoms, but this decrease in vividness cannot account for the time required for perceptual filling-in. 

Experiment 3: Flash suppression and phantom perception in inexperienced observers

In this experiment, we wished to confirm whether inexperienced observers would show the same temporal dynamics of visual phantom perception under conditions of flash suppression as was found in Experiment 2. Such a demonstration would indicate the generality of these findings and the active nature of perceptual filling-in. Moreover, we were curious as to whether varying the salience of the phantom-inducing gratings might influence the speed of propagation of the visual phantom. Although filling-in occurs with phantom inducers of different contrast levels, previous studies have found that the contrast of the surrounding inducers can affect the perceptual strength of phantoms (Gyoba, 1994). However, it is unknown whether such factors might alter the temporal dynamics of filling-in. One possible hypothesis is that inducing gratings of higher contrast might lead to more rapid propagation of the visual phantom. Alternatively, the mechanisms underlying visual phantom formation might operate in a fairly constant manner that is independent of the strength of the inducing stimulus. 

Methods

Observers

Observers consisted of twelve undergraduate or graduate students from Princeton University who received payment or course credit in an introductory psychology course for participation. All participants had normal or corrected-to-normal visual acuity, exhibited normal binocular perception of random-dot stereograms and reliable perception of visual phantoms. (A thirteenth observer was excluded due to impaired stereo-depth perception.) All observers were naïve to the purpose of the experiment and had little to no previous experience with visual phantoms or binocular rivalry. 

Stimuli and procedure

The experimental paradigm and visual stimuli were identical to those described for the flash suppression condition in Experiment 2, except for the following minor differences. First, the to-be-masked stimuli were presented for a shorter period of 2–3 s before the mask onset. Second, stimuli differed in contrast. Vertical phantom-inducing gratings were shown at 8% or 16% contrast. In one condition, the vertical gratings were masked by 80% contrast horizontal gratings. In the other condition, 2% contrast horizontal gratings were masked by the vertical gratings. Finally, eye assignment of stimulus displays was randomly mixed across trials. On half of the trials, vertical gratings were presented to the observer's left eye and horizontal gratings were presented to the right eye; for the other half, eye assignment was reversed. 

The task of reporting the appearance or disappearance of the visual phantoms was carried out in separate experimental blocks. Within each block, the following conditions were randomly varied across trials: contrast of the vertical inducing gratings, gap size between the gratings, and eye assignment of vertical and horizontal gratings. Observers received a total of 16 trials in every experimental condition. Data were collected within a single 1-hour session. 

Data analysis

Response times to the onset of the dichoptic mask were recorded for every trial that led to reliable flash suppression (95.0% of all trials). A group analysis was performed for this experiment because only a limited number of observations could be collected within the 1-hour session but many more observers were tested ( N = 12) than in Experiment 2. To reduce variability due to individual differences in response times, we normalized the response times obtained from each observer relative to that individual's mean response time averaged over all conditions. This normalization procedure removed the variability specific to an individual's absolute mean response time; any differences due to contrast level, gap size, or other experimental factors were preserved. Repeated measures analysis of variance was performed on mean normalized response times obtained from every condition of each observer to test for effects of phantom appearance vs. disappearance, gap size, and the contrast of the phantom inducers. Additional ANOVAs and planned linear contrasts were performed separately on phantom appearance and disappearance trials, to test for differential effects of gap size on response times. 

Results and discussion

The time course of visual phantom perception was very similar for the low- and high-contrast phantom inducers ( Figures 6A and 6B, respectively). When horizontal gratings were dichoptically masked by vertical phantom-inducing gratings, the time required for completion of the visual phantoms (red curves) steadily increased as a function of gap size ( F(3,33) = 12.78, p < 0.0001; significant linear trend for both low and high contrast conditions, t > 2.8 and t > 3.1, respectively, p < 0.01). However, when vertical phantom inducers were masked by horizontal gratings, mean response times for reporting the phenomenal disappearance of the visual phantoms (blue curves) did not vary as a function of gap size ( F(3,33) < 1, ns; t < 0.84, p > 0.40 for linear contrast). The effects of flash suppression on the time course of phantom perception revealed the same pattern of results as that found in experienced observers in Experiment 2 ( Figure 5A). 

Figure 6

 

Time course of visual phantom perception during flash suppression for phantom-inducing gratings of low contrast (A) or high contrast (B) in Experiment 3. Response times for reporting the appearance of visual phantoms, after horizontal gratings were masked by visual phantom inducers (red curves). Response times for reporting the disappearance of visual phantoms, after vertical phantom inducers were masked (blue curves). Mean normalized response times and standard errors are plotted as a function of gap size ( N = 12).

Response times were also significantly faster for reporting the disappearance of visual phantoms as compared to their appearance ( F(1,11) = 11.28, p < 0.01). However, neither the main factor of phantom-inducer contrast nor its interaction with any other factor was significant ( F < 2 in all cases). Thus, it appears that visual phantoms propagate at comparable speeds across visual space, despite variations in the strength of the surrounding inducers. The slow time course of phantom filling-in appears to be a robust effect that can be reliably demonstrated in a variety of observers. 

General discussion

The experiments provide converging evidence indicating that perceptual filling-in of visual phantoms depends on the outcome of rivalry. Phenomenal suppression of the inducing gratings abolished phantom filling-in during both spontaneous rivalry alternations ( Experiment 1) and flash suppression ( Experiments 2 and 3). The effects of flash suppression were virtually identical to the unambiguous physical presentation or removal of the phantom-inducing gratings ( Experiment 2), implying that visual responses to the surrounding inducers were strongly inhibited such that suppression effectively mimicked the removal of a physical stimulus. Finally, both experienced and inexperienced observers showed very similar effects of flash suppression on phantom filling-in ( Experiments 2 and 3, respectively), indicating the generality of these findings. 

The present findings allow us to rule out the possibility that rivalry suppression occurs at a later stage of processing than phantom filling-in. The fact that rivalry suppression consistently blocked filling-in indicates that suppression must be acting at an earlier or common stage of visual processing. Previously, we have shown that rivalry suppression can greatly suppress neural filling-in responses associated with visual phantoms (Meng et al., 2005). There is some other evidence to favor the notion that rivalry occurs prior to filling-in. Both psychophysical and neuroimaging studies suggest that rivalry suppression first emerges at monocular sites of the human visual system, and rivalry modulations have been found in monocular regions of the LGN and V1 (Freeman & Nguyen, 2001; Haynes et al., 2005; Lee & Blake, 2004; Tong & Engel, 2001; Wunderlich et al., 2005). In contrast, psychophysical studies have shown that visual phantom perception relies on binocular processing (Brown & Weisstein, 1991; Tynan & Sekuler, 1975), as was found here when the two phantom-inducing gratings were presented to separate eyes. Taken together, the data suggest that binocular rivalry emerges at monocular stages of visual processing, whereas phantom filling-in depends on subsequent binocular stages of processing. 

It remains possible that rivalry suppression and phantom filling-in involve common or overlapping stages of visual processing, in which case filling-in should also influence binocular rivalry. Perhaps consistent with this notion, in preliminary studies we have observed that two sets of visual phantoms with different orientations can rival, though this appears weaker than binocular rivalry between real gratings of the same perceived contrast (Meng, 2006). Such rivalry between visual phantoms could indicate one of two possibilities. One is that the filling-in mechanism can provide input to the mechanism underlying binocular rivalry, due to bidirectional interactions. Alternatively, visual phantoms may result from high-level pattern rivalry (Logothetis, Leopold, & Sheinberg, 1996), which has been proposed to take place at a later stage of processing than binocular rivalry (Wilson, 2003). We plan to investigate these issues in future studies, though they extend beyond the focus of the current study. From this study, we can conclude that binocular rivalry does not take place after the site of phantom filling-in. 

In conceptual agreement with the present findings, a previous study found that binocular rivalry can suppress the visibility of subjective contours (Sobel & Blake, 2003); thus, rivalry is effective at suppressing more than one form of perceptual filling-in. It remains an open question as to whether other forms of filling-in can also be gated by rivalry. It has been suggested that filled-in impressions at the blind spot can contribute to the predominance of a monocular stimulus during binocular rivalry (He & Davis, 2001). However, other studies have found that a perceptually filled-in grating encompassing the blind spot is less effective as a rival target than an intact grating (Tong & Engel, 2001), consistent with the notion that binocular rivalry precedes the site of filling-in of the blind spot. Further investigation of the impact of rivalry suppression on various forms of filling-in could help reveal distinctions between filling-in mechanisms. 

Temporal dynamics of phantom filling-in

Another key finding was that the time required for phantom filling-in increased linearly as a function of gap size. In comparison to the speed of most visual processes, these times were surprisingly long, ranging from 1163 ms for a 2° gap to 1612 ms for a 5° gap (averaged across all observers from Experiments 2 and 3). Although the response times found for the smallest 2° gap probably include sensory and decisional processes independent of filling-in, the additional time of ∼450 ms required for perceptual completion of the 5° gap reflects processes specific to filling-in of the phantom. These results cannot be readily explained by cognitive theories of filling-in, which propose that the visual system simply ignores the absence of information from the “filled-in” region (Dennett, 1991; Durgin et al., 1995). Instead, some type of active filling-in process appears to be responsible for the perceptual completion of visual phantoms across the blank gap, with larger gaps requiring more time to be filled-in. Our results provide the first evidence indicating that visual phantom formation involves an active time-dependent process. 

Certain other forms of perceptual filling-in have been shown to involve time-dependent mechanisms. For example, one psychophysical study found that the perceived brightness of a uniform stimulus propagates rapidly from the edge towards the stimulus' center, requiring on the order of 10's of milliseconds to travel a few degrees of visual angle (Paradiso & Nakayama, 1991). Filling-in of a textured surface seems to occur at slightly slower rates than brightness filling-in, typically within 120 ms or so (Caputo, 1998), though this study did not investigate effects of surface size. Studies of illusory contour perception suggest that amodal completion takes place within 100–200 ms of the onset of the inducers, but effects of gap size have not been reported (Gold & Shubel, 2006; Ringach & Shapley, 1996). Perceptual filling-in of artificial scotomas—the fading of a peripheral uniform target on a dynamic noise background—is highly dependent on target size and can require up to several seconds for filling-in to occur (Ramachandran & Gregory, 1991). However, the long durations necessary for perceptual fading under these conditions have been attributed to slow visual adaptation at the borders of the uniform target region (De Weerd et al., 1998). Once the target border is weakened by adaptation, the infiltration of visual signals from the dynamic surround can proceed rapidly. 

In the present study, we were able to manipulate the size of the intervening gap between the phantom-inducing gratings while maintaining a constant size and border length for the inducing gratings. Therefore, the additional time required for the filling-in of larger gaps (mean RTs for 2° and 5° gaps, 1163 ms and 1612 ms, respectively, difference 449 ms) cannot be attributed to border adaptation effects. Instead, these effects of gap size appear to reflect the operation of an active filling-in mechanism. Further investigation will be important to determine why different filling-in phenomena occur at different rates across the visual field. In comparison to other forms of filling-in, the completion of visual phantoms is unusually slow, perhaps suggesting the involvement of a distinct mechanism. 

What types of neural interactions might underlie the propagation of visual phantoms across the blank gap? Rapid feedforward connections are unlikely to account for these slow filling-in effects, especially their dependence on gap size. Myelinated feedback connections from higher visual areas with larger receptive fields might contribute to these effects (Shmuel et al., 2005), although feedback signals are believed to propagate much more rapidly than those involving unmyelinated lateral connections within a cortical area (Bullier, 2001). Long-range lateral interactions (Gilbert & Wiesel, 1989; Stettler, Das, Bennett, & Gilbert, 2002), perhaps in the primary visual cortex (Meng et al., 2005), appear to provide the most likely candidate for the filling-in of visual phantoms, although this hypothesis awaits further evidence. Excitatory lateral interactions in V1 tend to occur between iso-orientation columns that represent collinear regions of the visual field and have been implicated in contour integration (Kapadia, Ito, Gilbert, & Westheimer, 1995; Stettler et al., 2002). Interestingly, these facilitatory interactions are most prominent under low-contrast conditions (Kapadia, Westheimer, & Gilbert, 2000), which favor the formation of visual phantoms between collinear inducing gratings. It has also been proposed that weak, excitatory lateral interactions could contribute to other types of visual propagation, such as the tendency for rivalry dominance to spread along collinear orientations across the visual field (Wilson, Blake, & Lee, 2001). The rate at which rivalry dominance can spread along a coherent contour (∼9.6 degrees/s in parafovea) is quite comparable to the rates we find for phantom filling-in. Such slow propagation across the visual field, evident in both types of phenomena, could be due to the gradual spread of activity across the visual cortex mediated by local lateral interactions. 

Another finding in our study that may provide useful constraints on models of neural filling-in was the asymmetric effect of gap size, which differed for the appearance and disappearance of phantom-inducing gratings. Whereas visual phantoms required more time to propagate across larger gaps after the phantom-inducing gratings became visible, the phenomenal suppression or physical removal of the inducing gratings led to the relatively rapid disappearance of the visual phantoms, independent of gap size. This asymmetry cannot be easily explained if one considers filling-in to result solely from the propagation of visual signals from the surrounding region to the intervening blank region. Presumably, the onset of the phantom-inducing gratings leads to the rapid feedforward flow of signals to corresponding regions of visual cortex. This cortical activity, in turn, eventually spreads to adjacent regions that represent the intervening gap, with propagation times so slow that it takes an additional ∼450 ms for activity to traverse an additional 3° gap (i.e., 2° vs. 5°). When input to the cortical region representing the surrounding inducers comes to an abrupt stop, by either rivalry suppression or physical removal of the inducers, one would expect that the residual neural activity in this cortical region should still be able to propagate, unperturbed, to adjacent regions representing the blank gap. This leads to the prediction that visual phantoms that propagate across larger gaps should persist for longer durations, after the suppression or removal of the inducing gratings, a prediction not borne out by our results. How then might the different time courses observed for the appearance and disappearance of visual phantoms be explained? 

A possible account of phantom filling-in

We suggest a tentative model to account for the asymmetric temporal effects found for the appearance and disappearance of visual phantoms. This asymmetry can be understood if one considers perceptual filling-in to result from the summation of two sources of diffuse subthreshold input: a fast component and a slow component. The fast component might consist of rapid feedforward or feedback projections that are sufficiently diffuse to infiltrate the cortical region corresponding to the blank gap between the phantom-inducing gratings. The slow component could result from activity propagating laterally within a visual area, such as the primary visual cortex, along unmyelinated horizontal connections. 

According to this model, only the summation of these two sources of input can lead to the formation of visual phantoms. The arrival of signals from the fast component alone will initially lead to subthreshold levels of activity in the blank gap region, insufficient for evoking the impression of a visual phantom. Eventually however, activity will propagate laterally across the cortex from regions corresponding to the surrounding inducers to the blank region, leading to suprathreshold levels of activity in this area. Thus, the time required for the appearance of visual phantoms should be determined by the slowest component of these two subthreshold sources of input, that is, the time required for signals to travel laterally across the cortex. Larger gaps require more time to be filled-in because signals must propagate over a greater cortical distance. In contrast, disruption of the fastest component of the input to the blank gap region will determine how quickly the visual phantoms disappear. For example, many feedback projections to V1 are quite diffuse, so the speed of these feedback signals would not depend on gap size. Once these signals are cut-off, activity in the cortical representation of the blank gap would immediately fall below threshold. 

This model provides a viable account of the asymmetric effects observed for the appearance and disappearance of visual phantoms. However, the model should be considered tentative, as its validity remains to be tested in further studies. Consistent with this proposed model, optical imaging studies have shown that a small point stimulus evokes spiking activity in a restricted region of V1 but leads to subthreshold activity in a greatly extended region, up to 20 times the areal size of the spike-activated zone (Das & Gilbert, 1995; Grinvald, Lieke, Frostig, & Hildesheim, 1994). This subthreshold activity may reflect the contribution of both lateral interactions and feedback interactions in cortex. Other studies indicate that subthreshold activity in V1 may have an important role in contour integration and collinear facilitation effects (Das & Gilbert, 1999; Kapadia et al., 1995; Stettler et al., 2002). Although the neurophysiological basis of visual phantom perception has yet to be explored, human neuroimaging studies suggest that phantom filling-in involves long-range facilitatory interactions in early visual areas, including V1 and V2 (Meng et al., 2005). Important issues for future research will be to determine the relative contributions of feedforward, lateral, and feedback inputs in visual phantom formation and other forms of perceptual filling-in, and to develop a more formalized understanding of the temporal dynamics of these filling-in mechanisms. 

Acknowledgments

The authors would like to thank Devin Brady for technical assistance and Randolph Blake for helpful discussions. This work was supported by NIH grant R01 EY14202 to F.T. 

Commercial relationships: none. 

Corresponding author: Ming Meng. 

Email: [e-mail]. 

Address: 46-4089, 77 Massachusetts Avenue, Cambridge, MA 02139. 

Footnote

Footnotes

¹  The relative frequency of either grating appearing dominant, at any point in time following a reported switch, depends on the fact that rivalry dominance durations can vary in length from fractions of a second up to several seconds. After the onset of dominance of the vertical gratings at time 0, there is an increasing likelihood over time that observers will have experienced a perceptual switch to the horizontal gratings, but also an increasing likelihood of more than one reversal occurring by this point in time (e.g., switch to horizontal and back to vertical). Thus, the probability of the horizontal gratings appearing dominant at any point after time 0 is unlikely to reach 100% and instead may saturate at levels of about 50%, especially for subjects with more variable dominance durations.