Multistable binocular feature-integrated percepts are frozen by intermittent presentation

Abstract

When two different stimuli are presented continuously to each eye, the percept alternates over time between the left-eye stimulus and right-eye stimulus. The perceptual alternation can be slowed or even stopped, however, if the same stimuli are presented intermittently (D. A. Leopold, M. Wilke, A. Maier, & N. K. Logothetis, 2002; J. Orbach, D. Ehrlich, & H. A. Heath, 1963). A basic question is the nature of the persisting neural representation, which mediates the stabilized percept. Is it a representation for the dominant eye, for the stimulus in one eye or for a feature-integrated percept incorporating features presented separately to each eye? We define a feature-integrated percept as one constructed by the visual system but which never is presented as a stimulus. This was tested using a feature-integrated percept resulting from rivalrous, equiluminant chromatic patterns (S. W. Hong & S. K. Shevell, 2009). Measurements showed that the feature-integrated percept was stabilized by intermittent viewing: when the percept at the end of the initial viewing period was feature-integrated, this same integrated percept was seen on subsequent intermittent presentations. The results showed that the stabilized percept from these intermittent rivalrous patterns was due to a persisting neural representation at or after binocular feature integration, not to a persisting dominant eye or neural representation of a retinal stimulus.

Introduction

When two different images are presented continuously to each eye, the visual system resolves the conflicting responses from the two eyes by perceptual alternation between the left-eye image and right-eye image (Blake & Logothetis, 2002). Historically, binocular rivalry was viewed as a memoryless process due to its stochastic perceptual alternation and uncorrelated perceptual durations (Blake, Fox, & McIntyer, 1971; Fox & Herman, 1967; Walker, 1975). A recent study, however, reports a significant serial correlation in duration of perceptual dominance of one eye during rivalrous viewing (van Ee, 2009). Another study found mutual dependencies between percept probabilities and dominance durations during perceptual rivalry (Naber, Gruenhage, & Einhauser, 2010). This implies that perception during rivalry can be related to prior neural representations. 

The dependence of the current percept on a prior representation is particularly clear when rivalrous stimuli are presented intermittently with a blank period of several seconds or longer between presentations. In this case, perceptual alternation is slowed or even stopped (Leopold, Wilke, Maier, & Logothetis, 2002; Orbach, Ehrlich, & Heath, 1963). The same percept is perceived during a vast majority of successive presentations; this freezing of the percept has been called perceptual stabilization. The persistence of what one sees during successive intermittent presentations implies that information is maintained across the blank interval when the stimuli are off. The information maintained between intermittent presentations is viewed as some sort of perceptual memory (Maier, Wilke, Logothetis, & Leopold, 2003; Pearson & Brascamp, 2008). Perceptual memory does not simply reflect a record of the latest percept but rather can depend on the accumulated history of percepts (Brascamp et al., 2008; Maloney, Dal Martello, Sahm, & Spillmann, 2005). 

What type of information is kept across the blank interval? In addition, at what level, or levels, of the visual system is the persisting representation? First, perceptual memory may reflect a persisting retinotopic representation. Perceptual memory is not observed when the stimuli in both eyes change position, either horizontally or vertically, between intermittent presentations (Chen & He, 2004). The retinotopic location of intermittently presented stimuli, however, does not have to be at an exact location. Perceptual memory can be undisrupted up to about two degrees of visual angle from prior foveal presentation (Knapen, Brascamp, Adams, & Graf, 2009). Moreover, individual biases in perceptual interpretation at different retinotopic locations can occur: at different retinotopic locations, one percept can be preferred over another during the intermittent presentations (Carter & Cavanagh, 2007). Information from a specific eye also can be an important factor. When the rivalrous stimuli are switched between the eyes in successive intermittent presentations, the percept typically follows the same eye rather than the same stimulus (Chen & He, 2004; Pearson & Clifford, 2004). 

Second, perceptual memory may be due to persistence of the stimulus representation. When ambiguous stimuli are interleaved with one another, the separate memory traces are sustained during intermittent presentations (Maier et al., 2003). With rivalrous three-dimensional objects, exchanging them between the two eyes gives a percept that predominantly follows the same stimulus rather than the same eye (Grossmann & Dobbins, 2006). 

No previous study, however, considers the level of a feature-integrated binocularly combined representation. In contrast to previous work on rivalrous percepts, the experiments here used stimuli giving rise to multistable percepts that integrated features from dichoptically presented stimuli (feature-integrated percepts). Rivalrous, equiluminant chromatic patterns (Figure 1A) sometimes resulted in a percept of the stimulus in one eye (classical interocular rivalry) but often gave instead feature-integrated percepts that included colors from each of the eyes within a spatial pattern presented to only one eye (Figure 1B; Hong & Shevell, 2009). Feature-integrated percepts are not consistent with the stimuli presented to either eye so there is no direct correspondence between the percept and a retinal stimulus. These feature-integrated percepts are different from simple interocularly grouped percepts that may give a unified percept based on eye dominance within each small retinotopic region. 

Figure 1

 

Stimuli, reported percepts, and schematic diagram of experimental conditions. (A) A 2-cpd square-wave vertical grating was presented to one eye and a tooth-shaped vertically oriented grating (top half of grating phase-shifted by one-half cycle relative to bottom half) to the other eye. Each eye's stimuli were equiluminant. (B) Three types of reported percepts. Binocular feature-integrated percepts had perceptual misbinding of color to a form. Colors from both the left eye and right eye were perceived either in a left-eye form or right-eye form. Monocular percepts were from the stimulus in only the left eye or right eye. Piecemeal percepts were inhomogeneous fields with distinct regions from one eye or the other. (C, D) Schematic diagram of non-swapping and swapping conditions. Initially, both the left-eye and right-eye stimuli were presented continuously for 4 s and then extinguished. Then, both the left-eye and right-eye stimuli were presented for 0.5 s every 2.5 s (0.5 s on, 2 s off) for 1 min. For the swapping condition, the left-eye and right-eye stimuli were exchanged between the two eyes during each successive 0.5-s presentation.

When each eye is presented with either parts of concentric circles (Figure 2A) or parts with identical shapes but different colors (dots in Figure 2B), observers occasionally perceive only concentric circles or only a circle with a horizontal grating (stimuli in Figure 2A; as described by Diaz-Caneja in 1928 (cited by Alais, O'Shea, Mesana-Alais, & Wilson, 2000)), or all green dots or all red dots (stimuli in Figure 2B; Kovacs, Papathomas, Yang, & Feher, 1996). This interocular grouping can be explained by retinotopically local interocular rivalry within the stimuli, with local regions perceptually patched together to form a coherent object. The feature-integrated percept studied here (Figure 1B), on the other hand, cannot be explained by this kind of local rivalry. The feature-integrated percept is not consistent with any simple substitution from a corresponding retinotopic region from one or the other eye; rather it is the color—a feature—from the other eye that is misbound to a different form in the percept. 

Figure 2

 

Interocular grouping stimuli. (A) Diaz-Caneja rivalry stimuli. When each stimulus is presented to a different eye, one occasionally perceives alternation between concentric circles and horizontal lines (from O'Shea, 2004). (B) Patchwork color rivalry stimuli. When each stimulus is presented to a different eye, one occasionally perceives alternation between all red circles and all green circles (from Kovacs et al., 1996).

Here we tested whether the perceptual memory stores only retinotopic representations or also non-retinotopic feature-integrated representations. The results showed a persisting percept with intermittent presentation, for feature-integrated, monocular or retinotopically local (piecemeal) rivalrous percepts. This reveals a representation maintained during the blank interval at a locus at or beyond the level where signals from the two eyes are combined. 

Methods

Observers

Three observers participated in the study. Each had normal color vision as determined using a Neitz anomaloscope and normal color discrimination on the Farnsworth-Munsell 100-Hue test. Author P.K. (observer 3) had knowledge about the experiments. Observers 1 and 2 did not know the design or purpose of the study. Consent forms were completed in accordance with the policies of the University of Chicago's Institutional Review Board. 

Apparatus

Stimuli were generated using a Macintosh G4 computer and presented on a calibrated Sony color display (GDM-F520). The cathode ray tube (CRT) had 1360 × 1024 pixel resolution and a refresh rate of 75 Hz non-interlaced. Different stimuli were presented to each eye by projection through an eight-mirror haploscope. Binocular fusion was facilitated by adjusting two of the mirrors, attached to the same saddle on a triangular rail, to compensate for the distance between an observer's two eyes. 

Stimuli

A 2 cycle/deg square-wave vertical grating was presented to one eye and a tooth-shaped vertically oriented grating (top half of grating phase-shifted by one-half cycle relative to bottom half, Figure 1A) to the other. Each eye's stimuli were equiluminant at 8 cd/m². Equiluminance was determined separately for each observer using heterochromatic flicker photometry (HFP). The HFP field size was 1.6 deg and the temporal frequency was 12.5 Hz. 

Three chromaticities were used in the experiments. Chromaticities were specified in a cone-excitation space (MacLeod & Boynton, 1979). The chromaticities differed in both L/(L + M) and S/(L + M): [L/(L + M) = 0.718, S/(L + M) = 3.3], which appeared magenta; [L/(L + M) = 0.611, S/(L + M) = 0.3], which appeared green; and [L/(L + M) = 0.665, S/(L + M) = 1.0], which was metameric to equal-energy-spectrum (EES) “white.” The unit of S/(L + M) is arbitrary and set here to 1.0 for EES. A thin circular ring and nonius lines, all metameric to EES, guided fixation (Figure 1A). 

Procedure

Prior to making measurements, each observer dark adapted for 3 min. This was in order to bring the eyes of the observer to the same state of adaptation each day. A run began with presentation of the EES rings with nonius lines and an EES uniform field (8 cd/m²) in the center of each 2.0 deg circular aperture where the rivalrous stimuli would appear. When the observer pressed a button to indicate binocular fusion with all four nonius lines in view—two of them inline horizontally and two inline vertically—both the left-eye and right-eye rivalrous stimuli were presented continuously for 4 s and then extinguished. The 4-s duration of this initial viewing period was determined in pilot work that showed 4 s of continuous viewing gave on about half the trials a feature-integrated percept (Figure 1B) at the end of the viewing period. 

During the 4 s of continuous presentation in the initial period, observers used a game pad to report their percept at each instant by continuously pressing one of several response buttons. The last percept at the end of the 4-s presentation was determined from the last button press during this period. Separate buttons were used to report (1) dominance of the monocular tooth-shaped stimulus, (2) dominance of the monocular vertical grating stimulus, (3) both the left-eye and right-eye colors within the tooth-shaped pattern (feature integration), (4) both the left-eye and right-eye colors within the vertical grating pattern (feature integration), or (5) each of three different piecemeal percepts. Observers were instructed to report a two-color feature-integrated percept (response (3) or (4)) only when they perceived the left-eye and right-eye colors they had seen initially and no achromatic region in the pattern. A piecemeal percept was easily distinguished from percepts (1) to (4) and clearly inhomogeneous. Observers were confident in their judgments of a piecemeal percept though different piecemeal subtypes (inhomogeneous pattern, inhomogeneous color, or both) were more difficult to distinguish so the three piecemeal responses were combined into a single response category. 

After the initial viewing period, both the left-eye and right-eye stimuli were presented for 0.5 s every 2.5 s (2 s off, 0.5 s on) for 1 min (thus 24 intermittent presentations in a run). The display was dark during the stimulus-off period except for the fixation ring and nonius lines, which remained. After each stimulus-on period, the observer pressed a button on the game pad to report the percept. 

The study included two experiments. In the non-swapping experiment (Figure 1C), the same stimulus was presented to each eye on all 24 intermittent presentations within a run. In the swapping experiment, in half of the runs the left-eye and right-eye stimuli were exchanged on each intermittent presentation (swapping condition; Figure 1D), and in the other half of the runs, the non-swapping condition was repeated as a within-session control; swapping and non-swapping runs were ordered randomly within a session. Reported results are the average from 16 runs for each of four subconditions of stimulus presentation (64 repetitions in all). The four subconditions counterbalanced across the left and right eyes for (i) the two chromaticities and (ii) the two patterns (vertical grating and tooth-shaped grating). 

Results

Non-swapping experiment

Rivalrous equiluminant stimuli were presented intermittently with the image stimulating each eye unchanged in every successive brief presentation (Figure 1C). Measurements showed that the last percept at the end of the initial 4-s viewing period was strongly stabilized on subsequent intermittent presentations, whether the last percept was monocular, feature-integrated, or piecemeal. 

Consider the responses from observer 1. On average, a specific monocular percept (either response (1) [dominance of the monocular tooth-shaped stimulus] or (2) [dominance of the monocular vertical grating stimulus]) was seen at the end of the initial viewing period on 10% of the runs (see the black horizontal line across the “monocular” bar, top panel of Figure 3). If this frequency represents a baseline rate for a particular monocular percept, and that percept on each presentation is independent of what was seen previously, then about 10% of subsequent intermittent trials should evoke the same percept. Instead, that particular monocular percept was seen on nearly 90% of the following 24 intermittent presentations (dark gray bar, top panel of Figure 3). Results from observer 3 were similar: on 23% of the runs, one of the monocular images was perceived at the end of the initial viewing period, while that same percept occurred on over 70% of subsequent intermittent presentations. (Observer 2 never saw a monocular percept, either at the end of the initial viewing period or during intermittent presentations.) These results corroborate previous studies with rivalrous stimuli that evoke only alternating percepts of the left-eye and right-eye images (e.g., Chen & He, 2004; Pearson & Clifford, 2004). These results, however, cannot distinguish among (i) persisting dominance of one eye, (ii) a persisting neural representation for one eye's stimulus, or (iii) a persisting representation of a feature-integrated percept. 

Figure 3

 

Results from non-swapping condition. The percent of persistence during 60 s of intermittent stimulus presentation (vertical axis) is plotted for each percept experienced at the end of the initial viewing period (horizontal axis). Error bars are standard errors of the mean from measurements taken on four different days. The black horizontal line across each bar shows the percent of runs with the given percept at the end of the initial viewing period.

The critical observation for assessing the locus of the neural representation for the perceptual memory comes from runs with a feature-integrated percept at the end of the initial viewing period. On average, a specific binocular feature-integrated percept (either response (3) [both the left-eye and right-eye colors within the tooth-shaped pattern] or (4) [both the left-eye and right-eye colors within the vertical grating pattern]) occurred at the end of the initial viewing period on between 21% and 27% of the runs for each of the three observers (see the black horizontal line across the “feature-integrated” bar, middle of each panel in Figure 3). This percentage was by design, as described above, because the duration of the initial viewing period was selected to evoke a final percept that was feature-integrated on about half of the runs (that is, average of about 25% for each of the two binocular feature-integrated percepts). On runs when a binocular feature-integrated percept was seen at the end of the 4-s period, over 95% of subsequent intermittent presentations gave the same feature-integrated percept, for every observer (striped bars in the middle of each panel). Similarly, a piecemeal percept occurred at the end of the initial viewing period on 32%, 47%, or 12% of the runs for observers 1, 2, and 3, respectively, while on those runs the rate of perceiving a piecemeal percept on subsequent intermittent trials was 59% for observer 1 and over 90% for the other two observers. This is clear evidence that feature-integrated percepts, as well as piecemeal percepts with subregions from each eye's stimulus, persisted with intermittent presentation. This is expected if the perceptual memory maintained during the blank interval depended on a cortical neural representation at or after binocular combination. 

A possible alternative explanation, however, is that persistence of feature-integrated percepts resulted from a stabilized contribution from each eye. If the specific contribution from each eye persisted, with a following stage of binocular integration that combined the unchanging contribution from each eye, then alternating the stimulus between eyes on each successive presentation would disrupt the perceptual stabilization. This alternative was tested in the next experiment. 

Swapping experiment

The experiment described above was repeated except that the left-eye and right-eye stimuli were exchanged on each successive brief presentation of a run (swapping condition, Figure 1D). If the perceptual memory depends on persisting eye dominance or on a retinal representation re-excited by the same stimulus on each intermittent presentation, then the percept now should change on successive presentations because the stimulus in each eye always was different from the previous presentation. On the other hand, a cortical neural representation at or after binocular combination may integrate the swapped stimuli to give the same percept seen on the previous presentation. 

Results from the swapping condition again show strong perceptual stabilization. Observers 1 and 3 perceived a particular monocular image at the end of the initial viewing period on 8.5% of the runs but saw the same monocular percept on 82% or 54%, respectively, of the following intermittent presentations (dark gray bars, left panels of Figure 4A). All observers saw a feature-integrated percept at the end of the initial viewing period on between 24% and 33% of the runs (again by design) but saw the same feature-integrated percept on between 68% and 92% of the subsequent intermittent presentations (striped bars, middle of panels in Figure 4A). Piecemeal percepts showed a similar result: the last percept during initial viewing was piecemeal on 22%–34% of the runs, while 56–85% of percepts were piecemeal during the following intermittent presentations (light gray bars, right panels of Figure 4A). The percept, therefore, was stabilized even when the stimuli were exchanged between the two eyes on each successive presentation. 

Figure 4

 

Results from the swapping condition and from non-swapping runs interleaved in sessions with the swapping condition. The percent of persistence during 60 s of intermittent stimulus presentation (vertical axis) is plotted for each percept experienced at the end of the initial viewing period (horizontal axis). Error bars are standard errors of the mean from measurements taken on four different days. The black horizontal line across each bar shows the percent of runs with the given last percept at the end of the initial viewing period. (A) Results from swapping condition. (B) Results from interleaved non-swapping condition (non-swapping runs randomly interleaved with swapping runs in the same session).

As a control, half of the runs within each session repeated the non-swapping condition (randomly interleaved) so that results for swapping and non-swapping presentations could be compared within the same session. These interleaved non-swapping results were virtually the same as for the swapping measurements (compare Figures 4A and 4B). This supports the conclusion that swapping the stimuli between eyes on each presentation, or not, results in a comparable neural representation across the blank interval between each presentation and therefore supports perceptual memory from a feature-integrated representation at or after binocular combination. 

Discussion

Does binocular rivalry result from competition between coherent percepts, or from competition between signals from the two eyes? This has been widely debated (Blake, 2001; Blake & Fox, 1974; Blake, Westendorf, & Overton, 1980; Kovacs et al., 1996; Leopold & Logothetis, 1996; Logothetis, Leopold, & Sheinberg, 1996), but there now is general agreement that rivalry may be triggered by representations at various levels of the visual system (Blake & Logothetis, 2002; Tong, Meng, & Blake, 2006). The same question applies to the perceptual memory maintained when rivalrous stimuli are extinguished. Is the information maintained across the blank interval left- or right-eye dominance (Chen & He, 2004; Pearson & Clifford, 2004), a stored retinotopic representation (Carter & Cavanagh, 2007; Chen & He, 2004; Knapen et al., 2009), a stimulus representation (Grossmann & Dobbins, 2006; Maier et al., 2003), or a feature-integrated representation? Of course, a persisting representation may be maintained at more than one level of the visual system. 

The non-swapping experiment showed that feature integrated percepts are stabilized during intermittent presentations. This suggests that a feature-integrated percept is maintained between intermittent presentations, though an alternative possibility is that the persistence of a feature-integrated percept is due to stabilized contributions from each eye. This alternative explanation was eliminated, however, by the swapping experiment in which the left-eye and right-eye stimuli were exchanged on each successive intermittent presentation. Results from the non-swapping and swapping conditions were virtually the same. This showed that a feature-integrated neural representation can be maintained between intermittent presentations, not simply eye dominance, or a representation of one eye's stimulus, or a patchwork of retinotopically localized eye dominance. 

Previous studies report a strong influence of eye of origin on perceptual memory. When left-eye and right-eye stimuli are swapped between eyes, eye-of-origin information remains stable across a blank interval (Chen & He, 2004; Pearson & Clifford, 2004). Both studies, however, use stimuli with strong luminance contrast (30% and 99%, respectively), which results in eye-based perceptual dominance. Even modest luminance contrast within rivalrous stimuli is known to increase markedly the proportion of time of exclusive visibility for only one eye's stimulus (that is, eye-based perceptual dominance). With luminance contrast over 20%, observers seldom if ever perceive a feature-integrated percept (Hong & Shevell, 2006). Unsurprisingly, there is no persisting neural representation of a feature-integrated percept with luminance contrast over 20%. 

Interocular switch rivalry (Logothetis et al., 1996) shows that with lower luminance-contrast (25%) stimuli, stimulus rivalry is perceived more often than eye rivalry, whereas with higher luminance contrast (50%) it is the opposite (Lee & Blake, 1999). Moreover, with interocular switch rivalry presented intermittently, color is the best stabilized attribute of the percept (Pearson & Clifford, 2004). Separate contributions from the magno- and parvocellular streams to perceptual selection during interocular switch rivalry are reported (Denison, Hillenbrand, & Silver, 2010). Low spatial frequency gratings (4 cpd) and higher temporal frequencies (up to 30 Hz) promote eye rivalry whereas higher spatial frequencies (7 cpd) and equiluminant stimuli promote stimulus rivalry. This suggests that the magnocellular pathway contributes to eye rivalry and the parvocellular pathway to stimulus rivalry. 

In contrast to the experiments with luminance-contrast stimuli, the equiluminant stimuli used here revealed strong and comparable stabilization in both the swapping and non-swapping conditions. Two observers experienced stabilization with the swapping paradigm even for a monocular percept. Our results are consistent with Pearson and Clifford (2005). They report that when the rivalrous interrupting stimulus changes in either color or orientation during intermittent presentations, the percept is more stable than when the attributes are the same. Whether a prior unambiguous stimulus primes or induces switching depends on the strength of the stimulus (Pearson & Brascamp, 2008). Brief low luminance-contrast stimuli prime, causing perceptual repetition, whereas long high luminance-contrast stimuli tend to induce a switch in the percept (Brascamp, Knapen, Kanai, van Ee, & van den Berg, 2007). Equiluminant stimuli may have enhanced a priming effect here, which resulted in perceptual stabilization for the monocular percepts even during the eye-swapping condition. 

A recent study on ambiguous three-dimensional motion shows that the priming built up during the successive intermittent presentations can be diminished by applying TMS to human middle temporal cortex (Brascamp, Kanai, Walsh, & van Ee, 2010). The TMS is thought to interrupt a buildup of priming and so results in favoring individual percept biases. For example, after TMS observers perceive a perceptual interpretation that is most frequently reported during baseline measurements (individual bias) and perceive less of the unfavored interpretation. 

In conclusion, the results from both the non-swapping and swapping conditions here show that feature-integrated percepts are stabilized during intermittent presentation and therefore reveal that a feature-integrated representation can persist in perceptual memory. The perceptual stabilization of a feature-integrated percept cannot be due to eye dominance or to a monocular representation of the stimuli. Further, these results show that a percept that is never presented in either the left- or right-eye stimulus can be represented and held in perceptual memory. A feature-integrated perceptual memory is consistent with persistence of a neural representation for the percept evoked by the stimuli, not for the stimuli themselves. 

Acknowledgments

This work was supported by NIH Grant EY-04802. 

Commercial relationships: none. 

Corresponding author: Para Kang. 

Email: [e-mail]. 

Address: 940 E 57th St., Chicago, IL 60637, USA.