A short-term memory of multi-stable perception

Abstract

It is well known that pauses in the presentation of an ambiguous display may stabilize its perceptual appearance. Here we show that this stabilization depends on an extended history spanning several dominance periods, not merely on the most recent period. Specifically, appearance after a pause often reflects less recent (but longer) dominance periods rather than more recent (but shorter) periods. Our results imply the existence of a short-tem memory for perceptual appearance that builds up over seconds, decays over minutes, and is robust to perceptual reversals. Although this memory is most evident in paused displays, it influences perceptual reversals also when display presentation continues: while the memory of one appearance prevails over that of the other, successive dominance durations are positively correlated. This highly unusual successive dependence suggests that multi-stable perception is not the memoryless ‘renewal process’ as which it has long been regarded. Instead, a short-term memory of appearance must be added to the multiple processes that jointly produce reversals of perceptual appearance.

Introduction

While visual perception is stable and robust under most conditions, certain ambiguous displays induce spontaneous fluctuations between two or more alternative perceptions. This multi-stable perception, which has intrigued scientific minds for two centuries, continues to be investigated with a large variety of both binocular (Blake & Logothetis, 2002) and monocular displays (Leopold & Logothetis, 1999). Surprisingly, the perceptual fluctuations elicited by widely different kinds of displays exhibit a universal stochastic nature (e.g., Brascamp, van Ee, Pestman, & van den Berg, 2005): perceptual dominance times are distributed statistically (approximating a Gamma distribution) and successive dominance periods are independent (Fox & Herrmann, 1967; Levelt, 1967). 

Accordingly, multi-stable perception gives every appearance of a ‘renewal process’, in that each perceptual reversal seems to depend only on the immediately preceding reversal, not on the earlier history of reversals. For this reason, multi-stable perception has long been considered a “memoryless” process (Blake, Fox, & McIntyre, 1971; Borsellino, De Marco, Allazetta, Rinesi, & Bartolini, 1972; Fox & Herrmann, 1967; Walker, 1975). 

At least one observation, however, sits oddly with the supposed memorylessness of multi-stable perception: when presentation of an ambiguous display is interrupted and later resumed, the dominant appearance tends to remain the same in spite of even a long pause. This surprising persistence of dominant appearance stabilizes perception considerably, slowing or even arresting perceptual reversals for intermittently presented displays (Leopold, Wilke, Maier, & Logothetis, 2002; Maier, Wilke, Logothetis, & Leopold, 2003; Orbach, Ehrlich, & Heath, 1963). 

How perceptual appearance is able to persist without a stimulus remains unknown. One class of mechanisms might involve adaptation states (Noest, van Ee, Nijs, & van Wezel, 2007), which are known to influence appearance and to persist over gaps of stimulation (Nawrot & Blake, 1989; Petersik, 2002). Another possible mechanism, which is supported by recent imaging results, would be a short-term memory for dominant appearance, which would presumably reside at post-perceptual levels of processing (Maier et al., 2003; Sterzer & Rees, 2008). Yet another possibility would be that each observer and/or retinal location exhibits an intrinsic and permanent bias in favor of one appearance or another (Carter & Cavanagh, 2007). Such a permanent bias could explain the consistent appearance of stimulus onsets without any memory-like process. 

Here we investigate whether persistence of appearance over a stimulus interruption reflects only the last dominance period before the interruption, or also a longer history of dominance periods. The former possibility would be consistent with a ‘renewal process’: the last appearance before stimulation ceases would leave a ‘lingering bias’, which would then influence appearance once stimulation resumes. This was, in essence, the suggestion of Leopold et al. (2002). The latter possibility would violate the assumption of a ‘renewal process:’ in this case, several dominance periods before the offset would leave antagonistic ‘lingering biases,’ each favoring the appearance in question, and the appearance at renewed onset would then be determined by the saldo of the accumulated biases. 

Previous work on interrupted, multi-stable displays could not address this issue, as the times for which stimuli were presented (between interruptions) were too short for perceptual reversals to occur (Leopold et al., 2002; Maier et al., 2003; Orbach et al., 1963). As only one appearance dominated each presentation time, only one appearance had an opportunity to establish a ‘lingering bias.’ To avoid this limitation, we lengthened presentation times and interrupted the display only after a chosen number of perceptual reversals had occurred, so that both appearances could establish ‘lingering biases’ prior to the interruption. When stimulation resumed after the interruption, this allowed us to determine whether appearance depended only on the most recent ‘bias’ or whether it reflected a combination of several ‘biases.’ 

Our results show that phenomenal appearance of an interrupted, multi-stable display depends on an extended history of three or more dominance periods, not only on the period immediately preceding each interruption. We will argue that this demonstrates the existence of a short-term memory for perceptual appearance. The present findings have been reported in abstract form (Pastukhov & Braun, 2007a). Related observations have recently been reported by another group (Brascamp et al., 2008). 

Methods

Observers

Six observers (3 female, 3 male, including the first author AP) participated in Experiment 1 and four observers (1 female, 3 male, including AP) participated in Experiment 2. All observers had normal or corrected-to-normal vision and some were experienced psychophysical subjects. Apart from AP, all observers were naive to the purpose of the experiment and were paid to participate. Procedures conformed to local ethics guidelines and all observers gave informed written consent. 

Apparatus

Stimuli were generated by a computer (HP xw8000 PC, HP Invent, Palo Alto, CA; video card Quattro FX 1100, NVidia, Santa Clara, CA) and displayed on a 19” CRT screen (Vision Master Pro 454, Iiyama, Nagano, Japan), with spatial resolution of 1600 × 1200 pixels and refresh rate of 85 Hz. The viewing distance was 95 cm, so that each pixel subtended approximately 0.011°, and background luminance was 30 cd/m ². 

Rotating sphere

We used the kinetic-depth effect (Wallach & O'Connell, 1953) to create the appearance of a three-dimensional rotating sphere (see Figure 1A). One hundred dots (diameter 0.3°, luminance 64 cd/m²), uniformly covering the surface of a virtual sphere, were projected orthographically. The virtual sphere (diameter 5.6°) rotated 0.25 Hz about a vertical axis and was centered 3.2° above or below fixation (depending on the observer's preference). This display is ambiguous and consistent with two opposite rotations in depth. Phenomenal appearance is bi-stable and alternates between the two possible rotations. 

Figure 1

 

Stimulus and procedure. (A) Bi-stable display of a rotating sphere, which is perceived with front surface rotating either left or right. (B) Stimulation and phenomenal appearance in Experiment 1 (schematic). Display periods alternated with blank periods of equal duration. Display periods were terminated after one perceptual reversal (dominance periods T₁ and T₂). (C) Same for Experiment 2. Display periods were terminated after two perceptual reversals (dominance periods T₁, T₂, and T₃). See text for further details.

General procedure

Observers viewed the display continuously for 31 display periods and 30 blank periods (one ‘block’), reporting the direction of rotation of the ambiguous sphere by continuously pressing one of two keys: 〈Left〉 and 〈Right〉 depending on whether the front surface appeared to move left or right. Phenomenal reversals were typically swift and complete and observers were instructed to report each reversal as rapidly as possible. To minimize any residual memory, observers were required to pause for 5 min after each block. In total, we collected 76 blocks in Experiment 1 and 38 blocks in Experiment 2. 

To facilitate comparison with other publications on the subject, we analyzed the results in terms of a ‘survival probability’ ( P _survival), defined as the probability that the same appearance will dominate before and after the blank period (e.g., Leopold et al., 2002). If A₁, A₂, and A₃ denote the respective appearances of the first, second, and third dominance periods during a display period (see Figures 1B and 1C) and if i is the index of the display period, ‘survival’ of phenomenal appearance is defined as A₂ⁱ = A₁ⁱ⁺¹ in Experiment 1 and A₁ⁱ = A₃ⁱ = A₁ⁱ⁺¹ in Experiment 2. 

Survival probability from display period i to i + 1 was compared to the duration of dominance periods T ₁, T ₂, and T ₃ in display period i ( Figures 1B and 1C). As mean dominance times varied significantly between different observers and between different experimental sessions with each observer, dominance durations are reported as multiples of the mean dominance duration 〈 T _dom〉 for each session. This session mean was computed as the average of (up to) 30 values of T ₁ in Experiment 1 and 30 values of T ₁ and T ₂ in Experiment 2. 

Experiment 1 (two dominance periods)

Each display period was terminated at a time T ₂ after the first phenomenal reversal was reported ( Figure 1B). T ₂ remained constant during each block but ranged between 0.0 and 0.9 times 〈 T _dom〉 in different blocks. At the beginning of each session, the mean dominance duration 〈 T _dom〉 was established for each observer under conditions of continuous stimulation. A second estimate of 〈 T _dom〉 was obtained from the main experiment. 

In a control experiment (not shown), we measured the reaction time of observers as T _RT = 385 ± 100 ms and used this value to correct for the inevitable delay between a phenomenal reversal and its voluntary report by the observer. Specifically, dominance times T ₁ and T ₂ were computed as  

$T_1=T_{\mathrm{report}}-T_{\mathrm{RT}},$

(1)

 

$T_2=T_2+T_{\mathrm{RT}}.$

(2)

 

Each display period was followed by a blank period of equal duration ( T ₁ + T ₂) in order to minimize adaptation effects (Petersik, 2002). In absolute numbers, the blank period ranged from 1700 to 7700 ms with a mean of 2924 ms. In this range of durations, blank periods are expected to stabilize perception and reduce the frequency of reversals (Kornmeier, 2002; Kornmeier, Ehm, Bigalke, & Bach, 2007). 

In 21.18% of the display periods, observers reported two reversals before the period was terminated. These periods were disregarded in the analysis of the results. 

Experiment 2 (three dominance periods)

Each display period was terminated immediately after the second phenomenal reversal was reported ( Figure 1C). The duration of T ₃, which could not be measured, was assumed to be equal to the observer's reaction time T _RT (see above). To correct for reporting delays, dominance durations were computed as  

$T_1=T_{\mathrm{report}1}-T_{\mathrm{RT}},$

(3)

 

$T_2=T_{\mathrm{report}2}-T_{\mathrm{report}1},$

(4)

 

$T_3=T_{\mathrm{RT}}.$

(5)

 

Each display period was followed by a blank period of equal duration ( T ₁ + T ₂ + T ₃). In absolute terms, the blank period ranged from 1834 to 4321 ms in this experiment. 

Results

Experiment 1: Two dominance periods

In this experiment, each display period comprised two dominance periods ( Figure 1B). The first period T ₁ was terminated by a spontaneous reversal of appearance, while the second period T ₂ ended artificially with the display period. T ₂ was kept constant for each block but varied systematically between blocks. Each display period was followed by a blank period of equal duration ( T ₁ + T ₂). In the analysis of this experiment, P _survival ² is the probability that onset appearance agrees with the most recent dominance period (prior to the interruption), while P _survival ¹ = 1 − P _survival ² is the probability that it is consistent with the less recent dominance period. Further details may be found in the Methods section. 

Representative results are illustrated in Figures 2A and 2B, which shows the reported appearance during display periods and highlights whether the reported appearances immediately before and after the interruption agreed (red dots) or disagreed (green dots). The durations of T ₁ exhibited the characteristic variability of multi-stable percepts ( Figure 2C). Typically, both appearances were reported at the onset of display periods, although this varied between sessions and observers. Figure 2D shows the distribution of report fractions A _right/( A _right + A _left) over all sessions and observers (average 0.46 ± 0.28). Thus, we cannot confirm that observers consistently favor one particular onset appearance, as reported recently for another type of multi-stable display (Carter & Cavanagh, 2007). 

Figure 2

 

(A) Example block with reported appearance during display periods (blue lines), interruptions of the display (gaps in blue lines), and agreement (red dots) or disagreement (green dots) of appearance before and after each interruption. (B) Schematic of T₁, T₂, and P_survivalⁱ. (C) Distribution of dominance times T₁ across all observers and sessions, normalized to 〈T_dom〉, with Gamma distribution fit (α = 6.15, C_V = 0.41). (D) Distribution of right percept report-fractions at onset. (E) Survival probabilities of last appearance (P_survival²) for different observers.

Our main interest was the survival probability P _survival ², which lay below 0.5 for most observers, ranging from 0.268 to 0.54 for individual observers and averaging to 0.38 ± 0.12 over all observers ( Figure 2E). Thus, appearances before and after the interruption disagreed more often than not. This results contrast markedly from the values of P _survival > 0.8 obtained when only one dominance period is allowed per display period (Leopold et al., 2002; Maier et al., 2003; Pearson & Clifford, 2005). 

To investigate more closely the dependence of P _survival ⁱ on T ₁ and T ₂, we took advantage of the fact that T ₁ varied between display periods and T ₂ varied between blocks. Specifically, we sorted display periods into bins according to the value of T ₁ and computed the average value of P _survival ¹ for each bin ( Figure 3A). Similarly, we computed the dependence of P _survival ² on the value of T ₂ ( Figure 3B) and on the value of T ₂ − T ₁ ( Figure 3C). In each case, the results show that the probability of a particular appearance after the interruption increases with length of time for which this appearance was dominant prior to the interruption. 

Figure 3

 

(A) Survival of the first (less recent) appearance ( T ₁) as a function of its dominance time T ₁. (B) Survival of second (more recent) appearance ( T ₂) as a function of its dominance time T ₂. (C) Survival of the second (more recent) appearance ( T ₂) as a function of the difference in dominance times T ₂ − T ₁.

At first glance, this monotonically increasing relation appears more pronounced for (the less recent) T ₁ than for (the more recent) T ₂. Note, however, that the range of values sampled for T ₂ was less than half that for T ₁. Presumably, P _survival ² would have reached larger values if longer durations of T ₂ could have been explored. This expectation was borne out by the dependence of P _survival ² on T ₂ − T ₁ ( Figure 3C): for positive T ₂ − T ₁, P _survival rises steeply with the value of T ₂ − T ₁. Accordingly, the most recent appearance persists whenever it outlasts the less recent appearance, regardless of absolute duration. 

We now turn to another level of analysis and consider correlations between successive display periods. Analyzing ‘runs’ of successive onsets with identical appearance (termed “consistent onset periods” or COP), we found that the overall distribution of run lengths resembled a Poisson process with identical P _survival (KS test, n = 10, D = 0.2, p > 0.2). However, the mean run length decreased significantly with increasing T ₂ ( Figure 4A). This fits with the idea that onset appearance is determined by ‘lingering biases’: when T ₂ is much shorter than T ₁, the onset appearance leaves the stronger ‘bias’ and becomes self-perpetuating, resulting in long COPs. As T ₂ increases toward T ₁, the ‘bias’ from the second appearance grows, increasing the likelihood of an inconsistent onset and thus curtailing COPs. 

Figure 4

 

Consistent onset periods (COPs). (A) Mean duration of COPs as a function of T ₂. (B) Correlation coefficients for T ₁ values for successive display periods falling inside the same COP (red columns), outside any COP (yellow columns), and over the entire data set (blue columns).

Astonishingly, the values of T ₁ were significantly correlated between successive display periods ( Figure 4B). This violation of sequential independence was limited to successive display periods falling within the same COPs, with correlation coefficients as high as 0.55. For successive display periods falling outside COPs, no correlation between the values of T ₁ was evident. 

Experiment 2: Three dominance periods

The previous experiment showed that onset appearance depends on the durations of both more and less recent dominance periods. To generalize this finding, we modified the experiment to allow three dominance periods in each display period ( Figure 1C). The first two periods ( T ₁ and T ₂) lasted until appearance reversed spontaneously, while the third period ( T ₃) was terminated by the end of the display period. T ₃ was kept as short as possible and not varied systematically. Each display period was followed by a blank period of equal duration ( T ₁ + T ₂ + T ₃). In the context of this experiment, P _survival ^1,3 is the probability that onset appearance matches the most recent dominance period (prior to the interruption), while P _survival ² = 1 − P _survival ^1,3 is the probability that it matches the less recent dominance period. For further details, the reader is referred to the Methods section. 

Results from one representative block are illustrated in Figures 5A and 5B. The durations of T ₁ and T ₂ varied stochastically, exhibiting typical Gamma-like distributions ( Figure 5C). No observer consistently reported one particular appearance at the onset of display periods ( Figure 5D). The survival probability P _survival ^1,3 of the most recent appearance averaged 0.62 ± 0.17 across observers ( Figure 5E). 

Figure 5

 

(A) Example blocks with reported appearance during display periods (blue lines), interruptions of the display (gaps in blue lines), and agreement (red dots) or disagreement (green dots) of appearance before and after each interruption. (B) Schematic of dominance periods T ₁, T ₂, and T ₃ and P _survival ⁱ. (C) Distribution of dominance times T ₁ and T ₂ across all observers and sessions, normalized to 〈 T _dom〉, with Gamma distribution fit ( T ₁: α = 4.7, C _V = 0.46; T ₂: α = 6.37, C _V = 0.4). (D) Distribution of right percept report-fractions at onset. (E) Survival probabilities of most recent appearance P _survival ^1,3 for different observers.

To establish how P _survival ⁱ depends on the dominance times of the preceding display period, we sorted display periods into bins according to the value of T ₁ + T ₃ and computed the average value of P _survival ^1,3 for each bin ( Figure 6A). Similarly, we computed the dependence of P _survival ² on the value of T ₂ ( Figure 6B) and of P _survival ^1,3 on the value of T ₁ − T ₂ + T ₃ ( Figure 6C). As in the first experiment, we observed that a particular onset appearance becomes the more likely, the longer it had been dominant during the preceding display period. This demonstrates that at least three dominance periods influence a subsequent onset appearance. 

Figure 6

 

(A) Survival of first and third (last) appearances ( T ₁) as a function of its total dominance time T ₁ + T ₃. (B) Survival of the second (last-but-one) appearance ( T ₂) as a function of its dominance time T ₂. (C) Survival of the first and third (last) appearances ( T ₁) as a function of the difference in dominance times T ₁ + T ₃ − T ₂.

We also analyzed successive onsets dominated by the same appearance (“consistent onset periods” or COPs). The distribution of COP lengths resembled a Poisson process with the same P _survival (KS test, n = 10, D = 0.36, p > 0.1). As in Experiment 1, individual values of T ₁ and T ₂ were correlated in successive display periods, provided these fell into the same COP ( Figure 7). The observed correlation was stronger for the appearance not dominating the onsets of the COP ( T ₂). 

Figure 7

 

Correlation coefficients for T ₁ and T ₂ values observed in different presentation periods within the consistent onset periods (COPs).

A simple model

The results presented above are qualitatively consistent with a short-term memory of multi-stable perception. To make this idea more precise and to establish the extent to which it provides also a quantitative account for our observations, we formulated a simple model. The model assumes that, whenever one particular appearance dominates perception of a multi-stable display, a ‘bias’ in favor of this appearance gradually develops. When another appearance dominates, or when the display is interrupted, this ‘bias’ gradually decays. Importantly, the decay is far slower than the earlier rise. Over time, ‘biases’ accumulate in favor of both appearances. When an interrupted display is resumed, appearance follows the balance of the accumulated ‘biases’. 

Specifically, we assume that, while appearance A _i ( i ∈ {1,2}) dominates perception, the associated bias B _i increases exponentially with time constant τ _rise ( Equation 6). When A _i is suppressed or the display interrupted, bias B _i decreases exponentially with time constant τ _fall ( Equation 7).  

$B_{\mathit{i}}\left(t\right)=B_{\mathit{i}}\left(t_0\right)+[1-B_{\mathit{i}}(t_0\left)\right]·[1-\exp (-\frac{T}{{\tau }_{\mathrm{rise}}}\left)\right],$

(6)

 

$B_{\mathit{i}}\left(t\right)=B_{\mathit{i}}\left(t_0\right)·\exp (-\frac{T}{{\tau }_{\mathrm{fall}}}),$

(7)

where t is time during the dominance period, t ₀ is the time at the beginning of the dominance period, and τ _rise and τ _fall are the time constants of the rise and fall, respectively. 

At the renewed onset, appearance is governed by the differential bias (when it is large) or by a fixed observer preference (when the differential bias is small). This takes into account the observers' preferred appearance which dominates the beginning of each block (Carter & Cavanagh, 2007; Hupé & Rubin, 2003; Orbach et al., 1963): 

$A_{\mathrm{onset}}=\{\begin{array}{cc}{A}_1,& \mathrm{if}(B_1-B_2)>\mathrm{\Delta }B\\ A_2,& \mathrm{if}(B_2-B_1)>\mathrm{\Delta }B\\ A_{\mathrm{preferred}},& \mathrm{if}\mid B_2-B_1\mid \le \mathrm{\Delta }B\end{array},$

(8)

where ΔB is the minimal differential bias needed to sway appearance at onset and A_preferred is the intrinsic preference of the observer. 

Given the four parameters of the model ( τ _rise, τ _fall, Δ B, and A _preferred) and the reported dominance periods for each block of trials, we can calculate the absolute and relative bias strength at the time of each display onset and thus predict onset appearance. A _preferred was determined by noting the initial appearance at the start of each block, while τ _rise, τ _fall, and Δ B were determined by fitting the model to our observations. The fraction of correctly predicted onset appearances served as a measure of the quality of fit. 

Fitting to all blocks from both experiments, we obtained the best fits for τ _rise/〈 T _dom〉 = 0.3 ± 0.5, τ _fall/〈 T _dom〉 = 31 ± 18, and Δ B = 0.02 ± 0.005, where the error ranges correspond to a fit quality of >95% of the optimal fit. Although the error ranges of individual parameters are fairly broad, the narrow and elongated shape of the parameter region providing the best fits shows that τ _fall is approximately two orders of magnitude larger than τ _rise ( Figure 8B). These values for τ _rise and τ _fall are in good agreement with recent empirical measurements (Pastukhov & Braun, 2007b), which determined the dependence of P_survival on the duration of a single, preceding dominance period (to obtain τ_rise) and on the duration of the blank period (to obtain τ_fall). In those measurements, we had obtained values of τ_rise/〈T_dom〉 = 0.4 ± 0.1 and τ_fall/〈T_dom〉 = 30 ± 10. 

Figure 8

 

Figure on model fit results. (A) Distributions of COPs predicted by the model. (B) Region of best fits.

Model predictions are in good agreement with experimental observations. The predicted distribution of the lengths of consistent onset periods (COPs) is similar to the empirical distribution, but shifted slightly toward larger lengths ( Figure 8A). The dependence of P _survival ⁱ on dominance times is quantitatively reproduced ( Figures 9A– 9F), although at times the model predicts more extreme values of P _survival ⁱ than are actually observed. This includes the dependencies on T ₂ and T ₂ − T ₁ in Experiment 1 ( Figures 9B and 9C), as well as the dependencies on T ₁ + T ₃ and T ₁ − T ₂ + T ₃ in Experiment 2 ( Figures 9D– 9F). 

Figure 9

 

Model fit results. (A–C) Model fits compared to Experiment 1 ( T ₁, T ₂, and T ₂ − T ₁). (D–F) Model fits compared to Experiment 2 ( T ₁ + T ₃, T ₂, and T ₁ − T ₂ + T ₃).

The model of Noest et al. (2007)

The postulated ‘bias’ dynamics is formally similar to the dynamics of adaptation states in recent models of multi-stable perception (Laing & Chow, 2002; Noest et al., 2007; Wilson, 2003). The difference is, of course, that our postulated ‘bias’ raises the likelihood of the associated appearance, while adaptation states are thought to lower the likelihood of an appearance. 

Recently, Noest et al. (2007) have combined ‘positive’ and ‘negative’ biases in a single model for the phenomenal appearance of interrupted ambiguous displays model. This model (henceforth termed “Noest model”) predicts series of both consistent and alternating onsets, depending on the precise timing of ON/OFF periods. In the Noest model, adaptive states fulfill a dual role: strongly adapted states exert the traditional ‘negative bias’, whereas weakly adapted states have the opposite effect and exert a ‘positive bias’ on appearance. Accordingly, the Noest model may well account for the results described here. 

To investigate this possibility, we subjected the Noest model to our stimulation protocols ( T _ON terminated after one or two reversals, T _OFF equal to the preceding T _ON) and analyzed the predicted relationship between dominance times and onset appearance. As mentioned by Noest and colleagues, the particular (linear) type of adaptive decay implemented in their model predicts only marginal effects on onset appearance for T _OFF ≫ τ _adaptation, that is, for the regime of long T _OFF investigated by us. However, this shortcoming can easily be remedied by introducing a “long-tailed” adaptive decay. As an alternative to extending the persistence of adaptation effects, we computed the predicted onset appearance for shorter T _OFF (average T _OFF equal to τ _adaptation). Noise level and τ _adaptation were fit to our observations. All other parameters were left unchanged. 

The results were disappointing ( Figures 10A– 10C). While the Noest model predicts a weak dependence of P _survival on the duration of the most recent appearance (before the interruption, Figure 10B), it does not account for the dependence on less recent appearances ( Figures 10A– 10C), which constitutes the critical evidence for a memory-like process. 

Figure 10

 

Best fit of the model of Noest et al. (2007). (A–C) Model fits compared to Experiment 1 (T₁, T₂, and T₂ − T₁).

In its current form, the Noest model would seem to face two problems with our observations. The first difficulty lies in the disparate timescales. While τ _adaptation must be comparable to 〈 T _dom〉 in order to contribute to phenomenal reversals, τ _fall must be an order of magnitude larger than 〈 T _dom〉 to produce the observed persistence of ‘positive bias’. The second difficulty is posed by persistence across several dominance periods. While adaptive states must decay within a dominance period (to permit phenomenal reversals), the positive bias must ‘linger’ over several dominance periods to account for our observations. 

A recent study (Brascamp et al., 2008) reaches essentially the same conclusion when it models the history dependence of multi-stable appearance in terms of two adaptive states with distinct time constants. 

Discussion

We investigated the stabilization of phenomenal appearance that occurs when an ambiguous pattern is presented intermittently rather than continuously (Leopold et al., 2002; Maier et al., 2003; Orbach et al., 1963). We found this stabilizing effect to depend on the relative durations of several recent dominance periods. We conclude that there exists a visual short-term memory of phenomenal appearance which is robust to intervening stimulation and which disambiguates the renewed onset of ambiguous patterns. 

A visual short-term memory

In two experiments, observers viewed an ambiguous pattern that was presented intermittently. Each presentation lasted long enough to allow spontaneous reversals of phenomenal appearance and was followed by an equally long pause. The appearance after the pause often matched the less recent appearance before the pause. The probability of this outcome increased with the dominance time of the less recent appearance and decreased with the dominance time of the more recent appearance before the pause. In short, both more and less recent appearances before the pause stabilize appearance after the pause, to a degree that depends on the recent dominance history of each appearance. 

These results suggest that, while one appearance dominates before the pause, a ‘bias’ accumulates which favors this same appearance after the pause. They further suggest that the ‘bias’ in question survives periods of stimulation dominated by the alternative appearance. The results were qualitatively and quantitatively well described by assuming that these ‘biases’ accumulate and decay independently with time constants τ _rise and τ _fall, respectively. Quantitative analysis suggests that decay rates are at approximately two orders of magnitude slower than accumulation rates, consistent with earlier observations (Pastukhov & Braun, 2007b). 

Some prior reports have noted that the appearance of an intermittently presented display may reflect a longer history. For example, Maloney, Dal Martello, Sahm, & Spillmann (2005) noted that “responses to trials before the most recent one may exert their own cumulative influence on the current trial,” while Leopold et al. (2002) remarked on the fact that the most recent appearance persists to the next onset only after having dominating for a sufficient length of time. 

Very recently, Brascamp et al. (2008) reported that a history of consistent dominance (many display periods dominated by the same appearance) leaves a stabilizing ‘bias’ that is not erased by a single interval of the alternative appearance but that continues to influence the appearance of subsequent intervals. The present results concern a more natural situation, where appearance is permitted to reverse during every display period, rather than being held constant for many successive display periods. 

The novel aspect of the present finding, therefore, is that a single dominance period builds up a stabilizing ‘bias’ which is robust to intervening stimulation. Previous observations (Brascamp et al., 2008; Leopold et al., 2002; Pastukhov & Braun, 2007a) were consistent with the possibility that each new dominance period partially or completely erases ‘biases’ from earlier dominance periods, so that only the most recent dominance period is able to influence appearance after a pause (Figure 11A). Instead, we found that a renewed stimulus onset may be dominated reliably by a less recent (but longer) rather than a more recent (but shorter) dominance period (Figure 11B). 

Figure 11

 

Phenomenal appearance and hypothetical ‘lingering bias’. Time courses of interrupted stimulation (stim), alternative appearances (solid lines A₁, A₂), and associated ‘biases’ (dotted lines B₁, B₂) are shown schematically. Two dominance periods (T₁, T₂) and the renewed onset of stimulation are marked on the time axis. (A) Appearance at renewed onset (A₂) corresponds to the last dominance period (T₂), as it has established the larger ‘bias’ (B₂). Note that B₁ decays rapidly during T₂. In this scenario, the ‘bias’ is like an iconic memory. (B) Appearance at renewed onset (A₁) corresponds to the last-but-one dominance period (T₁), which now provides the larger ‘bias’ (B₁). Note that here B₁ decays slowly during T₂. In this case, the ‘bias’ is like a short-term memory.

In view of these characteristics, the ‘biasing’ effects investigated qualify as a visual short-term memory. This term usually refers to storage of visual information that is non-permanent but long lasting and robust to subsequent stimulation (Baddeley, 2003; Brascamp et al., 2008). 

Relation to adaptation

It has long been recognized that adaptation (also called “habituation,” “satiation,” or “fatigue”) contributes importantly to perceptual multi-stability (Blake, Sobel, & Gilroy, 2003; Kohler & Wallach, 1944; Nawrot & Blake, 1989; Wolfe, 1984). Adaptation is thought to progressively weaken the currently dominant state of phenomenal awareness and to thus curtail its duration. In computational models of multi-stability (Laing & Chow, 2002; Noest et al., 2007; Wilson, 2003), the adaptive state of each phenomenal appearance is assumed to rise while the appearance is dominant and to decay while it is suppressed. Thus, there is an intriguing formal analogy between the dynamics of adaptive states and the dynamics of the memory states investigated here. 

However, the destabilizing effects of adaptation and the stabilizing effects of a visual short-term memory seem to follow disparate time courses. Although adaptation occurs on a wide range of timescales, the persistence of adaptation is generally less than its period of induction (Nawrot & Blake, 1989; Petersik, 2002; Wolfe, 1984), implying that adaptive states decay more rapidly than they build up. Computational models of multi-stability require adaptation time constants comparable to or smaller than mean dominance times, with typical values on the order of 1 s (Laing & Chow, 2002; Noest et al., 2007; Wilson, 2003). While the biophysical basis of adaptation in multi-stable displays is unclear, this timescale is consistent with afterhyperpolarization mediated by Ca²⁺-gated K⁺-channels (McCormick & Williamson, 1989). 

In contrast, memory effects in multi-stable displays persist far longer than their period of induction. Leopold et al. (2002) report undiminished persistence 40 s of the appearance of an ambiguous rotating sphere after the end of a 3 s period of induction. Elsewhere, we have documented the dynamics of memory states by presenting an ambiguous rotating sphere twice, separated by a pause (Pastukhov & Braun, 2007b). The observed time constants for persistence were far higher than for induction, with an average ratio of 88 ± 26 across observers. The quantitative analysis of our present results, which again concern ambiguous rotating spheres, confirmed this large discrepancy and yielded a ratio of 103 ± 50. Other types of ambiguous displays (Necker cube, jumping dots, binocular rivalry) present a qualitatively similar picture (Braun & Pastukhov, 2007; Leopold et al., 2002). 

The experimental situation investigated here—intermittent presentation of an ambiguous pattern—reveals either adaptive or memory states, depending on stimulus timing. While long pauses in the presentation stabilize appearance (Leopold et al., 2002; Maier et al., 2003; Orbach et al., 1963), short pauses have the opposite effect and destabilize appearance. Reversal rates peak for pause durations of 300–400 ms, exceeding the rates observed with nearly continuous presentation and for some presentation schedules phenomenal appearance alternates from one onset to the next (Kornmeier, 2002; Kornmeier et al., 2007; Orbach et al., 1963). These observations suggest that adaptive states and the associated destabilization of the recently dominant appearance persist for at least some hundreds of milliseconds into the pause. However, the destabilizing effects disappear with increasing pause duration and at 1000 ms pause duration the opposite, stabilizing effect dominates (Leopold et al., 2002; Maier et al., 2003; Orbach et al., 1963). The present study employed average pause durations of 2924 ms (range 1700 to 7700 ms in Experiment 1, 1834 to 4321 ms in Experiment 2), thus minimizing the destabilizing effects of adaptation. 

Adaptive and memory states in multi-stable perception differ not only in their decay rates. While the destabilizing effects of adaptation may be elicited with both ambiguous and unambiguous patterns (Kanai & Verstraten, 2005; Nawrot & Blake, 1989; Petersik, 2002; Wolfe, 1984), the stabilizing effects of visual short-term memory are reliably observed only with ambiguous patterns (Pearson & Clifford, 2005; Wolfe, 1984). Comparing three instances of perceptual multi-stability and two instances of binocular rivalry, we observed the stabilizing effects of viewing the ambiguous patterns but no such effects for viewing the unambiguous version of each pattern (Braun & Pastukhov, 2007). Similarly, Sterzer and Rees (2008) observed both the behavioral and neural correlates of stabilization when two images were viewed dichoptically (and observers experienced binocular rivalry) but not when the same images were alternated physically (see below). 

Possible neural basis

What could be the neural basis of a visual short-term memory that stabilizes the appearance of ambiguous displays when these are presented intermittently? A recent study has investigated the stabilization of appearance in binocular rivalry with functional brain imaging (Sterzer & Rees, 2008). The aim was to identify brain activity that persists in the absence of stimulation and that also relates to the stabilization of appearance (i.e., to instances in which the same appearance dominated before and after the pause). The authors report such activity both in visual areas and in non-visual frontal and parietal regions similar to regions typically activated by working-memory tasks (Sterzer & Rees, 2008). These results are consistent with the possibility that the stabilization of appearance involves an interaction between visual representations and working memory (Maier et al., 2003). 

Alternatively, perceptual stabilization may involve an intrinsic facilitation of visual representations. An extension of the recent model of Noest et al. (2007) may offer a plausible scenario for this possibility. This model was developed for the regime of short pauses, in which both stabilizing and destabilizing effects are observed (see above). It equates the timescales of destabilizing and stabilizing effects and thus cannot accommodate our observations on stabilization in the regime of much longer pauses (see Results section). However, if this linkage is broken and independent dynamics for destabilizing and stabilizing factors are introduced (Brascamp et al., 2008), a suitably modified model might well account for the stabilization and destabilization of appearance across all time schedules of intermittent presentation. 

Sequential dependence

The notion that multi-stable perception is a memoryless renewal process is based largely on the sequential independence of dominance times (Blake et al., 1971; Borsellino et al., 1972; Fox & Herrmann, 1967; Walker, 1975). Previous studies find unanimously that the distribution of dominance times is independent of the duration of the preceding dominance periods. This is true both for binocular and perceptual rivalry and is considered a hallmark of multi-stability (Leopold & Logothetis, 1999; van Ee, 2005). Indeed, when we consider our entire data set, we find at best a weak correlation between consecutive dominance periods. 

Intriguingly, and in contrast to all previous studies known to us, successive dominance periods are strongly and positively correlated in a subset of our data. The subset in question comprises the epochs that we have termed “consistent onset period” (COP), that is, epochs in which display onsets are consistently dominated by the same appearance. To the extent that onset appearance is governed by a visual short-term memory, as we have argued, this subset comprises periods in which memory consistently favors one appearance over the other. The sequential dependence that we observe during these periods may therefore reflect variations in the influence of memory: when memory influence is comparatively large, dominance times of the favored appearance may be consistently longer and dominance times of the disfavored appearance consistently shorter than expected. When memory influence is small, the dominance times of both appearances revert to the mean. 

In short, we find that multi-stable perception ceases to be “memoryless” exactly when a short-term memory favors one particular appearance. As dominance times reflect spontaneous reversals, this sequential dependence implies that the postulated memory of appearance influences not merely onsets but also spontaneous reversals during the continuous presentation of an ambiguous display. Although the existence of this memory is most evident with intermittent displays, such a memory may silently but importantly contribute to alternations of perceptual appearance in continuous displays. Thus, our results suggest that a short-term memory of appearance should be added to the list of processes that jointly produce reversals of perceptual appearance. 

Acknowledgments

This research was supported by the Bernstein Network for Computational Neuroscience, Germany. 

Commercial relationships: none. 

Corresponding author: Alexander Pastukhov. 

Email: [e-mail]. 

Address: Leipzigerstr. 44, 39120 Magdeburg, Germany.