Unraveling adaptation and mutual inhibition in perceptual rivalry

Abstract

When the visual system is confronted with incompatible images in the same part of the visual field, the conscious percept switches back and forth between the rivaling stimuli. Such spontaneous flips provide important clues to the neuronal basis for visual awareness. The general idea is that two representations compete for dominance in a process of mutual inhibition, in which adaptation shifts the balance to and fro. The inherent nonlinear nature of the rivalrous flip-flop and its stochastic behavior, however, made it impossible to disentangle inhibition and adaptation. Here we report a general method to measure the time course, and asymmetries, of mechanisms involved in perceptual rivalry. Supported by model simulations, we show the dynamics of opponent interactions between mutual inhibition and adaptation. The findings not only provide new insight into the mechanism underlying rivalry but also offer new opportunities to study and compare a wide range of bistable processes in the brain and their relation to visual awareness.

Introduction

Perceptual rivalry behaves like a flip-flop in which percepts remain stable for a while and then abruptly switch from one percept to the other (Wheatstone, 1838). Changes in cortical processing associated with such spontaneous alternations have been studied extensively because they may expose the neuronal basis for visual awareness (Crick & Koch, 1998). Despite many neurophysiological (Leopold & Logothetis, 1996; Logothetis, 1998; Logothetis & Shall, 1989) and fMRI (Lee & Blake, 2002; Polonsky, Blake, Braun, & Heeger, 2000; Tong, Nakayama, Vaughan, & Kanwisher, 1998) studies aimed at identifying neuronal activity engaged in rivalry, the mechanisms involved remain puzzling (Blake & Logothetis, 2002). 

One of the main reasons is that the rivalrous flop-flop is highly nonlinear and stochastic. Noise in activity levels associated with both percepts causes a random distribution of dominance intervals (Levelt, 1965). In line with the notion of a nonlinear flip-flop is the finding that interactions between rivalrous percepts are strongly asymmetrical (Levelt, 1965). Increasing the contrast in one eye may have a large effect on the duration of the percept associated with the other eye, whereas it may hardly affect durations for the higher contrast stimulus. Since Levelt formulated his second law, this asymmetry has been a hallmark in explanations of rivalry processes. Rivalry models, such as the two-stage model proposed by Wilson (2003) and Laing and Chow's (2002) spiking neural model, pass this critical test. 

The asymmetries arise from the interplay between at least two dynamic processes. On the one hand there is mutual inhibition, operating on a relatively short time scale. On the other hand there is adaptation, operating on a larger time scale. Both mechanisms are activity dependent but in fundamentally different ways. High levels of activity favor inhibition of the opposite perceptual representation. This might shorten dominance intervals for the suppressed eye and increase durations for the dominant eye. Adaptation works the other way around: high activity levels cause stronger adaptation and may therefore decrease durations for the dominant eye. Because these two processes act on different time scales, activity changes at different moments during perceptual dominance intervals are likely to affect rivalry in different ways. Thus, to understand rivalry it is important to know the time course of rivalrous interactions. 

So far, distributions of interval durations have been the main source of information on dynamics of rivalry, but clearly this does not allow one to separate dynamics of inhibition and adaptation. One obvious approach to solve this problem would be to present probes of stimulus variations at different points in time during the flip cycle (Norman, Norman, & Bilotta, 2000). This has several disadvantages: First, timing of flashes relative to the next flip is variable and cannot be controlled directly. Second, to study possible mutual interactions, multiple combinations of probes in one stimulus and the other and at different strengths must be used. Such measurements would take a long time, especially because single dominance durations normally last seconds already. 

Instead of probes, we used a reverse correlation technique, in which signal strength in both stimuli is modulated according to a random sequence. By correlating perceptual flips to signal strengths, we recover the full time course of rivalrous interactions. Moreover, we can pinpoint any asymmetries in the process by comparing correlations for the two stimuli and the two directions of flips. The reverse correlation technique described here provides the necessary extra handle to disentangle effects of adaptation, inhibition, and noise and thus sheds a new light on previous and future results. 

Methods

Here we present results for binocular motion rivalry (Alais & Blake, 1998; Blake, Yu, Lokey, & Norman, 1998; Meng, Chen, & Qian, 2004; van de Grind, van Hof, van der Smagt, & Verstraten, 2001). Random dot patterns for the left and for the right eye were presented side by side on a 19-in. CRT display, running at 100 Hz. Observers viewed the stimuli with the help of a mirror stereoscope in a dimly lit room. Great care was taken to properly adjust the viewing apertures and vergence angle for each observer to create a single dichoptic image that was easily fixated. Proper alignment of left and right eye was supported by a central fixation mark and by the contours of the patterns. Each random dot pattern consisted of 6000 bright dots on a dark background, displayed in a 2.2 degree square window (200 × 200 pixels), resulting in 15% dot density. Stimuli were generated in real time and were updated on every frame. The coherent motion was to the upper left in one eye and to the upper right in the other, at a velocity of 1.6 deg/s (1 pixel/frame). Dot positions were kept as floats and rounded towards the nearest monitor pixel for drawing. This minimizes the effect of step size differences in horizontal/vertical directions and diagonal directions for the rectangular pixel array. Orthogonal motion directions in the two eyes were chosen to minimize “monocular” directional interactions. 

The difference in motion directions in the two eyes caused clear binocular rivalry. Observers reported dominance of one motion direction or the other, with clear flips in between. Piece-meal rivalry, in which different parts of the display seem to move in different directions, was mostly limited to a wave of transition across the display. Transitions generally occurred fast compared to perceptual durations. 

To measure the time course of activity changes affecting dominance flips, we modulated the coherence of the motion around a clearly supra-threshold value. The coherence value is defined as the percentage of dots taking part in the coherent motion while the remaining dots moved in a random direction. In area MT of macaques, which is strongly related to motion perception, variations in supra-threshold coherence values cause nearly linear variations in cell-activity (Britten, Shadlen, Newsome, & Movshon, 1992). We assumed that motion detectors in our brains that are involved in binocular motion rivalry were equally well modulated by motion coherence. The noncoherent dots of the pattern moved with the same step size, but in a direction drawn from a random distribution for each dot, on each frame. This type of noise causes minimal segregation of motion and noise. To further minimize visual segregation of stimulus and noise dots, dots were given a limited lifetime of 300 ms. Dots were refreshed asynchronously on every frame. 

Temporal modulations of motion coherence do not result in clearly visible transients. On a slower time scale, observers see variations in coherence, but fast variations do not reach visual awareness. As a result, modulations do affect the process of rivalry but do not inevitably force flips. Coherence values were drawn from a uniform noise distribution on each frame (frame rate 100 Hz) and subsequently filtered with a first order low-pass filter (time constant 500 ms), which resulted in Gaussian white noise. Amplitude and time constant of coherence variations were chosen to obtain sufficient resolution in both time and coherence domains. Temporal filtering assured that stimulus energy was confined to an effective range of temporal frequencies. Results described here were obtained with a mean coherence value of 50% and standard deviations of the modulations (after filtering) of 11.5%. 

All observers had ample experience in rivalry experiments and in other motion detection experiments. Monocular motion coherence thresholds were below 5% for each observer. All observers, except MLA, were naive regarding the outcome of the experiment. They were instructed to indicate their perceptual flips by pressing arrow keys on the keyboard. The stimuli were generated in real time and were shown for as long as required to finish a session. Flips were collected in three or four sessions of about 20 min, with long periods in between. Observers were also free to rest at any time during a session. The first three flips after a break were ignored, so as to rule out start-up transients. Coherence modulations and key presses were stored at a resolution of one frame for off-line analysis. 

Figure 1 shows an example of noise modulations for both stimuli and explains the analysis procedure. Flips from left to right and from right to left were used to calculate peri-flip motion strength correlation functions. These correlations describe sensitivity at each point in time preceding flips for coherence changes in one eye and in the other eye. This yields four correlation functions (Figure 2): two for averaged modulations while the right eye percept was dominant and two for left eye dominance. To avoid confounding effects of previous flips, we averaged for one complete interval only, plus a period of 2 s following flips. Figure 2b shows the cumulative distribution of interval durations. Mean interval duration for observer MLA was about 5 s. The number of averaged noise traces thus declines rapidly for increasing interval duration. As a result, the data become less reliable for long intervals. 

Figure 1

 

Time course of motion coherence values for left and right eye. A new coherence value was drawn from a uniform distribution on each frame (every 10 ms) and then passed through a single first order low-pass filter with a time constant of 500 ms. This resulted in a Gaussian distribution of coherence values with a standard deviation of 11.5% around the mean of 50%. The triangular symbols indicate the observers' reports of flips from left to rightward motion and vice versa. The red and blue interrupted lines show the resulting dominance intervals.

Figure 2

 

Peri-flip correlation functions (a), constructed by averaging the noise traces corresponding to separate dominance intervals, as indicated by the interrupted lines in Figure 1. This results in four flip-triggered averages: two for noise in the dominant stimulus (red and black curves) and two for noise in the suppressed stimulus (green and blue). (b) The number of intervals used in averaging each 10-ms time bin. The number decreases for longer intervals relative to the flip, and hence the variability in the averages increases. Data shown are for observer MLA.

Results

Results for five observers are summarized in Figure 3. The first column shows the distribution of interval durations. Observers differ in the parameters of the distribution; both in mean interval duration and in the shape of the distribution. Especially observer MLO flips more frequently than the other subjects, a finding consistent with results in other rivalry tasks. The lack of long intervals for this observer renders the correlations at times before about −3 s less reliable. Whereas the parameters of the interval distributions are the main source of information in other studies and have been analyzed in great detail, we only focus on the reverse correlation functions. 

Figure 3

 

Flip-triggered noise averages for different observers. The left-hand column shows the distributions of interval durations for left (red) and right eye (blue). Data were collected for 250–450 leftward and rightward flips. The right-hand column shows results for stimuli in the suppressed eye (black) and dominant eye (green), with coherence values on the left-hand axis. The thin red and blue lines show the number of intervals used in averaging (right-hand axis). Confidence intervals correspond to ±2 standard errors of the mean.

Because the two ipsilateral curves (noise for the dominant stimulus) and the two contralateral curves (suppressed stimuli) were very similar, we averaged the two. The second column in Figure 3 shows the averaged coherence values for noise modulations in the suppressed and dominant eye. The most surprising finding is that the correlation functions show a biphasic profile. Directly preceding a flip, there is a period of about 1 s in which increments in the suppressed eye and decrements in the dominant eye favor the occurrence of flips. At longer delays, about 1–6 s before flips, modulations have the opposite effect. This opposite effect is smaller in amplitude but lasts considerably longer. Despite the large differences in dominance interval durations, all observers show qualitatively the same result. 

Effects for the dominant eye are opposite to those for the suppressed eye. Closer examination, however, shows that the effects are not mirror images. Typically, the short latency phase is more pronounced for the suppressed eye than for the dominant eye. An increment in the suppressed eye is more effective than a decrement in the dominant eye. 

Modulation amplitudes differ between observers. Interestingly, the modulation amplitude seems to correlate positively with the mean interval duration. Observer MLO flips at a high frequency and shows a small modulation amplitude, whereas RB has the longest dominance durations and shows the highest modulation amplitude. MLA, JD, and RvW are intermediate, both in the amplitudes of the modulation functions and in the value of the mean interval duration. 

The difference in modulation amplitudes for the dominant and suppressed stimuli covaries with the overall amplitude of the modulation functions. It is especially strong for observers showing a relatively small effect. For observer MLO, the negative modulation for the dominant eye is nearly absent, whereas for RB, who shows the largest effect, this asymmetry between dominant and suppressed stimuli is much smaller. The effects for dominant and suppressed stimuli not only differ in amplitude but also have clearly different time courses. Sensitivity for variations in the dominant eye change gradually during the flip cycle, whereas the function for the suppressed eye shows a fairly abrupt change from a negative to a positive value. 

Model simulations

These results indicate that rivalry is a complex interaction between (at least) two processes, with different time constants. As a result, flip-triggered noise averages show a biphasic profile. At short latencies relative to flips, the effect is consistent with mutual inhibition, at longer latencies the sign of the effect reverses and is consistent with an adaptation effect. This seems counterintuitive at first but makes sense considering differences in dynamics between adaptation and mutual inhibition. To gain further insight in the interactions between adaptation and mutual inhibition underlying the flip-triggered averages, we performed a simple model study in which we simulate the experiments described above. 

The sole aim of the model simulation was to show that a minimal model already shows the behavior we observed, indicating that it reflects a key property of rivalry mechanisms. Other constraints will surely require more sophisticated models but that is of no concern here. 

Figure 4 shows a diagram of the rivalry model, which is a simplified version of previously proposed rivalry models (Lehky, 1988; Matsuoka, 1984; Wilson, 2003). It consists of two processing streams (or “neurons”), one for the left eye and one for the right eye. Each neuron receives the coherence level as input. In the model, processing for left and right eye is similar and to a high degree independent, except for the mutual negative feedback. Signals in each stream pass adaptation, a filter stage, and a compression stage. A Weber-type of adaptation is implemented by dividing the input by the sum of fixed constant, c, and a low-pass filtered version of the input. The time constant for adaptation, τ_ad, determines to a large extent the mean duration of dominance intervals. It was set to 1 s. To account for low temporal resolution and long temporal integration times in motion coherence detection, we included a low-pass filter with a time constant τ_i of 500 ms. The low-pass filtered activity level is converted into a final output by a nonlinear, Naka–Rushton type of compression stage (

$x=x^n/x^n+c$

), where c (set to 0.5) determines the position and n (set to 2) determines the width of the operating range. The final comparison stage reports a flip whenever the balance of responses reverses. The feedback is characterized by a gain factor (set to 96). An additional filter stage was included in the feedback path but its time constant, τ_fb, was short (20 ms) relative to the other time constants and therefore less influential. The model contains no intrinsic noise sources, and no response latency between flip detection and “button press” was included. 

Figure 4

 

Model diagram. The model consists of two streams, one for the right and one for the left eye. The output provides negative feedback to the contra lateral side. The input (stimulus plus feedback) passes an adaptation stage in which the input is divided by a low-pass filtered (time constant τ_ad) version of the input (plus a fixed constant C). The adapted signal is low-pass filtered (time constant τ_i) and passed through a nonlinear, Naka–Rushton type of compression stage. A “homunculus” compares left and right eye outputs and reports a flip whenever the balance of responses reverses.

The model was simulated numerically, feeding it the same coherence modulations that were used in the psychophysical experiments and using the same procedures for analysis. Model parameters were roughly adjusted by hand to fit the results qualitatively. No attempt was made to quantitatively match the data or to build a model that explains a large body of other experimental findings. Figure 5 shows the rivalrous model behavior. Mutual inhibition in combination with adaptation results in an unstable flip-flop in which dominance regularly flips from one eye to the other. Note that we did not introduce an internal noise source either, which resulted in a quite narrow spread of interval durations. 

Figure 5

 

Example of output signals for left and right eye.

Yet, this simple model reproduces many of the characteristics found in the real data (see Figure 6). Profiles are biphasic and show a similar asymmetry for modulations in the suppressed and dominant stimuli. The timing of correlations obviously deviates from the psychophysical data: the peak correlation occurs at

$t=0$

because no response latency was introduced. Thus, the difference in latency between model and real data reveals the response latency between the occurrence of internal (low-level) flips and the time of button presses. Peak latencies for different observers varied from 380 to 620 ms (mean 560 ± 127 ms,

$n=5$

). The zero crossings for the nonsuppressed signals were very consistent across observers (between 1600 and 1700 ms before button presses). Zero crossings for the suppressed signals showed a similar variation in latency as the positive modulation peaks: varying between 900 and 1350 ms. 

Figure 6

 

Simulation results. Flip-triggered averages of coherence values in the dominant eye (black curve) and nondominant eye (green curve). The simulation results are shown in the same format as the data in Figure 3.

Discussion

We have shown that dynamic interactions between rivalrous stimuli can be studied using a reverse correlation technique. Flip-triggered noise averages reveal the balance between adaptation and mutual inhibition and its dynamic change during the rivalry cycle. The important conclusion is that biphasic behavior has to be taken into account in perceptual rivalry studies in which signal strengths are varied. Changing input signals at different moments may result in opposite effects. 

The biphasic behavior is not an artifact of the reverse correlation procedure. Although the time average of the noise trace by definition equals the mean coherence value, only part of the noise, determined by the time of flips, is analyzed. Most importantly, each signal is based on only half of the total time, as determined by the dominance flips. The flip-triggered averages thus do not necessarily sum to the mean noise value. A comparison of model results and psychophysical results shows that the slow decay of the traces after the peak of the correlation function must be due to the temporal correlation in the noise signals. The model peaks sharply at time

$t=0$

, after which modulations cannot affect the flip anymore. Thus, decay after the peak must be due to temporal correlations. This correlation is due to low-pass filtering of the external noise. It should be noted that this slow decay might also contribute to modulations in the earlier part of the correlation functions. For the shorter interval durations, these deflections fall within the time window over which signals for the next flip are being averaged. Further model simulations, in which we skipped the first 1.5 s after flips in the analysis, however, showed similar results. For the experimental data, the effect is even smaller because the slow decay is almost completely finished at the time of button presses. Omitting 1.5 s after button presses from the analysis slightly increased the variance for long latencies but otherwise gave indistinguishable results. 

We used independent modulations of signal strength for the two eyes, which allowed us to study the asymmetries in this process as well. Our finding therefore sheds new light on the asymmetries in rivalry processes. Levelt's (1965, 1967) pioneering work clearly showed asymmetrical contrast effects in the dominant and in the suppressed eye. This finding has been a cornerstone for numerous psychophysical and modeling studies (Matsuoka, 1984; Mueller, 1990; Stollenwerk & Bode, 2003; Wilson, 2003). The biphasic profiles that we measured suggest that variations of fixed, continuous levels may largely average out the underlying dynamic effects. Asymmetries are not a static property; instead, our data show how the asymmetry arises from opposite effects during different phases of the rivalry cycle. 

Variations on the method as presented here open many new opportunities for studying dynamics and asymmetries in both monocular and binocular rivalry processes. Similar experiments could, for example, be performed for luminance contrast variations of rivalrous gratings in the two eyes. One can thus study and compare the dynamics of different rivalry processes directly, rather than indirectly comparing interval distributions. The method also provides a precise estimate of the time between the “internal reversal” and the moment of button presses. Differences in the time required for different rivalry mechanisms to reach visual awareness might thus be characterized. 

The reverse correlation method allows separating dynamic effects resulting from dominant and suppressed signals. This opens the opportunity to link, for example, cognitive manipulation of rivalry in much more detail to changes in the underlying mechanisms. Reverse correlation in combination with a cognitive manipulation task may very well reveal how observers are able to adjust sensitivity for the different stimuli. 

Observers clearly differed in their sensitivity to external noise. RB very much behaves like the noise free model. He shows a large amplitude effect, with relatively narrow confidence intervals. MLO on the other hand shows much smaller modulation amplitudes. The fact that RB has a long mean interval duration and MLO a very short mean interval suggests that this variation might correspond to different levels of internal noise. It would also imply that one could study the internal noise source by varying the properties of the external noise. By measuring noise thresholds for different temporal properties of the external noise, we might be able to measure the temporal frequency content of the internal noise. 

Conclusion

We conclude that one should take the dynamics of adaptation and inhibition into account in linking stimulus manipulations and noise sources to interval distributions. Reverse correlation provides an extra handle to unravel the different processes involved in perceptual rivalry. 

Acknowledgments

I am grateful to Andre Noest for inspiring discussions. 

Commercial relationships: none. 

Corresponding author: Martin J. M. Lankheet. 

Email: [e-mail]. 

Address: Padualaan 8, 3584 CH Utrecht, The Netherlands.