Which image is in awareness during binocular rivalry? Reading perceptual status from eye movements

Abstract

Binocular rivalry is a useful psychophysical tool to investigate neural correlates of visual consciousness because the alternation between awareness of the left and right eye images occurs without any accompanying change in visual input. The conventional experiments on binocular rivalry require participants to voluntarily report their perceptual state. Obtaining reliable reports from non-human primates about their subjective visual experience, however, requires long-term training, which has made electrophysiological experiments on binocular rivalry quite difficult. Here, we developed a new binocular rivalry stimulus that consists of two different object images that are phase-shifted to move in opposite directions from each other: One eye receives leftward motion while the other eye receives rightward motion, although both eyes' images are perceived to remain at the same position. Experiments on adult human participants showed that eye movements (optokinetic nystagmus, OKN) are involuntarily evoked during the observation of our stimulus. We also found that the evoked OKN can serve as a cue for accurate estimation about which object image was dominant during rivalry, since OKN follows the motion associated with the image in awareness at a given time. This novel visual presentation technique enables us to effectively explore the neural correlates of visual awareness using animal models.

Introduction

When completely different images are presented to the two eyes, they compete for perceptual dominance, such that only one image is consciously perceived at a time, with the dominant image alternating between the left and right eye images every few seconds. This visual phenomenon, referred to as binocular rivalry, has been used as an effective tool for investigating the neural correlates of visual awareness, since the fluctuation in perception is dissociated from changes in physical stimulation. 

Human fMRI studies (Lumer, Friston, & Rees, 1998; Tong, Nakayama, Vaughan, & Kanwisher, 1998) show that BOLD responses in higher visual cortical areas follow the alternation of perception, rather than retinal input, during binocular rivalry. Furthermore, recent studies indicate that even the BOLD signal in V1 (Polonsky, Blake, Braun, & Heeger, 2000; Tong & Engel, 2001) and LGN (Haynes, Deichmann, & Rees, 2005; Wunderlich, Schneider, & Kastner, 2005) is modulated depending on the perceptual alternations as well as the physical alternation of monocular inputs. One of the drawbacks of human fMRI studies, however, is that BOLD signals are not a direct measure of neural activity but are hemodynamic responses that accompany neural activity (Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001). The slow time course of hemodynamic responses makes it difficult to address the temporal dynamics of neural activity that are critical to understanding the control mechanisms of perceptual alternation in binocular rivalry. Although there are a few electrophysiological studies that have recorded spike activity from the temporal cortex of epilepsy patients (Kreiman, Fried, & Koch, 2002), thorough experiments remain to be conducted because human electrophysiology is limited to clinical patients. 

Alternatively, electrophysiological studies in non-human primates can directly measure where neural activity related with binocular rivalry occurs with a precise time course. Single-unit recordings in awake behaving monkeys have revealed that only a small percentage of neurons in V1/V2 (18%) show activity modulation depending on when their preferred stimulus is dominantly perceived during binocular rivalry (Leopold & Logothetis, 1996). This percentage increases as one ascends the hierarchy of cortical areas: approximately 40% in intermediate cortical visual areas (MT and V4) and approximately 90% in downstream cortical visual areas such as IT (Leopold & Logothetis, 1996; Sheinberg & Logothetis, 1997). These results suggest that neural competition during binocular rivalry is not fully resolved until higher visual areas, which stands in contrast to the findings of human fMRI studies in lower visual areas. Although the neural mechanisms of binocular rivalry could potentially be clarified through electrophysiology studies using non-human participants, non-human binocular rivalry experiments have an inherent problem: Participants must voluntarily report which eye's image is perceived at any point in time. 

Reliable reports on subjective experience are difficult to obtain from non-human animals (this will be also the case for human infants who will be needed to study the developmental aspects of binocular rivalry). The aim of the present study is to propose a new visual presentation technique that enables us to know one's perceptual state from an involuntary response. To this end, we conducted psychophysical experiments using human participants who can voluntarily report their visual experience and investigated involuntary responses that are highly correlated with the reports. Developing such a visual presentation technique is important not only for promoting binocular rivalry studies using animal models and/or infants but also to do human experiments based on more objective measures than participants' reports, which can fluctuate due to changes in criteria. 

Optokinetic nystagmus (OKN) is an eye movement response elicited reflexively by visual field motion, which alternates between the slow-phase movements that occur in the same direction as the field motion and the fast-phase (saccade-like) eye movements that jump toward the opposite direction to reset the eye position. When opponent motions are presented to the two eyes as rivalry stimuli, the direction of the slow-phase movement of the OKN corresponds to the direction of the visual motion dominantly perceived at the time. Thus, OKN can be used as an objective indicator of subjective motion perception during rivalry without requiring a voluntary report. This finding was first reported by Enoksson (1963) and then intensively investigated by Fox, Todd, and Bettinger (1975). Decades later, Logothetis and Schall (1990) showed that this measure works with non-human primates as well (see also review by Leopold, Maier, & Logothetis, 2003). OKN is also a good indicator of observer's experience in other type of bistable motion perception, such as transparent motion (Niemann, Ilg, & Hoffmann, 1994; Watanabe, 1999; Wei & Sun, 1998). The use of OKN for binocular rivalry research, however, has been limited to the studies of visual awareness of motion since the stimuli available so far have consisted of moving gratings or random-dot patterns. There has been no method that utilizes OKN for studying the visual awareness of object recognition, i.e., a function processed along the ventral stream (also known as the “what” pathway) that is usually investigated using stationary images, as opposed to motion processing in the dorsal stream (or “where” pathway; Goodale & Milner, 1992; Ungerleider & Mishkin, 1982; Ungerleider & Pasternak, 2003). 

To evoke OKN as an indicator of perceptual status during rivalry, the monocular images of stimuli have to continuously move in opposite directions from each other. On the other hand, the monocular images are required to remain at the same position to make the perception of object image constant and to keep the stimulus within the receptive field of a recorded neuron if applied in electrophysiological studies. In the present paper, we describe how to present a binocular rivalry display that can compromise these apparently contradicting requirements using motion stimuli generated by phase-shift operation, also known as quadrature motion (Carney & Shadlen, 1993; Hayashi, Miura, Tabata, & Kawano, 2008; Hayashi, Nishida, Tolias, & Logothetis, 2007; Shadlen & Carney, 1986). Then, we will show that OKN changes its direction depending on the motion direction associated with a dominantly perceived object image in our new stimulus. 

Methods

A method generating a new binocular rivalry stimulus

We developed a new method using phase-shift operation to create rival targets that consist of object images continuously moving in one direction while they are perceived to remain within the same spatial location as shown in Figure 1: An object image is Fourier transformed and the phase of all spatial frequency components is shifted by a fixed amount. Then, the inverse Fourier transform of the phase-shifted spatial frequency pattern was calculated to generate a new image for the next frame. Such operation makes all spatial frequencies move in one direction. (The motion signal generated by the phase-shift operation is first-order motion (Lu & Sperling, 2001), which is defined as the change in luminance, thus has motion energy (Adelson & Bergen, 1985; Watson & Ahumada, 1985). Contrary to customary position-shift motion stimulus, phase-shift motion stimulus contains multiple velocities since broad range spatial frequency components are temporally modulated at a frequency determined by the cycle of the phase shift). On the other hand, since the same sequence of images is cyclically presented and the positional change of the high spatial frequency components that provide edge information of the object image is small, the pattern of the image is perceived to stay at the same location. We generated two phase-shift motion stimuli originating from two different object images, which moved in opposite directions, and presented them to the left and right eyes dichoptically as binocular rivalry stimuli (see also Movie 1). 

Figure 1

 

Schematic of the method used to generate binocular rivalry stimuli: The left eye image is an upright human face and phase-shifted by a fixed amount (−pi/2 in this picture) every frame. The image returns to the first image, when the accumulated amount of phase shift reaches 2pi. The sinusoidal grating patterns in the left column are caricatures that illustrate the changes of individual spatial frequency components of the face image (the upper half of the image depicts the shift of a high-frequency component and the lower half of the image depicts those of a low-frequency component), showing that the phase-shift operation produces continuously moving leftward motion. On the other hand, the right eye image is the inversion of the left eye image and phase-shifted by the opposite amount every frame (pi/2 in this picture); thus, the right eye receives continuously moving rightward motion. As indicated in the left-hand column, although the phase-shift operation generates motion continuously moving in one direction, high spatial frequency components, which compose edge information of the face image, do not change position very much; thus, the face image is perceived to stay at the same position. The pictures shown on the third frame illustrate how the contrast polarity of images alternates in our stimuli when the accumulated amount of phase shift reaches pi. The phase shift amount for each frame used in the actual experiments was pi/12. The left eye images were 8-bit images of red color, while the right images were those of green color and two images were dichoptically presented using red–green color filter in our experiments.

Experimental procedure

In the following experiments, we tested in adult human participants whether OKN changes its direction depending on the change of dominantly perceived image in the proposed binocular rivalry stimulus. We used the upright and upside-down images of an identical human face as monocular inputs in order to minimize the difference in low-level visual features between the two eyes (mean luminance, contrast, distribution of spatial frequency, etc.) and to explore the effect of configural information of the face on rivalry, which is processed in specialized cortical visual areas in the temporal cortex along the ventral stream (Freiwald, Tsao, & Livingstone, 2009; Kanwisher, Tong, & Nakayama, 1998). Additionally, physiological studies and developmental studies on OKN have indicated that direct input from subcortical areas provides velocity signals only for temporonasal motion, whereas the cortical input supplies velocity signals for both temporonasal and nasotemporal motion (Braddick, 1996; Hoffmann, 1981). Therefore, we also investigated the nasotemporal asymmetry of OKN in our stimulus to gain insight into whether or not OKN contingent upon binocular rivalry is driven by cortical signals. 

Experiment 1

In the first experiment, as for control experiments, we examined whether continuous phase-shift motion stimuli can elicit OKN and, if so, whether the slow-phase movement of OKN flips its direction when the monocular inputs of opponent motion were physically alternated. A face image was arbitrarily chosen from 20 images of different individuals (from the face database provided by the Max Planck Institute for Biological Cybernetics; Troje & Bulthoff, 1996) whose mean luminance and contrast were adjusted to be equal across different faces. The image in front of the left eye was an upright face that moved either leftward or rightward, while the right eye image was an upside-down face that moved in the opposite direction. Each eye's stimulus was then presented monocularly in turns for 30 s: The moving face image was presented to either left or right eye while the blank image of the same mean luminance was presented to the other eye. After a random interval (2, 3, 4, 5, or 6 s), the opponent image was presented to the other eye while a blank image was presented to the first eye. The physical alternation of monocular inputs that mimicked the perception of binocular rivalry continued to the end of trial. We tested four conditions (two motion directions (leftward or rightward) × two presentation orders (left eye first or right eye first)) and each condition was repeated 10 times, in random order. The participants were instructed to report which image was being perceived by pressing one of two buttons during stimulus presentation. The eye movements of the left eye were recorded simultaneously using an eye tracker (EyeLink CL, SR Research) with a sampling rate of 1000 Hz. Participants were also required to maintain their eye position around the center of the screen. We did not set an explicit eye window during the experiments, but the data at the time when eye blink occurred or the eye position deviated from the center by 10 deg were eliminated from the post-experiment analysis. Visual stimuli (256 × 256 pixel image), which were viewed 57 cm away from the CRT monitor, subtended 15.7 deg × 15.7 deg on the screen. Monitor refresh rate was fixed at 100 Hz and the cycle of phase-shift operation was 24 frames, thus the contrast polarity of each image was alternated every 120 ms in our stimuli. The size of the stimulus and the cycle of phase shift were first adjusted by the author (RH) in order to (1) elucidate OKN of sufficient amplitude and (2) not to cause the piecemeal rivalry (Blake, O'Shea, & Mueller, 1992; O'Shea, Sims, & Govan, 1997) before running the experiments with naive participants. The stimulus size of 15.7 deg and the cycle of the phase shift (4.1 Hz in temporal frequency) used in the following experiments are within the optimal range for elucidating OKN reported by Schor and Narayan (1981). All naive participants reported that under this fixed parameter condition they mostly perceived a single image during the stimulus presentation except for the transition from one-eye image to the other (see also the later discussion about the optimization of the stimulus parameters). We used the anaglyphic method with red–green filters (Kodak gelatin filter no. 25 and no. 58) for dichoptic presentation of the stimuli. The luminance of the screen through color filters was gamma corrected for both eyes and the mean luminance was adjusted to 3.57 cd/m² (measured with CS-100A, Konica Minolta). 

Participants

Four adult participants (one was author RH) whose visual acuity was normal or corrected to normal participated in the first experiment and five adult participants including RH participated in the second and third experiments. We obtained written informed consent from all participants involved in our study. All experiments were conducted in accordance with the principles embodied in the Declaration of Helsinki (code of ethics of the World Medical Association) under the approval of the ethics committee of RIKEN Brain Science Institute. 

Analysis

In order to quantitatively evaluate the correspondence between the flip of OKN and reported perceptual alternation, we applied the following simple filter to the recorded eye movement responses: We first calculated the sign of the moving average of the sign of eye velocity within a 300-ms time window, then smoothed the output using a 1-s time window to estimate which direction the slow phase of OKN moved at each time point. Next, the cross-correlation between the button press reports and the filtered outputs was calculated to estimate the optimal time shift that compensated the time delay between button press action and OKN switch. Finally, the filtered output was discretized to [1, 0, −1] under the threshold of ±0.5 to reproduce the button press reports on perception. Button press responses and filtered outputs were sampled every 1 ms from 3 s after stimulus onset to stimulus offset. The matching index was defined as the percentage of the number of time points within which the filtered output matched the actual button press reports if the filtered output was non-zero, i.e., the eye movement showed clear OKN behavior. We used the average of the sign of eye velocity, instead of eye velocity itself, because our stimulus includes multiple velocities, thus eye velocity during the slow phase of OKN was not constant. Moreover, the velocity of the fast-phase eye movement would be another artifact if velocity itself were used. We found that the average of the sign of the eye velocities is a simple but robust method to these artifacts, providing very reliable estimation of the direction to which slow-phase eye movement drifted. Although our stimulus contains various velocities, phase-shift operation generates motion moving in the same direction across all spatial frequencies. This is in contrast to the motion generated by position shift of broadband stimulus, such as missing fundamental stimulus; the missing fundamental stimulus is generated by subtracting the fundamental Fourier components from square-wave grating whose position is shifted every frame by the quarter wavelength (Adelson & Bergen, 1985). In missing fundamental stimulus, the third, seventh, and 4n − 1 harmonics are shifted by −pi/2 in phase, while the fifth, ninth, and 4n + 1 harmonics are shifted by pi/2 in phase; thus, there are conflicts in motion direction across different spatial frequencies. 

Experiment 2

Stimuli and procedures used in Experiment 2 were the same as those used in Experiment 1, except stimuli were presented as binocular rivalry stimuli: Both eyes' images were dichoptically presented for 30 s. Additionally, the image to the left eye was either an upright face or an upside-down face that moved either leftward or rightward, while the right eye image was the inversion of the left eye image that moved in the opposite direction. Therefore, four conditions (two types of stimuli (upright or upside down) × two motion directions (leftward or rightward)) were tested and each condition was repeated for 10 trials in random order. Participants (n = 5) were asked to report which image was dominantly perceived by pressing buttons during stimulus presentation. The period when the participants pressed no button or pressed two buttons simultaneously (i.e., the participants could not decide to report either image or reported the two images due to the mixed perception of the two images) were eliminated from the analysis. The eye movement responses were recorded and analyzed in the same way as in Experiment 1 to test whether the participants' reports on the perceptual alternation during rivalry can be reproduced from the eye movement responses. 

Experiment 3

Stimuli used in Experiment 3 were the same as those used in Experiment 2, except that the binocular rivalry stimuli were presented for only 3 s and participants (n = 5) were instructed to report the dominant image at the very end of the stimulus presentation. Fifty trials were repeated for each condition. Then, the sign of eye velocity around the stimulus offset within a 300-ms time window (from 2850 to 3150 ms) was analyzed to compare with the participants' reports. 

Results

Experiment 1 (monocular alternation condition)

First, we examined whether continuous phase-shift motion stimuli can elicit OKN and, if so, whether the slow-phase movement of OKN flips its direction when the monocular inputs of opponent motion were physically alternated. In the first experiment, the monocular inputs (one eye received a phase-shifted motion stimulus, while the other eye received uniform blank input) were flipped every several seconds at random intervals. Participants (n = 4) were asked to report which of two images (upright or upside-down face) was perceived by pressing buttons during stimulus presentation. Figure 2 depicts a temporal profile of the eye movement response recorded from one participant during the observation of physical alternation of phase-shifted motion. The blue line in the first row indicates eye position and the black dashed line is the profile of the button press reports (positive is the button press report when the image moving rightward was dominantly perceived and negative is that when the image moving leftward was dominantly perceived. Zero indicates the period when the participants pressed no button or pressed two buttons simultaneously). As indicated in the top row of Figure 2, OKN was evoked by phase-shifted motion, similar to previous studies using position-shifted motion (Niemann et al., 1994; Watanabe, 1999; Wei & Sun, 1998): the time plot of eye position showed a sawtooth-like pattern, alternating slow-phase movements that followed the phase-shifted motion and fast-phase movements in the opposite direction. In addition, the direction of the slow-phase movement of the OKN flipped when the monocular input was flipped to change the direction of motion. The orange line in the second row of Figure 2 indicating eye velocity also shows the flip of the eye movement direction consistent with the physical change of motion stimulus. To evaluate how well the direction of the slow-phase movement of OKN corresponded to the button press report of perceived motion, we developed a simple filter that calculates the average of the sign of eye velocity within a moving time window. Since slow-phase responses usually last longer than fast-phase responses, summing the sign of eye velocity within a time window is expected to provide good estimation of which direction the slow-phase responses were moving at the time. The red line in the third row of Figure 2 indicates the outputs of this filtering. The results show that the filtered eye movement responses matched very well with the button press reports (black dashed line). The green line in the fourth row of Figure 2 shows the discretized output of the filtered eye movement responses, reproducing the button press responses (black dashed line) very well. We defined a matching index that quantifies in how many of the time bins the button reports corresponded with those reproduced from eye movements (see Methods section for details) and found that the mean matching index across all four participants was quite high (99.8%). The mean matching index was not statistically different between the conditions when the two eyes' motions moved in a temporonasal direction and when they moved in a nasotemporal direction (paired t test: p = 0.175). It is noteworthy that the mean time delay introduced to obtain the optimal matching index was 224.0 ms (std = 65.0 ms), which means the flip of the OKN tends to precede button press reports. It is considerable that this delay reflects the longer reaction time for voluntary button press response compared with the reflexive eye movement. 

Figure 2

 

Time plots of eye movement responses (blue line) and button press reports (black dashed line) from a representative participant when monocular inputs of opponent phase-shift motions were physically alternated. The horizontal axis indicates time and the vertical axis indicates eye position. The zero point in the vertical axis is the location of the fixation point shown at the center of the screen and positive values represent eye positions to the left of the center. As for the black dashed line (the button press reports), positive means that the participant reported the image moving leftward and negative means that the participant reported the image moving rightward. Zero means that no button was pressed or two buttons were pressed simultaneously. The orange line in the second row indicates eye velocity. The red line in the third row is the output of the filter that calculated the average of the sign of eye velocity, which provides a good estimation of which way the slow phase of OKN was moving (positive value means that the eye followed leftward motion and vice versa. See Methods section for details). The green line in the bottom row is the discretized filter output.

Experiment 2 (binocular rivalry condition)

In the second set of experiments, we tested whether binocular rivalry with our new stimulus can elicit the spontaneous alternation of OKN in correspondence with the perceptual alternation. All participants (n = 5) who observed our new binocular rivalry stimulus experienced the fluctuation of awareness between the left and right eye images, as expected. Furthermore, although the contrast polarity of our stimulus alternated every half-cycle (i.e., 120 ms in our experiments) due to the phase-shift operation (see Figure 1), a single phase of perceptual dominance usually lasted for several seconds, thus spanning multiple alternations of contrast. Figure 3 shows the behavioral results recorded from three participants during their observation of the binocular rivalry stimulus. As in Experiment 1, blue lines in the first row indicate the eye position over time, and the black dashed lines are the button press reports about the dominantly perceived image. OKN was elicited even by the dichoptic presentation of opponent phase-shift motions, and the direction of the slow-phase movement of OKN flipped when the dominantly perceived image had spontaneously flipped. The orange lines in the second row of Figure 3, indicating eye velocity, also show the flip of the eye movement direction consistent with the perceptual alternation of the image in awareness. To quantify how well the flip of OKN corresponded to that of visual perception, we applied the same filtering analysis as in the first experiment and calculated the matching index between the actual button press reports of the dominant image and those reproduced from filtered eye movement responses. Red lines in the third row of Figure 3 are the output of the filtering, which corresponds very well with the button press responses. Figure 4A shows the matching index between the participants' reports and those reproduced from eye movements from all five participants. The left bar is the condition when the two images moved in the temporonasal direction, while the right bar is the condition of the nasotemporal direction. The mean matching index was 91.7% and 95.4% for the temporonasal and nasotemporal conditions, respectively (the mean of two conditions was 93.6%), and their difference was not statistically different (paired t test: p = 0.168). We also calculated the correlation coefficient between the button press reports and the filtered eye movement responses and found that the correlation was also quite high for the two conditions (0.94 for temporonasal condition and 0.95 for nasotemporal condition). There was no statistically significant effect of face orientation (upright vs. inverted) on the duration of the dominant phase (mean ± std: 3.489 ± 2.290 s vs. 3.353 ± 2.248 s, paired t test: p = 0.106) and on the reproduction of perceptual alternation from eye movement (92.5% vs. 93.2%, paired t test: p = 0.453). However, we observed a slight but statistically significant difference (paired t test: p = 0.045) in the matching index of the reproduction if we compared the condition when the recorded eye's (the left eye) motion was nasotemporal motion (leftward motion) and OKN followed the nasotemporal motion (96.6%) with the condition when the recorded eye's (the left eye) motion was temporonasal motion (rightward motion) and OKN followed the temporonasal motion (88.7%). The poorer performance in the temporonasal condition could be accounted for by the idea that alternation of OKN that corresponds to the perceptual alternation in rivalry is driven by the cortical motion signal, while the temporonasal motion signal that directly passes through subcortical areas interferes with the rivalry-dependent modulation of the cortical motion signal. The best matching index and/or correlation coefficient between the button press reports and the filtered eye movement responses was obtained when the time lag of 448.2 ms (std = 74.9 ms) was introduced. This means that the flip of the OKN tends to precede button press reports. It is noteworthy that the time delay between the flip of the OKN and the button press response is larger in binocular rivalry condition than the time delay in Experiment 1, in which the participants were expected to press the button as soon as possible when monocular inputs were physically switched. One possible explanation for the extended time delay is that the flip of perception in the binocular rivalry condition may be triggered by the eye movement change. Another explanation is that making the decision to report the perceptual flip is delayed because the perceptual flip does not necessarily occur instantaneously, but the perceived image is sometimes ambiguous: For some periods, participants experienced the mosaic patch-wise mixture of the two eyes' images, which made it difficult to report either eye's image as dominant. Regardless of the cause of the precedence of the OKN changes, the results indicate that OKN flips its direction depending on what the participants see rather than whether monocular inputs were physically flipped or dichoptically presented. Therefore, OKN provides an objective behavioral index of which one of the two eyes' images is perceptually dominant at a time. 

Figure 3

 

Time plots of eye movement responses (blue lines) and button press reports (black dashed lines) from three participants while viewing binocular rivalry stimuli. Each column corresponds to the data from each participant (TS, RH, YM). The horizontal axis indicates time and the vertical axis indicates eye position for the blue lines. As for the black lines, positive values on the vertical axis indicate that the image moving leftward was dominantly perceived and negative values indicate that the image moving rightward was dominantly perceived. Zero means that no button was pressed or two buttons were pressed simultaneously. The orange lines in the second row indicate eye velocity. Red lines in the third row are the outputs of the filter that provide the estimation about which direction OKN moved (see Methods section for details) and correspond very well to the button press reports on the dominant image. The green lines in the bottom row are the discretized filter output.

Figure 4

 

(A) The matching index between the participants' reports on the dominant image during binocular rivalry and those reproduced from eye movement responses. The matching index was plotted for conditions when the two eyes' images moved in a nasotemporal direction and in a temporonasal direction. (B) The matching index between the participants' reports on the dominant image and the direction of the slow-phase movement of OKN at the very end of the 3-s stimulus presentation. The matching index was plotted for conditions when the two eyes' images moved in a nasotemporal direction and in a temporonasal direction.

Experiment 3 (binocular rivalry condition: Short stimulus duration)

Although the reproduction of perceptual alternation from eye movement responses was quite accurate in Experiment 2, a time delay regarding decision making to press a button and a delay executing the button press action after the decision are inevitable in that paradigm. In the third experiment, to prevent this time delay and exclude the situation where the participants experienced the mixture of the two eyes' images, we modified the visual sequence to show the stimuli for only 3 s and asked the participants (n = 5) to report which image was dominant at the end of the presentation by three alternative choices: the left eye image, the right eye image, or “not sure.” We then calculated the average of the sign of eye velocity within the time window (300 ms, the same time window as used in Experiments 1 and 2) at the stimulus offset to determine the direction of the slow-phase movement of OKN at the end. When the participants reported either eye's image as dominant, the reports corresponded quite well with the direction of OKN. As shown in Figure 4B, the matching indices for the temporonasal and nasotemporal conditions were 93.0% and 91.8%, respectively, and this difference was not statistically significant (paired t test: p = 0.535). The mean of the two conditions was 92.3%. 

Discussion

In summary, we developed a new binocular rivalry stimulus that consists of two different images associated with phase-shifted motion moving in opposite directions to each other. In this stimulus, the images were perceived to remain at the same position and a single phase of perceptual dominance lasted several seconds despite of the contrast alternation due to the phase shift; thus, the observer experiences the rivalry of images as similar to a stimulus that consists of static images. The results of our experiments also showed that phase-shifted motion signals elicited involuntary eye movements, i.e., OKN, similar to previous studies that used more customary position-shifted motion stimuli. Moreover, the trace of the observers' reports on the dominantly perceived image during binocular rivalry was reproducible from the elicited OKN, since the slow-phase movements of OKN correspond to the motion associated with an image in awareness. More than 90% of the button press reports on perceptual status matched those reproduced from the OKN. These findings indicate that our new binocular rivalry stimulus serves as a psychophysical tool to read out the alternation of perceived object images without requiring participants to voluntarily report it. 

The method proposed here depends on the OKN for the reproduction of the participants' perception. Therefore, stimulus parameters have to be adjusted to evoke OKN of sufficient amplitude. The parameters, especially the stimulus size, should also be adjusted not to produce frequent piecemeal rivalry (Blake et al., 1992; O'Shea et al., 1997). As for the optimization of stimulus parameter, Schor and Narayan (1981) have thoroughly investigated the influence of stimulus size, spatial frequency, and temporal frequency on OKN using moving grating pattern. They found that OKN is elicited by coarse moving gratings and is relatively insensitive to fine gratings. (The upper limit of the spatial frequency to evoke OKN is 8 cycle/deg in the stimulus size of 10 deg.) It was also reported that there is the upper limit of the temporal frequency (24 Hz regardless of the stimulus size). Therefore, images have to contain low spatial frequency components and to be modulated at a proper temporal frequency (4 Hz in the present study) to work in our paradigm. The exact values of optimal parameters for our stimulus configuration that contains wide range of spatial frequency may differ from those reported by Schor and Narayan who used moving grating, since non-linear interaction across different spatial frequencies (Hayashi, Sugita, Nishida, & Kawano, 2009), as well as interocular suppression, could be involved in the process integrating motion signals. Nevertheless, rough parameter adjustment by checking whether OKN is elucidated and whether the participants do not experience frequent piecemeal rivalry is enough to get reliable estimation of which eye image was perceived in our stimulus from OKN. The proposed paradigm is designed to use natural/object images, such as human faces, which contain broadband spatial frequencies. The drawback of applying phase-shift operation to such images is that the velocity of the stimulus is not uniquely defined; thus, we cannot investigate the effect of stimulus parameters based on a fine measure such as gain of OKN as previous studies did (Schor & Narayan, 1981). 

It is also noteworthy to point out that there is evidence showing that form rivalry and motion rivalry can be dissociated (Andrews & Blakemore, 1999) and, moreover, that eye movements and motion perception can be dissociated during rivalry (Spering, Pomplun, & Carrasco, 2010) when two orthogonally moving gratings were presented dichoptically. These results evidently indicate that perceived motion and eye movements may not always provide a complete picture of what is being experienced during rivalry between object images. The studies that used oppositely moving stimuli, as the present study (Fox et al., 1975; Logothetis & Schall, 1990), however, are all consistent in that eye movement corresponds to the perceived motion direction, probably because opponent motion signals are mutually inhibitory (Qian, Andersen, & Adelson, 1994; Stromeyer, Kronauer, Madsen, & Klein, 1984) and/or motion signal integrated by vector averaging (possible mechanism causing the dissociation between the eye movement and motion perception in Spering et al., 2010) is null. Intriguing questions on, e.g., how the rivalry/OKN correspondence varies depending on stimulus parameters and how various parameters interact with each other on rivalry dynamics in our stimulus remain to be solved as future work. 

The absence of clear nasotemporal asymmetry in the performance to reproduce the perceptual alternation from OKN suggests that motion processing in the cortex is at least involved in the perception-congruent alternation of OKN. On the other hand, the detailed analysis implies that temporonasal motion signals from subcortical areas that are presumably not congruent with perceptual alternation could interfere with this cortical modulation on OKN. It would, therefore, be better to move our binocular rivalry stimulus in nasotemporal directions to reduce the effect of motion signals that do not correlate with perceptual alternation. We chose upright and upside-down human faces as monocular inputs with the expectation of finding an effect of the holistic information of face on the dominance of the conscious perception. Although the predominance of upright face/familiar stimulus over inverted face/unfamiliar stimulus has been reported in previous studies (Jiang, Costello, & He, 2007; Yu & Blake, 1992), we did not find the advantage of context information on either perceptual alternation or on the switching of the OKN in the present study. This discrepancy may be partially due to the relatively weak constraint on eye movements in our experiment, which allows OKN during the stimulus presentation but also makes it easier to break the suppression from the other eye, thereby diminishing the slight advantage of the upright face. 

The results showed that the flip of OKN (or the motion perception that drives OKN) and the flip of object perception occurred more or less simultaneously and did not dissociate except for the period when a mixture of the left and right eye images was experienced. There are several possibilities that can account for the association between motion/OKN and object recognition even though the two modalities are processed by relatively separate cortical streams. The first possibility is that rivalry occurs at the level where motion and figural information are not completely separated yet, such as V1/V2. The second possibility is that rivalry occurs at the level where motion and figural information are reintegrated such as at the superior temporal polysensory area (STP; Baizer, Ungerleider, & Desimone, 1991; Boussaoud, Ungerleider, & Desimone, 1990). The third possibility is that rivalry occurs in areas along the two visual processing streams (e.g., MT for motion processing and IT for object recognition) and the visual awareness of motion or object perception affects the other processing. As support for the third possibility, psychophysical experiments studying 3D rotational motion from “Mooney face” (Ramachandran, Armel, Foster, & Stoddard, 1998) suggested the interaction between modules concerned with motion and those involved in high-level object recognition. Such interaction could be achieved through several corticocortical pathways, including a direct connection between MT and IT (a sparse connection between MT and the posterior part of IT (TEO) has been reported; Distler, Boussaoud, Desimone, & Ungerleider, 1993), an indirect connection via V4 (Maunsell & van Essen, 1983; Ungerleider & Desimone, 1986), an ascending relay through STP (Baizer et al., 1991; Boussaoud et al., 1990), and top-down modulation onto the common lower visual areas (V1/V2). These three possibilities are not mutually exclusive; thus, rivalry at multiple layers may underlie the processing of our binocular rivalry stimulus, consistent with the views of Blake and Logothetis (2002) and Tong, Meng, and Blake (2006). The study of visual awareness is attracting the attention of researchers in a wide range of fields because it could be one possible approach to the problem of consciousness (Crick & Koch, 1998). In particular, the discrepancy between human fMRI and monkey electrophysiology regarding the role of V1 in determining the visibility of a stimulus is one of the disputed issues in the study of visual awareness (Maier et al., 2008). Future electrophysiological experiments that simultaneously measure neural activity in MT, IT, and/or V1 using our stimulus may clarify the interaction between the lower and higher visual areas and how the two modalities processed along separate visual streams are bound during rivalry. 

Supplementary Materials

Supplementary Movie - Supplementary Movie 

Movie 1 

Acknowledgments

This research was supported by JST PRESTO Program (09152519) to RH, Grant-in-Aid for Scientific research 23700498 (Grant-in-Aid for Young Scientists B) to RH, 22300137 to MT, and Grant-in-Aid for Innovative Areas, “Face Perception and Recognition” to MT, from Ministry of Education, Culture, Sports, Science and Technology (MEXT). 

Author contributions: RH and MT conceived and designed the experiments. RH performed the experiments and analyzed the data. RH and MT wrote the paper. 

Commercial relationships: none. 

Corresponding author: Manabu Tanifuji. 

Email: [email protected]. 

Address: Laboratory for Integrative Neural Systems, RIKEN Brain Science Institute, Saitama, Japan.