Free
Research Article  |   August 2010
Audiovisual interactions in binocular rivalry
Author Affiliations
Journal of Vision August 2010, Vol.10, 27. doi:https://doi.org/10.1167/10.10.27
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Verena Conrad, Andreas Bartels, Mario Kleiner, Uta Noppeney; Audiovisual interactions in binocular rivalry. Journal of Vision 2010;10(10):27. https://doi.org/10.1167/10.10.27.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

When the two eyes are presented with dissimilar images, human observers report alternating percepts—a phenomenon coined binocular rivalry. These perceptual fluctuations reflect competition between the two visual inputs both at monocular and binocular processing stages. Here we investigated the influence of auditory stimulation on the temporal dynamics of binocular rivalry. In three psychophysics experiments, we investigated whether sounds that provide directionally congruent, incongruent, or non-motion information modulate the dominance periods of rivaling visual motion percepts. Visual stimuli were dichoptically presented random-dot kinematograms (RDKs) at different levels of motion coherence. The results show that directional motion sounds rather than auditory input per se influenced the temporal dynamics of binocular rivalry. In all experiments, motion sounds prolonged the dominance periods of the directionally congruent visual motion percept. In contrast, motion sounds abbreviated the suppression periods of the directionally congruent visual motion percepts only when they competed with directionally incongruent percepts. Therefore, analogous to visual contextual effects, auditory motion interacted primarily with consciously perceived visual input rather than visual input suppressed from awareness. Our findings suggest that auditory modulation of perceptual dominance times might be established in a top-down fashion by means of feedback mechanisms.

Introduction
Behavioral performance can be optimized when information from multiple sensory modalities is integrated into a unified and more reliable percept of our natural environment. For instance, in everyday life, we may more easily perceive an approaching car when combining the complementary auditory and visual cues about its speed or motion direction. Motion detection thresholds for auditory-visual signals are significantly lower for congruent motion signals than for signals that are not matched in location or direction (Meyer, Wuerger, Röhrbein, & Zetzsche, 2005). In humans, motion perception is dominated by the visual sense that usually provides more reliable motion information. Vision can even reverse the perceived direction of a conflicting auditory motion stimulus—a phenomenon coined visual capture (Mateef, Hohnsbein, & Noack, 1985; Soto-Faraco, Lyons, Gazzaniga, Spence, & Kingstone, 2002; Soto-Faraco, Spence, & Kingstone, 2004). However, if the visual stimulus is rendered unreliable, auditory motion can also bias the perception of visual motion direction (Maeda, Kanai, & Shimojo, 2004; Meyer & Wuerger, 2001) and influence the perceived velocity of a moving object (Ecker & Heller, 2005). At the neural level, audiovisual interactions have been shown at multiple hierarchical levels including subcortical regions, primary sensory, and higher order association cortices. In humans, integration of auditory and visual motion signals was found in both visual motion area V5+/hMT+ (Sadaghiani, Maier, & Noppeney, 2009) and “classical” higher order parietal convergence areas (Bremmer et al., 2001). 
While the role of auditory signals in disambiguating unreliable (or degraded) visual information is well established, only few studies have investigated multisensory interactions in multistable perception. Can an unambiguous signal from one sensory modality resolve the conflict between competing percepts or interpretations of incoming signals from another modality? Indeed, visual bistability of 3D rotation percepts induced by a “structure from motion” rotating globe was significantly influenced by concurrent unambiguous tactile motion cues (Blake, Sobel, & James, 2004). Similarly, tactile 3D shape information stabilized congruent visual percepts of the bistable Necker cube (Bruno, Jacomuzzi, Bertamini, & Meyer, 2007). Collectively, these results show that the human brain draws on information from multiple senses to resolve (visual) ambiguities and conflicts in multistable perception. 
The present study investigated the influence of auditory signals on the temporal dynamics of binocular rivalry. Binocular rivalry refers to the phenomenon that observers report alternating percepts when the two eyes are presented with dissimilar images. These perceptual fluctuations reflect competition between the two visual inputs, at monocular and binocular processing stages (Bartels & Logothetis, in revision; Blake & Logothetis, 2002). Even though perceptual transitions occur stochastically over time, their temporal dynamics can be modulated by changes in stimulus strength, perceptual or semantic context, and attention (Levelt, 1966; Meng & Tong, 2004; Paffen, Alais, & Verstraten, 2006; van Ee, van Boxtel, Parker, & Alais, 2009; van Ee, van Dam, & Brouwer, 2005; Wolf & Hochstein, 2007). While increases in stimulus strength, such as contrast, primarily abbreviate suppression phases of a percept (Levelt, 1968), attentional and contextual factors predominantly lengthen its dominance periods (Chong & Blake, 2006)—though this principle is not consistently applicable (Bossink, Stalmeier, & de Weert, 1993; Brascamp, van Ee, Noest, Jacobs, & van den Berg, 2006; Mueller & Blake, 1989; Paffen, van der Smagt, te Pas, & Verstraten, 2005). 
In terms of neural mechanisms, the currently emerging view attributes binocular rivalry to distributed processes at multiple cortical levels that are susceptible to feedback influence from higher order systems (e.g., attention; Leopold & Logothetis, 1999; Tong, Meng, & Blake, 2006). 
Only a few studies have investigated the influence of concurrent auditory signals on the temporal dynamics of rivaling visual percepts. Recently, Kang and Blake (2005) showed that amplitude-modulated sounds increased the dominance times of a flickering relative to a static grating, particularly when sound and visual grating's contrast oscillated in synchrony. Even though the sound's effect was considerably smaller relative to well-established effects of visual context (Sobel & Blake, 2002) or focused attention (Chong, Tadin, & Blake, 2005), these findings highlighted the importance of synchrony cues for audiovisual interactions in bistable perception in line with the “temporal” principle of multisensory integration known from non-rivaling perception. A second study (only reported as an abstract by Parker & Alais, 2006) demonstrated that congruent looming sounds that draw on attentional resources enhance the general bias or predominance of a looming (relative to a receding) visual percept. In both previous studies, auditory signals prolonged only the dominance periods of the congruent percept (e.g., looming sounds amplify the predominance of the looming percept) but did not influence its suppression periods or—from an alternative perspective—the dominance periods of the incongruent percept. Among many potential mechanisms, this asymmetry may at least in part be attributed to the predominance of this percept even in the absence of sound (i.e., flickering vs. static gratings; looming vs. receding). 
The current study investigated the influence of (i) auditory non-motion or (ii) 3D motion sounds along the horizontal trajectory (e.g., left to right) on the temporal dynamics of rivaling visual motion percepts. In a series of three experiments, observers dichoptically viewed random-dot kinematograms with different motion coherence levels and/or opposite directions. The auditory input could be (i) absent, (ii) non-motion, (iii) congruent, or (iv) incongruent with respect to the direction of the visual motion percept. This experimental paradigm enabled us to address the following questions: 
  1.  
    Does auditory input per se influence the temporal dynamics of rivaling motion percepts (e.g., slows down or speeds up)?
  2.  
    Do auditory motion signals influence and aid resolving the conflict of competing visual motion percepts? Based on previous studies, we expected auditory motion signals to prolong the dominance times of the directionally congruent visual motion percept. In addition, we investigated whether auditory motion signals could also influence the suppression time of a visual motion percept and hence interact with visual processing in the absence of awareness.
Methods
Observers
Observers in all three experiments had normal or corrected-to-normal vision and reported normal hearing. All had participated in previous rivalry experiments; yet, they were naive with regard to the purpose of this particular experiment except for one observer (VC) who is an author of the manuscript (n.b. her data were not different from the group results). Observers from Experiments 1 and 3 were identical. Observers gave written informed consent to participate in the study, which was approved by the local ethics committee of the University of Tübingen. 
Visual stimuli
Visual stimuli were presented dichoptically using a stereoscope consisting of two CRT monitors viewed via angled, silver-coated mirrors. The monitors were independently linearized and run with a resolution of 1024 × 768 at 60 Hz. In all conditions, visual stimuli were random-dot kinematograms (RDKs) showing 100 white dots (38 cd/m2) on a dark background (1.5 cd/m2, see Figure 1, modified from Albright, 1984; Britten, Shadlen, Newsome, & Movshon, 1993; Celebrini & Newsome, 1994; Maunsell & Van Essen, 1983; Moutoussis, Keliris, Kourtzi, & Logothetis, 2005; Rees, Friston, & Koch, 2000). Each dot was approximately 0.1° in diameter and had a lifetime of 2 s. The RDKs were presented foveally, contained an isoluminant fixation spot of 0.5 deg of visual angle and were framed by an isoluminant square aperture of 4 deg in diameter to aid binocular fusion. The viewing distance was 118 cm. Dot speed was 1 deg/s in either a signal or random direction. Signal direction was either left or right horizontal translational motion. Stimulus area and number of dots were identical for all experimental conditions. The level of motion coherence was determined by manipulating the fraction of dots moving coherently in the signal direction at constant speed. A 100% coherence value generates a smoothly drifting field of randomly positioned dots and 0% coherence results in an incoherent “snowstorm” with no directional information (see Moutoussis et al., 2005 for further details). Psychophysical stimuli were created on a PC running Windows XP using the Psychtoolbox Version (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; http://psychtoolbox.org) running on Matlab 7 (Mathworks, Nantucket, Massachusetts). 
Figure 1
 
Experimental paradigm, stimuli, and setup. (a) Experiment 1: The two eyes were presented with unidirectional motion of opposite directions at 100% coherence. (b) Experiment 2: The two eyes were presented with alternating motion of opposite directions at 100% coherence. (c) Experiment 3: Eye 1 was presented alternating motion at 60% coherence; eye 2 with random motion at 0% coherence. In all three experiments, the sound was (i) absent, (ii) non-motion, or (iii) directional motion. The arrows indicate the direction of motion of the RDKs. (e, f) Subjects viewed stimuli in a darkened stereo-setup. LE: left eye, RE: right eye. See text for details.
Figure 1
 
Experimental paradigm, stimuli, and setup. (a) Experiment 1: The two eyes were presented with unidirectional motion of opposite directions at 100% coherence. (b) Experiment 2: The two eyes were presented with alternating motion of opposite directions at 100% coherence. (c) Experiment 3: Eye 1 was presented alternating motion at 60% coherence; eye 2 with random motion at 0% coherence. In all three experiments, the sound was (i) absent, (ii) non-motion, or (iii) directional motion. The arrows indicate the direction of motion of the RDKs. (e, f) Subjects viewed stimuli in a darkened stereo-setup. LE: left eye, RE: right eye. See text for details.
Auditory stimuli
To maximize the influence of auditory motion on the visual motion percept, we created a persuasive auditory 3D motion stimulus. Auditory motion perception relies on binaural cues, i.e., interaural amplitude and time differences, and monoaural filtering cues due to the pinna shape, head, and torso (for a review, see Blauert, 1983) Previous studies of audiovisual motion integration have often used auditory motion stimuli that were defined only by interaural amplitude (Meyer & Wuerger, 2001) or time difference cues (Alais & Burr, 2004). However, audiovisual motion integration is thought to be facilitated when all three cues are combined leading to precise audiovisual spatiotemporal colocalization (Meyer et al., 2005). Since RDKs do not allow for precise spatiotemporal colocalization, we created auditory motion signals that matched motion direction and duration of the visual stimulus. The auditory motion stimulus was a white noise stimulus moving at constant velocity from left to right or vice versa in 2-s duration following a horizontal trajectory. The auditory motion stimulus combined binaural cues of interaural time and intensity difference and monoaural filtering cues based on generalized head-related transfer functions. 
The motion sound was created using OpenAL (2005) as implemented into Psychtoolbox 3. OpenAL is a free, open-source, cross-platform 3D audio API (OpenAL 1.1 Specification and Reference) that provides functions to control properties of the auditory motion stimulus such as position, velocity, and motion direction. These parameters determined how the sound traveled through 3D space with reference to the listener (for further reference for the use of OpenAL in psychophysical studies, see Lecuona & Canadas-Quesada, 2009; Perrin, Chavarriaga, Ray, Siegwart, & Milan, 2008). 
For the present study, a uniformly distributed random noise signal was played at a sampling rate of 22,050 Hz through a pair of headphones (Sennheiser HD 201). The 3D sound was generated using “Soundblaster XFi” soundcard from Creative Labs. This card supports calculation of the spatialized sound signal in hardware using generalized HRTFs. 
Experimental design
In all three experiments, observers dichoptically viewed random-dot kinematograms (RDKs) of translational motion with constant velocity. Across experiments, the RDKs differed with respect to motion coherence and whether or not the motion direction alternated over time. To investigate the influence of auditory input on the temporal dynamics of subjects' alternating percepts, each experiment included three conditions: (i) no sound, (ii) non-motion sound, and (iii) motion sound along a horizontal trajectory. 
Subjects reported their visual percept, while ignoring the concurrent auditory input that was congruent, incongruent, or absent. More specifically, they reported their current percept by holding down one of two buttons and indicated mixed or indeterminate percepts by pressing neither button. These instructions were used to attenuate effects of sound-induced response biases, which are most pronounced during piecemeal periods that do not provide a consistent visual motion signal. 
Prior to the main experiment, observers were familiarized with the auditory and visual stimuli and the possible percepts were explained using demonstration stimuli. In a series of practice trials, they indicated the direction of visual and auditory motions to ensure the functionality of the visual and auditory presentation systems. An additional psychophysics study also demonstrated that the motion cues provided by the auditory motion signal are sufficient for reliable discrimination of the direction of the auditory motion (see Supplementary materials for further details). 
Observers fixated throughout the experimental blocks. Each experimental block was preceded and followed by a 10-s fixation period. Each observer completed two experimental blocks per condition in each experiment resulting in 6 blocks for Experiments 1 and 2 and 8 blocks for Experiment 3. The order of conditions was counterbalanced for each subject. 
Experiment 1
Observers
Ten observers participated in Experiment 1 (age range 22–30 years, mean age 25.8 years, 6 males). 
Experimental design
Visual stimulus: Observers dichoptically viewed two RDKs of horizontal translational motion with 100% coherence level, equal absolute motion velocities, but opposite motion directions. The motion direction of each RDK and the motion velocities were held constant within each 200-s block. For instance, eye 1 was presented with left to right motion and eye 2 with right to left motion throughout the 200-s block. Over blocks, the direction of the visual motion was counterbalanced between the two eyes within subjects. 
Auditory stimulus: Three sound conditions were included: (i) No sound. The two RDKs were presented in the absence of auditory input. (ii) Non-motion. In the non-motion condition, participants were presented with an auditory stimulus that was similar to the directional motion sound with respect to its physical properties (i.e., white noise) but did not provide any motion information. The non-motion condition tested for overall influences of auditory input on the temporal dynamics of binocular rivalry. (iii) Directional motion sound. In each 200-s block, participants were presented with a sequence of 100 brief (i.e., 2-s duration) sweeps of directional motion sounds that were congruent to one of the two 100% RDKs with respect to motion direction. In other words, within a 200-s block, the auditory motion signal sweeps were unidirectional, congruent with the 100% RDK presented to one eye and incongruent to the 100% RDK presented to the other eye. Over blocks, the direction of the auditory sweeps (i.e., “left to right” vs. “right to left”) and its congruency with the left vs. right eye were counterbalanced within and across subjects. 
Observers reported whether they perceived left or right visual motion. Hence, if the percept of a given eye (e.g., left motion) was dominant for an extended temporal period, the observer reported one single percept (e.g., left motion) for that period of time. 
Experiment 2
Observers
Twelve observers participated in Experiment 2 (age range 24–36 years, mean age 27.75 years, 7 males). 
Experimental design
Visual stimulus: Observers dichoptically viewed two RDKs of horizontal translational motion with 100% coherence level, equal absolute motion velocities, but opposite motion directions. Yet, in contrast to Experiment 1, the motion direction of each RDK alternated every 2 s within a 200-s block. For instance, eye 1 was presented with [left–right–left–right…] motion and eye 2 with [right–left–right–left…] motion throughout the 200-s block. Over blocks, the initial direction of the visual motion was counterbalanced between the two eyes within subjects. 
Auditory stimulus: Three sound conditions were included: (i) No-sound, (ii) non-motion, and (iii) directional motion sound. In each 200-s block, participants were presented with a sequence of 100 brief (i.e., 2-s duration) sweeps of motion sounds of alternating direction. This sequence of [left–right–left…] sweeps was congruent with the motion sequence presented to one eye and hence incongruent to the motion sequence presented to the other eye. Over blocks, the direction of the initial auditory sweep (i.e., left to right or right to left) and its congruency with the left vs. right eye were counterbalanced within subjects. 
Observers reported whether they perceived left or right visual motion. Hence, if the percept of a given eye (e.g., left–right–left…) was dominant for an extended temporal period, the observer reported percepts that alternated in motion direction every 2 s (by alternating between pressing the left or right button every 2 s). 
Unless otherwise stated, the parameters from Experiment 1 were applied. 
Experiment 3
Observers
Ten observers participated in Experiment 3 (age range 22–30 years, mean age 25.8 years, 6 males). 
Experimental design
Visual stimulus: Observers dichoptically viewed two RDKs, one with 60% horizontal translational motion and another with 0% coherence level. A level of only 60% coherence was used in order to provide a strong directional motion signal while balancing the dominance times of the two RDKs (see Moutoussis et al., 2005 for further details). Similar to Experiment 2, the motion direction of the 60% RDK alternated every 2 s within a 200-s block. Hence, eye 1 was presented with [left–right–left–right…] motion and eye 2 with random motion throughout the 200-s block. 
Over blocks, the initial direction of the visual motion was counterbalanced between the two eyes within subjects. 
Auditory stimulus: Four sound conditions were included: (i) No sound, (ii) non-motion, (iii) congruent directional motion sound, and (iv) incongruent directional motion sound. In conditions (iii) and (iv), participants were presented with a sequence of 100 brief (i.e., 2-s duration) sweeps of auditory motion sounds of alternating direction that were congruent (i.e., condition (iii)) or incongruent (i.e., condition (iv)) with respect to the motion direction of the 60% RDK. Over blocks, the direction of the initial auditory sweep (i.e., left to right or right to left) and its congruency with the left vs. right eye were counterbalanced within subjects. 
Observers reported whether they perceived coherent or random visual motion. Hence, if the percept of a given eye (e.g., alternating left–right–left… coherent motion) was dominant for an extended temporal period, the observer continuously reported the coherent percept (by pressing one button throughout the dominance period). 
Unless otherwise stated, the parameters from Experiments 1 and 2 were applied. 
Results
In all three experiments, we characterized the influence of non-directional and directional sounds on the temporal dynamics of binocular rivalry using two complementary standard measures. (1) Mean perceptual dominance times inform us about the length of the dominance periods of a particular RDK percept during a given block. (2) Cumulative dominance durations refer to the cumulative duration of each percept expressed as the percentage of the total presentation time. Even though the two measures may often provide convergent profiles, they can diverge in some cases. For instance, if subject's percept frequently alternates between percept 1 and an indeterminate percept, the mean dominance time for percept 1 may be small despite a considerable cumulative dominance time. Conversely, holding the duration of piecemeal periods constant, an increased cumulative dominance time for percept 1 may arise because of longer mean dominance times for percept 1 or shorter dominance times for percept 2 (= suppression times of percept 1). Thus, mean perceptual dominance times and cumulative dominance durations highlight complementary characteristics of the dynamics of binocular rivalry. 
To evaluate the effect of directional congruency, the perceptual mean and cumulative dominance times during the motion sound conditions were separated according to whether the visual motion percept was directionally congruent or incongruent with respect to the auditory motion signal. 
Experiment 1
In Experiment 1, the two eyes were presented with 100% RDKs of constant velocity and constant yet opposite motion directions. In the auditory motion conditions, the auditory sound was a sequence of 2-s sweeps consistently moving into one direction throughout the entire block (Figures 2 and 3). 
Figure 2
 
Experiment 1: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. * Significant difference in mean dominance time at p < 0.05, see Table 1.
Figure 2
 
Experiment 1: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. * Significant difference in mean dominance time at p < 0.05, see Table 1.
Figure 3
 
Experiment 1: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. Percent cumulative dominance times for the stable percepts in the non-motion and no-sound conditions were averaged (not summed) across the two eyes for comparison with the percent cumulative dominance times of the congruent and incongruent percepts during the motion sound conditions. Therefore, percent cumulative dominance times during the non-motion and no-sound conditions do not sum to 100%. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 1.
Figure 3
 
Experiment 1: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. Percent cumulative dominance times for the stable percepts in the non-motion and no-sound conditions were averaged (not summed) across the two eyes for comparison with the percent cumulative dominance times of the congruent and incongruent percepts during the motion sound conditions. Therefore, percent cumulative dominance times during the non-motion and no-sound conditions do not sum to 100%. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 1.
Mean perceptual dominance times: A one-way repeated-measures ANOVA with the factor sound (congruent, incongruent, non-motion sound, and no sound) revealed a significant main effect of sound on perceptual dominance times (F(3, 27) = 8.01, p < 0.05). Post-hoc comparisons (one-tailed t-tests) demonstrated that the mean dominance times for the congruent visual motion percept were significantly prolonged relative to the incongruent visual motion percept, the non-motion, and no-sound conditions. The suppression times of the congruent percept (= dominance times of the incongruent motion percept) was not significantly different from the non-motion or no-sound condition. The mean dominance times for non-motion and no-sound conditions were not significantly different (see Table 1). 
Table 1
 
Experiment 1: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
Table 1
 
Experiment 1: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
df:(9) Motion-sound incongruent Non-motion sound No sound
Mean dominance times in seconds
Motion-sound congruent t = 2.95, p < 0.05* t = 5.48, p < 0.05* t = 2.88, p < 0.05*
Motion-sound incongruent t = 0.84, p > 0.05 t = 1.45, p > 0.05
Non-motion sound t = 1.72, p > 0.05
Cumulative dominance in percent
Motion-sound congruent t = 3.06, p < 0.05* t = 3.00, p < 0.05* t = 2.40, p < 0.05*
Motion-sound incongruent t = 1.10, p > 0.05 t = 2.17, p = 0.06
Non-motion sound t = 1.40, p > 0.05
 

*Significant difference at p < 0.05.

Percent cumulative dominance durations: A one-way repeated-measures ANOVA with the factor sound (congruent, incongruent, non-motion sound, and no sound) revealed a significant main effect of sound (F(3, 27) = 5.98, p < 0.05). Post-hoc pairwise comparisons (see Table 1) revealed a significant increase in percent cumulative dominance duration for the directionally congruent percept relative to the directionally incongruent percept, the non-motion sound, and no-sound conditions. Further, suppression time of the congruent percept (= dominance of the incongruent percept) was shortened relative to the no-sound condition (non-significant trend: p = 0.06). 
In summary, sounds without directional signals (i.e., non-motion) did not significantly modulate the mean or percent cumulative dominance times of two rivaling 100% RDK percepts with opposite directions. In contrast, directional motion sounds primarily increased the mean/cumulative dominance times of the directionally congruent motion percept and only slightly decreased its cumulative suppression times relative to the no-sound condition. This profile of dominance/suppression times can alternatively be described from the perspective of the incongruent motion percept, which would then be modulated primarily during its suppression rather than dominance periods. These interpretational ambiguities will be discussed in greater depth later in the manuscript. 
Experiment 2
In Experiment 1, the effect of motion sound on the dynamics of binocular rivalry may have been attenuated because of differences in the temporal structure between the auditory (discontinuous) and visual (continuous) motion stimuli. While RDKs present translational motion without any interruption over an infinite period of time, a strong auditory motion percept could only be induced by a rapid sequence of brief (2 s) sweeps that were consistently moving into one direction, e.g., left to right throughout the entire 100-s block. To enhance temporal congruity between visual and auditory motion signals, Experiment 2 presents a rapid (i.e., 2 s) alternating sequence of right and left motion “sweeps” in both visual and auditory modalities. Thus, Experiment 2 provides not only congruity of auditory and visual motion directions but also synchronous alternations of motion direction. Temporal synchrony and directional congruity are known as essential cues to enable audiovisual interactions (e.g., see Figure 1b). 
Mean perceptual dominance times: A one-way repeated-measures ANOVA with the factor sound (congruent, incongruent, non-motion sound, and no sound) revealed a significant main effect of sound direction on perceptual dominance times (F(2.01, 23.06) = 24.33; p < 0.001, Huynh–Feldt correction). Post-hoc comparisons (one-tailed t-tests) revealed a significant effect of directional congruency. Directional motion sounds increased the mean dominance times of the congruent visual motion percept relative to non-motion sound and no sound. Furthermore, the suppression times of the congruent motion percept (= dominance time of the incongruent motion percept) were decreased relative to the non-motion sound and no-sound conditions (see Table 2). 
Table 2
 
Experiment 2: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
Table 2
 
Experiment 2: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
df:(11) Motion-sound incongruent Non-motion sound No sound
Mean dominance times in seconds
Motion-sound congruent t = 6.01, p < 0.05* t = 4.61, p < 0.05* t = 4.84, p < 0.05*
Motion-sound incongruent t = 3.91, p < 0.05* t = 3.33, p < 0.05*
Non-motion sound t = 1.53, p > 0.05
Cumulative dominance in percent
Motion-sound congruent t = 6.16, p < 0.05* t = 5.32, p < 0.05* t = 5.39, p < 0.05*
Motion-sound incongruent t = 4.44, p < 0.05* t = 3.93, p < 0.05*
Non-motion sound t = 1.69, p > 0.05
 

*Significant difference at p < 0.05.

Percent cumulative dominance durations: A one-way repeated-measures ANOVA with the factor sound (congruent, incongruent, non-motion sound, and no sound) revealed a significant main effect of sound on cumulative dominance durations (F(1.83, 20.07) = 28.86, p < 0.001). Post-hoc comparisons showed that directional motion sound significantly increased the percent cumulative dominance durations of the congruent visual motion percept relative to the incongruent sound, non-motion sound, and no sound. Comparisons between the directionally incongruent and the non-motion sound and no-sound conditions revealed non-significant effects (Figures 4 and 5). 
Figure 4
 
Experiment 2: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in mean dominance time at p < 0.05, see Table 2.
Figure 4
 
Experiment 2: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in mean dominance time at p < 0.05, see Table 2.
Figure 5
 
Experiment 2: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 2.
Figure 5
 
Experiment 2: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 2.
In line with the results from Experiment 1, Experiment 2 revealed a significant effect of directional motion sound. Auditory motion signals primarily increased the mean/cumulative dominance times of the directionally congruent motion percept and decreased its mean suppression times (= dominance time of the incongruent percept) relative to the non-motion and no-sound conditions. 
Experiment 3
Experiments 1 and 2 demonstrated that motion sounds prolong the mean and cumulative dominance times of directionally congruent RDK percepts. Experiment 1 and particularly Experiment 2 have also provided some evidence for shortened suppression periods of the congruent RDK motion percept. However, these shorter suppression periods of the congruent percept may in fact result from motion sounds reducing the dominance periods of directionally incongruent RDKs. To further disentangle these two interpretations, Experiment 3 presented RDKs at 60% and 0% coherence together with motion sounds that were directionally congruent and incongruent to the 60% RDK. Since 0% RDKs do not entail a direction signal (hence reducing the incongruency between motion sound and 0% RDK), a persistence of abbreviated suppression periods for 60% RDKs that are directionally congruent to the sound would provide further evidence for a true effect of auditory motion signal on the suppressed visual signal. Given the strong congruency effects in Experiment 2, Experiment 3 likewise presented alternating motion directions in both visual and auditory modalities (Figures 6 and 7). 
Figure 6
 
Experiment 3: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept. *Significant difference in mean dominance time at p < 0.05, see Table 3.
Figure 6
 
Experiment 3: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept. *Significant difference in mean dominance time at p < 0.05, see Table 3.
Figure 7
 
Experiment 3: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept; white = indeterminate percept *Significant difference in percent cumulative dominance times at p < 0.05, see Table 3.
Figure 7
 
Experiment 3: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept; white = indeterminate percept *Significant difference in percent cumulative dominance times at p < 0.05, see Table 3.
Mean perceptual dominance times: A doubly multivariate repeated-measures ANOVA with the factor auditory signal (congruent motion sound, incongruent motion sound, non-motion sound, no sound) was performed for the two dependent variables mean dominance times of the coherent and random percepts and revealed a significant main effect of auditory signal (F(3, 27) = 3.55, p < 0.05). 
This significant result was followed up by two separate repeated-measures ANOVAs for the two dependent variables. As expected, a significant effect of motion direction was revealed only for the coherent percept (F(2.58, 23.19) = 3.54, p < 0.05, Huynh–Feldt correction) but not for the random percept (F(1.44, 12.96) = 0.50, p > 0.05, Huynh–Feldt correction). Post-hoc pairwise comparisons revealed that congruent motion sound significantly increased the mean dominance time of the congruent visual motion percept relative to the incongruent sound, non-motion sound, and no sound. Comparisons between the directionally incongruent and the non-motion sound and no-sound conditions were not significant. 
Percent cumulative dominance durations: A doubly multivariate repeated-measures ANOVA with the factor auditory signal (congruent motion sound, incongruent motion sound, non-motion sound, no sound) for the two dependent variables percent cumulative dominance time of the coherent and random percepts revealed a significant main effect of motion direction (F(3, 27) = 4.66, p < 0.05). This significant result was followed up by two separate repeated-measures ANOVAs for the two dependent variables. As expected, a significant effect of auditory signal was revealed only for the coherent percept (F (3, 27) = 4.12, p < 0.05) but not for the random percept (F(3, 27) = 0.83, p > 0.05). Post-hoc pairwise comparisons revealed that congruent motion sound significantly increased the percent cumulative dominance duration of the congruent visual motion percept relative to the incongruent sound, non-motion sound, and no sound. Comparisons between the directionally incongruent and the non-motion sound and no-sound conditions were not significant (see Table 3). 
Table 3
 
Experiment 3: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
Table 3
 
Experiment 3: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
df:(9) Motion-sound incongruent Noise No sound
Mean dominance times in seconds
Motion-sound congruent t = 2.31, p < 0.05* t = 2.50, p < 0.05* t = 1.93, p < 0.05*
Motion-sound incongruent t = 0.51, p > 0.05 t = 0.13, p > 0.05
Noise t = 0.44, p > 0.05
Cumulative dominance in percent
Motion-sound congruent t = 2.04, p < 0.05* t = 2.87, p < 0.05* t = 3.98, p < 0.05*
Motion-sound incongruent t = 0.10, p > 0.05 t = 0.13, p > 0.05
Noise t = 0.42, p > 0.05
 

*Significant difference at p < 0.05.

Experiment 3 revealed an effect of auditory motion signal only on the coherent motion percept but not on the random motion percept. Auditory motion signals primarily increased the mean/cumulative dominance times of the directionally congruent motion relative to the incongruent, non-motion, and no-sound conditions. 
Discussion
The current study demonstrates that directional motion sounds—rather than auditory input per se—influence the temporal dynamics of rivaling visual motion percepts. Motion sounds primarily prolonged the dominance times of the congruent visual motion percept. They also abbreviated those of incongruent motion percepts when they directly competed with a congruent percept. 
The role of auditory signals in disambiguating unreliable visual input is well established (Frassinetti, Bolognini, & Ladavas, 2002; Shams, Kamitani, & Shimojo, 2002). In particular, auditory motion input has been shown to influence and bias the perception of visual motion direction if the visual stimulus is rendered unreliable (Meyer & Wuerger, 2001; Sadaghiani et al., 2009). In a series of experiments, we investigated the influence of auditory non-motion and motion signals on the temporal dynamics of rivaling visual motion percepts. Our results suggest that directional congruence of auditory motion rather than the presence of auditory inputs per se influences the temporal dynamics of binocular rivalry. Both mean and percent cumulative dominance times were comparable for “non-motion” and the “no-sound” control conditions. This “null finding” is consistent with the notion that the human brain should integrate only “congruent” auditory and visual signals emanating from identical sources (Calvert, Spence, & Stein, 2004). Since the non-motion sounds did not entail temporal structure as synchrony or motion cues, the non-motion conditions lacked essential multisensory congruency cues to enable audiovisual integration. 
Conversely, concurrent auditory motion along a horizontal trajectory prolonged the percent cumulative and mean dominance times of the directionally congruent visual motion percept consistently across all three experiments. Previous studies have demonstrated that amplitude-modulated sounds increase the dominance times of flickering relative to static gratings (Kang & Blake, 2005) especially when the rates of amplitude and visual contrast modulation were matched. These findings highlight the importance of synchrony cues for audiovisual interactions. In contrast, in our Experiments 1 and—to a certain degree—2, the auditory influence relies purely on congruency of motion direction rather than strict temporal coincidence or synchrony of transient audiovisual signals. Synchrony cues were only provided in Experiment 3. Beyond providing synchrony cues, the direction changes of the visual motion stimuli in Experiments 2 and 3 are also important because of their role in inducing perceptual switches with or without changes in eye dominance. For instance, when the stimuli were interchanged between the two eyes, subjects may continue seeing with the dominant eye and report a perceptual alternation (Blake, Westerndorf, & Overton, 1980). Further, several rivalry experiments have demonstrated that transients in the stimuli can induce perceptual switches with and without changes in eye dominance (Bartels & Logothetis, in revision; Blake & Fox, 1974; Kanai, Moradi, Shimojo, & Verstraten, 2005; Wolfe, 1984). Indeed, additional analyses of perceptual switch probabilities as a function of time after a physical direction change demonstrated elevated switch probabilities immediately following a physical change in Experiment 2 (see Supplementary Figure 1). Taking into account subjects' reaction times, these early perceptual switches indicate that subjects tend to continue seeing with the currently dominant eye with a clear advantage for the eye that is presented with visual motion congruent with respect to the auditory motion. In Experiment 3, the transients had less influence, probably due to the lower motion coherence and the unequal dominance of the random and coherent motion percepts (see Supplementary Figure 2). Thus, both transients as well as steady-state motion information appear to account for our results. 
Our results are also related to a recent finding that looming sounds lengthen the dominance times of looming relative to receding visual signals (Parker & Alais, 2006). However, since looming signals are indices for rapidly approaching objects (Maier & Ghazanfar, 2007; Seifritz et al., 2002) and engage attentional resources, the interpretation of auditory looming effects in binocular rivalry is more ambiguous. In fact, to our knowledge, all previous multisensory studies of binocular rivalry have shown sound-induced amplifications of dominance times only for visual percepts that were more dominant already in the absence of an auditory signal, such as the flickering (vs. static) grating and the looming (vs. receding) motion. Presenting observers with visual motion that differs only with respect to motion direction, Experiments 1 and 2 demonstrate that auditory motion can selectively prolong the directionally congruent motion percept even if the two rivaling visual motion percepts were matched with respect to their visual dominance in the absence of sound. 
Having established the influence of auditory motion signal on perceptual dominance times, we investigated its effect on the suppression times of the congruent visual motion percept. In particular, we asked whether auditory motion may rescue a directionally congruent, yet “unaware” motion percept from suppression. Recent neurophysiological studies in non-human primates have demonstrated interactions already at the primary cortical level in anesthetized non-human primates indicating that audiovisual interactions do not necessitate awareness (Kayser, Petkov, & Logothetis, 2008). Further, self-generated actions were able not only to prolong the dominance periods of the congruent visual motion percept but also to abbreviate its suppression periods (Maruya, Yang, & Blake, 2007). Indeed, Experiments 1 and 2 demonstrated that auditory motion did not only prolong the dominance times but also abbreviated the suppression times of the directionally congruent motion percept. However, since in both experiments, the two eyes were presented with motion of opposite directions, we may equally well attribute these findings to a shortening of the dominance times of the incongruent percept. Experiment 3 aimed to disentangle these two interpretations by presenting the two eyes with random and 60% coherent motion that could either be congruent or incongruent with respect to the auditory motion. In this case, the effect of motion sound influenced only the dominance times of the congruent motion percept but not its suppression times. Nor did motion sound influence the dominance or suppression times of the incongruent percept. Interestingly, while the prolonged dominance times of the congruent percept occurred at the cost of the indeterminate percept in Experiment 3, they occurred at the cost of the incongruent percept in Experiment 2. This profile suggests that the neural representation of the incongruent percept is attenuated only in direct competition with a congruent (but not a random) motion percept. Even though the failure of Experiment 3 to reveal any effects of auditory motion on the dominance times of the incongruent motion percept may be attributed to the reduced (60%) level of motion coherence, this explanation is unlikely since the effects on congruent motion in Experiment 3 were larger than those observed in Experiment 2. Instead, the asymmetric effect of auditory motion on congruent and incongruent percepts may reflect the fact that directionally congruent auditory and visual signals are integrated into a unified percept, while directionally incongruent sound is simply treated as a separate stimulus like no-motion sound. Alternatively, the suppressive influence of incongruent motion sound on the coherent visual motion percept may be reduced because the dominance times of the 60% and 0% motion percepts were already unbalanced and biased toward coherent motion percept in the absence of sound (Breese, 1909; Walker & Powell, 1979). In line with this conjecture, previous audiovisual rivalry experiments with unbalanced percepts (e.g., flickering vs. static gratings or looming vs. receding motion) similarly failed to detect auditory influences on the suppression times of the congruent percept. To further investigate influences of auditory motion on suppression times of the congruent visual motion percept, a future experiment may therefore present the two eyes with 100% coherent and 0% random motion but adjust the contrasts of the two RDKs such that their dominance times are fully matched in the absence of sound. 
What are the potential neural mechanisms that mediate auditory influences on rivaling visual motion percepts? Dominant visual stimuli are thought to be processed throughout the same cortical hierarchy that is involved during normal non-rivaling viewing. Thus, auditory motion may influence the dynamics of visual motion percepts at multiple cortical hierarchical levels (Sadaghiani et al., 2009; Soto-Faraco, Kingstone, & Spence, 2003; Soto-Faraco et al., 2004) via (1) genuine multisensory integration, (2) attention, and (3) decisional processes. (1) Over the past decade, multisensory interactions have been shown at multiple cortical levels ranging from primary sensory to higher order association cortices (Bremmer et al., 2001; Kayser et al., 2008; Schroeder & Foxe, 2005; Werner & Noppeney, 2010). Since true multisensory integration breaks down when sensory inputs are brought into conflict, the dominance periods of the congruent visual motion percept may be stabilized by the integration of congruent auditory and visual inputs into a unified percept. Multisensory integration or binding could then be considered a natural extension of the well-established perceptual grouping (Sobel & Blake, 2002) to the multisensory domain and may be related to attentional bottom-up effects. Given that binding within a sensory modality is generally stronger than between sensory modalities, it is not surprising that our multisensory congruency effect of approximately 30% rise in dominance time is smaller than “classical” visual perceptual grouping effects (Sobel & Blake, 2002), which lengthened dominance durations by 40% or more. In addition to “true” multisensory integration, auditory motion may influence the dominance times of rivaling percepts via additional endogenous attentional mechanisms and voluntary control (Chong & Blake, 2006; Chong et al., 2005; Lack, 1978; Maruya et al., 2007; Meng & Tong, 2004). First, motion sound may withdraw attention from the visual stimulus (Alais, Parker, Boxtel, Paffen, & van Ee, 2007; Paffen et al., 2006; Pastukhov & Braun, 2007). However, a “withdrawal mechanism” should generally slow down temporal dynamics and cannot easily explain the congruency-selective effect observed in the current study. Second, a congruent audiovisual motion percept may attract more attention leading to longer dominance times. Though we cannot exclude this effect of attention, it is interesting to note that under non-rivaling conditions incongruent rather than congruent task-irrelevant auditory inputs lead to protracted response times (Adam & Noppeney, 2010; Laurienti, Kraft, Maldijan, Burdette, & Wallace, 2004; Molholm, Ritter, Javitt, & Foxe, 2004; Noppeney, Josephs, Hocking, Price, & Friston, 2008; Noppeney, Ostwald, & Werner, 2010; Weissman, Warner, & Woldorff, 2004) and increased processing in the fronto-parietal attentional system. Third, auditory input has recently been shown to enhance voluntary control and attentional selection in binocular rivalry. However, this auditory-induced attentional selection gain was only observed when subjects attended to the sounds but not when they ignored the auditory input as in our paradigm (van Ee et al., 2009). Collectively, these arguments suggest that endogenous top-down attention may not be the primary mechanism but rather amplify auditory influences initiated by genuine audiovisual integration. Finally, changes in dominance and suppression times may be attributed to high-level decisional biases induced by concurrent motion sound. For instance, observers may tend to report rightward motion for piecemeal periods with concurrent rightward sounds. While additional decisional influences cannot be fully excluded, we note that the dissociation of perceptual from decisional effects on subjects' report is a rather general problem in studies of binocular rivalry. 
Collectively, our results suggest that auditory motion may influence the dominance times of the congruent and incongruent percepts via genuine multisensory and attentional mechanisms at multiple levels of the cortical hierarchy: First, motion congruency enables the integration of auditory and visual inputs into a coherent more reliable percept possibly within motion area V5+/hMT+ (Arjen, Singer, & Muckli, 2008; Sadaghiani et al., 2009). Auditory motion signal thus resolves conflict within the visual modality and boosts the congruent visual motion percept analogous to bottom-up attentional mechanisms. Second, higher order endogenous attentional processes may further amplify the auditory influences and prolong the dominance times of the coherent motion percept via top-down or feedback influences (probably from parietal cortices). From a more theoretical perspective, multisensory influences in binocular rivalry are compatible with the idea that the human brain forms one unified and coherent generative model to explain inputs from multiple sensory modalities (Dayan, 1998; Hohwy, Roepstorff, & Friston, 2008; Leopold & Logothetis, 1999). 
Conclusion
The present study characterized the influence of auditory motion and non-motion signals on the temporal dynamics of rivaling visual motion percepts: 
  1.  
    Continuous auditory “non-motion” input without prominent temporal structure did not influence the temporal dynamics of rivaling visual motion percepts highlighting the role of spatiotemporal congruency for audiovisual interactions.
  2.  
    Auditory motion, even when unattended, prolonged the dominance periods of the congruent visual motion percept and abbreviated its suppression periods to a certain degree (or the dominance periods of the incongruent motion percept).
Our results demonstrate that auditory motion signals assist conflict resolution between the two rivaling visual motion percepts. Auditory motion may influence the dominance times of the congruent and incongruent percepts via genuine multisensory and higher order attentional mechanisms at multiple levels of the cortical hierarchy. From a more theoretical perspective, our results suggest that the temporal dynamics in binocular rivalry is determined by the brain's attempt to provide one unified interpretation explaining the inputs from all sensory modalities. 
Supplementary Materials
Supplementary Figure 1 - Supplementary Figure 1 
Supplementary Figure 2 - Supplementary Figure 2 
Supplementary materials - Supplementary materials 
Acknowledgments
This research was supported by the Max-Planck-Gesellschaft. 
Commercial relationships: none. 
Corresponding author: Verena Conrad. 
Email: verena.conrad@tuebingen.mpg.de. 
Address: Max Planck Institute for Biological Cybernetics, Spemannstr. 41, 72076 Tübingen, Germany. 
References
Adam R. Noppeney U. (2010). Prior auditory information shapes visual category selectivity in ventral-occipito-temporal cortex. Neuroimage, May 7, Epub ahead of print.
Alais D. Burr D. (2004). No direction-specific bimodal facilitation for audiovisual motion detection. Cognitive Brain Research, 19, 185–194. [CrossRef] [PubMed]
Alais D. Parker A. Boxtel J. Paffen C. van Ee R. (2007). Attending to auditory signals slows binocular rivalry. Perception, 36, ECVP Abstract Supplement.
Albright T. D. (1984). Direction and orientation selectivity of neurons in visual area MT of the macaque. Journal of Neurophysiology, 52, 1106–1130. [PubMed]
Arjen A. Singer W. Muckli L. (2008). Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. Journal of Neuroscience, 28, 2690–2697. [CrossRef] [PubMed]
Bartels A. Logothetis N. K. (in revision). Rivalry: A time dependence of monocular and percept contributions.
Blake R. Fox R. (1974). Binocular rivalry suppression: Insensitive to spatial frequency and orientation change. Vision Research, 14, 687–692. [CrossRef] [PubMed]
Blake R. Logothetis N. K. (2002). Visual competition. Nature Reviews Neuroscience, 3, 13–21. [CrossRef] [PubMed]
Blake R. Sobel K. V. James T. W. (2004). Neural synergy between kinetic vision and touch. Psychological Science, 15, 397–402. [CrossRef] [PubMed]
Blake R. Westerndorf D. H. Overton R. (1980). What is suppressed during binocular rivalry? Perception, 9, 223–231. [CrossRef] [PubMed]
Blauert J. (1983). Spatial hearing. Cambridge, MA: MIT Press.
Bossink C. J. H. Stalmeier P. F. M. de Weert C. M. M. (1993). A test of Levelt's second proposition for binocular rivalry. Vision Research, 33, 1413–1419. [CrossRef] [PubMed]
Brainard D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 443–446. [CrossRef] [PubMed]
Brascamp J. W. van Ee R. Noest A. J. Jacobs R. H. A. van den Berg A. V. (2006). The time course of binocular rivalry reveals a fundamental role of noise. Journal of Vision, 6, (11):8, 1244–1256, http://www.journalofvision.org/content/6/11/8, doi:10.1167/6.11.8. [PubMed] [Article] [CrossRef] [PubMed]
Breese B. (1909). Binocular rivalry. Psychological Review, 16, 410–415. [CrossRef]
Bremmer F. Schlack A. Shah N. J. Zafiris O. Kubischik M. Hoffmann K. P. , et al. (2001). Polymodal motion processing in posterior parietal and premotor cortex: A human fMRI study strongly implies equivalencies between humans and monkeys. Neuron, 29, 287–296. [CrossRef] [PubMed]
Britten K. H. Shadlen M. N. Newsome W. T. Movshon J. A. (1993). Responses of neurons in macaque MT to stochastic motion signals. Visual Neuroscience, 10, 1157–1169. [CrossRef] [PubMed]
Bruno N. Jacomuzzi A. C. Bertamini M. Meyer G. (2007). A visual‐haptic Necker cube reveals temporal constraints on intersensory merging during perceptual exploration. Neuropsychologia, 45, 469–475. [CrossRef] [PubMed]
Calvert G. Spence C. Stein B. E. (Eds.) (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.
Celebrini S. Newsome W. T. (1994). Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. Journal of Neuroscience, 14, 4109–4124. [PubMed]
Chong S. C. Blake R. (2006). Exogenous attention and endogenous attention influence initial dominance in binocular rivalry. Vision Research, 46, 1794–1803. [CrossRef] [PubMed]
Chong S. C. Tadin D. Blake R. (2005). Endogenous attention prolongs dominance durations in binocular rivalry. Journal of Vision, 5, (11):6, 1004–1012, http://www.journalofvision.org/content/5/11/6, doi:10.1167/5.11.6. [PubMed] [Article] [CrossRef]
Dayan P. (1998). A hierarchical model of binocular rivalry. Neural Computation, 10, 1119–1135. [CrossRef] [PubMed]
Ecker A. J. Heller L. M. (2005). Auditory-visual interactions in the perception of a ball's path. Perception, 34, 59–75. [CrossRef] [PubMed]
Frassinetti F. Bolognini N. Ladavas E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research, 147, 332–343. [CrossRef] [PubMed]
Hohwy J. Roepstorff A. Friston K. J. (2008). Predictive coding explains binocular rivalry: An epistemological review. Cognition, 108, 687–701. [CrossRef] [PubMed]
Kanai R. Moradi F. Shimojo S. Verstraten F. A. (2005). Perceptual alternation induced by visual transients. Perception, 34, 803–822. [CrossRef] [PubMed]
Kang M. Blake R. (2005). Perceptual synergy between seeing and hearing revealed during binocular rivalry. Psichologija, 32, 7–15.
Kayser C. Petkov C. I. Logothetis N. K. (2008). Visual modulation of neurons in auditory cortex. Cerebral Cortex, 18, 1560–1574. [CrossRef] [PubMed]
Kleiner M. Brainard D. Pelli D. (2007). What's new in Psychtoolbox-3? Perception, 36, ECVP Abstract Supplement.
Lack L. (1978). Selective attention and the control of binocular rivalry. The Hague, The Netherlands: Mouton.
Laurienti P. J. Kraft R. A. Maldijan J. H. Burdette J. H. Wallace M. T. (2004). Semantic congruence is a critical factor in multisensory behavioural performance. Experimental Brain Research, 158, 405–414. [CrossRef] [PubMed]
Lecuona A. R. Canadas-Quesada F. J. (2009).Interference of auditory information with haptic perception of stiffness in virtual reality. In Redondo, M. Bravo, C. Ortega M. (Eds.), Engineering the user interface: From research to practice (pp. 41–51). London, UK: Springer.
Leopold D. A. Logothetis N. K. (1999). Multistable phenomena: Changing views in perception. Trends in Cognitive Sciences, 3, 254–264. [CrossRef] [PubMed]
Levelt W. J. M. (1966). The alternation process in binocular rivalry. British Journal of Psychology, 57, 225–238. [CrossRef]
Levelt W. J. M. (1968). On binocular rivalry. The Hague, The Netherlands: Mouton.
Maeda F. Kanai R. Shimojo S. (2004). Changing pitch induced visual motion illusion. Current Biology, 14, R990.
Maier J. X. Ghazanfar A. A. (2007). Looming biases in monkey auditory cortex. Journal of Neuroscience, 27, 4093–4100. [CrossRef] [PubMed]
Maruya K. Yang E. Blake R. (2007). Voluntary action influences visual competition. Psychological Science, 18, 1090–1098. [CrossRef] [PubMed]
Mateef S. Hohnsbein J. Noack T. (1985). Dynamic visual capture: Apparent auditory motion induced by a moving visual target. Perception, 14, 721–727. [CrossRef] [PubMed]
Maunsell J. H. Van Essen D. C. (1983). Functional properties of neurons in middle temporal visual area of the macaque monkey. I Selectivity for stimulus direction, speed, and orientation. Journal of Neurophysiology, 49, 1127–1147. [PubMed]
Meng M. Tong F. (2004). Can attention selectively bias bistable perception Differences between binocular rivalry and ambiguous figures. Journal of Vision, 4, (7):2, 539–551, http://www.journalofvision.org/content/4/7/2, doi:10.1167/4.7.2. [PubMed] [Article] [CrossRef]
Meyer G. F. Wuerger S. (2001). Cross-modal integration of auditory and visual motion signals. Neuroreport, 12, 2557–2600. [CrossRef] [PubMed]
Meyer G. F. Wuerger S. M. Röhrbein F. Zetzsche C. (2005). Low-level integration of auditory and visual spatial colocalisation. Experimental Brain Research, 166, 538–547. [CrossRef] [PubMed]
Moutoussis K. Keliris G. Kourtzi Z. Logothetis N. K. (2005). A binocular rivalry study of motion perception in the human brain. Vision Research, 45, 2231–2243. [CrossRef] [PubMed]
Mueller T. J. Blake R. (1989). A fresh look at the temporal dynamics of binocular rivalry. Biological Cybernetics, 61, 223–232. [CrossRef] [PubMed]
Molholm S. Ritter W. Javitt D. C. Foxe J. J. (2004). Multisensory visual-auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex, 14, 452–465. [CrossRef] [PubMed]
Noppeney U. Josephs O. Hocking J. Price C. J. Friston K. J. (2008). The effect of prior visual information on recognition of speech and sounds. Cerebral Cortex, 18, 598–609. [CrossRef] [PubMed]
Noppeney U. Ostwald D. Werner S. (2010). Perceptual decisions formed by accumulation of audio-visual evidence in prefrontal cortex. Journal of Neuroscience, 30, 7434–7446. [CrossRef] [PubMed]
Paffen C. L. Alais D. Verstraten F. A. (2006). Attention speeds binocular rivalry. Psychological Science, 17, 752–756. [CrossRef] [PubMed]
Paffen C. L. van der Smagt M. J. te Pas S. F. Verstraten F. A. (2005). Center–surround inhibition and facilitation as a function of size and contrast at multiple levels of visual processing. Journal of Vision, 5, (6):8, 571–578, http://www.journalofvision.org/content/5/6/8, doi:10.1167/5.6.8. [PubMed] [Article] [CrossRef]
Parker A. L. Alais D. M. (2006). Auditory modulation of binocular rivalry [Abstract]. Journal of Vision, 6, (6):855, 855a, http://www.journalofvision.org/content/6/6/855, doi:10.1167/6.6.855. [CrossRef]
Pastukhov A. Braun J. (2007). Perceptual reversals need no prompting by attention. Journal of Vision, 7, (10):5, 1–17, http://www.journalofvision.org/content/7/10/5, doi:10.1167/7.10.5. [PubMed] [Article] [CrossRef] [PubMed]
Perrin X. Chavarriaga R. Ray C. Siegwart R. Milan J. (2008). A comparative psychophysical and EEG study of different feedback modalities for HRI. In Proceedings of the 3rd ACM/IEEE International Conference on Human‐Robot Interaction (pp. 41–48). New York, NY, USA.
Rees G. Friston K. Koch C. (2000). A direct quantitative relationship between the functional properties of human and macaque V5. Nature Neuroscience, 3, 716–723. [CrossRef] [PubMed]
Sadaghiani S. Maier J. X. Noppeney U. (2009). Natural, metaphoric and linguistic auditory direction signals have distinct influences on visual motion processing. Journal of Neuroscience, 29, 6490–6499. [CrossRef] [PubMed]
Schroeder C. E. Foxe J. J. (2005). Multisensory contributions to low-level, “unisensory” processing. Current Opinion in Neurobiology, 15, 454–458. [CrossRef] [PubMed]
Seifritz E. Neuhoff J. G. Bilecen D. Scheffler K. Mustovic H. Schächinger H. et al. (2002). Neural processing of auditory looming in the human brain. Current Biology, 12, 2147–2151. [CrossRef] [PubMed]
Shams L. Kamitani Y. Shimojo S. (2002). Visual illusion induced by sound. Cognitive Brain Research, 14, 147–152. [CrossRef] [PubMed]
Sobel K. V. Blake R. (2002). How context influences predominance during binocular rivalry. Perception, 31, 813–824. [CrossRef] [PubMed]
Soto-Faraco S. Kingstone A. Spence C. (2003). Multisensory contributions to the perception of motion. Neuropsychologia, 41, 1847–1862. [CrossRef] [PubMed]
Soto-Faraco S. Lyons J. Gazzaniga M. Spence C. Kingstone A. (2002). The ventriloquist in motion: Illusory capture of dynamic information across sensory modalities. Cognitive Brain Research, 14, 139–146. [CrossRef] [PubMed]
Soto-Faraco S. Spence C. Kingstone A. (2004). Moving multisensory research along: Motion perception across sensory modalities. Current Directions in Psychological Science, 13, 29–32. [CrossRef]
Tong F. Meng M. Blake R. (2006). Neural bases of binocular rivalry. Trends in Cognitive Sciences, 10, 502–511. [CrossRef] [PubMed]
van Ee R. van Boxtel J. J. A. Parker A. Alais D. (2009). Multisensory congruency as a mechanism for attentional control over perceptual selection. Journal of Neuroscience, 29, 11641–11647. [CrossRef] [PubMed]
van Ee R. van Dam L. C. Brouwer G. J. (2005). Voluntary control and the dynamics of perceptual bi-stability. Vision Research, 45, 41–55. [CrossRef] [PubMed]
Walker P. Powell D. J. (1979). The sensitivity of binocular rivalry to changes in the nondominant stimulus. Vision Research, 19, 247–249. [CrossRef] [PubMed]
Weissman D. H. Warner L. M. Woldorff M. G. (2004). The neural mechanisms for minimizing cross-modal distraction. Journal of Neuroscience, 24, 10941–10949. [CrossRef] [PubMed]
Werner S. Noppeney U. (2010). Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cerebral Cortex, 20, 1829–1842. [CrossRef] [PubMed]
Wolf M. Hochstein S. (2007). High-level effects in binocular rivalry. Perception, 36, ECVP Abstract Supplement.
Wolfe J. M. (1984). Reversing ocular dominance and suppression in a single flash. Vision Research, 24, 471–478. [CrossRef] [PubMed]
Figure 1
 
Experimental paradigm, stimuli, and setup. (a) Experiment 1: The two eyes were presented with unidirectional motion of opposite directions at 100% coherence. (b) Experiment 2: The two eyes were presented with alternating motion of opposite directions at 100% coherence. (c) Experiment 3: Eye 1 was presented alternating motion at 60% coherence; eye 2 with random motion at 0% coherence. In all three experiments, the sound was (i) absent, (ii) non-motion, or (iii) directional motion. The arrows indicate the direction of motion of the RDKs. (e, f) Subjects viewed stimuli in a darkened stereo-setup. LE: left eye, RE: right eye. See text for details.
Figure 1
 
Experimental paradigm, stimuli, and setup. (a) Experiment 1: The two eyes were presented with unidirectional motion of opposite directions at 100% coherence. (b) Experiment 2: The two eyes were presented with alternating motion of opposite directions at 100% coherence. (c) Experiment 3: Eye 1 was presented alternating motion at 60% coherence; eye 2 with random motion at 0% coherence. In all three experiments, the sound was (i) absent, (ii) non-motion, or (iii) directional motion. The arrows indicate the direction of motion of the RDKs. (e, f) Subjects viewed stimuli in a darkened stereo-setup. LE: left eye, RE: right eye. See text for details.
Figure 2
 
Experiment 1: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. * Significant difference in mean dominance time at p < 0.05, see Table 1.
Figure 2
 
Experiment 1: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. * Significant difference in mean dominance time at p < 0.05, see Table 1.
Figure 3
 
Experiment 1: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. Percent cumulative dominance times for the stable percepts in the non-motion and no-sound conditions were averaged (not summed) across the two eyes for comparison with the percent cumulative dominance times of the congruent and incongruent percepts during the motion sound conditions. Therefore, percent cumulative dominance times during the non-motion and no-sound conditions do not sum to 100%. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 1.
Figure 3
 
Experiment 1: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. Percent cumulative dominance times for the stable percepts in the non-motion and no-sound conditions were averaged (not summed) across the two eyes for comparison with the percent cumulative dominance times of the congruent and incongruent percepts during the motion sound conditions. Therefore, percent cumulative dominance times during the non-motion and no-sound conditions do not sum to 100%. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 1.
Figure 4
 
Experiment 2: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in mean dominance time at p < 0.05, see Table 2.
Figure 4
 
Experiment 2: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in mean dominance time at p < 0.05, see Table 2.
Figure 5
 
Experiment 2: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 2.
Figure 5
 
Experiment 2: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = congruent percept; light gray = incongruent percept; white = indeterminate percept; dark gray = dominance durations averaged across motion percepts of both directions separately for the non-motion and no-sound conditions. *Significant difference in percent cumulative dominance time at p < 0.05, see Table 2.
Figure 6
 
Experiment 3: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept. *Significant difference in mean dominance time at p < 0.05, see Table 3.
Figure 6
 
Experiment 3: Bar plot of mean dominance durations (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept. *Significant difference in mean dominance time at p < 0.05, see Table 3.
Figure 7
 
Experiment 3: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept; white = indeterminate percept *Significant difference in percent cumulative dominance times at p < 0.05, see Table 3.
Figure 7
 
Experiment 3: Bar plot of percent cumulative dominance time (across subjects mean ± SE). Percepts are color coded: black = coherent motion percept; dark gray = random motion percept; white = indeterminate percept *Significant difference in percent cumulative dominance times at p < 0.05, see Table 3.
Table 1
 
Experiment 1: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
Table 1
 
Experiment 1: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
df:(9) Motion-sound incongruent Non-motion sound No sound
Mean dominance times in seconds
Motion-sound congruent t = 2.95, p < 0.05* t = 5.48, p < 0.05* t = 2.88, p < 0.05*
Motion-sound incongruent t = 0.84, p > 0.05 t = 1.45, p > 0.05
Non-motion sound t = 1.72, p > 0.05
Cumulative dominance in percent
Motion-sound congruent t = 3.06, p < 0.05* t = 3.00, p < 0.05* t = 2.40, p < 0.05*
Motion-sound incongruent t = 1.10, p > 0.05 t = 2.17, p = 0.06
Non-motion sound t = 1.40, p > 0.05
 

*Significant difference at p < 0.05.

Table 2
 
Experiment 2: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
Table 2
 
Experiment 2: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
df:(11) Motion-sound incongruent Non-motion sound No sound
Mean dominance times in seconds
Motion-sound congruent t = 6.01, p < 0.05* t = 4.61, p < 0.05* t = 4.84, p < 0.05*
Motion-sound incongruent t = 3.91, p < 0.05* t = 3.33, p < 0.05*
Non-motion sound t = 1.53, p > 0.05
Cumulative dominance in percent
Motion-sound congruent t = 6.16, p < 0.05* t = 5.32, p < 0.05* t = 5.39, p < 0.05*
Motion-sound incongruent t = 4.44, p < 0.05* t = 3.93, p < 0.05*
Non-motion sound t = 1.69, p > 0.05
 

*Significant difference at p < 0.05.

Table 3
 
Experiment 3: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
Table 3
 
Experiment 3: Pairwise statistical comparisons of mean dominance times and percent cumulative dominance with congruent/incongruent percepts separated.
df:(9) Motion-sound incongruent Noise No sound
Mean dominance times in seconds
Motion-sound congruent t = 2.31, p < 0.05* t = 2.50, p < 0.05* t = 1.93, p < 0.05*
Motion-sound incongruent t = 0.51, p > 0.05 t = 0.13, p > 0.05
Noise t = 0.44, p > 0.05
Cumulative dominance in percent
Motion-sound congruent t = 2.04, p < 0.05* t = 2.87, p < 0.05* t = 3.98, p < 0.05*
Motion-sound incongruent t = 0.10, p > 0.05 t = 0.13, p > 0.05
Noise t = 0.42, p > 0.05
 

*Significant difference at p < 0.05.

Supplementary Figure 1
Supplementary Figure 2
Supplementary materials
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×