August 2020
Volume 20, Issue 8
Open Access
Article  |   August 2020
Brief period of monocular deprivation drives changes in audiovisual temporal perception
Author Affiliations
  • Collins Opoku-Baah
    Neuroscience Graduate Program, Vanderbilt University, Nashville, TN, USA
    Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
    [email protected]
  • Mark T. Wallace
    Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
    Department of Psychology, Vanderbilt University, Nashville, TN, USA
    Department of Hearing and Speech, Vanderbilt University Medical Center, Nashville, TN, USA
    Vanderbilt Vision Research Center, Nashville, TN, USA
    Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
    Department of Pharmacology, Vanderbilt University, Nashville, TN, USA
    [email protected]
Journal of Vision August 2020, Vol.20, 8. doi:https://doi.org/10.1167/jov.20.8.8
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Collins Opoku-Baah, Mark T. Wallace; Brief period of monocular deprivation drives changes in audiovisual temporal perception. Journal of Vision 2020;20(8):8. https://doi.org/10.1167/jov.20.8.8.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The human brain retains a striking degree of plasticity into adulthood. Recent studies have demonstrated that a short period of altered visual experience (via monocular deprivation) can change the dynamics of binocular rivalry in favor of the deprived eye, a compensatory action thought to be mediated by an upregulation of cortical gain control mechanisms. Here, we sought to better understand the impact of monocular deprivation on multisensory abilities, specifically examining audiovisual temporal perception. Using an audiovisual simultaneity judgment task, we discovered that 90 minutes of monocular deprivation produced opposing effects on the temporal binding window depending on the eye used in the task. Thus, in those who performed the task with their deprived eye there was a narrowing of the temporal binding window, whereas in those performing the task with their nondeprived eye there was a widening of the temporal binding window. The effect was short lived, being observed only in the first 10 minutes of postdeprivation testing. These findings indicate that changes in visual experience in the adult can rapidly impact multisensory perceptual processes, a finding that has important clinical implications for those patients with adult-onset visual deprivation and for therapies founded on monocular deprivation.

Introduction
The brain possesses a remarkable degree of plasticity that enables it to reorganize to meet the demands of the changing environment (Pascual-Leone, Amedi, Fregni, & Merabet, 2005). This intrinsic property of the nervous system peaks during a defined period in development, the so-called critical or sensitive period (Berardi, Pizzorusso, & Maffei, 2000). During this period of plasticity, changes in sensory experience can engender morphological and functional changes that have enduring effects on sensory function. 
Seminal work by Hubel and Wiesel discovered that early visual deprivation produced several changes in the structural and neurophysiological properties of neurons and circuits in the lateral geniculate nucleus and the visual cortex (Hubel & Wiesel, 1970; Wiesel & Hubel, 1963, 1965a, 1965b). They described that, after a few weeks to months of visual deprivation, there was a decrease in the number and overall activity of neurons in these brain regions that responded to the deprived eye, as well as changes in receptive field properties of these cells and a reorganization of ocular dominance columns in favor of the open eye (Hubel & Wiesel, 1970; Wiesel & Hubel, 1963). Importantly, these deprivation-induced changes endured after years of restoring normal sensory experience to the deprived eye (Wiesel & Hubel, 1965b). Furthermore, these changes were found to be more profound in response to monocular versus binocular deprivation, suggesting that an imbalance of input to the two eyes is a critical factor (Wiesel & Hubel, 1965a). In humans, early visual deprivation can occur through conditions such as untreated congenital cataracts and uncorrected refractive errors and is associated with permanent defects in performance of visual perceptual tasks, such as visual acuity, contrast detection, and shape and depth perception (Fine, Smallman, Doyle, & MacLeod, 2002; Fine et al., 2003; Levi, McKee, & Movshon, 2011; Ostrovsky, Andalman, & Sinha, 2006). 
In recent years, the study of multisensory processing has received a great deal of attention. A growing body of compelling evidence supports the interaction and/or integration of information from different sensory modalities at various brain levels, especially in areas which were once considered to be classically “unisensory” (Foxe & Schroeder, 2005; Ghazanfar & Schroeder, 2006; Stein & Stanford, 2008). Integrating information from multiple senses has several benefits, including striking response gains at the neural level (Stein & Meredith, 1993; Stein & Stanford, 2008), as well as enhanced detection (Frassinetti, Bolognini, & Làdavas, 2002; Lovelace, Stein, & Wallace, 2003), discrimination (Ernst & Banks, 2002), localization (Zou, Müller, & Shi, 2012), and speeding response times (Diederich & Colonius, 2004) at the perceptual level. These neural and perceptual benefits depend on the physical characteristics of the combined stimuli, with the temporal proximity between the different cues that make up a multisensory stimulus being one of the important factors. Intuitively, natural events that originate from a common source are likely to arrive at the brain centers in close temporal proximity and, thus, integrating them may enhance perceptual saliency and performance. Indeed, the largest neural and perceptual gains are achieved when the two inputs making up a multisensory stimulus are close in time (Meredith, Nemitz, & Stein, 1987). 
Extending this work into the psychophysical arena, a number of studies have focused on characterizing the window of stimulus onset asynchronies (SOAs) within which two stimuli are likely to be integrated or perceptually “bound” (Wallace & Stevenson, 2014). This window, termed the temporal binding window (TBW), is essentially a measure of multisensory temporal acuity with a larger window indicating poorer acuity and an increased likelihood that the brain will bind events that do not originate from a common source. Over the years, several studies have revealed some notable characteristics of the TBW. For example, the size of the TBW varies significantly across participants (Stevenson, Zemtsov, & Wallace, 2012). This individual variability has been linked with multisensory integrative abilities, such as individual susceptibility to the McGurk effect, scored as an illusory perception of /da/ or /ta/ when an observer fuses an auditory syllable /ba/ and a visual syllable /ga/ (Stevenson et al., 2012). In addition, the TBW size has been shown to be larger in clinical populations such as those with autism and schizophrenia (Foss-Feig et al., 2010; Stevenson et al., 2014; Stevenson et al., 2017; Noel, Stevenson, & Wallace, 2018). As with the adult plasticity seen in the visual system, multisensory temporal function has been shown to be highly malleable. For example, a short interval of perceptual training can significantly narrow the TBW (Powers, Hevey, & Wallace, 2012; Powers, Hillock, & Wallace, 2009). 
Interestingly, altered visual experience has been shown to influence the maturation of multisensory integration during development. For example, work in animal models has shown that early visual deprivation impairs the ability of neurons to synthesize multisensory information (Carriere et al., 2007; Wallace, Perrault, Hairston, & Stein, 2004). In humans, imaging studies have revealed large-scale cross-modal reorganization of the visual cortex in patients who were visually deprived briefly during their early life (Collignon et al., 2015; Guerreiro, Putzar, & Röder, 2015). Perceptually, the effects of early visual deprivation in humans can also impair audiovisual interactions, most notably in the temporal domain (Chen, Lewis, Shore, & Maurer, 2017; Putzar, Goerendt, Lange, Rösler, & Röder, 2007; Putzar, Hötting, & Röder, 2010). For instance, patients who experienced visual deprivation owing to congenital cataracts showed wider TBWs when compared with age-matched typically developed controls during the performance of an audiovisual simultaneity judgment task (Chen et al., 2017). In addition, although monocularly deprived patients exhibited widened TBWs characteristic of an immature audiovisual integration system, binocularly deprived patients showed widened TBW that did not match any stage in typical development (Chen et al., 2017). Together, these findings indicate that normal visual experience is crucial for the development of both visual and multisensory (i.e., visual–nonvisual) functions. 
Although the impact of atypical early visual experience on visual and multisensory functions have been well-established, the question of whether a short-term change in visual experience in adults can drive plastic changes in multisensory temporal function remains unanswered. Recent studies have shown that, despite the decreased plasticity in the brain after the closure of the critical period, the adult brain still retains marked plasticity that can impact perception and behavior (Berardi et al., 2000; Sabel, 2008). One example of such adult plasticity is the perceptual effect of short-term monocular deprivation. Occluding one eye with a translucent patch for 150 minutes boosts the perceptual dominance of the deprived eye during subsequent binocular rivalry and increases the apparent contrast of viewed gratings (Lunghi, Burr, & Morrone, 2011). According to Lunghi and her colleagues (Lunghi et al., 2011), this finding reflects a form of rapid homeostatic plasticity to visual deprivation and is mediated by an upregulation of contrast gain control mechanisms of the deprived eye in an attempt to optimize the attenuated or absent visual information. This enhanced performance in the deprived eye has been demonstrated across different visual tasks, such as phase combination, global motion coherence, and contrast matching and detection (Zhou, Clavagnier, & Hess, 2013), and is independent of the nature of deprivation (Chadnova, Reynaud, Clavagnier, & Hess, 2017; Kim, Kim, & Blake, 2017; Zhou et al., 2013). Given the evidence of the malleability of the TBW to effects of visual deprivation-induced plasticity during development, we hypothesized that a brief period of monocular deprivation will impact the TBW in adult humans through homeostatic plasticity. Such a result would reinforce the importance of adult visual experience in maintaining not only visual representations, but also in maintaining multisensory (e.g., audiovisual) representations. 
Methods
Participants
Twenty undergraduates at Vanderbilt University were recruited to perform an audiovisual simultaneity judgment task using either their deprived or nondeprived eye before and after 90 minutes of monocular deprivation. Each participant was randomly assigned to either the deprived eye group, n = 10, 3 males, mean age 19.1 ± 3.0 years, or the nondeprived eye group, n = 10, 4 males, mean age 18.9 ± 1.2 years. All students had normal or corrected-to-normal vision and normal hearing. Normal vision was confirmed using a visual acuity task. Participants gave informed consent before being allowed to participate and received course credits for their participation. All recruitment and experimental procedures were approved by the Vanderbilt University Institutional Review Board. This work was carried out in accordance with the Declaration of Helsinki. 
Stimulus and apparatus
The experimental stimuli were generated and presented using MATLAB (Math Works Inc., Natick, MA) software with the Psychophysics Toolbox Version 3 (Brainard & Vision, 1997; Pelli, 1997). The visual stimulus was displayed on a gamma-corrected monitor (21-inch Asus LCD) with a 120-Hz refresh rate and the auditory stimulus was presented binaurally through headphones (Sennheiser HD559). The visual stimulus was a gabor patch, sigma = 0.5°, which had a spatial frequency of 4 cycles per degree, subtended an angle of 6° and was presented at the center of fixation. The gabor patch was displayed at 20% contrast on a uniform gray background with a luminance of 10 cd/m2. The auditory stimulus comprised a white noise burst at approximately 75 dB and with a sampling frequency of 41 kHz. Both stimuli were presented for four frame periods equaling 33 ms. Although the intensity of the auditory stimulus was linearly ramped up and down, each for 25% of the stimulus duration, a similar effect was achieved with the visual stimulus by presenting the first and last frames at one-half of the actual contrast of the stimulus. A Minolta Chroma Meter CS-100 and a sound level meter were used to verify the luminance and sound intensity levels, respectively. The durations of all visual and auditory stimuli, as well as the SOAs, were confirmed using a Hameg 507 oscilloscope with a photovoltaic cell and microphone. 
Procedure
Participants performed the experiment inside a WhisperRoom (SE 2000 Series) with their forehead and chin placed comfortably against a HeadSpot (University of Houston Optometry). The whisper room was only lit by the background luminance of the screen. Before the experiment began, each participant was randomly assigned to either the deprived eye group or the nondeprived eye group. The eye to be deprived was randomly chosen for each participant and counterbalanced across all participants. The deprived eye group performed the task with their deprived eye and the nondeprived eye group performed the task with their nondeprived eye before and after 90 minutes of monocular deprivation with an opaque eye patch (Figure 1A). Both the predeprivation and the postdeprivation sessions comprised two blocks of an unspeeded audiovisual simultaneity judgment task. A block took approximately 10 minutes to complete. During each block, the participant viewed a gabor patch and heard an auditory white noise burst after a brief fixation period lasting between 600 and 1000 ms (Figure 1B). The white circular dot that marked the fixation period was always present and participants were instructed to always fixate at it. The onsets of the two stimuli were separated by 13 parametrically varied SOAs (in ms): −500, −350, −200, −150, −100, −50, 0, 50, 100, 150, 200, 350, and 500. Negative and positive SOA values corresponded with auditory preceding vision and vision preceding auditory SOAs, respectively. Each SOA was presented 20 times (totaling 260 trials) in randomized fashion. After each presentation, participants accurately delivered their response by pressing “1” on the keyboard if the pair of audiovisual stimuli was synchronous or by pressing “2”, if the pair was asynchronous. Each participant was given two brief initial practice sessions, each consisting of nine SOAs and 10 trials per SOA. The rationale for this initial practice was to, first, exclude participants who could not perform adequately well on the task and, second, decrease familiarization effects during the main experiment. No feedback was provided on the correctness of their responses during the main experiment. 
Figure 1.
 
Experimental procedure. (A) Participants were randomly assigned to either the deprived eye or nondeprived eye group. Participants in the deprived eye group performed the audiovisual temporal simultaneity judgment task with their deprived eye before and after 90 minutes of deprivation. For the nondeprived eye group, the deprivation phase started with the first predeprivation block of the nondeprived eye. The postdeprivation blocks were then performed using the nondeprived eye after 90 minutes from the start of the first predeprivation block. (B) On each block, participants judged the simultaneity of a gabor patch (33 ms) and an auditory white noise burst (33 ms). The onsets of the two stimuli were separated by a range of preselected SOAs between –500 ms and 500 ms. Negative and positive SOAs signified auditory leading and visual leading respectively. On each trial, there was a brief fixation period (600–1000 ms), followed by the stimulus presentation. Participants were then asked to respond by pressing the keyboard after which the next trial began automatically.
Figure 1.
 
Experimental procedure. (A) Participants were randomly assigned to either the deprived eye or nondeprived eye group. Participants in the deprived eye group performed the audiovisual temporal simultaneity judgment task with their deprived eye before and after 90 minutes of deprivation. For the nondeprived eye group, the deprivation phase started with the first predeprivation block of the nondeprived eye. The postdeprivation blocks were then performed using the nondeprived eye after 90 minutes from the start of the first predeprivation block. (B) On each block, participants judged the simultaneity of a gabor patch (33 ms) and an auditory white noise burst (33 ms). The onsets of the two stimuli were separated by a range of preselected SOAs between –500 ms and 500 ms. Negative and positive SOAs signified auditory leading and visual leading respectively. On each trial, there was a brief fixation period (600–1000 ms), followed by the stimulus presentation. Participants were then asked to respond by pressing the keyboard after which the next trial began automatically.
Analysis
Each participant completed two predeprivation and two postdeprivation blocks of the audiovisual simultaneity judgment task. Proportions of synchrony reports as a function of SOA were computed for each block and each participant. We then pooled (i.e., averaged) the data from the two predeprivation blocks into one combined predeprivation block and likewise, the two postdeprivation blocks into one postdeprivation block. These data were then fitted separately with a single-term Gaussian distribution model (through MATLAB fit.m) whose amplitude, mean and standard deviation were free to vary (Noel et al., 2018; Simon, Noel, & Wallace, 2017). Unlike the mean and the standard deviation parameters, the range of possible amplitude values was bound between 0 and 1. The mean and the standard deviation of the best fitting model were then taken as the point of subjective simultaneity (PSS) and the temporal window of simultaneity (TBW), respectively. The averaged r2 values across participants in both groups showed reasonable fits to the data for the combined predeprivation block (0.92 ± 0.01) and the combined postdeprivation block (0.91 ± 0.02). To determine whether the deprivation effect differed between the first and second postdeprivation blocks, the Gaussian distribution described elsewhere in this article was fitted to the data for the two postdeprivation blocks separately and their PSS and TBW measures were derived. Again, the averaged r2 values across participants for the two postdeprivation blocks were satisfactory: postblock 1 (0.89 ± 0.03) and postblock 2 (0.87 ± 0.04). Next, we computed the effect of deprivation on the performance measures for each participant by normalizing the postdeprivation measure (either combined or divided) to the baseline predeprivation measure. For the TBW, the relative change calculation ((post–pre)/pre × 100%) was used, whereas for the PSS an absolute change (post–pre) computation was used because of the inclusion of negative numbers. 
The following statistical analyses were then carried out for each performance measure using the JASP software version 0.11 (JASP Team, 2018). First, an independent samples t-test was done to determine whether participants in both groups were comparable in their predeprivation performance. Second, a one-sample t-test against zero (baseline) was used to determine whether the computed deprivation effect (i.e., absolute or relative change) was statistically significant for each eye group. Third, we conducted an independent samples t-test to determine whether the absolute change in PSS differed between the deprived and nondeprived eye groups. For the TBW, a one-way analysis of covariance test was done to determine whether the relative change in TBW differed between the deprived and nondeprived eye groups while adjusting for the individual variability in baseline TBW. For the divided blocks, we conducted multiple t-tests (for four comparisons in total) to determine 1) whether the deprivation effect differed between the first and second postdeprivation blocks for each group and 2) whether the deprivation effect differed between the two groups for each postdeprivation block. Note that, in the latter scenario, a one-way analysis of covariance test was conducted to adjust for the between-participant differences in the predeprivation TBW. Correlation analyses were done using Pearson's correlation. All statistical analyses were two-tailed and significance level, alpha, was set to 0.05. In the case of multiple tests, alpha was adjusted with a Bonferroni-Holm correction. Furthermore, the statistical results were reported in both frequentist (i.e., p values) and Bayesian (i.e. Bayes factors) frameworks of hypothesis testing. 
Results
The proportions of synchrony reports averaged across participants are plotted as a function of SOA for the deprived eye group (red) and the nondeprived eye group (blue) in Figure 2. Whereas Figure 2A displays the results between the averaged predeprivation blocks (dashed line) and the first postdeprivation block (solid line), Figure 2B shows the results for the averaged predeprivation blocks (dashed line) and the second postdeprivation block (solid line). For each performance measure (i.e., PSS and TBW), an independent samples t-test on the combined predeprivation blocks showed no statistically significant difference between the two eye groups, indicating that both groups were comparable at baseline, PSS: t(18) = 0.52, p = 0.611, BF10 = 0.437; TBW: t(18) = −1.55, p = 0.138, BF10 = 0.904. 
Figure 2.
 
Mean proportion of synchrony reports. Proportion of synchrony reports averaged across participants in the deprived eye group (red) and the nondeprived eye group (blue) is plotted as a function of SOA for (A) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line) and for (B) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line).
Figure 2.
 
Mean proportion of synchrony reports. Proportion of synchrony reports averaged across participants in the deprived eye group (red) and the nondeprived eye group (blue) is plotted as a function of SOA for (A) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line) and for (B) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line).
No effect of deprivation on PSS
The PSS is the mean of the TBW and is defined as the SOA at which maximum perceived simultaneity is perceived. Essentially, the PSS measures which modality is given more weight in determining audiovisual temporal simultaneity. Interestingly, the PSS is not usually zero and may be shifted toward the auditory-leading (i.e., more visual weight or bias) or visual-leading (i.e., more auditory weight or bias) side, depending on stimulus-related factors such as stimulus duration and intensity (Boenke, Deliano, & Ohl, 2009; Jaśkowski, 1999; Sanford, 1971), task-related factors such as judging the onset versus the offset in an simultaneity–judgment task (Wen, Opoku-Baah, Park, & Blake, 2020), and attention-related factors such as being asked to attend to one modality (Schneider & Bavelier, 2003; Stelmach & Herdman, 1991; Zampini, Shore, & Spence, 2005). To determine the effect of monocular deprivation on the PSS, we computed the absolute change in the PSS before and after deprivation. Based on this computation, a negative change signified an increase in visual bias or weighting while a positive change signified a decrease in visual bias. 
The effect of 90 minutes of monocular deprivation on the PSS is displayed in Figure 3. Our results showed that the absolute change in PSS was not statistically significant for either the deprived eye, DG: t(9) = −0.45, Bonferroni-Holm adjusted alpha = 0.05, p = 0.663, Cohen d = −0.142, BF10 = 0.337, or nondeprived eye, NG: t(9) = 1.68, Bonferroni-Holm adjusted alpha = 0.025, p = 0.128, Cohen d = −0.53, BF10 = 0.884, groups when data were pooled across the entire postdeprivation phase. Comparing the mean absolute change in PSS between the deprived and nondeprived eye groups showed no significant difference, t(18) = −1.59, p = 0.128, Cohen d = −0.713, BF10 = 0.943. Next, we determined whether the absolute change in PSS differed between the groups for each postdeprivation phase as well as between the postdeprivation phases for each group. Our results revealed that for both the first and second postdeprivation phases, there was no statistically significant difference between the mean absolute change in PSS for the deprived group and that of the nondeprived group, first: t(18) = –1.28, Bonferroni-Holm adjusted alpha = 0.0167, p = 0.215, Cohen d = −0.574, BF10 = 0.702; second: t(18) = −1.64, Bonferroni-Holm adjusted alpha = 0.0125, p = 0.118, Cohen d = −0.734, BF10 = 0.992. Also, the mean absolute change in PSS did not differ significantly between the first and second postdeprivation phases for the deprived group, t(9) = −0.71, Bonferroni-Holm adjusted alpha = 0.05, p = 0.499, Cohen d = −0.223, BF10 = 0.381, and for the nondeprived group, t(9) = −0.98, Bonferroni-Holm adjusted alpha = 0.025, p = 0.353, Cohen d = −0.309, BF10 = 0.457. 
Figure 3.
 
Effects of deprivation on the PSS. (A) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Absolute change was computed as postdeprivation – predeprivation. (B) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Figure 3.
 
Effects of deprivation on the PSS. (A) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Absolute change was computed as postdeprivation – predeprivation. (B) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Deprivation has contrasting effects on the TBW when indexed via the deprived and nondeprived eyes
As mentioned elsewhere in this article, the TBW is an index for multisensory temporal acuity with a narrow window signifying greater acuity and vice versa. To assess the impact of monocular deprivation on the size of the TBW, we computed the relative change in TBW between the postdeprivation and predeprivation phases when the data was pooled across the entire postdeprivation phase. In contrast with the lack of changes in PSS, deprivation resulted in a number of significant changes in the TBW (Figure 4). First, results from the combined postdeprivation block showed a significant decrease (i.e., negative relative change) in the TBW for the deprived eye group, DG: t(9) = −4.99, Bonferroni-Holm adjusted alpha = 0.025, p < 0.001, Cohen d = −1.578, BF10 = 52.012, and a significant increase (i.e., positive relative change) in the TBW for the nondeprived eye group, NG: t(9) = 2.53, Bonferroni-Holm adjusted alpha = 0.05, p = 0.032, Cohen d = 0.801, BF10 = 2.477, after deprivation (Figure 4A). To determine whether the two groups differed in their mean relative change in TBW, we conducted a one-way analysis of covariance test controlling for the individual differences in predeprivation TBW. Our analysis revealed a significant difference between the mean relative change in TBW for the deprived eye group versus that for the nondeprived eye, F(1, 17) = 16.47, p < 0.001, η2p = 0.492, BF10 = 107.06, while adjusting for predeprivation TBW. 
Figure 4.
 
Effects of deprivation on the TBW. (A) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Relative change was computed as (postdeprivation – predeprivation)/predeprivation × 100%. (B) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Figure 4.
 
Effects of deprivation on the TBW. (A) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Relative change was computed as (postdeprivation – predeprivation)/predeprivation × 100%. (B) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Comparing the mean relative change in TBW between the deprived eye and nondeprived eye groups while adjusting for predeprivation TBW showed that the mean difference between the groups was significant during the first postdeprivation phase, F(1, 17) = 24.31, Bonferroni-Holm adjusted alpha = 0.0125, p < 0.001, η2p = 0.588, BF10 = 619.3, and nonsignificant during the second postdeprivation phase, F(1, 17) = 2.42, Bonferroni-Holm adjusted alpha = 0.025, p = 0.138, η2p = 0.125, BF10 = 2.167 (Figure 4B). In contrast, the mean relative change in TBW differed significantly between the first postdeprivation phase and second postdeprivation phase for the deprived eye group, DG: t(9) = –3.38, Bonferroni-Holm adjusted alpha = 0.0167, p = 0.008, Cohen d = −1.07, BF10 = 7.338. However, for the nondeprived eye group, there was no significant difference between the mean relative change in TBW for the first postdeprivation and second postdeprivation phases, NG: t(9) = −0.71, Bonferroni-Holm adjusted alpha = 0.05, p = 0.499, Cohen d = −0.223, BF10 = 0.381 (Figure 4B). These results emphasize two points: first, the differential impact of deprivation on the TBW of the deprived eye versus the nondeprived eye occurred only in the first deprivation phase and, second, the lack of significant difference observed during the second deprivation phase is attributable to the waning of the deprivation-induced effect observed in the deprived eye. 
Relative change in TBW strongly correlates with the absolute change in PSS
Finally, we conducted a Pearson's correlation between the absolute change in PSS values pooled across all participants and postdeprivation blocks and the corresponding relative change in TBW values (Figure 5). Our analysis revealed a strong positive correlation between the absolute change in PSS values and the relative change in TBW values, r = 0.7, p < 0.001, log(BF10) = 10.607. This finding indicates that an enhancement in sensitivity to audiovisual temporal asynchrony (i.e., narrowing of the TBW), which was observed mostly for the deprived eye group, is associated with PSS shifts toward the auditory leading side (i.e., an increase in visual bias or weighting). In contrast, a decrease in temporal acuity or a widening of the TBW, which was observed mostly in the nondeprived eye group, is associated with PSS shifts toward the visual leading side (i.e., a decrease in visual bias or weighting). 
Figure 5.
 
Correlation between the absolute change in PSS and the relative change in temporal window of simultaneity (TWS). Plotted on the x-axis the is relative change in TWS data pooled from participants across both eye groups (deprived eye: red; nondeprived eye group: blue) and the two postdeprivation blocks (first: circle; second: diamond) and plotted on the y-axis is the corresponding absolute change in PSS data. Positive and negative data points on the x-axis signify enhancement (narrowing of TWS) and depreciation (widening of TWS) of sensitivity to audiovisual temporal asynchrony, whereas positive and negative values on the y-axis signify more (negative shift in PSS) and less (positive shift in PSS) dominance (dom.) of the visual modality respectively. Our results revealed a strong positive correlation between relative change in TWS and the absolute change in PSS, r = 0.7, p < 0.001.
Figure 5.
 
Correlation between the absolute change in PSS and the relative change in temporal window of simultaneity (TWS). Plotted on the x-axis the is relative change in TWS data pooled from participants across both eye groups (deprived eye: red; nondeprived eye group: blue) and the two postdeprivation blocks (first: circle; second: diamond) and plotted on the y-axis is the corresponding absolute change in PSS data. Positive and negative data points on the x-axis signify enhancement (narrowing of TWS) and depreciation (widening of TWS) of sensitivity to audiovisual temporal asynchrony, whereas positive and negative values on the y-axis signify more (negative shift in PSS) and less (positive shift in PSS) dominance (dom.) of the visual modality respectively. Our results revealed a strong positive correlation between relative change in TWS and the absolute change in PSS, r = 0.7, p < 0.001.
Discussion
Even after the closure of the critical period, the adult brain retains a marked degree of plasticity that can impact perception and behavior, and moreover, support recovery and rehabilitation after injury (Berardi et al., 2000; Sabel, 2008). Much of the evidence supporting this claim for the visual system has been provided through studies of perceptual learning (Goldstone, 1998; Huxlin, 2008) and, recently, through brief periods of monocular deprivation (Lunghi et al., 2011; Zhou et al., 2013). Adding to this body of knowledge, our study provides the first clear evidence that short-term monocular deprivation can induce changes in multisensory (i.e., audiovisual) temporal perception. Using an audiovisual simultaneity judgment task, we discovered that 90 minutes of monocular deprivation produced contrasting effects on the TBW (a measure of audiovisual temporal acuity), narrowing the TBW of participants who performed the task with their deprived eye and widening the TBW of those who used their nondeprived eye. Moreover, we observed that this effect was short lived, occurring only in the first phase of postdeprivation testing and took place within 10 minutes of the removal of deprivation. 
During visual deprivation, the loss of visual information also impacts any visual-based multisensory processing (Carriere et al., 2007). In fact, the absence (often through the removal of visual information) or alteration (by manipulating the spatial and temporal properties of the cross-modal cues) of multisensory experience can influence normal multisensory function both before and after the critical period (Carriere et al., 2007; Odegaard, Wozny, & Shams, 2017; Stein, Stanford, & Rowland, 2014; Wallace et al., 2004). Although the deprivation effect observed in our study could be a result of compensation to the decrease in visually based multisensory information, we believe this is highly unlikely for two reasons. First, the monocular deprivation paradigm likely has very little impact on visually based multisensory experience, given the remaining information coming from the nondeprived eye. Second, the contrasting nature of the effects observed in the deprived and nondeprived eyes in our study are consistent with results that have been observed in purely visual studies of monocular deprivation (Lunghi et al., 2011; Lunghi et al., 2013; Zhou et al., 2013) and hence, points to the effects being driven in large measure by visual, as opposed to multisensory based compensatory mechanisms. 
Monocular deprivation represents a powerful tool to study plasticity in the adult visual system. Indeed, evidence from past studies points to more detrimental effects of monocular as opposed to binocular deprivation on visual cortical organization and behavioral performance (Lewis, Maurer, & Brent, 1995; Wiesel & Hubel, 1965a). Mechanistically, monocular deprivation triggers various forms of homeostatic plasticity in an effort to restore the loss of visual drive and maintain the average levels of neural activity (Maffei, Nelson, & Turrigiano, 2004; Maffei & Turrigiano, 2008; Wang, Fontanini, & Maffei, 2012). These forms of homeostatic plasticity induced by monocular deprivation involve alterations in the balance of cortical excitation and inhibition via a complex interplay of excitation, inhibition, and intrinsic excitability (Maffei et al., 2004; Maffei & Turrigiano, 2008; Wang et al., 2012). In adult humans, short-term monocular deprivation has been associated with the downregulation of gamma aminobutyric acid (GABA) concentration in the primary visual cortex (Lunghi, Emir, Morrone, & Bridge, 2015). Using electroencephalographic recordings, other studies have also demonstrated that short-term monocular deprivation increases the amplitude of visual evoked potentials and neural oscillations believed to originate from the primary visual cortex when the deprived eye is stimulated and decreases these measures when the nondeprived eye is stimulated (Lunghi et al., 2015; Zhou, Baker, Simard, Saint-Amour, & Hess, 2015). Although these results emphasize an increase in overall neural activity after monocular deprivation, a recent 7T functional magnetic resonance imaging study has shown that besides an increase in the blood oxygen level-dependent signal in V1 for the deprived eye, monocular deprivation also shifts ocular dominance distributions in favor of the deprived eye (Binda et al., 2018). Collectively, these findings indicate that monocular deprivation induces plastic mechanisms geared at enhancing the visual information coming from the deprived eye. 
Extending these findings to our current results, we believe that the changes observed in the TBW could arise from two main causes, which can occur separately or in tandem. Using the causal inference model, these two factors can be expressed in terms of changes in visual unisensory encoding and changes in the tendency to bind the audiovisual information (Beierholm, Quartz, & Shams, 2009; Körding et al., 2007; Magnotti, Ma, & Beauchamp, 2013). First, changes in the TBW as a result of changes in visual encoding after monocular deprivation is consistent with previous visual studies showing that monocular deprivation enhances the perception of the deprived eye while depressing that of the nondeprived eye (Lunghi et al., 2011; Zhou et al., 2013). In the multisensory domain, several studies have provided evidence that support the hypothesis that manipulating stimulus reliability, which occurs in the context of monocular deprivation, can influence multisensory interactions. For instance, Fister, Stevenson, Nidiffer, Barnett, and Wallace (2016) demonstrated that the proportion of perceived simultaneity of the paired audiovisual stimuli across SOA decreased faster for high than low saliency conditions. Although this study did not compute TBWs, the finding indicates that increasing stimulus effectiveness decrease the tolerance to audiovisual temporal asynchrony, which is comparable to narrowing of the TBW. Furthermore, Magnotti et al. (2013) demonstrated that decreasing the reliability (via blurring) of the visual stimulus during the performance of a simultaneity–judgment task widened the TBW. Studies using causal inference models have demonstrated that such effects emerge from changes in sensory noise, influencing the process of visual encoding (Beierholm et al., 2009; Magnotti et al., 2013). 
Second, it is plausible that the changes observed in the TBW after monocular deprivation could arise from genuine cross-modal plasticity, reflecting changes in the tendency to bind the audiovisual stimulus pair. In a very interesting study, Lo Verde, Morrone, and Lunghi (2017) discovered using a binocular rivalry paradigm that, after monocular deprivation, the effect of visuo-haptic interaction on perceptual dominance disappeared for the deprived eye, which was potentiated in the visual domain, but was not affected for the nondeprived eye, which was weakened. In a separate control experiment where a postdeprivation effect was simulated by increasing the intensity of the visual stimulus for one eye and decreasing the intensity for the other eye, this unequal effect of monocular deprivation on cross-modal interaction for the deprived and nondeprived eyes disappeared, indicating that the effects on monocular deprivation were due to mechanisms that exceeded changes in contrast gain of the deprived eye (Lo Verde et al., 2017). 
The idea that changes in the TBW can result from changes in binding tendency is consistent with studies that have demonstrated a relationship between TBW and intracortical excitation and inhibition balance—the underlying mechanism for homeostatic plasticity induced by monocular deprivation. For example, a recent study demonstrated that the size of the audio-tactile TBW measured across participants was associated with cortical excitation and inhibition balance indexed as glutamate/GABA concentrations in the left primary auditory cortex (Ferri et al., 2017), a region noted for integration of auditory and tactile information (Kayser, Petkov, Augath, & Logothetis, 2005; Schürmann, Caetano, Hlushchuk, Jousmäki, & Hari, 2006). Specifically, they showed that moderate excitation and inhibition balance was associated with the narrowest TBW sizes, whereas extreme (i.e., relatively high or low) excitation and inhibition balance was associated with wider TBW sizes (Ferri et al., 2017). Additionally, the link between changes in cortical excitation and inhibition balance mediating deprivation-induced homeostatic plasticity and the TBW can also be observed in clinical conditions or in perceptual mechanisms where excitation and inhibition balance is affected. For example, individuals with obesity, known to exhibit impaired homeostatic plasticity (suggestive of the presence of an altered excitation and inhibition balance) owing to a stronger GABAergic inhibition (Lunghi et al., 2019) also show widened TBW compared with healthy-weight controls (Scarpina et al., 2016). In addition, the TBW has been shown to be highly malleable to perceptual learning (Powers et al., 2009; Powers et al., 2012), which is known mechanistically to involve Hebbian plasticity, a form of experience-dependent plasticity that depends on the net intracortical excitation and inhibition balance like the homeostatic plasticity induced by monocular deprivation (see Binda et al., 2018, for a discussion). Together, these studies provide evidence that suggests that the TBW can be modulated to some extent by a direct influence of changes in cortical excitation and inhibition levels and or through changes in visual representations occurring as a result of changes in excitation and inhibition balance after monocular deprivation. To determine the relative contributions of these two mechanisms to the changes in TBW, further studies using neuroimaging techniques will be necessary to unravel the brain areas involved, the time course of these effects, and the changes in connectivity patterns induced by these effects. 
Apart from changes in the TBW, we found that depriving participants monocularly for 90 minutes did not reveal any significant changes in the PSS within and between the eye conditions. The PSS is a central tendency of the TBW and represents a measure of perceptual weighting between the two modalities whose simultaneity is being judged. Mechanistically, it is known to stem from a combination of 1) the difference between the arrival time latencies and 2) the difference between the processing time of the two modalities (García-Pérez & Alcalá-Quintana, 2012). The lack of deprivation effect on the PSS seen in our study can be explained by prior electroencephalographic work that showed that monocular deprivation did not affect the latencies of C1, P1, and P2 components of the visual evoked potentials (Lunghi, Berchicci, Morrone, & Di Russo, 2015). Although the change in PSS was nonsignificant, we did observe a strong positive correlation between the absolute change in PSS and the relative change in the TBW when the data were pooled across the two postdeprivation blocks and participants in both eye groups. This finding implies that the two measures, that is, PSS and TBW, were driven by similar mechanisms underlying monocular deprivation, albeit less strongly for the PSS than for the TBW. 
In conclusion, we have shown that a short perturbation in normal visual experience in adults through monocular deprivation results in significant, albeit transient, changes in multisensory temporal perception. These findings have implications on how the adult brain may react to visual impairment and how the clinical use of monocular deprivation as therapy may impact the restoration of balanced binocular function in adult patients with amblyopia. First, changes in visual experience in the adult can influence the way the visual system interacts with other sensory modalities, and thus such changes in multisensory perception may contribute to the overall compensatory behaviors observed in patients with adult-onset visual deprivation. Second, several studies have reported intact homeostatic plasticity among clinical populations such as amblyopia (Lunghi, Morrone, Secci, & Caputo, 2016; Zhou, Thompson, & Hess, 2013) and, consequently, have used monocular deprivation as therapy for improving visual perception in the amblyopic eye (Lunghi et al., 2019; Zhou et al., 2019). Our findings imply that, besides improving visual function in the amblyopic eye, monocular deprivation therapy may also ameliorate the impaired multisensory temporal processing observed in adult patients with amblyopia (Richards, Goltz, & Wong, 2017). 
Acknowledgments
The authors thank Brian Hou for his contribution during the data collection period. 
Commercial relationships: none. 
Corresponding author: Collins Opoku-Baah. 
Address: 465 21st Avenue South, MRB III Suite 7110, Nashville, TN 37240, USA. 
References
Beierholm, U. R., Quartz, S. R., & Shams, L. (2009). Bayesian priors are encoded independently from likelihoods in human multisensory perception. Journal of Vision, 9(5), 23–23.
Berardi, N., Pizzorusso, T., & Maffei, L. (2000). Critical periods during sensory development. Current Opinion in Neurobiology, 10(1), 138–145.
Binda, P., Kurzawski, J. W., Lunghi, C., Biagi, L., Tosetti, M., & Morrone, M. C. (2018). Response to short-term deprivation of the human adult visual cortex measured with 7T BOLD. Elife, 7, e40014.
Boenke, L. T., Deliano, M., & Ohl, F. W. (2009). Stimulus duration influences perceived simultaneity in audiovisual temporal-order judgment. Experimental Brain Research, 198(2–3), 233–244.
Brainard, D. H., & Vision, S. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436.
Carriere, B. N., Royal, D. W., Perrault, T. J., Morrison, S. P., Vaughan, J. W., Stein, B. E., & Wallace, M. T. (2007). Visual deprivation alters the development of cortical multisensory integration. Journal of Neurophysiology, 98(5), 2858–2867.
Chadnova, E., Reynaud, A., Clavagnier, S., & Hess, R. F. (2017). Short-term monocular occlusion produces changes in ocular dominance by a reciprocal modulation of interocular inhibition. Scientific Reports, 7, 41747.
Chen, Y.-C., Lewis, T. L., Shore, D. I., & Maurer, D. (2017). Early binocular input is critical for development of audiovisual but not visuotactile simultaneity perception. Current Biology, 27(4), 583–589.
Collignon, O., Dormal, G., De Heering, A., Lepore, F., Lewis, T. L., & Maurer, D. (2015). Long-lasting crossmodal cortical reorganization triggered by brief postnatal visual deprivation. Current Biology, 25(18), 2379–2383.
Diederich, A., & Colonius, H. (2004). Bimodal and trimodal multisensory enhancement: Effects of stimulus onset and intensity on reaction time. Perception & Psychophysics, 66(8), 1388–1404.
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429.
Ferri, F., Nikolova, Y. S., Perrucci, M. G., Costantini, M., Ferretti, A., Gatta, V., … Sibille, E. (2017). A neural “tuning curve” for multisensory experience and cognitive-perceptual schizotypy. Schizophrenia Bulletin, 43(4), 801–813.
Fine, I., Smallman, H. S., Doyle, P., & MacLeod, D. I. (2002). Visual function before and after the removal of bilateral congenital cataracts in adulthood. Vision Research, 42(2), 191–210.
Fine, I., Wade, A. R., Brewer, A. A., May, M. G., Goodman, D. F., Boynton, G. M., … MacLeod, D. I. (2003). Long-term deprivation affects visual perception and cortex. Nature Neuroscience, 6(9), 915.
Fister, J. K., Stevenson, R. A., Nidiffer, A. R., Barnett, Z. P., & Wallace, M. T. (2016). Stimulus intensity modulates multisensory temporal processing. Neuropsychologia, 88, 92–100.
Foss-Feig, J. H., Kwakye, L. D., Cascio, C. J., Burnette, C. P., Kadivar, H., Stone, W. L., & Wallace, M. T. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Experimental Brain Research, 203(2), 381–389.
Foxe, J. J., & Schroeder, C. E. (2005). The case for feedforward multisensory convergence during early cortical processing. Neuroreport, 16(5), 419–423.
Frassinetti, F., Bolognini, N., & Làdavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research, 147(3), 332–343.
García-Pérez, M. A., & Alcalá-Quintana, R. (2012). On the discrepant results in synchrony judgment and temporal-order judgment tasks: A quantitative model. Psychonomic Bulletin & Review, 19(5), 820–846.
Ghazanfar, A. A., & Schroeder, C. E. (2006). Is neocortex essentially multisensory? Trends in Cognitive Sciences, 10(6), 278–285.
Goldstone, R. L. (1998). Perceptual learning. Annual Review of Psychology, 49(1), 585–612.
Guerreiro, M. J., Putzar, L., & Röder, B. (2015). The effect of early visual deprivation on the neural bases of multisensory processing. Brain, 138(6), 1499–1504.
Hubel, D. H., & Wiesel, T. N. (1970). The period of susceptibility to the physiological effects of unilateral eye closure in kittens. Journal of Physiology, 206(2), 419–436.
Huxlin, K. R. (2008). Perceptual plasticity in damaged adult visual systems. Vision Research, 48(20), 2154–2166.
Jaśkowski, P. (1999). Reaction time and temporal-order judgment as measures of perceptual latency: The problem of dissociations. Advances in Psychology, 129, 265–282.
JASP Team. (2018). JASP [Computer software, Version 0.11. 0.1], https://jasp-stats.org/faq/how-do-i-cite-jasp/.
Kayser, C., Petkov, C. I., Augath, M., & Logothetis, N. K. (2005). Integration of touch and sound in auditory cortex. Neuron, 48(2), 373–384.
Kim, H.-W., Kim, C.-Y., & Blake, R. (2017). Monocular perceptual deprivation from interocular suppression temporarily imbalances ocular dominance. Current Biology, 27(6), 884–889.
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007). Causal inference in multisensory perception. PLoS One, 2(9), 943.
Levi, D. M., McKee, S. P., & Movshon, J. A. (2011). Visual deficits in anisometropia. Vision Research, 51(1), 48–57.
Lewis, T. L., Maurer, D., & Brent, H. P. (1995). Development of grating acuity in children treated for unilateral or bilateral congenital cataract. Investigative Ophthalmology & Visual Science, 36(10), 2080–2095.
Lovelace, C. T., Stein, B. E., & Wallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: A psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research, 17(2), 447–453.
Lo Verde, L., Morrone, M. C., & Lunghi, C. (2017). Early cross-modal plasticity in adults. Journal of Cognitive Neuroscience, 29(3), 520–529.
Lunghi, C., Berchicci, M., Morrone, M. C., & Russo, Di (2015). Short-term monocular deprivation alters early components of visual evoked potentials. Journal of Physiology, 593(19), 4361–4372.
Lunghi, C., Burr, D. C., & Morrone, C. (2011). Brief periods of monocular deprivation disrupt ocular balance in human adult visual cortex. Current Biology, 21(14), R538–R539.
Lunghi, C., Burr, D. C., & Morrone, M. C. (2013). Long-term effects of monocular deprivation revealed with binocular rivalry gratings modulated in luminance and in color. Journal of Vision, 13 (6), 1–1.
Lunghi, C., Daniele, G., Binda, P., Dardano, A., Ceccarini, G., Santini, F., … Morrone, M. C. (2019). Altered visual plasticity in morbidly obese subjects. iScience, 22, 206–213.
Lunghi, C., Emir, U. E., Morrone, M. C., & Bridge, H. (2015). Short-term monocular deprivation alters GABA in the adult human visual cortex. Current Biology, 25(11), 1496–1501.
Lunghi, C., Morrone, M. C., Secci, J., & Caputo, R. (2016). Binocular rivalry measured 2 hours after occlusion therapy predicts the recovery rate of the amblyopic eye in anisometropic children. Investigative Ophthalmology & Visual Science, 57(4), 1537–1546.
Lunghi, C., Sframeli, A. T., Lepri, A., Lepri, M., Lisi, D., Sale, A., & Morrone, M. C. (2019). A new counterintuitive training for adult amblyopia. Annals of Clinical and Translational Neurology, 6(2), 274–284.
Maffei, A., Nelson, S. B., & Turrigiano, G. G. (2004). Selective reconfiguration of layer 4 visual cortical circuitry by visual deprivation. Nature Neuroscience, 7(12), 1353–1359.
Maffei, A., & Turrigiano, G. G. (2008). Multiple modes of network homeostasis in visual cortical layer 2/3. Journal of Neuroscience, 28(17), 4377–4384.
Magnotti, J. F., Ma, W. J., & Beauchamp, M. S. (2013). Causal inference of asynchronous audiovisual speech. Frontiers in Psychology, 4, 798.
Meredith, M. A., Nemitz, J. W., & Stein, B. E. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. Journal of Neuroscience, 7(10), 3215–3229.
Noel, J. P., Stevenson, R. A., & Wallace, M. T. (2018). Atypical audiovisual temporal function in autism and schizophrenia: Similar phenotype, different cause. European Journal of Neuroscience, 47(10), 1230–1241.
Odegaard, B., Wozny, D. R., & Shams, L. (2017). A simple and efficient method to enhance audiovisual binding tendencies. PeerJ, 5, e3143.
Ostrovsky, Y., Andalman, A., & Sinha, P. (2006). Vision following extended congenital blindness. Psychological Science, 17(12), 1009–1014.
Pascual-Leone, A., Amedi, A., Fregni, F., & Merabet, L. B. (2005). The plastic human brain cortex. Annual Review of Neuroscience, 28, 377–401. [PubMed]
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10(4), 437–442.
Powers, A. R., Hevey, M. A., & Wallace, M. T. (2012). Neural correlates of multisensory perceptual learning. Journal of Neuroscience, 32(18), 6263–6274.
Powers, A. R., Hillock, A. R., & Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. Journal of Neuroscience, 29(39), 12265–12274.
Putzar, L., Goerendt, I., Lange, K., Rösler, F., & Röder, B. (2007). Early visual deprivation impairs multisensory interactions in humans. Nature Neuroscience, 10(10), 1243.
Putzar, L., Hötting, K., & Röder, B. (2010). Early visual deprivation affects the development of face recognition and of audio-visual speech perception. Restorative Neurology and Neuroscience, 28(2), 251–257.
Richards, M. D., Goltz, H. C., & Wong, A. M. (2017). Alterations in audiovisual simultaneity perception in amblyopia. PloS One, 12(6), e0179516.
Sabel, B. A. (2008). Plasticity and restoration of vision after visual system damage: An update. Restorative Neurology and Neuroscience, 26(4, 5), 243–247.
Sanford, A. (1971). Effects of changes in the intensity of white noise on simultaneity judgements and simple reaction time. Quarterly Journal of Experimental Psychology, 23(3), 296–303.
Scarpina, F., Migliorati, D., Marzullo, P., Mauro, A., Scacchi, M., & Costantini, M. (2016). Altered multisensory temporal integration in obesity. Scientific Reports, 6, 28382.
Schneider, K. A., & Bavelier, D. (2003). Components of visual prior entry. Cognitive Psychology, 47(4), 333–366.
Schürmann, M., Caetano, G., Hlushchuk, Y., Jousmäki, V., & Hari, R. (2006). Touch activates human auditory cortex. Neuroimage, 30(4), 1325–1331.
Simon, D. M., Noel, J. P., & Wallace, M. T. (2017). Event related potentials index rapid recalibration to audiovisual temporal asynchrony. Frontiers in Integrative Neuroscience, 11, 8.
Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: The MIT Press.
Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255.
Stein, B. E., Stanford, T. R., & Rowland, B. A. (2014). Development of multisensory integration from the perspective of the individual neuron. Nature Reviews Neuroscience, 15(8), 520–535.
Stelmach, L. B., & Herdman, C. M. (1991). Directed attention and perception of temporal order. Journal of Experimental Psychology: Human Perception and Performance, 17(2), 539.
Stevenson, R. A., Park, S., Cochran, C., McIntosh, L. G., Noel, J. P., Barense, M. D., & Wallace, M. T. (2017). The associations between multisensory temporal processing and symptoms of schizophrenia. Schizophrenia Research, 179, 97–103.
Stevenson, R. A., Siemann, J. K., Schneider, B. C., Eberly, H. E., Woynaroski, T. G., Camarata, S. M., & Wallace, M. T. (2014). Multisensory temporal integration in autism spectrum disorders. Journal of Neuroscience, 34(3), 691–697.
Stevenson, R. A., Zemtsov, R. K., & Wallace, M. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. Journal of Experimental Psychology: Human Perception and Performance, 38(6), 1517.
Wallace, M. T., Perrault, T. J., Hairston, W. D., & Stein, B. E. (2004). Visual experience is necessary for the development of multisensory integration. Journal of Neuroscience, 24(43), 9580–9584.
Wallace, M. T., & Stevenson, R. A. (2014). The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia, 64, 105–123.
Wang, L., Fontanini, A., & Maffei, A. (2012). Experience-dependent switch in sign and mechanisms for plasticity in layer 4 of primary visual cortex. Journal of Neuroscience, 32(31), 10562–10573.
Wen, P., Opoku-Baah, C., Park, M., & Blake, R. (2020). Judging Relative Onsets and Offsets of Audiovisual Events. Vision, 4(1), 17.
Wiesel, T. N., & Hubel, D. H. (1963). Effects of visual deprivation on morphology and physiology of cells in the cat's lateral geniculate body. Journal of Neurophysiology, 26(6), 978–993.
Wiesel, T. N., & Hubel, D. H. (1965a). Comparison of the effects of unilateral and bilateral eye closure on cortical unit responses in kittens. Journal of Neurophysiology, 28(6), 1029–1040.
Wiesel, T. N., & Hubel, D. H. (1965b). Extent of recovery from the effects of visual deprivation in kittens. Journal of Neurophysiology, 28(6), 1060–1072.
Zampini, M., Shore, D. I., & Spence, C. (2005). Audiovisual prior entry. Neuroscience letters, 381(3), 217–222.
Zhou, J., Baker, D. H., Simard, M., Saint-Amour, D., & Hess, R. F. (2015). Short-term monocular patching boosts the patched eye's response in visual cortex. Restorative Neurology and Neuroscience, 33(3), 381–387.
Zhou, J., Clavagnier, S., & Hess, R. F. (2013). Short-term monocular deprivation strengthens the patched eye's contribution to binocular combination. Journal of Vision, 13(5), 12–12.
Zhou, J., He, Z., Wu, Y., Chen, Y., Chen, X., Liang, Y., … Hess, R. F. (2019). Inverse occlusion: A binocularly motivated treatment for amblyopia. Neural Plasticity, 2019, 5157628.
Zhou, J., Thompson, B., & Hess, R. F. (2013). A new form of rapid binocular plasticity in adult with amblyopia. Scientific Reports, 3, 2638.
Zou, H., Müller, H. J., & Shi, Z. (2012). Non-spatial sounds regulate eye movements and enhance visual search. Journal of Vision, 12(5), 2–2.
Figure 1.
 
Experimental procedure. (A) Participants were randomly assigned to either the deprived eye or nondeprived eye group. Participants in the deprived eye group performed the audiovisual temporal simultaneity judgment task with their deprived eye before and after 90 minutes of deprivation. For the nondeprived eye group, the deprivation phase started with the first predeprivation block of the nondeprived eye. The postdeprivation blocks were then performed using the nondeprived eye after 90 minutes from the start of the first predeprivation block. (B) On each block, participants judged the simultaneity of a gabor patch (33 ms) and an auditory white noise burst (33 ms). The onsets of the two stimuli were separated by a range of preselected SOAs between –500 ms and 500 ms. Negative and positive SOAs signified auditory leading and visual leading respectively. On each trial, there was a brief fixation period (600–1000 ms), followed by the stimulus presentation. Participants were then asked to respond by pressing the keyboard after which the next trial began automatically.
Figure 1.
 
Experimental procedure. (A) Participants were randomly assigned to either the deprived eye or nondeprived eye group. Participants in the deprived eye group performed the audiovisual temporal simultaneity judgment task with their deprived eye before and after 90 minutes of deprivation. For the nondeprived eye group, the deprivation phase started with the first predeprivation block of the nondeprived eye. The postdeprivation blocks were then performed using the nondeprived eye after 90 minutes from the start of the first predeprivation block. (B) On each block, participants judged the simultaneity of a gabor patch (33 ms) and an auditory white noise burst (33 ms). The onsets of the two stimuli were separated by a range of preselected SOAs between –500 ms and 500 ms. Negative and positive SOAs signified auditory leading and visual leading respectively. On each trial, there was a brief fixation period (600–1000 ms), followed by the stimulus presentation. Participants were then asked to respond by pressing the keyboard after which the next trial began automatically.
Figure 2.
 
Mean proportion of synchrony reports. Proportion of synchrony reports averaged across participants in the deprived eye group (red) and the nondeprived eye group (blue) is plotted as a function of SOA for (A) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line) and for (B) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line).
Figure 2.
 
Mean proportion of synchrony reports. Proportion of synchrony reports averaged across participants in the deprived eye group (red) and the nondeprived eye group (blue) is plotted as a function of SOA for (A) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line) and for (B) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line).
Figure 3.
 
Effects of deprivation on the PSS. (A) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Absolute change was computed as postdeprivation – predeprivation. (B) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Figure 3.
 
Effects of deprivation on the PSS. (A) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Absolute change was computed as postdeprivation – predeprivation. (B) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Figure 4.
 
Effects of deprivation on the TBW. (A) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Relative change was computed as (postdeprivation – predeprivation)/predeprivation × 100%. (B) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Figure 4.
 
Effects of deprivation on the TBW. (A) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Relative change was computed as (postdeprivation – predeprivation)/predeprivation × 100%. (B) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.
Figure 5.
 
Correlation between the absolute change in PSS and the relative change in temporal window of simultaneity (TWS). Plotted on the x-axis the is relative change in TWS data pooled from participants across both eye groups (deprived eye: red; nondeprived eye group: blue) and the two postdeprivation blocks (first: circle; second: diamond) and plotted on the y-axis is the corresponding absolute change in PSS data. Positive and negative data points on the x-axis signify enhancement (narrowing of TWS) and depreciation (widening of TWS) of sensitivity to audiovisual temporal asynchrony, whereas positive and negative values on the y-axis signify more (negative shift in PSS) and less (positive shift in PSS) dominance (dom.) of the visual modality respectively. Our results revealed a strong positive correlation between relative change in TWS and the absolute change in PSS, r = 0.7, p < 0.001.
Figure 5.
 
Correlation between the absolute change in PSS and the relative change in temporal window of simultaneity (TWS). Plotted on the x-axis the is relative change in TWS data pooled from participants across both eye groups (deprived eye: red; nondeprived eye group: blue) and the two postdeprivation blocks (first: circle; second: diamond) and plotted on the y-axis is the corresponding absolute change in PSS data. Positive and negative data points on the x-axis signify enhancement (narrowing of TWS) and depreciation (widening of TWS) of sensitivity to audiovisual temporal asynchrony, whereas positive and negative values on the y-axis signify more (negative shift in PSS) and less (positive shift in PSS) dominance (dom.) of the visual modality respectively. Our results revealed a strong positive correlation between relative change in TWS and the absolute change in PSS, r = 0.7, p < 0.001.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×