A neural hierarchy for illusions of time: Duration adaptation precedes multisensory integration

Abstract

Abstract

Abstract:

Abstract  Perceived time is inherently malleable. For example, adaptation to relatively long or short sensory events leads to a repulsive aftereffect such that subsequent events appear to be contracted or expanded (duration adaptation). Perceived visual duration can also be distorted via concurrent presentation of discrepant auditory durations (multisensory integration). The neural loci of both distortions remain unknown. In the current study we use a psychophysical approach to establish their relative positioning within the sensory processing hierarchy. We show that audiovisual integration induces marked distortions of perceived visual duration. We proceed to use these distorted durations as visual adapting stimuli yet find subsequent visual duration aftereffects to be consistent with physical rather than perceived visual duration. Conversely, the concurrent presentation of adapted auditory durations with nonadapted visual durations results in multisensory integration patterns consistent with perceived, rather than physical, auditory duration. These results demonstrate that recent sensory history modifies human duration perception prior to the combination of temporal information across sensory modalities and provides support for adaptation mechanisms mediated by duration selective neurons situated in early areas of the visual and auditory nervous system (Aubie, Sayegh, & Faure, 2012; Duysens, Schaafsma, & Orban, 1996; Leary, Edwards, & Rose, 2008).

Introduction

Despite the ecological importance of accurate time perception, a broadly reproduced finding is the remarkable extent to which perceived durations are modified by the context in which they are presented. Two notable examples of these misperceptions include duration distortions induced via multisensory integration (MSI) and those induced via duration adaptation (DA). The former occurs when perceived visual duration is pulled in the direction of concurrently presented—but physically discrepant—auditory durations (Chen & Yeh, 2009; Klink, Montijn, & van Wezel, 2011). The latter arises when recent sensory history contains consistent duration information. Postadaptation, perception is characterized by repulsive duration aftereffects that are sensory specific, bidirectional, and tuned around the adapting duration (Heron, Aaen-Stockdale, et al., 2012; Walker, Irion, & Gordon, 1981). 

An unresolved issue is the relative positioning of these perceptual distortions within the processing hierarchy. In principle, DA could be mediated via neurons known to respond selectively to a narrow range of stimulus durations centered on their preferred duration. These neurons are located in relatively early areas of the visual and auditory nervous systems including the primary visual cortex (Duysens, Schaafsma, & Orban, 1996) and auditory midbrain (Brand, Urban, & Grothe, 2000; Casseday, Ehrlich, & Covey, 1994; Faure, Fremouw, Casseday, & Covey, 2003; Leary, Edwards, & Rose, 2008; Perez-Gonzalez, Malmierca, Moore, Hernandez, & Covey, 2006). Alternatively, DA could have a much later neural locus associated with duration-specific firing patterns observed in higher cortical areas including prefrontal/frontal (Genovesio, Tsujimoto, & Wise, 2006, 2009; Shinomoto et al., 2011), motor/premotor (Lebedev, O'Doherty, & Nicolelis, 2008; Merchant, Zarco, Perez, Prado, & Bartolo, 2011; Mita, Mushiake, Shima, Matsuzaka, & Tanji, 2009) or lateral intraparietal regions (Leon & Shadlen, 2003). 

Although the neural locus of MSI-based duration illusions remains unknown, the integration of visual-vestibular directional cues and audio-visual speech components are increasingly being ascribed to higher, extrastriate areas such as dorsal medial superior temporal area (Fetsch, Pouget, DeAngelis, & Angelaki, 2012; Gu, Angelaki, & DeAngelis, 2008) and the superior temporal sulcus (Nath & Beauchamp, 2011), respectively. Nevertheless, a recent study also suggests a role for the primary sensory cortices in multisensory integration (Helbig et al., 2012). 

If the two illusions are indeed generated by neural mechanisms with distinct positions in the processing hierarchy, the interaction between them should be unidirectional. For example, if MSI occurs at a relatively early stage of processing, any distortions of perceived duration it generates will feed forward to influence subsequent adaptation mechanisms. Inducing MSI-based distortions during a period of adaptation should thus produce duration aftereffects commensurate with the perceived rather than physical duration of the adapting stimulus. In this scenario, it also follows that patterns of MSI should be unaffected by prior adaptation, remaining consistent with the physical duration of each stimulus. In the current study, we present psychophysical data inconsistent with this hypothesis. Rather, our findings reveal duration adaptation to be a process completed by the nervous system prior to the integration of temporal information across the senses. 

Materials and methods

Participants

Seven observers (three authors, four naïve) participated in these experiments (median age 28 years). All participants had reported an absence of visual, auditory, or somatosensory disorders. All participants gave written informed consent in accordance with the Declaration of Helsinki (2008). Experiments were approved by the Ethics committee of the University of Bradford. 

Stimuli and apparatus

The visual stimulus was a Gaussian luminance blob with a peak luminance of 94 cd/m², presented on a uniform 47 cd/m² gray background. The size of the Gaussian blob was defined by its standard deviation (σ) which measured 2.1° at a viewing distance of 57 cm. The blob was presented in the center of a gamma-corrected monitor screen. Visual stimuli were presented in the center of either a Compaq P1220 CRT display (resolution of 1280 × 1024, refresh rate 100 Hz), or a Sony Trinitron GDM FW 900 CRT display (resolution = 1920 × 1200, refresh rate = 75 Hz) CRT display. Both displays were driven by a dual-quad-core Apple Mac Pro desktop computer running Mac OS 10.4. Auditory and visual stimuli were generated using MATLAB 7 (Mathworks, Natick, MA) and Psychophysics Toolbox 3 (http://www.psychtoolbox.org). The auditory stimulus consisted of a burst of white noise of approximately 75 dB presented via Sennheiser HD 280 headphones. The delivery of visual and auditory stimuli and the collection of subject's responses were controlled from within MATLAB using custom software. The physical duration of visual and auditory stimuli were given uniform onset-offset profiles. All timings were verified via simultaneous capture on a dual-channel oscilloscope. 

Multisensory integration experiment

Subjects were asked to make a two interval, forced choice duration discrimination judgment between a 320-ms test stimulus and a variable reference stimulus presented to the same modality. The duration of the reference stimulus varied in seven steps centered on 320 ms, according to the method of constant stimuli. The test stimulus was accompanied by a concurrently presented 200 ms or 510 ms distracter stimulus from the opposite modality (e.g., Figures 1A and B). These durations were chosen to provide ±0.2 log units of physical duration discrepancy with the 320 ms test stimulus. The temporal midpoints of the visual and auditory components of the test stimulus were coincident. Within a block of trials, the sensory modality of test, reference, and distracter stimuli was held constant and the presentation order of all (bimodal) test and (unimodal) reference durations was randomly interleaved. The presentation order of test and reference stimuli within each trial was also randomized. This had the advantage of removing any putative time order error effect(s) from our reported point of subjective equality (PSE) values but may have slightly elevated duration discrimination threshold values in the process (Hellstrom, 1985). During the presentation of bimodal test stimuli subjects were asked to ignore the distracter duration whilst making a unimodal judgment as to, “Which was longer, the first or second visual/auditory duration?” Subjects responded by keyboard which triggered the next trial. 

Figure 1

 

(A) and (B) A schematic of the multisensory integration experiment where subjects were presented with a test stimulus consisting of 320 ms duration from one modality (either vision [blue rectangles] or audition [red rectangles]) and a concurrently presented distracter duration from the opposite modality. Following the presentation of the reference stimulus (200–440 ms) subjects made unimodal duration discrimination judgments (“Which had the longer duration, test or reference stimulus?”) whilst attempting to ignore the distracter duration. In the example provided above the distracter stimulus has a physically longer duration than the test stimulus but were in fact equally likely to be longer or shorter (see Figure 3 for resultant data) (C) Schematic of a control experiment where observers adapted to concurrently presented auditory and visual stimuli with physically matched durations before reproducing the duration of (i) auditory, (ii) bimodal (again, of matched duration), or (iii) visual test stimuli (see Figure 5B for resultant data and Methods for details).

A block of trials comprised 10 repetitions of the seven reference durations and two distracter durations for a given modality combination (i.e., Figures 1A and 1B, respectively). Each subject completed a minimum of three blocks per modality combination. 

Adaptation experiments

In separate blocks, observers adapted to bimodal stimuli where a 320 ms visual duration was coupled with an auditory distracter stimulus of 200 ms or 510 ms (Figure 2A). During the adaptation phase, bimodal adapting stimuli were presented 100 times, followed by a further four top-up adaptation stimuli. All adapting and top-up stimuli were separated by an interstimulus interval that was jittered between 500 and 1000 ms. In the test phase subjects were presented with one of three visual durations, 50% of which were 320 ms in duration and the other 50% (catch trials) were divided equally between 160 ms and 640 ms durations (all randomly interleaved). To evaluate perceived duration, we used a method of reproduction in which subjects were asked to depress a computer key for a duration corresponding to that of the test stimulus. The release of the computer key triggered the next top-up/test cycle. The use of a method of reproduction allowed for an absolute unimodal estimate of duration without requiring any crossmodal comparison which may itself have been distorted by adaptation to the distracter duration. The use of 160 ms and 640 ms catch trials ensured that subjects were unable to adopt a repetitive motor response associated with repeated reproductions of identical durations. 

Figure 2

 

(A) Schematic showing an adaptation experiment where visual durations (blue rectangles) were assigned physically matched durations in both adaptation and test phases. In the adaptation phase, concurrently presented auditory durations (red rectangles) were assigned physically discrepant durations which allowed multisensory integration mechanisms to induce perceptual differences between visual adapting and test durations (see Figure 4B). Following adaptation, subjects reproduced the perceived duration of the visual test stimulus. (B) A subsequent adaptation experiment where a unimodal auditory adaptation phase was followed by a two interval visual duration discrimination judgment similar to that depicted in Figure 1A (“Which was longer, the visual test or reference stimulus?”) with the exception that the auditory and visual components of the test stimulus were assigned the same (320 ms) duration. In this situation, duration adaptation mechanisms were responsible for inducing perceptual differences between the (physically matched) auditory and visual components of the test stimuli (see Figures 6C, D). (C) A subexperiment where the perceived duration of the auditory distracter stimulus in (B)'s test phase was measured via comparing its duration to a variable visual test stimulus. In (A), (B), and (C) the four top-up stimuli depicted (see Adaptation phase above) were preceded a train of 100 identical stimuli and the subjects response triggered the next top-up/test cycle.

Figure 3

 

Data from the multisensory integration experiment depicted in Figures 1A and B. Bar heights points represent PSE (the point of subjective equality—the physical reference duration that induced perceptual equivalence with the 320 ms test duration from within the same modality). Black and white bars represent conditions where test and reference durations were visual and distracter durations were 200 ms (white bar) or 510 ms (black bar) auditory stimuli, with the reverse applying to dark gray and light gray bars. PSE values have been averaged across observers (n = 7) and error bars represent the standard error of the mean.

Figure 4

 

(A) Visual adapting stimuli (in blue) are presented concurrently with longer or shorter duration auditory distracter stimuli (red). (B) Multisensory integration (MSI) expands or contracts the perceived duration of the visual stimulus (C) and (D). If later-stage duration adaptation (DA) mechanisms receive distorted visual information from earlier-stage MSI mechanisms, these distortions will induce perceived differences between—physically identical—visual and adapting and test durations. These differences predict repulsive duration aftereffects which would be manifest as perceptual compression (left hand panel) or expansion (right hand panel) of visual test duration. (E) Alternatively, if DA mechanisms operate at a pre-MSI processing stage, they will receive veridical visual duration information during adaptation. In this scenario, an absence of adapt-test duration differences would predict an absence of duration aftereffects: The perceived duration of the test stimulus will be consistent with the physical duration of the adapting stimulus.

Figure 5

 

(A) Mean aftereffect magnitude (the arithmetic difference between post-adaptation reproduced durations—see Methods for details) following adaptation to perceptual differences between adapting and test durations (black bar). These perceptual differences were introduced via the MSI associated with stimulus configurations shown in Figures 4A and B. These effects were compared with equivalent physical adapt-test differences (gray bar). (B) Data from a control experiment where mean aftereffect magnitude was compared for the three adapt-test conditions shown in Figure 1C. Observers reproduced visual, auditory, and bimodal test stimuli following adaptation to the concurrent presentation of bimodal stimuli consisting of identical (160 ms or 640 ms) visual and auditory durations. Errors bars indicate the standard error of the mean (n = 7).

Figure 6

 

(A) Observers adapt to relatively long (left hand panels) or short (right hand panels) duration auditory stimuli before (B) estimating the visual component of an intermediate duration bimodal test stimulus (see Figure 2B). (C) and (D) Adaptation induces perceptual compression of the test stimulus' auditory component. (C) If the nervous system completes multisensory integration (MSI) before the implementation of duration adaptation (DA) mechanisms, the visual test duration will be integrated with its pre-adaptation (i.e., veridical) auditory counterpart. In this situation, the outcome of MSI will be consistent with physical auditory duration. (D) Conversely, if later-stage MSI mechanisms receive distorted duration information from earlier-stage DA mechanisms, the auditory distortion will be transferred to its visual counterpart. In this situation, the outcome of MSI will be consistent with perceived auditory duration (see Figure 7 for resultant data).

Figure 7

 

(A) Visual duration discrimination judgments (made by a naïve representative observer AR) between a bimodal test stimulus (concurrently presented 320 ms visual and auditory durations) and a variable visual reference stimulus (see test phase in Figure 2B). These judgments were made following adaptation to 640 ms (green circles) or 160 ms (orange squares) auditory durations. The combined effects of adaptation and sensory integration are signified by the shift in PSE: the lateral separation between the function's midpoints (i.e., the horizontal distance between vertical dashed gray lines) (B) Gray bar represents the mean PSE shift (n = 7). The white bar represents PSE shifts induced by pairing visual stimuli with auditory distracter durations that were the physical equivalent of the (adaptation induced) perceptual distracter durations deployed in Figure 7A (see main text for details). Error bars indicate the standard error of the mean.

A block of trials was completed when the 320 ms test duration had been presented 30 times. Each subject completed a minimum of three blocks per adapting stimulus condition giving a total of 90 reproductions per condition. All conditions were presented in a randomized order. Before beginning the experiment, subjects completed a practice session where they were given immediate feedback on the direction and magnitude of any reproduction errors; however no feedback was given during adaptation conditions. 

In a follow-up experiment we employed the same paradigm with the following exceptions. The adaptation phase consisted of 160 ms or 640 ms unimodal auditory durations (presented in separate blocks—see Figure 2B). The test phase was identical to the multisensory integration experiment described above (Figure 1A) with exception that the bimodal test stimulus always consisted of concurrently presented 320 ms auditory and visual durations. As above, subjects were instructed to ignore auditory stimuli and whilst making a two-interval visual duration discrimination judgment. Within each experiment, all blocks were randomized so as to ensure any training effects were spread across as many conditions as possible. 

Results

Multisensory integration

For each observer, the percentage of reference longer than test (Figures 1A, B) responses for each condition was plotted as a function of reference duration and fitted with a logistic function of the form  

 

where μ is the reference duration corresponding to the 50% response level on the psychometric function. We refer to this as the point of subjective equality (PSE), which represents the physical reference duration that is perceptually equivalent to the 320 ms test duration. θ provides an estimate of duration discrimination threshold (approximately half the offset between the 27% and 73% response levels). In this way, PSE values were obtained for all observers in all conditions. Figure 3 plots these PSE values (averaged across observers) as a function of distracter duration and modality. Black and white bars denote the PSE when test and reference stimuli are visual, and the distracter stimulus is auditory (e.g., Figure 1A) whilst gray bars denote the converse situation (e.g., Figure 1B). When the auditory distracter stimulus is shorter than its 320 ms visual counterpart (Figure 3, white bar), the visual reference duration must be compressed to 277 ms in order to maintain perceptual equivalence with the 320 ms visual test stimulus. This compression is mirrored by marked perceptual expansion of the test stimulus (375 ms) when the auditory distracter stimulus is longer than 320 ms. A mixed model analysis of variance (ANOVA) was used to test for any significant difference between the two conditions, and to make a precautionary check that the pattern of results was the same for naïve and nonnaïve participants. For the auditory distracter, the difference in PSE for the two distracter durations was highly significant, F(1, 5) = 14.1, p < 0.05, and there was no interaction between effect and participant type, F(1, 5) = 0.28, p > 0.05, showing that the adaptation effect is common across observers. In the case of the visual distractor, there was no significant effect of duration, F(1, 5) = 0.662, p > 0.05, and again no interaction effect, F(1, 5) = 0.529, p > 0.05. 

In addition to changes in perceived visual duration, we also found changes in our observer's visual duration discrimination sensitivity. Specifically, the 200 ms and 510 ms auditory distracter conditions produced mean thresholds of 47 ms and 67 ms, respectively. The fact that thresholds were higher when perceived duration was expanded was confirmed with a one-tailed t test, t(6) = 2.28, p = 0.031. One of the fundamental properties of duration perception is the proportional relationship between duration discrimination thresholds and mean estimated duration, known as Weber's law for duration (for reviews see Gibbon, 1977; Lewis & Miall, 2009). Dividing our individual observer's visual thresholds by their corresponding PSEs gave Weber fractions which, when averaged across observers, gave mean Weber fractions of 0.17 and 0.18 for the 200 ms and 510 ms distracter conditions, respectively. The similarity of these values suggests that perceptual compression (Figure 3, white bar) and expansion (Figure 3, black bar) of visual duration may be paralleled by a proportional reduction and elevation in duration discrimination sensitivity. Thus, the perceptual consequences of MSI-based duration distortions appear analogous to equivalent changes in physical stimulus duration. 

In line with recent reports (Chen & Yeh, 2009; Klink et al., 2011), the data of Figure 3 show that sensory integration produces marked distortions of visual duration. We proceed to examine the relationship between sensory integration-based mechanisms and those underpinning duration adaptation-induced illusions. Specifically, what are their relative positions within the nervous system's sensory processing hierarchy? If sensory integration precedes duration adaptation, employing the former to induce biases in an adapting stimulus (Figure 4B) should result in aftereffects that are consistent with its perceived rather than physical adapting duration (Figure 4D). Conversely, if the mechanisms underpinning duration adaptation have their neural locus at an earlier processing stage, duration aftereffects will remain consistent with the physical duration of the adapting stimulus (Figure 4E). 

Adaptation versus multisensory integration

Observers adapted to the concurrent presentation of 320 ms visual durations alongside auditory distracter durations of 200 ms or 510 ms (see Figure 2A and Methods for details) which provided robust perceptual differences between (physically identical) visual adapting and test stimuli (e.g., Figure 4B). Duration aftereffects are only manifest when adapting and test stimuli have different durations (Heron, Aaen-Stockdale, et al., 2012). Therefore, the early MSI, late DA scenario predicts that the perceptually compressed and expanded adapting stimuli will produce aftereffects in opposing directions (Figure 4D). However, the ‘early DA, late MSI’ scenario predicts equivalence of pre- and postadaptation visual duration perception (Figure 4E). These ‘perceptually different’ conditions were compared to duration-matched ‘physically different’ conditions where observers reproduced a 320 ms test stimulus following adaptation to unimodal visual durations of 277 ms and 375 ms. These values were selected to match the visual distortion generated in our multisensory integration experiment (see height of Figure 3's white and black bars, respectively). 

For all adapting conditions, aftereffect magnitude was calculated by subtracting the reproduced test duration following adaptation to the longer stimulus from that following adaptation to the shorter stimulus. Any repulsive duration aftereffects would therefore be manifest as a positive value. Figure 5A plots aftereffect magnitude following adaptation to perceptual (black bar) and physical (gray bar) differences between adapting and test stimuli. Clearly, the two conditions induce very different aftereffect magnitudes, t(6) = 8.09, p < 0.05. Specifically, physical differences between adapting and test durations generate robust duration aftereffects, t(6) = 9.858, p < 0.0001, comparable with those reported elsewhere in the literature (Heron, Aaen-Stockdale, et al., 2012). However, consistent with the scenario depicted in Figure 4E, perceptual differences fail to generate any significant aftereffects, t(6) = −0.778, p > 0.1, a result which suggests that duration adaptation mechanisms are unaffected by the duration distortions generated via multisensory integration. 

Alternatively, the absence of effect for the illusory stimuli may arise from the bimodal nature of the adapting stimulus and the unimodal nature of the test stimulus (Figure 2A). For example, the auditory stimulus may divert attention from its visual counterpart during adaptation—a scenario known to ameliorate other aftereffects (Chaudhuri, 1990; Heron, Roach, Whitaker, & Hanson, 2010; Montaser-Kouhsari & Rajimehr, 2004). Equally, a greater degree of categorical similarity between adapting and test stimuli (e.g., adapt bimodal, test bimodal) may be necessary to activate duration adaptation mechanisms (e.g., Bestelmeyer et al., 2008; Little, DeBruine, & Jones, 2011; Rotshtein, Henson, Treves, Driver, & Dolan, 2005). These possibilities were investigated by conducting a control experiment identical to the previous experiment with the following exceptions: Auditory and visual components of the bimodal adapting stimuli had physically identical durations, either 640 or 160 ms (durations known to produce reliable duration aftereffects in both visual and auditory domains, Heron, Aaen-Stockdale et al., 2012), presented in separate blocks. Test stimuli were either 320 ms auditory, bimodal (again, of matched physical duration), or visual durations (see Figures 1C(i), (ii), and (iii), respectively). If attending to audition explains the lack of aftereffect seen in Figure 5A (black bar), aftereffects should only be manifest when test stimuli contain auditory durations (i.e., Figures 1C(i) and (ii) only). If effective adaptation requires categorical similarity between adapting and test stimuli configurations we should only see aftereffects in the adapt bimodal, test bimodal condition (Figure 1C[ii]). Figure 5B shows that neither of these scenarios are a credible explanation for the data shown in Figure 5A: aftereffects are manifest with all three test stimuli, thus demonstrating that unisensory auditory and visual durations can be independently distorted via the auditory and visual components of a multisensory adapting stimulus. A repeated measures ANOVA shows no significant difference between the height of the three bars shown in Figure 5B or Figure 5A's ‘physical’ condition (gray bar), F(3, 18) = 0.75, p > 0.1. 

The data of Figure 5A suggest a processing hierarchy where duration perception is modulated by recent experience before the integration of temporal information across the senses. A further prediction of the ‘early adaptation’ hypothesis is that duration distortions generated via adaptation should feed forward to influence later-stage multisensory integration (Figure 6). We tested this hypothesis using an experimental design which exploited the sensory specificity of duration aftereffects. Observers adapted to relatively long or short auditory durations before a test phase consisting of visual duration discrimination judgments between a test duration (concurrently presented 320 ms visual and auditory durations) and a variable visual reference (see Figure 2B and Methods for details). Auditory duration adaptation will induce duration aftereffects confined to the auditory—but not visual (Heron, Aaen-Stockdale, et al., 2012; Walker et al., 1981)—component of the test stimulus (Figures 6C, D). Relatively early multisensory integration would be unaffected by (later stage) duration adaptation (Figure 6C). Alternatively, late stage multisensory integration would inherit duration aftereffects generated by (earlier stage) adaptation mechanisms and transfer these distortions to the perceived duration of the visual test stimuli (Figure 6D). Figure 7A shows duration discrimination data from a representative naïve observer where the lateral separation in the resultant psychometric functions demonstrates a clear modulation of visual duration perception. Specifically, 640 ms auditory adapting durations (Figure 7A—green curve, circle symbols) induce compression in the perception of the 320 ms auditory component of the test stimulus. Subsequently, MSI mechanisms combine this biased auditory percept with its concurrently presented visual counterpart to induce a corresponding compression of perceived visual test duration (as per the scenario depicted in Figure 6D). This leads to an increase in reference longer than test responses and a corresponding leftward shift in the psychometric function's lateral position. A reverse pattern of results is observed following adaptation to 160 ms auditory durations (Figure 7A—orange curve, square symbols). The height of the gray bar shown in Figure 7B represents PSE shift—the arithmetic difference between PSE values for the two adapting conditions (i.e., the horizontal separation between gray dashed lines in Figure 7A), averaged across observers. 

If these patterns of multisensory integration are determined solely by the perceived duration of the auditory stimulus in Figure 2B's test phase, it should be possible to replicate this effect by repeating our initial multisensory integration experiment using physical auditory distracter durations that match the perceived auditory durations depicted in Figures 6C, D. To measure these durations, observers adapted to 160 ms or 640 ms unimodal auditory durations before judging the duration of a fixed 320 ms auditory reference stimulus against a visual test stimulus whose duration varied around 320 ms (Figure 2C). In line with previous reports (Heron, Aaen-Stockdale, et al., 2012), adaptation to short or long auditory durations generated PSEs of 357 ms (perceptual expansion) and 281 ms (perceptual compression), respectively. We then generated physical auditory distracter durations of 357 ms and 281 ms and repeated our multisensory integration experiment (without adaptation—see Figure 1A) by pairing these distracters with a 320 ms visual test stimulus. The height of the white bar shown in Figure 7B represents the PSE shift (the arithmetic difference between visual PSEs from the 357 ms and 281 ms auditory distracter conditions) induced by these auditory distracters. Both perceptual duration discrepancies, t(6) = 6.12, p < 0.005, and their duration-matched physical counterparts, t(6) = 3.06, p < 0.05, induced PSE shifts that were significantly different from zero (see Figure 7B's gray and white bars, respectively). The close correspondence, t(6) = 0.50, p > 0.1, between these two conditions strongly suggests that the PSE shifts associated with perceptual adapt-test differences are a product of adaptation-induced distortions which are subsequently modified by sensory integration (e.g., Figure 6D). 

Discussion

In the current study we set out to investigate the functional hierarchy of temporal processing by measuring the interdependency of two duration distortions, one induced via multisensory integration and one via adaptation. In our first experiment, we show that the integration of concurrently presented visual and auditory duration information induces marked distortions of perceived duration: Visual duration is expanded or contracted in the direction of the auditory stimulus yet—in relative terms—auditory judgments are impervious to the concurrent presentation of visual durations (Figure 3). We then compared conditions where either physical or perceptual differences were introduced between adapting and test durations. Whilst physical adapt-test differences generated robust aftereffects, test stimuli were perceived veridically when perceptual adapt-test differences were generated by sensory integration mechanisms (Figure 5A). Conversely, when duration adaptation mechanisms generated perceptual differences between physically identical auditory and visual test durations (Figure 6), the observed duration distortions were quantitatively consistent with a prediction based on the successive effects of (relatively early) duration aftereffects which feed forward to influence (relatively late) sensory integration mechanisms (Figure 7B). Taken together, these findings reveal a sensory hierarchy where duration information at the level of multisensory integration arrives having undergone modification via adaptation (Figures 4E and 6D). 

Unidirectional interaction between the two temporal illusions we have investigated has implications for the neural loci at which they originate. Firstly, the integration of duration information across the senses appears to be a relatively late stage process, making extrastriate regions such as the superior temporal sulcus (audio-visual speech integration, Beauchamp, Nath, & Pasalar, 2010; Nath & Beauchamp, 2011; Pasalar, Ro, & Beauchamp, 2010) and dorsal medial superior temporal area (visual-vestibular motion integration, Fetsch et al., 2012; Gu et al., 2008) more credible neural sites than the primary sensory cortices. Conversely, our duration adaptation effects displayed characteristics consistent with having arisen at a relatively early stage of sensory processing, a finding which provides support for perceptual mechanisms underpinned by the duration selective neurons found in cat primary visual cortex (Duysens et al., 1996), cat auditory cortex (He, Hashikawa, Ojima, & Kinouchi, 1997), and auditory midbrain of amphibians (Gooler & Feng, 1992; Leary et al., 2008), bats (Casseday et al., 1994; Casseday, Ehrlich, & Covey, 2000; Faure et al., 2003; Mora & Kossl, 2004; Sayegh, Aubie, & Faure, 2011), rats (Perez-Gonzalez et al., 2006), guinea pigs (Wang, Van Wijhe, Chen, & Yin, 2006; Yin, Chen, Yu, Feng, & Wang, 2008), and mice (Brand et al., 2000; Xia, Qi, & Shen, 2000). A neural locus for adaptation within the primary sensory pathways would help to explain why related temporal aftereffects show spatial specificity (e.g., Ayhan, Bruno, Nishida, & Johnston, 2009; Heron, Roach, Hanson, McGraw, & Whitaker, 2012, but see Burr, Tozzi, & Morrone, 2007; Roseboom & Arnold, 2011), and a lack of dichoptic transfer (Bruno, Ayhan, & Johnston, 2010). An interesting question for further work will be to address the extent to which within-modality perception is influenced by adaptation-based duration distortions. For example, if adaptation induced a sufficiently early expansion of perceived duration, we should see the perceived luminance/loudness of very brief stimuli increase in line with an equivalent increase in physical duration. Regardless of the processing stage at which auditory and visual durations are integrated, the fact that unimodal test stimuli are perceived veridically following adaptation to a bimodal test stimulus (Figure 2A and 5A) rules out not only a pre-adaptation locus for multisensory integration but also the regulation of unimodal duration perception via a feedback loop from (late-stage) integration sites (Driver & Noesselt, 2008). For example, our findings do not support the possibility that MSI-distortions could be fed back into a pre-DA processing stage which would regulate the unimodal input to the DA mechanism. 

In summary, we have shown that duration perception is a highly flexible process that is modified via the twin mechanisms of sensory integration and temporal adaptation. Although these mechanisms produce superficially similar distortions of temporal perception, our experiments tease apart a clear separation in their processing order. It appears that, similar to visual spatial frequency or auditory pitch, event duration is a low-level stimulus attribute that undergoes early, adaptation-based recalibration which precedes the integration of temporal information across the senses. 

Acknowledgments

The work was supported by The Wellcome Trust (WT85315 & WT097387), and The Federation of Ophthalmic and Dispensing Opticians. 

Commercial relationships: none. 

Corresponding author: James Heron. 

Email: [e-mail]. 

Address: Bradford School of Optometry and Vision Science, University of Bradford, Bradford, UK.