Time dilation in dynamic visual display

Abstract

How does the brain estimate time? This old question has led to many biological and psychological models of time perception (R. A. Block, 1989; P. Fraisse, 1963; J. Gibbon, 1977; D. L. I. Zakay, 1989). Because time cannot be directly measured at a given moment, it has been proposed that the brain estimates time based on the number of changes in an event (S. W. Brown, 1995; P. Fraisse, 1963; W. D. Poynter, 1989). Consistent with this idea, dynamic visual stimuli are known to lengthen perceived time (J. F. Brown, 1931; S. Goldstone & W. T. Lhamon, 1974; W. T. Lhamon & S. Goldstone, 1974, C. O. Z. Roelofs & W. P. C. Zeeman, 1951). However, the kind of information that constitutes the basis for time perception remains unresolved. Here, we show that the temporal frequency of a stimulus serves as the “clock” for perceived duration. Other aspects of changes, such as speed or coherence, were found to be inconsequential. Time dilation saturated at a temporal frequency of 4–8 Hz. These results suggest that the clock governing perceived time has its basis at early processing stages. The possible links between models of time perception and neurophysiological functions of early visual areas are discussed.

Introduction

The ability of biological systems to utilize multiple scales of temporal information—from sub-milliseconds (Grothe, 2003) to seconds (Fraisse, 1963; Pöppel, 1988) to days (Czeisler et al., 1999)—rests upon diverse neural machineries distributed across several brain regions (Buhusi & Meck, 2005; Lewis & Miall, 2003). Time estimation for short durations, that is, a few hundred milliseconds to a few seconds, is considered to be a rather automatic sensory process. For longer durations, time estimation requires a more cognitive, modality-independent process (Lewis & Miall, 2003; Mauk & Buonomano, 2004; Rammsayer, 1999). Generally, duration cannot be determined at a given moment but requires internally generated and/or externally triggered signals over the interval to be estimated. It has therefore been proposed that time perception is based on the number of changes present during the event (Brown, 1995; Fraisse, 1963; Poynter, 1989). Stimulus motion—a continuous change in space—is thought of as a fundamental cue used for estimating the duration of a certain time interval (Gibson, 1975; Poynter, 1989). Indeed, several studies have shown that time perception for rapidly moving stimuli is lengthened as compared with slower or stationary stimuli (Brown, 1995; Goldstone & Lhamon, 1974; Lhamon & Goldstone, 1974; Roelofs & Zeeman, 1951), a phenomenon referred to as (subjective) time dilation. 

So far, it is unknown what aspect of changes—whether speed, traveled distance, or temporal frequency—is actually critical for time dilation. Determining the crucial factor is the first step toward linking change-based models of time perception to known neurophysiological and psychophysical properties of visual processing (Born & Bradley, 2005; Parker & Newsome, 1998). To do so, we investigated the effects of speed, motion coherence, spatial frequency, and temporal frequency on the induction of time dilation. 

On the basis of a number of experiments, we find that it is the temporal frequency of a stimulus that determines the magnitude of time dilation. Even when no spatial changes were involved in a stimulus, dynamic changes in the stimulus as such are sufficient to induce time dilation. Our results suggest that the clock signals governing perceived duration originate from relatively early processing stages, possibly as early as the primary visual cortex (V1). This dependency on temporal frequency provides a concrete ground for previously proposed change-based models and opens up the possibility to link these models to neurophysiological properties of visual processing areas. 

Methods

Apparatus

In all experiments, stimuli were presented on a LaCie Electron Blue II 22-in. CRT monitor with a 1,024 × 768 resolution at a refresh rate of 100 Hz. Viewing distance was 57 cm, and head movements were restrained by a chin rest. 

Observers

Eight observers participated in each experiment. 

Task

In all experiments, the task was to reproduce the duration of a stimulus by depressing the space bar on a keyboard after each stimulus presentation. During the reproduction period, only the fixation point was presented. One second after the key release, the next trial automatically started. No feedback was given to the observers. 

Stimuli

In all experiments, stimuli were presented for one of five durations (200, 400, 600, 800, or 1,000 ms) against a gray background. Details on the parameters and particular settings of stimuli for each experiment are described below. Each observer completed 15 trials per condition in Experiment 1 and 10 trials in all other experiments. Each experiment consisted of two blocks, and different durations and stimulus conditions were interleaved across trials. 

Experiment 1

We used a moving or stationary black square (3.2° × 3.2°) presented for a variable duration (see above). The stimulus started moving from the position just below fixation, which was presented 400 pixels (16°) to the left of the horizontal center. The observers were seated right in front of fixation. The velocity of the stimulus varied between 0, 2, 4, 8, 16, 24, and 32 deg/s. 

Experiment 2

The stimulus consisted of 300 randomly moving black (0.04° × 0.04°) dots presented within an 8° diameter circular aperture just below fixation. Dots were updated when they moved out of the aperture. Two factors were manipulated: motion coherence (0%, 30%, 60%, and 90%) and velocity (1, 2, 4, 8, and 16 deg/s). The direction of coherent motion was randomly chosen between rightward and leftward motion. 

Experiment 3

In this experiment, we used a concentric sinusoidal grating moving outward (expansion), which was spatially enveloped by a Gaussian profile ( σ = 1°) centered at a fixation point. Two factors were varied independently: speed between 4, 8, 16, and 32 deg/s and temporal frequency between 0.5, 1.0, 2.0, and 4.0 Hz. Spatial frequency was set accordingly for each condition as the ratio between the two. 

Experiment 4

The stimuli in this experiment were identical to those in Experiment 3, except that temporal frequency and spatial frequency were varied independently. Temporal frequency was varied between 0.5, 1.0, 2.0, and 4.0 Hz. Spatial frequency was varied between 0.5, 1.0, 2.0, and 4.0 cpd. 

Experiment 5

The stimulus was a Gaussian luminance blob with a σ of 1°, and the luminance was modulated with a temporal frequency of 2, 4, 6, 8, 10, or 12 Hz. The peak contrast was 1.0 (Michelson). 

Analysis

In all experiments, we calculated the response error by subtracting the physical duration of the stimulus from its response duration. The mean was obtained for each observer per condition and then submitted to a repeated measures ANOVA for statistical analysis. We used α < .05 as a significance threshold. 

Results

Motion-induced time dilation

In Experiment 1, we used moving stimuli to observe the phenomenon of motion-dependent time dilation (Bonnet, 1965; Brown, 1931; Goldstone & Lhamon, 1974; Lhamon & Goldstone, 1974; Roelofs & Zeeman, 1951). The stimulus was a black square moving horizontally against a gray background. After the stimulus had disappeared, observers were asked to press and hold a key to reproduce its duration. To encourage the observers to make their duration judgments as accurate as possible, we varied the presentation duration between 200 and 1,000 ms. The amount of overestimation was calculated as the difference between the reported duration and the physical duration of the stimulus. 

The results are shown in Figure 1B. Consistent with previous findings (Brown, 1931; Goldstone & Lhamon, 1974; Lhamon & Goldstone, 1974; Roelofs & Zeeman, 1951), a moving stimulus was perceived to last longer than a stationary one, even when their actual presentation times were physically equal. In addition, increasing stimulus speed resulted in larger overestimations, F(4,168) = 60.64, p < .001. This speed dependency is consistent with the notion of change-based time perception (Brown, 1995; Poynter, 1989), in which faster speeds imply larger amounts of change in space and, as such, lengthen the perceived duration. 

Figure 1

 

Time dilation from translational object motion (Experiment 1). (A) A trial is illustrated schematically. (B) The overestimation (response–stimulus duration) is plotted as a function of stimulus velocity. An exponential saturation curve (dashed line) is fitted to the data. The error bars indicate the standard error of the mean.

The physical stimulus duration had a significant effect on the perceived duration as well, F(6,168) = 40.32, p < .001. Overall, the durations of brief stimuli were overestimated, whereas longer stimuli were underestimated. This pattern of bias has been known as Vierordt's law (Allan, 1979; Vierordt, 1868) and is thought of as a type of contextual biases or time-order error (Hellström, 1985). In addition, it is possible that holding down a key as brief as 200 ms in some conditions was a difficult task, resulting in an overestimation. This could have also manifested itself as part of the bias. 

Moreover, we found a significant interaction between the stimulus speed and duration, F(24,168) = 3.198, p < .001. To characterize the nature of the interaction, we fitted a soft saturation curve to the data from each stimulus duration for each subject. The curve was a simple three-parameter model  

$r=a\times (1-\exp (-v/b\left)\right)+c,$

where r is the response error, a is the saturation asymptote point, b is the velocity where the response reaches 63% saturation, and c is the baseline of the Vierordt bias. 

The parameters were estimated for the group data. They are shown in Figure 2 as a function of stimulus duration. In Figure 2A, the velocity point of saturation is shown. The saturation was reached at a slower speed when the stimulus duration was brief. As the stimulus duration increased, much faster speed was required to reach saturation. In Figure 2B, the mean baseline bias (parameter c) is plotted. The baseline bias shifted from positive (+200 ms) to negative (−200 ms) as the stimulus duration increased from 200 to 1,000 ms. This decreasing trend confirms the contextual bias that whereas short durations are overestimated, long durations are underestimated. In Figure 2C, the maximum dilation magnitudes (parameter a) are plotted as a function of stimulus duration. Note that these values reflect the difference between the maximum dilation and the baseline (instead of physical duration). This graph shows that a larger amount of time dilation can be obtained if the motion stimulus is presented for a longer duration. 

Figure 2

 

Saturation parameters fitted on data from Experiment 1 are shown as a function of stimulus duration. (A) Mean saturation velocity. (B) Mean baseline bias. (C) Maximum dilation effect.

What might be the origin of this complex interaction? One possibility is that there is an upper limit in the overestimation of a duration about 400 ms, beyond which overestimation is prohibited. Because of the Vierordt bias, short-duration stimuli approach this limit with small amount of motion-induced dilation, resulting in the earlier saturation. However, this is one of many possible hypotheses, and it remains to be seen what underlies the interaction between speed and stimulus duration. Despite the complicated results, the data clearly show the basic fact that in the conditions where faster motion was used, subjects overestimated the stimulus duration. 

Stimuli moving at a fast speed cover a larger space than stimuli moving at a slow speed, when their durations are identical. Consequently, the distance traveled by a stimulus could be used as a cue to estimate perceived duration. For example, if perceived duration were to be determined by the ratio between estimated distance and perceived speed (Brown, 1931), systematic errors in the speed estimate and/or spatial extent could account for our data as well. 

Local versus global motion

To disentangle the contribution of speed and traveled distance, we used a random dot motion display in which the coherence and the speed of individual dots were varied independently ( Experiment 2). In contrast to the moving square used in Experiment 1, this stimulus was constrained within a fixed window on the display. Therefore, cues based on distance were not directly available for time estimation unless the total distance traveled by the stimulus is estimated by the integration of global motion over time. As an extreme condition, we included a zero coherence condition, in which all the dots moved in random directions. For this condition, distance cues cannot be used for time estimation. Moreover, this condition provides insight into the processing stage(s) underlying time dilation. In general, visual motion is processed by multiple, hierarchically organized processing stages. At early stages (i.e., V1), visual input is processed locally over a small spatial extent, whereas at a later stage (i.e., MT/V5), the output from the early stage is integrated into global and coherent motion (Kanai, Paffen, Gerbino, & Verstraten, 2004; Snowden & Verstraten, 1999). Therefore, the manipulation of motion coherence is informative about the role of the early and late stages in time dilation. A dependency on motion coherence would imply the involvement of the late stage in time estimation, whereas a lack of dependency would imply that time estimation is primarily mediated by the early stage. 

The results for this experiment are plotted in Figure 3B separately for each coherence level. We have performed a three-way repeated measures ANOVA to see how the factors of motion coherence, speed of the dots, and stimulus duration affected time estimation. As expected from Vierordt's law, we obtained the main effect of stimulus duration, F(4,28) = 46.879, p < .001. The coherence did not affect perceived duration, F(3,21) < 1, and did not interact with either stimulus duration, F(12,84) < 1, or speed of dots, F(12,84) < 1. By contrast, the speed of individual dots had a significant effect; perceived duration was lengthened as the speed of the individual dots increased, F(4,28) = 17.902, p < .001. Consistent with Experiment 1, time dilation saturated with a lower speed when stimulus presentation was brief, resulting in a significant interaction between the speed of dots and stimulus duration, F(16,112) = 3.023, p < .001. 

Figure 3

 

Motion coherence versus velocity ( Experiment 2). (A) A trial is illustrated schematically. (B) The overestimation is plotted as a function of coherence. Each line indicates a different speed of individual dots. (C) The same data are shown as a function of speed of individual dots. Each line indicates a different coherence. The average standard error of the mean is shown to indicate intersubject variability.

These results indicate that time dilation is determined solely by the speed of individual dots and that coherence has no influence on time perception. Even in the zero coherence condition, time dilation increased with increasing speed of individual dots. This means that a coherent motion is not necessary for motion-induced time dilation. Thus, our time estimates for a visual event may not be dependent on explicit representations of spatial translations but may originate from more implicit information present in early visual motion processing. 

Speed versus temporal frequency

Neurons at early processing stages (V1) are tuned to both spatial frequency and temporal frequency (Foster, Gaska, Nagler, & Pollen, 1985), and their ratio determines their speed tuning. In the next set of experiments, we investigated which of these stimulus parameters—speed, temporal, or spatial frequency—is critical for time dilation. In this experiment (Experiment 3), we used a concentric grating that contains expanding motion and we varied the temporal frequency and speed independently. Using this manipulation, once speed and temporal frequency were fixed for a condition, spatial frequency was uniquely determined. Thus, faster motion was always associated with a lower spatial frequency. To test whether the manipulation of spatial frequency as such affects time dilation, we varied temporal frequency and spatial frequency independently in Experiment 4. 

The results of Experiment 3 are shown in Figures 4B and 4D. We performed a three-way repeated measures ANOVA to see the effects of stimulus duration, speed, and temporal frequency. As expected from the previous experiments, stimulus duration had a significant effect on perceived duration, F(4,28) = 9.696, p < .001. Overall, the characteristics of the dependency on stimulus duration accord with Vierordt's law. However, there are two peculiarities that should be noted. First, we did not obtain a negative bias, as we did in previous experiments. Even stimuli as long as 1,000 ms were positively biased, that is, overestimated in all stimulus conditions. Second, the bias for the shortest duration (200 ms) was smaller than for the second shortest duration (400 ms). These differences may have come from the difference in stimuli (e.g., translation versus expansion) or from the variability across subjects. However, the reasons for these discrepancies are unclear. We leave the issue of how Vierordt's law is affected by different stimulus conditions to future studies. 

Figure 4

 

Temporal frequency versus speed ( Experiment 3). (A) A trial is illustrated schematically. (B) The overestimation is plotted as a function of speed. Each line indicates a different temporal frequency. (C) The same data are shown as a function of temporal frequency. Each line indicates a different speed. The average standard error of the mean is shown to indicate intersubject variability.

The manipulation of speed itself did not have a significant effect on perceived duration, F(3,21) = 2.686, p = .073. Instead, time dilation was produced by increasing the temporal frequency of the stimulus, F(3,21) = 36.43, p < .001. This implies that the main cause of time dilation is not motion as such but the dynamic nature of the stimulus. 

Although increasing stimulus speed did not lengthen perceived duration monotonically, it had modulatory effects on time dilation (interaction with temporal frequency), F(9,63) = 7.64, p < .001. Because stimulus duration did not have a significant interaction with speed, F(12,84) = 1.177, p = .313, or with temporal frequency, F(12,84) = 1.569, p = .117, we plotted the data collapsed across different presentation durations to illustrate the source of the interaction between temporal frequency and speed ( Figures 4C and 4D). It can be seen in Figure 3C that the range of time dilation is more limited when the stimuli moved slowly. In other words, for stimuli with low temporal frequency, increase in speed (i.e., decreasing spatial frequency from medium to low) resulted in a smaller time dilation, whereas for stimuli with high temporal frequency, increase in speed (i.e., decrease in spatial frequency from high to medium) resulted in a larger time dilation. While the origin of this interactive effect of speed remains mysterious, our results support the idea that time dilation is primarily dependent on the temporal frequency of the stimulus, and other constituents of a movement do not have much, if there is any, influence. 

In the interest of completeness, we manipulated spatial frequency and temporal frequency independently in Experiment 4. Consistent with all other experiments, the two factors stimulus duration and temporal frequency reached significance, F(4,28) = 2.319, p < .001, and F(3,21) = 30.46, p < .001, respectively, but spatial frequency did not show a significant effect on time dilation, F(3,21) = 1.396, p = .272. Spatial frequency did not interact with temporal frequency, F(12,84) < 1, but showed a significant interaction with stimulus duration, F(12,84) = 4.936, p = .001. We do not have a clear explanation for this interaction. Similar to the previous experiments ( Experiments 1 and 2), we found a significant interaction between stimulus duration and temporal frequency, F(12,84) = 4.463, p < .001. This appears to come from the saturation of the dilation effect at low temporal frequency in the conditions where stimuli were briefly presented ( Figure 5B). 

Figure 5

 

Temporal frequency versus spatial frequency ( Experiment 4). (A) A trial is illustrated schematically. (B) The overestimation is plotted as a function of spatial frequency. Each line indicates a different temporal frequency. (C) The same data are shown as a function of temporal frequency. Each line indicates a different spatial frequency. The average standard error of the mean is shown to indicate intersubject variability.

Taken together, these results support the idea that temporal frequency, not speed or spatial frequency, is the determining factor for time dilation in dynamic display. Although both speed and spatial frequency seem to have some modulatory effects on time dilation, those factors by themselves do not monotonically regulate the magnitude of time dilation. 

Flicker-induced time dilation

To directly assess the effect of temporal frequency without any contamination of other factors, we tested if stimulus flicker can produce time dilation and if time dilation is produced in a frequency-dependent fashion ( Experiment 5). The flicker stimulus was Gaussian blob modulating its luminance and did not involve motion at all. 

The results of this experiment are shown in Figure 6B. As expected, we found a significant effect of duration (Vierordt's law), F(4,140) = 13.358, p < .01. The magnitude of time dilation increased with temporal frequency, F(5,140) = 100.589, p < .001, supporting the hypothesis that temporal modulation of a stimulus can lead to perceptual time dilation without directional motion. Also, there was a significant interaction between stimulus duration and temporal frequency, F(20,140) = 14.469, p < .001, a pattern we have consistently observed in the previous experiments. 

Figure 6

 

Time dilation from a flickering dot ( Experiment 5). (A) A trial is illustrated schematically. (B) The overestimation is plotted as a function of temporal frequency. An exponential saturation curve (dashed line) is fitted to the data. The error bars indicate the standard error of the mean.

Discussion

We have demonstrated that time dilation in a dynamic visual display is determined primarily by its temporal frequency: Apparent duration of a visual event increases with increasing temporal frequency. Other spatiotemporal characteristics of dynamic stimuli such as coherence, speed, and spatial frequency had only minor modulatory effects. 

Generally speaking, time estimation must rely on temporal cues of some form, whether they are externally driven by stimulus input or internally regulated (e.g., neural oscillators). Many theorists have proposed that the number of changes present in a stimulus might be used as a cue to estimate the passage of time (Fraisse, 1963; Gibson, 1975; Poynter, 1989). The idea is that the more changes a stimulus undergoes, the more time must have elapsed. In particular, these models suggest that motion might be one of the most informative temporal cues because motion by definition involves changes in position over time. This idea has been supported by the fact that visual motion expands perceived time (Brown, 1931; Brown, 1995). However, visual motion has several characteristics, and in contrast to what is believed so far, our present study shows that temporal frequency is the most fundamental factor in time dilation, if not the only factor. In fact, our results indicate that motion as such is not critical at all: Time dilation was observed in noise (zero coherence) displays and simple flicker stimuli. The finding that temporal frequency can be used as an indicator for how rapidly a scene is changing is consistent with the change-based models of time perception. 

Models of time perception often assume the existence of an internal clock that generates “pulses” (Creelman, 1962; Treisman, 1963), which are subsequently integrated at an accumulation stage. However, this idea of pulse generator is highly hypothetical and the connection between cognitive models and neurophysiological properties of visual neurons is barely established. In the context of the pulse generator models, our present findings may imply that the neural responses in early visual areas, which modulate in accordance with the contrast modulation of input stimulus, may play the role of the temporal pulses in the visual domain. 

In a recent study, Johnston, Arnold, and Nishida (2006) have found a distortion of perceived time that was restricted to an area adapted to a visual flicker. This suggests that the source of temporal cues resides in early visual areas, where inputs are constrained within small receptive fields. While it is likely that each sensory modality uses different temporal cues as pulses, our results indicate that temporal frequency plays the critical role in visual time perception. 

In our final experiment ( Experiment 5), we observed that time dilation saturates at a temporal frequency around 4–8 Hz ( Figure 7A). To see whether this saturation frequency was specific for this experiment, we calculated the saturation frequency for the other four experiments (see the Methods section). Compared to the limit of flicker perception (Roufs, 1972), this temporal rate is quite low. Therefore, not all events of contrast reversals appear to add up to the final time estimation for stimuli with a high temporal frequency. 

Figure 7

 

Saturation parameters fitted on data from Experiment 5 are shown as a function of stimulus duration. (A) Mean saturation velocity. (B) Mean baseline bias. (C) Maximum dilation effect.

In summary, our present study shows that temporal frequency plays a critical role in time dilation. In a more global perspective, however, temporal frequency is certainly not the only factor that determines perceived duration. For example, we can estimate the duration of a stimulus that does not involve any changes. This means that even in the absence of changes in a stimulus, time can be estimated. Moreover, information of the onset and offset of an interval is also important in time estimation, and delays in those timing signals result in errors in perceived duration (Kanai & Watanabe, in press). Thus, time perception is a result of orchestrated multifaceted processing. Yet, as regards the time dilation of dynamic stimuli, temporal frequency, not motion per se, is the primary determinant of the effect. How the signals driven by temporal frequency interact with more internally driven signals, as would be the case for stationary stimuli, warrants further investigations. 

Acknowledgment

We would like to thank Alan Johnston for discussion. 

Commercial relationships: none. 

Corresponding author: Ryota Kanai. 

Email: [e-mail]. 

Address: California Institute of Technology, Division of Biology, M/C 114-96, Pasadena, CA 91125, USA.