The weight of time: Gravitational force enhances discrimination of visual motion duration

Abstract

In contrast with the anisotropies in spatial and motion vision, anisotropies in the perception of motion duration have not been investigated to our knowledge. Here, we addressed this issue by asking observers to judge the duration of motion of a target accelerating over a fixed length path in one of different directions. Observers watched either a pictorial or a quasi-blank scene, while being upright or tilted by 45° relative to the monitor and Earth's gravity. Finally, observers were upright and we tilted the scene by 45°. We found systematic anisotropies in the precision of the responses, the performance being better for downward motion than for upward motion relative to the scene both when the observer and the scene were upright and when either the observer or the scene were tilted by 45°, although tilting decreased the size of the effect. We argue that implicit knowledge about gravity force is incorporated in the neural mechanisms computing elapsed time. Furthermore, the results suggest that the effects of a virtual gravity can be represented with respect to a vertical direction concordant with the visual scene orientation and discordant with the direction of Earth's gravity.

Introduction

According to an influential view, encoding mechanisms in the visual system may have evolved to better process the prevailing contents in the visual world (e.g., Barlow, 1959; Gros, Blake, & Hiris, 1998; Howe & Purves, 2005; Simoncelli & Olshausen, 2001). The object orientation in the visual field is especially significant from an ecological standpoint. Most images of our gravity-bound natural or man-made environments are anisotropic, with more image structure at orientations parallel or orthogonal to the direction of gravity in a fronto-parallel plane (e.g., Hansen & Essock, 2004). These image anisotropies can be matched by corresponding anisotropies in perceptual responses, lending support to the idea that the visual system takes into account the statistics of natural environments. Thus, contours are detected and discriminated more easily when they are oriented vertically or horizontally (cardinal directions), compared with when they are oriented obliquely (the so-called “oblique effect”; Appelle, 1972; Campbell, Kulikowski, & Levinson, 1966). This oblique effect is typically observed when the stimuli consist of isolated lines, gratings, and other narrowband stimuli. Instead, orientation discrimination is best at obliques and worst at horizontal with broadband stimuli, increasing the relative salience of a target when viewed against a typical natural background (the so-called “horizontal effect”; Hansen & Essock, 2004). 

An oblique effect has also been demonstrated for the perception of motion direction: subtle directional differences between two sets of moving random dots are more easily discriminated along cardinal than oblique axes (Ball & Sekuler, 1987; Gros et al., 1998; Matthews & Qian, 1999). However, no systematic differences across the cardinal directions (upward, downward, leftward, and rightward) have been reported (Ball & Sekuler, 1987; Gros et al., 1998; Matthews & Qian, 1999), although centripetal hemifield motion can be detected and discriminated better than centrifugal motion (Giaschi, Zwicker, Young, & Bjornson, 2007; Raymond, 1994). As for speed and acceleration perception, they appear to be isotropic (Calderone & Kaiser, 1989; Matthews & Qian, 1999). In particular, the detection threshold of acceleration in a vertically moving target does not differ significantly between the downward and upward directions (Calderone & Kaiser, 1989; Chang & Troje, 2009). Thus, motion perception tends to be isotropic relative to the up/down directions. 

Anisotropies in the perception of motion duration have not been investigated to our knowledge. Generally, time duration cannot be directly measured at a given moment but requires internally generated and/or externally triggered signals over the interval to be estimated (see Eagleman, 2008). It has been proposed that, to solve this indeterminacy, the brain constantly calibrates its time estimation against physical laws from the outside world (Eagleman, 2004). Thus, the position of a moving object at a given time in the near future might be predicted by a forward internal model of Newtonian mechanics (Davidson & Wolpert, 2005; Zago, McIntyre, Senot, & Lacquaniti, 2009) and compared with sensory feedback to keep the perceived time calibrated (Eagleman, 2004). 

Because Earth's gravity is locally constant, the motion of any object accelerated by gravity has a fixed duration over a given path (neglecting air drag). Therefore, implicit knowledge about the direction and the effects of gravity could be used by the brain for consistent timekeeping. Indeed, the manual interception of a vertically falling ball is accurately timed (Lacquaniti & Maioli, 1989; see Zago et al., 2009), as is the indication of the time of landing of a computer-animated target that rolls off a horizontal surface and falls hidden from view (Huber & Krist, 2004). Furthermore, the final position of a horizontally moving target that is suddenly halted is misremembered as being displaced downward below the path of motion, consistent with the idea that gravity effects are implicitly assumed by the observers (Hubbard, 1995). The ability to detect unnatural features in vertical visual motion related to gravity can be demonstrated early in life. Between 5 and 7 months, infants begin to implicitly expect a downwardly moving object to accelerate and an upwardly moving object to decelerate, as shown by an abnormally prolonged attention at displays with the unnatural motion (Friedman, 2002; Kim & Spelke, 1992). 

Here, we consider the possibility that an internal model of gravity also affects the perceptual judgment of temporal duration of a visual motion. If relative time in perceptually structured displays is efficiently encoded because of the availability of models of physics, time discrimination should be more precise when the motion of an object complies with gravity constraints than when it artificially violates such constraints. One way of testing this hypothesis is to compare the time discrimination for linear motion of a virtual object across the four cardinal directions: downward, upward, rightward, and leftward. With a constant positive value of target acceleration, target kinematics would be congruent with the effects of gravity only for the downward direction, and one would expect a corresponding anisotropy in time discrimination: durations should be discriminated more precisely during downward motion than during the other motion directions. 

The next issue is to evaluate whether the sensitivity to gravity constraints is tied to retinal (or other egocentric) coordinates, to Earth's gravity, or to visual references intrinsic to the scene (see, for instance, Chang, Harris, & Troje, 2010; Kushiro, Taga, & Watanabe, 2007; Lopez, Bachofner, Mercier, & Blanke, 2009; Troje, 2003). Thus, the previously described anisotropy in motion direction discrimination between cardinal and oblique axes is tied to retinal coordinates. Indeed, when the observer's head was rolled by 45°, performance was better when the reference direction was 45° oblique with respect to the monitor and thus a cardinal direction with respect to the retina (Gros et al., 1998). However, the direction of perceptual “down” generally depends on several different sensory and internal cues about visual reference, Earth's gravity, and body orientation (De Vrijer, Medendorp, & Van Gisbergen, 2008; MacNeilage, Banks, Berger, & Bülthoff, 2007; Van Beuzekom & Van Gisbergen, 2000; Zupan, Merfeld, & Darlot, 2002). Notice, in particular, that the gravity apparently acting on people or objects in a distant visual scene may not be spatially aligned with the physical gravity and/or ourselves, as it happens, for instance, when we are upright and watching a movie on a tilted monitor, or we are tilted. In such cases, built-in pictorial cues (e.g., familiar size, linear perspective, shading, texture gradient) of the visual scene may help gauge the approximate spatial scale and orientation of the scene and help estimate the effects of the apparent gravity concordant with the scene reference frame. The question then is whether pictorial cues also contribute to enhancing time discrimination along the direction of the apparent gravity. A critical test would be to keep the head and body aligned with the physical vertical and to tilt the pictorial scene. Alternatively, one can tilt the observer and keep the pictorial scene aligned with the physical vertical. In both cases, downward motion in the scene reference frame would produce oblique motion relative to the retinal coordinates. However, the results of these two conditions may not necessarily coincide due to the different relative weight of visual, gravity, and body orientation cues (Chang et al., 2010; Jenkin, Jenkin, Dyde, & Harris, 2004). 

To address the above issues, we performed six different experiments in each of which we compared time discrimination across different directions of target motion. In Experiments 1 and 2, the directions were downward, upward, rightward, or leftward. In Experiment 1, target motions were superimposed on a pictorial background, whereas in Experiment 2 the same motions were superimposed on a non-pictorial, quasi-uniform background. In Experiments 3 and 4, the target moved along oblique lines, rotated by 45° relative to the cardinal axes, on the non-pictorial background. In Experiment 5, we used the same pictorial background and up/down motions as in Experiment 1, but we tilted the observer by 45° relative to the monitor and Earth's gravity. Finally, in Experiment 6, we kept the observer upright, and we tilted the monitor and the pictorial scene by 45°. 

Experiment 1

Here, target motion was embedded in a scene of an inhabited room, rendered with computer graphics in perspective view (Figure 1). We included pictorial cues (familiar size, linear perspective, shading, and texture gradient) sufficient to roughly gauge the scale of the scene and to estimate the effects of gravity on target motion in the reference frame of the visual scene. We chose an indoor, man-made environment because it is known to carry strong orientation cues (Haji-Khamneh & Harris, 2010). 

Figure 1

 

Background scene and target of Experiment 1. The target sphere moved at constant acceleration between two opposite holes of the room. Motion direction was downward, upward, rightward, or leftward in different blocks of trials, kinematics being congruent with the effects of gravity only in the downward direction. The red dot in the center was the fixation point. The image depicted here is from a downward motion trial. All directions of motion are shown in the Supplementary movie.

The magnitude of target acceleration was always 9.81 m s⁻² (typical gravitational acceleration), while the direction was downward, upward, rightward, or leftward in different blocks of trials. Thus, target kinematics was congruent with the effects of gravity only for downward motion. 

Methods

Rationale

To assess the perceptual discrimination of motion duration, we used a hybrid of the method of constant stimuli (MCS) and the method of single stimuli (MSS, e.g., Morgan, Watamaniuk, & McKee, 2000). Classically, with MCS an explicit standard is presented on each trial, whereas with MSS the standard is implicitly defined by the mean of the whole set of trials. It has been shown that thresholds measured with the MSS procedure are just as precise as those measured with the traditional MCS procedure, but the former procedure halves the duration of the experiment relative to the latter procedure (Morgan et al., 2000). To increase the reliability of the reference for MSS, we presented a first set of consecutive standard stimuli with constant duration during a preview phase. This initial phase was followed by the test phase in which the subjects had to judge whether a given test stimulus—randomly drawn from a sample with 9 different durations (with mean equal to the standard duration)—lasted longer or shorter than the reference standard. 

In each block of trials, the standard stimuli moved in the same direction as the following test stimuli. While the acceleration of the targets was fixed (9.81 m s⁻²), their speed ranged between 18° s⁻¹ and 40° s⁻¹. These values are well within the range of speeds that are known to be best discriminated: Weber fractions (WFs) have been shown to be lowest (about 7%) for speeds between about 4° s⁻¹ and 64° s⁻¹ (De Bruyn & Orban, 1988). As for acceleration detection, it is often assessed in terms of a ratio analogous to a WF: (V _final − V _initial)/V _average, where V _final, V _initial, and V _average are the final, initial, and average speeds, respectively (Calderone & Kaiser, 1989; Regan, Kaufman, & Lincoln, 1986). In our experiments, this ratio ranged between 0.4 and 1.96 (corresponding to the shortest and longest durations of motion, respectively). These values are above the detection thresholds of acceleration (0.17–0.25), which are typically reported in the literature (Brouwer, Brenner, & Smeets, 2002; Calderone & Kaiser, 1989; Regan et al., 1986; Werkhoven, Snippe, & Toet, 1992). 

Participants

Seven subjects participated in this experiment (6 naive subjects plus author A.M.; 3 females and 4 males, 27 ± 3 years old, mean ± SD). They were right-handed (as assessed by a short questionnaire based on the Edinburgh scale). All participants in this and the following experiments had normal or corrected-to-normal vision and gave informed consent to procedures approved by the Institutional Review Board of Santa Lucia Foundation, in conformity with the Declaration of Helsinki on the use of human subjects in research. 

Apparatus and stimuli

Participants binocularly viewed a display (HP p1130, 21″ CRT Monitor, 1024 × 768 pixels, 85-Hz refresh rate, controlled by HP xw4600 with nVidia GeForce 8800 GTX graphics card) placed 0.6 m in front of them in a dimly illuminated room. The height of the chair where they sat was adjusted so that subject's eyes were at roughly the same height as the display midpoint. In order to reduce the effects due to vertical and horizontal visual references outside the display, we placed a large non-textured black panel behind the computer monitor, as well as uniform, non-textured panels at the sides of the observer. Button-press responses were recorded by sampling the USB output of a standard HP mouse at 125 Hz. All visual stimuli were generated with NBS Presentation (Neurobehavioral System Presentation 13.0, Albany, Canada). They were defined in a left-handed coordinate system with leftward X-axis and upward Y-axis in the frontal plane, plus in-depth Z-axis. Scene projection was computed using on-axis linear perspective, assuming a viewpoint at [0, 3 m, −10 m] and looking at point [0, 3 m, 0]. The fixation point was located at [0, 3 m, 6 m] of this frame, while the center of mass of the target moved in a frontal plane through the origin [0, 0, 0]. We displayed a colored scene (35° by 26°, horizontal and vertical visual angles, respectively) with the image of a large room (Figure 1). Eight human figures were placed at different positions in the room to provide an approximate metric reference. Photographs of the adults were downloaded with permission from www.vyonyx.com (copyrights owned by VYONYX). The photograph of the child was downloaded with permission from www.imagecels.com (copyrights owned by Realworld Imagery). The fixation point was a red dot (0.38°) placed in the center of the rear wall. There were four circular holes in the room: on the ceiling, on the floor, and on each of the sidewalls. The distance between each pair of opposite holes was 6 m in the scale of the scene (corresponding to 20°, ±10° around fixation). A white stripe painted on the ceiling, floor, and sidewalls connected the holes. 

During each trial, a textured black-and-white sphere (0.8°), representing a soccer ball, accelerated along a straight path between two opposite holes. The ball was displayed as emerging initially from within the start hole, and then shifting with the prescribed law of motion to finally disappear within the opposite hole. In order to remove the traveled distance as a cue to estimate time duration, the ball always shifted over the whole, fixed path. The magnitude of the acceleration was always 9.81 m s⁻² (33° s⁻²), while the direction was downward, upward, rightward, or leftward, depending on the specific block of trials (see Procedures section below). Thus, target kinematics was congruent with the effects of gravity only in the downward block. The initial speed of the target could vary in different trials, resulting in a variable total duration of the visible motion (see below). Luminance and RGB color coordinates of the background were determined over the region corresponding to the rear wall of the room: average luminance was 23 cd m⁻² (as measured by means of Tektronix J17 LumaColor photometer) and average RGB coordinates were 171, 113, and 78 (as determined by NBS program). 

Procedures

Before the experiments, participants received general instructions and were familiarized with the setup in a brief training session. Specific instructions regarding the different phases of the experiment were written on the monitor prior to each phase. Downward, upward, rightward, and leftward motions were blocked (order counterbalanced across subjects), so that there were four blocks of trials in each experiment (with a brief rest between blocks). Each block started with the presentation of 60 consecutive standard trials (ISI = 500 ms) in which the initial target speed was fixed (12° s⁻¹), resulting in a total duration of motion T = 800 ms and in an average speed of 25° s⁻¹ over the displayed trajectory. During this phase, participants were instructed to watch each video so as to memorize the standard flight duration. After this phase, 360 test trials (ISI = 2500 ms) were presented with the same direction of motion as the standard trials but with a variable initial speed and total duration of motion. These stimuli could have one of nine possible durations (within a range of 500–1100 ms, centered on 800 ms), randomized across trials. Each exact duration was an integer multiple of the monitor frame duration. The average target speed ranged between 18° s⁻¹ and 40° s⁻¹ (corresponding to the longest and shortest durations, respectively). In each test trial, after 500 ms from target disappearance, a question mark appeared over the fixation point, prompting the participants to provide a response in 2 s. (If they responded before or after the allocated time window, the trial was rejected and repeated at the end of the experiment.) They indicated whether the test stimulus was longer or shorter in duration than the standard stimuli by pressing the right or left mouse button, respectively. Each stimulus duration was presented 40 times (9 durations × 40 repetitions = 360 test trials) in each block. Participants were asked to fixate the central red dot during the presentation of both standard and test stimuli. No performance feedback was provided. 

Data analysis

Analyses were carried out in R software (R 2.10, R Development Core Team, 2009, R Foundation for Statistical Computing, Vienna, Austria, www.R-project.org). For each subject and condition (down, up, left, right), we computed a psychometric function based on the percentage of trials in which the test was judged as longer than the standard (using all 360 responses). In any given trial, we defined a “Longer” response as Y = 1, and a “Shorter” response as Y = 0. Because the responses were asymmetrically distributed about P(Y = 1) = 0.5, they were fitted with the log–log link function: 

$\log [-\log (P(Y=1)\left)\right]={\alpha }_{\mathrm{p}}+{\beta }_{\mathrm{p}}x,$

(1)

where P() is the probability of response for test duration equal to x, and α _p and β _p are the intercept and slope of the model, respectively. The log–log model fitted the data better than a logistic model according to the Akaike information criterion (AIC; Akaike, 1973): at the population level, the difference in AIC was 131. Although the choice of the log–log model was primarily suggested by its good fit, there was also an a priori reason to prefer it over the logistic model. As pointed out by Miller and Ulrich (2001), Weber's law provides a very general argument that ideal psychometric functions should be positively skewed (as in the log–log model), rather than symmetric functions (as in the logistic or probit models), because a given change in stimulus value should have a greater effect at the bottom of the stimulus range than at the top. 

The psychometric function derived from Equation 1 is given by 

$P(Y=1)=\exp [-\exp ({\alpha }_{\mathrm{p}}+{\beta }_{\mathrm{p}}x\left)\right].$

(2)

The slope β _p provides a measure of the precision of discrimination (the higher the slope, the greater the precision). The point of subjective equivalence (PSE) estimates the accuracy of the judgment. It was computed from the psychometric function of Equation 2 according to 

$\mathrm{P}\mathrm{S}\mathrm{E}=\frac{\log [-\log (0.5\left)\right]-{\alpha }_{\mathrm{p}}}{{\beta }_{\mathrm{p}}}.$

(3)

The 95% confidence interval of the PSE for each subject was estimated using the delta method (Casella & Berger, 2002). 

Population responses for each given condition were derived in two different ways to verify the consistency of the results. First, a population psychometric function was obtained by simply pooling the responses over all participants. Second, the data of all subjects were fitted by means of a Generalized Linear Mixed Model that separately accounts for random effects and fixed effects (GLMM; Agresti, 2002). GLMM was fitted using the R package “repeated” (Lindsey, 2009, www.commanster.eu). For a log–log function, the model is given as follows: 

$\log [-\log (P(Y=1|u_i)\left)\right]=u_i+\mathit{\beta }\mathbf{X},$

(4)

where u _i is the random effect for subject i, X is the design matrix, and β is the vector of coefficients of the fixed effects. In our case, the GLMM included a single random effect parameter (the random intercept) and eight parameters of fixed effects estimating the intercept, the flight duration (that is, the slope with the downward condition as the baseline), three dummy variables corresponding to the three remaining conditions (leftward, rightward, and upward), and the interaction between flight duration and the three dummy variables. For each parameter, we computed the following Wald statistics: 

$z=\frac{\widehat{\beta }}{SE},$

(5)

where

$\widehat{\beta }$

is the estimated parameter and SE is its estimated standard error, and we derived the corresponding two-sided p-values (Agresti, 2002). The same statistics was used to test the statistical significance of the parameters of the population psychometric functions. Because the standard deviation of the random location factor turned out to be significantly different from zero (z = 20.5, p < 0.001), the use of a mixed model was statistically justified (Agresti, 2002). To compare the precision of discrimination across conditions and experiments, the values of the slope of the population responses were normalized by dividing each value by the slope of the downward condition. 

An alternative estimate of precision is given by the discrimination threshold ΔT. This can be derived from the psychometric function as ΔT = 0.5(T _0.75 − T _0.25), where T _0.75 and T _0.25 are the values of flight duration yielding 0.75 and 0.25 probabilities of “Longer” responses. The Weber fraction then is WF = ΔT/T _standard, where T _standard is the standard duration (800 ms). 

Statistical significance of all tests was set at α = 0.05, after adjusting for multiple comparisons according to the false discovery rate procedure (Benjamini & Hochberg, 1995). 

Results and discussion

Figure 2a compares the psychometric function obtained by pooling the responses over all participants for the downward motion condition (blue) with the psychometric function for the upward motion condition (red). The data were well fit as shown by a non-significant deviance (Agresti, 2002; Wichmann & Hill, 2001). The psychometric function was significantly (p < 0.05) steeper (higher slope) for downward motion than for all other tested directions of motion, indicating a higher precision of judgment or, equivalently, a better discrimination of the stimulus duration for the downward motion. This trend was observed also at the level of single subjects: the response slope was higher for downward motion than that for the other directions in most (6/7) participants. In one participant, the slope for downward was the second highest after that for leftward. 

Figure 2

 

Results from Experiment 1. (a) Psychometric functions for downward motion (blue) and upward motion (red). Data were pooled over all participants (n = 7). The graphs show the proportion of times the test stimulus appeared to last longer than the standard. Each data point corresponds to one of 9 test durations and is the average of 280 responses (40 repetitions for each subject). The data were fitted with the log–log link function. The vertical line denotes the standard duration (800 ms). The horizontal line crosses the 50% point of the psychometric functions and identifies the PSE of the different conditions. (b) For each motion direction, the precision of discrimination was assessed as the slope of the GLMM population response. For the sake of comparison, all values were normalized relative to the downward condition. Error bars denote ±1 SD; *** and * correspond to a difference with p < 0.001 and p < 0.05, respectively.

Population responses were also estimated using the GLMM (see Data analysis in Methods section). First, we searched for any significant effect of block ordinal position on the slope averaged across directions of motion, and we found none (z = 0.536, p = 0.59). Next, we computed the values of the slope of the population responses obtained for each motion direction (Figure 2b). The slope was significantly higher in downward motion than upward (z = 4.468, p < 0.001), rightward (z = 3.919, p < 0.001), and leftward motions (z = 2.454, p = 0.016). Absolute values of the slope were 0.0083 ± 0.00032 (mean ± SD, over all subjects), 0.0064 ± 0.00026, 0.0072 ± 0.00029, and 0.0066 ± 0.00027 for down, up, left, and right, respectively. On average, the slope for downward motion was 23, 13, and 20% higher than the slope for upward, leftward, and rightward motions, respectively. For downward motion, average discrimination threshold (ΔT) was 87 ± 3 ms (mean ± SD, over all subjects) and average Weber Fraction (WF) was 0.109 ± 0.004. For upward motion, ΔT = 110 ± 4 ms, and WF = 0.137 ± 0.005. 

The population PSE values did not differ significantly (at the 95% confidence interval) between the four motion directions nor did they differ significantly from the reference value of the standard duration (800 ms, see Figure 2a). On average, the PSE values were 809 ± 46 ms (mean ± SD) for downward, 824 ± 50 ms for upward, 826 ± 49 ms for leftward, and 805 ± 65 ms for rightward, indicating fairly accurate estimates of stimulus duration. This shows that the method we used to present reference and test stimuli did not create any bias. 

The results of the first experiment showed a clear directional anisotropy in the perceptual judgment of target motion duration. Durations were discriminated significantly better when the target moved in the downward direction than when it moved in the other tested directions. 

It should be noticed that targets translated across a blank scene in previous studies of speed and acceleration discrimination (Calderone & Kaiser, 1989; Matthews & Qian, 1999), whereas our targets moved in a pseudo-3D scene with several pictorial cues that might help process visual motion. Therefore, in the next experiment, we removed size/distance cues to verify whether perceived motion duration remained anisotropic. 

Experiment 2

Here, we projected the target motion on a quasi-uniform background image (Figure 3). Both the moving target and the static landmarks in the background were 2D abstract geometrical figures providing clear vertical and horizontal references but no motion metric in allocentric reference frames. 

Figure 3

 

Background scene and target of Experiment 2. The target (gray square) moved at constant acceleration between two opposite black squares. Motion direction was downward, upward, rightward, or leftward in different blocks of trials. The red dot in the center was the fixation point. The image depicted here is from a downward motion trial.

Methods

Participants

Seven naive subjects participated in this study (4 females and 3 males, 26 ± 7 years old, mean ± SD). None of them had previously participated in Experiment 1. All were right-handed, except an ambidextrous subject. 

Apparatus and stimuli

The apparatus was identical to that described for Experiment 1. Visual stimuli were generated with NBS Presentation and were defined in 2D coordinates. We displayed a colored scene (35° by 26°) with a central red dot (0.38°) as fixation point and four black squares (1° side) as the start/end points for target motion. The squares were placed at the same angular distance (20°, ±10° around fixation) between each other as the four holes of Experiment 1. The target was a gray square (0.8°). Luminance and RGB color coordinates of the background matched the respective values averaged over the rear wall of the room of Experiment 1 (see above). Similarly, all three color coordinates of the target square were set to the mean value (80) of the corresponding coordinates of the soccer ball of Experiment 1. 

During each trial, the target accelerated (33° s⁻²) along a straight path between two opposite landmarks, as in Experiment 1. The direction of target motion was downward, upward, rightward, or leftward, depending on the specific block of trials (order counterbalanced across subjects). 

Procedures and data analysis

They were identical to those described for Experiment 1. 

Results and discussion

As in the previous experiment, the population PSE values did not differ significantly between the four motion directions nor did they differ significantly from the reference value of the standard (see Figure 4a). On average, the PSE values were 800 ± 36 ms (mean ± SD) for downward, 802 ± 52 ms for upward, 800 ± 36 ms for leftward, and 802 ± 50 ms for rightward. 

Figure 4

 

Results from Experiment 2 (same format as in Figure 2). (a) Psychometric functions for downward motion (blue) and upward motion (red). Data were pooled over all participants (n = 7). (b) For each motion direction, the precision of discrimination was assessed as the normalized slope of the GLMM population response; *** corresponds to a difference with p < 0.001.

There was no significant effect of block order on the average slope of the GLMM population responses (z = 0.253, p = 0.80). The slope for downward motion was significantly higher than that for upward motion (z = 3.834, p < 0.001, see Figure 4b), but it was not significantly different from that for rightward (z = 1.387, p = 0.22) or leftward (z = 0.090, p = 0.93) motion. Thus, when pictorial cues were removed from the scene, average discrimination performance for downward motion remained superior (by 19%) to that for upward motion, but it lost the superiority over horizontal motions. This trend was observed in the psychometric functions of most (6/7) participants. The average WFs were 0.095 ± 0.003 (mean ± SD over all subjects) and 0.108 ± 0.004 for downward and upward motions, respectively. 

In the first two experiments, the vertical of the scene was roughly aligned with several other key references: the retinal vertical meridian, the body midline, and the Earth's gravitational vertical. Therefore, in line of principle, the superior discrimination for downward motion could be related to any one of these references. One may ask whether the differential up/down performance is tied exclusively to the vertical direction or it also applies to oblique directions. To address this question, in the next experiments, we measured time discrimination with targets moving along oblique lines. 

Experiment 3

We used a quasi-uniform background as in the previous experiment, but the target moved along lines tilted by 45° relative to the cardinal axes (Figure 5). 

Figure 5

 

Background scene and target of Experiments 3 and 4. The target (gray square) moved at constant acceleration along oblique lines between two opposite black squares. Motion direction was down-and-rightward, down-and-leftward, up-and-rightward, or up-and-leftward in different blocks of trials. The red dot in the center was the fixation point. The image depicted here is from a down-and-leftward motion trial.

Methods

Participants

Seven subjects participated in this study (6 naive subjects plus author A.M.; 4 females and 3 males, 25 ± 6 years old, mean ± SD). None of them, except subject A.M., had participated in previous experiments. All were right-handed, except an ambidextrous subject. 

Apparatus and stimuli

The apparatus was identical to that used in Experiments 1 and 2. The background scene, the central fixation dot, and the target were the same as in Experiment 2. The four black squares that served as the start/end points for target motion, however, were placed along virtual lines at 45° relative to the cardinal axes of Experiment 2. The squares were placed at the same angular distance (20°, ±10° around fixation) between each other as the start/end points of Experiments 1 and 2. During each trial, the target accelerated (33° s⁻²) along a straight path between two opposite landmarks. The direction of target motion was down-and-rightward, down-and-leftward, up-and-rightward, or up-and-leftward, depending on the specific block of trials (order counterbalanced across subjects). 

Procedures and data analysis

They were identical to those described for Experiments 1 and 2. 

Results and discussion

The population PSE values did not differ significantly between the four motion directions nor did they differ significantly from the reference value of the standard, although the PSE tended to be slightly smaller than the standard. On average, the PSE values were 780 ± 54 ms (mean ± SD over all subjects) for down-and-leftward, 776 ± 37 ms for up-and-rightward, 774 ± 45 ms for down-and-rightward, and 770 ± 42 ms for up-and-leftward. 

The slope of the GLMM population response did not differ significantly between the two conditions involving a downward motion (down-and-rightward versus down-and-leftward, z = 1.6, p = 0.19) nor did it differ significantly between the two conditions involving an upward motion (up-and-rightward versus up-and-leftward, z = 0.5, p = 0.66). Therefore, we pooled the data for each pair of conditions and found that there was no significant difference of the slope between downward and upward directions (z = 1.7, p = 0.08, Figure 6a). Although the difference was not significant, the slope for downward was slightly higher (by 7%) than that for upward. This superiority was observed in 5/7 participants. The average WF was 0.096 ± 0.008 (mean ± SD over all subjects) for downward motions. 

Figure 6

 

Results from (a) Experiments 3, (b) 4, and (c) 5. Normalized slopes of the GLMM population response for downward and upward motions. In Experiments 3 and 4, the data for down-and-rightward and down-and-leftward were pooled together, as were the data for up-and-rightward and up-and-leftward.

We wondered whether the lack of a significant differential up/down performance when the target moved along oblique axes could depend on the specific protocol employed to test the time estimates. We then designed a different protocol with the aim of increasing the signal-to-noise discrimination. 

Experiment 4

The visual stimuli were identical to those of the previous experiment, but we increased the overall number of repetitions for each condition and presented the stimuli in shorter blocks so as to reduce the time span during which the standard duration had to be held in memory. 

Methods

Participants

Seven subjects participated in this study (6 naive subjects plus author A.M.; 2 females and 5 males, 26 ± 4 years old, mean ± SD). All were right-handed, except a left-handed subject. 

Apparatus and stimuli

They were identical to those used in Experiment 3. 

Procedures and data analysis

There were five sessions in an experiment. Each session consisted of four blocks of trials, each one with a different direction of motion (down-and-leftward, down-and-rightward, up-and-leftward, or up-and-rightward). A block started with the presentation of 60 consecutive standard trials, followed by 90 test trials. Both the standard and the test trials were similar to those of Experiments 1–3. The order of the blocks was balanced across different sessions using a Latin Square design (session 0 = session 4). The order of the sessions was randomized across subjects. Overall, for each motion direction, a given stimulus duration was presented 50 times in each experiment, yielding a total of 450 test trials (9 durations × 50 repetitions). All other design parameters, testing procedures, and data analyses were identical to those described for Experiments 1–3. 

Results and discussion

Using a protocol different from that of the previous experiment, we confirmed the lack of a significant anisotropy between upward and downward motions along oblique axes. The slope of the GLMM population response did not differ significantly between the two downward conditions (down-and-rightward versus down-and-leftward, z = 0.9, p = 0.45) nor did it differ significantly between the two upward conditions (up-and-rightward versus up-and-leftward, z = 1.7, p = 0.14). Critically, there was no significant difference of the slope between downward and upward conditions (z = 1.47, p = 0.14, Figure 6b), although the slope for downward was slightly higher (by 5%) than that for upward as in the previous experiment. 

Likewise, in agreement with the previous experiment, the population PSE values were slightly smaller than the standard. For all four motion directions, this difference was significant (the 95% confidence interval of the distribution of the PSE did not include the duration of the standard). On average, the PSE values were 773 ± 28 ms in down-and-leftward sessions, 759 ± 23 ms in up-and-rightward sessions, 759 ± 37 ms in up-and-leftward session, and 771 ± 23 ms in down-and-rightward session. 

Experiment 5

To assess the role of cues about egocentric orientation in space, we tilted the observer by 45° relative to the Earth's vertical. Visual stimuli were identical to those of Experiment 1 (pictorial experiment), but we tested only downward and upward directions of motion to reduce the duration of experiments that involved a slightly uncomfortable position for the participants. 

Methods

Participants

Seven subjects participated in this study (6 naive subjects plus author A.M.; 3 females and 4 males, 28 ± 4 years old, mean ± SD). All were right-handed, except a left-handed subject. Four of them had participated in previous experiments (the author A.M. plus subjects G.C., M.Z., and A.Z. who had been involved in Experiment 4). 

Apparatus and stimuli

The participants were lying down on their left side on a recliner placed 0.6 m in front of the computer screen. The backrest and footrest of the chair were tilted by 45° relative to the vertical using a spirit level, and the chair height was adjusted so that subject's eyes were at about the same height as the display midpoint. Otherwise, the apparatus and stimuli were identical to those used in Experiment 1. 

Procedures and data analysis

The general procedures were the same as in Experiment 4. There were five sessions in each experiment with a brief rest between sessions (when participants stood up and stretched). Each session consisted of two blocks of trials, each one with a different direction of motion (either downward or upward). A block started with the presentation of 60 consecutive standard trials, followed by 90 test trials. Both the standard and the test trials were the same as those of Experiment 1. The order of the blocks was counterbalanced across sessions and subjects (either downward first or upward first). Overall, for each motion direction, a given stimulus duration was presented 50 times in each experiment, yielding a total of 450 test trials (9 durations × 50 repetitions). All other design parameters, testing procedures, and data analyses were identical to those described for Experiment 1. 

Results and discussion

The slope of the GLMM population response was significantly higher (by 11%) for downward motion than for upward motion (z = 2.4, p = 0.017, see Figure 6c). This trend was observed in 5/7 subjects. The average WFs were 0.067 ± 0.03 (mean ± SD over all subjects) and 0.074 ± 0.02 for downward and upward motions, respectively. Thus, despite the tilt of the observers relative to the picture, the up/down anisotropy in the responses was still present, although the size of the effect was about one half that found in the canonical upright position (Experiment 1). 

The population PSE values did not differ significantly (at the 95% confidence interval) between the two directions of motion nor did they differ significantly from the reference value of the standard duration. On average, the PSE values were 795 ± 27 (mean ± SD over all subjects) for downward and 794 ± 27 for upward. 

Experiment 6

The experiments reported so far showed that the time discrimination for downward motion was better than that for upward motion along the physical vertical, both with and without pictorial cues. Without pictorial cues, no statistically significant anisotropy between downward and upward motions was detectable along oblique axes, although the discrimination for downward directions was slightly better than that for upward directions. The question remains as to whether a pictorial vertical built in to the scene can contribute to time discrimination even when it is tilted relative to the retinal, body, and Earth's vertical. To address this issue, we rotated the monitor and the scene by 45° while keeping the observer upright (Figure 7). 

Figure 7

 

Background scene and target of Experiment 6. The visual scene was identical to that of Experiment 1, but the monitor was rotated by 45° relative to the Earth's vertical.

Methods

Participants

Seven subjects participated in this study (6 naive subjects plus author A.M.; 1 female and 6 males, 25 ± 3 years old, mean ± SD). All were right-handed, except for a left-handed subject. Three of them had participated in previous experiments (the author A.M. plus subjects G.C. and M.Z. who had been involved in Experiments 4 and 5). 

Apparatus and stimuli

Subjects binocularly viewed a monitor (ViewSonic VP2030b, 20″ LCD monitor, 1024 × 768 pixels, 60-Hz refresh rate, controlled by HP xw4600 with nVidia GeForce 8800 GTX graphics card) that was tilted by 45° relative to the Earth's vertical using a spirit level. Otherwise, the visual scene and stimuli were identical to those of Experiment 1, so that the target could move downward, upward, rightward, or leftward relative to the scene (in the following, denoted as pictorial downward, upward, rightward, or leftward). Here, pictorial downward and upward correspond to down-and-leftward and up-and-rightward relative to the Earth's vertical, respectively. 

Procedures

The protocol was identical to that of Experiment 4. All other design parameters, testing procedures, and data analyses were identical to those described for Experiments 1–4. 

Results and discussion

The slope of the GLMM population response for pictorial downward motion was higher than that for the other directions (Figure 8). The difference was statistically significant for the comparison with pictorial upward (z = 3.2, p = 0.002), leftward (z = 3.08, p = 0.002), and rightward (z = 4.1, p < 0.001). On average, the slope for pictorial downward was 14, 14, and 18% higher than the slope for upward, leftward, and rightward, respectively. The slope associated with downward motion was the highest among all other conditions in 5/7 tested subjects. Average WFs were 0.065 ± 0.02 (mean ± SD, over all subjects), 0.076 ± 0.029, 0.077 ± 0.038, and 0.079 ± 0.031 for downward, upward, leftward, and rightward motions, respectively. 

Figure 8

 

Results from Experiment 6 (same format as in Figures 2 and 4). (a) Psychometric functions for downward (blue) and upward (red) directions relative to the scene. Data were pooled over all participants (n = 7). (b) For each motion direction, the precision of discrimination was assessed as the normalized slope of the GLMM population response; *** and ** correspond to a difference with p < 0.001 and p < 0.01, respectively.

On average, the PSE values were 771 ± 30 ms (mean ± SD) for downward, 762 ± 37 ms for upward, 768 ± 42 ms for leftward, and 771 ± 35 ms for rightward, indicating fairly accurate estimates of stimulus duration. 

In sum, the discrimination of target motion duration was better in a vertical downward direction than in the opposite direction even when the vertical direction defined by pictorial cues was tilted by 45° relative to the retinal, body, and Earth's vertical. 

General discussion

Different models of time perception have been proposed, from the classical “counter” model in which internal pulses are collected during the presence of a stimulus and subsequently integrated at an accumulation stage, to more recent models according to which the passage of time can be encoded in the evolving patterns of activity in neural networks (see review in Eagleman, 2008). The gist of the work we presented here is that, whatever the specific implementation, the brain may calibrate its estimation of motion duration by comparing the predictions of an internal model of gravity effects with visual measurements of target kinematics. Before we discuss this hypothesis further, we first review the main results and their implications. 

Motion perception anisotropies

Six experiments addressed the issue of whether the perceptual judgment of motion duration depends on motion direction. In each experiment, a target accelerated over a fixed length path in different directions of a fronto-parallel plane. Overall, we found systematic anisotropies in the precision of the responses, average performance being better in the downward direction relative to the scene than in the upward direction in all experiments. The difference in precision between downward and upward motions was statistically significant in all experiments except Experiments 3 and 4. Moreover, the discrimination in the downward direction was significantly better than that in the orthogonal (leftward and rightward) directions with pictorial cues. 

The present results imply that discrimination of motion duration is enhanced when target motion complies with gravity constraints (as in the downward trajectories) than when it artificially breaks such constraints (as in the other tested trajectories). The increment in precision brought about by the compliance with gravity constraints was not very large (about 10–20% depending on the conditions), but one should consider that even the pictorial scenes provided only approximate size/distance cues and lacked an immersive character. It is reasonable to expect that the precision would be greater in the presence of more realistic cues to assess gravity effects on target motion (Zago et al., 2004). 

Because stimulus duration here covaried with average speed, in line of principle subjects could rely on either time or speed estimates to provide a response. However, direction anisotropies for the judgment of motion duration in our experiments cannot be simply reduced to direction-dependent differences in processing arbitrary speeds and accelerations. In fact, Matthews and Qian (1999) found that speed discrimination of moving random dots is isotropic across the cardinal directions (up, down, left, and right) and the corresponding 45° oblique directions. Moreover, the detection threshold of acceleration of a single target does not differ significantly between a target moving downward and one moving upward nor does it differ systematically between vertical and horizontal directions of motion (Calderone & Kaiser, 1989; Chang & Troje, 2009), in sharp contrast with the present perceptual anisotropies. On the other hand, anisotropies have previously been described for visual motion processing and generally involve a lower discrimination threshold of motion direction along cardinal than oblique axes (Ball & Sekuler, 1987; Gros et al., 1998; Matthews & Qian, 1999). Critically, however, no systematic differences between downward and upward (or between leftward and rightward) directions of motion have been reported. At fast speeds, centripetal hemifield motion has been shown to be associated with lower discrimination thresholds than centrifugal motion (Giaschi et al., 2007). In our experiments, all motion trajectories traversed the central fixation point, so that they were first centripetal and then centrifugal. Notice further that the previously described anisotropy in motion direction discrimination between cardinal and oblique axes is strictly tied to retinal coordinates (Gros et al., 1998). By contrast, the anisotropy in motion duration discrimination we described is not necessarily tied to retinal coordinates, as shown by the results of Experiments 5 and 6 where downward motion in the pictorial scene produced oblique motion on the retina. 

The experimental protocols we used were aimed at investigating differences in duration judgments as a function of motion direction rather than determining the specific strategy used by observers to estimate durations. It is likely that, given the opportunity, all available cues would be combined to estimate motion duration, including temporal frequency and speed (Eagleman, 2008). 

Egocentric, gravicentric, and visual reference cues

The difference in precision between downward and upward motions was not constant across experiments but varied in a graded manner as a function of the conditions, being highest when both the observer and the pictorial scene were upright (Experiment 1) and lowest when the target direction in the non-pictorial scene was tilted by 45° relative to an upright observer (Experiments 3 and 4). To model this graded behavior, we used a linear combination of the 3 types of cues experimentally manipulated: pictorial cues (P), orientation of the observer (O), and orientation of target motion (T) relative to the physical vertical. Accordingly, β _p(Upward)/β _p(Downward) (the ratio of the slopes of upward and downward psychometric functions) was modeled as 

$ratio=a+b·O+c·T+d·P,$

(6)

where O, T, and P could take the value of 1 or 0 depending on whether or not they corresponded to the default in a given experiment (e.g., O = T = P = 0 in Experiment 1). The regression parameters (a–d) were obtained by fitting the data of all experiments together (except Experiment 4 that was redundant with Experiment 3). The resulting weighing coefficients were 43, 37, and 20% (of the overall response) for O, T, and P, respectively. The results changed very little if Experiment 3 (involving a non-significant down/up difference) was excluded from the analysis. 

The observation that egocentric cues specifying the observer's orientation (O) dominate is in line with much previous work on the perceptual discrimination of scenes, people, and actions (e.g., Chang et al., 2010; Kushiro et al., 2007; Troje, 2003). On the other hand, the substantial contribution of visual references intrinsic to the scene, such as the direction of target motion (T) and the presence of additional pictorial cues (P), agrees with the previous observation that viewing a photograph with strong polarization cues, which indicate relative “up” and “down” directions in the picture, can alter the perceived direction of absolute “up” and “down” directions in the real world (Jenkin et al., 2004). 

We hypothesize that spatial representations for computing time are flexible and may be anchored to a variety of different egocentric and allocentric references. A similar viewpoint has recently emerged from studies on adaptation-based duration compression, showing that visual event timers may remain anchored to retinal coordinates (Bruno, Ayhan, & Johnston, 2010) or may exhibit a genuine spatial tuning in external space (Burr, Cicchini, Arrighi, & Morrone, 2011). In fact, adaptation to high temporal frequency induces spatially specific reductions in the apparent duration of sub-second intervals containing medium frequency drift or flicker. 

A neural estimate of gravity direction in a world-centered frame of reference might be computed by the central nervous system as a Bayesian weighted average of multi-cue information, including vestibular, visual, neck, and truncal signals, plus a prior distribution about head and body orientations (De Vrijer et al., 2008; MacNeilage et al., 2007; Van Beuzekom & Van Gisbergen, 2000; Zupan et al., 2002). Interestingly, vestibular stimulation can impair the production of time intervals (Capelli & Israel, 2007). In addition, the ability to produce regularly timed intervals is partially impaired in weightlessness (Semjen, Leone, & Lipshits, 1998). 

On the other hand, it is known that an implicit expectation of the effects of Earth's gravity can affect the behavior independently of sensory cues to gravity, as shown by the observation that astronauts continue to anticipate gravity effects when catching a ball descending along a visual vertical in weightlessness (McIntyre, Zago, Berthoz, & Lacquaniti, 2001). The present results showed that time discrimination was enhanced both when the down direction defined by pictorial cues in the scene reference frame was aligned with physical gravity and when it was tilted by 45°. This implies that observers could represent the effects of a remote gravity with respect to a vertical direction concordant with the visual scene orientation and discordant with the direction of local physical gravity. 

The down/up anisotropy observed here for time discrimination may represent a perceptual counterpart of an asymmetry of motor responses previously reported (Senot, Zago, Lacquaniti, & McIntyre, 2005). The manual interception of a ball accelerating at 9.81 m s⁻² toward the subject, presented stereoscopically in an immersive virtual environment, was more accurate when the ball was launched downward from above (obeying gravity) than when it was launched upward from below (violating gravity). In addition, the observation that the down/up anisotropy was enhanced by embedding motion in a pictorial scene nicely parallels the previous observation that a similar pictorial context facilitated interception of gravitational acceleration over the interception of an unnatural acceleration, whereas a blank scene reduced such bias (Miller et al., 2008). 

Candidate neural mechanisms

The present results are compatible with the idea that the brain calibrates its time estimation by comparing the predictions of a forward model of the object's kinematics with the visual feedback (Eagleman, 2004). In particular, we surmise that, for motion consistent with gravity, elapsed time might be computed by means of an internal model of gravity effects. From elementary physics, we know that the time duration D = t ₂ − t ₁ of a vertical fall, subject to gravity, is given by 

$D=\frac{v_2-v_1}{g},$

(7)

where v ₁ and v ₂ are the speed at time t ₁ and t ₂, respectively, and g is the gravitational acceleration. Whereas speed estimates might be derived from visual motion processing, the value of gravitational acceleration could be internalized either in exact or approximate form (see Zago, McIntyre, Senot, & Lacquaniti, 2008). 

One possible mechanism is that the internal model generates neural inputs temporally coincident with those arising from the visual motion inputs, evoking an increase in spike precision and supralinear spike summation (Cardin, Kumbhani, Contreras, & Palmer, 2010). In turn, this would result in enhanced perceptual acuity for temporal discrimination of a target motion that is consistent with gravity effects. 

The neural bases of the described anisotropy, then, may arise at the processing stage at which the analysis of visual motion (direction and speed) has been combined with the internal model of gravity. A candidate network is given by a set of (possibly interconnected) cortical regions: the putative human homolog of middle temporal area (hMT+), parieto-insular vestibular cortex (PIVC), and posterior parietal cortex (PPC). All three regions are involved in processing visual motion and encoding time, but they play different roles vis-à-vis gravitational stimuli. Thus, while neural populations in hMT+ can feed direction and speed information for each motion direction into downstream regions (Born & Bradley, 2005), neural populations of PIVC appear to be tuned preferentially to object motion related to gravity. Indeed, functional magnetic resonance (fMRI) studies (Indovina et al., 2005; Maffei, Macaluso, Indovina, Orban, & Lacquaniti, 2010; Miller et al., 2008) showed activation of hMT+ with object motion either coherent with or violating gravity in a distant scene. Instead, PIVC was activated selectively by motion coherent with gravity. Conversely, transcranial magnetic stimulation (TMS) of hMT+ affected the interception timing for both gravity-coherent and gravity-incoherent target motions in the vertical or horizontal direction, whereas TMS of TPJ affected only the interception timing for vertical motion coherent with gravity (Bosco, Carrozzo, & Lacquaniti, 2008). Internal time signals may arise directly in visual motion regions through the modulation of local horizontal connections, resulting in trailing inhibition left behind by the moving object (Sundberg, Fallah, & Reynolds, 2006). 

In monkeys, MT/MST feed visual information into PPC regions, such as lateral intraparietal (LIP) area and 7a, which in turn are interconnected with PIVC. Direct correlates of elapsed time in the sub-second range have been found in PPC, where populations of neurons exhibit ramping activities whose slope tightly correlates with the perceived duration in a time discrimination task (Leon & Shadlen, 2003) or with motor response timing in an interception task (Merchant, Battaglia-Mayer, & Georgopoulos, 2004). The slope is probably shaped by spatiotemporal integration of excitatory and inhibitory inputs related to visual motion, motor intention, and high-order contextual signals. We conjecture that neural attributes of the internal model of gravity fed by PIVC may affect both the slope and its temporal variability, thereby contributing to the internal time estimates. 

Conclusions

The main timekeepers used by mankind for centuries—the water clock and the pendulum clock—relied on gravity. Here, we considered the hypothesis that the brain evolved neural mechanisms that exploit gravity effects to estimate time. Our experiments showed that the temporal resolution of visual processing is enhanced for stimuli compatible with expected gravity constraints, as compared with stimuli violating such constraints. This adds to previous evidence that information about physics (such as frictional interaction or momentum) is embedded within the neural mechanisms processing visual motion (Freyd & Finke, 1984; Gilroy & Blake, 2004). 

Supplementary Materials

Supplementary Movie - Supplementary Movie 

Acknowledgments

We thank Dr. Maura Mezzetti for statistical advice on GLMM and Dr. Myrka Zago for help with the computer graphics setup. This work was supported by the Italian Health Ministry, Italian University Ministry (PRIN project), and Italian Space Agency (CRUSOE Grant). 

Some observations of this study will be presented at the 2011 VSS Annual Meeting in Naples, Florida (The Weight of Time: Implied Gravitational Force Enhances Discrimination of Visual Motion Duration). 

Commercial relationships: none. 

Corresponding author: Alessandro Moscatelli. 

Email: [e-mail]. 

Address: Via Ardetina 354, Rome 00179, Italy.