Corrective mechanisms of motion extrapolation

Abstract

Transmission and processing of sensory information in the visual system takes time. For motion perception, our brain can overcome this intrinsic neural delay through extrapolation mechanisms and accurately predict the current position of a continuously moving object. But how does the system behave when the motion abruptly changes and the prediction becomes wrong? Here we address this question by studying the perceived position of a moving object with various abrupt motion changes by human observers. We developed a task in which a bar is monotonously moving horizontally, and then motion suddenly stops, reverses, or disappears-then-reverses around two vertical stationary reference lines. Our results showed that participants overestimated the position of the stopping bar but did not perceive an overshoot in the motion reversal condition. When a temporal gap was added at the reverse point, the perceptual overshoot of the end point scaled with the gap durations. Our model suggests that the overestimation of the object position when it disappears is not linear as a function of its speeds but gradually fades out. These results can thus be reconciled in a single process where there is an interplay of the cortical motion prediction mechanisms and the late sensory transient visual inputs.

Introduction

Transmission of visual information from retina to visual cortex takes time. Neural processing of visual information takes time as well. This means the position of a time-varying object, such as a moving stimulus, is outdated when it reaches the brain (Bullier, 2001). Because the object continues moving in the physical world, it should cause a disparity between the perceived position and the actual physical position. In actuality, we are able to accurately perceive and interact with moving objects like, for instance, when catching a ball, suggesting that there are compensatory mechanisms for these neural delays in our visual system. One such strategy is through motion extrapolation: Our visual system can use the past trajectory of a moving object to predict its current position (Nijhawan, 1994; Hogendoorn, 2020; Nijhawan, 2008). 

Strong evidence for motion extrapolation mechanisms has been made on the basis of a wide range of motion-induced illusions. The most-studied illusion is probably the flash-lag effect, in which a flashed stimulus is perceived as lagging behind a moving object when the two objects are physically aligned (Nijhawan, 1994; Hogendoorn, 2020). By using this illusion, many studies investigated how the visual system extrapolates continuous motion (Nijhawan, 2002; Wojtach, Sung, Truong, & Purves, 2008; Maus, Ward, Nijhawan, & Whitney, 2013; Hubbard, 2014; Subramaniyan et al., 2018) and determined which parameters can modulate the motion extrapolation mechanism. For example, faster motion speed (Nijhawan, 1994; Lee, Khuu, Li, & Hayes, 2008; Wojtach et al., 2008) and lower contrast of flashed objects (Ogmen, Patel, Bedell, & Camuz, 2004; Wang, Reynaud, & Hess, 2021) induces a more manifest flash-lag effect, which suggests an ampler motion extrapolation. 

A few investigations focused on how the brain computes abrupt motion alterations, such as changes in direction or disappearance. In the flash-grab illusion, when an object is flashed at the motion reversal point, its position is perceived as displaced towards the perceived location of the trajectory endpoint (Cavanagh & Anstis, 2013). In the flash-terminated condition in which a moving object stops moving at the time of the flash, no flash-lag effect is observed (Kanai, Sheth, & Shimojo, 2004) as well as when the object reverses its direction (Whitney & Murakami, 1998). These observations suggest that the perceived terminal position of a moving object does not overshoot the perceived position of a transient stationery object (see also the trinkle-goes illusion, in which no shift is perceived when two bars move horizontally toward each other and suddenly disappear when they are aligned [Nakayama & Holcombe, 2021]). However, some studies demonstrated that a predictive overshoot does exist when motion suddenly changes in certain conditions, such as low-contrast (Kanai et al., 2004), blur (Fu, Shen, & Dan, 2001), and the gradual fading of the moving stimulus (Maus & Nijhawan, 2006). 

In the aforementioned visual illusions, the subjects did not report the endpoint of the moving object in absolute coordinates. Quantification of the amplitude of the extrapolation mechanism was calculated by comparing the position of the moving object relative to a nearby flash (i.e., a transient dynamic reference). However, the motion signals could indeed distort the visual space of the transient flash (Murai & Murakami, 2016), causing the perceived position of the flash to shift in the direction of motion (Whitney & Cavanagh, 2000; Watanabe, Nijhawan, & Shimojo, 2002; Cavanagh & Anstis, 2013). In the current study, we wanted to assess the extrapolation mechanism in the visual system per se, compared to a static reference, when the motion suddenly stops or reverses. Therefore we developed a task in which a bar is monotonously moving horizontally and then disappears or reverses suddenly. In this task, the traditional briefly flashed object is replaced by two constant reference lines. This design enables us to directly measure whether there is a perceptual overshoot at the final position. In Experiments 1 and 2, we explore whether experimental parameters such as speed, contrast, and luminance affect the perception of the position of a bar that is either disappearing (motion stop) or reversing its direction (motion reverse). In Experiment 3, we further investigate whether an overshoot in the position of the stimulus is perceived when these two conditions are combined, that is to say, when the bar disappears and then, after a variable delay, reverses its motion (motion gap). 

Experiments 1 and 2: Motion stop and reverse conditions

Methods

Participants

Twelve adults (average age: 25.7 years old; range, 21–32 years old; seven females) with normal or corrected-to-normal vision participated in Experiments 1 and 2. All subjects had no history of any eye disease or surgery. Participants performed the experiments with their best optical correction when needed. Written informed consent was obtained from each participant. The study follows the tenets of the Declaration of Helsinki and was approved by the Ethics Committee of West China Hospital of Sichuan University. 

Apparatus

The stimuli were generated by Matlab R2018b (the MathWorks) using the PsychToolBox extensions 3.0.9 (Kleiner et al., 2007) on a MacBook Pro computer. All stimuli were displayed on a gamma-corrected CRT monitor (Sony Sun GDM-5510, 21 in.; Sony, Tokyo, Japan). The resolution of the monitor was 1280 × 1024 px, and the refresh rate was 100 Hz. The maximal luminance of the monitor was 72 cd/m². During the test, participants were placed in a dimly lit room and kept a constant distance of 57 cm from the screen. Participants viewed the stimuli monocularly with their left eye, wearing a dark opaque patch over their right eye. 

Stimuli and procedure

Stimuli were presented on a gray background. Two vertical black reference lines were presented at a fixed position of 3° from the vertical meridian throughout the experiment. The vertical distance between the nearest edges of the lines was 4.4°. On a horizontal path between these two lines, a bar (4° × 1°) moved from left to right, toward the fixation point and disappeared (stop condition) or reversed (reverse condition) at a given azimuth varied within 11 values (−1.5, −0.9, −0.6, −0.3, −0.15, 0, 0.15, 0.3, 0.6, 0.9, 1.5°) from the reference lines. Negative values indicate that the moving bar disappeared/reversed ahead (to the left) of the reference lines whereas positive values indicate that the bar disappeared/reversed behind (to the right) the reference lines. Each azimuth was tested for 10 repetitions per block. The stimuli were always presented in the left hemifield, which is known to produce the largest motion-induced illusion (Kanai et al., 2004; Suzuki, Atmaca, & Laeng, 2023). 

Several physical parameters of the stimuli were tested: (1) Contrast: The contrast of the bar was varied between −1, 0.2, and 1 at a fixed speed of 18°/s. (2) Speed: The speed of the moving bar was varied within 9°/sec, 18°/sec, and 36°/sec with a fixed contrast of 1. (3) Luminance: The global luminance seen by the eye was diminished using a neutral density (ND) filter of intensity: 0ND (no filter), 1ND, or 2ND, with a fixed contrast of 1 and a speed of 18°/sec. In summary, there was a total of seven conditions in both the motion stop and reverse experiments. Each condition was tested twice (two blocks of 10 repetitions). 

Experiment 1: Motion stop condition

In Experiment 1, the subjects were tested with the motion stop task in which a bar moved horizontally toward the fixation point before suddenly disappearing at an azimuth close to the two vertical black reference lines (Figure 1). In each trial, the subject was asked to stare at the orange fixation point. After the stimulus disappeared, the subject judged whether the moving bar (the right edge of the bar) disappeared ahead of or behind the reference lines and indicated their decision using a keyboard. The order of test conditions (contrast, speed, and luminance) was randomized. 

Figure 1.

 

Illustration of the motion stop condition. A bar moves horizontally from left to right, toward the fixation point, and suddenly disappears at an azimuth close to the two vertical black reference lines.

Experiment 2: Motion reverse condition

In Experiment 2, we tested the subjects with the motion reverse task in which a bar moving from left to right suddenly reversed its direction at an azimuth close to the reference lines (Figure 2). In each trial, after the stimulus disappeared, the subject was asked to judge whether the moving bar (the right edge of the bar) reversed ahead or behind of the reference lines. The test order of different parameters was randomized. 

Figure 2.

 

Illustration of the motion reverse condition. A bar moves horizontally from left to right toward the fixation point and suddenly reverses its direction at an azimuth close to the two vertical black reference lines.

Data analysis

Data was analyzed with Matlab R2018b (MathWorks, Inc., Natick, MA, USA). Participants’ psychometric functions describing the proportion of “behind” response at each spatial point were fitted with a logistic function forced between 0 and 1. The estimated midpoint of the logistic function was defined as the point of subjective equality (PSE), the point at which a subject gives 50% “ahead” and 50% “behind” responses, which indicates the perceived alignment of the moving bar and the reference lines. Significant PSE shifts from zero characterize the magnitude of motion-induced shifts. 

Results

Experiment 1: Motion stop

We present the psychometric functions of one representative subject under different contrast (Figure 3A), speed (Figure 3B), and luminance (Figure 3C) conditions in the motion stop experiment, fitted with logistic functions. For all subjects, the average coefficient of determination R² is 0.984 ± 0.002 (mean ± standard error), indicating that the logistic function fits are accurate. The estimated PSEs are significantly negative in all conditions of the motion stop condition, with an average value of −0.24 ± 0.03° (p ≤ 0.023, one-tail, Figures 3D through 3F), which indicates that participants overestimated the position of the stopping bar by approximately 0.24° in all tested conditions. 

Figure 3.

 

Psychometric functions of one representative subject in the motion stop condition under different contrast (A), speed (B), and luminance (C) conditions. Boxplots of the PSEs of all subjects in the motion stop condition under different contrast (D), speed (E), and luminance (F) conditions. The colored solid line within each box represents the median. The colored square with a black outline represents the mean PSE of each condition. The box represents the interquartile range (IQR) of the data (25% to 75%). The whiskers represent 1.5 × IQR either above the third quartile or below the first quartile. *p < 0.05; # indicates significant PSE shift from 0.

We analyzed the estimated PSEs of the psychometric functions of the motion stop conditions (Figures 3D through 3F). The main effect of contrast was not significant (F_{1.2, 13.4} = 2.18, p = 0.162) (Figure 3D). However, we did observe that the PSE was less negative in the contrast −1 condition, which suggests that a smaller overshoot was perceived with a black bar. We found a significant main effect of speed (F_{2, 22} = 4.41, p = 0.025) (Figure 3E). The post hoc analysis confirms that the PSE of the low speed (9°/sec) presents a significantly smaller PSE magnitude (−0.19° ± 0.05°) compared to the moderate (18°/s, −0.28° ± 0.05°) (p = 0.023) and high (36°/sec, −0.32° ± 0.07°) (p = 0.012) speed conditions. We did not find a significant difference in PSE between the different luminance conditions (F_{1.2, 13.3} = 1.56, p = 0.239) (Figure 3F). 

Experiment 2: Motion reverse

The psychometric functions of one representative subject and the estimated PSEs of all participants under the motion reverse condition are plotted in Figure 4. Logistic functions fits are again accurate across all conditions (R², 0.983 ± 0.002). Actually, PSEs were not different from 0 in any of the contrast or luminance conditions or in the two slowest speed conditions. The only PSE significantly different from 0 was found in the high-speed condition with a value of −0.18° ± 0.08° (Figure 4E, p = 0.024, one-tail). 

Figure 4.

 

Psychometric functions of one representative subject in the motion reverse condition under different contrast (A), speed (B), and luminance (C) conditions. Boxplots of the PSEs in the motion reverse condition under different contrast (D), speed (E), and luminance (F) conditions. he colored solid line within each box represents the median. The colored square with a black outline represents the mean PSE of each condition. The box represents the interquartile range (IQR) of the data (25% to 75%). The whiskers represent 1.5 × IQR either above the third quartile or below the first quartile. *p < 0.05; # indicates significant PSE shift from 0.

To compare the mean PSE magnitude between the different conditions in the motion reverse experiment, we performed a one-way analysis of variance with contrast, speed and luminance, respectively. Neither the main effect of contrast (F_{2, 22} = 0.55, p = 0.585) nor luminance (F_{2, 22} = 0.27, p = 0.763) was significant (Figure 4D and 4F). However, we did find a significant main effect of speed (F_{2, 22} = 5.78, p = 0.01) (Figure 4E). The results of our post hoc analysis show that the PSE at high speed (−0.18° ± 0.08°) was larger than the other two speeds (for both, p = 0.023); so the faster the speed, the larger the overshoot. 

Motion stop versus motion reverse

The mean PSEs of motion stop and reverse across all test conditions were −0.24° ± 0.03° and −0.07° ± 0.19°, respectively. To properly compare whether there is a perceptual difference between the motion stop and reverse conditions, we performed within-subject repeated measures analysis of variance tests. We found that the difference in mean PSE between motion stop and reverse was significant for all contrast (F_1,11 = 14.8, p = 0.003), speed (F_1,11 = 8.44, p = 0.014), and luminance (F_1,11 = 12.805, p = 0.005) conditions. The post hoc comparisons showed that (1) the mean PSE of motion stop was more negative than that of motion reverse across all the contrast conditions (Figure 5A, p ≤ 0.034); (2) for the speed conditions, subjects showed a larger effect in the motion stop experiment for both low and moderate speeds (Figure 5B, p ≤ 0.08), but not for the high speed (p = 0.136); and (3) motion stop showed lower mean values of PSE under 0 ND (with ND filter) and 2 ND conditions (Figure 5C, p ≤ 0.12). These results indicate that there is a larger overshoot in the perception of the position of a moving object at its final position when it abruptly stops compared to when it reverses. 

Figure 5.

 

Comparison of the mean PSE magnitude of motion stop and reverse under different contrast (A), speed (B), and luminance (C) conditions. Open and filled dots represent the results of motion stop and reverse respectively. Error bars represent standard error. *p < 0.05; **p < 0.01.

Interim discussion

In Experiments 1 and 2, we investigated how the brain computes abrupt motion changes (stops and reversals). We found that the final perceived position of a moving bar was in front of its physical position when it abruptly disappears (i.e., a perceptual overshoot). Additionally, the magnitude of the overshoot increased with speed. These findings suggest that our brains over-extrapolate a moving signal when it suddenly disappears. However, there was no perceptual overshoot in the motion reverse experiment, except at a high speed. It is likely that a moving object with an abrupt trajectory change, such as a direction reversal, would trigger a strong correction for extrapolation, causing an absence of an overshoot (Cavanagh & Anstis, 2013; Blom, Liang, & Hogendoorn, 2019; Blom, Bode, & Hogendoorn, 2021). Also, both pre- and post-reverse motion signals contribute to the perception of the motion reversal position (Blom et al., 2019; Takao, Sarodo, Anstis, Watanabe, & Cavanagh, 2022). Thus our results raise an interesting question: What if there was a temporal gap between the pre- and post- reverse motions? Would we perceive an overshoot in a moving stimulus position if it temporarily disappeared before reversing its direction? To address these questions, we ran a follow-up experiment in which the moving bar temporarily disappeared before switching direction. 

Experiment 3: Motion gap reverse condition

Methods

Participants

Eight participants (average age: 27.4 years old; range, 23–39 years old; three females) were tested in this experiment. All participants had no history of any eye disease or surgery. They were all naïve to the purpose of the experiment. Participants performed the experiment with their best optical correction if needed. The study followed the tenets of the Declaration of Helsinki and was approved by the Ethics Review Board of the Research Institute of the McGill University Health Center. 

Apparatus

Stimuli were generated by a Mac Computer (OSX, 10.10.5) and displayed on a CRT monitor (Iiyama MA203DTD, 19.5 inch; Iiyama, Tokyo, Japan). The display had a resolution of 1280 × 1024 px with a refresh rate of 100 HZ. The maximal luminance of the monitor was 82 cd/m². Participants viewed the screen through their left eye at a viewing distance of 57 cm. 

Stimuli and procedure

Figure 6 illustrates the stimulus used in the Experiment 3. A bar moved from left to right before disappearing at a position close to the black reference lines. After a temporal gap, the bar reappeared at the same location but with a reversed motion direction. The gap duration between the pre- and post-reverse motion was varied within 0, 10, 30, and 100 ms (note that the 0 gap is equivalent to the motion-reverse condition in Experiment 2). We tested four different speeds with a fixed contrast of 1 in this experiment: 9°/sec, 18°/sec, 36°/sec, and 54°/sec. Altogether, there was a total of 16 conditions (4 gaps × 4 speeds). Each condition was tested twice (two blocks of 10 repetitions). The order of the test conditions was randomized. As in Experiments 1 and 2, the azimuth where the bar disappeared and reversed was varied within 11 values (−1.5°, −0.9°, −0.6°, −0.3°, −0.15°, 0°, 0.15°, 0.3°, 0.6°, 0.9°, 1.5°) from the reference lines. In each trial, subjects were asked to judge whether the moving bar (the right edge of the bar) reversed ahead or behind of the reference lines (bar disappearance was not mentioned to them; however, most of them noticed it for long gap durations). Each azimuth was tested for 10 repetitions in each block. 

Figure 6.

 

Illustration of the motion gap reverse experiment. A bar moved horizontally from left to right towards the fixation point before disappearing at a position close to the black reference lines. After a temporal gap of 0 to 100 ms, the bar reappeared at the same location but with a reversed motion direction.

Model

In the last section of this article, we introduce a simple model that unifies the data of the three experiments: motion stop, reverse, and gap reverse. Over these three experiments, it globally describes the amplitude of the overshoot as a function of the gap duration and speed. 

\begin{eqnarray}Overshoot{\rm{\ }} = {\rm{\ }}a{\rm{\ }} \times {\rm{\ }}spee{d^{{\beta _s}}}{\rm{\ }} \times {\rm{\ }}ga{p^{{\beta _g}}}\end{eqnarray}

(1)

 

The model (Equation 1) has only 3 free parameters to fit all data: β_s and β_g are compressive exponents of the speed and gap duration respectively; and a is a scaling coefficient. To perform the fitting, the data of motion reverse (Experiment 2) is averaged with the data of gap 0 (Experiment 3). And the data of motion stop (Experiment 1) that would correspond to an infinite gap is assigned as a gap of 1000 ms. Data were averaged across subjects. Fitting was performed with Matlab's function nlinfit. 

Results

Psychometric functions of one representative subject are plotted in Figure 7A. Psychometric functions were fitted with a logistic function to estimate the PSE. The mean coefficient of determination R² for all subjects and conditions was high (0.982 ± 0.003), proving that the fits are accurate. 

Figure 7.

 

Psychometric functions of one representative subject in the motion gap reverse experiment for different gap durations at 36°/sec (A). The mean PSE at different gap durations for different speeds (B). Boxplots of the PSEs in the motion gap reverse condition at different speeds (C–F). The colored solid line within each box represents the median. The colored square dots with a black outline within each box represent the mean PSE of each condition. The box represents the interquartile range (IQR) of the data (25% to 75%). The whiskers represent 1.5 × IQR either above the third quartile or below the first quartile. PSE, point of subjective equality; *p < 0.05; # indicates a significant PSE shift from 0.

We plotted the mean PSE of different speeds under 4 temporal gap durations in Figure 4B. Similar to the findings in Experiment 2, the mean PSE decreases with increasing speed (F_1.3,9.2 = 9.8, p = 0.009). Also, the results show that mean PSE tends to shift towards negative values when there is a gap. To better quantify this observation, we further analyzed the effect of the temporal gap on the mean PSE. First, we found that the PSEs were significantly negative, indicating a perceptual overshoot, with a gap of 30ms and 100ms at 36°/sec (Figure 7E) and 54°/sec (Figure 7F) (p < 0.05, one-tail). Then, the mean PSE scaled with the temporal gap (F_3,21 = 7.61, p = 0.001). The results of post hoc analysis showed that (1) the PSE for a gap of 10, 30, or 100 ms was more negative than for a gap of 0 ms (i.e., equivalent to motion reverse) at speeds of 9°/sec and 18°/sec (p ≤ 0.036, Figures 7C and 7D); (2) at speed of 36°/sec, a gap of 100 ms induced a more negative PSE compared to other gaps (p < 0.05, Figure 7E); (3) there was no significant difference between the different gap durations at the high speed of 54°/sec (p ≥ 0.1, Figure 7F). 

In these conditions, we observed that the perceptual overshoot was both proportional to the duration of the gap and to the speed. Thus, in the motion gap reverse condition, the visual system successively integrates the extrapolation of the pre-reverse motion and the new correction of the post-reverse motion, which both contribute to an intermediate perception between the reverse and stop conditions, proportional to the gap duration. 

To better understand the relationships among the motion stop, reverse, and gap reverse, we reconcile the data of the three experiments by using a simple descriptive model (see Methods). It globally describes the amplitude of the overshoot as a function of the gap duration and speed. The model includes three free parameters: a scaling coefficient and two nonlinear exponents of the speed and gap duration. 

In Figure 8, we concatenate the results of the three experiments: motion reverse, stop, and gap reverse experiments, where the gap durations of all three experiments are combined on the x-axis: the data of motion reverse (Experiment 2) is averaged with the data of gap 0 (Experiment 3). And the data of motion stop (Experiment 1) is indicated as infinite (see Methods). The magnitude of the overshoot is then plotted as a function of the gap in the different speed conditions. The solid lines represent our model fits with coefficient of determination R² = 0.6848. The estimate of the two nonlinearity exponent for speed and gap duration were β_s = 0.44 and β_g = 0.22. The exponent values <1 indicate that those nonlinearities are compressive. The compressive speed exponent reveals that the overestimation of the object position when it disappears is not linear as a function of its speeds but gradually fades out. And the compressive gap duration exponent reveals that the correction mechanism, initiated by the reappearance of the bar, gets relatively faster to intervene as the gap extends. 

Figure 8.

 

Model of the overshoot as a function of the gap duration and speed. Continuous lines represent the model fit (R² = 0.6848) with different shades of blue indicating different speeds. Note that the motion stop data is presented as Infinite gap, but fitted with an assigned value of 1000 ms (see Methods). Data are averaged across all subjects.

Discussion

In the present study, we investigated how our brains accurately compute and perceive the final position of a moving object when its motion abruptly changes. We found that when the object disappeared, the final perceived position was shifted forward from its physical end position—a perceptual overshoot, although no such effect was observed when the object reversed its direction. However, when a temporal gap was added at the reverse point, the perceptual overshoot of the end point scaled with the gap duration. 

It has been reported that the location of the stimuli related to fovea could affect the perception of motion-induced illusions (Kanai et al., 2004; Shi & Nijhawan, 2012). For example, the flash-lag effect was dramatically reduced when the moving and flashed objects were near the fovea (Kanai et al., 2004). Additionally, eye movements may impair illusion perception (Nijhawan, 1997; van Beers, Wolpert, & Haggard, 2001). The smooth pursuit of the moving object could decrease the perceived magnitude of the flash-lag illusion (Nijhawan, 2001). Thus, in our study, we asked the subjects to stare at the fixation dot throughout the experiment to maintain their fixation stability. 

Sensory input takes time to travel through the visual system which may pose a computational challenge as the information available to the brain lags behind its corresponding real-world event. Krekelberg and Lappe proposed an explanation of temporal integration for the flash-lag effect, which is that the visual system collects the motion position signal over a certain period of time and estimates the position based on the integrated signal (Krekelberg & Lappe, 2000). Under this hypothesis, the integrated position of the reference lines in our experiments equals their actual position, because the positions do not change. However, for the moving bar, unless the integration window would also sum predicted signals, the integrated position would be behind of where the bar disappeared, because the integration window would only sum the signals before its disappearance (Krekelberg & Lappe, 2001). This is inconsistent with our finding of a perceptual overshoot in the motion stop condition. 

So, accurately perceiving a moving object typically involves motion extrapolation mechanisms to compensate for the neural delays (Nijhawan, 1994; Nijhawan, 2008; Hogendoorn, 2020; Johnson et al., 2023). In cortical retinotopic maps, an object moving across the visual field triggers a wave of neural activity ahead of its motion path thanks to the horizontal connections between neurons (Jancke, Erlhagen, Schoner, & Dinse, 2004; Jancke, Chavane, Naaman, & Grinvald, 2004; Muller, Reynaud, Chavane, & Destexhe, 2014; Subramaniyan et al., 2018). Computational models have suggested that these traveling waves may continue to travel in the direction of motion even without further sensory input (Erlhagen, 2003; Khoei, Masson, & Perrinet, 2013; Kaplan, Lansner, Masson, & Perrinet, 2013). It has been shown using functional magnetic resonance imaging that these traveling waves can indeed convey a motion signal in unstimulated retinotopic regions of V1 (Muckli, Kohler, Kriegeskorte, & Singer, 2005; Ekman, Kok, & de Lange, 2017). Time-resolved electroencephalographic decoding has also showed that predictive mechanisms can activate sensory-like neural representations of the future position of moving objects, prior to the arrival of the afferent sensory input (Blom, Feuerriegel, Johnson, Bode, & Hogendoorn, 2020). Thus these prediction mechanisms in our visual system may lead to the perception of a forward shift beyond the moving object's actual position of disappearance. 

This area is quite debated. Previous studies based on the flash-terminal condition, when the termination of motion is compared to a transient flashed reference, observed that when the moving object disappears at the time of the flash, it does not perceptually overshoot the point of disappearance (Eagleman & Sejnowski, 2000). However, the motion itself may bias the position of the nearby transient object, causing a shift in its perceived position in the direction of motion (Whitney & Cavanagh, 2000; Watanabe et al., 2002). In the current study, we replaced the transient flash with two fixed reference lines, which should reduce the distortions from motion (Whitney & Cavanagh, 2000). We observed that the perceived final position of the moving bar overshot its physical disappearance point, which suggests that its position was indeed extrapolated by an internal mechanism at the cortical level based on the past trajectory of the moving object. This extrapolation mechanism could lead to the perception of a moving object at unstimulated positions. Indeed, similar results have been found in the case of gradually fading motion termination (Maus & Nijhawan, 2006; Maus, Weigelt, Nijhawan, & Muckli, 2010), motion in the blind spot (Maus & Nijhawan, 2008), in the blue scotoma of the fovea (Shi & Nijhawan, 2012), or during eyeblinks (Maus, Goh, & Lisi, 2020); conditions that are characterized by subthreshold or no stimulation. 

In flash-lag effect related studies, the average forward displacement of the moving object compared to the flash was reported from 0.4° to 2°, which is equivalent to approximately ∼60ms at various moving speeds (Nijhawan, 1994; Wojtach et al., 2008; Maus et al., 2013; Wang et al., 2021). The magnitude of the forward displacement observed in the current study was around 0.2–0.3° (characterizing a temporal forward shift of approximately 10–20 ms) in the motion stop condition (Figure 5, circle symbols), which is quite small compared to the magnitude of motion extrapolation measured by the flash-lag effect (roughly 60 ms) (Nijhawan, 1994; Maus et al., 2013; Wang et al., 2021). This discrepancy of magnitudes between sudden disappearance of motion and continuous motion was also found in a similar study (Maus & Nijhawan, 2009) in which a continuously moving bar was perceived in advance of an abruptly disappearing bar when the two bars were physically aligned. In our case, when a moving object suddenly disappears, bottom-up inputs and predictive mechanisms are in conflict. However, the corrective mechanisms triggered by sensory input may arrive too late to prevent the motion signal from being represented by the visual system (Blom et al., 2020) and will therefore not fully correct the prediction. This leads to a small overextrapolation of the motion signal, smaller than the perceptual position shift observed in the flash-lag effect where no corrective mechanisms are involved (Khoei, Masson, & Perrinet, 2017). 

In contrast, we did not observe an overshoot of the motion reverse or gap 0 condition, consistent with previous studies (Whitney & Murakami, 1998; Cavanagh & Anstis, 2013), which may have even reported an undershoot perception (Sinico, Parovel, Casco, & Anstis, 2009; Cavanagh & Anstis, 2013). The motion reversal is a stronger transient signal compared to the disappearance of the bar. It may therefore trigger a strong correction for the prediction error to help the visual system catch up to the motion trajectory in the new direction (Schwartz, Taylor, Fisher, Harris, & Berry, 2007; Khoei et al., 2017; Blom et al., 2019). To test this hypothesis, we introduced a gap before the motion reversal in Experiment 3. This gap provides a short temporal window for the neural representation of the overextrapolation at the time of motion disappearance before the subsequent sensory corrective input. After the gap, the motion resumes in the opposite direction and generates strong corrective signals (Khoei et al., 2017). Indeed, in these conditions, we observed that the perceptual overshoot was both nonlinearly proportional to the duration of the gap and to the speed as described in our model (Figure 8). Thus the model reveals that the visual system successively integrates the extrapolation of the pre-reverse motion and the new correction of the post-reverse motion. 

The overestimation of the object's position when it disappears is not linear as a function of its speeds but gradually fades out. This observation is consistent with the results of Khoei et al. (2013). This study shows that when a moving object disappears and then resumes its trajectory, there is a rapid drop in perceived speed when the moving object disappears. The speed information would be stored in an infraliminal way, so that when the moving object reappears and is consistent with a linear trajectory, the estimate is resumed almost instantaneously. This degree of memory depends on the duration of the gap, which could correspond to the compressive nonlinearity on the gap duration β_g presented in our model. 

In a second time, our modeling suggests that the correction mechanism, initiated by the reappearance of the bar, gets relatively faster to intervene as the gap extends as characterized by the parameter β_s. Again, this is consistent with the results of Khoei et al. (2017), which suggest that the FLE depends on speed, up to a certain saturation. The compression effect could be due to the existence of a prior that favors low velocities. 

Conclusions

Our results suggest that there is an interplay between the cortical motion prediction mechanisms and the late sensory transient visual inputs when the visual system processes abrupt changes in visual motion. The overestimation of the object position when it disappears gradually fades out. And the correction mechanism, initiated by the reappearance of the bar, gets relatively faster to intervene as the gap extends. The interaction of these two signals and their weights drives an integrated perception of moving objects, allowing us to interact with our dynamic environment. 

Acknowledgments

Supported by the National Natural Science Foundation of China (NSFC 82201233) and Sichuan Science and Technology Program (2023NSFSC1669) to XW as well as a start-up fund from the Research Institute of the McGill University Health Center to AR. The authors thank their participants and Daniel Gurman for language correction. 

Commercial relationships: none. 

Corresponding author: Longqian Liu and Xi Wang. 

Email: [e-mail] and [e-mail]. 

Address: West China Hospital Sichuan University, Sichuan, China.