A dynamic noise background reveals perceptual motion extrapolation: The twinkle-goes illusion

Abstract

We find that on a dynamic noise background, the perceived disappearance location of a moving object is shifted in the direction of motion. This “twinkle-goes” illusion does not require luminance- or chromaticity-based confusability of the object with the background, or on the amount of background motion energy in the same direction as the object motion. This suggests that the illusion is enabled by the dynamic noise masking the offset transients that otherwise accompany an object's disappearance. While these results are consistent with an anticipatory process that pre-activates positions ahead of the object's current position, additional findings suggest an alternative account: a continuation of attentional tracking after the object disappears. First, the shift increased with speed until over 1.2 revolutions per second (rps), nearing the attentional tracking limit. Second, the shift was greatly reduced when attention was divided between two moving objects. Finally, the illusion was associated with a delay in simple reaction time to the disappearance of the object. We propose that in the absence of offset transients, attentional tracking keeps moving for several tens of milliseconds after the target disappearance, and this causes one to hallucinate a moving object at the position of attention.

Introduction

Objects are not always perceived at the location corresponding to the position they stimulate on the retinas. Object motion, for example, can strongly influence perceived object location (motion-induced position shift [MIPS]; De Valois & De Valois, 1991; Ramachandran & Anstis, 1990). A popular theory of such phenomena has been that they reflect anticipatory extrapolation by the brain to compensate for neural delays (Nijhawan, 1994). 

Anticipatory extrapolation predicts that if an object suddenly disappears, its disappearance location should be perceived ahead of where it actually disappeared. But this does not occur (Kerzel, 2000; Whitney, Murakami, & Cavanagh, 2000). To explain this, advocates of the anticipatory extrapolation theory have suggested that the abrupt disappearance and associated luminance transient results in a correction or reset of extrapolation (Hogendoorn, 2020; Nijhawan, 2002, 2008). 

A potentially major concern about the transient correction hypothesis is that it appears to have been created post hoc to preserve the extrapolation theory. However, Nijhawan and colleagues (Maus & Nijhawan, 2008; Shi & Nijhawan, 2012) found novel support for the transient correction hypothesis by assessing the perceived location of objects after they entered the retinal blind spot. The perceived disappearance location of objects moving into the blind spot was beyond the blind spot's proximal edge. This is consistent with the transient correction theory, because entering the blind spot prevents the disappearance of the moving object from causing a luminance transient. Nijhawan et al. concluded that the perceived disappearance location of an object moving into the blind spot reflects anticipatory activation of positions (inside the blind spot) ahead of the object's currently sensed location. That is, throughout the time that the object was presented, it was perceived ahead of its sensed location, and its movement into the blind spot prevented the transient correction process that normally occurs upon object disappearance. 

Here, we find support for an alternative theory of Nijhawan's results and one that also explains a striking new phenomenon of position and motion that we call the “twinkle-goes“ illusion. In our basic demonstration (Supplementary Movie S1), two smoothly moving objects are presented on a static background for one interval, and on a dynamic noise background during a second interval. In the dynamic background portion of the movie, when the moving objects disappear, they appear to be shifted in the direction of motion, whereas on the static background portion they do not. 

Experiment 1 validates the phenomenon and shows that it occurs whenever the dynamic noise is presented during a short interval after the moving object's disappearance. Experiment 2 shows that the illusion occurs even when the moving objects and background are distinguished with different colors, suggesting the illusion is not due to simple similarity of the background to the moving object. Experiment 3 shows that the illusion does not require the presence of motion signals in the background compatible with the motion of the object. These findings suggest that the offset transient that normally accompanies an object's disappearance is what prevents the illusion in ordinary displays – the dynamic noise enables the illusion by masking the offset transient. 

Experiment 4 shows that the size of the twinkle-goes illusion increases approximately linearly with speed, up to a high speed of over 30 deg/s. This suggests a process that shifts the perceived location of the object to where it would be in a certain amount of time, specifically 38 ms (SEM = 11 ms) after disappearance. These results of the experiments so far are all compatible with the anticipatory extrapolation theory of Nijhawan (2008). 

The results of Experiment 5 suggest that attentional tracking (Cavanagh, 1992; Pylyshyn & Storm, 1988) may be critical for the phenomenon. We exploit the rotational speed limit of attentional tracking and, in Experiment 5a, finds evidence that the illusion saturates near the attentional tracking limit. Experiment 5b indicates that when participants were forced to split attention between two displays, the illusion is diminished. 

To explain these findings, we propose a tracking continuation theory of the illusion. Under this theory, attention continues moving in the direction the target was moving for dozens of milliseconds after the object disappears. This attentional tracking process provides a top-down prediction that causes hallucination of the moving object in the associated positions. Without a dynamic background, this does not occur because attention is captured by the luminance transient associated with an object's disappearance. 

A prediction of tracking continuation theory is that, if the illusion occurs, the moment that the object is perceived to disappear should be several tens of milliseconds later than if it does not occur. This is what we found in Experiment 6. 

In summary, the twinkle-goes illusion may be caused by the continuation of attentional tracking when an object disappears, which is enabled by the absence of a salient offset transient. This implies that the attentional tracking process can cause one to hallucinate an object in novel locations. 

Experiments 1a and 1b

To better understand the role of the dynamic noise background, these experiments varied the time when the dynamic noise was presented relative to the time of the disappearance of the object. In Experiment 1a, the onset time of the background manipulation was varied for each of two different target luminances and, in Experiment 1b, the offset time was varied as well as the onset time. 

Methods

Participants

Seven observers (6 females and 1 author) participated in Experiment 1a and a different seven observers (5 females) participated in Experiment 1b. All participants had normal or corrected-to-normal vision and provided written informed consent. All experiments were conducted in accordance with the Declaration of Helsinki (2003) and were approved by the ethics committee of the University of Sydney (Australia), the National Institute of Information and Communications Technology, or the University of Tokyo (Japan). 

Apparatus

In all experiments, images were displayed on a gamma-corrected 22-inch CRT screen (1280 × 960 pixels) with a frame rate of 75 Hz. The CRT resolution was 3.6 min/pixel at a viewing distance of 30 cm. The software was created with Psychtoolbox-3 (Kleiner, Brainard, & Pelli, 2007). 

Stimuli

A pair of rectangular objects (2.9 degrees wide × 7.7 degrees high) moved horizontally at 18.1 deg/s from opposite sides toward the center with a vertical distance of 7.7 degrees from a black fixation point (0.5 degrees in diameter), in upper and lower visual fields, respectively (as described in Figure 1). The moving objects were presented for a random duration between 0.8 and 1 second and then disappeared at predetermined positions, after which the background was presented for an additional 400 ms. The central area of the background (77.1 degrees wide and 36.2 degrees high) was filled with square dots (0.2 × 0.2 degrees), except for a circular gray part (1.1 degrees in diameter) around the fixation point. The luminance values of the background square dots were randomly chosen from a uniform distribution between the minimum and maximum values of the CRT and, in dynamic background conditions, refreshed every movie frame (75 Hz). 

Figure 1.

 

Schematic display of stimuli used here to elicit the “twinkle-goes” illusion. One object in the upper visual field and another in the lower visual field moves horizontally and abruptly disappears near the display's center. Yellow arrows (not shown in the actual display) indicate the movement directions, and dashed rectangles indicate the physical disappearance location (middle panel) and perceived position (right panel), which is shifted in the direction of motion if the random-dot background is dynamically modulated in luminance after the object disappearance. In the experiments, the magnitude of the shift was estimated with alignment judgements where a staircase (1 up and 1 down) adjusted the misalignment across trials.

In Experiment 1a, the background was static or dynamic throughout the stimulus presentation, or the modulation started 80 ms before, at the same time with (0 ms after), or 80 ms after the object disappearance. The objects were either white (high luminance contrast with the background) or gray (low luminance contrast with the background). In Experiment 1b, while the objects were invariably gray, the background was static or dynamic throughout, or the modulation started 0, 27, 54, or 80 ms after the disappearance, or the modulation started at the same time with the motion onset then stopped 0, 27, 54, or 80 ms after the disappearance (background changed from dynamic to static). 

Procedure

In each trial, participants viewed the stimulus display with steady fixation and reported whether the direction of vertical misalignment at disappearance of moving objects was to the left (top object offset to the left of the bottom object) or to the right. A blank (uniform gray) display was presented during the response period and after that a new static noise background with a fixation point was presented for 800 ms until the next trial started. The physical misalignment was adjusted by a staircase with a factor of 2 (e.g. −1.8, −0.9, −0.5, −0.2, 0, 0.2, 0.5, 0.9, and 1.8 degrees) and a 1 up 1 down rule, which targets a 50% proportion of “right” direction responses, with 120 trials per condition. 

The dynamic and static background conditions were randomly interleaved along with the corresponding staircases. The motion directions of the upper and lower visual fields were swapped every trial. For each condition, we estimated the point of subjective equality (PSE) as the vertical alignment corresponding to chance reporting of the direction, by fitting a logistic curve via maximum likelihood. 

Results and discussion

The perceived shift magnitude for the disappearance location is plotted in Figure 2A, as a function of timing between disappearance and when the background became dynamic rather than static (Experiment 1a). The result shows that when the dynamic noise precedes or coincides with the object disappearance and continues until the end of each trial, the illusion occurs (z values > 2.15, p values < 0.04) except when the object has high luminance coincident with the dynamic noise in existence throughout the trials (z = 1.89, p = 0.06). The illusion does not occur (z values < 0.88, p values > 0.38) when the dynamic noise does not start until 80 ms after the disappearance or is totally absent (static background). 

Figure 2.

 

Results of experiments 1a and 1b. The abscissa indicates when the background underwent luminance modulation. The ordinate shows the average shift in perceived position of the object disappearance in the motion direction, in degrees of visual angle. The label of dynamic or static BG indicates the presence or absence of the background modulation throughout each trial. Other numbers in the abscissa indicate when the background modulation started (displayed by circles or triangles) or stopped (displayed by triangles; tested only in Experiment 1b) relative to the object disappearance. The shift for gray objects (low luminance contrast with the background) is displayed by circles or triangles and the shift for white objects (high luminance contrast) is displayed by squares (tested only in Experiment 1a). Error bars represent ± SEM.

The different target luminance values (white and gray) had little effect on the illusion (main effect of object luminance, F_{(1, 6)} = 2.26, p = 0.18, η_p² = 0.02, in a 2-way repeated measures ANOVA). Although the illusion was smaller when the objects were white (interaction of object luminance with background being static versus dynamic, F_{(4, 24)} = 6.73, p = 0.0009, η_p² = 0.05; simple main effect of luminance when the background was dynamic throughout: F_{(1, 30)} = 12.98, p = 0.001, η_p² = 0.04), the illusion was still present: the shift was larger on the dynamic background than on the static background (t₍₄₈₎ = 6.97, p < 0.0001, d_z = 2.63 for gray objects; t₍₄₈₎ = 3.36, p = 0.001, d_z = 1.27 for white objects; main effect of background being static versus dynamic, F_{(4, 24)} = 18.97, p < 0.0001, η_p² = 0.26). 

Experiment 1a revealed that the critical period for the background being dynamic to create the illusion did not extend more than 80 ms after the time of the object disappearance. Experiment 1b extended these results by also investigating the effect of the duration of the dynamic aspect of the background, by varying both its onset and offset time. 

In half of the trials, the background was dynamic at the end of the trial, but the time when that dynamic modulation started was varied, more finely than in Experiment 1a. In the other half of the trials, the background was dynamic at the beginning of the trial, but when it ceased was varied (Experiment 1b). The results revealed (Figure 2B, circles) that the noise must commence by very soon after the object disappearance, as the perceptual shift decreases as the dynamic noise starts later (1-way repeated measures ANOVA, F_{(4, 24)} = 9.334, p = 0.0001, η_p² = 0.61), falling to approximately zero by 80 ms after the objects’ disappearance (z = 0.37, p = 0.24). If the dynamic noise commences at the beginning of the trial, there is no illusion if it ceases at the same time as the object disappearance (z = −0.87, p = 0.38; see Figure 2B, triangles), as the background must be dynamic after the object disappearance. The shift increases as the dynamic noise is extended through several tens of milliseconds after object disappearance (F_{(4, 24)} = 6.50, p = 0.001, η_p² = 0.52). 

In summary, the illusion is strongest when the background is dynamic for the 80 ms following object disappearance. The longer the background is dynamic during that interval, the greater the perceptual shift. 

Experiment 2

The perceptual extrapolation might conceivably happen because the human visual system confuses luminance components contained in the dynamic noise with a part of the moving objects, resulting in an apparent extension of their trajectory. We tested whether the illusion occurred when the moving objects had a color not shared by the background. 

Methods

Participants

Six observers (1 female and 1 author) participated in this experiment. 

Stimuli

The same stimuli as Experiment 1 were used except that the moving objects and the background were completely defined by blue and red components. Either color was randomly assigned to the objects and the other color to the background (i.e. if the background was blue, only the blue gun of the CRT screen was used for the background, with the red and green set to zero [black], and only the red gun was used for the objects). The luminance of the object was 15.0 cd/m² for red (0.62, 0.35 in CIE 1931 xy chromaticity) and 9.9 cd/m² for blue (0.15, 0.08). The luminance values of the background square dots were randomly chosen from a uniform distribution between 0 and 9.9 cd/m² for blue and between 0 and 15.0 cd/m² for red. 

The background luminance modulation (which did not occur in the static background condition) started 80 ms before the object disappearance (and continued until 400 ms after the disappearance) because a pronounced effect was observed for this interval of dynamic noise presentation in Experiment 1. The same staircase and logistic fitting were used as for Experiment 1. 

Results and discussion

Even in these circumstances of different color of the objects and the background, the illusion still occurred (Figure 3) – the magnitude of the shift was positive (z = 1.84 relative to zero, p = 0.03) with the dynamic background and greater than on the static background (t₍₅₎ = 16.29, p < 0.0001, d_z = 6.65), where it was not statistically significant (z = −0.14, p = 0.55) and may have been zero or negative (positive numbers indicate a shift in the direction of object motion). This result indicates that the illusion does not depend on simple confusability of the objects with their background. 

Figure 3.

 

Results of Experiment 2. The objects had a different color from their background. The forward shift in the disappearance location is shown for each of the dynamic and static backgrounds. Empty and filled squares display individual and average results, respectively. Error bars represent ± SEM.

Experiment 3

In the conditions studied so far, the dynamic background contained an equal amount of motion components, or energy, in all directions. Conceivably, the motion energy in the object movement direction might drive the effect, as the expectation of motion in the forward direction might cause attention to weight that stimulus component more heavily. To assess this possibility, the magnitude of the extrapolation was estimated with objects moving on a dynamic background filled with orthogonal movement of the dots rather than random luminance modulation. 

Methods

Participants

Six observers (1 female and 1 author) participated in this experiment. 

Stimuli and procedure

The same stimuli as Experiment 1 were used except that the objects were white and the dynamic background (achromatic) was a moving background: instead of refreshing luminance values, half of the (76800) square dots (0.2 × 0.2 degrees) filling the background area (77.1 degrees wide and 36.2 degrees high) moved upward and the other half moved downward, at 9.0 deg/s. The luminance values of the dots were randomly chosen from a uniform distribution and for the parts of the dots overlapping with each other when they moved, the luminance values were added by the alpha value of 0.5. The dots that had reached either the top or bottom edge of the background area (67.5% of dots did not reach the edge during the stimulus presentation on average) disappeared and new dots appeared at the other edge, yielding the maximum lifetime of each dot of 480 ms. The movement of the dots (which did not occur in the static background condition) started 80 ms before the object disappearance, and continued until 400 ms after the disappearance. Measurements and analyses were done in the same way as Experiment 1. 

Results and discussion

Even with a dynamic background filled with orthogonal movement of the dots rather than random luminance modulation, the illusion was still robust (Figure 4). A marginally positive shift was found (z = 1.62, p = 0.05) with the moving background but not the static background (z = 0.48, p = 0.32), with a significant effect of the dynamic/static background (t₍₅₎ = 4.48, p = 0.007, d_z = 1.83). 

Figure 4.

 

Results of Experiment 3. The dynamic background was entirely composed of vertically moving dots whereas the objects moved horizontally. The forward shift in the disappearance location is shown for each of the dynamic and static backgrounds. Empty and filled squares display individual and average results, respectively. Error bars represent ± SEM.

Rather than the motion energy of the dynamic background or confusability with the moving objects being critical to its effect, there must be some other mechanism. We speculate that the dynamic noise enables the illusion by masking the luminance transient associated with the object disappearance. 

Experiment 4

To better understand the underlying motion processes, this experiment investigated the effect of speed on the amount of perceptual shift. 

Although the underlying mechanisms for motion perception that have been well characterized are temporal frequency tuned (Burr & Ross, 1982; Kelly, 1979) or speed tuned (Perrone & Thiele, 2001; Reisbeck & Gegenfurtner, 1999), humans do have the ability to attentionally track multiple objects at high speeds (Cavanagh, 1992; Pylyshyn & Storm, 1988). This suggests that if attentional tracking is critical to an illusion, the size of the illusion may increase linearly with speed until high speeds. 

Methods

Participants

Eight observers (6 females and 1 author) participated in this experiment. 

Stimuli and procedure

A pair of gray rectangle objects (2.9 degrees wide and 7.7 degrees high) moved horizontally in the same way as in Experiment 1. For the dynamic background condition, the background was dynamic throughout the stimulus presentation and the object speed was 2.3, 4.0, 6.9, 12.0, 20.9, or 36.2 deg/s. For the static background condition, the only speed used was 12.0 deg/s. Measurements and analyses were done in the same way as Experiment 1. 

To assess the linearity of the effect of speed, the value of the quadratic term was examined after fitting a quadratic function (y = ax² + bx + c) to the data for each participant via maximum likelihood, where x and y denote speed and shift magnitude, respectively, with free parameters of a, b, and c. We considered that a magnitude of the quadratic term of greater than 0.002 would be an indication of substantial nonlinearity, at least as much as that previously found for the MIPS illusion and for speed- and velocity-tuned mechanisms; see the dotted line in Figure 5 for an example of a line curved by that amount. We preregistered this criterion at https://osf.io/65pb4 before commencing data collection. 

Figure 5.

 

Results of Experiment 4. The effect of object speed on perceived forward shift from the disappearance location is shown. The background underwent luminance modulation in six-sevenths of the trials (dynamic BG; displayed by circles), with no modulation in the other trials (static BG; displayed by a triangle). Empty and filled markers display individual and average results respectively. For comparison to the data, the dotted curve has a quadratic term of −0.002. Error bars represent ± SEM.

Results and discussion

The shift increased with speed (1-way repeated measures ANOVA, F_{(5, 35)} = 15.58, p < 0.0001, η_p² = 0.46; see Figure 5) and the effect of speed was approximately linear. The statistical test we preregistered is a z-test to compare to 0.002 the absolute value of the fitted quadratic terms for the participants. The absolute value of the quadratic term, 0.0006 (SEM = 0.0003), was significantly smaller than 0.002 (z = −4.61, p < 0.0001) and, on average, the fit of the quadratic curve was very good (r² = 0.98, SEM = 0.005). The slope of the line was 38 ms (SEM = 11 ms), which given the linearity of the effect, is consistent with a process that shifts the objects’ position by a further 30 to 50 ms after disappearance. 

The linear effect of speed here is quite different from speed's effect on the MIPS phenomenon, which saturates at low speeds (Bressler & Whitney, 2006; Chung, Patel, Bedell, & Yilmaz., 2007; De Valois & De Valois, 1991). The present illusion may critically depend on attentional tracking or high-level motion processes, while also involving low-level motion processes. 

Experiment 5a

To assess the role of attentional tracking, we tested objects that rotated about fixation as that allowed us to test at speeds near the attentional tracking limit, because previous work has found that participants can only attentionally track such objects up to about 2 revolutions per second (rps), with performance declining for some participants already by 1.6 rps (Holcombe & Chen, 2013; Verstraten, Cavanagh, & Labianca, 2000). 

Methods

Participants

Seven observers (3 females and 1 author) participated in this experiment. 

Stimuli

A pair of white circular objects (2.4 degrees in diameter) revolved about the fixation point with a radius of 8.7 degrees and a speed of 0.15, 0.3, 0.6, 1.2, or 2.4 rps. The direction of revolution for each trial was reversed relative to the previous trial. The background modulation started 80 ms before the disappearance, and continued until 400 ms after the objects’ disappearance. A static background condition was also included, with only 0.6 rps tested. Other aspects were the same as Experiment 1 except that the moving objects were presented for a random time between 0.8 and 1.6 seconds and the background area subtended 48.2 degrees times 48.2 degrees. 

Procedure

Measurements and analyses were done in the same way as Experiment 1, except that the final orientation relative to vertical of the revolving objects was adjusted by a staircase with a step size of 4 degrees of polar angle (e.g., −16, −12, −8, −4, 0, 4, 8, 12, and 16 degrees) and a 1 up 1 down rule. Such a staircase was run for each condition for 120 trials. 

Results and discussion

The illusory position shift increased with speed steadily to 0.6 rps after which it began to saturate and then decline at a speed between 1.2 rps (corresponding to 65 degrees of visual angle per second) and 2.4 rps (Figure 6), consistent with the possibility that the ability to attentionally track constrains the effect. 

Figure 6.

 

Results of experiments 5a. The perceived shift from the disappearance location, in degrees of polar angle, is shown as a function of object speed in revolution. The background underwent luminance modulation in five-sixths of the trials (dynamic BG; displayed by squares), with no modulation in the other trials (static BG; displayed by a triangle). Error bars represent ± SEM. One revolution per second corresponds to 54.5 degrees of visual angle per second.

A 1-way repeated measures ANOVA up to 1.2 rps supports the increase with speed, F_{(3, 18)} = 6.45, p = 0.004, η_p² = 0.26. We also assessed the evidence by calculating the slope for each pair of successive speeds tested, which supported a decrease in slope around 1 rps, and a decline at high speeds. Specifically, whereas the slope from 0.15 rps to 0.3 rps (12.7 deg/rps, SEM = 4.13 deg/rps) was similar to the slope from 0.3 rps to 0.6 rps (13.7 deg/rps, SEM = 3.00 deg/rps), t₍₆₎ = 0.34, p = 0.74, d_z = 0.13, both were larger than the slope from 0.6 rps to 1.2 rps (1.79 deg/rps, SEM = 2.16 deg/rps) as well as the slope from 1.2 rps to 2.4 rps (−3.78 deg/rps, SEM = 1.36 deg/rps), t₍₆₎ > 3.64, p < 0.002, d_z > 1.37. 

Note that the size of the shift cannot be interpreted as straightforwardly as those of the previous experiment (4), where we calculated, for example, that the linear effect of speed would correspond to 38 ms of extrapolation. Here, instead of using a linear trajectory, we measured rotational shift following the objects’ circular trajectory and we do not know to what extent the process that generates the illusion takes into account the curvature of a trajectory. 

A limitation of this experiment is that the highest rotational speed (2.4 rps) is so fast that the displacement between successive frames at the 75 Hz refresh rate tested is 11.5 degrees of polar angle, potentially impairing the response of motion detectors. 

Experiment 5b

To further investigate a possible role of attentional tracking in the twinkle-goes illusion, in Experiment 5b, we assessed the effect of varying the amount of available attention resource. To manipulate the amount of resource available per target, we added a second pair of potential targets to the display (Figure 7A) and varied whether participants knew which pair they would have to judge the final position of at the end of the trial. 

Figure 7.

 

Schematic stimulus display and results of Experiment 5b. (A) Two pairs of circular objects revolve clockwise and counter-clockwise as indicated by yellow arrows (not shown in the actual display) and then disappear. The target pair was indicated both before and after the stimulus presentation in half of trials (pre-cue condition), and only after the stimulus presentation in the other half of trials (post-cue condition). (B) The forward shift in the disappearance location, in degrees of polar angle, is shown for the pre-cue and the post-cue conditions, for the dynamic background (squares) and the static background (triangles) conditions. Error bars represent ± SEM.

Figure 8.

 

Results of Experiment 6. The response time to the object disappearance on the dynamic/static noise background. Participants pressed a button as rapidly as possible when the object disappeared. The object moved in half of trials (moving object; displayed by circles) and did not in the other half of trials (stationary object; displayed by triangles). Error bars represent ± SEM.

In half of the trials, the target pair was indicated before the stimulus presentation (the pre-cue condition), so participants could attend solely to that pair. In the other half of trials, the target pair was not indicated until after (the post-cue condition), so participants had to split their attention to track both pairs, which impairs tracking performance (e.g. Alvarez & Cavanagh, 2005; Holcombe & Chen, 2013; Pylyshyn & Storm, 1988). 

Methods

Participants

Twenty observers (8 females and 1 author) participated. Based on informal observation and previous work with the manipulation of attention, we expected a small effect with high variance for our per-/post-cue experiment, which indicated that we would need a lot more participants than for the other experiments to achieve high statistical power. 

Stimuli

Two pairs of white circular objects (2.4 degrees in diameter) revolved clockwise and counter-clockwise at 1.3 rps in the upper and lower visual fields respectively, with a trajectory radius of 4.3 degrees (see Figure 7A). The centroids of the object pairs were 13.5 degrees above and below the fixation point. As a reference for the vertical orientation judgments, the objects within a pair were connected by a gray thin line (0.5 degrees wide and 6.3 degrees high). 

Procedure

Half of trials were the pre-cue condition and half were the post-cue condition, which appeared in random order. At the beginning of a pre-cue trial, a white vertical line (0.5 degrees wide and 3.9 degrees tall) was shown for 1.2 seconds centered either 2.9 degrees above or below the fixation point to cue the location where the target pair would appear (upper or lower visual field). In the post-cue condition, two of the white vertical lines were shown, one above and one below fixation, to indicate that the participant needed to attend to both pairs. After the stimulus presentation, in all trials, the white vertical line indicating the pair to report the orientation of was shown until the response. Both conditions were further divided into dynamic versus static background conditions, which were randomly interleaved, with the final orientation of each controlled by its own staircase (480 trials in total). The revolving directions were swapped every trial and the target pair was assigned randomly. The final orientation of the distracter pair was determined at random between −16 degrees and 16 degrees. Measurements and analyses were otherwise the same as those of Experiment 5a. 

Results and discussion

When only half the attention resource was available (the post-cue condition), the perceived final position of the target on the dynamic background significantly lagged its final position (negative shift, z = −2.93, p = 0.004; Figure 7B). Two-way repeated measures ANOVA confirms a main effect of the pre-/post-cue (F_{(1, 19)} = 11.42, p = 0.003, η_p² = 0.08) and that of the dynamic/static background (F_{(1, 19)} = 52.76, p < 0.0001, η_p² = 0.21). Examining the dynamic background condition alone, a simple main effect analysis supported the effect of pre/post cue (F_(1,38) = 18.14, p = 0.0001, η_p² = 0.08). With the static background the pre-/post-cue had less (and possibly no) effect, as indicated by the significant interaction with dynamic versus static background, F_{(1, 19)} = 7.73, p = 0.01, η_p² = 0.01; the simple main effect of pre/post cue in the static background condition was not statistically significant, F_(1,38) = 3.26, p = 0.08, η_p² = 0.01. For completeness, we analyzed the data with the author's data removed and the interaction was still statistically significant F_{(1, 18)} = 6.51, p = 0.02, η_p² = 0.01. 

Perceptual lags like that found for the post-cue conditions here have been investigated before with static backgrounds and are thought to reflect intermittent sampling of position by attentional tracking (Holcombe & Chen, 2013; Howard, Masom, & Holcombe, 2011; Howard & Holcombe, 2008; VanRullen, Carlson, & Cavanagh, 2007). In addition, a few previous studies find a lag with a single moving target even with fairly slow speeds (e.g. Nakajima & Sakaguchi, 2016), suggesting that the visual system temporally averages position estimates. Under our attentional tracking continuation account, at the time of disappearance, the last sampled position may be substantially behind the actual disappearance position, but in the dynamic background condition, attentional tracking can continue for several dozen milliseconds, reducing or eliminating the lag. 

In this experiment, in most conditions, there was a significant lag, and even in the dynamic pre-cue condition the shift was not statistically significant (z = 0.31, p = 0.76). Possibly, the presentation of multiple moving objects can distract attention somewhat in a bottom-up manner (e.g. Abrams & Christ, 2003), causing more infrequent sampling, resulting in more negative shifts. In addition, if the underlying process (e.g. continuation of attentional tracking) does not fully use the curvature of the object's expected future trajectory, the smaller radius (greater curvature) of the object trajectories would have diminished the shift. 

About why post-cuing did not completely eliminate the (relative) shift, the possibility remains that low-level motion somewhat contributes for the twinkle-goes illusion. 

Experiment 6

Under the tracking continuation theory, the perceived time of disappearance of the moving object should be several tens of milliseconds later than in conditions where the illusion does not occur, because the final perceived position is the result of a time-consuming continuation of tracking after object disappearance. This is unlike the anticipatory extrapolation theory of Nijhawan and others, where the shift reduces or eliminates the lag caused by neural delay, so the shifted perceived position is not something created after object disappearance – instead, the represented position has already been extrapolated at the moment that the final location is registered by the brain. 

To test the contrasting predictions of the anticipatory theory and the tracking continuation theory, simple response time to the disappearance was measured with versus without the illusion. Although it has been suggested that an anticipatory extrapolation should be suppressed, via a kind of backward-masking, by the offset transient (Nijhawan, 2008), simple response time would be immune to such a postdictive process. That is, observers can respond to a visual target at a similar reaction time even when that target is very effectively backward masked (Taylor & McCloskey, 1990, 1996; also see Lachter & Durgin, 1999). 

Methods

Participants

Eight right-handed observers (3 females) participated in this experiment. 

Stimuli

A gray circular object remained stationary or, in the moving condition, revolved about fixation at 0.8 rps, with a radius of 4.3 degrees. The object's initial position (which was its only position in the stationary condition) was randomized on each trial, as was its movement direction. In the dynamic background condition, the background was dynamic throughout the stimulus presentation. Other aspects were the same as in Experiment 5a except that the object was presented for a random duration between 2 and 3 seconds and then disappeared, after which the background remained on the screen for an additional 800 ms. 

Procedure

In each trial, participants were instructed to view the stimulus display with steady fixation and press a button with their right middle finger as rapidly as possible when the object disappeared. The background (static or dynamic) was presented for an additional 800 ms, and then a new static noise background with a fixation point was presented for 800 ms to allow the participant to prepare for the next trial. 

If the button was pressed before the object disappeared or was not pressed until 800 ms after, a beep was played to inform participants that they responded inappropriately on that trial. 

The four conditions (stationary versus moving object crossed with static versus dynamic noise background) were randomly interleaved, with 120 trials per condition. After the experiment, any trials with response times (intervals from the disappearance to the button press) smaller than 0 ms or larger than 800 ms were excluded. These averaged 3.9% (SD = 4.0%) of trials. 

Results and discussion

The prediction that the response time would be longer to offset of the moving object than the stationary object particularly on the dynamic background was borne out by the data – there was a statistically significant interaction of moving/stationary object with dynamic/static background (F_{(1, 7)} = 46.44, p = 0.0002, η_p² = 0.39; see Figure 8). The response time was increased by the dynamic background by 65 ms more for the moving object than for the stationary object. 

The results are consistent with the notion that the illusion delays the perceived disappearance time by several tens of milliseconds, consistent with the theory that tracking continues for this interval after object disappearance and continues to mediate visibility. 

For the stationary object too, although the effect was not nearly as big, the response time was longer for the dynamic background than for the static background (F_{(1, 7)} = 46.44, p = 0.0002, η_p² = 0.39). Why is this? A moving object can be thought of as a sequence of transients, evoked at successive locations, which makes the transient evoked by its disappearance less salient than that of the offset of a static object. 

For completeness we report the main effects from the ANOVA: the effect of object motion (F_{(1, 7)} = 82.51, p < 0.0001, η_p² = 0.78) and the dynamic background (F_{(1, 7)} = 87.58, p < 0.0001, η_p² = 0.70). 

Discussion

On a dynamic background, the perceived disappearance location of a moving object was shifted forward relative to with a static background. This “twinkle-goes” illusion does not require the moving object to be highly confusable with the background (Experiment 2), nor does it require substantial amounts of background motion energy (Experiment 3) in common with the object movement direction. 

We propose that the dynamic noise enables the illusion by masking the offset transient associated with the disappearance of the object. In this, we agree with Nijhawan that offset transients prevent perception of a moving object in a forward-shifted position (Nijhawan, 2002, 2008). Less clear to us, however, is the nature of the process that yields the forward-shifted position. 

Nijhawan (2002, 2008) proposed an anticipatory prediction theory, according to which moving objects are continually represented in an extrapolated position ahead of the corresponding sensory signal, unless and until an offset transient occurs to suppress the perception of the final extrapolated position. 

An alternative that we advance here, however, is that the percept occurs after the cessation of afferent signals from the moving object, and that it reflects a continuation of attentional tracking along the object trajectory. Evidence that may favor this account is our findings regarding the effect of speed (Experiment 4). 

Because low-level motion mechanisms show speed or temporal frequency tuning rather than a monotonic effect of speed (Burr & Ross, 1982; Kelly, 1979; Perrone & Thiele, 2002), the linear increase in shift over a broad range that we observed suggests that the amount of shift is determined by a high-level motion mechanism, such as attentional tracking. This is unlike the MIPS illusion, which is temporal-frequency tuned in a manner that yields an increase with speed over only a narrow range of speeds (Bressler & Whitney, 2006; De Valois & De Valois, 1991), and the flash-drag illusion, for which speed has little effect (Whitney & Cavanagh, 2000, 2002). 

Further supporting a role for high-level motion and tracking is that the shift declines when object speed nears the attentional tracking limit (Experiment 5a), and perceived positions increasingly show lag rather than extrapolation when attention is divided (Experiment 5b). Finally, the time the object is perceived to disappear is later when the illusion occurs than when not, by a duration (approximately 65 ms) roughly consistent with the amount of extrapolation (Experiment 6). 

In summary, our account is that, if not captured by the offset transient, attentional tracking can continue for several tens of milliseconds, causing one to hallucinate a moving object in the corresponding positions. This does not deny the existence of motion and position illusions that are not caused by attentional tracking. It is compatible, for instance, with neurons that represent locations ahead of a moving object being activated in advance, facilitating perception of the moving object in that location before it reaches that location (Berry, Brivanlou, Jordan, & Meister, 1999; Blom, Feuerriegel, Johnson, Bode, & Hogendoorn, 2020). It is not yet fully understood how such processes relate to attentional tracking, although Robinson, Grootswagers, Shatek, Gerboni, Holcombe, and Carlson (2021) found electroencephalogram (EEG) evidence for anticipatory activation of locations when participants continued stepping their attention to consecutive locations after a stepping object disappeared. 

Relation to adaptive integration of uncertain position and motion signals

Spatial uncertainty is one factor that seems to affect certain position and motion illusions, such as the MIPS (De Valois & De Valois, 1991) illusion, which is greater when positional uncertainty is higher (Fu, Shen, & Dan, 2001; Kwon, Tadin, & Knill, 2015), with supporting evidence regarding its neural substrate (Fu, Shen, Gao, & Dan, 2004; Whitney, Goltz, Thomas, Gati, Menon, & Goodale, 2003). 

Kwon et al. (2015) suggested a computational model of MIPS based on Kalman filtering that accounts for some quantitative characteristics of MIPS. Their model calculates object position by integrating sensory position signals with current motion signals attributed to the object, weighted by position uncertainty relative to motion uncertainty. Kwon et al. conceptualizes attentional tracking as the integration locus of low-level motion signals with position signals. However, their model would not expect a significant extrapolation for our stimuli because our stimuli have no pattern motion that could be attributed to position change. On the other hand, perhaps the model takes some time to stop tracking when a moving object disappears. This lag might correspond to a continuation of attentional tracking. 

On the framework of adaptive integration of uncertain signals, however, a linear increase in shift with speed would not be expected (Fu et al., 2001; Kanai, Sheth, & Shimojo, 2004) partly because the visual system has a slow speed prior (Stocker & Simoncelli, 2006; Weiss, Simoncelli, & Adelson, 2002) that underestimates object speed (and position shift as a result) particularly for high speeds, especially because fast objects should yield more position uncertainty (Brenner, Van Beers, Rotman, & Smeets, 2006; Linares, Holcombe, & White, 2009). 

The present illusion has different characteristics from the MIPS phenomenon that the uncertainty-based theory has had success explaining. Whereas dividing attention reduced the twinkle-goes, previous work showed that for the MIPS, dividing attention and even the absence of awareness of the motion has little effect (Haladjian, Lisi, & Cavanagh, 2018; Nakayama & Holcombe, 2020; Whitney, 2005). In addition, Kwon et al. (2015) found that when a moving object is shifted in position by orthogonal pattern motion, the abrupt disappearance of the object on a static background still results in the shift being perceived, rather than eliminated by the offset transient. In summary, the visual system may bias position estimates to compensate for positional uncertainty, which may chiefly cause the MIPS, but the present results support a distinct mechanism for the twinkle-goes illusion. 

Relation to mislocalization of brief stimuli by motion

The twinkle-goes illusion seems fundamentally different from the mislocalization of an unmoving flash that is adjacent to a moving pattern, which predominantly depends on motion after the flash (flash-grab illusion, Cavanagh & Anstis, 2013; Roach & McGraw, 2009; Whitney & Cavanagh, 2000). Similar to the present effect, however, Cavanagh and Anstis (2013) found evidence that the flash-grab illusion requires attention (unlike the flash-drag illusion; Fukiage, Murakami, & Whitney, 2011) and shows a near-linear increase in speed, with a saturation by 0.75 rps, which is compatible with our results. However, one difference is that temporal averaging of position could conceivably yield the linear effects of speed in the flash-grab and the related trajectory shortening illusion (Sinico, Parovel, Casco, & Anstis, 2009), whereas temporal averaging could not result in the perception of position beyond the trajectory as seen in the twinkle-goes illusion. Still, high-level motion processes may underlie both and, if so, the difference might reflect distinct properties of attention like what happens in the case of a reversal of motion compared to simple continuation of tracking. At the same time, low-level motion processes may contribute to both (e.g. Kohler, Cavanagh, & Tse, 2015). 

Conclusion

The present study suggests that the top-down prediction due to attentional tracking can in this instance cause a percept, possibly with a minor contribution of bottom-up motion signals. The object's disappearance may provide a prediction error signal (Rao & Ballard, 1999) that normally eliminates the continuation of top-down prediction signals. While predictive coding and related approaches have been popular, we are not aware of other psychophysical evidence supporting an attention-created suprathreshold percept. 

Acknowledgments

The authors thank Chaz Firestone for discussions. 

Supported by JSPS KAKENHI Grant JP18J01398 to RN. The main part of this study was carried out when RN was a JSPS research fellow; a visiting researcher at CiNet, NICT; and a visiting researcher at the University of Sydney. 

Commercial relationships: none. 

Corresponding author: Ryohei Nakayama. 

Email: [email protected]. 

Address: Department of Psychology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan. 

Supplementary movie

Movie S1. “Twinkle-goes” effect, a new motion illusion of extrapolation. The revolving discs disappear when they are vertically aligned. But in the part of the movie where the disappearance is followed immediately by twinkle (dynamic noise), the discs appear shifted in the direction of motion. This does not occur when the background remains static.