The flash-lag effect and the flash-drag effect in the same display

Abstract

Visual motion distorts the perceived position of a stimulus. In the flash-drag effect (FDE), the perceived position of a flash appears to be shifted in the direction of nearby motion. In the flash-lag effect (FLE), a flash adjacent to a moving stimulus appears to lag behind. The FLE has been explained by several models, including the differential latency hypothesis, that a moving stimulus has a shorter processing latency than a flash does. The FDE even occurs when the flash is presented earlier than the moving stimulus, and it has been discussed whether this temporal property can be explained by the differential latency model. In the present study, we simultaneously quantified the FDE and FLE using the random jump technique (Murakami, 2001b) and compared their temporal properties. While the positional offset between a randomly jumping stimulus and a flashed stimulus determined the FLE, a drifting grating appeared next to the flash at various stimulus-onset asynchronies to induce the FDE. The grating presented up to 200 ms after the flash onset induced the FDE, whose temporal tuning was explained by a simple convolution model incorporating stochastic fluctuations of differential latency estimated from the FLE data and a transient-sustained temporal profile of motion signals. Thus, a common temporal mechanism to compute the stimulus position in reference to surrounding stimuli governs both the FDE and the FLE.

Introduction

Our visual system must solve the difficult problem of localizing objects surrounding us. Reaching and eye movements in daily life require accurate position coding. Many psychophysical studies have indicated that the visual system utilizes various clues to optimally estimate the object's position (Cai, Pouget, SchlagRey, & Schlag, 1997; Mateeff, 1978; Matsumiya & Uchikawa, 2000; Ross, Morrone, & Burr, 1997; Suzuki & Cavanagh, 1997; Whitaker, McGraw, & Levi, 1997). In particular, considerable research has focused on how visual motion affects perceived stimulus position (for a review, see Whitney, 2002). The flash-drag effect (FDE) is a representative example of these phenomena: The perceived position of a flashed stimulus appears to be shifted in the direction of nearby motion (Whitney & Cavanagh, 2000). 

According to past studies, motion signals within a certain interval around the flash onset time affect the perceived position in the FDE, and this interval skews toward the time after the flash (Durant & Johnston, 2004; Roach & McGraw, 2009; Whitney & Cavanagh, 2000). Therefore, even if no moving stimulus is present at the flash onset, the perceived position can be influenced by a moving stimulus presented after the flash. Why are motion signals after the flash more critical for the FDE than those before the flash? One possible explanation involves the difference in processing latency between flash and moving stimulus. 

It has been proposed that a flashed stimulus has a longer latency than a moving one, which provides an explanation for another visual illusion called flash-lag effect (FLE): The position of a flash presented in alignment with a moving stimulus is perceived to lag behind (Mackay, 1958; Metzger, 1932). This phenomenon was once interpreted as a spatial illusion (Nijhawan, 1994, 1997). In this interpretation termed “motion extrapolation,” the visual system extrapolates the position of a moving stimulus in the direction of a predicted trajectory to compensate for the latency the moving stimulus inherently requires. However, in the majority of studies, the FLE has been interpreted as a temporal illusion (Brenner & Smeets, 2000; Lappe & Krekelberg, 1998; Patel, Ogmen, Bedell, & Sampath, 2000; Purushothaman, Patel, Bedell, & Ogmen, 1998; Whitney & Murakami, 1998; Whitney, Murakami, & Cavanagh, 2000). In one of these interpretations, termed “differential latency,” the FLE results from a longer processing delay of the flashed stimulus compared to the moving one (Whitney & Murakami, 1998). Murakami (2001a, 2001b) indicated that the FLE occurs even when the moving stimulus randomly changes its position so that observers cannot predict its next position, thus making motion extrapolation impossible. 

In the FDE configuration, the FLE may also occur between the flash and moving stimulus. If the moving stimulus is processed earlier than the flash, the FLE may cause a flash presented before the motion onset to be perceived after it. If the FDE occurs after the processing stage at which the differential latency between flashed and moving stimuli is made explicit, the time course of the FDE may be shifted relative to the interval of the moving stimulus. 

There has been some controversy regarding the temporal relationship between the FDE and FLE. Some studies reported that motion signals 60–80 ms after the flash onset have the largest impact on the FDE (Durant & Johnston, 2004; Fukiage, Whitney, & Murakami, 2011). This temporal shift is comparable to the previously reported magnitude of the FLE expressed in time (Krekelberg & Lappe, 1999; Whitney & Murakami, 1998; Whitney et al., 2000). In contrast, Whitney and Cavanagh (2000) indicated that motion signals even 200 ms after the flash onset could induce the FDE, and this time lag was much longer than the previously reported differential latency. In these studies, however, the two illusions were not measured at the same time and for the same stimulus configuration. The magnitude of the FLE is dramatically influenced by many factors such as the eccentricities of the flashed and moving stimuli, the spatiotemporal predictability of the flash, and differences in task strategy (Baldo & Klein, 1995; Kanai, Sheth, & Shimojo, 2004; Murakami, 2001b; Namba & Baldo, 2004; Purushothaman et al., 1998; Vreven & Verghese, 2005); thus, one should be cautious about directly comparing the time windows for the FLE and FDE measured under different experimental settings. 

Although the FLE may occur in the stimulus configuration designed for the FDE (Figure 1), no studies have measured the FLE and FDE simultaneously. Part of the difficulty in measuring both illusions for the same dataset comes from the lack of methodology as to how to analyze perceptual data from spatiotemporally complicated situations. As shown in Figure 1, observers compare the positions of the flashed and moving stimuli in the FLE experiment, whereas the positions of the flashed stimulus and of some reference point (the fixation point in the illustrated example) are compared in the FDE experiment. Therefore, it is usually impossible to measure the FDE and FLE in a single task. 

Figure 1

 

The FLE and FDE can occur in the same stimulus configuration. A stationary flash presented in alignment with a moving stimulus is perceived to lag behind (FLE). On the other hand, in the presence of a moving stimulus, the position of the flash presented in alignment with a fixation cross is subjectively shifted in the direction of motion (FDE). The dotted and filled rectangles represent physical and perceived positions, respectively.

Additionally, both the FDE and the FLE are usually experienced as spatial illusions. When the FLE is described as a temporal illusion, researchers actually measure the spatial offset at which the flashed and moving stimuli appear aligned, and then this spatial offset is transformed to an equivalent temporal delay by the relationship (temporal delay = spatial offset / speed). In this case, the experimenter is confronted with the difficult problem of dissociating a single relative position judgment between stimuli into two different components, the FLE and FDE. 

In the present study, we propose a new method to measure the FDE and FLE simultaneously. By using the random jump technique (Murakami, 2001a, 2001b), the FLE was measured as a purely temporal illusion. In this situation, any systematic spatial shifts were attributed to the FDE. At the same time, by carefully manipulating the relative timings of the flashed and moving stimuli, the time course of the moving stimulus and of the FDE were compared. Comparing this FDE time course with the FLE magnitude revealed their temporal relationship. 

Methods

Observers

One of the authors (YM) and three adults who were naive to the purpose of the experiment (aged 19–26) participated in the study. All had normal or corrected-to-normal visual acuity. Each observer provided written informed consent. This study, which followed the Declaration of Helsinki guidelines, was approved by the Ethics Committee of the College of Arts and Sciences, The University of Tokyo. 

Apparatus

Stimuli were presented on a CRT monitor (Mitsubishi Electric RDF223H, 1024 × 768 pixels, mean luminance 42.8 cd/m²) in a dark room. The refresh rate of the monitor was 60 Hz, so whenever the time dimension is henceforth described in video frames, one frame corresponds to 16.7 ms. Each observer viewed the stimuli binocularly from a viewing distance of 43 cm with his/her head mildly constrained with a chin rest. 

Stimuli and procedure

All visual stimuli were generated using MATLAB and the Psychophysics Toolbox (Brainard, 1997). A schematic of the stimulus configuration is shown in Figure 2 (see also Supplementary Movie). A fixation cross was presented at the center of the display throughout the experiment. Two pairs of drifting sinusoidal gratings were used as inducers of the FDE. All gratings had a 0.14-cpd spatial frequency, 4-Hz temporal frequency, >99% contrast, and occupied a rectangular region subtending 5° × 36°. The inner and outer gratings were centered at 7.7° and 14°, respectively, from the central fixation cross. The outer two gratings always translated in the direction opposite to the inner two gratings to minimize the intrusion of optokinetic reflexive eye movements. The initial phases of the gratings were the same for each grating pair and were randomized for every presentation. The motion direction was chosen randomly for every presentation. 

Figure 2

 

Schematic of the stimulus configuration. Two pairs of sinusoidal gratings were used as the inducer of the FDE. A randomly jumping bar was presented to the immediate left of the fixation cross; a flash (horizontal bar) was presented to the left of the leftmost grating. The randomly jumping bar changed its position every 11 frames (183 ms). The drifting gratings repeatedly appeared and disappeared every 30 frames (500 ms) throughout each session.

A randomly jumping bar and a flashed bar were centered at 2.8° and 18.8°, respectively, to the left of the central fixation cross. The randomly jumping bar was continuously presented throughout the experiment. Both the jumping bar and flash were dark gray (8.6 cd/m²) rectangles (3° × 27 min). The spatiotemporal plot of the stimulus sequence is shown in Figure 3. 

Figure 3

 

Spatiotemporal relation between the flash and randomly jumping bar (adapted from Murakami, 2001b). Coloring is for illustrative purpose. The randomly jumping bar (green) stayed at a random vertical position for 183 ms (11 frames) and then jumped to another random position. At a given moment, the flash (red) was presented for 16.7 ms (one frame) at a random vertical position. The spatiotemporal plot of these stimuli is illustrated in the top right panel. Each point on the spatiotemporal correlogram represents when and where the flash was presented relative to the onset time and position of a given jumping bar. Note that each individual flash is plotted multiple times in the spatiotemporal correlogram (four times in the bottom right panel).

The drifting gratings repeatedly appeared for 30 frames (500 ms) and disappeared for 30 frames (500 ms) throughout each session. Every 11 frames (183 ms), the jumping bar was vertically displaced to a randomly chosen position (chosen from a range of ±96 min around the horizontal meridian), stayed there for 11 frames, and then jumped to the next random position. We chose this duration because it was known to produce a strong FLE in random motion (Murakami, 2001b), and because it should be long enough to be relatively immune to the illusory position shift that might occur in the jumping bar itself and short enough to yield relatively vivid apparent motion. The flashed bar, on the other hand, was presented for one frame with random timing (with the interflash interval randomly chosen from a range of 2500 ± 500 ms) and constrained to have a predetermined stimulus-onset asynchrony (SOA) relative to the onset of the drifting grating. The vertical position of the flashed bar was randomly chosen from a range of ±32 min around the horizontal meridian. 

Observers were requested to judge whether the flash was above (“upper”) or below (“lower”) the jumping bar at the time of the flash onset and to press one of two buttons to report their judgment by the time of the next flash presentation. The next flash was presented no matter whether a button press occurred or not. The experiment continued until the number of repeated trials for every SOA between flash and drifting grating reached at least 1,100. 

Data visualization and analysis

As in Murakami's (2001a, 2001b) studies, a spatiotemporal correlogram was drawn for each grating-to-flash SOA condition for the purpose of visualizing each observer's positional judgment for the flash relative to the jumping bar. Each point on the correlogram represents when and where the flash was presented relative to the onset time and position of a given jumping bar (Figure 3). Each observer's responses in each grating-to-flash SOA condition are superimposed in one correlogram, and the percentage of the observer's “upper” responses when the outer grating (the grating most proximal to the flash) moved upward are shown as a grayscale plot (Figure 4). The relative vertical positions and the observer's responses when the outer grating moved downward were flipped and merged with the data under the upward condition. Therefore, the gray scale in the correlogram indicates the percentage of the trials in which the observer reported that the flash was seen displaced in the motion direction of the outer grating relative to the jumping bar. For simplicity, the gray scale in the correlogram is subsequently described as if it were the percentage at which the observer reported that the flash was seen as “upper” compared to the jumping bar when the nearest grating was drifting upward. 

Figure 4

 

Predicted shifts of the correlogram under the occurrence of the FDE, FLE, and both. (a) Ideal observer model. An ideal observer who knows exactly what is being presented on the monitor at each instant can perfectly judge the position of the flash relative to the jumping bar that is presented simultaneously. (b) Correlogram when only the FDE occurs. The correlogram is spatially shifted relative to the ideal observer's performance. (c) Correlogram when only the FLE occurs. The correlogram is temporally shifted relative to the ideal observer's performance. (d) Correlogram that would be obtained from a real observer who experiences both the FDE and the FLE. The correlogram is spatially as well as temporally shifted relative to the ideal observer's performance.

The key assumptions are that the internal representation of the flash was spatially displaced upward in the presence of the upward drifting grating (i.e., manifestation of the FDE), and furthermore, that the flash temporally lagged relative to the jumping bar (i.e., manifestation of the FLE). These assumptions made it possible to estimate the magnitudes of both the FDE and FLE from the spatial and temporal offsets between the flash and jumping bar in the same dataset. The amplitude of the FDE can be quantified as the point of subjective alignment (PSA), namely, the spatial offset at which they were perceived as aligned, and the amplitude of the FLE can be quantified as the temporal offset at which the observer's response best reflected the relative position between the temporally offset flash and jumping bar. 

An ideal observer who knows exactly what is being presented on the monitor at each instant can perfectly judge the position of the flash relative to the jumping bar that is presented simultaneously. The spatiotemporal correlogram of the ideal observer is shown in Figure 4a. If only the FDE occurred, the flash should appear to be shifted upward, and therefore, the observer would report “upper” even when the flash was physically lower than the jumping bar. As a result, the correlogram would be shifted downward (Figure 4b) relative to the ideal observer's correlogram. If only the FLE occurred, the correlogram would be shifted backward in time (Figure 4c) relative to the ideal observer's correlogram. Consequently, for a real observer for whom both the FDE and FLE occurred, the correlogram would be spatially as well as temporally shifted (Figure 4d). 

Because it was impossible to predict the next spatial position of the randomly jumping bar, deviations from the ideal observer's performance in the spatial dimension cannot be explained by the spatial account of the FLE, namely “motion extrapolation.” Therefore, if the correlogram had any systematic spatial shift, it could be safely used as the operational definition of the FDE, and any systematic temporal shift in the correlogram would be orthogonal to the contribution of the FDE, and therefore would be used as the operational definition of the FLE. 

Since successive positions of the randomly jumping bar were uncorrelated, the correlogram should stay at the baseline chance level if the flashes were temporally too far backward or forward compared with the current jumping bar. Hence, for the first step of the analysis, the FDE was calculated from each correlogram by temporally averaging data from −183 to 367 ms relative to the onset of each jumping bar. These averaged data, in which information of relative time between bar and flash was lost, represented the probability of the “upper” response as a function of relative position between bar and flash. By fitting a cumulative normal distribution as a psychometric function, the PSA was calculated as the estimate of the FDE (Figure 5, right panel). 

Figure 5

 

Visual explanation of how to obtain the FDE and FLE from the dataset of the present study. The analysis was conducted separately for each grating-to-flash SOA condition. The displayed spatiotemporal correlogram is for subject HM in the 100-ms SOA condition. Data from −183 to 367 ms relative to the onset of each jumping bar were averaged across time and fitted with a psychometric function, from which the PSA was calculated. The kernel function p(−τ) for the FLE was estimated by deconvolving c(t), or the human observer's behavior (the filled circles represent raw data, whereas the black curve indicates noise-cut filtered time series on which deconvolution was performed), with b(t), or the hypothetical observer's FLE- and noise-free performance (green profile).

After determining the PSA, the FLE was estimated. The percentage of “correct” responses with respect to the current jumping bar is plotted as filled circles in the bottom panel of Figure 5. A “correct” response means an “upper” (“lower”) response to the flash that is physically upper (lower) than the PSA. For an FLE- and noise-free hypothetical observer who experiences the FDE but otherwise judges the relative position perfectly (Figure 4b)—that is, who always reports “upper” (“lower”) when the flash is upper (lower) than the PSA, the “correct” rate will always be 1 when bar and flash are simultaneously presented and at chance level (0.5) otherwise. The green profile represents this theoretically predicted performance. 

In the current framework, the FLE is assumed to result from the differential latency that obeys a certain temporal distribution (Fukiage & Murakami, 2010; Murakami, 2001a, 2001b). For clarity, the theoretical curve of the “correct” response will be termed c(t), and the FLE- and noise-free hypothetical observer's performance will be termed b(t), where t denotes the flash onset time relative to that of the jumping bar. The differential latency distribution is represented by p(τ), where τ denotes differential latency. Thus, the correlation function c(t) would be simply determined as ∫p(−τ)b(t − τ)dτ. While p(τ) denotes the distribution of differential latency, the kernel function p(−τ) describes the distribution of the onset time of the flash that is perceptually simultaneous with the jumping bar at time zero. The kernel function p(−τ) was estimated by deconvolving the correlation function c(t) by the hypothetical ideal observer's performance b(t). To reduce random noise, c(t) was noise-cut filtered with the optimal Wiener filter. The actual human data and the noise-cut filtered data are shown in the bottom panel of Figure 5 for comparison. 

Results

Time course of the FDE

The FDE data for each observer and their average are shown in Figure 6. The magnitude of spatial mislocalization of the flash is plotted as a function of the SOA between flash and drifting grating. The positive FDE values indicate that the flash appeared to be displaced in the motion direction of the nearest grating. If the FDE occurred only when the flash was presented during the presentation of the grating, we would obtain positive values for SOAs of 0–500 ms and otherwise zero. However, if the motion signals after the flash also induced the FDE, the time course of the FDE would be shifted backward relative to that of the presentation of the grating. 

Figure 6

 

(a) Time course of the FDE for the four observers as a function of SOA between flash and drifting grating. Error bars indicate 95% confidence intervals. Shaded regions represent grating-to-flash SOAs at which the flash was presented during the grating presentation; 0-ms SOA corresponds to the condition in which the flash and drifting grating onset occurred simultaneously; 500-ms SOA corresponds to the condition in which the flash was displayed at the time when the drifting gratings were turned off. (b) Average time course of the FDE across all four observers after normalization relative to each observer's peak FDE value. Error bars indicate standard error of the mean.

The latter scenario was confirmed. The FDE significantly occurred at −100-ms SOA for all observers, whereas no significant FDE was observed at 400-, 450-, and 500-ms SOAs for observers HM, SS, and YuM, and at 500-ms SOA for observer YoM. This pattern of temporal shift is consistent with the previous finding that the motion signals after the flash presentation can induce the FDE (Roach & McGraw, 2009). We also found that the FDE reached a peak when the flash was presented 50–100 ms before the onset of the drifting grating. In addition, spatial mislocalization in the direction opposite to the grating motion around the offset time of the grating was significantly observed at 500- and 550-ms SOAs for observer HM, at 400- to 550-ms SOAs for observer SS, at 500-ms SOA for observer YuM, and at 550- and 600-ms SOAs for observer YoM. These results are largely consistent with Roach and McGraw's (2009) findings of a transient increase in the FDE around the motion onset and of a repulsive FDE around the motion offset. Similar contrasting effects in motion perception have been demonstrated in different contexts (e.g., Raymond & Isaak, 1998; Wexler, Glennerster, Cavanagh, Ito, & Seno, 2013). 

Distribution of the FLE

The time at which the “correct” position between flash and jumping bar was estimated with maximal frequency was temporally shifted relative to their actual temporal relationship. This phenomenon is consistent with Murakami's (2001a, 2001b) studies and is considered evidence for the FLE in random jump. 

To quantify the estimated distribution of the differential latency, we integrated the deconvolution results within the latency range from −500 to 500 ms, and calculated the latency values giving 25, 50, and 75 percentiles of the integration result. The left column of Figure 7 shows the estimated distribution of the differential latency as a function of the SOA between flash and drifting grating separately for each observer. The bottom, middle, and top curves correspond to 25, 50, and 75 percentiles of the differential latency distribution, respectively. Consistent with previous studies (Fukiage & Murakami, 2010; Murakami, 2001a, 2001b), the differential latency exhibited a broad temporal distribution. A one-way repeated-measures analysis of variance for the median of the estimated probability distribution revealed no significant effect of SOA, F(13, 39) = 1.61, p = 0.12. Accordingly, we averaged the task performance (the percentage of “correct” responses as shown in the bottom panel of Figure 5) across all SOA conditions for each observer and estimated the differential latency with this averaged task performance. The estimated distribution of the differential latency averaged across all SOA conditions is shown in the right column of Figure 7, and the median of the distribution was 117.7, 117.9, 143.0, and 85.1 ms for observers HM, SS, YuM, and YoM, respectively. 

Figure 7

 

(Left panels) 25, 50, and 75 percentiles of the estimated distribution of the differential latency p(τ) plotted as a function of the SOA between flash and drifting grating. Shaded regions represent SOA conditions in which the flash was presented during the grating presentation. Positive latency values indicate that the flash had a longer latency than the randomly jumping bar. (Right panels) The differential latency distribution obtained by the deconvolution of the actual observer's noise-cut filtered data c(t) averaged across all SOA conditions by the hypothetical observer's performance b(t).

Temporal properties of the FDE and FLE

The next question was whether the temporal shift of the FDE related to the presentation interval of the drifting gratings could be explained by the magnitude of the FLE expressed in terms of differential latency. To test this hypothesis, we further modeled the time course of the FDE using the experimentally estimated FLE. 

In this model (Figure 8a), a time series of the impact of motion signals, M(t), which obeys a boxcar function, is convolved with a kernel p(−τ) reflecting the distribution of the differential latency of the flash relative to the drifting grating, where “impact” is a dimensionless variable that simply represents any influential power the grating has on the perceived position of the flash. We assume that the time course of the FDE, F(t), is simply determined as ∫p(−τ)M(t − τ)dτ, where p(τ) is the estimated distribution of the FLE as plotted in the right panels of Figure 7. The observed and simulated time courses of the FDE are superimposed in the right panel of Figure 8a. 

Figure 8

 

A convolution model of the time course of the FDE. (a) The time course of the FDE for observer HM was simulated by a convolution of hypothetical motion impact signals, M(t), with a kernel reflecting the estimated distribution of differential latency, p(−τ). Red circles indicate the observed magnitude of the FDE in this experiment and the solid curve the convolution results. (b) (Left) M(t) with a transient-sustained profile customized for each observer. (Middle) The kernel function p(−τ) for each observer. (Right) Observed and simulated time courses of the FDE.

The simulation result appeared to explain the actual measurement value of the FDE fairly well except for a transient increase around the motion onset. From this appearance, we reasoned that the hypothetical motion impact signals might need to form some transient-sustained profile reflecting neuronal outputs to a moving stimulus (cf. Roach & McGraw, 2009) rather than a boxcar function. As such, we modified M(t). In the revised model, the motion signals had three components: a transient component time-locked to the onset of the drifting grating, another transient component time-locked to the offset of the drifting grating, and a sustained component lasting throughout the presentation of the drifting grating (Figure 8b). The new M(t) was formulated as:  

 

where t denotes the SOA (ms) between flash and drifting grating, and a, b, and T are free parameters.  

The observed and simulated time courses of the FDE are shown in the right column of Figure 8b. The best-fit parameters were (a, b, T) = (49.3, 0.45, 49.5 ms), (77.5, 0.16, 58.6 ms), (25.5, 0.99, 113.8 ms), and (246.6, 0.29, 30.2 ms) for observers HM, SS, YuM, and YoM, respectively. This model more accurately predicted the observed time course of the FDE (R² = 0.816, 0.955, 0.852, and 0.916 for observers HM, SS, YuM, and YoM, respectively). By taking into account the transient-sustained nature of motion impact signals, the temporal shift of the time course of the FDE relative to that of the motion presentation was reasonably well explained by the distribution of the FLE. 

Discussion

FLE and FDE share the same differential latency

In the present study, we simultaneously quantified the FLE and FDE using the random jump technique and compared their temporal characteristics. The FDE induced by the drifting grating started to occur over 100 ms earlier than the onset of the grating, and the flash was perceived 100–150 ms later than the randomly jumping bar. From these experimental results, we proposed a model in which the time course of the FDE originated from an interplay between the transient-sustained time course of motion signals and distribution of the differential latency. The model well mimicked the observed time course of the FDE, suggesting that the FDE and the FLE are both subject to the same differential latency between flashed and moving stimuli. 

Previous FDE studies reported that motion signals within a certain interval around the flash onset affect the perceived position of the flash, and that this interval skews toward the time after the flash (Durant & Johnston, 2004; Fukiage et al., 2011; Whitney & Cavanagh, 2000). These studies created a controversy over this interval and its relationship with the FLE. Whitney and Cavanagh (2000) reported that motion signals even 200 ms after the flash onset could induce the FDE, and this time lag was much longer than the previously reported FLE magnitudes of tens of milliseconds (Krekelberg & Lappe, 1999; Whitney & Murakami, 1998; Whitney et al., 2000). On the other hand, in other FDE studies, the effective time range of motion signals was 60–80 ms after the flash onset (Durant & Johnston, 2004; Fukiage et al., 2011) and comparable to the reported FLE magnitudes. The apparent contradiction for the temporal tunings of the FDE and FLE might be explained within the framework proposed in the present study. 

The key assumption of our convolution model is that the temporal tuning of the FDE is determined by a combination of the time course of the motion signals and the probability distribution of the differential latency of the flash. First, consideration of differential latency needs to address the substantial differences in stimulus configuration between typical FLE and FDE experiments. In typical FLE experiments, the flash is presented foveally or parafoveally (Baldo & Klein, 1995; Krekelberg & Lappe, 1999, 2000; Murakami, 2001a, 2001b; Nijhawan, 1994; Whitney et al., 2000). By contrast, in typical FDE experiments, especially in studies reporting long time lags such as Whitney and Cavanagh's (2000) and ours, the flash was presented peripherally, whereas other FDE studies reporting relatively short time lags presented the flash foveally or parafoveally (Durant & Johnston, 2004; Fukiage et al., 2011). These differences in flash eccentricity partially explain the discrepancy. Because the FLE increases with flash eccentricity, the longer FLE we observed (80–150 ms) might have occurred also in previous FDE studies placing the flash in the periphery. In addition, many factors including the detectability of the flash (Purushothaman et al., 1998), eccentricity of the moving stimulus (Kanai et al., 2004), and spatiotemporal uncertainty of the flash (Brenner & Smeets, 2000; Kanai et al., 2004; Murakami, 2001a) can modulate the FLE magnitude. At the same time, differently moving stimuli convey different time courses of motion signals. As these factors lead to different time courses of the FDE, caution is warranted when comparing FLE and FDE data measured under different experimental settings. 

Second, our model assumes that the time course of the motion signals also contributes to the temporal characteristics of the FDE. Previous studies have quantified the temporal feature of the FDE with various motion stimuli, including continuous rotation (Durant & Johnston, 2004), apparent motion (Fukiage et al., 2011), grating motion with direction reversal (Whitney & Cavanagh, 2000), and grating motion with explicit motion onset and offset (Roach & McGraw, 2009). These various stimuli may yield different time courses of motion signals, and thus different time courses of the FDE. For example, when the temporal lag of the FDE is measured relative to the timing of the motion direction reversal, the motion signals after the reversal could be larger than those before due to a transient signal produced by the motion reversal. Such temporal asymmetry of the motion signals may amplify the apparent temporal lag of the FDE. 

FDE and FLE in the random motion sequence

By using the random jump technique, we measured the FLE as a purely temporal illusion. While the FLE has often been interpreted as a temporal illusion (Brenner & Smeets, 2000; Lappe & Krekelberg, 1998; Patel et al., 2000; Purushothaman et al., 1998; Whitney & Murakami, 1998; Whitney et al., 2000), some studies have proposed that the spatial hysteresis of a moving object also contributes to the FLE (Chappell & Hine, 2004; Maus & Nijhawan, 2008). In contrast to motion with a continuous trajectory, the jumping bar randomly changes its position so that observers cannot predict its next position based on its past trajectory, which makes motion extrapolation impossible in principle. However, this does not necessarily negate the involvement of a spatial mechanism. Because the bar jumped up and down randomly, any effect of spatial hysteresis on the perceived position of the bar should have been cancelled out in our correlation analysis. One possible method to test such a spatial effect in a random motion sequence is to draw second-order spatiotemporal correlograms conditional on the previous position of the bar. However, it was difficult to draw such second-order correlograms with an acceptable signal-to-noise ratio because of the limited number of trials. Further study will be needed to investigate spatial mechanisms of the FLE in random motion. 

The random motion used in the present study was an apparent motion like stimulus that was static for a certain duration (183 ms) before jumping to another position. This stimulus sequence could be viewed as a series of flash presentations. Thus, one might argue that the FDE could also occur for each jumping bar. Considering our stimulus configuration, however, this is unlikely because the FDE magnitude decreases (a) for flashes presented repetitively in the same region, (b) with increasing flash duration (Whitney & Cavanagh, 2000), and (c) with decreasing flash eccentricity (Durant & Johnston, 2004). 

What constitutes differential latency?

The present study demonstrated that the FDE and FLE are subject to the same differential latency between flashed and moving stimuli. What kind of psychological and neural processes for position computation are revealed by the new observations of the present study concerning the temporal properties of the FDE and FLE? 

Several studies have proposed that the processing delay of the flash compared to the moving stimulus results from the time required to capture or shift attention between the flashed and moving stimuli, so as to take a snapshot of the moving stimulus—that is, the flash triggers a time-consuming process whereby the position of the moving stimulus is sampled (Fukiage & Murakami, 2010). However, in the present study, the time course of the FDE also shifted relative to the presentation interval of the drifting grating. In this paradigm, observers compared the position of the flash relative to the randomly jumping bar, not relative to the drifting grating. Therefore, observers did not need to shift their attention from the flash to the grating to sample the position of the grating. Nevertheless, the time lag comparable to the FLE occurred between the FDE and the motion presentation. Therefore, the present results suggest that the differential latency between flashed and moving stimuli does not reflect the temporal characteristics of an attentional shift or position sampling, and that the differential latency is already determined before the relative position is computed. This notion is consistent with a previous study reporting that as few as two frames of unpredictable apparent motion can preattentively cause the FDE (Fukiage et al., 2011). Of course, this does not imply that attention never affects the FLE and FDE. Various studies have reported that attention indeed modulates the magnitude of the FLE (Baldo, Kihara, Namba, & Klein, 2002; Chappell, Hine, Acworth, & Hardwick, 2006; Namba & Baldo, 2004; Shioiri, Yamamoto, Oshida, Matsubara, & Yaguchi, 2010) as well as the FDE (Shim & Cavanagh, 2005; Tse, Whitney, Anstis, & Cavanagh, 2011). In the present study, both the estimated shape of the motion signal and the differential latency varied widely across individuals, which could reflect individual differences in the subjects' attentional state. Further studies should investigate whether attentional modulations in the temporal characteristics of the FLE and FDE are independent. 

In the present study, the differential latency was estimated from the FLE and successfully applied to explain the time course of the FDE. In other words, even though we independently defined and measured the differential latency between the random motion and the flash, and the temporal offset between the drifting grating and the flash, the former explained the latter. However, this conclusion cannot necessarily be generalized to other experimental settings. The differential latency between flash and moving stimulus may be caused by a general computational process that computes flash position in reference to the surrounding environment. However, environmental objects may have variable latencies on their own, as we discussed above; for example, if we set the jumping bar just above its contrast threshold while leaving the drifting grating as it is, the estimated differential latency to explain the FLE might become too small to explain the time course of the FDE. 

As for the neural basis of the differential latency, physiological studies have indicated that the latency difference between stationary and moving stimuli could arise at early stages of visual processing, such as the lateral geniculate nucleus (Orban, Hoffmann, & Duysens, 1985) and area V1 (Jancke, Erlhagen, Schoner, & Dinse, 2004), and remain at all subsequent stages. However, the reported temporal advantage for moving stimuli in these early areas is approximately 15 ms, which is shorter than the previously reported FLE, and cannot explain the broad distribution of the FLE of over 200 ms estimated in the present study. Psychophysical studies demonstrating the relative independence between the FLE or FDE and the tilt aftereffect—that is, after adaptation to slightly tilted lines, vertical lines appear tilted away from the adapted orientation—also support the notion that the FLE as well as the FDE may originate at later stages (Fukiage & Murakami, 2010; Kosovicheva et al., 2012), given our knowledge that the mechanism underlying the tilt aftereffect originates in V1 (De Valois, Yund, & Hepler, 1982; Maffei, Fiorentini, & Bisti, 1973). 

One probable neural basis for the FLE and FDE is hMT+. Transcranial magnetic stimulation (TMS) applied to hMT+ reduces both the FLE (Maus, Ward, Nijhawan, & Whitney, 2013) and FDE (Whitney et al., 2007). The pattern of fMRI activity in hMT+ produced by flashes presented near motion is similar to that produced by physically shifted flashes without motion (Maus, Fischer, & Whitney, 2013). The time course of the FDE has similar temporal properties as the transient-sustained response profile of MT neurons (Roach & McGraw, 2009). These lines of evidence strongly support the involvement of hMT+ in the FLE and FDE. In line with these studies, we were able to propose a comprehensive model for the temporal characteristics of the FDE and FLE by introducing a transient-sustained profile of the motion signal. 

The present study and our previous studies have demonstrated that the differential latency obeys a broad probability distribution spreading over ∼300 ms (Fukiage & Murakami, 2010; Murakami, 2001a, 2001b). Several studies have suggested that the FLE is mediated by a sluggish computational process that binds the flash and the moving stimulus (Arnold, Durant, & Johnston, 2003; Arnold, Ong, & Roseboom, 2009; Brenner & Smeets, 2000; Fukiage & Murakami, 2010), and the broad temporal tuning of the differential latency observed in the present study might reflect such a sluggish binding process. By using a psychophysical reverse correlation method, recent motion perception studies have demonstrated that the spatiotemporal information of motion is dynamically bound within a time window of 200–300 ms (Iyer, Freeman, McDonald, & Clifford, 2011; Neri, 2014; Neri & Levi, 2008). Considering that both lines of studies share the width of the processing time window or the biased spatiotemporal readout for the moving stimulus, the differential latency could be potentially defined in the context of a more general computational principle beyond the motion-position interaction. 

Conclusion

The present study demonstrated that the FDE and FLE can be quantified simultaneously using the random jump technique. The time course of the FDE was shifted 100–200 ms relative to that of the motion presentation, and this temporal offset was well explained by the probability distribution of the FLE and a nonlinear temporal property of the motion signals. These results suggest that a common temporal mechanism to compute the flash position in reference to the surrounding stimuli governs both the FDE and FLE. 

Acknowledgments

This present study was supported by a JSPS Grant-in-Aid for Scientific Research on Innovative Areas (25119003) to IM and a Grant-in-Aid for JSPS Research Fellow (14J03550) to YM. 

Commercial relationships: None. 

Corresponding author: Yuki Murai. 

Email: [e-mail]. 

Address: Department of Life Sciences, The University of Tokyo, Meguro-ku, Tokyo, Japan.