Inhibition of return modulates the flash-lag effect

Abstract

Transient events are known to draw exogenous attention, and visual processing at the attended location is transiently facilitated, but after several hundred milliseconds, attentional processing at the cued location becomes poorer than processing elsewhere, resulting in a slower reaction to a target stimulus that subsequently appears at the cued location. Despite a number of previous studies on this effect, termed inhibition of return (IOR), it is still unclear whether a perceptual process related to the subjective onset time of the target stimulus is disrupted when IOR occurs. In the present study, we used a distinct visual phenomenon termed the flash-lag effect (FLE) as a tool to quantify IOR. The FLE is an illusion in which a flashed stimulus appears to lag behind a moving stimulus, despite being physically aligned. We used an identical stimulus configuration and asked observers to conduct two independent tasks in separate sessions. The first was a simple reaction task to measure the onset reaction time (RT) to an abruptly appearing target. The second was an orientation judgment task to measure the degree of the FLE. Both the RT and the FLE were found to be altered in accordance with IOR, and a significant correlation was demonstrated between the changes in the RT and those in the FLE. These results demonstrate that the perceptual process related to the stimulus onset can be compromised by IOR.

Introduction

Although our eyes are constantly exposed to many visual inputs, our visual system cannot equally process all of the inputs due to limitations in cognitive resources. Implementing the most appropriate allocation of the limited resources is one of the most important problems that the visual system is faced with. One crucial function relating to such allocation of resources is location-based attention. It is well known that a transient event automatically draws attention to the location where the event has just occurred; this type of attention, often termed exogenous attention, has been extensively studied (for a review, see Carrasco, 2011; Klein, 2005; Mulckhuyse & Theeuwes, 2010). The visual system prioritizes and facilitates processing at the location where something new is occurring, resulting in a faster reaction time (RT) to the onset of the stimulus that occurs there (e.g., Osugi, Hayashi, & Murakami, 2016; Yantis & Jonides, 1984). However, such facilitation does not persist for an extended period. In several hundred milliseconds, the same location is known to be lowered in attentional priority, hence “inhibited,” compared to other locations, often termed “uncued” locations, to which exogenous attention has not recently been afforded; this inhibitory phenomenon, discovered by Posner and Cohen (1984), was later named “inhibition of return (IOR)” by Posner, Rafal, Choate, and Vaughan (1985). 

In laboratories, IOR may be operationally defined as a longer onset RT required for observers to react to a stimulus at a recently cued location than that at an uncued location. In a typical experimental setup of IOR, an uninformative peripheral cue is presented briefly, and observers are instructed to rapidly react to the onset of a subsequently presented target, which may appear at either the cued or uncued location. When the cue-target onset asynchrony (CTOA) is sufficiently short (<150 ms), the RT is typically faster at the cued location. At longer CTOAs (>300 ms), the RT is increased at the cued location, and the RT difference between the cued and uncued locations tends to increase with increasing CTOA, resulting in a sluggish time course of IOR (Posner & Cohen, 1984) (for reviews, see Klein, 2000; Lupiáñez, Klein, & Bartolomeo, 2006; Samuel & Kat, 2003; Taylor & Klein, 1998). IOR is known to occur not only in simple tasks involving reactions to stimulus onset, but also in choice reaction tasks when the CTOA is sufficiently long (Lupiáñez, Milan, Tornay, Madrid, & Tudela, 1997; Lupiáñez, Milliken, Solano, Weaver, & Tipper, 2001). The disengagement of attention from locations to which exogenous attention has once been drawn is thought to be important for IOR to be observed (Faust & Balota, 1997; Law, Pratt, & Abrams, 1995; Pratt, 1995; Pratt & Fischer, 2002), whereas some studies indicate that such attentional disengagement may not be necessary for perceptual/physiological signatures related to IOR to be observed (Chica & Lupiáñez, 2009; Chica, Lupiáñez, & Bartolomeo, 2006; Rafal, Davies, & Lauder, 2006; Martín-Arévalo, Chica, & Lupiáñez, 2014). The importance of disengagement suggests that the mechanism of IOR may discourage reorienting toward previously attended locations to make the visual search as efficient as possible (Klein, 2000; Klein & MacInnes, 1999; Tipper, Weaver, Jerreat, & Burak, 1994). 

What process is actually involved in the so-called “inhibition” when IOR occurs? IOR is almost exclusively measured in terms of RT, and the response is executed with a manual key press (e.g., Posner & Cohen, 1984) or eye movement (e.g., Abrams & Dobkin, 1994; Rafal, Calabresi, Brennan, & Sciolto, 1989). Both response modes include at least two consecutive processes. The first is a perceptual process, starting from visual inputs and ending with the perception of the stimulus. The second is a motor process, triggered by the perception and involving motor execution to move one's hand or eye to accomplish the requested reaction. An intermediate process, decision making, is also considered to occur in-between and, particularly in choice reaction tasks, to make a substantial contribution in the selection of one of the alternatives. Various studies have attempted to determine the process involved in the “inhibition” in IOR, suggesting that if decision making has only a negligible impact on RT changes in simple onset reaction, either the perception or action processes can be affected by exogenous attention, depending on the response mode (Hunt & Kingstone, 2003; Taylor & Klein, 2000). When the response is executed with eye movement, typically via a saccade, modulations in the motor process are likely to occur as implicated in psychophysical (Hunt & Kingstone, 2003; Kingstone & Pratt, 1999; Rafal, Egly, & Rhodes, 1994; Taylor & Klein, 2000) and neurophysiological data (Dorris, Klein, Everling, & Munoz, 2002; Lepsien & Pollmann, 2002; Sapir, Soroker, Berger, & Henik, 1999). On the other hand, the difference in the RT between the cued and uncued locations is thought to reflect some difference within an attentional process, which may be part of the perceptual process in the above nomenclature, when manual responses are used (Hunt & Kingstone, 2003; Kingstone & Pratt, 1999; Pratt, Kingstone, & Khoe, 1997; Reuter-Lorenz, Jha, & Rosenquist, 1996). However, these studies consistently use RT as a manifestation of IOR, which inevitably involves both the perceptual and motor processes. 

Several studies measuring IOR use indices other than RT in an attempt to reveal what is actually altered in IOR. For example, Handy, Jha, and Mangun (1999) measured not only RT but also the orientation discrimination accuracy for a bar stimulus and demonstrated that accuracy as well as RT was affected when IOR occurred. Ivanoff and Klein (2006) used a speed-accuracy trade-off procedure and showed that shape discrimination sensitivity was lower at the cued than uncued location. Chica, Taylor, Lupiáñez, and Klein (2010) showed that the effect of IOR on the accuracy of color discrimination occurred only when the observer's eye movements were prevented. Another recent study using a Gabor patch revealed a modulation in contrast sensitivity, which was reported to be a psychophysical manifestation of the perceptual process responsible for IOR (Sapir, Jackson, Butler, Paul, & Abrams, 2014). These studies demonstrate alteration of certain perceptual aspects of the target when IOR occurs. However, it is not easy to convert the dependent variables in these studies, namely discriminability and sensitivity, into the cost in time, namely the difference in RT between the cued and uncued locations, as measured in conventional IOR studies. Therefore, it is not guaranteed that these perceptual consequences actually produce RT differences. 

To examine the possible involvement of the perceptual process in IOR, other studies have used temporal measures. If the perceptual process is modulated in IOR, it is predicted that the onset of the target is perceived later at the cued than the uncued location. Thus, several studies used temporal measures related to the subjective onset time, such as the perceptual judgment of temporal order between two stimuli (Maylor, 1985; Posner et al., 1985; for a review, see Klein, Schmidt, & Muller, 1998) and a visual illusion in which illusory motion is consistently perceived in a static bar (Schmidt, 1996). In these studies, although exogenous attention could lead to facilitatory effects at short CTOAs, both in the subjective temporal order (Shore, Spence, & Klein, 2001) and in the motion illusion (Hikosaka, Miyauchi, & Shimojo, 1993), no effects occurred at longer CTOAs in favor of IOR, indicating that IOR is not correlated with these two perceptual phenomena (Klein et al., 1998; Maylor, 1985; Posner et al., 1985; Schmidt, 1996; but see also Redden, Hurst, & Klein, 2018). Another study using a speed-accuracy trade-off procedure has shown that the processing speed of the target may be slower at the cued than uncued location when the target remains visible until response (Zhao, Heinke, Ivanoff, Klein, & Humphreys, 2011), but such modulation in processing speed is not observed when the target is presented only briefly, the very condition in which a slower RT at the cued than uncued location is evident (Lupiáñez et al., 1997; Reuter-Lorenz et al., 1996). Another type of temporal measure, perceived duration, was used in a recent study (Osugi, Takeda, & Murakami, 2016), demonstrating that the stimulus duration appeared to be shorter at the cued location than at the uncued location. In this study, however, there are three possible accounts: delayed subjective onset, shortened subjective duration, and quickened subjective offset at the cued rather than uncued location. In summary, although various studies about IOR have suggested modulation in perceptual processing, there is still little evidence for delayed perception of the stimulus onset at the cued location. 

To address this question, it is necessary to obtain the RT with a conventional measurement such as the manual simple reaction as a replication of the conventional IOR, and also to measure some aspect of temporal vision that relates to the perception of stimulus onset and does not stem from motor processing. As one of the most appropriate visual phenomena for this purpose, we took advantage of the flash-lag effect (FLE), which has never been used in IOR studies. 

The FLE refers to the illusion that emerges as a perceived position offset between moving and flashed stimuli (e.g., Nijhawan, 1994, 1997; for a review, see Eagleman & Sejnowski, 2007; Hubbard, 2014; Krekelberg & Lappe, 2001; Nijhawan, 2002; Whitney, 2002). In a typical display, as a flashed bar stimulus is presented in physical alignment with a moving stimulus, the position of the flash stimulus appears to lag behind the moving stimulus. It is highly likely that the FLE is an adequate phenomenon for the investigation of the relationship between IOR and the perceptual process related to the stimulus onset for the following reasons. First, the FLE purely reflects the consequence of the perceptual process managing interactions between the moving and flashed stimuli, and is fully independent of the motor process (Khurana, Watanabe, & Nijhawan, 2000). Second, although the phenomenal description of the FLE is typically based on spatial position, numerous studies have unequivocally indicated that the degree of FLE is better described in terms of the temporal lag that is necessary for cancelling the apparent misalignment to establish the perceptual alignment between the flashed and moving stimuli (e.g., Kirschfeld & Kammer, 1999; Murakami, 2001; Purushothaman, Patel, Bedell, & Ogmen, 1998; Whitney & Murakami, 1998). Third, the FLE when viewed as a time index has a sufficient temporal resolution for the investigation of IOR. The FLE typically results in a 60–80 ms difference between the moving and flashed visual stimuli in terms of effective “latency”; that is, they establish the perceptual alignment when the flash is presented earlier than the moving stimulus by this amount of time (Krekelberg & Lappe, 1999; Whitney & Murakami, 1998). When IOR occurs, the difference in RT between the cued and uncued locations is typically 10–30 ms (Osugi et al., 2016; Pratt et al., 1997; Reuter-Lorenz et al., 1996). Such a difference in time is detectable in the FLE experiment with statistical significance (Baldo, Kihara, Namba, & Klein, 2002; Baldo & Namba, 2002; Ichikawa & Masakura, 2010). Fourth, the FLE is explained in terms of a delay in the perceptual onset of the flashed stimulus (e.g., Patel, Ogmen, Bedell, & Sampath, 2000; Purushothaman et al., 1998; Whitney & Murakami, 1998). Therefore, only by measuring both the onset RT and the degree of FLE with the same display does it become possible to test whether the perceived time of the stimulus onset can be modulated when IOR is observed. This approach also makes it possible to perform a quantitative comparison between the degree of IOR as determined by the difference in RT and another visual measure of perceived time as determined by the FLE experiment. 

In the present study, we measured the degree of FLE at the cued and uncued locations in a visual display in a typical setup wherein a substantial degree of IOR was expected to occur. More specifically, we presented a rotating bar stimulus and a briefly flashed stationary bar target in the periphery, and, using the same display, we measured the onset RT to the flashed target in one session, and measured the degree of FLE between the rotating bar and the flashed target in another session. If there is relative deterioration of the perceptual process related to the stimulus onset in IOR, not only the onset RT, but also the degree of FLE should differ between the cued and uncued locations. If, on the other hand, the delay of some other process than the perceptual process related to the stimulus onset is the determinant of IOR, no significant modulation in the degree of FLE between the cued and uncued locations will be observed. 

Methods

Participants

Participants were fifteen adults with normal or corrected-to-normal visual acuity. All participants were naïve to the purpose of the experiment. The participants provided informed written consent. The experiment was conducted following the Declaration of Helsinki and was permitted by the ethics committee of the Graduate School of Humanities and Sociology, The University of Tokyo. 

Apparatus

Visual stimuli were generated with a computer (Apple MacPro Mid 2010) and displayed on a CRT display (Mitsubishi Electric Diamondtron M2 RDF223G) with a spatial resolution of 1600 × 1200 pixels and a refresh rate of 100 Hz. The luminance profile of the display was gamma-corrected. The experimental programs were written in MATLAB R2013a (MathWorks, Natick, MA) with the Psychophysics Toolbox extension version 3.0.11 (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). Each observer's head was constrained in terms of viewing distance by using a chin rest. Observers viewed the visual stimuli with both eyes open and at a distance of 57.3 cm from the display in a dark room. 

Stimuli

A snapshot of the visual display is presented in Figure 1. The background (40° wide and 30° high) was uniform gray (27.5 cd/m²). The stimuli described below were all black (0.16 cd/m²), with the exception of peripheral and central cues (see Procedure). As a fixation point, a bull's eye, which concentrically comprised of a smaller circle of 0.4° in diameter and a larger circle of 0.8° diameter, was displayed at the center of the screen. A thick circle was presented at the center, concentrically with the fixation point. Two other thick circles were also presented to the left and right of the central circle. The diameter of these three circles was 3.25° and their line width was 0.25°. The center-to-center distance between the central circle and each of the peripheral circles was 5.625°. Each of the peripheral circles had two outward bars, with line lengths and widths of 0.75° and 0.25°, respectively. These outward bars at each circle were collinear to each other with their interpolation passing through the center of the circle, and the orientations formed by the aligned bars were always orthogonal to each other between the left and right circles. Another bar, 2.0° long and 0.25° wide, served as the target. It was presented inside either the left or right peripheral circle in each trial. The target was oriented vertically, horizontally, or diagonally (±45°). Among these orientations, the 45° tilted target appeared only in catch trials that were not included in data analysis. 

Figure 1

 

Visual stimulus. Throughout each trial, three large circles were presented; one was placed concentrically to the fixation point (the bull's eye), and the remaining two were to the left and right of the fixation point. Each of the peripheral circles had two outward bars, which were rotating about the center of each circle throughout each trial. A stationary bar was briefly flashed inside either the left or right circle and served as the target.

Procedure

A schematic of the procedure is provided in Figure 2. Throughout each trial, the outward bars at the peripheral circles always rotated clockwise about the center of each circle at 33.3 rpm whenever the circles were present. Each trial started with the presentation of the fixation point and the three circles with the rotating bars, and, 740 ms later, the peripheral circle and rotating bars at either the left or right side abruptly turned white (52.3 cd/m²) and remained for 50 ms, reverted to black for the next 50 ms, turned white again for the next 50 ms, and reverted to black. This manipulation produced the perception of double twinkles in the periphery that were expected to function as a peripheral cue to draw exogenous attention (following a recent study on IOR and perceived duration; Osugi et al., 2016). After an interval of 350 ms, the central circle turned white and remained for 250 ms, to serve as a central cue, which was introduced to ensure that attention initially drawn to the cued location was then disengaged and returned to the center of the display (Faust & Balota, 1997; Law et al., 1995; Pratt, 1995; Pratt & Fischer, 2002). After one of various intervals (110–430 ms) from the cessation of the central cue, the target was presented inside either the left or right peripheral circle for 30 ms, and the observer was asked to respond by pressing a key (see below for the specific tasks). The next trial started 1 s after the observer's response. The observers were instructed to fixate the bull's eye at the center of the screen throughout each trial. The CTOA was defined as the time from the onset of the peripheral cue to the onset of the target, and varied among nine levels (860, 900, 940, 980, 1,020, 1,060, 1,100, 1,140, and 1,180 ms). Each level was sufficiently long for the occurrence of IOR (Lupiáñez et al., 1997, 2001). 

Figure 2

 

Time course of the stimulus presentation. Throughout each trial, the outward bars at the peripheral circles rotated clockwise. A peripheral cue was delivered in the form of double twinkles at either the left or right circle. A central cue was successively presented in the form of a single twinkle so as to disengage attention from the cued location. The target was flashed briefly with various cue-target onset asynchronies (CTOA).

Design

There were two conditions with respect to the relationship between the peripheral cue and the target (Figure 3). In the “cued” condition, the target was presented at the same side as the peripheral cue. In the “uncued” condition, the target was presented at the opposite side. These two conditions were mixed at equal probability in a random order within a block, such that the peripheral cue was uninformative about the location of the impending target. 

Figure 3

 

Design. There were two conditions. In the cued condition, the target was presented at the same location as the peripheral cue. In the uncued condition, the target was presented at the location opposite to the peripheral cue. These conditions were mixed and presented in equal probability, so that the peripheral cue was uninformative of the target location.

There were two tasks tested in separate sessions: a simple reaction to the onset of the target for RT measurement, and an orientation judgment for FLE measurement. Each session had nine blocks, of which the first was run as a practice block and was not included in the data analysis. In each block, the data were acquired for 72 trials: 2 peripheral cue locations × 2 target locations × 2 target orientations (vertical and horizontal) × 9 CTOAs; they were presented in a random order and also mixed with eight catch trials with a tilted target. 

Simple reaction

In the simple reaction task, observers were instructed to press the space key of the computer keyboard as soon as they detected the target. In the data analysis, the RT means were calculated separately for the cued and uncued conditions within each observer. In total, there were 288 trials for each condition for each observer. RTs of <100 ms or >1,000 ms were judged as erroneous and excluded from the data analysis (4.61% of all trials). 

Orientation judgment

The purpose of the orientation judgment task was to determine the degree of FLE with the method of constant stimuli. Observers were instructed to indicate whether the orientation formed by the rotating bars was inclined clockwise or counterclockwise relative to the orientation of the target (which was either vertical or horizontal) at the moment of target onset. The physical orientation relationship between the rotating bars and target covaried with CTOA; when it was 1,060 ms, the rotating bars and target were physically aligned (Figure 4). Relative to this CTOA, the varieties in time lag we prepared for the method of constant stimuli were {−200, 160, −120, −80, −40, 0, 40, 80, 120 ms} in time, and {−40°, −32°, −24°, −16°, −8°, 0°, 8°, 16°, 24°} in relative orientation, with the positive values indicating that the rotating bars had physically moved ahead at the target's arrival. In each block, each level was tested four times for both the cued and uncued conditions. In total, each level for each condition was tested 32 times for each observer. 

Figure 4

 

Relationship between cue-target onset asynchrony and the relative angle between the target and rotating bars. When the cue-target onset asynchrony (CTOA) was 1,060 ms, the target and rotating bars were physically aligned. The figure indicates the time relative to this time of physical alignment. When this value was negative, the rotating bars were physically counterclockwise to the target angle.

For each observer, the rate of judging the rotating bars as clockwise (i.e., observed to have moved ahead) was plotted against the physical time lag. We fitted the data with the cumulative normal distribution function with the maximum likelihood method using the Palamedes toolbox extension for MATLAB (Prins & Kingdom, 2009). The mean and standard deviation were free parameters whereas the floor and ceiling rates were fixed at 1/(2 × 32) and 1 – 1/(2 × 32), respectively (Prins, 2012). The mean of the best-fit function was taken as the time lag when the rotating bars and target achieved the point of subjective alignment (PSA), with the negative PSA values indicating the occurrence of the conventional FLE. In addition, the difference between the PSA and the time lag corresponding to the 0.75 probability of judging the rotating bars as clockwise was determined as the just noticeable difference (JND) in relative orientation. 

Among the fifteen observers, two observers consistently exhibited the probability of >0.5 of judging the rotating bars as clockwise and failed to exhibit a systematic increase in response rate against time lag; hence, these observers' orientation discrimination performances did not meet the criteria for our study purpose. As the data for these observers could not be fitted with the cumulative normal in any meaningful manner, they had to be excluded. 

Results

Simple reaction time

Figure 5a presents the results of the simple reaction task in the form of the interobserver mean of RTs averaged across all CTOAs. The effect of cuing on RT, which was quantified by the RT in the cued condition minus the RT in the uncued condition, is also shown. A paired t test revealed a significant difference between conditions, t(12) = 5.89, p < 0.0001, r = 0.86. The RT in the cued condition was longer than that in the uncued condition, indicating that a typical IOR occurred. The degree of IOR, as characterized by the effect of cuing on RT (approximately 15 ms), was also comparable to that in previous studies (Posner & Cohen, 1984). 

Figure 5

 

Results. The interobserver mean is provided for each condition. The error bars indicate the standard error of the mean. (a) Results of simple reaction time (RT) in the measurement of IOR. The effect of cuing was quantified by calculating the result in the cued condition minus the result in the uncued condition for each individual and then averaging the data across observers; thus, the error bar no longer contains individual variability in absolute RT. (b) Results of the point of subjective alignment (PSA) in the flash-lag effect (FLE) measurement. The effects of cuing was quantified by the result in the uncued condition minus the result in the cued condition. (c) The just noticeable difference (JND) in the FLE measurement. The effect of cuing was quantified by the result in the cued condition minus the result in the uncued condition.

A significant main effect of CTOA, F(8, 96) = 24.35, p < 0.0001, Display Formula\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\unicode[Times]{x1D6C2}}\)\(\def\bupbeta{\unicode[Times]{x1D6C3}}\)\(\def\bupgamma{\unicode[Times]{x1D6C4}}\)\(\def\bupdelta{\unicode[Times]{x1D6C5}}\)\(\def\bupepsilon{\unicode[Times]{x1D6C6}}\)\(\def\bupvarepsilon{\unicode[Times]{x1D6DC}}\)\(\def\bupzeta{\unicode[Times]{x1D6C7}}\)\(\def\bupeta{\unicode[Times]{x1D6C8}}\)\(\def\buptheta{\unicode[Times]{x1D6C9}}\)\(\def\bupiota{\unicode[Times]{x1D6CA}}\)\(\def\bupkappa{\unicode[Times]{x1D6CB}}\)\(\def\buplambda{\unicode[Times]{x1D6CC}}\)\(\def\bupmu{\unicode[Times]{x1D6CD}}\)\(\def\bupnu{\unicode[Times]{x1D6CE}}\)\(\def\bupxi{\unicode[Times]{x1D6CF}}\)\(\def\bupomicron{\unicode[Times]{x1D6D0}}\)\(\def\buppi{\unicode[Times]{x1D6D1}}\)\(\def\buprho{\unicode[Times]{x1D6D2}}\)\(\def\bupsigma{\unicode[Times]{x1D6D4}}\)\(\def\buptau{\unicode[Times]{x1D6D5}}\)\(\def\bupupsilon{\unicode[Times]{x1D6D6}}\)\(\def\bupphi{\unicode[Times]{x1D6D7}}\)\(\def\bupchi{\unicode[Times]{x1D6D8}}\)\(\def\buppsy{\unicode[Times]{x1D6D9}}\)\(\def\bupomega{\unicode[Times]{x1D6DA}}\)\(\def\bupvartheta{\unicode[Times]{x1D6DD}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bUpsilon{\bf{\Upsilon}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(\def\iGamma{\unicode[Times]{x1D6E4}}\)\(\def\iDelta{\unicode[Times]{x1D6E5}}\)\(\def\iTheta{\unicode[Times]{x1D6E9}}\)\(\def\iLambda{\unicode[Times]{x1D6EC}}\)\(\def\iXi{\unicode[Times]{x1D6EF}}\)\(\def\iPi{\unicode[Times]{x1D6F1}}\)\(\def\iSigma{\unicode[Times]{x1D6F4}}\)\(\def\iUpsilon{\unicode[Times]{x1D6F6}}\)\(\def\iPhi{\unicode[Times]{x1D6F7}}\)\(\def\iPsi{\unicode[Times]{x1D6F9}}\)\(\def\iOmega{\unicode[Times]{x1D6FA}}\)\(\def\biGamma{\unicode[Times]{x1D71E}}\)\(\def\biDelta{\unicode[Times]{x1D71F}}\)\(\def\biTheta{\unicode[Times]{x1D723}}\)\(\def\biLambda{\unicode[Times]{x1D726}}\)\(\def\biXi{\unicode[Times]{x1D729}}\)\(\def\biPi{\unicode[Times]{x1D72B}}\)\(\def\biSigma{\unicode[Times]{x1D72E}}\)\(\def\biUpsilon{\unicode[Times]{x1D730}}\)\(\def\biPhi{\unicode[Times]{x1D731}}\)\(\def\biPsi{\unicode[Times]{x1D733}}\)\(\def\biOmega{\unicode[Times]{x1D734}}\)\(\eta _p^2\) = 0.67, was found in the two (cued and uncued locations) × nine (CTOAs) repeated-measures analyses of variance (ANOVA), indicating that the RT was reduced as the CTOA increased (Supplementary Figure S1). This tendency has been observed in several previous studies on IOR (Faust & Balota, 1997; Gibson & Egeth, 1994; Rafal et al., 1989). Importantly, the main effect of location was also significant, F(1, 12) = 38.04, p < 0.0001, Display Formula\(\eta _p^2\) = 0.76, whereas the interaction between location and CTOA was not, F(8, 96) = 1.65, p = 0.12, Display Formula\(\eta _p^2\) = 0.12. These results indicate that no systematic differences across CTOAs were observed in the RT difference between the cued and uncued locations, thus confirming that the CTOAs we selected were within the appropriate range for IOR to occur. 

Flash-lag effect

Figure 5b presents the results of the orientation judgment task in the form of the interobserver mean of the PSAs. The effect of cuing on PSA, which was quantified by the PSA in the uncued condition minus the PSA in the cued condition, is also shown. One sample t tests revealed that the PSAs in both conditions were significantly smaller than 0, t(12) = 5.20, p = 0.00022, r = 0.83 for the cued condition, t(12) = 4.77, p = 0.00046, r = 0.81 for the uncued condition, indicating the occurrence of a conventional FLE, i.e., the flashed target appeared to lag behind the rotating bars when they were in physical alignment (e.g., Nijhawan, 1994). Importantly, the paired t test revealed a significant difference in the PSA between conditions, t(12) = 4.65, p = 0.00056, r = 0.80. Thus, the FLE was greater in the cued condition than in the uncued condition. Hence, in the cued condition, the rotating bars appeared to have moved farther ahead at the subjective onset of the flashed target; that is, the flashed target in the cued condition was perceived at a later time than that in the uncued condition. Therefore, the difference in the PSA between conditions in this orientation judgment task suggests that the subjective onset of the target was delayed at the cued location relative to the uncued location, which was consistent with the difference in onset RT obtained in the simple reaction task. 

Figure 5c provides the interobserver mean of the JNDs. The effect of cuing on the JND, which was quantified by the JND in the cued condition minus the JND in the uncued condition, is also shown. A paired t test revealed a significant difference between conditions, t(12) = 2.55, p = 0.026, r = 0.59, indicating poorer discriminability in the cued condition than in the uncued condition. This is consistent with the notion that when IOR occurs, the cued location receives less attention, and thus should result in relatively compromised visual sensitivity as predicted from previous studies demonstrating better sensitivities at attended locations (Carrasco & McElree, 2001; Sapir et al., 2014). 

The RT was recorded in the orientation judgment task as well, although speeded reaction was not explicitly requested. Although the RT in the cued condition (837.0 Display Formula\( \pm \) 31.6 ms) tended to be longer than that in the uncued condition (808.5 Display Formula\( \pm \) 29.3 ms), the difference was not statistically significant, t(12) = 2.06, p = 0.061, r = 0.51. 

Correlations between simple reaction time and the degree of flash-lag effect

We further examined interobserver correlations between the effects of cuing on RT and each index of FLE. A scatterplot of the effect of cuing on the PSA against the effect of cuing on RT is presented in Figure 6a. Pearson's correlation coefficient was significant, r = 0.592, t(11) = 2.44, p = 0.033, and the linear regression model was well fit to the data, y = 1.52x + 7.48, R² = 0.351, thereby supporting the statistical significance of the slope of the linear model, and the intercept of the model did not significantly differ from 0, t(11) = 0.70, p = 0.50, r = 0.21. In addition, a scatterplot of the effect on the JND against the effect on RT is provided in Figure 6b. Pearson's correlation coefficient was significant, r = 0.643, t(11) = 2.78, p = 0.018, and the linear regression model was well fit to the data, y = 1.95x – 9.40, R² = 0.413, thereby supporting the significance of the slope of the model, and the intercept did not significantly differ from 0, t(11) = 0.79, p = 0.45, r = 0.23. These comparisons clearly demonstrate that the observer who exhibited a larger difference in RT between conditions yielded a larger change in the degree of FLE and discriminability, indicating the possibility that the differences in RT, PSA, and JND reflected a change within a common process. 

Figure 6

 

Correlations between the effects of cuing on the reaction time, point of subjective alignment, and just noticeable difference. The effect of cuing was determined as the difference between the cued and uncued conditions in each index. More specifically, the effects of cuing on reaction time (RT) and just noticeable difference (JND) were quantified by the result in the cued condition minus the result in the uncued condition. In contrast, the effect of cuing on the point of subjective alignment (PSA) was quantified by the result in the uncued condition minus the result in the cued condition. (a) The scatterplot between the effect of cuing on RT and that on the PSA. (b) The scatterplot between the effect of cuing on RT and that on the JND.

Discussion

The present study investigated whether the degree of FLE varied for a flashed target when IOR was also induced by an uninformative peripheral cue. The measurement of simple reaction time confirmed that the onset RT was indeed longer at the recently cued location than at the uncued location, which is consistent with the typical observations in conventional IOR experiments. The measurement of the orientation judgment between the constantly rotating bars and the abruptly flashed target revealed that the degree of FLE as determined by the physical orientation difference at the PSA was larger at the recently cued location than at the uncued location, indicating that the target subjectively appeared later at the cued location than at the uncued location. Furthermore, these two independent measures had a positive interobserver correlation. In addition, the JND in the FLE measurement was also larger at the cued location than at the uncued location. In short, not only the RT but also the degree of FLE was modulated in the display in which IOR was induced by an uninformative peripheral cue. 

Some previous studies have reported the modulation in the perceptual process in situations of IOR on the basis of changes in stimulus detectability/discriminability (Chica et al., 2010; Handy et al., 1999; Ivanoff & Klein, 2006; Sapir et al., 2014), processing time in a speed-accuracy trade-off procedure (Zhao et al., 2011), and perceived duration (Osugi et al., 2016). In contrast, other studies have demonstrated no effect on perceptual phenomena (Klein et al., 1998; Maylor, 1985; Posner et al., 1985; Schmidt, 1996; but see also Redden et al., 2018). While none of these studies provided evidence for alteration in the perceptual process related to the subjective onset time of a brief stimulus in situations of IOR, the present experiments clearly demonstrated a delay of perceived onset in a purely perceptual phenomenon, i.e., FLE enhancement. 

The relationship between attention and the FLE is also notable. The degree of FLE is known to be affected by the amount of attention afforded to the stimulus (Sarich, Chappell & Burgess, 2007; Shioiri, Yamamoto, Oshida, Matsubara, & Yaguchi, 2010), as well as the predictability of the stimulus (Baldo & Namba, 2002; Namba & Baldo, 2004; Vreven & Verghese, 2005). Some studies have suggested that the manner in which the FLE itself occurs can be partly explained in terms of attention (Baldo et al., 2002; Baldo & Klein, 1995; Khurana et al., 2000). The present results can be interpreted as another instance of psychophysical findings demonstrating the relationship between attention and the FLE. However, the present study used an uninformative peripheral cue and focused on transient exogenous attention and its subsequent disengagement, whereas previous studies have primarily focused on endogenous attention. Thus, considering the notion that endogenous attention and exogenous attention have distinct characteristics (e.g., Carrasco, 2011; Carrasco & Barbot, 2014; Chica, Bartolomeo, & Lupiáñez, 2013; Klein, 2009), the present study provides novel information regarding the relationship between attention and the FLE. 

Next, let us consider the possible relationship between the proposed underlying mechanisms of the FLE and the occurrence of IOR, which may aid our understanding of why the FLE should be larger at the cued than at the uncued location. Various mechanisms have been proposed, such as motion extrapolation (Nijhawan, 1994, 1997) and postdiction (Eagleman & Sejnowski, 2000). Among those, there are two seemingly influential mechanisms at present. One is the differential latency model (Murakami, 2001; Patel, Ogmen, Bedell, & Sampath, 2000; Purushothaman et al., 1998; Whitney & Cavanagh, 2000; Whitney, Cavanagh & Murakami, 2000; Whitney, Murakami, & Cavanagh, 2000; Whitney & Murakami, 1998), which is based on the concept that the latency of visual processing for flashed stimuli is longer than that for moving stimuli. Due to this delay in processing, more time is needed for a flashed stimulus to reach our consciousness, resulting in the flash “lag” in time. The differential latency model can be interpreted as a model of the characteristics of the perceptual process for flashed stimuli (i.e., the target in the present study). Given that the onset latency of the flashed stimulus becomes longer at the recently cued and then attentionally disengaged location than at the uncued location when IOR occurs and that the processing latency of the preexisting bars in constant rotation are immune to IOR, the FLE originating from the latency difference should also be larger at the cued location, as was observed in the present study. 

Another influential model, namely the temporal averaging model (Brenner & Smeets, 2000; Krekelberg & Lappe, 2000a, 2000b; Lappe & Krekelberg, 1998), posits that the onset of a flashed stimulus triggers positional averaging of a moving stimulus within a certain temporal window, resulting in the moving stimulus appearing ahead of the flashed stimulus. The temporal averaging model can be interpreted as a model describing the nature of the perceptual process for moving stimuli (i.e., the rotating bars in the present study). Assuming that the sampling process is started later at the cued location than at the uncued location when IOR occurs, the temporal window of averaging can shift forward at the cued location, resulting in a larger FLE at the cued location. 

As such, both models can explain the difference in visual processing between the cued and uncued locations in accord with the occurrence of IOR and thus predict our results, making it difficult to determine which model is more plausible for the explanation of the FLE in the present study; it is even possible that both mechanisms operate, one for the flashed stimulus and the other for the moving stimulus. Because the effect of IOR spatially spreads around the recently cued location (Bennett & Pratt, 2001), both flashed and moving stimuli can be affected by attentional disengagement. In addition, although both temporal modulations as argued above predict a larger FLE at the recently cued location, the difference in RT should reflect only the modulation of the processing of the flashed target because the speeded reaction to the target onset can be made with no reference to the moving stimulus. Indeed, the difference between the cued and uncued locations was approximately twice as large when the FLE was used as an index of the effect of IOR than when the RT was used, suggesting that, while the RT reflects the modulation of the processing of the flashed target itself, the FLE reflects this modulation plus an additional modulation as a result of interactions between the flashed target and rotating bars (Figure 5a and b). 

These two types of visual processing modulations are consistent with previous notions of attention. It has been suggested that attention can enhance visual signals (Cameron, Tai, & Carrasco, 2002; Carrasco, Williams, & Yeshurun, 2002; Hikosaka et al., 1993), and such enhancement can shorten the latency of stimuli presented at exogenously attended locations (Hikosaka et al., 1993; Kirschfeld & Kammer, 2000; Shore et al., 2001). Given that this attentional modulation occurred in the present study, the differential latency model can explain the larger FLE at the recently cued and then attentionally disengaged location and the smaller FLE at the uncued location. Additionally, it has been posited that the sampling of visual information is started earlier at locations to which attention is directed (Hubbard, 2014), suggesting a relatively forward shift of the temporal window of sampling at locations to which attention is no longer directed. Given this attentional modulation, the temporal averaging model can also explain the larger FLE at the recently cued and then attentionally disengaged location. 

The change in the JND in the FLE experiment (Figure 5c) can also be explained in terms of attentional modulation. Given that a stimulus to which attention is directed becomes more discriminable (Carrasco & McElree, 2001; Giordano, McElree, & Carrasco, 2009), a stimulus at an attentionally disengaged location may become less discriminable, as was found in the orientation stimuli presented at the recently cued location in the current study. 

The present study, which quantified the IOR and FLE in the same display, has important implications for future studies. First, these phenomena may be interpreted as useful psychophysical tools to quantify the degree of attention directed to a certain stimulus and/or location. For example, several studies have used IOR as an index to determine whether exogenous attention has recently been directed to a certain stimulus, such as a singleton (Theeuwes & Godijn, 2002) and a face (Theeuwes & Van der Stigchel, 2006). In addition, the FLE can be a useful index in measuring the degree of attention that is directed to a specific location (Shioiri et al., 2010), often with smaller within-observer noises than when using onset RT as an index. Therefore, the combination of these two phenomena can offer another method of quantification of the attentional capture, dwelling, and disengagement with higher precision. Another possible future direction is the extension of the present combination between the IOR and FLE to other aspects of perceptual processing. For example, both the FLE and IOR have been observed in auditory and audiovisual processing (for the FLE, Alais & Burr, 2003; Arrighi, Alais, & Burr, 2005; Hine, White, & Chappell, 2003; for the IOR, Spence & Driver, 1998). In addition, there are several lines of research regarding the relationship between each phenomenon and the retinal and environmental coordinates of vision (for the FLE, Cai, Jacobson, Baloh, Schlag-Rey, & Schlag, 2000; Nijhawan, 1997, 2001; Schlag, Cai, Dorfman, Mohempour, & Schlag-Rey, 2000; Watanabe & Yokoi, 2007; for the IOR, Abrams & Dobkin, 1994; Tipper, Driver, & Weaver, 1991; Tipper et al., 1994). Using the paradigm of the present study may enable us to disentangle these characteristics of perceptual processing in more detail while isolating the effect on the perceptual process from those on the decision-making and motor processes. 

The limitation of the present study and remaining questions for future studies should be mentioned. We measured the onset RT and the degree of FLE in separate sessions. Although we used an identical physical display between sessions, the observer's mental state might have differed, somehow affecting the behavioral data, as in Folk, Remington, and Johnston's (1992) study indicating mental-state dependence of attentional capture. This possibility makes it difficult to refute the concern that the IOR during the simple reaction task and that during the orientation judgment task involve two qualitatively different processes. Also, two major accounts have emerged from studies examining the relationship between eye-movement state and IOR type. On the one hand, some studies have argued that the oculomotor system pertains to some types of IOR; accordingly, a perceptual process is modulated when fixation is maintained, whereas a motoric process is modulated when eye movements are made to a cued location (Chica et al., 2010; Taylor & Klein, 2000). On the other hand, several studies have argued that the activation state of the reflexive oculomotor system controls which type of IOR is generated; accordingly, a perceptual form is generated when the system is suppressed, whereas a motoric form is generated when the system is not suppressed (Hilchey, Dohmen, Crowder, & Klein, 2016; Hilchey et al., 2014; Redden, Hilchey, & Klein, 2016; for a review, see Klein & Redden, 2018). In either case, it has been suggested that a blocked design is not an optimal procedure to segregate these different types of IOR (Hilchey, Klein, & Ivanoff, 2012). Thus, we cannot fully exclude the possibility that the blocked design in the present study caused different internal states between tasks, hence different types of IOR. To assess this issue, there will be two possible extensions of the present study. First, it will be useful to mix two tasks, namely the simple reaction to the stimulus onset to measure the IOR and the orientation judgment to measure the FLE, within the same block. This approach will enable us to explore the possibility of the mental state confound in the present study. Second, to record and/or manipulate the observer's eye movements is also an intriguing approach. As an oculomotor equivalent of the current manual response procedure, suppose we ask observers to move their eyes to the cue and then back to the center (c.f. Chica et al., 2010), to compare the FLE/IOR between the cued and uncued locations, or to compare the FLE/IOR between prosaccade and antisaccade conditions (c.f. Redden et al., 2016). These experiments in future studies will make more comprehensive understanding about what is actually altered when IOR occurs. 

Conclusion

The present study investigated whether the perceptual process related to the subjective onset of the stimulus is modulated in IOR by utilizing an independent perceptual phenomenon termed the FLE. The results clearly indicated that not only the RT to the target, but also the FLE between the flashed target and moving stimulus was affected in the situation of IOR that occurred due to the shift of exogenous attention. The study also demonstrated that the effects of IOR on RT and FLE are correlated. Altogether, the present study showing a modulation of a purely perceptual phenomenon in a display in which IOR is shown to occur suggests that a perceptual process related to the subjective onset of the target can be modulated in the case of IOR. 

Acknowledgments

The authors thank Takayuki Osugi for constructive discussion from the start of the present study, and also thank Sohei Kito for technical supports in constructing experimental setups. This study was supported by JSPS KAKENHI Grants-in-Aid for Scientific Research on Innovative Areas (25119003 and 18H05523). 

Commercial relationships: none. 

Corresponding author: Daisuke Hayashi. 

Email: . 

Address: Faculty of Human Informatics, Aichi Shukutoku University, Japan. 

Reference