What do fast response times tell us about attentional control?

Abstract

Abstract

Abstract:

Abstract  The mental experience of attention capture has attracted a great deal of intrigue among researchers seeking to explain the interactions between stimulus-driven and goal-driven attentional control. In recent years, researchers have increasingly begun to analyze cumulative response time (RT) distributions to test modern accounts of the capture phenomenon, particularly accounts claiming that there are changes in susceptibility to distraction as a function of time. In this paper, we raise a criticism of this approach, which centers on a problematic assumption. The assumption is that variability in these distributions is primarily determined by fluctuations in the observer's internal control state (e.g., readiness for the trial). However, it is also the case that faster segments of the distributions are overrepresented by trials that are objectively easy, while slower segments are overrepresented by trials that are objectively difficult. That is, incidental aspects of the trial stimuli influence task performance independently of the observer's internal control state. Here, we demonstrate empirically that contributions of incidental stimulus factors distort cumulative RT distributions in such a way that confounds proper interpretation of experimental data. The results have implications for theoretical accounts of attentional control, and they also raise caution about the assumptions that are sometimes made when analyzing cumulative RT distributions.

Introduction

What is the extent of our ability to ignore salient, irrelevant information in our environment? Some have argued that we have strong goal-driven abilities to resist distraction (e.g., Folk, Remington, & Johnston, 1992), although we may not always employ them (Bacon & Egeth, 1994). Others have argued that stimulus-driven factors primarily control our allocation of attention (e.g., Theeuwes, 1992). In the past few decades, the research literature has amassed a wealth of conflicting data that have been taken as evidence for either one viewpoint or the other, and attempts to reconcile the data have been fraught with further controversy (e.g., see Burnham, 2007; Theeuwes, 2010). 

Recently, several researchers have adopted an explanatory framework based on the time-course of salience processing, a perspective that has attracted a great deal of intrigue. One such time-course based explanation is often referred to as the rapid disengagement account (Theeuwes, Atchley, & Kramer, 2000; Theeuwes, 2010), by which salient stimuli initially capture attention but observers are able to implement goal-driven control to rapidly disengage from the irrelevant stimulus. By this account, individuals can mask any outwardly observable effects of distraction by delaying their responses until after any potential effects of capture have been overcome. A related account, the short-lived salience account (Donk & van Zoest, 2008) similarly proposes that visual selection in the moments following the initial view of a stimulus is wholly determined by stimulus-driven salience, but this salience quickly wanes. By this account, individuals who delay their search until the salience wanes will evade distraction by salient, irrelevant stimuli. What the two accounts have in common is that time-course is critical for attentional control, and behavioral effects of distraction are generally only measurable when the responses—either manual or saccadic—are speeded (see also Hunt, von Mühlenen, & Kingstone, 2007; van Zoest, Donk, & Theeuwes, 2004; Theeuwes, 2010). 

Some of the most impactful evidence in favor of these time-course based accounts has come from analysis of cumulative response time (RT) distributions (e.g., van Zoest et al., 2004). In this analysis approach, trial data are typically Vincentized (Ratcliff, 1979), by sorting RTs into multiple ordinal bins of data, starting from the fastest RTs all the way through the slowest RTs (e.g., Ansorge, Horstmann, & Carbone, 2005; Godijn & Theeuwes, 2002; Theeuwes & Burger, 1998). For instance, one could sort RTs into quintiles, so that each bin summarizes 20% of the cumulative distribution. When examining these data, the time-course dependent accounts make very specific predictions. By the short-lived salience account, observers should experience the strongest distraction effects when they commence their search, while the initial period of salience-dominated processing is active. Since quickly commenced searches should correspond with fast RTs, then the short-lived salience account predicts that fast RTs should be accompanied by the greatest degree of distraction. In contrast, when observers withhold their search until after the initial wave of salience has waned and the potency of the distractor is reduced, then slower RTs should be accompanied by weaker distraction. Indeed, van Zoest et al. (2004) found just this pattern when measuring proportion correct for saccadic eye-movements toward the target. That is, in the fastest bin of saccadic RTs, a large proportion of saccades were directed to irrelevant distractors; proportion correct then increased gradually as a function of bin, with more than 90% of saccades correctly directed toward the target at the slowest of the five bins (see also van Zoest & Donk, 2005; van Zoest, Hunt, & Kingstone, 2010). 

However, not all researchers have corroborated this finding. For instance, Ansorge et al. (2005), using a two-alternative forced choice manual RT task in which a target discrimination was performed, found that distraction effects were actually smallest in the fastest bin of RTs (see also Ansorge & Horstmann, 2007). Other researchers who have reported such an analysis have found either smaller distraction at the fastest RT bins or at least no modulation of distraction across the bins (e.g., Costello, Madden, Shepler, Mitroff, & Leber, 2010; Gibson & Bryant, 2008). 

Taken together, it may appear unclear why the cumulative RT results are so conflicting, although some explanations have been offered. One leading explanation highlights differences in processing steps in manual versus saccadic tasks (see e.g., Hickey, van Zoest, & Theeuwes, 2010), focusing on the point that manual RTs employing a target discrimination are slowed on trials in which capture occurs; this is due to initially shifting covert attention to the distractor, then disengaging, and then shifting attention to the target. This slowing, which we can refer to as a redirection cost, causes trials in which capture occurs to be sorted to slower segments of the RT distribution, leaving trials in which capture is successfully avoided—and thus not slowed following the initial deployment of attention—in the fastest segments. Thus, some have argued that rapid disengagement and short-lived salience accounts cannot be properly tested with cumulative RT analysis in a manual response task (e.g., Hickey et al., 2010). Note that most saccadic RT studies do not suffer a redirection cost, since each trial in these studies is completed when the eyes land on any item (typically the target or distractor). This is because the dependent variable in these studies has always been the proportion of saccades to the target, rather than discrimination RT (e.g., van Zoest et al. 2004). It should also be noted that not all manual RT tasks need be susceptible to redirection costs. For instance, Hunt et al. (2007) used joystick movements to the target, yielding a dependent measure that matches the measure used in the saccadic RT studies (i.e., proportion to target). 

The existence of the redirection cost in typical manual discrimination tasks has offered a parsimonious explanation of the conflicting results of cumulative RT analyses. However, a closer look reveals that it cannot explain small or negligible capture effects at the fastest cumulative RT bins in the manual tasks. Consider that in the manual RT studies, capture at all bins is computed by examining the differences in distributions of trials expected to cause distraction versus those that are not. For instance, Ansorge et al. (2005) compared quintiled bins of trials in which the distractor appeared at the target location to quintiled bins of trials in which the distractor appeared at a different location. If capture is present, it should cause a selective slowing on the different-location trials due to the need to redirect attention to the target location. Thus, if capture is greatest when search is most rapidly commenced, these different-location trials might not end up in the fastest RT bin. However, if these trials are moved to a slower bin, the remaining trials that populate the fastest bin are necessarily those in which the search was less rapidly commenced. As a result, when comparing the fastest bin for same and different location trials, a net positive RT difference must still emerge. Thus, it remains unclear how the time course explanations of capture can be reconciled with existing results from manual RT tasks. 

In the present paper, we offer another explanation. As we will illustrate, the general enterprise of analyzing cumulative RT distributions in attention capture studies is susceptible to a potentially major pitfall. That is, because the sorting of trials by their RTs is inherently post hoc, the causes for why some RTs are fast versus slow can be greatly misinterpreted. We draw a critical distinction between two sources of variability in behavior that can influence cumulative RT analysis: (a) the observer's internal control state, or momentary readiness to perform the task (e.g., commence the search), and (b) incidental stimulus factors, or aspects of the environment that can be attributed to momentary “luckiness” (i.e., the observer will sometimes encounter trials whose stimulus aspects render them objectively easy compared to other trials). These two sources of variability are arguably confounded in cumulative RT analysis. 

The key to the problem is that incidental factors of the environment can influence behavioral outcomes independently of internal control. Consider by analogy a roulette player who bets exactly the same way prior to every spin of the wheel. On some spins, the ball will come to rest on a number that yields the gambler a substantial win, while on others the ball will rest on a number that yields a substantial loss. Few researchers are superstitious enough to believe that the gambler's spin-by-spin performance is due to anything other than incidental aspects of the environment (e.g., the rotational speed of the roulette wheel as well as the starting point, speed and finesse placed on the ball); certainly, the gambler's internal control state could not possibly affect the results. However, this kind of superstition may creep into our experimental data interpretation, specifically when we employ cumulative RT analysis. Here, we demonstrate how incidental factors, which contribute largely to the variance in behavioral outcomes (i.e., getting lucky or unlucky on each trial due to some idiosyncratic visual aspect of the task), confound interpretations of data. 

To build our argument, we focus on a typical behavioral task with a manual response, in which observers perform visual search in the face of distracting information. The left panel of Figure 1 shows the task—an adaptation of the additional singleton paradigm introduced by Theeuwes (1991) —in which the participant is asked to find the single circle among squares as rapidly as possible and demonstrate his/her success by discriminating the orientation of the line segment inside of it. On half of the trials, the participant is confronted with a distracting, task-irrelevant stimulus, in the form of a color singleton, shown in the right panel of Figure 1. While the task typically uses only the inside ring of shapes, our current adaptation adds an outside ring to boost the visual salience of the color singleton (see Theeuwes, 2004). Note that in our version of the task, neither targets nor distractors ever appear in the outside ring. It is easy to see the appeal of cumulative RT analysis in the present context. Yet, as we previewed, we are concerned about how the results of this important analysis method, and from related approaches, are interpreted. To articulate our concerns, we present Vincentized RT data collected from the additional singleton paradigm and then proceed to carefully scrutinize the results. Specifically, we aim to reveal whether incidental stimulus factors confound the analysis of Vincentized RT data, and if so, how. To preview, we will argue that susceptibility to distraction when observers are responding quickly is generally underestimated, due to the confounding influence of the incidental factors. This potentially resolves the inconsistent results of cumulative RT analysis and might be taken as support for time-course accounts such as short-lived salience and rapid disengagement. However, we ultimately will not endorse any particular theoretical framework, arguing that several critical issues remain open for well-deserved debate. 

Figure 1

 

Representation of stimuli used in the experiment. Targets and singleton distractors always appeared within the ring of items closest to fixation. See Methods for additional details.

Method

Participants

Nineteen undergraduates from the University of New Hampshire, with self-reported normal or corrected-to-normal visual acuity and normal color vision participated for course credit. The session lasted approximately 40 min. The research protocol adhered to the tenets of the Declaration of Helsinki. 

Materials

Stimuli were generated with Apple G4 or G5 computers running Mac OS X, using Matlab (Mathworks, Natick, MA), with the Psychophysics Toolbox 3 extensions (Brainard, 1997; Pelli, 1997). Displays were presented on a 19-in. VGA monitor and viewed from a typical distance of 50 cm (head position was not fixed). When present, a white fixation point occupied the center of the screen, subtending 0.23° × 0.23° visual angle. Search items consisted of two concentric rings of objects (see Figure 1). On the inside ring, six outline shapes were spaced evenly on the circumference of an imaginary circle, with each shape centered at 3.87° eccentricity from fixation. The set of six shapes was symmetrical about the vertical and horizontal axes, with three shapes appearing in the left hemifield and three in the right. On the outside ring, ten outline shapes were evenly spaced, with each shape centered at 6.94° eccentricity. These 10 shapes were also symmetrical about both vertical and horizontal axes, with five shapes appearing in the left hemifield and five in the right. In distractor absent displays, all shapes were colored green. One of these shapes—the target—was a circle, measuring 1.78° in diameter, with a stroke of 0.16°. The target shape was always presented in the inside ring. The remaining shapes were all squares, subtending 1.55° in height and width, with a stroke of 0.16°. Centered inside each of the 16 total shapes was a vertical or horizontal line segment, drawn in white, subtending 0.78°, with a stroke of 0.16°. The line segment orientation for each object was selected randomly with replacement on each trial, such that both the inside and outside rings each contained an equal number of vertical and horizontal lines. The locations of these lines were determined randomly on each trial. The distractor-present displays were identical to the distractor-absent displays, except one of the nontarget shapes in the inside ring was drawn in red. All stimuli were presented on a black background. 

Design

Three independent variables determined the basic trial characteristics. (a) Distractor presence contained two levels (present vs. absent), each occurring on 50% of the trials. (b) Target location contained six levels, one for each possible display location in the inside ring of shapes; each of these levels was presented equally often. (c) Target-distractor distance contained five levels, representing the number of item positions in the inside ring separating the two items, measured clockwise and ranging from one to five (zero was not included, as targets and distractors could never appear at the same location); each of these levels was presented equally often. This variable was “dummy coded” on distractor-absent trials. The three independent variables were factorially crossed to generate 60 minimum trial conditions, and each of these were presented twice per block to yield 120 trials. Participants each completed six main blocks, yielding 720 total trials. Presentation order of the trials was randomized within blocks. 

Procedure

Participants were instructed to search for the target circle and report the orientation of the line segment appearing inside of it. They registered their responses by pressing “1” on the number pad of a standard keyboard for vertical bars and “2” for horizontal bars. Participants were informed of the color singleton and told to try their best to ignore it, while also responding as quickly as possible while keeping errors to a minimum. 

The experiment consisted of 24 practice trials, followed by the six blocks of 120 main trials each. Trials began with the presentation of the fixation point. After 500 ms, the search array was added to the display. It remained until response, or for 2500 ms, whichever came first. Trials in which participants did not respond within 2500 ms were counted as errors. On these trials, and on trials in which the participants registered an incorrect response, an error message was displayed on the screen for 1000 ms. Successive trials were separated by a 1000-ms intertrial interval during which all stimuli were removed. Breaks were given between blocks. 

Results and discussion part I: Conventional analysis

Overall behavior

We begin with the conventional analysis of the behavioral data, in which participants' means for each condition are averaged across the session. For the analysis of RT, error trials were excluded as were trials in which RT exceeded three SD above participants' means for distractor-present and distractor-absent conditions (2.0% and 2.1% of correct trials, respectively). Results replicated the typical capture effect observed with this task (e.g., Theeuwes, 1992), where RTs on distractor present trials (M = 733 ms) were reliably slower than on distractor absent trials (M = 707 ms), yielding 26 ms of distraction, t(18) = 3.994, p = 0.0009. Accuracy was close to ceiling and did not vary between distractor present (M = 96.8%) and distractor absent (M = 96.8%) trials. 

Cumulative RT analysis

The data were next Vincentized using decile cutoffs, producing 10 ordinal bins, each summarizing 10% of the cumulative RT distribution. This procedure was done for correct trials only, and distractor-present and -absent trials were Vincentized separately, allowing for estimates of distractor interference at each bin. Overall means for each bin are plotted in Figure 2A, and distractor interference effects (i.e., distractor-present minus distractor-absent) are plotted in Figure 2B. A repeated-measures one-way ANOVA was carried out to determine if the distractor interference effect varied as a function of bin. Mauchly's test revealed unequal variances across bins, so we corrected the degrees of freedom using the Greenhouse-Geisser estimate of sphericity. This violation of sphericity was not surprising, as RTs tend to be substantially more variable at the slower tail of the distribution. (Note that the Greenhouse-Geisser correction will be used for all subsequent sphericity violations.) The ANOVA confirmed significant modulation in the interference effect across the bins, F(1.67, 30.12) = 5.321, p = 0.014. An inspection of the means suggests increasing interference with increasing bin, and trend analysis confirmed this linear relationship to be significant, F(1, 18) = 8.526, p = 0.009. We might thus summarize these data by stating that when you are responding most quickly, you are least susceptible to attentional distraction. 

Figure 2

 

(A) Vincentized RT distributions for distractor-present and -absent conditions, from the experiment. (B) Distractor interference effects, computed by subtracting distractor-present minus distractor-absent RT at each bin. Error bars indicate SE of the difference.

Results and discussion part II: Demonstrating the confounds

Why is distraction minimal at the fastest RTs? These data appear to violate accounts such as short-lived salience (Donk & van Zoest, 2008) and rapid disengagement (Theeuwes, 2010). It is tempting to infer that the fastest trials might represent a state of internal control that both rapidly commences shifts of attention and also is most immune to distraction. However, as we proposed in the Introduction, the results could have also been driven in large part by incidental stimulus factors. Specifically, trials whose incidental factors made the task objectively easy may have been sorted into the fastest RT bins, while trials whose incidental factors made the task objectively difficult may have been sorted into the slower RT bins. In the present task, there are numerous stimulus factors that can vary the difficulty of search, irrespective of the observer's task preparedness. We attempted to generate an exhaustive list of these factors, although it is likely we have overlooked a few. The factors—and the ways in which they influence cumulative RT analysis—are described below. Note that we will report a sizeable number of statistical tests, and a variety of approaches for correcting for multiple comparisons could be justified. We elected not to control for experiment-wise error and rather to leave it to the reader to evaluate the uncorrected statistics accordingly. 

Distractor-to-target distance

It has been shown previously that a distractor's ability to cause interference depends on its spatial proximity to the target (e.g., Ansorge et al., 2005; Turatto & Galfano, 2000). We examined our own data for such an effect, sorting trials into distractor-target distances of 1 (i.e., when the objects occupy adjacent positions, measured either clockwise or counterclockwise), 2 (i.e., when they are two spatial locations away), or 3 (i.e., when they are on opposite sides of the display). Results revealed a distance effect in which RT on distractor present trials decreased as the distractor-target distance increased (see Figure 3A); a one-way ANOVA across the three distances was significant, F(2, 36) = 8.685, p = 0.001. This result provides our first demonstration that RT can vary independently of the observer's internal control state. That is, the observer could not prepare any differently for a trial with a distractor-target distance of 1 compared to a distractor-target distance of 3; this is because such stimulus factors are randomized within blocks and are thus unpredictable. As such, distractor-target distance confounds the cumulative RT analysis. That is, because trials with larger distractor-target distances are objectively easier and confer faster RTs, they should be more likely to be sorted into the faster RT bins. 

Figure 3

 

RT for distractor-present and -absent trials from the experiment, plotted as a function of distractor-to-target distance (A), n – 1 target-to-current-target distance (B), n – 1 target-to-current-distractor distance (C), n – 1 distractor-to-current-target distance (D), n – 1 distractor-to-current-distractor distance (E), and previous response (F). Distractor absent data in panels A and C represent the respective means for those conditions, as these data did not have associated distance values.

We directly evaluated this prediction by computing mean distractor-target distance separately for each RT bin (see Figure 4A; note that distractor-absent trials are not included, as they do not have a corresponding distance measure). We entered these data into a one-way ANOVA (across the 10 levels of bin) for distractor present trials, and results approached significance, F(9, 162) = 1.779, p = 0.076. Trend analysis confirmed that the pattern across bins was linear, F(1, 18) = 8.288, p = 0.010. That is, distractor-target distance varied across RT bin, such that distances decreased with increasing bin. This result confirms that trials placed in different RT bins varied in their objective degree of difficulty. 

Figure 4

 

Makeup of stimulus characteristics across cumulative RT bins, from the experiment. Means distance measures are plotted at each bin for distractor to target (A), n – 1 target to current target (B), n – 1 target to current distractor (C), n – 1 distractor to current target (D), and n – 1 distractor to current distractor (E). Also shown is mean probability of response repetition at each bin (F).

Previous target to current target distance

It has been shown that targets of visual search are found more rapidly when they appear in the same location on consecutive trials, a phenomenon termed “position priming” (Maljkovic & Nakayama, 1996). In Figure 3B, we plot RT as a function of the current target's distance from the target on the previous, or n – 1, trial. We computed a two-factor ANOVA to examine the effects of distance (four levels) and distractor presence (2) on RT. Note that we will not report the main effect of distractor presence on RT in this ANOVA or subsequent multifactor ANOVAs using this independent variable, as such reporting would be repetitive—and nonindependent—from the initial t test that reported significant distractor interference. There was a significant main effect of distance, F(3, 54) = 24.865, p < 0.001, replicating the position priming effect; RT was considerably faster on trials in which the target repeated in the same location twice (i.e., distance = 0), compared to the other conditions. This position priming effect simply shows that trials with location repetitions are easier, independent of internal control. Furthermore, distractor presence interacted significantly with distance, F(3, 54) = 14.475, p < 0.001. This shows that a trial's objective difficulty (with respect to n – 1 target-to-current-target distance) varies with distractor interference and thus demonstrates again that such interference can vary independently of internal control. 

We next evaluated whether n – 1 target-to-current-target distance varied with RT bin (see Figure 4B). These distance scores were entered into a two-factor (bin × distractor presence) ANOVA, which yielded a main effect of bin, F(4.36, 78.44) = 23.185, p < 0.001. The main effect of distractor presence was not significant, nor was the interaction (both Fs < 1). The linear trend of increasing distance across bin was significant, F(1, 18) = 72.079, p < 0.001. Because trials sorted into the faster RT bins have lower n – 1 target-to-current-target distances, we again demonstrate that task difficulty was not equivalent across the RT bins. 

Previous target to current distractor distance

If the location of a previous target confers facilitated processing of the current target, then it stands to reason that it will also confer facilitated, albeit unwanted, processing of a distractor. In Figure 3C, we plot RT as a function of the current distractor's distance from the n – 1 target (for distractor-present trials only). RTs were numerically slowest when the distractors appeared in previous target locations (i.e., distance = 0), and they decreased as the distance increased. A one-way ANOVA across the four levels of distance did not reach significance, F(3, 54) = 1.983, p = 0.127, but the pattern of data suggests that easier trials—with respect to their n – 1 target-to-distractor distance—might also come along with smaller distractor interference effects, independently of internal control. 

We next evaluated whether n – 1 target-to-current-target distance varied across the RT bins (Figure 4C). A one-way ANOVA computed on the distance scores across the 10 bins was significant, F(9, 162) = 2.634, p = 0.007, and trend analysis confirmed a significant linear trend, F(1, 18) = 14.641, p = 0.001. Because distractor-present trials in the faster RT bins generally had larger n – 1 target to current distractor distances, these bins were comprised of objectively easier trials. 

Previous distractor to current target distance

In addition to the position priming effects just discussed, there also exists a phenomenon of “negative position priming,” in which processing of an object is slowed when it occupies the location of a previous distractor, presumably due to spatially specific inhibition persisting across trials (Kumada & Humphreys, 2002). In Figure 3D, we plot RT as a function of the current target's distance from the n – 1 distractor (only including trials in which a distractor was present on the n – 1 trial). A distance (i.e., n – 1 distractor to current target) × distractor presence ANOVA yielded a significant main effect of distance, F(3, 54) = 4.320, p = 0.008, replicating the results of Kumada and Humphreys (2002). Moreover, the slowing of RT at locations closest to the n – 1 distractor seemed more pronounced on distractor present trials; accordingly, the distance × distractor presence interaction approached significance, F(3, 54) = 2.252, p = 0.093. Thus, negative position priming can create objectively difficult trials, independent of internal control, that yield slower overall RT and produce greater capture effects. 

We next evaluated whether n – 1 distractor-to-current-target distance varied across the RT bins (see Figure 4D). The distance scores were entered into a two-factor (bin × distractor presence) ANOVA, which yielded a main effect of bin, F(9, 162) = 3.088, p < 0.002. The main effect of distractor presence was not significant, nor was the interaction (both Fs < 1.25). The linear trend of decreasing distance across bin was significant, F(1, 18) = 16.689, p = 0.001. Because trials in the faster RT bins had numerically larger n – 1 distractor-to-current-target distances, we conclude that the trials sorted into the faster bins were objectively easier. 

Previous distractor to current distractor distance

In addition to impacting processing of the current target, negative position priming carries consequences for the current distractor (Kumada & Humphreys, 2002). In Figure 3E, we plot distractor-present RT as a function of the current distractor's distance from the n – 1 distractor (when the n – 1 trial also contained a distractor). For reference, distractor-absent RT is also shown (for trials whose n – 1 trial contained a distractor). Results showed that RTs were fastest when distractors appeared in the n – 1 distractor location. A one-way ANOVA failed to show that distractor-present RT varied significantly as a function of n – 1 distractor-to-current-distractor distance, F(3, 54) = 1.711, p = 0.176, although the numerical trend was consistent with data from Kumada and Humphreys (2002). 

We next evaluated whether n – 1 distractor to current distractor distance varied across RT bin (see Figure 4E). A one-way ANOVA on these distance scores across the 10 RT bins failed to reach significance, F(9, 162) = 1.050, n.s. Nevertheless, the numerically smaller n – 1 distractor to current distractor distances at the fastest RT bins suggests that trials sorted into these bins may have been objectively easier. 

Previous response

Response repetitions have long been known to facilitate RT (Bertelson, 1965), and we present such a result in Figure 3F, in which RT was faster for both distractor present and distractor absent trials when the response was the same on the n – 1 trial (note that due to the 1:1 mapping of response feature to responses—i.e., vertical bars to left button and horizontal bars to right button—what we label response repetition effects can also include contributions of perceptual priming). A response type (repetition or switch) × distractor presence ANOVA confirmed the main effect of response type, F(1, 18) = 7.665, p = 0.013, although the response type × distractor presence interaction was not significant, F < 1. This result shows that repetitions of the response constitute easy trials that speed RT independently of internal control, although these faster trials are not likely to confer reductions in distractor interference. 

We next evaluated whether the frequency of response repetitions varied as a function of RT bin (see Figure 4F). Using the response repetition frequency as a dependent measure, a two-way (distractor presence × bin) ANOVA was computed, revealing a significant main effect of bin, F(3.31, 59.48) = 7.753, p < 0.001. Neither the main effect of distractor presence nor the interaction were significant (both Fs < 1.17). The linear trend of decreasing frequency of response repetition with increasing RT bin was significant, F(1, 18) = 6.864, p = 0.017. That is, trials sorted into the fastest RT bins had a greater proportion of response repetitions, and these trials were thus objectively easier. 

Additional extraneous variables

There exist additional incidental stimulus factors not described above. For instance, some trials could be considered easier than others by virtue of the target's spatial location, irrespective of the observer's internal control. Our preliminary analysis of this variable did not yield any overall biases for certain display locations, across the sample of participants. However, it is likely that each participant has his/her own idiosyncratic biases. Likewise, individuals could also vary in their abilities to ignore distractors appearing at the six display locations (although, once again, preliminary analysis did not show an overall effect). Moreover, there were several additional incidental stimulus factors that we could not measure in this study. For example, it is possible that each participant's location preferences varied further on a trial-by-trial basis. This would be particularly plausible for an observer who guesses randomly on each trial where the target will appear. He/she focuses spatial attention intently at that location and is rewarded with a fast RT and small distractor interference on the 1/6 trials in which the target fortuitously appears at the guessed location. While the propensity to guess may reflect some aspect of internal control, the outcome of guessing is completely due to chance. Thus, successful guesses would get sorted into faster RT bins more often than unsuccessful guesses, further confounding the cumulative RT analysis. It might not be possible to assess spatial guessing on a trial-by-trial basis, although it remains a cause for concern. Additional variables we did not measure or control that could further influence the cumulative RT analysis include things such as transient auditory noise and room temperature variations. 

Further considerations

We described in detail how an assortment of variables from the n – 1 trial could influence performance on trial n. However, it is also probable that more distally occurring trials (i.e., n – 2 and earlier) influenced RT. For instance, Maljkovic and Nakayama (1996) showed effects of position priming persisting for more than five trials. This consideration further compounds the concern we have raised about how incidental stimulus factors can affect cumulative RT analysis independently of internal control. 

General discussion

We began by speculating that cumulative RT analysis of the distractor interference effect confounds internal control and incidental stimulus factors. We then enumerated a series of incidental factors that distort the cumulative RT distribution independently of internal control. That is, due to the post-hoc nature of sorting trials based on RT, there is a general tendency for trials that are objectively easier to wind up in the faster segments of the RT distribution while trials that are objectively more difficult wind up in the slower segments. As a result, the presence of incidental factors necessarily and artifactually reduces distraction effects in the faster RT bins. 

The results carry important implications for theories of attention capture. A key prediction of the time-course based accounts of attentional control, such as the rapid disengagement and short-lived salience accounts is that attention capture should be greatest at the fastest RTs. Our initial results from the Vincentized data, showing weakest capture at the fastest RT bins would appear to be at odds with such a proposal, potentially drawing the argument to a standstill (see also Ansorge & Horstmann, 2007; Ansorge et al., 2005). However, our subsequent analysis of the confounds due to incidental factors strongly suggests that the capture at the fastest RT bins is underestimated. Thus, Vincentized data—without some correction applied—arguably cannot be used to present a challenge to the rapid disengagement and short-lived salience accounts. 

Given that the current analysis was carried out on a data set that employed a manual RT task with a target discrimination, one might question what implications it carries for the saccadic RT studies measuring proportion of responses to the target (e.g., van Zoest et al., 2004). When examining cumulative RT distributions, the saccadic RT studies have been assumed to be superior to manual tasks, since the dependent measure is likely to more closely correlate with the commencement of attentional shifts (see Hickey et al., 2010). However, insofar as incidental factors do exist in these studies, it is likely that they serve to yield an underestimate of capture at the fastest RT bins. Thus, while the saccadic RT studies have still consistently shown greatest capture at the fastest bins, interpretation of these data should be mindful of the potential influence of incidental factors. 

Taken together, our data might be interpreted to be consistent with time-course based accounts of attentional control that emphasize greater distractibility early in processing. Nevertheless, while we agree that distraction could be greatest at the fastest RTs after hypothetically controlling for all incidental factors, we believe other plausible interpretations of the data exist. Several criticisms of the time-course based accounts have been promoted without the need to rely on cumulative RT analysis (for summaries, see Chen & Mordkoff, 2007; de Fockert, 2010; Eimer & Kiss, 2010; Folk & Remington, 2010; Lamy, 2010; Müller et al., 2010; Nordfang & Bundesen, 2010; Rauschenberger, 2010). For example, we have argued elsewhere that slowing down while avoiding distraction need not be due to waiting for salience to subside. Rather, observers in a state of high cognitive control could simply exhibit a more conservative response strategy, such that the slower RTs could manifest at the response level of processing rather than at the level of attentional selection (Egeth, Leonard, & Leber, 2010). This interpretation is consistent with models accounting for data from Stroop and flanker interference tasks (e.g., Botvinick, Braver, Barch, Carter, & Cohen, 2001). Further research will be necessary to distinguish between the short-lived salience proposal and our conservative response threshold account. 

The implications of this work also extend beyond purely behavioral research to physiological and neuroimaging techniques, such as single-unit recording, functional magnetic resonance imaging, and electroencephalography. Much has been learned in recent years by linking trial-by-trial variability in neural activity to behavioral performance (e.g., Hahn, Ross, & Stein, 2007; Leber, Turk-Browne, & Chun, 2008; Weissman, Roberts, Visscher, & Woldorff, 2006). However, trouble could emerge in approaches where trials are sorted by neural measures, as much as it is for the approach in which trials are sorted based on RT. That is, objectively easy trials could yield more or less evoked neural activity, independently of the observer's internal control state. We have argued that one approach is to sort trials based on pretrial baseline activity instead of trial-evoked activity, since pretrial activity cannot be influenced by incidental stimulus factors (see Leber, 2010; Leber et al., 2008). 

Final remarks

It would be misguided to recommend categorically dispensing with RT distribution analysis in an effort to understand attentional control, but caution is warranted. If a researcher is interested in examining variations of the observer's internal control within sessions, he/she must carefully consider how incidental factors might influence the shape of the cumulative RT distribution. In many cases, the researcher might conclude that the independent contributions of internal control and incidental factors cannot be separated. We should note that we attempted to develop a model to exhaustively account for the contributions of all incidental factors on RT described in this study; this was done with the goal of isolating RT variance due to internal control. However, the sheer number of incidental factors involved rendered this task unachievable for us. We do not rule out the possibility that other researchers will succeed, particularly those with considerably larger data sets. Nevertheless, it is inevitable that some incidental factors will always be impossible to account for; trial-by-trial guessing, transient auditory noise, and room temperature variations, for instance, could remain as confounds. Thus, should researchers proceed with cumulative RT analysis to assess variations in internal control as a function of overall speed, the burden to prove that incidental factors do not distort the distribution may be prohibitively cumbersome. 

In conclusion, researchers stand to gain tremendous insight into the mechanisms of attentional capture by scrutinizing intra-individual variability. While the current data prompt careful consideration of how cumulative RT analysis are used, we are hopeful that efforts to resolve the intense debate regarding the time-course of attentional control will eventually succeed. 

Acknowledgments

We thank Wieske van Zoest, Andrew Heathcote, Denis Cousineau, and an anonymous reviewer for incisive comments on previous versions of this manuscript, and we also thank Justin Jungé and Rachael Gwinn for helpful suggestions. This work was supported by NSF BCS-1027054 and US-Israel BSF 2009424 to A. B. L. 

Commercial relationships: none. 

Corresponding author: Andrew B. Leber. 

Email: [e-mail]. 

Address: Department of Psychology, The Ohio State University, Columbus, OH, USA.