Temporal order judgment and simple reaction times: Evidence for a common processing system

Abstract

We present a simple reaction time (RT) versus temporal order judgment (TOJ) experiment as a test of the perception–action relationship. The experiment improves on previous ones in that it assesses for the first time RT and TOJ on a trial-by-trial basis, hence allowing the study of the two behaviors within the same task context and, most importantly, the association of RT to “correct” and “incorrect” TOJs. RTs to pairs of stimuli are significantly different depending on the associated TOJs, an indication that perceptual and motor decisions are based on the same internal response. Simulations with the simplest one-system model (J. Gibbon & R. Rutschmann, 1969) using the means and standard deviations of the RT to stimuli presented in isolation yield excellent fits of the mean RT to these increments when presented in sequence and moderately good fits of the RT when classified according to the TOJ categories. The present observation that the point of subjective simultaneity for stimulus pairs is systematically smaller than the difference in RT to each of the two increments in the same pairs pleads, however, in favor of distinct decision criteria for perception and action with the former below the latter. For such a case, standard one-system race models require that the internal noise associated with the TOJ be less than the one associated with the RT to the same stimulus pair. The present data show the reverse state of affairs. In short, data and simulations comply with “one-system–two-decision” models of perceptual and motor behaviors, while prompting further testing and modeling to account for the apparent discrepancy between the ordering of the two decisions.

Introduction

A central issue in cognitive sciences is the quest for understanding how physical stimulations give rise to conscious experience and motor behavior. The contemporary popular view advocated by Goodale and Milner (1992) posits the existence of two distinct, possibly interacting visual pathways (Goodale & Westwood, 2004): one ventral (occipitotemporal), dedicated to “vision-for-perception,” and one dorsal (occipitoparietal), dedicated to “vision-for-action”. Despite the experimental craze it gave rise to (for a recent review, see Schmidt & Vorberg, 2006), the two-pathway stand has been criticized on both anatomophysiological (e.g., Guillery, 2003, 2005; Merigan & Maunsell, 1993) and behavioral (Franz, Gegenfurtner, Bulthoff, & Fahle, 2000) grounds. Even the neuropsychological distinction between optic ataxia and visual agnosia, a cornerstone argument in favor of the perception–action dissociation (see Goodale & Humphrey, 1998), has been severely criticized (Pisella, Binkofski, Lasek, Toni, & Rossetti, 2006; Rossetti, Pisella, & Vighetto, 2003). Instead, this literature promoted convincing experimental observations supporting the inexorable link between action and perception. 

Among the numerous paradigms used to assess the perception–action relationship, the comparison between motor and perceptual latencies to the same visual events has generated perhaps the largest number of studies. In this paradigm (already suggested by Exner, 1868), the difference between simple reaction times (RTs) to two unequally salient stimuli (ΔRT) is compared with the physical delay between these same stimuli required to yield a perceptual simultaneity judgment (point of subjective simultaneity [PSS]) in a temporal order judgment (TOJ) task. In the standard TOJ task, participants are asked to specify which of two sequentially presented stimuli, S1 and S2, separated by a variable stimulus onset asynchrony (SOA), occurred first. This yields a psychometric function (percentage of “S1 seen first” responses as a function of SOA) typically fitted with a cumulative Gaussian distribution constrained by its mean (equivalent to the PSS) and variance (σ_TOJ²), with the latter being a measure directly related to observer's temporal order discrimination threshold. The equality of ΔRT and PSS is generally taken as direct evidence that motor and perceptual behaviors are determined by the same internal response and decision process (the one-system–one-decision hypothesis). For a sequence of two noninterfering signals, such a one-system–one-decision model also requires that σ_TOJ² be equal to the sum of the RT variances for each of the two stimuli (σ_RT–S1² + σ_RT–S2²). Instead, according to the two (parallel)-system hypothesis, neither ΔRT and PSS nor σ_TOJ² and σ_RT–S1² + σ_RT–S2² should be related. 

With a very few exceptions (e.g., Roufs, 1963), the literature, having used the RT–TOJ paradigm, has found that PSS and ΔRT are unequally affected by a number of stimulus manipulations (e.g., salience: Adams & Mamassian, 2004; spatial frequency: Barr, 1983; Tappe, Niepel, & Neumann, 1994; luminance rise times: Jaśkowski, 1993; stimulus duration: Jaśkowski, 1991, 1992). In particular, it was found that stimulus intensity affects ΔRT about twice more than it affects PSS (Jaśkowski, 1992; Jaśkowski & Verleger, 2000; Menendez & Lit, 1983; Roufs, 1974; but see Roufs, 1963). Such results were generally taken as supporting the two-independent-system view (Neumann, Esselmann, & Klotz, 1993; Steglich & Neumann, 2000; Tappe et al., 1994), but serial processing models positing distinct decision processes operating on the same internal response at different times have also been advocated. Sternberg and Knoll (1973), for example, proposed that RTs are triggered as the internal response evoked by the stimuli exceeds a motor threshold, whereas TOJs are based on the instant when the internal response reaches its peak value. More recently, Miller and Schwarz (2006) presented a one-system diffusion model with the motor triggering response level higher than the perceptual decision variable. 

The fact remains that the data on which either the one- or two-system views are based reveal a range of inconsistencies across studies (e.g., Roufs, 1974; Tappe et al., 1994), experimental conditions within the same study (e.g., Jaśkowski, 1992), and even across subjects within the same study and for the same experimental conditions (e.g., Jaśkowski, 1993). Critically, all these studies involved RT and TOJ experiments with unmatched stimulation conditions, for example, with RT and PSS assessed in independent sessions for one and two successive stimuli, respectively. As a consequence, RT and TOJ performances could have been subject to different response strategy and/or set effects. Here, we avoid such putative drawbacks by means of an experimental design where RT and TOJ are assessed within the same experimental session and for the same trial. This allows a trial-by-trial RT–TOJ association and, hence, the classification of RT according to the perceived order of the stimuli in the TOJ task. A trial-by-trial RT–TOJ assessment permits a direct test of the one-system model claim that RT to a sequence of two stimuli and the corresponding TOJ can be predicted from one another. If motor and perceptual responses result from independent processes, then RT and TOJ should be uncorrelated. To assess the generality of our conclusions, the present experimental design involved two sorts of visual events, that is, contrast and orientation increments applied to suprathreshold Gabor patches. Participants were shown a sequence of two such events and were required to press a key as soon as they perceived any of them and subsequently indicate which of them occurred first. 

Methods

Stimuli

The stimuli were two +45° oriented Gabor patches displayed on a 19-in. IIyama Vision Master Pro454 screen (1,024 × 768 pixels) with a 150-Hz refresh rate at 100 cm from the observer. The Gabors had a standard deviation of 2°, a spatial frequency carrier of 4 cycles/deg, and a luminance contrast of 20% and were presented at ±2° eccentricity about a white fixation cross (0.4° × 0.4°; thickness, 0.04°) along the horizontal meridian at a mean luminance of 40 cd/m ². Stimulus presentation and response recording were controlled using the Psychtoolbox (Brainard, 1997; Pelli, 1997) under Matlab. 

Either the contrast (C) or the orientation (O) of each of the two Gabors (also referred to as pedestals) were incremented during each trial by one of two possible low- and high-salience amounts as determined in preliminary experiments (see below). One increment was delayed with respect to the other (SOA), by one of seven possible SOAs, that is, −100, −66, −33, 0, 33, 66, or 100 ms. 

Procedure

Preliminary experiments

Preliminary experiments were needed to determine the low- and high-salience C and O increment values (C1, O1 and C2, O2, respectively) for each observer. Low-salience C and O increments were set at three times their detection threshold as assessed in separate blocks using two interleaved 1-up–2-down staircases that were ended after at least 15 reversals each. The increment–decrement step size was 6 dB for the first two reversals, 4 dB for the next two reversals, and 2 dB thereafter. As in the main experiment (see below), the two pedestals appeared after 1,300 ms of fixation and one of them was incremented in either C or O after a random period of 500–1,300 ms after their onset. The increment was applied randomly from trial to trial to the left- or to the right-side Gabor, and the observer was asked to indicate where the change had occurred. The Gabors were offset 1,700 ms after their onset. C and O thresholds were computed as the mean increment over the last six (or more) reversals. The procedure was repeated twice for each observer. 

The high-salience C and O increments (C2 and O2) were determined following the same procedure as that used by Adams and Mamassian (2004). The temporal sequence of events within one trial was similar to the low-saliency preliminary experiment above. Ten contrast and 10 orientation increments (ranging between three times their respective thresholds to upper values of 80% contrast and 65° orientation) were chosen for each observer and presented randomly 50 times for every C and O session. Observers were asked to press a key as soon as they detected the increment. Simple RT as a function of the increment size (x = ΔC or ΔO) were fitted with the exponential RT = c₁ exp(−xc₂) + c₃ (Barbur, Wolf, & Lennie, 1998), where c₁, c₂, and c₃ are free parameters. The high-salience increments used in the main experiment were chosen to be the ΔC or ΔO values for which the fitted RTs came within 5% of their asymptotic values for each observer. RTs to C and O increments were measured in separate blocks (of 500 trials each) repeated twice according to an ABBA sequence. 

Main experiment

The temporal configuration of the stimuli is illustrated in Figure 1. One experimental block started with the observer pressing a key that triggered the display of the fixation cross. After a constant 1,300-ms interval, the two Gabor pedestals appeared and remained unchanged for a uniformly distributed random period of 500 to 1,300 ms. At the end of this period, each of the two Gabors was incremented in either C or O with one increment delayed with respect to the other by one of seven equally spaced SOA ranging from −100 to 100 ms. The two “incremented” Gabors were simultaneously removed from the screen 1,700 ms after their onset (discarding stimulus offsets as possible clues for the TOJ; Jaśkowski, 1991, 1992). SOA values and the sides allocated to each increment were randomized across trials. Observers were required to press a key as soon as they detected any of the two increments (speeded simple RT) and, after the offset of the stimuli, press another key (out of two) to indicate which of the left- or right-hand increment occurred first (TOJ). 

Figure 1

 

One trial sequence illustrating two successive orientation changes.

As the temporal interval between the onsets of the pedestals and their increment was a uniform random variable, the probability of occurrence of the latter increased over time and so did observer's expectancy. Each experimental block (see below) also included 5% no-increment (catch) trials to discourage anticipatory RT. For these trials, observers were instructed to withhold their key press until the stimuli offset and then press yet another key to terminate the trial. Different feedback tones were provided after each “catch” trial depending on whether or not the observer had produced an anticipatory key press. Each experimental block also included single-increment trials (equivalent to an infinite SOA) with the same frequency as any of the seven SOAs. The means and standard deviations of the RT distributions to these single increments were used to simulate the RT and TOJ responses with the 0-free-parameter, one-system model (see the Model and data fits section). 

One experimental block was specified in terms of the incremental pair applied to the two Gabor pedestals. As there were two C (C1, C2) and two O (O1, O2) increments; their combination yielded 10 stimulus pairs/blocks, that is, four identical stimulus pairs (C1–C1, C2–C2, O1–O1, and O2–O2) and six different stimulus pairs (C1–C2, C1–O1, C1–O2, C2–O1, C2–O2, and O1–O2). Within each block, the seven SOAs and the single increments (infinite SOA) were presented randomly 50 times each. One block also included 23 (no increment) catch trials so that each block consisted of 450 + 23 = 473 trials run over an average of 30 min with a break every 100 trials. Trials yielding either anticipatory (<0 ms) or very long (>1 s) RTs were repeated at the end of the block without informing the observer. Trials including RTs larger than ±2 SD of the mean were excluded offline from further analysis. The 10 experimental blocks were run in a random order for each subject and were repeated three times. Hence, each experimental point (i.e., mean RT and percentage TOJ per SOA and per stimulus pair) was computed out of 150 trials. 

Observers

Four right-handed observers (one woman and three men—including the first two authors—23 to 54 years old) participated in all experiments. 

Model and data fits

The present RT–TOJ data analysis focuses on the simplest one-system model according to which RT to and TOJ of two sequential sensory events are based on the same internal signal and on the same unbiased and deterministic decision process (Adams & Mamassian, 2004; Gibbon & Rutschmann, 1969; Sternberg & Knoll, 1973). According to standard race models (e.g., Gold & Shadlen, 2001; Luce, 1986; Smith & Ratcliff, 2004), each of the two perceptual events evokes independent noisy internal responses increasing over time at a rate proportional to these events' intensity and triggering both a motor response and a perceptual detection once any of these responses exceeds some unknown decision value. The decision time, that is, the duration between the stimuli onsets and the first criterion crossing, is modelled as a Gaussian stochastic variable characterized by its means (μ) and standard deviations (σ) specific to the given stimuli. Although simplistic, this model allows the computation of RT distributions to a pair of events, the TOJ psychometric functions (of the time interval between these events, SOA), as well as the RT distributions classified according to observers' TOJ (hereafter referred to as RT_TOJ) with the same set of parameters. According to this one-system race model, the μ and σ parameters are directly available from the RT distributions measured for each observer and for each event when occurring in isolation.¹ In the present experiment, the two stimuli sequences were interleaved with trials where only one of the two stimuli occurred (see the Methods section) so that RT distributions, that is, their μ and σ, could be assessed for each isolated stimulus in each of the 10 experimental conditions out of 750 trials each. 

In the remainder, the model fed with the measured μ and σ variables (i.e., 4 μ and 4 σ values, i.e., 1 μ and 1 σ  for each of the two increments and for each of the two stimulus attributes, C and O) is referred to as a 0-free-parameter model. A second analysis consisted in fitting the model separately to RT and RT _TOJ data with μ and σ  as free parameters. The fitting procedure provided either 8- or 32-free-parameters per observer. In the 8-free-parameter case, 4 μ and 4 σ values (see above) were used to fit RT and RT _TOJ to the whole set of stimulus pairs (i.e., 10). In this case, as in the 0-free-parameter model, only one distribution (1 μ and σ value) is associated to each stimulus, regardless of the experimental condition. The difference between the 0- and the 8-free-parameter fits is that, in the first case, the parameters are measured (the μ and σ of the RT distribution to single stimuli), and in the second case, they are fit to the data. For the 32-parameter fits, different μ and σ values were fitted to each of the 10 stimulus pairings, yielding a total of 4 (identical-stimuli conditions) × 2 (parameters) + 6 (different-stimuli conditions) × 4 (parameters) = 32 (the largest possible number of) free parameters. Fitted and measured μ and σ values were subsequently used to infer the parameters of the corresponding TOJ psychometric functions (of SOA), that is, their inflection points (PSS) and slopes ( σ _TOJ). These inferred parameters were then compared with those obtained by fitting the raw TOJ with cumulative Gaussians. The fits were obtained via Monte Carlo simulations of random draws (500,000 per experimental condition and observer) from normally distributed distributions whose μ and σ were either given (0 free parameter) or adjusted for the best fit of the data (8- and 32-free-parameters). 

As the 0-free-parameter fits make use of 8 measured (4 μ and 4 σ) parameters, their comparison with the 8 free parameters was meant as a qualitative evaluation of the extent to which RT distributions for single stimuli are good predictors, within the simplest one-system model framework, of the RT distributions to stimulus pairs and of the corresponding TOJ. The 32-free-parameter fits correspond to the upper goodness-of-fit bound provided by the simplest one-system model when all its parameters are set free. This upper goodness-of-fit bound could be used as a benchmark for testing future improvements of this simplest model. Significant differences between the 0- and 8-free-parameter fits, on the one hand, and between the latter two and the 32-free-parameter fits, on the other hand, would call into question the validity of the simplest one-system model and/or the independent processing of the two stimuli in a stimulus pair. The absence of a significant difference between the measured and fitted (8 and 32) parameters will instead sustain the validity of the former in predicting RT to stimulus pairs independently of their specific combinations. 

Goodness of fit: R ² computation

To evaluate the goodness of fit of the simplest one-system model with 0-, 8-, and 32-free-parameters, we computed the determination coefficient, R ², for each observer as 1 − SSE/SST, where SSE is the sum of squares of errors of the predictions, that is, a measure of how close the points are to the regression line, and SST is the total sum of squares about the mean of the measured values. Accordingly, R ² can be negative when SSE is larger than SST, which means that the model describes the data less well than their mean (Neter, Kutner, Wasserman, & Nachtsheim, 1996). In the present case, SSE was computed relative to the major diagonal (slope of 1 and intercept of 0), that is, the perfect fit. It should be noted that this R² index differs form the R² of the simple linear regression analysis where the slope and intercept of the regression line are free parameters. The simple linear regression analysis yields higher R² values but was discarded as the nonunit slope and nonzero intercept regression line makes no theoretical sense. 

Results

In this section, experimental data and their simulations are first presented separately for the RT and TOJ tasks (first two subsections) and are subsequently confronted with each other (third subsection). To anticipate, the one-system model yields relatively good RT data fits (better for the mean RT than for their standard deviations) but generates rather mixed implications concerning the parameters (PSS and slopes) of the TOJ psychometric functions. At the same time, however, RTs differ significantly according to their associated TOJs: Depending on whether or not the TOJs conform with the physical order of the stimuli, the associated RTs are respectively faster and slower, with the difference between the two increasing with the absolute temporal delay between the two stimuli. This observation that we regard as the main finding of the present study confirms the inescapable link between perceptual and motor behaviors and attunes to the existence of a unique processing stream for the corresponding decisions. 

Predictability of RT distributions to two-stimuli sequences

Out of the 10 stimulus pairs presently studied, Figure 2 displays data (solid circles) and model fits (curves) for one representative condition (O1–O2) and for each of the four observers (columns). RT distribution means ( μ _RT) and standard deviations ( σ _RT) as a function of the O1–O2 SOA are shown in the top and bottom rows, respectively. Negative SOAs are for cases where O2 was presented before O1. Black curves are data fits obtained with the 0-free-parameter simple stochastic model, that is, based on the measured RT distributions ( μ _RT and σ _RT) to O1 and to O2 when presented in isolation (see the Model and data fits section). Red curves are the best fits of the 32-free-parameter stochastic model (i.e., with μ _RT and σ _RT for each of the two stimuli in the pair as free parameters). Data fits using only 8-free-parameters (i.e., with the same μ _RT and σ _RT for each of the four stimuli independently of how they were combined) lie somewhere in between the 0- and the 32-free-parameter fits. 

Figure 2

 

Means ( μ _RT; top row) and standard deviations ( σ _RT; bottom-row) of the RT distributions for the representative O1–O2 condition as a function of the O1–O2 SOA for each observer (columns). Horizontal solid lines in the top and bottom panels show μ _RT and σ _RT for each of the two stimuli (O2 and O1 on the left- and right-hand sides of each panel) in the pair when presented alone (negative SOAs indicate trials where O2 was presented first). Solid circles show the actual data. Vertical bars in the top panels are ±1 SE _RT. Black curves are the simplest one-system model's predictions based on the μ _RT and σ _RT assessed with O1 and O2 stimuli presented in isolation (i.e., 0-free-parameter fits). Red curves are this same model's best fits with μ _RT and σ _RT for each of the two stimuli as free parameters (referred in the text as the 32-free-parameter fits). 8-free-parameter fits are omitted for clarity. See text for more details.

The observed dependence on SOA of both data and model fits can be understood intuitively: (a) at large positive and negative SOA, μ _RT and σ _RT tend asymptotically toward the respective values assessed for the first and last stimulus in the sequence when presented alone (horizontal lines in each panel); (b) μ _RT and σ _RT dips observed at short SOA result from probability summation over the stochastic internal responses evoked by the two stimuli (see Adams & Mamassian, 2004). Clearly, RT distributions to the O1–O2 (but also to the remaining nine) stimulus pair(s) as a function of SOA are satisfactorily predicted by the 0-free-parameter model with only marginal benefits brought about by the 32-free-parameter fits. As no systematic differences between data and fits were observed across the two attributes (C and O) and their combinations, further analyses of the goodness of fits were performed over all experimental conditions lumped together. 

The relative goodness of fits with 0-, 8-, and 32-free-parameters could, in principle, be assessed via a nested model hypothesis testing. This procedure requires, however, the estimation of the variability of the σ _RT datum points (the variance of the variance of the data for each experimental condition) via time-consuming Monte Carlo simulations of an unworthy cost. As an alternative, Figure 3 displays for each absolute SOA the average (over the four observers) coefficients of determination ( R ²) between data and the 0-, 8-, and 32-free-parameter fits (black, light, and dark gray histogram bars, respectively) computed over μ _RT and σ _RT values lumped together ( Figure 3A) and over each of them separately ( Figures 3B and 3C). As noted above, μ _RT and σ _RT values for stimuli in a pair separated by large positive or negative SOA are bound to converge on the corresponding values assessed for each of these stimuli when presented alone (horizontal lines in Figure 2). Consequently, the R ² coefficients are also compelled to improve with ±SOA, as actually observed. 

Figure 3

 

Average percentage of variance ( R ²) accounted for by the simplest one-system model fits (with 0-, 8-, and 32-free-parameters; see inset) of the RT distributions ( μ _RT and σ _RT; a) and, separately, of their means (b) and standard deviations (c) as a function of absolute SOA. Error bars show +1 SE computed over the four observers.

R ² coefficients for the global RT distributions ( μ _RT and σ _RT lumped together; Figure 3A) are no less than .70 (0-free-parameter fits) at 0 SOA and increase with absolute SOA. These correlations are mostly generated by the fits of the μ _RT data ( R ² in between .77 and .98; Figure 3B) with the fits of the σ _RT data yielding systematically lower R ² (.58–.88; Figure 3C). Overall, Figure 3 supports the notion that RT distributions to sequential stimulations are predictable from the RT distributions to single stimuli (Adams & Mamassian, 2004). Not surprisingly, the goodness of the fits increases with the number of free parameters. In comparison with the R² error bars across observers, however, this gain is relatively minor for the lumped and for the μ_RT data (Figures 3A and 3B) but not for the σ_RT data (Figure 3C) where the 32-free-parameter fits are better than the 0- and 8-free-parameter fits. Student t tests between the measured (0-free-parameter) and the fitted (8- and 32-free-parameters) parameters do not yield a significant difference, either for the means, 0 versus 8: t(15) = −1.17, p = .260; 0 versus 32: t(63) = −1.67, p = .101, or for the standard deviations, 0 versus 8: t(15) = 0.036, p = .972; 0 versus 32: t(63) = 0.695, p = .493. Hence, within the framework of the simplest one-system model, measured and fitted μ_RT and σ_RT are statistically equivalent, which implies that allowing these parameters to vary with the different stimulus pairings is a useless endeavor. 

Consistency of the TOJ

Before evaluating the TOJ predictability from the RT distributions, one would like to know the extent to which the parameters (PSS and σ _TOJ) of the TOJ psychometric function (of SOA, hereafter referred to as Ψ functions) for an arbitrary stimulus pair can be inferred from the TOJ Ψ functions for other stimulus pairs given the assumptions underlying the present one-system model (a “consistency” test). This approach is similar but not equivalent to the one in the subsection above where distributions of RT to two-stimulus sequences were tested against predictions based on RT distributions to single stimuli (0-free-parameter fits). It is similar in the sense that both approaches test predictions of the model but differs from the previous one in the specific way the consistency test was performed. Indeed, the TOJ task requires, by design, the presentation of two stimuli, whereas RTs can be (and were) measured for both single and dual stimulations. The TOJ consistency issue was therefore grappled with via two independent analyses: one that tests the transitivity of the measured PSS and one that tests the predictability of the slopes of the TOJ Ψ functions obtained with nonidentical stimuli pairs from the Ψ functions assessed with identical stimuli pairs. As a first step for either of the two tests, percentages of “S1 seen first” responses as a function of SOA were fitted with cumulative Gaussians (constrained by their means, μ _TOJ—equivalent to the PSS—and by their standard deviations, σ _TOJ—equivalent to the slope of the Ψ function—for each of the 10 experimental conditions and for each observer). It must be noted that, by construction, the predictability of the RT distributions for stimulus pairs from the distributions for single stimuli ( Figure 3) necessarily involves an indirect test of their consistency as it implies the transitivity of their means and the summation of the RT variances for single stimuli when these stimuli are presented in pairs. 

PSS transitivity test

The four conditions with identical stimuli in a pair (see the Methods section) were excluded as they yield, by design, a PSS of 0. For the remaining six conditions, each of the four stimuli (C1, C2, O1, and O2) is paired with the remaining three (e.g., C1–C2, C1–O1, and C1–O2). Hence, given transitivity, any PSS should be predictable from two other PSSs provided that the latter two are assessed with stimulus pairs sharing one stimulus and each of them sharing the to-be-predicted pair from one of the two remaining stimuli. For example, the PSS for the C1–C2 pair, PSS _C1–C2 = PSS _C1–O2 − PSS _C2–O2. With four stimuli, there are four such different triplets as each of these four stimuli must be excluded once. The choice of the to-be-predicted PSS in a given PSS triplet is arbitrary. 

Figure 4A displays four measured against four predicted PSSs given PSS transitivity (different symbols are for the different observers). Perfect transitivity would yield data points along the major diagonal. A linear regression computed on these values for all subjects (note that whereas datum points of three of four observers cluster together, that of the remaining observer—open triangles—displays a more erratic behavior) fails to support the transitivity hypothesis (PSS _M = −0.06 PSS _P + 30.2, where subscripts M and P stand for “measured” and “predicted”), R ² = .003, t(14) = 0.204, p = .841. In substance, observers' PSSs are not consistent across experimental conditions, a variability frequently reported (e.g., Gibbon & Rutschmann, 1969; Jaśkowski, 1993, 1996, 1999) and consistently ignored. It may be due not only to cognitive factors (e.g., Frey, 1990; Schneider & Bavelier, 2003; Shore, Spence, & Klein, 2001) not intervening in the RT task but also to some high-level interactions between the two stimuli. Occasional transient fading of a low-saliency stimulus induced by a distant, higher saliency stimulus has been reported both in previous studies (Kanai & Kamitani, 2003) and by all four observers in the present experiments. Such interactions, however, do not seem to affect the RT distributions. 

Figure 4

 

Measured versus predicted PSS (a) and TOJ slopes, σ_TOJ (b) (different symbols are for different observers). Each of the four (out of six) PSS predictions (per observer) is given by the algebraic sum of two other PSSs. Each of the six σ_TOJ predictions (per observer) is derived from the σ_TOJ obtained with stimulus pairs consisting of identical stimuli (see text for details). The black, solid, unitary slope straight lines show the perfect correspondence between predictions and data. The red straight lines are linear regressions. Red dashed curves show 95% confidence intervals delimiting the area that has a 95% chance of containing the true regression lines. Blue dashed curves are 95% prediction intervals; that is, they delimit the area where 95% of all datum points are expected to fall.

Predictability of the slopes of the TOJ Ψ functions

This test concerned the predictability of the Ψ function slopes obtained with the six stimulus pairs composed of different stimuli from the Ψ function slopes for the four pairs with identical stimuli. On the critical assumption that the internal events evoked by each stimulus ( i, j) in a pair are independent stochastic events, their combined variance σ _{i, j} ² is the sum of the two variances, σ _i ² + σ _j ². Hence, for pairs of identical stimuli, the predicted σ _i is given by the standard deviation of the fitted Ψ function, σ _{i, i}, divided by

$\sqrt{2}$

. The slopes of the six Ψ functions for nonidentical stimuli, σ _{i, j}, were therefore computed as  

${\sigma }_{i,j}=\sqrt{{\sigma }_i^2+{\sigma }_j^2}$

(1)

and are shown in Figure 4B against the actually measured Ψ function slopes for each observer (different symbols; as for the PSS, the observer whose datum points are shown as open triangles displays a relatively deviant behavior). From inspection of the figure, these predictions are fair as the datum points lie close, although slightly above the main diagonal (particularly the open triangles). A linear regression analysis confirms this observation ( σ _{TOJ_M} = 0.954 σ _{TOJ_P} + 23.3), R ² = .295, t(22) = 3.03, p < .01, as it yields a slope not significantly different from 1, t(22) = 0.144, p = .44. 

In sum, observers' behavior in the TOJ task yields mixed trends: It is rather reliable as regards the slopes of the Ψ functions (i.e., complies with the variance summation rule), but it shows PSS inconsistency (i.e., violates transitivity). This pattern of results is compatible with the TOJ being based on independent Gaussian random variables associated to a pair of stimuli but is not compatible with a deterministic, unbiased decision rule according to which PSSs are entirely determined by the processing latencies. Instead, the inconsistency of the PSS suggests that the decision variable is affected by unknown (cognitive?) factors. 

Although the consistencies of the RT and TOJ data cannot be directly compared (as they were derived with different procedures), the high determination coefficients between measured and predicted RT ( Figures 3A and 3B), on the one hand, and the poor PSS transitivity index for the TOJ data ( Figure 4A), on the other hand, suggest that the two tasks address qualitatively distinct processes. The RT data are compatible with a fixed decision criterion (the verified predictability of the μ _RT implies their transitivity), whereas the TOJ data are not. 

Comparing TOJ and RT

The most straightforward test of the RT–TOJ relationship and, thereby, of the simple one-system model is to check the equivalence (i.e., a 1:1 relationship) between the amount by which RTs to each of the two stimuli ( i, j) making up a sequential stimulus pair differ (ΔRT _{i, j} = RT _i − RT _j) and the PSS assessed for this same pair (PSS _{i, j}). According to the simplest one-system model (Gibbon & Rutschmann, 1969), the stimulus whose evoked internal response triggers the motor response must also be the one perceived first, with the difference between RT to each of the two events in a sequence (ΔRT) being equal to the SOA, entailing their simultaneous perception (PSS). This PSS–ΔRT relationship is shown in Figure 5A for all observers (different symbols) and for the six different-stimuli pairs.² An equivalent 1:1 relationship should also be observed between the measured slopes of the TOJ Ψ functions (σ_TOJi,j, where i and j are the two stimuli in a pair) and these same slopes as predicted from the standard deviations of the measured RT distributions to each of the stimuli in a pair (σ_RTi and σ_RTj). These predictions are given by 

$\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d}{\sigma }_{\mathrm{T}\mathrm{O}\mathrm{J}}=\sqrt{{\sigma }_{\mathrm{R}\mathrm{T}i}^2+{\sigma }_{\mathrm{R}\mathrm{T}j}^2}$

(2)

and are presented against the measured slopes in Figure 5B. 

Figure 5

 

Measured PSS against measured ΔRT (a) and measured TOJ Ψ function slopes, σ_TOJ, against their values predicted from the standard deviations of the RT distributions to each of the two stimuli in the corresponding stimulus pair. Different symbols are for different observers. The solid black lines of unitary slope show the perfect correspondence between the two pairs of values. The red straight lines are linear regressions, with the red and blue dashed curves showing 95% confidence (see Figure 4 caption) and prediction intervals, respectively.

As expected from previous studies (e.g., Gibbon & Rutschmann, 1969; Jaśkowski, 1993, 1996, 1999), the ΔRT–PSS relationship is rather erratic (particularly so for the observer whose data are shown as open triangles). A linear regression analysis yields a nonsignificant ΔRT–PSS correlation coefficient (PSS = 0.474 ΔRT − 6.12), R² = .05, t(22) = 1.11, p = .279. The nonsignificance of this relation is probably due to the large dispersion of the data shown as triangles, the only ones to ever exceed the 95% prediction intervals (i.e., the intervals supposed to contain 95% of the datum points). Excluding this observer's data from the linear regression analysis does not change the parameters of the fitted regression line but yields a highly significant R² (PSS = 0.474 ΔRT + 3.31), R² = .411, t(16) = 3.34, p < .005, as frequently reported (Jaśkowski, 1992; Jaśkowski & Verleger, 2000; Menendez & Lit, 1983; Roufs, 1974). A binomial test on all data shows that most of these datum points are below the major diagonal, B(6, 24), p < .02, and above the 0 ordinate (PSS) point, B(5, 24), p < .01. This supports the notion that the PSSs are less affected by the difference between stimulus saliencies than ΔRT, in line with most previous studies that report a ΔRT:PSS ratio of about 2 (but see Roufs, 1963). Within the framework of a one-system model, this difference between ΔRT and PSS suggests that perceptual decisions are taken before motor ones. 

An equivalent regression analysis on the predicted versus measured slopes of the TOJ Ψ functions ( Figure 5B) yields significant R ² coefficients ( σ _{TOJ_M} = 1.46 σ _{TOJ_P} − 2.16), R ² = .298, t(38) = 4.01, p < .001. A binomial test shows that most datum points lie above the major diagonal, B(4, 40), p < .001. This analysis (which, to the best of our knowledge, was never performed before) confirms once again the relationship between RT and TOJ tasks, while revealing a significantly higher variability associated with the latter (i.e., measured σ _TOJ > predicted σ _TOJ). Assuming that the noise of the evoked internal response increases with its magnitude over time (e.g., Gold & Shadlen, 2001; Luce, 1986; Smith & Ratcliff, 2004), one may speculate that this larger variance associated with the TOJ task reflects the existence of a perceptual criterion above the motor criterion (see also Miller & Schwarz, 2006), a conclusion opposite to the one based on ΔRT–PSS differences. 

Overall, the observed correlations between the measured parameters of the TOJ Ψ functions (PSS and σ _TOJ) and their values predicted from the RT distributions (ΔRT and σ _RT) substantiate the notion that motor (RT) and perceptual (TOJ) behaviors are (at least partly) based on the same internal response. However, the fact that data and predictions do not lie along unitary slope lines calls into question the validity of the one-system, one-decision model and suggests instead distinct decision processes for the two tasks. The data analysis below strengthens this analysis. 

As participants performed the RT and TOJ tasks within the same trial, it was possible to classify their RTs according to the two TOJ categories (RT _TOJ), namely, “S1 seen first” and “S2 seen first”. ³ This feature is the main methodological improvement of the present study over previous ones. Figure 6 displays once more the data sample of Figure 2 (means and standard deviations of the RT distributions for the O1–O2 pair in the top and bottom rows, respectively; circles) and their 0- and 32-free-parameter fits (black and red curves, respectively) classified this time according to the “O1 seen first” (open circles and dotted curves) and “O2 seen first” (solid circles and curves) TOJ categories. Note that “O1 seen first” and “O2 seen first” responses are in agreement with the physical temporal display for negative and positive SOAs, respectively. From the perspective of the simplest one-system model, for O2 to be seen first despite being presented second (−SOA), it must be that the arrival time of its evoked response at the decision stage is delayed in average relatively to the arrival time of the response evoked by O1 (for the same −SOA). The same logic holds true for “O1 seen first” responses and +SOAs. Hence, provided that TOJs and RTs are both based on the same internal response, one expects that, overall, RTs associated with “wrong” TOJs be longer than those associated with “correct” TOJs (see Footnote 3). Moreover, the difference between the two RT _TOJ types should increase with the absolute physical delay between the two stimuli. Mean RT _TOJ and their fits in the top panels of Figure 6 sustain these observations and thereby the notion that motor and perceptual behaviors are most likely based on the same (or on strongly correlated) internal response(s). An equivalent account of the dependency on SOA of the standard deviation associated with “correct” and “wrong” RT _TOJ (bottom panels of Figure 6) is less intuitive ⁴ and also less backed up by the data. 

Figure 6

 

Same data as in Figure 2 (i.e., for the O1–O2 stimulus pair) and their 0- and 32-free-parameter fits, but with the RT datum points and their predictions classified according to the two types of TOJ, that is, “S1 seen first” (open symbols and dotted curves) and “S2 seen first” (solid symbols and curves). Black and red curves are 0- and 32-free-parameter fits, respectively. Top and bottom rows display the means and the standard deviations of the RT data/predictions, respectively. The 8-free-parameter fits are omitted for clarity.

The determination coefficients R ² between the 0-, 8-, and 32-free-parameter fits of the measured μ _RT and σ _RT classified according to TOJ and the data were assessed as before over the whole data set (10 stimulus pairs) and for each observer (see the Methods section). Figure 7 presents the average (over the four observers) R ² for each set of free-parameter fits (black, light, and dark gray histogram bars for 0-, 8-, and 32-free-parameters, respectively) with μ _RT and σ _RT values lumped together ( Figure 7A) and over each of them separately ( Figures 7B and 7C). The global fits yield R ² values ranging from .77 to .23 for 0 and ±100 ms SOA, respectively. Once again, these correlations are mainly contributed to by the partial μ _RT fits ( R ² ranging from about .7 to .4 for 0 to ±100 ms SOA, respectively; Figure 7B), with R ² for the partial σ _RT fits below .12 for the nonzero SOA ( Figure 7C). ⁵ As for the fits of the RT distributions noncontingent on observers' TOJ ( Figure 3), the goodness of the present fits increases with the number of free parameters, but the improvement is negligible relatively to the error bars across observers. Compared to those fits, the present R ² coefficients are globally lower, and instead of increasing, they decrease with absolute SOA. The latter discrepancy is at least partly due to the progressive decrease with the absolute SOA of the number of trials on which the estimated μ _{RT_TOJ} and σ _{RT_TOJ} values are based. Indeed, as the absolute SOA increases, the number of reports “seen first” for the second stimulus in the physical sequence decreases so that the assessment of the corresponding μ _{RT_TOJ} and σ _{RT_TOJ} values gets progressively less reliable. Most certainly, additional factors related to the variability of the TOJ should also account for the lower R ² coefficients in this TOJ-contingent RT analysis. 

Figure 7

 

Average percentage of variance ( R ²) of the RT _TOJ distributions ( μ _{RT_TOJ} and σ _{RT_TOJ}; a) and, separately, of their means (b) and standard deviations (c) explained by the simple one-system model fits using 0-, 8-, and 32-free-parameters (see inset) as a function of the absolute SOA value. Error bars show +1 SE of the means computed over the four observers.

Overall, Figures 6 and 7 demonstrate the fact that RT distributions to sequential stimulations follow distinct functions of SOA when categorized according to the associated TOJ. This appears to be the most important observation of the present study as it reveals beyond doubt an intrinsic relation between motor and perceptual responses. As such, it supports the notion that a one-system model may fully account for perceptual and motor behaviors given some additional assumptions (to be explored) on the underlying decision processes. 

Discussion

The design of the present study adds two new features to the extensive literature on the relationship between RTs and TOJs: (a) a trial-by-trial RT–TOJ analysis (of their means [PSS] and of their standard deviations [slopes of the TOJ Ψ functions]), which allows a direct assessment of their relationship; (b) the test of the simplest one-system model (positing that RT and TOJ are based on the same internal signal and on the same decision process; Gibbon & Rutschmann, 1969) via Monte Carlo simulations using 0-, 8-, and 32-free-parameters. Also, these assessments and tests were performed within the same experiment over two distinct visual dimensions (contrast and orientation) and their combinations (i.e., 10 stimulus pairs), hence strengthening the generality of the present findings. 

Inspection of the data yields the following six main observations:

1.  
   The means and standard deviations of the RT distributions to stimulus pairs decrease for small SOA ( Figure 2), as predicted by a simple race model assuming independent channels and a deterministic decision process (Adams & Mamassian, 2004).
2.  
   In contrast, as already documented in the literature, the PSSs of the TOJ Ψ functions ( Figure 4A) show large scatters with little consistency (i.e., transitivity) across experimental conditions. This result contrasts with the standard deviations of the TOJ Ψ functions, which are consistent across observers and experimental conditions ( Figure 4B). This pattern of results is compatible with the idea that TOJs are biased in a nonsystematic, idiosyncratic manner (Gibbon & Rutschmann, 1969).
3.  
   Also in agreement with the literature and contrary to the simplest one-system model's prediction, the difference between RT (ΔRT) to each of the two events used in the 10 stimulus pairings is typically two times larger than the PSS for these same stimulus pairs ( Figure 5A).
4.  
   Without precedence in the literature, the present data analysis also indicates that the slopes of the TOJ Ψ functions are typically shallower (i.e., larger standard deviations) than those inferred from the RT variability ( Figure 5B); this observation, which is tantamount to the fact that perceptual decisions are noisier than motor decisions, suggests that perceptual decisions are taken after (and relatively to a higher criterion than) the motor decisions.
5.  
   Despite the abovementioned inconsistencies between RT and TOJ, all observers show distinct RT functions of SOA depending on the associated TOJ. This excludes the possibility of these responses being performed by two independent subsystems and is regarded as the main finding of the present study.
6.  
   Finally, no qualitative differences related to the nature of the stimuli (i.e., contrast vs. orientation increments) were observed, supporting the generality of the above observations.

Observations 1 and 2 may bear on the fact that, unlike TOJs (e.g., Frey, 1990; Shore et al., 2001; Sternberg & Knoll, 1973), simple RTs presumably characterize an automatic response behavior and, as such, are independent of stimulus context and of subjective decisional factors (Waszak & Gorea 2004; Waszak, Gorea, & Cardoso-Leite, in revision).⁶ Instead, perceptual decisions are presumably referenced to a context-dependent criterion as defined by standard Signal Detection Theory (SDT; Green & Swets, 1966). According to SDT, the setting of a perceptual criterion results from a subjective optimization process (e.g., maximizing the number of correct responses) contingent on the observer's knowledge of the noise associated with a given task/stimulus and on contextual factors such as stimulus' a priori probability and payoff. Accordingly, TOJs (but not RTs) have been shown to comply with temporal Bayesian calibration whereby subjects use all the available information (sensory and contextual) to infer the physical onsets of the stimuli (Miyazaki, Yamamoto, Uchida, & Kitazawa, 2006). By contrast, inasmuch as observers' motor task does not require the maximization of the correct responses (but rather the minimization of their RTs), the motor threshold is presumably set as low as possible irrespective of the processing context (Sternberg & Knoll, 1973). 

That motor and perceptual responses may well be triggered at different levels of the same internal signal increasing over time is a current conjecture made by defendants of the one-system view (Miller & Schwarz, 2006; Sternberg & Knoll, 1973; Waszak & Gorea, 2004). While these authors may disagree as to the primacy of one of these levels over the other, the debate may just reflect the fact that, as perceptual criteria are both task and stimulus dependent, they can be set either below or above the motor threshold. It remains that within this dual-threshold theoretical framework, the present data yield contradictory messages. On the one hand, Observation 3 (i.e., TOJ being less affected than RT by stimulus intensity) suggests that the perceptual criterion is below the motor threshold. On the other hand, Observation 4 (i.e., larger perceptual than motor variability) suggests the opposite conclusion. Indeed, standard race or diffusion models (e.g., Gold & Shadlen, 2001; Luce, 1986; Smith & Ratcliff, 2004) posit that the variance of the internal signal increases over time. Hence, if the perceptual criterion is lower than the motor threshold, then the perceptual variability should be smaller than the RT variability. Future research should focus on this discrepancy. Be it as it may, Observation 5 (i.e., distinct RT distributions according to the observers' perceptual response) speaks beyond doubt for the relationship between RT and TOJ, which is to say against the strong version of a perception–action dissociation. Although the data fits of the simplest one-system model (Gibbon & Rutschmann, 1969) are substantially poorer when applied to the RT distributions classified according to observers' TOJ (Figures 6 and 7), they still account for at least 20% of the variance in the data. As already noted, this drop in the goodness of the fits is at least partly due to the smaller size of the trial samples on which the measured RT distributions are based. In addition, the goodness of these TOJ-contingent fits is most certainly affected by the intrinsic variability of the TOJ, probably due to contextual (decisional) and observer-dependent response strategy factors not included in the model and affecting only the perceptual decisions. 

In short, data and simulations comply with “one-system–two-decision” models of perceptual and motor behaviors, while prompting further testing and modeling to account for the apparent discrepancy between the ordering of the two decisions and of their associated variances. 

To conclude, the present study conveys the following three main ideas. First, simple RTs to and TOJs of a pair of visual events result from correlated processes: For all the studied SOAs, RTs differ with the associated TOJs. Second, this relationship is moderately accounted for by a simple race model positing that RT and TOJ are based on the same internal signal and a unique decision process. Third, this modeling fails, however, to account for the systematic differences between perceptual and motor responses. These differences could be partly explained by assuming that motor and perceptual responses rely on distinct decision processes, with the latter subjected to contextual variables yet to be characterized. 

Acknowledgments

This work was supported by a grant from the Agence Nationale de la Recherche (ANR) to A. Gorea. 

Commercial relationships: none. 

Corresponding author: Pedro Cardoso-Leite. 

Email: [e-mail]. 

Address: 45 rue des Saints-Pères, 75018 Paris, France. 

Footnotes

Footnotes

¹  Note that the measured RT means include some unknown execution time, which cancels out when computing their difference (ΔRT).

Footnotes

²  Because ΔRT and PSS are relative measures (one stimulus relative to another), their sign is arbitrary, whereas their sign relative to each other is not. ΔRT was computed so as to always yield positive values, with the sign of the PSS changed correspondingly.

Footnotes

³  TOJ is a subjective task so that the PSS derived for unequal saliency stimuli need not and in fact do not correspond to the physical simultaneity of the stimuli onsets (i.e., 0 SOA). As a consequence, the classification of TOJ as “correct” versus “incorrect” is meaningless under such conditions.

Footnotes

⁴  The model assumes that the decision “S1 seen first” is a nonlinear operation (minimum of two values). The probability density function resulting from the minimum of two Gaussian random variables is thus not necessarily Gaussian and clearly depends on the separation between the two Gaussian distributions.

Footnotes

⁵  Histograms for the 66- and 100-ms SOA in Figure 7A are missing because the corresponding R ² values are negative and, thus, outside the plot's range.

Footnotes

⁶  The concept of an automatic process has been and still is an object of frantic debate (see Pashler, 1998, chapter 8). For the purpose of this discussion, we go along with (Waszak and Gorea's 2004) proposal that simple RTs are “automatic” in the sense that they are triggered once an internal response exceeds a fixed motor threshold as opposed to a contextually dependent perceptual criterion (as defined by high-threshold and signal detection theories; Green & Swets, 1966).