Automatic adjustment of visuomotor readiness

Abstract

Participants initiated a reaching movement to a single target more rapidly than to an odd-color target among distractors when the two trial types were presented in separate blocks, reflecting differentiated states of sensorimotor readiness for a relatively easy (single target) versus harder (odd-color target) tasks. This pattern was eliminated when the two trial types were randomly mixed. Latencies for the easy single trials increased, and those for the harder odd-color trials decreased, showing homogenization. The faster movement initiation in the odd-color target task was accompanied by curved trajectories, directed toward a distractor initially but corrected in midflight. Two possible hypotheses could account for this differentiated adjustment in visuomotor readiness: (1) explicit knowledge of upcoming trial types and (2) implicit leaning derived from history of the very recent past, that is, repetition of the same type of trials. To distinguish between these two accounts, we included a third condition where the trial types were predictably alternated. Contrary to the explicit knowledge hypothesis, this also led to homogenization of initiation latencies, and curved trajectories. We conclude that visuomotor readiness is automatically adjusted by the recent experience of trial difficulty.

Introduction

Human visual perception and behavior directed toward the external world are strongly influenced by context and especially by recent experience. Effects of previous trials on current trials have been observed in response time, accuracy, ambiguous motion perception, and hitting movements (de Lussanet, Smeets, & Brenner, 2002; Luce, 1986; Maljkovic & Nakayama, 1994; Maloney, Dal Martello, Sahm, & Spillmann, 2005; Taylor & Lupker, 2001). Such effects of preceding trials are not limited to when previous and current trials share similar stimulus features or responses. Rather, such short-term dependencies have been claimed for far more abstract types of similarity, particularly for task difficulty (Mozer, Kinoshita, & Shettel, 2007). 

When easy and hard trials are separated into different blocks, responses to hard trials are slower than responses to easy trials. At first glance, this should not seem surprising because harder tasks are likely to require additional processing than easy tasks, which is the assumption of additive factor models of reaction times (Sternberg, 1969). Yet, recent work has shown that reaction times are not simply dependent on the difficulty of the current trial but depend critically on context, namely, on whether trials of a given sequence are all the same or of mixed difficulty. For instance, when easy and hard trials are mixed randomly instead of blocked, the differences in reaction time are greatly diminished. This homogenization effect of reaction times for tasks of varying difficulty has been reported for simple arithmetic, lexical decision, reading words aloud, and memory search (Lupker, Brown, & Colombo, 1997; Lupker, Kinoshita, Coltheart, & Taylor, 2003; Rastle, Kinoshita, Lupker, & Coltheart, 2003; Strayer & Kramer, 1994a, 1994b). These findings show that reaction times are not simply determined by the task at hand (Kinoshita & Mozer, 2006). There seems to be a strategy, or at least some level of readiness, established prior to any situation where action is required. 

However, the underlying basis of this differentiated adjustment in readiness has not been fully examined. The differentiated responsiveness in the blocked trials could be based on explicit knowledge of upcoming trial types (Monsell, Sumner, & Waters, 2003; Sperling & Dosher, 1986; Stoffels, 1996; Van Duren & Sanders, 1988). Alternatively, it could be based on simple repetitions of trials of the same type building up passive cumulative learning, over which participants have very little control (Los, 1994; Strayer & Kramer, 1994a, 1994b; Treisman & Williams, 1984). 

The explicit knowledge account predicts that if participants know the next trial type, they can actively prepare for it optimally. This also fits well with our intuition that humans have intrinsic control over our actions (Arrington & Logan, 2004). In contrast, the cumulative learning account predicts that simply repeating trials of the same type will elicit the proper adjustment of strategies before the next trial. This passive learning account suggests that a short-term memory system based on very recent experience determines our strategy. Explicit prior knowledge plays no role in this account. 

In the current study, we used a simple, visually guided manual-pointing task to examine how the sequence of trials, especially the previous trial's difficulty, affects the “visuomotor readiness” of the subsequent goal-directed motor responses and what type of mechanism adjusts this readiness. We use the term visuomotor readiness to describe changes in motor strategies. This is best captured by the measurement of movement initiation latency (see below). 

This continuous pointing task can show whether the homogenization effect can be generalized to simple sensorimotor behaviors, which are not mediated by higher level conceptual knowledge such as arithmetic or language processing. It also provides a favorable opportunity to examine the role of strategies and their subsequent consequences in richer detail. In comparison to previous tasks requiring discrete responses, pointing reveals not only when behavior is initiated but also how it plays out in time and space. In a continuous response such as visually guided action, we can measure the real-time temporal dynamics of ongoing cognitive processes in a spatial context (Song & Nakayama, 2006). In each trial, we asked participants to point to either a single target (easy) or an odd-color target among two distractors (hard) and intermixed these two types of trials in various ways. 

To examine whether explicit knowledge, cumulative learning, or both contribute to the visuomotor strategies, we conducted a series of experiments in which we pit explicit knowledge of the upcoming trial against trial-type repetitions. In Experiment 1, we intermixed easy single-target trials and more difficult odd-color-target trials in three ways: blocked ( Figures 1A and 1B), mixed ( Figure 1C), and alternated conditions ( Figure 1D). In the blocked condition, single- and odd-color-target trials were presented in separate blocks. In contrast, in the mixed condition, the two types were randomly mixed. Finally, in the alternated condition, single- and odd-color-target trials were alternated and participants were informed about the alternation in advance. The task was always to reach for and touch a single or an odd-color target with their index finger as quickly and accurately as possible. 

Figure 1

 

Schematic diagram of experimental conditions in Experiment 1. Two types of trials were mixed in three different ways. In single-target trials (A), a single target was presented without distractors, whereas in odd-color-target trials (B), one odd-color target was presented with two distractors. Single- and odd-color-target trials were mixed in three different ways: blocked (A and B), mixed (C), and alternated conditions (D). In the blocked condition, single- and odd-color-target trials were presented in separate blocks. In the mixed condition, these were randomly mixed. In the alternated condition, single- and odd-color-target trials were alternated.

Both explicit knowledge and cumulative learning accounts predict that differentiated motor execution strategies would be adopted for the two trial types in the blocked condition due to either explicit knowledge for upcoming trials or cumulative learning based on repetitions of the same type of trials. Of interest is the alternated condition, where diverging results are predicted. According to the explicit knowledge account, the alternated condition would be similar to the blocked condition because participants have full knowledge of upcoming trials. However, according to the cumulative learning account, it should be similar to the mixed condition because frequent trial-type alternations prevent any possibility for cumulative learning. 

In Experiment 2, we displayed a long sequence of each type of trial and informed participants of the exact switching point of trial types. According to the explicit knowledge account, participants should show a differentiated strategy for each trial of these predictable/deterministic sequences, regardless of the length of repetition, whereas according to the cumulative learning account, they can only adjust their strategy after experiencing a sequence of repetitions. 

General methods

Participants

Harvard University students participated for course credit. They were all right-handed with normal color vision and normal visual acuity. Nine participants participated in Experiment 1, and nine new participants participated in Experiment 2. 

Stimuli

Solid red or green diamond-shaped stimuli (1.5° × 1.5°) were presented against a black background. The red and green were chosen to be approximately equiluminant using a flicker photometry. The stimuli were arranged uniformly around an imaginary circle with a radius of 14°. The target was positioned randomly from trial to trial at one of three possible positions, corresponding to 4, 8, and 12 o'clock. 

Task

Participants were tested individually in a semidarkened room. They were seated 48 cm in front of the visual display. There were two types of trials: single- and odd-color-target trials. In single-target trials, the single target was presented without distractors, whereas in odd-color-target trials, the odd-color target was presented with two distractors. The target color was randomly either red or green in each trial. If the target was red, then the distractors were green and vice versa in odd-color-target trials. Participants were asked to touch the odd-color target with their index finger as quickly and accurately as possible. They were also instructed to fixate the central crosshair throughout the trial. 

In each trial, the black screen with a white fixation mark was presented for 1,000 ms. Then, the stimulus display was presented on the screen until the participant responded. Auditory feedback was provided to inform whether participants touched the correct target. This was followed by a 500-ms blank interval. Thus, the total response stimulus interval was 1,500 ms. Participants practiced 80–100 randomly mixed single- and odd-color-target trials. 

In Experiment 1, participants performed two blocks of trials in each of the following conditions: blocked (one was a single-target trial block [ Figure 1A] and the other was an odd-color-target block [ Figure 1B]), mixed ( Figure 1C), and alternated ( Figure 1D). Each block contained 48 trials. The order of the six blocks was randomly assigned across participants. In Experiment 2, sequences of five single- and five odd-color-target trials switched regularly. Participants were informed about the five-trial-switch rule. An additional colored numeric cue was presented at the center for 500 ms during the blank interval before each trial began and indicated the trial type and position of the trial in the sequence (first, second, etc.). Yellow numbers were used for single-target trials, and blue ones were used for odd-color-target trials. Participants performed five blocks of trials (90 trials/block). 

Measuring hand movements

Hand movements were tracked with a Fastrak electromagnetic position and orientation measuring system (Polhemus Inc.) with an update rate of 120 Hz. The small position-tracking sensor (0.89 × 0.50 × 0.45 in.) was attached to the index fingertip of the right hand. The starting position (3 × 3 cm) was marked on the table, which was approximately aligned with the body midline and 20 cm in front of the participants. Participants were required to put their index finger on the starting position to initiate each trial. The tracking system was calibrated in each block with nine distributed points. 

Data analysis

Movement data were transmitted to a Power Mac G4 by Vision Shell library for off-line analysis to identify the onset and offset of movements. Hand velocity exceeding a threshold of 10 cm/s demarcated the onset of the movement. Each trajectory was visually inspected to verify the appropriateness of this criterion, which was adjusted by hand if necessary. 

Only trials in which participants selected the correct target were included in further analyses. Initiation latency was defined as the interval between stimulus and movement onsets. Trials in which latencies were below 100 ms or in excess of 1,500 ms were excluded as anticipatory movements and outliers. Movement duration was the interval between movement onset and offset. Total time was the sum of initiation latency and movement duration. Less than 2% of the trials were eliminated because of selection errors or latency criteria. 

We characterized the global movement trajectory by computing maximum curvature. Maximum curvature is defined as the ratio of the largest deviation (perpendicular distance) of the trajectory from the straight line connecting the start and end points (target position) of the movements to the length of this line (Atkeson & Hollerbach, 1985; Desmurget, Jordan, Prablanc, & Jeannerod, 1997; Smit & Van Gisbergen, 1990). Thus, maximum curvature was computed in each trial with respect to the target position. For example, a maximum curvature of zero means a straight trajectory. 

Results and discussion

Experiment 1: Dissociation of explicit knowledge and past experience

In Experiment 1, we intermixed easy single-target trials and difficult odd-color-target trials in the blocked, mixed, and alternated conditions to determine whether visually guided manual-pointing strategies are adjusted by predictability for upcoming trials or by cumulative learning based on trial repetitions ( Figure 1). 

Initiation latency adjustment

We observed that participants typically initiated reaching to a single target more rapidly than to an odd-color target among distractors, reflecting differentiated states of sensorimotor readiness for easy versus hard tasks, F(1, 8) = 16.7, p < .005. This is in accord with the prediction that participants can adjust their initiation latencies appropriately for the difficulty level of each trial ( Figure 2). 

Figure 2

 

Initiation latencies of single- and odd-color-target trials in the blocked, mixed, and alternated conditions. The error bars represent between-participants standard errors.

Yet, this occurred only when trial types were presented in separate blocks, t(8) = 6.3, p < .001, showing a significant interaction effect between trial type and block type, F(2, 16) = 36.5, p < .001. The difference between the two trial types vanished, on average, in the mixed condition, t(8) = 1.6, p < 1. Compared to the blocked condition, single-target trials became significantly slower, t(8) = 6.87, p < .001, and odd-color-target trials became faster, t(8) = 3.09, p < .02, showing a homogenization effect. In all conditions, target selection accuracies were more than 98%, F values < 1. 

The alternated condition represented the most critical result, where the two predictions diverge. Our results showed that there was no difference between the two trial types, t(8) = .4, p < 1, demonstrating the homogenization effect like that occurring in the mixed condition. Also, we observed that, compared to those in the blocked condition, single-target trials became significantly slower, t(8) = 6.57, p < .0001, and odd-color-target trials became faster, t(8) = 4.75, p < .001. 

Thus, in contrast to the blocked condition, the mixed and alternated conditions demonstrated that, on average, motor execution strategies were not differentiated for each trial type. It is worth noting that participants have perfect explicit knowledge of upcoming trial types in the alternated condition. Yet, participants did not actively adjust their motor execution strategies by relying on their explicit knowledge. Therefore, the results support the passive cumulative learning hypothesis that trial repetitions are critical for adjusting motor readiness. 

Trial-by-trial sequential adjustment

Figure 2 reveals that participants adjust to the average difficulty for all the trials in the mixed blocks. They appear to adopt a single strategy by showing no differences in initial latencies. Yet, if cumulative learning is operative and important, we should see its effects on a trial-by-trial basis even in this mixed condition. We categorized sequences into three groups based on the number of same trial-type repetitions in the mixed condition: zero, one, and more than two repetitions. Then, the initiation latency difference between single- and odd-color-target trials was calculated in each repetition condition. Figure 3 shows that with increasing numbers of repetitions, the difference between single- and odd-color-target trials also increased, F(2, 7) = 2.9, p < .05, although participants did not have any explicit knowledge of upcoming trials. 

Figure 3

 

Initiation latency differences between odd-color- and single-target trials in the mixed condition as increasing numbers of same trial-type repetitions. The error bars represent between-participants standard errors.

Thus, we dissociated influences of explicit knowledge and past experience by demonstrating that initiation latencies are adjusted not only by the full knowledge of upcoming trials in the alternated condition but also by recent experience when trials of the same type are unpredictably repeated in the mixed condition. 

Cost of suboptimal strategies—Curved trajectories

We also examined whether there is any cost of latency homogenization in the mixed and alternated conditions ( Figure 2). Although participants successfully fulfilled their ultimate task goal of target selection in all conditions as we reported earlier, the presence of curved movement trajectories shows costs of suboptimal motor execution strategies ( Figure 4). Figure 4 shows more curved trajectories toward distractors in the mixed and alternated conditions than in the blocked condition. 

Figure 4

 

Manual-pointing trajectories to the three target locations from one participant. The first column (cyan) represents trajectories from single-target trials, and the second column shows odd-color-target trials. In the odd-color target trials, trajectories associated with each target location are depicted by three distinct colors: 8 o'clock position (green), 12 o'clock position (red), and 4 o'clock position (blue). These trajectories are three-dimensional, but for clarity, we show only the x and y dimensions where the difference between trajectory types is most evident. Trajectories from the blocked (A), mixed, (B) and alternated (C) conditions are depicted in separate rows.

To confirm the impression from Figure 4, we quantified the magnitude of trajectories by calculating maximum curvature. Figure 5 shows higher maximum curvatures of the odd-color trials in the mixed, t(8) = 3.2, p < .02, and alternated conditions, t(8) = 2.5, p < .05, than in the blocked condition, supporting the above interpretation of Figure 4. Overall, single-target trials are less curved than odd-color-target trials, F(1, 8) = 33.1, p < .001. We also observed a significant block type effect, F(2, 16) = 3.9, p < .05, and an interaction effect between trial type and block type, F(2, 16) = 4.3, p < .05. 

Figure 5

 

Mean maximum curvatures of single- and odd-color-target trials in the blocked, mixed, and alternated conditions. The error bars represent between-participants standard errors.

Thus, undifferentiated initiation strategies (homogenization) resulted in some cost reflected in curved trajectories, although there was no evident cost in final selection accuracies. The time cost caused by curved trajectories was negligible. Overall, movement duration was shorter in single- than in odd-color-target trials, F(1, 8) = 35, p < .001, but there were no other significant effects: 408, 406, and 403 ms for single-target trials and 447, 459, and 454 ms for odd-color-target trials, respectively, in the blocked, mixed, and alternated conditions. 

Experiment 2: Cumulative learning in predictable sequences

To further extend the finding from the sequence analysis of the mixed condition ( Figure 3) and investigate the time course of the adjustment of initiation latency criteria based on cumulative learning, we introduced longer predictable/deterministic sequences in Experiment 2. In this task, runs of five single- and five odd-color-target trials switched regularly (Allport, Styles, & Hsieh, 1994; Rogers & Monsell, 1995). Participants were informed about the five-trial-switch rule, and visual cues for trial types and which trials in the sequence (first, second, etc.) were also presented in each trial. Thus, participants had perfect cognitive knowledge for the upcoming trial types. 

Figure 6 demonstrates that initiation latencies for the two types were gradually differentiated as numbers of repetitions increased, consistent with results depicted in Figure 3. We observed a significant trial-type effect, F(1, 8) = 7.7, p < .05, repetition effect, F(4, 32) = 13.9, p < .001, and interaction effect between them, F(4, 32) = 42.1, p < .001. 

Figure 6

 

Initiation latencies of odd-color- and single-target trials as a function of the number of the same trial-type repetitions. The error bars represent between-participants standard errors.

This indicated that participants gradually initiated their movements faster in single-target trials and slower in odd-color-target trials with increasing repetitions, which could not be explained as a simple perceptual priming effect because trial-type repetitions slowed down odd-color-target trials. 

Initiation latency differences between single- and odd-color-target trials were statistically significant except for those in the zero-repetition condition, where the response for the type of trials was reversed, t(8) = 1.5, p < 1. The difference emerged only after one (35 ms), t(8) = 2.9, p < .02, or two repetitions (50 ms), t(8) = 3.1 p < .015, and gradually increased up to 62 ms, t(8) = 4.3, p < .003. Thus, with increasing the number of repetitions, initiation latencies of single- and odd-color-target trials were clearly separated as in the blocked condition of Experiment 1 ( Figure 2). 

This, again, supported the cumulative learning account by trial-type repetitions. In addition, the lack of latency difference at the switching point (zero repetition) showed that prior knowledge is not directly involved in this adjustment. 

General discussion

In the current study, we dissociated influences of explicit predictive knowledge and recent experience on visuomotor readiness, showing that cumulative learning based on recent repetitive experiences is critical. 

We showed that participants fail to adjust their motor strategies appropriately for trials of varying difficulty, although they have explicit knowledge of upcoming trials ( Figures 2 and 6, zero repetition). Considering our general intuition that humans have a degree of control over their actions, it is quite surprising that certain knowledge has no effect in overall visuomotor readiness, although it is conceivable that participants could explicitly adjust their visuomotor strategies under conditions of differentiated rewards and penalties, which we have not examined. Yet, in the current study, participants can apply distinctive motor initiation strategies for easy and hard trials by repetitions regardless whether they have explicit knowledge of upcoming trials ( Figures 3 and 6). As repetitions of same types increase, initiation latencies for easy trials became faster and those for difficult trials became slower, so that these are clearly separated. Thus, we argue that the visuomotor system determines motor initiation criteria based on very recent events. This cannot be simply explained by perceptual priming effects because priming would produce decreased reaction times for both trial types (Kristjansson, Wang, & Nakayama, 2002; Maljkovic & Nakayama, 1994). 

The initiation latency differences between easy and difficult trials vanished when the two trial types were intermixed because easy trials became slower and difficult trials became faster compared to when they were separated in blocks ( Figure 2). This homogenization effect differs in this respect from the typical task-switching costs, which would be expected to produce performance decrements in both tasks when they were intermixed. 

As mentioned earlier, the homogenization effect of easy and hard trials has been observed in a variety of other tasks (Kinoshita & Mozer, 2006), although domain-specific accounts have been raised to explain this homogenization effect, such as a dual-route model assuming a relative speed change in lexical and nonlexical routes in reading (Rastle & Coltheart, 1999). 

One possible hypothesis to explain similar reaction time modulation observed from higher level cognitive tasks and our simple reaching tasks is to assume a central mechanism independent from any input and output systems. This conjectured central monitoring system determines sensorimotor readiness for a current trial based on recent experiences. The current state of response readiness is not simply determined by the task but modulated by recent experience. Separate parallel channels could accumulate information for each of the competing responses over time. Then, as soon as accumulated information for one response over the other reaches the threshold, a motor response could be executed. Recent experience could adjust either the gain of information accumulation or threshold of movement onsets (Palmer, Huk, & Shadlen, 2005; Ratcliff, Cherian, & Segraves, 2003; Smith & Ratcliff, 2004; Stuphorn & Schall, 2002). 

This view is elaborated further by our trajectory measurements. More difficult trials not only are faster but also show more curved trajectories when mixed with easy trials. This supports the view that the initiation of motor behavior, but not its full planning, is triggered more quickly, either by lowered threshold or by faster activation along a decision axis. 

Our trajectory measurement demonstrated unique characteristics of visually guided actions. In contrast to discrete responses typically measured in cognitive tasks, reflecting only final outputs of internal cognitive process, our task shows how suboptimally adjusted initial strategy interacts with behaviors over the time until participants finally obtain the right target (Jeannerod, 1988; Song & Nakayama, 2006; Spivey, Grosjean, & Knoblich, 2005). Reaching trajectories initially attracted to distractors and then corrected to the target demonstrate how competition between the target and distractors is resolved over time. This interactive process between target selection and motor control provides an alternative to traditional cognitive theories based on the assumption that perception, cognition, and action are distinctive and serially processed domains (Marr, 1980; Newell & Simon, 1972; Sanders, 1980; Sternberg, 1969). 

For simplicity, we have assumed that a single central system is responsible for determining a very unified strategic control of the sensorimotor system. Furthermore, it is also possible that a single system is involved in determining a general control level for a wide range of cognitive and motor actions, given the fact that homogenization effects are widely observed. However, it is an empirical question as to whether a single system or perhaps multiple systems, each for different types of processing (language, phonology, eye movements, hand movements, etc.), are capable of making their own separate adjustments. 

In conclusion, we show that the strategy to deal with task difficulty is in fact regulated automatically by recent experience alone. There seems to be no role for explicit knowledge of upcoming tasks in determining visuomotor readiness. 

Acknowledgments

J.H.S. is supported by the Korea Foundation for Advanced Studies. We thank Patrick Cavanagh, Yuhong Jiang, Sander Los, and anonymous reviewers for comments. 

Commercial relationships: none. 

Corresponding author: Joo-Hyun Song. 

Email: [e-mail]. 

Address: The Smith–Kettlewell Eye Research Institute, 2318 Fillmore Street, San Francisco, CA 94115, USA.