Driving in the future: Temporal visuomotor adaptation and generalization

Abstract

Rapid and accurate visuomotor coordination requires tight spatial and temporal sensorimotor synchronization. The introduction of a sensorimotor or intersensory misalignment (either spatial or temporal) impairs performance on most tasks. For more than a century, it has been known that a few minutes of exposure to a spatial misalignment can induce a recalibration of sensorimotor spatial relationships, a phenomenon that may be referred to as spatial visuomotor adaptation. Here, we use a high-fidelity driving simulator to demonstrate that the sensorimotor system can adapt to temporal misalignments on very complex tasks, a phenomenon that we refer to as temporal visuomotor adaptation. We demonstrate that adapting on a single street produces an adaptive state that generalizes to other streets. This shows that temporal visuomotor adaptation is not specific to a single visuomotor transformation, but generalizes across a class of transformations. Temporal visuomotor adaptation is strikingly parallel to spatial visuomotor adaptation, and has strong implications for the understanding of visuomotor coordination and intersensory integration.

Introduction

In any visually guided behavior, the visual location of an object must be mapped to the appropriate motor coordinates. For example, when we wish to pick up a coffee cup, the visual coordinates of the cup must be converted to the appropriate motor commands to move our hand to the proper point in space. One traditional method for studying visuomotor coordination involves perturbing the spatial relationship between vision and action. A number of different types of perturbations have been used, including altering the visuomotor gain (eg, so that the hand physically moves less than the resultant visual displacement [Ingram et al, 2000]). The most traditional method, however, is to alter the visually perceived location of an object, so that it appears to be at a point in space different from its actual location. This is usually done by laterally offsetting, rotating, or left-right reversing the visual field (for reviews, see Bedford, 1993; Welch, 1978). The resulting discrepancy between the seen and felt location of an object greatly impairs visually guided behaviors. Consider, for example, when the visual field is shifted to the left with prism goggles. When subjects are asked to reach toward a target without being able to see their hand, they initially reach toward the visual location of the object (to the left of the actual location) and fail to touch the object. A few minutes of acting with the new visuomotor relationship (and receiving both visual and haptic feedback regarding the correct location of the object) returns behavior to nearly normal performance levels. Subsequent removal of the spatial perturbation (eg, taking off the prism goggles) leads to a renewed disruption of performance. This aftereffect is in the opposite direction of the initial error (eg, subjects now reach to the right of the object when they cannot see their hand) and as such is generally referred to as a negative aftereffect. This form of visuomotor learning is best described as spatial visuomotor adaptation, and the change in performance when the perturbation is removed is one of the most common measurements of the strength of adaptation (Welch, 1978). 

In addition to spatial coordination, the visual control of behavior also requires very tight temporal synchronization. For example, when we wish to catch a moving object, perceiving the location of the object is insufficient; we must also perceive when it was at that location. The internal delays associated with processing the visual stimulation and with initiating a motor act further complicate the temporal coordination of perception and action. That is, we do not move our hand to where the object was when we began processing the visual information (its sensed location) but to where it will be when our hand can finally arrive at the proper location. The lag between the beginning of perception and the beginning of the motor act (reaction time) varies considerably. Simple tasks (eg, pressing a button as soon as a light turns on) can yield reaction times as short as 150 milliseconds (Teichner & Krebs, 1972), whereas more complex situations yield larger reaction times (eg, while driving a car, reaction times of 1 to 2 seconds are commonly observed between seeing a signal and starting a braking maneuver [Fambro, Koppa, Picha, & Fitzpatrick, 1998]). We routinely compensate for these internal delays in an effortless and unconscious fashion, and in many cases even fail to notice that there is a lag between perception and action. Despite the ubiquity of this temporal compensation, the nature of the underlying mechanism is still largely unknown and is the source of considerable debate. 

Given the tight temporal restrictions on rapid interaction with the world, it should not be surprising that altering the temporal relationship between perception and action drastically impairs many closed-loop behaviors. One common way of altering the temporal visuomotor relationship is to delay visual feedback (eg, inserting a lag between the time an object is pushed and when it is seen moving). Delays as small as 40 milliseconds significantly impair rapid visually guided behaviors like pursuit tracking (Warrick, 1949, 1955). Delays of a second or more can effectively prohibit the rapid and accurate closed-loop interaction with the world (Sheridan & Ferrel, 1963; Smith, McCrary, & Smith, 1962; Smith, Wargo, Jones, & Smith, 1963). 

Despite the formal similarity between spatial and temporal perturbations in visuomotor coordination, and the fact that humans already compensate for an internal lag between perception and action, previous research has found little evidence of compensation for external delays (Held, Efstathiou, & Greene, 1966; Sheridan & Ferrel, 1963; Smith et al, 1962; Smith et al, 1963). In these studies, subjects tended to slow down when the delay was introduced, and slowing down should prevent visuomotor adaptation from occurring. More specifically, for visuomotor adaptation to occur, people must be exposed to the altered sensory relationship (Welch, 1978), and it can be readily shown that slowing down essentially negates the effects of the delay. For example, a driver traveling 72 km/h in a car with a 1-second delay must turn the steering wheel 20 meters prior to reaching an intersection. When traveling at 7.2 km/h, however, the driver needs to turn only 2 meters early; the driver can act as if there were no delay and turn once in the intersection. 

Recently, Cunningham, Billock, and Tsou (2001) demonstrated that when people are prevented from slowing down, and are thus exposed to the consequences of the temporal perturbations, they do adapt. In their study, a small figure descended from the top of the monitor at a constant vertical speed. Subjects were asked to use an isometric mouse to maneuver the figure through a dense field of obstacles. Each subject’s ability to perform the task without a delay was measured before and after training with a 235-millisecond visual feedback delay. The introduction of the delay initially impaired performance, but with a small amount of practice (between 5 and 20 minutes) subjects learned to perform almost equally well with delayed feedback as they could with immediate feedback. Subsequent removal of the delay produced a large drop in performance. The full pattern of results, as well as the failure of previous work to find such improvements, is best explained if the improvement is due to temporal visuomotor adaptation. 

In most forms of adaptation, aftereffects are obtained only when the adapting and testing situations are similar, and spatial visuomotor adaptation is not an exception to this rule. Determining which dimensions affect the strength of the aftereffect provides important insights into the mechanism behind the adaptation. For prism adaptation and other forms of spatial visuomotor adaptation, there is a great degree of specificity for the response pattern, although this point is often overlooked. For example, there is little or no intermanual transfer of prism adaptation (adapting to right-handed reaching produces little or no adaptation for reaching with the left hand and vice versa [Welch, 1978]). Even within a given motor system, changes in the trajectory of the motion can limit adaptation. For example, learning to throw clay balls underhanded with prism goggles does not transfer to overhanded throwing and vice versa (Martin, Keating, Goodkin, Bastian, & Thach, 1996). Moreover, even when the motion trajectory is the same, the velocity of the response is important (Kitazawa, Kimura, & Uka, 1997). Kitazawa et al found that when reaching, the size of the negative aftereffect decreased the more the reaching speed used during the test differed from the adapted reaching speed. These and other results strongly suggest that prism adaptation is not only a “recalibration” of the relationship between the seen and felt positions but also involves changes in the visuomotor transformation (Welch, Choe, & Heinrich, 1974). 

One notable dimension across which prism adaptation does generalize is spatial location. In a pioneering study, Bedford (1989) demonstrated that adaptation to reaching toward one point in space generalizes to other points. That is, the same size negative aftereffect was obtained for reaching toward the trained spatial location as was obtained for reaching toward untrained spatial locations. This suggests that spatial visuomotor adaptation is not simply the alteration of a single visuomotor transformation (eg, reaching to a specific location in space), but instead affects an entire class of transformations (eg, reaching in general), at least within the constraint that the reaching trajectories need to be relatively similar. 

Here we begin to enumerate the specificity of temporal visuomotor adaptation. Of particular interest is whether temporal visuomotor adaptation alters only a single specific visuomotor transformation or produces a more general shift for an entire class of transformations. To examine this, we employed a high-fidelity driving simulator. Driving has the advantage of being inherently path based, allowing the controlled manipulation of the complexity and similarity between trained and untrained paths. Driving also has the advantage of being a very complex skill that cannot be learned in a few minutes. Thus, improvements seen during training with a delay cannot be due to subjects completely relearning how to drive. Within the context of driving, the primary question of interest is whether training to navigate along a single path with a delay produces an adaptive state only for the specific turns and sequence of turns of that path, or whether the adaptation generalize to other turns and sequences. 

Experiment 1

Our driving simulator used a first-person perspective, unlike the top-down view in the task used by Cunningham et al (2001). Naturally, a first person perspective limits the amount of the street that can be seen. It is unclear how such limits affect temporal visuomotor adaptation. Because successful driving with delayed feedback requires the driver to turn earlier than usual, limits on how far ahead the driver can see may impair adaptation. Additionally, driving uses an acceleration-based input device (a steering wheel) that controls angular heading, rather than the isometric position-based input used by Cunningham et al (2001), which may also affect temporal visuomotor adaptation. Therefore, before investigating the generalization of temporal visuomotor adaptation, we first needed to determine if it can occur for driving. We ensured that subjects were exposed to the delay (and thus had the chance to adapt to it) by allowing them to control only the direction of travel. The speed was constant for the duration of each trial, with each subject being exposed to several different speeds. We also examined a range of delay magnitudes. 

Methods

Subjects

Twenty-one paid volunteers participated in the experiment. Five subjects developed simulator sickness and did not complete the experiment. Their data have been eliminated from the analyses. The data from one additional subject were eliminated because the subject had never driven and proved unable to control the car, even at a very slow speed with immediate feedback. Subjects were randomly assigned to one of the three delay magnitude conditions until each condition had a total of five subjects. 

Apparatus

The virtual road environment was projected onto a half-cylindrical 180-degree screen (3.15 m high, 7 m diameter) by a Silicon Graphics Onyx 2 Infinite Reality Engine (Mountain View, CA) (for a demonstration of the task with a 400-millisecond delay, see Movie 1). Subjects controlled the car via a custom-designed, forced-feedback steering wheel. During each trial, the steering wheel angle and the position and heading of the virtual car were recorded at 36 samples per second. 

 

Movie 1

 

A demonstration of the experimental setup and task with a 430-millisecond delay.

Stimuli

The same street (Figure 1) was used in all sections of Experiment 1. To provide a realistic and familiar driving environment, the street was generated according the formulas that the German government uses in designing real streets (Forschungsgesellschaft für Straßen- und Verkehrswesen: Arbeitsgruppe Straßenentwurf, 1995). Note that the street was generated using spirals and clothoids, so that the curvature of any given turn changed gradually. This means that one could drive along the entire street using smooth steering maneuvers. To make the task difficult and to reduce the length of the experiment, the curves were considerably sharper than the average German street and there were no straight sections. 

Figure 1

 

Top-down view of the street used in Experiment 1. Each trial began at the end of the road nearest the figure caption.

Procedure

The subjects maneuvered a virtual car (approximately 1.9 m wide by 4.5 m long) along a curved street (10 m wide) in a high-fidelity virtual environment, using a forced-feedback steering wheel. Subjects were asked to drive to the end of the street without leaving the road. The street consisted of four lanes. Subjects were asked to remain in the second lane from the right if possible, but were told that staying on the road was a higher priority (ie, they were encouraged to use all four lanes if necessary to stay on the road). 

Just as slowing down minimizes the effects of the delay, increasing the speed increases the effects of the delay. To maximize the effects of the delay, fast speeds were chosen: between 64 and 108 km/h (18, 22, 26, and 30 m/s). In general, the speeds were fast enough to force subjects to drive at or near the upper limits of their abilities. Of course, driving along a narrow curved street at high speeds is a very demanding task that requires intense concentration. To minimize the possibility of fatigue, the number of speeds, the number of repetitions per speed, and the length of the road were all chosen so that the experiment lasted only 30 to 45 minutes. To familiarize the subjects with the experimental setup and the control of the virtual car in particular, each subject was given several practice trials (with no feedback delay) prior to the start of the experiment. All practice trials were at the slowest speed (18 m/s). The experiment proper consisted of three sections: pretest, training, and posttest. 

Pretest

Traditionally, the pre- and posttest sections of a visuomotor adaptation experiment are presented without feedback. That is, the task is performed in an open-loop manner. It is, however, not possible to drive along a long, winding road without visual feedback. Moreover, open-loop driving is known to be qualitatively different from closed-loop driving (Cavallo, Brun-Dei, Laya, & Neboit, 1988). Thus, the pre- and posttest sections of this experiment were presented in a closed-loop manner. The pretest, then, provided a baseline measurement of how well subjects could drive a virtual car with immediate feedback. During the pretest, each subject was presented with five repetitions of four speeds in random order, for a total of 20 trials. The fastest speed at which a subject could successfully drive to the end of the street on at least four of the five repetitions was recorded as his or her top speed. This speed played an important role in the posttest. 

Training

During the training section, the steering wheel controlled the car in the same manner as in the pretest, with the sole exception that there was a delay between when steering wheel was turned and the resultant motion of the car. For one third of the subjects, the delay was 130 milliseconds. For another one third of the subjects, the delay was 230 milliseconds. For the remaining one third, the delay was 430 milliseconds. Prior to the onset of training, subjects were informed that there would be a delay between use of the steering wheel and motion of the car, but were not told how long the delay would be. 

The order in which the speeds were presented was determined using a shaping-by-approximation training procedure. Specifically, the slowest speed was repeatedly presented until one of three criteria was met. If a subject successfully reached the end of the street four times in a row (success criterion), the speed was increased and training continued. If a subject drove off the street 10 times in a row (collision criterion), training ended and the posttest began. If neither the success nor the collision criteria were met within 20 trials (stalemate criterion), training ended, and the posttest began. In a shaping procedure, it is important that the initial task not be too difficult and that the increase in difficulty between subsequent levels of training not be too great. If the initial task is too difficult, or if there are large differences between levels of training, a larger number of repetitions per level of training is required to achieve mastery of that level. To reduce the number of repetitions per speed level that were required and thereby minimize the risk of fatigue, the difficulty of the initial task was reduced by using a slower speed (14 m/s; this speed was determined based on pilot data with the 230-millisecond delay). 

Although gradual training procedures are not the standard in visuomotor adaptation studies, they have been used (Field, Shipley, & Cunningham, 1999; Ingram et al, 2000; Kagerer, Contreras-Vidal, & Stelmach, 1997) and offer some distinct advantages. For example, Kagerer et al compared visuomotor adaptation to a spatial perturbation (the visual display was rotated by 90 degrees) for both gradual and sudden changes in the spatial offset. The form of adaptation was the same in both cases, but the adaptation was stronger and more robust when the spatial perturbation was gradually changed from none to a 90-degree rotation. Ingram et al (2000) also found that gradual training procedures produced stronger adaptive states. Additionally, a general advantage of gradually exposing subjects to changes in a complex task is that such a procedure can reduce the number of errors, training time, and the amount of subjects’ frustration. 

Posttest

During the posttest, performance with immediate feedback (no delay) was re-measured. At the start of the posttest, subjects were informed that there would no longer be a delay, and that this section of the experiment was the same as the first (pretest). The difference in performance between the pre- and posttest sections provides a measurement of the aftereffects of training. For a proper comparison of the pre- and posttests, the two sections were as similar as possible. It is possible, however, that subjects could re-adapt to the immediate feedback during the posttest, masking any aftereffect of training. To avoid re-adaptation to immediate feedback, only five trials were presented during the posttest. For each subject, all five trials were presented at their top speed from the pretest. 

Analyses

Three complementary analyses were performed. First, the number of times that the subject could drive all the way to the end of the street was calculated (street completion). Although this is a rather coarse measure of performance, it proved to be a very accurate measure of driving competence, at least under these conditions (ie, when subjects were driving at the upper limits of their abilities). 

We also measured each subject’s position on the road (lateral position error). Specifically, the lateral deviation of the car from the center of the assigned lane was calculated (Figure 2). The standard deviation of lateral position provides a measure of the subject’s ability to accurately control the virtual car. To ensure a reliable estimate, only trials where subjects successfully drove at least as far as the second curve were analyzed. The fact that a trial ended when the subject left the road places an upper limit on the maximum possible deviation of the car’s position from the center of the assigned lane. This, in turn, means that the lateral position error may provide an underestimate of a subject’s driving ability when that subject is having extreme difficulty. It also suggests that the lateral position error should be well correlated with street completion metric. 

Figure 2

 

An illustration of the lateral position error calculation. The street is four lanes wide with the thick solid black line depicting the center of the road. The green line represents the center of the second lane from the right: the location where subjects were supposed to be driving. The blue line shows the path actually driven by one subject. The red arrow represents the lateral difference between assigned and actual position for one point in time. This lateral difference was calculated for the car’s location on every frame, and the standard deviation of these differences is the lateral position error.

Finally, we measured the subject’s steering behavior (number of reversals). If a subject was having trouble controlling the car, there should have been an increase in corrective-steering behavior. This should show up as an increase in steering wheel oscillations. The number of reversals was obtained by finding all zero crossings in the first derivative (Figure 3). To eliminate measurement artifacts introduced by digitizing the steering wheel’s angle, the wheel’s change over time was closely approximated with a smoothed curve (Figure 3). Specifically, the smoothing was implemented with a zero-phase forward and reverse digital filter with cosine profile and a length of 140 milliseconds. As with the lateral position error measurement, a reliable baseline for measurement was ensured by examining only those trials when the subject completed at least the first two curves. 

Figure 3

 

An illustration of the number of reversals calculation. The steering wheel angle is plotted as a function of time. The original measurement is in black, and smoothed measurement is plotted in green. The red dots depict the reversals.

Results and Discussion

Overall, the pattern of results for the 230-millisecond delay group is remarkably similar to that found by Cunningham et al (2001). The introduction of a delay impaired performance, but with practice, each subject’s performance improved considerably. The subsequent removal of the delay produced a renewed decrease in performance. The results for each of the three portions of the experiment are discussed in more detail below. 

Pretest

The average top speeds during the pretest were 90.72, 99.36, and 99.36 km/h for the 130-, 230-, and 430-millisecond delay groups, respectively. These speeds do not differ significantly [t(8) < 0.85]. (All within group t tests were dependent measures tests, and all between group t tests were independent measures tests.) Table 1 lists the average performance at each subject’s respective top speed during the pretest and posttest. Overall, the three groups had roughly similar driving abilities at the onset of the experiment, with one possible exception: the 430-millisecond group’s lateral position error was slightly lower than the 230-millisecond group’s error during the pretest. This difference did not reach significance [t (8) = 2.125, P < .067]. For all other measures, performance for the 3 groups during the pretest were not significantly different [t (8) < 1.75, all P values > .11]. 

Table 1

 

Performance at each subject’s top speed for all three groups during the pretest and posttest

	130-ms Delay Group		230-ms Delay Group		430-ms Delay Group
	Pretest	Posttest	Pretest	Posttest	Posttest
Street Completion, %	100	92	92	28	92	84
Lateral position error, m	1.26	1.28	1.46	1.78	1.19	1.19
Number of reversals, per 100 m	5.2	4.33	3.95	4.7	5.57	5.34

Training

For all three groups, the introduction of the delay impaired performance and altered subjects’ driving patterns (Table 2). For the 130- and 230-millisecond groups, subjects used significantly more of the street on the first trial during training than they did on the slowest speed presented in the pretest [t (4) = 2.92, P < .03, and t (4) = 2.29, P < .042, for the 130- and 230-millisecond delay groups, respectively]. Although both groups showed a trend toward an increased number of reversals, these increases were significant only for the 230-millisecond group [t (4) = 0.9275, P > .2, and t (4) = 2.030, P < .056 for the 130- and 230-millisecond delay groups, respectively]. For the 430-millisecond delay group, the impairment in driving caused by the delay was so large that only one subject made it past the first curve during the first few trials. 

Table 2

 

Performance for the slowest speed during pretest (18 m/v) and training (14 m/s)

	130-ms Delay Group			230-ms Delay Group
	Pretest (18 m/s)	First Taining Trial at 14 m/s	Last Training Trail at 14 m/s	Pretest (18 m/s)	First Training Trial at 14 m/s	Last Training Trial at 14 m/s
Lateral position error, m	0.93	1.25	1.27	1.41	1.78	1.32
Number of reversals, per 100 m	5.99	7.61	6.47	5.53	7.10	6.02

For all three groups, there was a tendency toward improved performance during training on the first speed (14 m/s). For the 130-millisecond groups, the number of reversals decreased by the last trial for this initial speed, on average, although this change was not significant [t(4) = 1.582, P < .095]. For the 230-millisecond group, the lateral position error decreased and the number of reversals increased by the last trial for this initial speed, although only the decrease in lateral position error was significant [t (4) > 2.966, P < .021 and t (4) = 0.798 for the lateral position error and the number of reversals, respectively]. For the 430-millisecond group, there was a large increase in the amount of the street completed from the first training trial to the last. One of the subjects completed the street on the first trial, but the remaining four were able to remain on the street for only 6.2 seconds, on average, at the onset of training. By the end of training, these four subjects could remain on the street for 14 seconds on average [this change approached significance; t (3) = 1.90, P < .077]. It appears, then, that subjects were adapting to the delay in all three conditions, but the growth of adaptation was slow in the 430-millisecond group. Notice that the larger the delay is, the more difficult a given speed will be (ie, the earlier one would need to turn). Thus, an initial task that is easy with a 230-millisecond delay (at 14 m/s, subjects only have to turn 3.2 m early on the 10-m wide road) might be very difficult with a 430-millisecond delay (where subjects need to turn 6.0 m early). As mentioned earlier, learning can be hampered in a shaping procedure when the initial task is too difficult and an insufficient number of training trials are given. Given that subjects were improving in the 430-millisecond delay, it is likely that more trials at this speed level would strengthen the adaptive state. Likewise, reducing the relative difficulty of the initial speed should speed the growth of adaptation. This latter possibility is explored in Experiment 2. 

Posttest

All groups showed a drop in the ability to drive to the end of the street, but this drop was small for the 130- and 430-millisecond delay magnitudes. A three-way analysis of variance (ANOVA) was performed on the street completion percentages, with delay magnitude a between-subjects factor and test type (pre- versus posttest) a within-subjects factor. Overall, there were significant main effects for delay magnitude [F(2,12) = 35.73, P < .0001] and for test type [F(1,12) = 34.78, P < .0001]. The two-way interaction was also significant [F(2,12) = 17.04, P < .0001]: The 230-millisecond group experienced a larger drop in performance than the other two groups. 

Although subjects in the 130-millisecond group tended to do worse in the posttest than they did in the pretest (Table 1), none of these changes were significant [t (4) < 1.63, P > .09, t (4) = 0.29, and t (4) = 1.555, P < .098 for the street completion, lateral position error, and number of reversals measures, respectively]. One possible reason for this lack of significant aftereffect is that a 130-millisecond delay is quite similar to immediacy, at least for the present task. It is worth noting that real cars do not respond immediately (inertia and various plant dynamic factors introduce delays). Thus, although the delay involved in real cars is less than 130 milliseconds, it may be sufficiently similar that previous driving experience generalizes to a 130-millisecond delay. This explanation is, however, undermined by the fact that the introduction of a 130-millisecond delay significantly impaired driving performance (see the training section analyses above). 

The results from the 230-millisecond group show a clear aftereffect: after removal of the delay, subjects’ ability to control the car decreased drastically (Table 1). Street completion was between 40% and 80% lower in the posttest than for the same speed during the pretest, with an average drop of 64% [t (4) = 8.55, P < .0005]. These results are very similar to those that Cunningham et al (2001) found for a simple top-down view obstacle avoidance task with a 235-millisecond delay. Interestingly, the average size of the aftereffect is remarkably consistent between the two experiments (64% and 52%, respectively). Given that subjects could not stay on the road, it should not be surprising that they could not stay in the assigned lane, either. There was a significant increase in the lateral position error from pre- to posttest [t (4) = 2.966, P < .03]. There was also an increase in the number of reversals [t (4) = 2.1829, P < .05]. These results indicate that subjects were having trouble controlling the car. It is worth noting that the 130-millisecond delay group had a similar amount of training (36.6 and 30 trials, on average, for the 230- and 130-millisecond groups, respectively), which strongly suggests that the drop in performance for the 230-millisecond delay is not due to fatigue. 

For the 430-millisecond group, there was little change in performance between the pre- and posttests [t (4) = 0.78, t (4) = 0.34, and t (4) = 0.57 for the street completion, lateral position error, number of reversals measures, respectively]. Given that subjects did not adapt very well during training, there is little reason to believe training affected posttest performance. One subject did manage to complete training for the slowest speed (14 m/s), but did not perform well at the next speed level (18 m/s). This subject showed a 40% drop in completion rate during the posttest, suggesting that to the degree that subjects can learn to drive with a 430-millisecond delay, this improvement will produce an aftereffect when the delay is removed. 

Experiment 2 Experiment 1 showed that temporal visuomotor adaptation can occur in a driving task. Experiment 2 examined the generalizability of temporal visuomotor adaptation within a class of visuomotor transformations. Experiment 2 was largely identical to Experiment 1. In particular, three groups of subjects participated (130 millisecond-, 230 millisecond-, and 430 millisecond-delay groups), and the experiment itself was divided into three sections (baseline, training, generalization). There were, however, two important changes. First, unlike Experiment 1, the effects of steering were delayed during all three sections of Experiment 2. Second, several different streets were used. During the baseline section, subjects were asked to drive along four streets with delayed visual feedback (Figure 4as–Figure 4d). During the training section, they were trained with the same delay on the street from Experiment 1. In the final section (generalization), they were re-tested with the same delay on the four streets from the baseline section, and on four new streets (Figure 4e–Figure 4h).

Figure 4

 

Eight of the streets used in Experiment 2. Streets a-d were used in the baseline section. Streets a-h were used in the generalization section. All trials started at the end of the road that is closest to the figure label.

Experiment 2 addressed two primary questions. First, if temporal visuomotor adaptation is specific to a particular visuomotor transformation, then performance should be similar during the baseline and generalization sections. If, on the other hand, temporal visuomotor adaptation affects an entire class of visuomotor transformations, one would expect to see higher performance in the generalization section than in the baseline. Second, if the degree of novelty of the untrained streets is important, then subjects should be better during the generalization section on the streets they saw in the baseline section than on the four completely new streets. Although each street in the baseline was seen only four times (once at each of the four speeds), recent work on the memorization and replication of turns has shown that humans can accurately reproduce a sequence of three turns after only two repetitions (von der Heyde, 2001). So, one might refer to the baseline streets as old and the four new streets as novel. 

Methods

Subjects

Sixteen subjects participated in the experiment. One subject developed simulator sickness and did not complete the experiment. These data have been eliminated from analysis. Subjects were randomly assigned to one of the three delay conditions until each condition had a total of five subjects. 

Apparatus and Stimuli

The apparatus and stimuli were the same as in Experiment 1, except that eight additional streets were used (Figure 4). All of the streets were 10-meters wide and of similar complexity. 

Procedure

The procedure was similar to that used in Experiment 1, with a major exception that all sections of the experiment had delayed feedback. During the baseline section, four streets (Figure 4a–4d) were presented once at each of the four speeds in random order, for a total of 16 trials. The training section used the same street (Figure 1), training procedure, and training criterion as in Experiment 1. The generalization section presented eight streets (Figure 4a–4h) once at each of the four speeds in a random order, for a total of 32 trials. 

Simulator sickness was an issue in Experiment 1, with approximately 25% of the subjects unable to complete the experiment. Although a discussion of simulator sickness is beyond the scope of this article, it is generally accepted that a conflict between the visual and vestibular perception of acceleration is one major cause. In Experiment 1, the high speeds and sharp corners produced a large visual angular acceleration, but there was no corresponding vestibular simulation. To reduce this conflict, slower speeds were used in Experiment 2. Specifically, the 130- and 230-millisecond delay groups were presented with speeds of 16, 18, 20, 22, and 24 m/s, whereas the 430-millisecond delay group was presented with speeds of 12, 14, 16, 18, and 20 m/s. As with Experiment 1, the slowest speed was only presented during the training section. The new set of speeds, which were chosen based on Experiment 1, not only should reduce the incidence of simulator sickness but also provide a tighter measurement of subjects’ driving abilities (because the speeds were more tightly clustered). 

Analyses

All three measures from Experiment 1 were examined in Experiment 2. The street completion metric proved to be quite sensitive and the other two measures added no additional information or insights. For this reason, the latter two measures are not discussed. 

Results and Discussion

For all three groups, performance in the generalization section was better than performance in the baseline section (Figure 5), demonstrating that training with a delay on one street improves performance with the same delay on novel streets. There was no difference between the old and novel streets in the generalization section, suggesting that the novelty of a street does not have a great influence on generalization. 

Figure 5

 

Results of Experiment 2. The blue bars depict the percentage of streets that were successfully completed during the baseline section, and the red bars depict performance during the generalization section. Error bars represent SEM.

The street completion percentages were pooled across streets and three two-way ANOVAs were performed (one for each delay group), with speed and test type (baseline versus generalization) as within subjects factors. For each of the three groups, there was a significant main effect of speed [all F(3,4) > 4, all P values < .04]: faster speeds were harder. There was a significant main effect of test type for the larger two delays [all F(1,4) > 29, all P values < .005]. Performance accuracy was higher in the generalization section than in the baseline section. Although every subject in the 130-millisecond group showed an increase in performance, this increase did not reach significance [F(1,4) = 2.663, P > .17]. This is most likely because of a ceiling effect. 

As expected, the use of slower speeds in Experiment 2 resulted in fewer people developing simulator sickness than in Experiment 1. This confirms the role played by intersensory acceleration differences in simulator sickness, and suggests that delayed feedback does not seem to cause simulator sickness--at least in this experimental setup. 

Adding a slower speed during training allowed subjects to adapt to a 430-millisecond delay. All five subjects in Experiment 2 reached the end of the street several times at a speed of 18 m/s, whereas no subject in Experiment 1 could complete a street at this speed. Furthermore, two of the subjects in Experiment 2 completed training for the fastest speed (20 m/s). These results contrast strongly with those from Experiment 1 and provide evidence that the inability of subjects to adapt fully to a 430-millisecond delay in Experiment 1 was due to the difficulty of the first speed seen during training and the paucity of repetitions at that speed. 

General Discussion

We found that strict temporal contiguity between an action and its consequences is not necessary for rapid and accurate interaction with the world. Although the introduction of a temporal delay in a visuomotor task does impair behavior at first, a few minutes of the proper experience improves performance considerably. Cunningham et al (2001) found that training can improve performance in a simple, unfamiliar obstacle-avoidance task. The present study extends this finding to a complex, highly familiar task: driving. Despite the complexity of driving (which takes a considerable amount of time to learn), a few minutes of exposure to delayed feedback enabled the subjects to drive with the delay nearly as well as they could without the delay. This suggests that subjects are not re-learning how to drive (ie, treating the delay task as an entirely new skill to be learned, which would take more than a few minutes), but are instead altering the existing visuomotor transformations used in driving. 

Experiment 2 showed that once subjects adapted to delayed feedback on one street, they were able to drive with a delay on other streets. Thus, the improvement seen during training could not have been caused by subjects merely memorizing the training street or memorizing a particular sequence of arm and hand movements, but rather seems to reflect a fundamental change in the visuomotor system. That is, much or all of the visuomotor transformations used for driving seem to have been altered. 

These results, combined with those of Cunningham et al (2001), clearly indicate that the improvement found during training is the result of temporal visuomotor adaptation. In his classic book, Welch (1978) defines adaptation to perceptual rearrangements as “a semipermanent change of perception or perceptual motor coordination that serves to reduce or eliminate a registered discrepancy between or within sensory modalities or the errors in behavior induced by this discrepancy” (p. 8). Not only do the present results meet this definition, but the overall pattern of temporal adaptation is strikingly similar to spatial visuomotor adaptation: (a) the introduction of a visuomotor or intersensory discrepancy impairs performance; (b) a few minutes of exposure to the consequences of the discrepancy improves performance; (c) adaptation to the discrepancy produces a strong aftereffect; (d) adaptation to the discrepancy seems to result in a change in the perceived relationship between the two sensory modalities; and (e) adaptation affects an entire class of visuomotor transformations. 

The present results cannot be caused by some cognitive or behavioral strategy (eg, simply trying to turn early or anticipate the turns). There are several reasons for this. First, there is some evidence that humans might not be able to consciously make predictions (either spatially or temporally) precise enough about exactly when to turn for such strategies to be successful (Gottsdanker, 1952; Rouse, 1976; Runeson, 1975; Tulga & Sheridan, 1980; Waganaar & Sagaria, 1975; Wickens, 1984). Second, the conditions used by all of the previous research on practice with delayed feedback are just as appropriate for such strategies, yet none of the previous work found an improvement in performance. The primary difference between the present study and those previous is that the present study was designed to encourage adaptation (by forcing subjects to be exposed to the effects of the delay). Third, and perhaps most critically, in Experiment 1, subjects knew that there was no delay during the posttest and were able to directly experience the immediacy of the feedback during the posttest. Thus, if subjects had learned to drive with the delay merely by using some conscious strategy, the removal of the delay should have led the subjects to change their strategy (Bedford, 1993; Welch, 1978), and there would have been no aftereffect. Even with these reminders to change strategies, subjects were unable to complete the task for most of the posttest. 

As mentioned already, the improvement seen during training cannot be due to subjects re-learning how to drive. Could subjects have treated the time delay as a minor change in previously learned control dynamics (eg, treating the delay as a loose steering wheel)? In other words, did subjects simply learn some new control dynamics rather than re-learn how to drive? For example, Readinger, Chatziastros, Cunningham, and Bülthoff (2001) have shown that subjects can rather quickly learn to drive a virtual car whose steering wheel is left-right reversed, so that turning the steering wheel to the left causes the car to go to the right. Such an alternate explanation may explain the improvement in performance and the generalization to new streets, but ultimately suffers from many of the same problems as strategy-based explanations: no form of skill learning can simultaneously explain the aftereffect seen in Experiment 1 and the generalization seen in Experiment 2. The car seen in the posttest was one that subjects knew already, had recently driven (during the pretest), and could drive immediately the first time they saw it. Subjects knew the control dynamics of the nondelayed car, yet knowingly returning to this familiar car led to a drop in performance. Why would learning new control dynamics impair one’s ability to use a familiar skill? There are only two possibilities. First, one might suggest that overtraining on the new skill might impair performance during the posttest. Overtraining would not, however, allow the generalization found in Experiment 2 to occur because overtraining is highly specific to the exact motor sequence. Second, one might suggest that the rules learned for the new control dynamics persisted somehow, and this persistence impaired the usage of previously known skills. This, of course, begs the question as to what the source of the persistence is. It also leads us right back to Welch’s definition of adaptation as “a semipermanent change of perception or perceptual motor coordination…” In summary, the best way to explain our results is to posit some change in visuomotor coordination that affects an entire class of visuomotor transformations and persists even in situations where the change is no longer appropriate. In short, temporal visuomotor adaptation provides the best explanation for the full pattern of results. 

These results have strong implications for the use of high-fidelity vehicle and flight simulators. One of the concerns with training simulators is the tradeoff between the fidelity of the simulation and the resultant delay. Increasing the visual fidelity of the simulator increases the computational demand on the computer, which in turn increases the visuomotor delay. Although a greater realism is definitely desirable in a simulator, one might ask if the increased visuomotor delay reduces the transfer of training. By showing that the opposite transfer can occur (ie, that skills learned without a delay can be used when a delay is present), our results imply that training with a delay can improve performance without a delay. Of course, the aftereffect produced by temporal visuomotor adaptation demands that caution be exercised when using a simulator. Of critical importance is that rapidly switching from a simulator to a real vehicle might lead to disastrously poor performance in the real world. There should be some rest period between simulator use and real-world driving. A determination of how long the aftereffect lasts is necessary to determine how large this rest period should be, and would be vitally important when using simulators.