Learning novel mappings from optic flow to the control of action

Abstract

Many perceptual-motor skills can be captured in terms of mappings from information in sensory arrays to movements of the body, but little is known about how these mappings are acquired and updated. The present study demonstrates that people adapt to changes in the dynamics of a controlled system by learning novel mappings from information in optic flow to movement of the system. Two groups of subjects performed a simulated braking task, using a foot pedal to decelerate to a stop at a target. Brake dynamics were manipulated such that deceleration was proportional to pedal position for one group, and both pedal position and current speed for the other group. Both groups adapted to their respective brake dynamics by learning to use different combinations of optic flow field variables, revealing a form of adaptation that has not been previously reported, but likely plays a critical role in robust visuomotor control.

Introduction

Over the course of a lifetime, humans acquire numerous perceptual-motor skills, many of which involve a tight coupling between continuously available information in sensory arrays and continuously controlled movements of the body. People learn to steer bicycles, catch fly balls, drive automobiles, pilot aircraft, and so on. It is well established that behavior in these tasks can be characterized in terms of mappings from information in sensory flow fields to movements of the body (or an input device, as in the case of vehicle and aircraft control) (Warren, 1998; Warren & Fajen, 2004). These perceptual-motor mappings are often captured by mathematical functions (called laws of control) that map specific sensory variables onto movements. For example, mappings from optic flow field variables to movements have been proposed and tested for tasks such as steering (Wann & Swapp, 2000; Wann & Wilkie, 2004; Warren, Kay, Zosh, Duchon, & Sahuc, 2001; Wilkie & Wann, 2003), braking (Fajen, 2005a; Lee, 1976; Yilmaz & Warren, 1995), catching fly balls (McBeath, Shaffer, & Kaiser, 1995; McLeod, Reed, & Dienes, 2003, 2006), and intercepting moving targets on foot (Chardenon, Montagne, Buekers, & Laurent, 2002; Fajen & Warren, 2007). However, very little is known about how these mappings are acquired in the first place, and how they are updated with experience and changes in the body, environment, or task constraints. 

The ability to learn novel sensorimotor mappings would seem to play a fundamental role in the acquisition of new perceptual-motor skills. Indeed, improvement in performance on catching, hitting, and collision avoidance tasks in both infants and adults has been linked to changes in the optical information upon which people rely at various stages of practice (Fajen, 2008; Jacobs & Michaels, 2006; Smith, Flach, Dittman, & Stanard, 2001; van Hof, van der Kamp, & Savelsbergh, 2006), suggesting that learning a new perceptual-motor skill is (in part) a matter of discovering better sources of information within complex, sensory arrays. In Smith et al. (2001), who provided inspiration for the present study, subjects had to time the release of a pendulum to hit an approaching ball in a desktop virtual environment. Initially, subjects released the pendulum when the ball's optical expansion rate reached a critical value. This strategy resulted in a systematic bias to respond too early for large balls and balls that approached slowly. With practice, performance gradually improved as subjects learned to rely on a linear combination of optical angle and expansion rate. Interestingly, the weights associated with optical angle and expansion rate were tuned to the range of ball sizes and approach speeds that were encountered, revealing a degree of flexibility that cannot be explained by traditional “single-optical-invariant” models. 

The ability to update sensorimotor mappings with experience may also underlie adaptation to changes in the dynamics of the controlled system (i.e., the body or vehicle) or the environment. It is already well established that people can adapt to complex changes in dynamics in certain classes of visuomotor tasks. In studies of visually guided reaching, the dynamics of the arm have been modified by imposing velocity-dependent forces on the limb during execution of the movement (Lackner & Dizio, 1994; Shadmehr & Mussa-Ivaldi, 1994). Reaches were initially distorted, but gradually converged onto trajectories that resemble those produced in the absence of external perturbations. People also modify their control strategy to adapt to different kinds of signal-dependent noise, making smaller movements when noise is proportional to the magnitude of the control signal and larger movements when noise is inversely proportional (Chhabra & Jacobs, 2006). 

In these and most other previous studies on adaptation to changes in dynamics, the visual component of the visuomotor task was rather trivial. Subjects were simply required to perceive the location of a stationary target in one or two dimensions. Despite the enduring interest in adaptation to changes in dynamics, none of the previous studies on this topic involved tasks with a richer visual component, such as those that are characterized by a tight coupling between information in optic flow and the control of self-motion. In such tasks, factors such as fatigue, load, surface traction, changes in plant and controller dynamics, as well as disturbances from the environment can dramatically affect the movements that result from particular motor commands to move the body or adjust an input device. If motor commands are a function of information in optic flow, then changes in the dynamics can alter the effectiveness of different optic flow field variables. An acceptable level of performance could be regained by updating the mapping from optic flow field variables to movements. 

This hypothesis was investigated in the present study by instructing subjects to perform a simulated braking task, using a foot pedal to decelerate to a stop as close as possible to a target, which consisted of a row of three stop signs ( Figure 1). To investigate adaptation, the dynamics of the brake pedal were programmed in one of two ways. In one condition, the simulated rate of deceleration was proportional to the position of the foot pedal ( Figure 2A). For example, assuming a gain of 10 m/s ², gradually displacing the pedal from 0% (neutral position) to 50% (as in the top panel of Figure 2A) resulted in an increase in deceleration from 0 to 5 m/s ². Whenever pedal position was constant (e.g., after ∼3 s in Figure 2A), deceleration was also constant and speed decreases linearly. This will be referred to as the Normal Brake dynamics because it describes the dynamics used in several previous studies of visually guided braking (Fajen, 2005a, 2005b; Rock & Harris, 2006; Yilmaz & Warren, 1995). 

In the other condition, which will be referred to as the Velocity Brake, deceleration was proportional to both pedal position and current speed ( Figure 2B). The initial spike in the deceleration plot in Figure 2B is due to the fact that deceleration was proportional to speed, and speed is greatest up until the point that braking is initiated. Unlike the Normal Brake, the same Velocity Brake input profile resulted in a different deceleration profile depending on initial speed, with a higher peak when initial speed was faster. Whenever brake position was constant, deceleration gradually decreased because speed decreased. 

Figure 1

 

Sample screen shot of simulation used in Experiment 1.

Figure 2

 

Comparison of Normal and Velocity brake dynamics. In the Normal Brake condition (A), deceleration was proportional to pedal position. In the Velocity Brake condition (B), deceleration was proportional to both pedal position and current speed ( v). The second and third rows show the deceleration and speed profiles, respectively, resulting from the brake position profile shown in the first row for two initial speeds.

The goal of the present study was to determine if people could adapt to these very different brake dynamics by learning novel mappings from information in optic flow to movement of the brake pedal. Consider the situation in which an actor is moving at a speed v toward some object at a distance z. At that moment, the constant pedal position p _n* on a Normal Brake that will bring the actor to a stop exactly at the object is:  

$p_n^{*}=G_n^{-1}\times \left(\frac{v^2}{2\times z}\right)$

(1)

where G _n is the gain of the Normal Brake and p _n* is given in units of maximum brake displacement. Thus, if at any point p _n = p _n*, then holding the pedal in a fixed position will bring the actor to a stop exactly at the target. Any difference between p _n and p _n* necessitates a change in pedal position to avoid colliding with or stopping before the target. 

Equation 1 can be rewritten in terms of optical variables as follows. v/ z is the inverse of the amount of time until contact with the object assuming constant velocity (i.e., time-to-contact), which is optically specified by

$\dot{\theta }$

/ θ, where θ and

$\dot{\theta }$

are the optical angle and expansion rate of the approached object (Lee, 1976). Speed (v) is also optically specified. When an observer translates over a textured ground surface at a fixed eye height, the optical velocity of each point on the ground surface depends on the point's azimuth and declination. The optical velocity of each point is also proportional to the ratio of observer speed (v) to eye height (e). Thus, v/e is a global multiplier that affects the optical motion of all points on the ground surface in the same way. This ratio (v/e) is referred to as global optic flow rate (GOFR) (Larish & Flach, 1990; Warren, 1982). As long as eye height is fixed (which it typically is for tasks that involve braking), v is proportional to GOFR. Thus, Equation 1 can be expressed in terms of optical variables by substituting

$\dot{\theta }$

/θ for v/z and GOFR for v, yielding: 

$p_n^{*}=k_n\times \left(\frac{\mathrm{G}\mathrm{O}\mathrm{F}\mathrm{R}\times \dot{\theta }}{\theta }\right)$

(2)

where the constant k_n is equal to [e/(2 × G_n)]. Recent studies using Normal Brake dynamics (e.g., Fajen, 2005a, 2008) provide evidence that deceleration is modulated on the basis of GOFR ×

$\dot{\theta }$

/θ. 

Because the Velocity Brake dynamics differ from those of the Normal Brake, the optical information that tells actors how to adjust the foot pedal also differs. For the Velocity Brake, the constant position p _v* that will bring the actor to a stop exactly at the object is:  

$p_v^{*}=G_v^{-1}\times \left(\frac{v}{z}\right)$

(3)

which can be translated into optical variables to yield:  

$p_v^{*}=k_v\times \left(\frac{\dot{\theta }}{\theta }\right)$

(4)

where the constant k _v is equal to (1/ G _v). Thus, Equations 2 and 4 describe possible mappings from information in optic flow to movements of the foot pedal for the Normal Brake and Velocity Brake, respectively. Note that these mappings differ in that GOFR is needed to control the Normal Brake, but not the Velocity Brake. 

The first of the two experiments in this study was designed confirm that subjects can, in fact, adapt to the two different brake dynamics. Because there is no previous work on adaptation to complex changes in system dynamics in visual control tasks involving optic flow (e.g., braking or steering), it cannot automatically be assumed that subjects will adapt to the two different brake dynamics, especially within the relatively short duration of an experimental session. They may partially adapt, or not adapt at all, in which case additional mappings other than those in Equations 2 and 4 would have to be considered. 

In Experiment 2, additional conditions were tested that, when compared with Experiment 1, provide a test of whether subjects adapt to the Normal Brake and Velocity Brake by learning the different mappings described in Equations 2 and 4. 

Experiment 1

Two groups of subjects completed 200 trials (5 initial speeds × 40 repetitions). In Group A, 90% of the trials were Normal Brake trials. The remaining 10% of the trials were Velocity Brake trials, which were randomly interspersed among Normal Brake trials. Because almost all of the trials were Normal Brake trials, it was assumed that subjects in Group A would adopt a mapping that would work effectively for the Normal Brake. Subjects in Group B received the opposite distribution of trials. Ninety percent of the trials were Velocity Brake trials and 10% of the trails were Normal Brake trials. Thus, subjects in Group B should adopt a mapping that works effectively for the Velocity Brake. 

Method

Participants

Twenty students participated in Experiment 1. All participants had normal or corrected-to-normal vision and no visual or motor impairments. Ten participants were randomly assigned to each group. Data from two participants were excluded because they did not follow instructions. 

Stimuli

Displays were generated using OpenGL running on a Dell Precision 530 Workstation and were rear-projected by a Barco Cine 8 CRT projector onto a large (1.8 m × 1.2 m) screen at a frame rate of 60 Hz. Trials were initiated by completely releasing the foot pedal and pressing a trigger button. The scene appeared and simulated linear motion at one of five initial speeds (8, 10, 12, 14, and 16 m/s) began immediately. The target consisted of a row of three red and white octagonal stop signs with a radius of 0.375 m, located at an initial distance that varied randomly between 50 and 60 m. 

Braking was controlled using an ECCI Trackstar 6000 (Minneapolis, MN) foot pedal system. Participants increased deceleration by pushing on the leftmost of two foot pedals that were spring loaded to provide resistance in proportion to displacement from the neutral position. Pedal position ( p) was sampled every frame and used to update the simulated rate of deceleration ( d) on the subsequent frame according to the following equations: d = G _n × p on Normal Brake trials, and d = G _v × v × p on Velocity Brake trials. G _n and G _v, which refer to the gains for the Normal and Velocity brakes, were set to 10 m/s ² and 2 s ⁻¹, respectively. 

Procedure

Prior to the experiment, informed consent was obtained. Participants completed 50 practice trials to familiarize themselves with the task before starting the actual experiment. All 50 practice trials were Normal Brake trials in Group A and Velocity Brake trials in Group B. The entire experiment lasted approximately 1 hour. 

Data analyses

Two measures of behavior were analyzed: mean initial brake adjustment magnitude and final position error. Mean initial brake adjustment magnitude is a measure of how hard subjects hit the brakes on the initial adjustment, and is scaled so that values greater than and less than 1.0 correspond to overshoots and undershoots, respectively. The onset of braking was determined by looking for the first window lasting 200 ms (12 frames) over which pedal position ( p) changed by at least 5% of maximum pedal displacement. The end of the initial brake adjustment was found by searching for the first 200 ms window after brake onset during which p was either held constant (±1% of maximum displacement) or decreased. The magnitude of each initial brake adjustment was calculated by first taking the change in p from the beginning to the end of the adjustment. The change in p was then scaled to the change in p that was required to null the difference between p and p _n* (see Equation 1) on Normal Brake trials or p _v* (see Equation 3) on Velocity Brake trials, resulting in a number greater than 1.0 for overshoots and less than 1.0 for undershoots. The second measure, mean final position, was derived from subjects' location relative to the target at the end of the trial. Final position error was positive when subjects stopped before reaching the target and negative when subjects stopped after passing through the target. 

Separate two-way mixed design ANOVAs were performed on initial brake adjustment magnitude and final position. The assumption of sphericity was tested using Mauchly's test, and violations of the sphericity assumption were corrected using the Greenhouse–Geisser correction. 

Results and discussion

Figure 3C shows mean initial brake adjustment magnitude as a function of initial speed on Velocity Brake trials for both groups. Initial speed affected subjects in Groups A and B differently, resulting in a significant Group × Initial speed interaction ( F _{4, 64} = 6.20, p < .001). Recall that the strength of the Velocity Brake was proportional to speed. Not surprisingly, subjects in Group A, who were adapted to the Normal Brake exhibited a large bias on Velocity Brake catch trials, resulting in a significant simple main effect of initial speed ( F _{4, 28} = 7.21, p < .001). Specifically, Group A tended to overshoot (i.e., brake harder than necessary) when initial speed was fast, but not when initial speed was slow. By comparison, subjects in Group B, who were adapted to the Velocity Brake exhibited no bias on Velocity Brake trials ( F _{4, 36} = 1.00, p = .42). These subjects applied just slightly more than the needed amount of deceleration, and were unaffected by initial speed. 

Figure 3

 

Initial speed × Group interactions for Experiment 1. Normal Brake trials (blue lines) are in panels A and B and Velocity Brake trials (red lines) are in panels C and D. Dotted lines indicate catch trials.

A similar pattern was found for final stopping distance ( Figure 3D), although the interaction did not reach significance ( F _{4, 64} = 1.74, p = .20). The absence of a significant interaction is likely due to the fact that final stopping distance is a less sensitive measure than initial brake adjustment magnitude. This is because trials lasted several seconds, and so subjects had ample time to use continuously available visual information to correct for overshoots and undershoots. Interestingly, there was a significant main effect of Group on mean final stopping distance ( F _{1, 16} = 4.76, p = .04), with subjects in Group A stopping later than those in Group B. This is likely due to the fact that the Velocity Brake is much weaker than the Normal brake at the very slow speeds encountered near the end of the approach. Hence, subjects who were adapted to the Normal Brake (Group A) applied insufficient deceleration near the very end of the approach, resulting in later stops. By comparison, subjects in Group B consistently stopped shortly before reaching the target, providing additional evidence that they were able to adapt to the Velocity Brake. 

As expected, the effect of initial speed was greater for subjects in Group B than Group A on Normal Brake trials, resulting in significant interactions on both measures ( F _{4, 64} = 2.97, p = .047 for initial brake adjustment magnitude; F _{4, 64} = 4.96, p = .016 for final stopping distance). When initial speed was fast, the Normal Brake was weaker than the Velocity Brake, which explains why subjects in Group B tended to undershoot ( Figure 3A) and stop later ( Figure 3B) on Normal Brake catch trials. Note that the direction of bias is opposite that which was found for subjects in Group A on Velocity Brake catch trials (compare dotted lines in Figures 3A and 3B with those in Figures 3C and 3D). 

One finding that was not predicted is the significant simple main effect of initial speed for subjects in Group A in Figure 3A ( F _{4, 28} = 15.62, p < .001). This is a robust effect that has been observed in other braking studies (e.g., Fajen, 2008) but is not predicted by the GOFR ×

$\dot{\theta }$

/θ hypothesis. Why might subjects brake harder when initial speed is slow? There is clearly an overall bias to brake harder than necessary (i.e., mean brake adjustment magnitude is greater than 1.0), most likely due to the fact that the consequences of braking insufficiently are more severe. When initial speed is fast, it is more difficult to overshoot because the deceleration needed to avoid a collision is greater than when initial speed is slow. So the main effect of initial speed could simply be due to the fact that there is more room to overshoot when initial speed is slow. 

To summarize, both groups performed well on non-catch trials (solid lines in Figure 3) and exhibited biases on catch trials (dotted lines) that were consistent with being adapted to the brake type experienced on the majority of trials (i.e., Normal Brake for Group A and Velocity Brake for Group B). These results confirm that subjects can, in fact, adapt to the two different brake types. Experiment 1 also shows the degree to which behavior depends on the dynamics of the brake to which one is adapted. This is important because the most widely accepted theory of visually guided braking (i.e., David Lee's tau-dot model) ignores the dynamics of the brake. Lee (1976) proposed that braking is controlled by

$\dot{\tau }$

, the first temporal derivative of τ = θ/

$\dot{\theta }$

.

$\dot{\tau }$

does not specify p* for either brake type but rather the sufficiency of the actor's current rate of deceleration. When

$\dot{\tau }$

< −0.5, current deceleration is insufficient and the actor needs to increase deceleration to avoid a collision; when

$\dot{\tau }$

> −0.5, current deceleration is excessive and the actor may decrease deceleration to avoid stopping too soon. Thus, the

$\dot{\tau }$

model predicts that actors modulate deceleration around the critical value of

$\dot{\tau }$

equal to −0.5, regardless of the brake dynamics. 

The results of Experiment 1 do not support this prediction. When subjects were adapted to the Velocity Brake, they consistently applied more deceleration than was needed to keep

$\dot{\tau }$

at −0.5. This is illustrated in Figures 4A and 4B, which shows data from a representative Velocity Brake trial for a subject in Group B. Figure 4A shows the actual rate of deceleration (solid line) and the rate of deceleration that corresponds to

$\dot{\tau }$

= −0.5 (dotted line). Note that actual deceleration after the end of the initial brake adjustment (∼4 s) is well in excess of that which was needed to make

$\dot{\tau }$

equal to −0.5. This was typical across trials and subjects. When subjects were adapted to the Velocity Brake (Group B), the overall mean initial change in deceleration on Velocity Brake trials was 1.88× that which was necessary to make

$\dot{\tau }$

equal to −0.5. From the perspective of

$\dot{\tau }$

, the tendency to overshoot looks like a systematic error. In fact, the overshoot bias is adaptive given the dynamics of the Velocity Brake, which becomes progressively weaker as speed decreases throughout the approach. To keep

$\dot{\tau }$

close to −0.5, one would have to continually increase pedal position. Rather than keeping

$\dot{\tau }$

at −0.5, subjects discovered a better solution for controlling deceleration using the Velocity Brake. Figure 4B shows the actual pedal position, the pedal position specified by

$\dot{\theta }$

/ θ ( Equation 4), and the pedal position needed to make

$\dot{\tau }$

equal to −0.5 for the same trial. Note that adjusting the pedal according to

$\dot{\theta }$

/ θ allows the subject to automatically apply the deceleration needed to stop at the target even as the strength of the brake decreased. Thus, braking behavior in Group B was clearly more consistent with the use of

$\dot{\theta }$

/ θ than

$\dot{\tau }$

. 

Figure 4

 

Representative Velocity Brake trial from a subject in Group B of Experiment 1.

Experiment 2 provides a stronger test of the hypothesis that subjects would adapt to the Normal Brake by relying on GOFR ×

$\dot{\theta }$

/ θ and to the Velocity Brake by relying on

$\dot{\theta }$

/ θ. This hypothesis was tested by replicating Experiment 1 but removing the textured ground plane, which was the source of GOFR. The two experiments were otherwise identical. 

Experiment 2

Three predictions were tested in Experiment 2. The first prediction, which follows from the hypothesis that subjects adapt to the Normal Brake by relying on GOFR ×

$\dot{\theta }$

/ θ, is that removing the ground plane should impair performance on Normal Brake trials, resulting in a stronger effect of initial speed. The second prediction concerns the performance of subjects in Group A on Velocity Brake catch trials. In the absence of ground plane texture, Group A subjects cannot adapt to the Normal Brake by relying on GOFR ×

$\dot{\theta }$

/ θ and must rely on another optical variable that allows them to perform as well as possible given the available information. One possibility is that they will rely on

$\dot{\theta }$

/ θ; that is, the same optical variable that specifies pedal position for the Velocity Brake. This leads to the prediction that removing the ground plane should actually improve performance on Velocity Brake catch trials for subjects in Group A, in the sense that they should not exhibit the speed bias that was found for their counterparts in Experiment 1. Finally, if subjects in Group B adapt to the Velocity Brake by relying on

$\dot{\theta }$

/ θ, then removing the ground plane should not affect performance on either trial type. Thus, the third prediction is that subjects in Group B of Experiment 2 should perform the same as those in Group B of Experiment 1 on both Normal Brake and Velocity Brake trials. That is, subjects should exhibit a strong speed effect on Normal Brake catch trials, but not on Velocity Brake trials. 

Methods

Participants

Twenty different students participated in Experiment 2. All participants had normal or corrected-to-normal vision and no visual or motor impairments. Ten participants were randomly assigned to each group. 

Stimuli, procedures, and data analyses

Stimuli, procedures, and data analyses were identical to those used in Experiment 1 with the exception that the gray cement-textured ground plane and the white posts below the stop signs were absent in Experiment 2. 

Results and discussion

Prediction #1

The first prediction is that removing the ground plane should impair performance on Normal Brake trials for subjects in Group A. Figures 5A and 5B compare data from Experiments 1 and 2 for subjects in Group A on Normal Brake trials. Even though there was already a significant main effect of initial speed on initial brake adjustment magnitude in Experiment 1, the Experiment × Initial speed interaction was still significant ( F _{4, 64} = 3.42, p = .042). Subjects in both experiments were affected by initial speed, but the effect was greater in Experiment 2. The interaction was also significant for final position error ( F _4,64 = 11.20, p < .001). As expected, subjects who were adapted to the Normal Brake were impaired by removing GOFR. This supports the hypothesis that, when ground plane texture is present to provide GOFR (as it normally is), subjects adapt to the Normal Brake by relying on GOFR ×

$\dot{\theta }$

/ θ. The result also provides further evidence against the

$\dot{\tau }$

hypothesis.

$\dot{\tau }$

is defined by the optical angle (and its temporal derivatives) of the approached object and is independent of the optic flow corresponding to the ground plane. Hence, the fact that subjects who were adapted to the Normal Brake were affected by removing the ground plane cannot be explained by the

$\dot{\tau }$

hypothesis (see also Fajen, 2005a; Rock & Harris, 2006). 

Figure 5

 

Initial speed × Experiment interactions for Group A. Normal Brake trials (blue lines) are in panels A and B and Velocity Brake trials (red lines) are in panels C and D. Dotted lines indicate catch trials.

Prediction #2

The second prediction is that removing the ground plane should improve performance on Velocity Brake trials for subjects in Group A; that is, Group A should not exhibit the speed bias that was found in Experiment 1. Indeed, the interaction of Experiment × Initial speed on initial brake adjustment magnitude ( Figure 5C) was significant ( F _{4, 64} = 2.52, p < .049). Whereas subjects in Experiment 1 were affected by initial speed, those in Experiment 2 were not ( F _{4, 36} < 1). A similar pattern was found for mean final position ( Figure 5D), although the interaction was not significant ( F _{4, 64} = 1.43, p = .24) due to the weak initial speed effect in Experiment 1 (see above). Thus, removing GOFR forced subjects to adapt to the Normal Brake by relying on

$\dot{\theta }$

/ θ, resulting in an exaggerated effect of initial speed on Normal Brake trials and a weakened effect of initial speed on Velocity Brake catch trials. 

Prediction #3

The third prediction is that subjects in Group B of Experiment 2 should behave similarly to those in Group B of Experiment 1. That is, they should exhibit a strong speed effect on Normal Brake catch trials, but not on Velocity Brake trials, as in Experiment 1. As expected, there was a significant main effect of initial speed ( F _{4, 72} = 26.43, p < .001) and a non-significant interaction ( F _{4, 72} = 1.57, p = .192; see Figure 6A). Similarly, the analysis of mean final position ( Figure 6B) revealed that subjects in both experiments stopped later when initial speed was faster ( F _{4, 72} = 30.85, p < .001). 

Figure 6

 

Initial speed × Experiment interactions for Group B. Normal Brake trials (blue lines) are in panels A and B and Velocity Brake trials (red lines) are in panels C and D. Dotted lines indicate catch trials.

Although subjects in Group B exhibited in the same speed effect on Normal Brake catch trials in Experiments 1 and 2, there was an unexpected main effect of experiment ( F _{1, 18} = 6.76, p < .05). As shown in Figure 6A, subjects in Experiment 1 tended to brake harder than subjects in Experiment 2. Despite making weaker brake adjustments, subjects in Experiment 2 still stopped in the same place as subjects in Experiment 1 on all but the fastest initial speeds ( Figure 6B). (The Experiment × Initial speed interaction for final stopping position was marginally significant, F _{4, 72} = 2.97, p = .066). One possible explanation for this effect is that the ground plane may have enhanced the salience of the local optical expansion of the target region. It is generally assumed that θ and

$\dot{\theta }$

correspond to the angle and expansion rate of the stop signs. Because the stop signs were small (0.375 m radius), the values of θ and

$\dot{\theta }$

at the distances at which braking was typically initiated were low and may not have been salient. However, when the ground plane was present (as in Experiment 1), θ and

$\dot{\theta }$

could also correspond to the angle and expansion rate of the point on the ground directly beneath the stop signs. Because eye height (1.1 m) was greater than sign radius, both θ and

$\dot{\theta }$

corresponding to the point on the ground were always greater (and therefore more salient at larger distances) than θ and

$\dot{\theta }$

for the stop signs. This could explain why subjects made stronger brake adjustments when the ground plane was present. It is important to emphasize, however, that even if subjects in Group B used θ and

$\dot{\theta }$

corresponding to the ground plane, they did not use GOFR ×

$\dot{\theta }$

/ θ. 

On Velocity Brake trials, Group B subjects performed the task successfully both with and without GOFR across the range of initial speeds (see Figures 6C and 6D), resulting in no significant effects of initial speed ( F _{4, 72} = 1.26, p = .30 for magnitude; F _{4, 72} < 1 for final position error) and no significant interactions ( F _{4, 72} < 1 for magnitude; F _{4, 72} < 1 for final position error) on either measure. Subjects applied roughly the right amount of deceleration on the initial adjustment, and consistently stopped shortly before reaching the target. Again, subjects in Experiment 1 tended to brake harder than subjects in Experiment 2, although the effect was only marginally significant ( F _{1, 18} = 3.94, p = .06). Thus, whereas subjects in Group A were affected by removing GOFR, those in Group B were not, providing compelling evidence that subjects adapted to the Velocity Brake by relying on

$\dot{\theta }$

/ θ. 

Alternative explanations

The two experiments in this study were designed to test the predictions that subjects would adapt to the Normal Brake by learning to use GOFR ×

$\dot{\theta }$

/ θ and to the Velocity Brake by learning to use

$\dot{\theta }$

/ θ. In this section, alternative optical variables will be considered. 

Lee's (1976) tau-dot strategy

The results of the present study provide little evidence that subjects relied on

$\dot{\tau }$

, regardless of which brake type was used. First, because

$\dot{\tau }$

is defined independently of optic flow from the ground plane, subjects should be able to control deceleration regardless of the presence or absence of the ground plane if they rely on

$\dot{\tau }$

. So the fact that performance in Group A degraded when the ground plane was removed suggests that subjects used something other than

$\dot{\tau }$

(such as GOFR ×

$\dot{\theta }$

/ θ) to control the Normal Brake. A possible counterargument is that performance in Group A degraded when the ground plane was removed not because GOFR was unavailable, but because the ground plane enhanced the saliency of local optical target expansion and hence made it easier for subjects to detect

$\dot{\tau }$

. However, the role of GOFR in controlling deceleration using Normal Brake dynamics has been demonstrated in other ways that are more difficult to reconcile with the

$\dot{\tau }$

hypothesis. For example, because GOFR is inversely proportional to eye height, doubling or halving eye height changes GOFR by a factor of 0.5 or 2.0, respectively. As one would expect if subjects relied on GOFR ×

$\dot{\theta }$

/ θ, Fajen (2005a) found that doubling eye height results in insufficient braking and halving eye height results in excessive braking. Second, the conclusion that subjects did not use

$\dot{\tau }$

to control the Velocity Brake is supported by the finding that they consistently applied more deceleration than was needed to keep

$\dot{\tau }$

at −0.5. This looks like an error from the perspective of

$\dot{\tau }$

but is in fact an adaptive strategy for controlling deceleration using the Velocity Brake. Finally, the flexibility that subjects exhibited in the present study raises questions about any model that assumes that people rely on a single optical variable regardless of the dynamics of the controlled system. 

Strategies based on a constant target size

The optical variables in Equations 2 and 4 invariantly specify p* across variations in the size of the approached object. Because target size was fixed in both experiments, the possibility that subjects used an optical variable that was not size-invariant must be considered. In their paper on braking, Yilmaz and Warren (1995) proposed two such variables: p_n* = k_n × [τ² × tan(θ/2)]⁻¹ and p_n* = k_n × GOFR² × tan(θ/2). (Note that both variables specify p* for the Normal Brake, as the brake type used in Yilmaz and Warren was identical to the Normal Brake in the present study.) If subjects relied on [τ² × tan(θ/2)]⁻¹ to control the Normal Brake, then removing the ground plane should not affect performance. Therefore, the ground plane effect that was observed across Experiments 1 and 2 allows us to rule out [τ² × tan(θ/2)]⁻¹. The second variable, GOFR² × tan(θ/2), depends on optic flow from the ground plane, and hence cannot be ruled out on the basis of the present study. However, Yilmaz and Warren found that braking behavior was similar regardless of whether target size was fixed or variable. More recently, Fajen (2008) manipulated target size as an independent variable, and found no evidence that size affects braking behavior. Hence, GOFR ×

$\dot{\theta }$

/θ provides the best explanation for the ground plane and target size effects that have been reported. 

Conclusions

The findings demonstrate that people are capable of adapting to changes in the dynamics of a controlled system, and that the ability to learn novel mappings from information in optic flow to movement underlies such adaptation. Thus, sensorimotor mappings that are used to guide movement are not fixed but rather easily molded based on experience. Such learning is analogous to the form of learning demonstrated by Fajen and Devaney (2006) and Smith et al. (2001) in that combinations of optical variables that correspond to stable, successful performance are discovered through experience. There are also important differences between these studies. First, in the two earlier studies, the effectiveness of different optical variables was manipulated by varying the task constraints and the range of conditions. In the present study, the effectiveness of different optical variables depends on the dynamics of the controlled system. This allows us to show that the ability to learn new combinations of optical variables also underlies adaptation to changes in such dynamics. Second, the present study used a continuously controlled visually guided action rather than a ballistic task. This is important because learning in a ballistic task must be based on feedback about the outcome of the trial. By comparison, learning in a continuously controlled task such as visually guided braking is likely to be based on information in the optical consequences of ongoing adjustments (Fajen, 2007). 

Implications for research on visually guided action

The conclusion that people can learn new sensorimotor mappings has important implications for research on perceptually guided actions such as steering, braking, and interception, which is generally aimed at identifying the sensory variable or the law of control. An oftentimes implicit assumption has been that each task has associated with it a single optical variable and single law of control, and that the goal of research is to discover the relevant variable and control law for each task. Lee's

$\dot{\tau }$

strategy is one example of a “single-optical-invariant” model, but such models exist for other tasks as well, including steering, catching fly balls, and intercepting moving targets on foot. Despite the appeal of the single-optical-invariant assumption, there is mounting evidence that the visual system is capable of flexibly tuning to different sources of information, providing greater robustness across variations in the actor and environment (Warren, 2007). The single-optical-invariant assumption overlooks the fact that the usefulness of any given variable depends on multiple factors, one of which (i.e., the dynamics of the controlled system) was investigated in the present study. This means that any claims about the role of a particular variable must be qualified by the conditions under which that particular variable is informative. It also means that more empirical and theoretical research is needed to (1) identify the factors that influence which variables are detected and how they are used and (2) understand how novel mappings are learned based on experience interacting with the world. 

Reinforcement learning (Kaelbling, Littman, & Moore, 1996; Sutton & Barto, 1998) provides a potentially useful framework for building models of the kind of learning demonstrated in this study. In a reinforcement learning model, the agent learns a policy (i.e., a mapping from states to actions) that maximizes performance in terms of some reward function (e.g., stopping within a target region). Learning is incremental and proceeds by updating the values of previously visited states based on subsequent rewards. As values are updated, the agent learns to move into states from which it can accumulate reward. One appealing aspect of reinforcement learning is that the agent learns based on experience interacting with the world, without a teacher as in supervised learning. 

Sims and Fajen (2007) developed a reinforcement learning model of braking that exhibits behavior similar to that of human subjects in earlier experiments. Although Normal Brake dynamics were used in their model, there is no reason why the agent could not learn a policy for the Velocity Brake. In the current version of the model, the dimensions of the state space are speed and distance, and the agent is given these parameters without having to rely on information in optic flow. To fully capture the kind of learning that was demonstrated in the present study, it would be necessary to build a more realistic perceptual component that learns mappings from optic flow to movement. In principle, such a model could learn a different mapping depending on the brake dynamics. 

Implications for perceptual-motor skill acquisition

It is widely recognized that adaptation to changes in the dynamics of the body and environment plays a crucial role in the acquisition and execution of skills such as reaching (Lackner & Dizio, 1994; Shadmehr & Mussa-Ivaldi, 1994; Shadmehr & Wise, 2005), throwing (Bruggeman, Pick, & Rieser, 2005), and walking (Bruggeman, Zosh, & Warren, 2007; Choi & Bastian, 2007; Gordon & Ferris, 2007). Factors such as growth, fatigue, and tool use can significantly alter the limb movements that result from a particular pattern of muscle forces. The ability to adapt to such changes, which has been thoroughly investigated by imposing forces on the body during execution of the task, provides the means by which flexible, robust control is achieved. 

The form of adaptation demonstrated in the present study may play an equally important role in the acquisition of perceptual-motor skills involving the use of devices that augment human locomotor capabilities (e.g., bicycles, automobiles, airplanes). Because many of these devices allow people to move in novel ways through unfamiliar environments, sensory information may be become available that is not available during normal legged locomotion. Moreover, these devices have dynamics that differ from those of the human body and are controlled using apparatus (e.g., handlebars, steering wheels, foot pedals, flight sticks) that map onto movements in somewhat arbitrary ways. Thus, the use of these devices is accompanied by changes in the available information and the dynamics of the controlled system. The ability to learn novel mappings from sensory flow fields to movement is needed to acquire skill in controlling such systems. In other words, the same form of learning exhibited by subjects in the present study is also at work whenever people learn to ride bicycles and steer vehicles. 

Acknowledgments

This research was supported by grants from the National Science Foundation (BCS 0236734 and BCS 0545141). 

Commercial relationships: none. 

Corresponding author: Brett R. Fajen. 

Email: [e-mail]. 

Address: Department of Cognitive Science, Rensselaer Polytechnic Institute, Carnegie Building 308, 110 8th Street, Troy, NY 12180-3590.