Flexibility in intercepting moving objects

Abstract

When hitting moving targets, the hand does not always move to the point of interception in the same manner as it would if the target were not moving. This could be because the point at which the target will be intercepted is initially misjudged, or even not judged at all, but it could also be because a different path is optimal for intercepting a moving target. Here we examine the extent to which performance is degraded if people have to follow a different path than their preferred one. Forcing people to make small adjustments to their path by placing obstacles near the path hardly influenced their performance. When the orientation of elongated targets was manipulated, people adjusted their paths, but not quite enough to avoid intercepting the targets at a sub-optimal angle, probably because following a more curved path would have reduced the spatial accuracy and taken more time. When the task was to hit targets in certain directions, people had to sometimes follow much more curved paths. This gave rise to larger errors and longer movement times. An asymmetry in performance between hitting moving targets further in the direction in which they were moving and hitting them back from where they came is consistent with the different consequences of timing errors for the two directions of target motion. We conclude that the path that people take to intercept moving targets depends on the precise constraints under the prevailing conditions rather than being a consequence of judgment errors or of limitations in the way in which movements can be controlled.

Introduction

There are many ways in which we could move to perform a specific task. When we move in a certain manner, is this by and large the best choice for performing the task successfully? This is only a meaningful question if moving in different ways clearly influences performance. If it does, people should be reluctant to change their trajectory and perform worse when they do. Since we are currently unable to make more specific predictions, we will examine the influence of several manipulations on various aspects of task performance. We will manipulate the path that the hand takes to intercept moving targets and examine whether performance becomes slightly but consistently worse whenever people are forced to move differently than they would without such manipulation. 

It may seem evident that issues such as energy expenditure and the biomechanics of the arm must be considered when selecting a trajectory. Since movements of the hand are driven by rotations at the elbow, wrist, and shoulder, it is not surprising that the path taken between two points depends on the posture of the arm (Boessenkool, Nijhof, & Erkelens, 1998). One can expect paths that minimize joint rotation (Micci-Barreca & Guenther, 2001) or torque change (Nakano et al., 1999). Optimizing task performance could be consistent with such choices of trajectory because in many cases the most comfortable and efficient movements are likely to be the ones that result in the best performance. 

Arriving at the target from a certain direction could also be beneficial under certain conditions for mechanical reasons (Brenner & Smeets, 1995). However, the chosen paths are certainly not only determined by biomechanical factors (Osu, Uno, Koike, & Kawato, 1997), as is evident from the fact that perceptual errors influence the chosen path (Brenner, Smeets, & Remijnse-Tamerius, 2002; Smeets & Brenner, 2004; Wolpert, Ghahramani, & Jordan, 1994). The relevance of perceptual feedback is most directly demonstrated by the finding that when visual feedback about the hand's path is deformed, subjects make curved movements to keep the visual feedback straight (Flanagan & Rao, 1995). Thus, the trajectory of the hand is clearly not arbitrary. 

When moving toward moving targets timing is also an important issue. In order to predict the point of interception one must consider that where one should reach the target depends on when one reaches it, and vice versa. It has been proposed that people may avoid having to predict the point of interception by relying on continuous visual control to bring the hand to the target (e.g., Montagne, Laurent, Durey, & Bootsma, 1999; Peper, Bootsma, Mestre, & Bakker, 1994). Finding that the hand moves differently toward targets that are moving at different velocities but are hit at the same position (see Figure 1) has been considered to support the idea that the point of interception is not predicted before the hand starts to move, or at least not correctly, but that one relies on visual information during the movement to guide the hand to the target (Bairstow, 1987; Brenner & Smeets, 1996; Brouwer, Brenner, & Smeets, 2002; Smeets & Brenner, 1995; van Donkelaar, Lee, & Gellman, 1992). Such models obviously cannot guarantee an optimal overall choice of path. 

Figure 1

 

Reproduction of data from Figure 3A of de Lussanet et al. (2004) showing a top view of the average trajectories when hitting targets moving to the right across a frontal screen. The starting position is at the bottom of the figure. The arrows show the targets' motion as the hand moved toward the screen. Carefully selected starting positions provided two pairs of targets that moved at different velocities but were hit at about the same positions, while three targets moving at different velocities were at about the same position when the hand started to move. The path toward the 6 cm/s target was very similar to that toward the 12 cm/s target that was hit at the same position. In contrast, the path toward the 18 cm/s target started in the same direction as that toward the 12 cm/s target that was at the same position at that moment (for further details, see the original paper).

In this paper, we will limit ourselves to targets that are moving more or less orthogonally to the direction in which the hand has to move to intercept them. Different ways of guiding the hand to such a moving target give rise to different paths. For instance, always heading straight for the target predicts that the hand will approach targets from behind, lagging further behind the faster the target moves. This does occur for some ranges of velocity, but not for all velocities (de Lussanet, Smeets, & Brenner, 2004). It is difficult to think of a control strategy that would give rise to the asymmetry between fast and slow targets that is illustrated in Figure 1, and that could deal with a visuomotor delay of 110 ms (Brenner & Smeets, 1997). 

The error that arises from reaching a chosen target position at the wrong time increases with the target's velocity, so timing errors increase in importance relative to spatial errors as the target velocity increases (Brenner, de Lussanet, & Smeets, 2002; Brouwer, Smeets, & Brenner, 2005; Tresilian & Lonergan, 2002). We recently proposed that the different paths taken to intercept targets moving at different velocities arise from there being a velocity-dependent advantage in following a curved path (Brenner & Smeets, 2005). If people are more uncertain about when they will reach the target's path than about where the target will be at a certain moment, then it is advantageous for them to be moving along with the target near the moment of interception. If so, arriving slightly earlier or later than anticipated will be less detrimental. However, since a more curved path probably also gives rise to larger spatial errors, the optimal strategy is likely to be a compromise between moving straight toward the interception point and moving in a way that makes the hand move along with the target near the moment of interception. A model based on such a compromise, on the movements being smooth (minimal jerk; Flash & Hogan, 1985), and on the spatial accuracy depending on the trajectory (based loosely on signal-dependent noise; Harris & Wolpert, 1998) could account for the pattern of results shown in Figure 1 (Brenner & Smeets, 2005). This would be consistent with the way in which people move to intercept moving targets being the result of optimizing performance. 

In the present study, we examine how flexible people are in adapting the path that their hand takes to intercept moving targets. Introducing obstacles near the path to the target does not force people to adapt their hand's path, but there is a risk in not doing so. Using oriented targets introduces an evident advantage of adapting the hand's path. Having people ‘hit’ the targets in a specified direction more or less imposes a certain path. We examine how such manipulations influence the hand's path and whether being ‘forced’ to use a different path results in much poorer performance. 

Methods

We conducted three experiments in which subjects had to intercept moving targets by moving a pen across a large (WACOM A2) drawing tablet. Although the pen looked like a normal pen and was held like a normal pen, it did not leave any trace when it was moved. Instead, the tablet determined the position of its tip at 200 Hz. Figure 2 is a schematic depiction of our setup. Images (including a grey background) were projected onto a screen, 20 cm above a half-silvered mirror. The tablet was 20 cm below the mirror, so that its position coincided with the apparent position of the screen as seen through the mirror. Lamps between the half-silvered mirror and the drawing tablet ensured that subjects could clearly see the pen and their hand as well as the target. A simple calibration whereby the experimenter aligned the tip of the pen with small disks presented on the screen allowed us to later relate any position in the image to a position on the surface of the drawing tablet, and vice versa. Images were generated at 85 Hz with a resolution of 1024 by 768 pixels. The experiments were conducted in a normally illuminated room. 

Figure 2

 

The setup. Subjects moved their hand, which they could see through the mirror, to a target that was reflected by the mirror, while the tablet recorded the movements of the pen that they held in their hands. The task was either to move the tip of the pen through the target as quickly as possible (Experiments 1 and 2) or to hit the target toward a goal (Experiment 3).

In order for the movements to be made toward approximately the same position(s) in different conditions, we chose the position at which we would like subjects to hit the target for each condition and adapted the target's starting position on each trial to the time that it took the subject to intercept the target on the previous trial of that condition, so that if the subject would take as long on the new trial he or she would hit the target at exactly the place that we would like them to hit the target. On the very first trial, the target reached that position in 700 ms (600 ms in Experiment 3). Further details for each of the three experiments are given below, followed by a combined description of the data analysis for the three experiments. 

Experiment 1: Obstacles

Nine subjects took part in the first experiment, including the two authors. The other seven subjects were unaware of the hypothesis under study. Each subject took part in two sessions of 150 trials (25 for each of the 6 conditions). In one session, the targets moved to the right and in the other the targets moved to the left. In both cases the targets moved at 20 cm/s. Within each session 6 conditions were presented in random order. The trials of the different conditions differed in the position along the target's path at which we expected the subject to hit the target (left or right), and in whether or not there was an obstacle near the path, and if so where it was (none, left, or right). 

Trials started with a 2-cm diameter black disk appearing at a fixed position on the screen, close to the subject. Targets only appeared when the tip of the pen had been moved to this starting point and was held still within its bounds, so subjects had control of when the next trial would start. The positions at which we expected subjects to hit the target were 25 cm from the starting point in the sagittal direction (away from the subject) and either 3 cm to the left or 3 cm to the right of the starting point. If there was an obstacle (a 4-cm diameter red disk) it was 10 cm from the starting point in the sagittal direction and 5 cm left or right of the starting point. The obstacle appeared at the same time as the starting point and remained visible for the whole trial. 

The starting point disappeared shortly after the subject placed the pen within its bounds, at the moment that the moving target (a green 2-cm diameter disk) appeared. The subjects' task was to hit the target as quickly as possible without hitting the obstacle (if present). We considered the target to have been hit as soon as the (interpolated path of the) pen made contact with the contour of the target (at some interpolated position). If the pen hit the target, the latter stopped moving and subjects heard a short sound. If they hit the obstacle, a message appeared on the screen telling them that they had done so. 

Experiment 2: Oriented targets

Nine subjects took part in the second experiment, including the two authors. The other seven subjects were unaware of the hypothesis under study. Five of them had also taken part in Experiment 1. Each subject took part in one session of 180 trials (15 for each of the 12 conditions). The task and procedures were identical to those in Experiment 1, except that there were no obstacles, the target always moved to the right at 20 cm/s, and the target was a green 3 by 0.5 cm bar that could be oriented at one of 6 different angles (orientations differing by 30 deg, including targets oriented along and orthogonal to their motion). Trials of the 12 conditions (left or right position; 6 orientations) were presented in random order. 

Experiment 3: Hitting toward a goal

Nine subjects took part in the third experiment, including one of the authors. The other eight subjects were unaware of the hypothesis under study. Five of them had taken part in at least one of the previous experiments. Each subject took part in one session of 300 trials (20 for each of the 15 conditions). Their task was to hit a target (a 3-cm diameter green disk) toward a goal (a 6 cm diameter blue disk). The target could be static or it could move either to the left or to the right at 20 cm/s. For each of the three kinds of target motion, the goal could be at one of five different positions, giving 15 conditions that were presented in random order. 

The position at which the moving target appeared was manipulated to try to make subjects always hit the target 15 cm from the starting point in the sagittal direction. The starting point (a 1-cm diameter black disk) disappeared shortly after the subject placed the pen within its bounds, at the moment that the static or moving target appeared. The goal was visible from the moment that the starting point appeared (i.e., well before the target appeared) and it remained visible until the end of the trial. The center of the goal was 10 cm from the position at which we wanted the target to be hit, either further in the sagittal direction or 30° or 60° to the left or right of sagittal. The subjects were not instructed to hit fast, but to hit the target to the goal. 

As soon as the pen hit the edge of the target, the target moved away in the direction orthogonal to its surface at that position, with a velocity that corresponded with the component of the pen's velocity in that direction (to increase the impression of the target's new motion being a consequence of the ‘impact’). The initial motion of the target itself and the direction of motion of the pen were not considered when determining the target's motion after contact, only the position of contact really mattered. Although this behavior is not completely ‘natural’ (e.g. no simulated conservation of momentum, no haptic feedback), it only took subjects a few trials to grasp the way that the target responded to the impact. Due to the delays in the tablet and projection system, the target actually moved on a bit before jumping to the appropriate position on the new trajectory (see example in Results) but subjects did not notice this (perhaps partly because their hand was in the way at that moment). If the target hit the goal subjects heard a short sound and the goal turned yellow. 

In this experiment, we also measured the subjects' eye and head movements. The movements of both eyes relative to the head were recorded at 500 Hz (Eyelink II, SR Research Ltd., Canada). Head movements were recorded at 250 Hz using custom software and the head tracking capabilities of the Eyelink II. Each session began with a simple calibration of the eye movement recordings. Since we were not interested in vergence, we averaged the orientations of the two eyes. We determined changes in eye and head orientation as well as displacements of gaze across the tablet surface (as calculated from the displacements of the head and the rotations of the head and eyes). 

Analysis

Trials were removed from the analysis if the target disappeared before it was hit (because the subject started moving too late or not at all, or moved too slowly), if any data were lost (because the subject lifted the pen too far off the surface before reaching the target), or if the target presentation was not perfect (because the computer failed to present a new image on every frame). Whenever possible, we analyzed all remaining trials, irrespective of whether the target was hit. However, for many measures, trials in which subjects missed the target could not be used, because the end of the movement was undefined. Thus, our analyses of the paths and movement times (and of the movements of the eyes and head) are based only on trials in which the target was hit. For the remaining measures of performance, misses were usually considered. Whether misses were considered is indicated for each measure. 

Paths

For Experiments 1 and 3, we determined the average path for each condition. We only considered trials in which the target was hit, and only if the pen remained on the tablet until it reached the target so that no data were missing. We consider the pen's position at the moment that the target appeared to be the beginning of the path. The end of the path was the (interpolated) point at which the pen made contact with the target. We divided each path into 100 parts of equal length, providing us with coordinates of 101 equidistant points along each path, and then averaged the coordinates of these points across all paths. We first averaged the coordinates per subject, and then across subjects, because the number of trials was not completely identical across conditions and subjects. To evaluate whether differences between the paths for the different conditions were consistent across subjects, we conducted repeated measures analyses of variance on the pen's average lateral position half way to the target. 

For Experiment 2, the average path is not a good measure of how subjects move because for some orientations there are two suitable paths, with intermediate paths being less suitable. For instance, for a bar oriented along the main direction of the pen's movement, the pen can best take a strongly curved path to approach the bar from the front or behind, so that it reaches one of the long sides of the bar. Averaging paths arriving from the front with ones approaching from behind would give the wrong impression that subjects aim for the narrow part of the bar. We therefore determined the direction in which the pen was moving when it hit the target on each trial and related this to the spatial errors on a trial-by-trial basis, rather than relying on changes in the average path. 

Performance

An obvious measure of performance in all experiments was whether the pen hit the target. In the third experiment, an additional measure was whether the target hit the goal. In addition to these measures, we determined measures of the accuracy of the hit that differed between the experiments (as explained in the next paragraph). Unless stated otherwise, we determined the median value for each subject and condition. We determined the median rather than the mean in order not to have to worry about outliers and skewed distributions. We used these median values to evaluate the consistency in the differences between conditions across subjects (using repeated measures analyses of variance) and to calculate the mean value (and standard error) for each condition across subjects (for presentation in the figures). 

For Experiment 1, our measure of accuracy was how close the pen came to the target center on each trial (even if the target was not hit). This distance is an unsigned measure. For Experiment 2, we used the signed distance between the pen and the center of the edge that was hit at the moment of the hit. If the pen missed the target, we determined what this value would have been if the target had been long enough to have been hit. We will refer to this measure as the error. For Experiment 3, we used the distance along the edge of the target between where the pen hit the target and where hitting the target would have made the target hit the center of the goal (considering the exact position of the target when it was hit). We consider both the unsigned error (distance in cm) and the systematic and variable components of the signed error (in terms of the angle at which the target moves after impact). Obviously these three measures only consider trials in which the target was hit. Moreover for the systematic and variable components, we determined the mean and standard deviation rather than the median. 

For Experiment 2, we present errors for individual trials (for the reasons mentioned above). In addition, we calculated the variability in the errors (standard deviations) for 10° bins of the direction in which the pen was moving when it hit the target (either relative to the target's path or relative to the target's orientation). For each subject, we only considered the standard deviation within a bin if there were at least 7 values within that bin. We only considered bins to which at least 2 subjects contributed. Since different subjects contribute to different bins, there is a danger of interpreting systematic differences between subjects as differences between directions. To avoid this, we normalized the standard deviations by scaling all values for each subject in relation to the subject's value for the bin at 0° (so the value at 0° is 1 by definition). 

Movement time

For all trials in which the target was hit, we also determined the movement time. The movement was considered to have started when the pen had moved 1 mm from the starting point in the sagittal direction, and to have ended when the target was hit. Here too we determined the median for each subject and condition and used the median values for the statistical analysis as well as averaging the median movement times across subjects for the figures. 

For Experiment 2, we also averaged the movement time for 10° bins of the direction in which the pen was moving when it hit the target. We only considered a subject's movement time within a bin if there were at least 3 values within that bin, and only considered bins to which at least 2 subjects contributed. To prevent differences between subjects from being interpreted as differences between directions, we scaled all the values for each subject so that each subject's value for the bin at 0° was equal to the average value of all the subjects' 0° bins before scaling. We will refer to the resulting values as normalized movement times. 

Eye and head movement

One possible reason for the pen's path curving in a target-motion-dependent manner is that subjects pursue the target with their eyes. If the movement of the pen is tightly linked to that of the eyes, then the pen may initially aim too far toward the instantaneous direction of the target as the eyes saccade in that direction, and then follow the motion of the eyes as the latter pursue the target. In the third experiment, in which the pen sometimes had to move in the opposite direction than the target, we therefore determined the average velocity at which the eyes (and head) were rotating (horizontally) near the moment of the hit. We also determined when subjects made saccades during this period. To do so, we determined the average velocity of the eyes during a moving window of 20 ms. Saccades were initially identified as moments at which the eyes moved faster than eight times the subject in question's median eye speed (during the whole experiment). The onset of the saccade was then found by searching back for the first moment at which the rotation of the eye between two samples was slower than the above-mentioned median velocity. 

In addition to analyzing the eye movements near the moment of the hit, we also used the eye and head measurements to estimate where the subject was looking (gaze). We distinguished between looking at the starting point, target, goal, and pen. We considered subjects to be looking at the item (of these four) that was closest to where we estimated that their gaze was directed. Since we considered the whole item (rather than its center) when we did so, we are biased toward assuming that subjects were looking at the larger object when the two overlap or are close to each other. 

Results

Experiment 1: Obstacles

In total, we lost 52 of the 1350 trials (almost 4%), mainly due to the pen being lifted too far off the surface during the movement, but occasionally a subject simply did not move the pen toward the target. Only two subjects ever hit an obstacle, and each only did so once. The target was hit on 1049 trials. Figure 3 shows the average path for each condition. Comparing the solid, dashed, and dotted curves, we see that placing an obstacle near the pen's path makes people move on a slightly different path in order to remain further from the obstacle ( p = 0.006 for the influence of the presence and position of the obstacle on the lateral position of the pen midway to the target). The influence of the obstacle depended on the position at which the target was to be hit (significant interaction between the presence and position of the obstacle and the position at which the target was to be hit; p = 0.001). 

Figure 3

 

Average path for each of the twelve conditions: two directions of target motion (green for leftward and red for rightward motion), two intended target positions (left and right grey discs), and three options for the obstacle (dashed for obstacle on the left, dotted for obstacle on the right, and solid for no obstacle).

The paths toward targets moving to the left and to the right (red and green curves) did not differ significantly half way to the target, but the small difference in curvature that is visible in the figure is consistent with a tendency to move along with the target near the moment of contact. Subjects probably aimed for the target center because they tended to hit the near-front edge of the targets, where the front is defined relative to the target's motion. The paths are certainly not mirror images across the midline, so presumably biomechanical factors or the fact that the hand is more likely to occlude objects on the right (only one subject was left-handed) influenced the path. 

The median movement time ( Figure 4) was significantly shorter for targets that crossed the midline before being hit (significant interaction between position and direction of motion; p = 0.0002). There was also a significant main effect of position ( p = 0.03) and a significant interaction between position and the presence and position of the obstacle ( p = 0.04). Most importantly, subjects' movement times were not consistently shorter when there was no obstacle. 

Figure 4

 

Median movement time on trials in which the target was hit for each condition of Experiment 1, averaged across subjects. Green bars are for leftward and red ones for rightward target motion. From left to right, the pairs of bars of each color are for no obstacle, obstacle on the left, and obstacle on the right. Within each pair the left bar is for the target on the left and the right bar is for the target on the right. Error bars are standard errors across subjects.

The number of targets that were hit ( Figure 5) was not influenced significantly by any of the factors (position; direction of motion; presence and position of obstacle). For the more sensitive measure, the median distance at which the pen crosses the target center ( Figure 6), there was a significant interaction between position and direction of motion ( p = 0.02). Subjects hit closer to the target center in the same conditions as they hit faster, so this is not the consequence of a speed-accuracy trade-off. It may have to do with the fact that the hand can approach the target slightly more from behind (moving along with the target) without having to follow a more curved path if the target has moved further before being hit. 

Figure 5

 

Average percentage of targets that was hit successfully for each condition of Experiment 1. For further details, see caption of Figure 4.

Figure 6

 

Median nearest distance that the pen came to the target center for each condition of Experiment 1, averaged across subjects. For further details, see caption of Figure 4.

Experiment 2: Oriented targets

We lost 56 of the 1620 trials of the second experiment (about 3%), again mainly because subjects lifted the pen. Subjects hit the target on 1392 trials. As in Experiment 1, the movement time was shorter for targets that had passed the midline (black bars in Figure 7; p = 0.003). It also depended on the target orientation ( p = 0.0001): It was larger when the target was oriented in a manner that makes it necessary to follow a more curved path to arrive orthogonal to the long side of the target. The interaction between the position and the orientation was also significant ( p = 0.02). The proportion of targets that was hit did not differ significantly between the conditions ( Figure 8). 

Figure 7

 

Median movement time on trials in which the target was hit for each condition of Experiment 2, averaged across subjects. Grey bars are for the target on the left and black bars are for the target on the right. The symbols below each pair of bars indicate the orientation of the target, with target motion being to the right and the pen moving ‘upwards’. Thus, the green horizontal target is oriented along its direction of motion with its long edge more or less facing the approaching pen. Error bars are standard errors across subjects.

Figure 8

 

Average percentage of targets that was hit successfully for each condition of Experiment 2. For further details, see caption of Figure 7.

When changing the target orientation, one must consider that moving along the same path will give rise to different errors for differently oriented targets, and that changing the path will change the distance and therefore perhaps also the movement time and variability. We therefore analyzed the data in relation to the paths on individual trials. To quantify the path with a single number, we took the direction in which the pen was moving at the moment that it hit the target. Figure 9 shows the distribution of such directions for all target orientations. Figures 10, 11, and 12 show the movement time and hitting error as a function of this direction. 

Figure 9

 

Histogram of directions in which the pen is moving when it hits the target for each target orientation in Experiment 2 (10 deg bins; zero indicates sagittal motion; positive values indicate motion to the right). The colors indicate the target orientations (as shown by the symbols at the top, assuming target motion to the right and the pen moving ‘upwards’). The dotted lines indicate directions orthogonal to the long side of the target.

Figure 10

 

Movement time as a function of the direction in which the pen is moving when it hits the target (zero indicates sagittal motion; positive values indicate motion to the right). Each dot represents one trial of Experiment 2. Its color indicates the target orientation, as coded by the symbols at the top. These symbols' horizontal positions indicate directions of pen motion for which the target would be hit orthogonal to its long side. Note that each target orientation is represented twice. The yellow filled circles show normalized average movement times for 10° bins, with standard errors (for details, see text).

Figure 11

 

Hitting accuracy as a function of the direction in which the pen is moving when it hits the target. Error is the distance from the target center, with positive values indicating deviations to the right with respect to the direction of approach. For details, see caption of Figure 10. The yellow filled circles show normalized average values of the standard deviation in the point at which the target is hit (see right axis) for 10° bins.

Figure 12

 

Hitting accuracy as a function of the angle between the direction in which the pen is moving when it hits the target and a line orthogonal to the long side of the target. Error is the distance from the target center, with positive values indicating deviations to the right with respect to the direction of approach. The horizontal positions of the symbols at the top indicate the angle at which the pen would approach the target if it were to move straight to the target in the sagittal direction. The yellow filled circles show normalized average standard deviations in the point at which the target is hit (see right axis) for 10° bins. The yellow curve indicates the normalized standard deviation that one would expect for this angle of approach for purely geometrical reasons. For further details, see text and caption of Figure 10.

From Figure 9 and the horizontal positions of the differently colored points in Figures 10 and 11, it is clear that subjects adjusted their movements to the target orientation, but the paths are not curved enough to ensure an approach of the target orthogonal to its long side (dots of each color are not aligned with symbols of the same color at the top; also see systematic horizontal shifts in Figure 12). There appears to be a tendency for the pen to be moving to the right (a general shift of the distribution toward positive values on the horizontal axis), along with the target. This could have to do with our subjects being right-handed, although it could also arise from an attempt to limit the relative (lateral) velocity between target and pen (Brenner & Smeets, 2005). 

It is difficult to interpret the vertical distribution of the dots in Figures 10, 11, and 12. The movement time appears to increase for paths that are presumably more curved (directions further from zero in Figure 10), and the variable error increases when the target is approached almost parallel to its long side (angles close to ±90° in Figure 12), but otherwise it is difficult to tell whether there are any systematic effects in the data. In particular, it is impossible to differentiate between differences related to the subject and differences related to the path. To overcome this problem, we also determined average normalized values within 10° bins. 

The average normalized movement times (yellow filled circles) in Figure 10 confirm that movements take longer if the final direction is far from zero (indicating that the path was curved) and that the fastest movements are made when the pen approaches the target moving slightly to the right (in the direction of target motion). There is no systematic relationship between the normalized standard deviation in the position of the hit and the direction of approach (yellow filled circles in Figure 11). We see a modest effect of the angle of approach relative to the target, which is consistent with the geometrical disadvantage of a non-orthogonal approach (yellow filled circles follow yellow curve in Figure 12). Together, these findings suggest that subjects adjusted their paths and movement times to maintain a more or less consistent precision across target orientations. Residual differences in precision are primarily due to the fact that subjects hit some targets at a less advantageous angle ( Figure 12), possibly because they wanted to comply with the request to hit the targets as quickly as possible, but it is also possible that they could not have done any better: Moving even more slowly along a more curved path may not improve the precision because moving more slowly increases the temporal error as well as decreasing the spatial error. 

Experiment 3: Hitting toward a goal

We lost 137 of the 2700 trials of the third experiment (about 5%), again mainly because subjects lifted the pen. The task now forced subjects to move on very different paths for hitting the targets toward the different goals (see Figure 13). Half way to the target, the paths were significantly different for the 5 goal positions ( p < 0.0001), but not for the different target motion conditions. At the moment that the target was hit, the lateral position of the pen was significantly different for the different goal positions ( p < 0.0001) and target motion conditions ( p = 0.0006), and there was a significant interaction between the two ( p = 0.0004). The latter effects are at least partly due to the fact that we did not succeed in having the subjects hit the target at the same position under all conditions (see bars in Figure 13). In particular, if the target was moving toward the goal as it passed the position at which we wanted them to hit it, subjects tended to ‘wait’: They moved much more slowly (see Figure 14). This contributed to a significant effect of goal position ( p = 0.001) and gave rise to a significant interaction between goal position and target motion condition ( p = 0.007) for the median movement times. 

Figure 13

 

Average path for each of the fifteen conditions of Experiment 3: three kinds of target motion (black curves for static targets, green ones for leftward target motion, and red ones for rightward target motion) and five positions of the goal (curves from left to right correspond with goals from right to left). The size and positions of the goals are shown by the circles at the top of the figure. The bar chart shows how far left or right of the position indicated by the grey disk the target was at the moment that it was hit (average with standard error across subjects; drawn to the same scale as the rest of the figure, with bars from top to bottom corresponding with goals from left to right).

Figure 14

 

Movement times on trials in which the target was hit for each of the fifteen conditions of Experiment 3. Bars are averages of the median movement times of all nine subjects (with standard errors). The white symbols are averages excluding one subject who moved extremely slowly. Red bars are for rightward target motion, black ones are for static targets, and green ones are for leftward target motion. For each kind of target motion, the leftmost bar is for the leftmost goal, and so on.

In this experiment, we did not instruct subjects to be as quick as possible, and there is a clear geometrical advantage to hitting the targets later in some conditions because they are then closer to the goal. Our procedure ensures that subjects cannot keep hitting targets earlier than we intend them to do so, but they can keep hitting later. One subject in particular did this. However, removing his data does not change the pattern of results (see symbols in Figure 14). We see little tendency for subjects to take less time to hit targets that pass the goal before being hit; they take about as long to hit such targets as to hit the corresponding static ones. For static targets, subjects clearly tend to take more time when the pen follows a more curved path (black bars). This too contributes to the abovementioned significant effect of goal position. 

Figures 15, 16, and 17 summarize the subjects' performance in this task. Subjects hit the target on almost all trials (symbols in Figure 15). Nevertheless, there was a significant effect of target motion condition ( p = 0.02) and a significant interaction between target motion condition and goal position ( p = 0.003) on the percentages of targets that were hit. The goal position ( p = 0.005) and target motion condition ( p < 0.0001) both influenced the percentage of goals that were hit (bars in Figure 15), and the interaction between the two factors was significant ( p = 0.0003). In accordance with this, the goal position ( p = 0.002) and target motion condition ( p < 0.0001) both influenced the median errors ( Figure 16), and again the interaction between the two factors was significant ( p = 0.0004). Similarly, the mean and standard deviation of the angular error ( Figure 17) depended on the goal position (respectively p < 0.0001 and p = 0.002), target motion condition (both p < 0.0001), and the interaction between the two (respectively p = 0.004 and p = 0.0004). 

Figure 15

 

Percentages of targets hit (symbols) and of goals hit by the target (bars) for each of the fifteen conditions of Experiment 3. For further details, see caption of Figure 14.

Figure 16

 

Distance along the target's edge between the point of contact and the point for which the target would move straight toward the goal (considering the actual target position). Average of the 9 subjects' values for each of the fifteen conditions of Experiment 3. Only trials in which the target was hit are considered. For further details, see caption of Figure 14.

Figure 17

 

Average systematic (symbols) and variable (bars) components of the 9 subjects' errors for each of the fifteen conditions of Experiment 3. The errors are expressed as the mean and standard deviation of the angle between the direction in which the target is hit (see Methods) and the direction to the goal. Positive values indicate a mean error in the counterclockwise direction. For further details, see caption of Figure 14.

When the target was static, subjects performed best when the goal was straight ahead in the sagittal direction. They hit most goals (black bars in Figure 15), hit closest on the target edge to the optimal position ( Figure 16), and had the smallest standard deviation in the direction in which they hit the target ( Figure 17). When the target was moving, subjects' performance depended on the relationship between the direction of target motion and the position of the goal: They performed best on all our measures when they had to hit the target back in the direction from which it came. 

When the target was moving away from the goal at the moment that it was hit (goals at the left and rightward target motion or goals at the right and leftward target motion), performance was about as good as when the target was static. Performance was clearly worse when subjects had to hit the target further in the direction in which it was moving. This is so for the number of goals that were hit ( Figure 15) as well as for the other measures of accuracy ( Figures 16 and 17). The movement times were longer for the conditions with poorer performance ( Figure 14), and the target was hit closer to the goal ( Figure 13), so the poorer performance must be the result of some fundamental difference between the conditions rather than the result of a speed-accuracy trade-off. Within conditions, and in particular those conditions in which performance was poor, we do see signs of a conventional speed-accuracy trade-off ( Figure 18): We see that faster movements (shorter movement times) are associated with larger unsigned errors (hitting the target's edge further from the ideal position). 

Figure 18

 

Correlation between movement time and error along the target's edge, within each subject's trials for each condition of Experiment 3. Symbols show averages across subjects with standard errors. Negative values indicate a tendency to make larger errors when moving faster.

Figure 17 also shows that subjects tend to systematically hit the target too far in the sagittal direction (symbols). This suggests that subjects could have improved their performance by moving along more curved paths. However, the performance for static targets shows that moving on more curved paths also has substantial costs. Thus, the systematic errors may arise from a compromise between these two effects of more curved paths. 

Subjects also tend to hit the moving targets slightly too far back in the direction from which they came (see shift between the three sets of symbols in Figure 17). This could mean that subjects considered the fact that the target was moving when evaluating how it would move after the impact (although this was not considered in the simulation; see Methods), or it could be a result of differences between the influences of spatial and timing errors (we will return to this in the Discussion). 

In this experiment, we also measured eye and head movements. Figures 19 and 20 show the average horizontal angular velocity of the eyes and head near the moment that the pen hits the target. The eyes clearly pursue the target until about 80 ms before the pen hits it, irrespective of the goal position (see Figure 19). After that time, the traces diverge, indicating that the eyes are influenced by the goal position. Figure 21 shows that subjects start to make more saccades at this time, presumably shifting their gaze toward the goal. As one may expect, the head did not contribute much to the pursuit of the target, although there does seem to be some effect (see Figure 20). 

Figure 19

 

Average horizontal angular velocity of the eyes (relative to the head) from 250 ms before the pen hits the target to 50 ms after it does so. Positive is to the right. Obviously only trials in which the target was hit are included. Eye movements were first averaged within subjects and then across subjects. The data are so ‘noisy’ despite averaging many trials because saccades were not removed. Red curves are for trials in which the target moved to the right. Green curves are for trials in which the target moved to the left. Black curves are for trials in which the target was static. When it moved, the target moved at about 20 deg/s. The five curves of each color are for the different goal positions.

Figure 20

 

Average horizontal angular velocity of the head from 250 ms before the pen hits the target to 50 ms after it does so. For further details, see caption of Figure 19.

Figure 21

 

Frequency of occurrence of saccade onset in 10-ms bins from 300 ms before the pen hits the target to 100 ms after it does so.

Figure 22 shows how gaze shifts across the surface, together with the target and pen, on a single trial. To determine the shifts in gaze, we considered the position and orientation of the head as well as the orientation of the eyes in the head. It is evident from this figure that gaze is not always directed exactly at one of the relevant structures. It could be that precise fixation of such structures is not required for this task, but errors in calibration (for instance due to a small shift of the cameras that determine the eye orientation) probably also contribute to this. 

Figure 22

 

Example of shifts in gaze (thin line), pen position (thick line), and target position (small circles, each representing the position on one frame) during a single trial. Black lines and circles are for the period before the pen starts to move, blue ones for the period during which the pen is moving toward the target, and pink ones for a short period after the pen hits the target. The grey disks indicate the starting position, the position of the target when it is hit, and the position of the goal. The subject made a first saccade to a position close to where he would later hit the target. At about the time that his hand started to move, he made a second saccade that directed his gaze at the target that he then pursued until just before hitting it. At that time, he made a third saccade reaching the goal at about the same time as the pen hit the target. Note that the target continues to move along its original path for about 62 ms after impact before jumping to the appropriate positions on the new path (three pink circles at top right). This is due to the delays in our system, but was not noticed by the subjects.

Assuming that the subjects were always looking at the item (starting point, target, goal, or pen) that was closest to our estimate of where their eyes were directed, we could examine what they were looking at throughout the action. Figure 23 summarizes this for all the movements. What it shows is the fraction of time spent looking at each item as a function of the stage of the action. Naturally subjects tend to look at the starting point in order to bring the pen to it to start a trial. Close to when the target appears, they tend to direct their gaze toward the region near the goal (also see example in Figure 22). By the time that they start to move the pen, they have usually directed their gaze toward the target. They fixate or pursue the target until shortly before hitting it, at which time they direct their gaze toward the goal (and then toward the starting point for the next trial). This pattern of eye movements is consistent with that of previous studies in which the eyes also tended to fixate items that were relevant to a manual task until just before the hand reached them (Johansson, Westling, Bäckström, & Flanagan, 2001; Land & Hayhoe, 2001) and is understandable if one considers that the movement of the hand can no longer be influenced by new visual information during the last 110 ms anyway (Brenner & Smeets, 1997). 

Figure 23

 

Summary of the structures that were closest to where gaze was directed throughout Experiment 3. Time is expressed as a percentage of the time between the target appearing and it being hit. The approximate equivalent time in ms is shown at the top of the figure. The grey-shaded area represents the time during which the pen is moving toward the target (its left edge is at the median reaction time). Initially, the subject is often either still looking at the starting point (green curve) or has made a saccade toward a position near the goal (blue curve). While the pen is moving, gaze is usually directed toward the target (red curve). Near the time that the pen hits the target, gaze shifts from the target to the goal. Since the tip of the pen is small, and we measure distance from the structures' edges, we are biased toward considering gaze to be directed at the starting point and target when the pen is on those structures (see dip in the grey curve at the moment that the target is hit).

Discussion

It is evident that subjects could draw on a variety of trajectories to successfully intercept moving targets. This is not surprising because in many sports people have to intercept moving objects in a specific manner (as they do in our third experiment). We here show that subjects do not need extensive practice to adapt their movements to new task constraints. We also demonstrate that there is indeed a cost to moving on a curved path (or more precisely to moving on a path that is curved differently than the path that one would normally use to reach the point of interception). This cost is most evident in the movement times ( Figure 14) and accuracy ( Figures 15, 16, and 17) when hitting static targets in Experiment 3, and in the movement times in Experiment 2 ( Figure 10). 

The most obvious reason for performance being poorer when following a more curved path is the increased path length. However, the additional forces that the muscles have to exert in order to achieve specific paths probably also contribute to this, both because there will be some variability in the amplitude of each additional force, and because more complicated patterns of muscle activation are likely to be more sensitive to timing errors. For movements away from the body, moving on a curved path may also increase the sensitivity to inaccuracies in visual judgments of the target's distance. In all three of our experiments, subjects had to find a compromise between the disadvantages of following a more curved path and other aspects of the task. 

In Experiment 1, the advantage of following a more curved path was that the likelihood of hitting the obstacle is reduced. Subjects could adapt the path sufficiently to avoid hitting the obstacle with barely noticeable consequences for other aspects of their performance. In Experiment 2, the advantage of following a more curved path for some targets is that approaching such targets from a more advantageous angle (with respect to the orientation of the target) reduces the influence of any spatial or temporal errors in the control of the movement of the hand. Subjects found a compromise between arriving at the target from a sub-optimal direction and moving along a sub-optimal trajectory, for which there was little loss in performance (in terms of accuracy and movement time; see Figures 11 and 12). 

In Experiment 3, the task itself explicitly required people to follow a curved path because the position at which they hit the edge of the target was relevant for the task: It determined whether the target hit the goal. Here too there is an advantage in approaching the edge orthogonally, so we could expect very curved paths. Moreover subjects were not instructed to hit as fast as possible, so at least one reason for not following a very curved path was eliminated (timing was obviously still constrained for moving targets, but subjects did not move exceptionally slowly toward static targets so this cannot have been very important). The fact that subjects tended to hit the targets in a too sagittal direction (symbols in Figure 17) could be the result of trying to reduce path curvature. However, a more notable finding for the interpretation of the data of Experiment 3 is that there were certain conditions in which performance was exceptionally poor: when hitting a target toward a goal that was largely in the direction of target motion. Why should this be so? 

As mentioned in the Introduction, we previously proposed that it is advantageous to move along with the target (Brenner & Smeets, 2005). So why do we find exceptionally poor performance when the pen moves along with the target in Experiment 3? There is an important difference between the experiments of the present study and ones in which subjects have to hit a position on a screen at the moment that a moving target crosses that position. In the latter case, the only moment that counts is when the hand hits the surface, irrespective of whether the target is at that position at the time, so one can minimize the consequence of misjudging when the hand will reach the surface by moving along with the target near the anticipated moment of contact. In the present study, the decisive moment is when the pen makes contact with the target. In Experiment 3, it is important to hit the appropriate part of the target at the correct moment. If the pen is moving in the opposite direction than the target, misjudging when and where the target will be hit will not be too detrimental to performance, because the pen and target are approaching each other very quickly, so the actual position at which they make contact will never be very incorrect. In contrast, if the pen is moving in the same direction as the target, the pen approaches the target less fast, so the errors introduced by arriving at the anticipated point of contact slightly earlier or later than the target are larger (in terms of where in space the target is hit as well as which part of the target is hit). 

In order to determine whether such timing issues could account for the very poor performance in certain conditions of Experiment 3, we modelled the results for the moving targets of Experiment 3 in a very simple manner. We assume that the variance related to path curvature is identical for moving and static targets and that it is the same for paths that curve leftward and rightward. We therefore take the average measured standard deviation in the position at which static targets were hit (when the goal was at midline and when it was ±30 and ±60 deg from midline) as our estimates of curvature-related variability. These three values are to be found as the five black bars in Figure 24. 

Figure 24

 

Standard deviation in where along its edge the target is hit, for each of the conditions of Experiment 3. Pale bars: measured data (equivalent to the bars in Figure 17). Bright bars: predictions of a simple model that combines larger variability for more curved paths with a timing uncertainty of 20 ms. For further details, see legend of Figure 14.

For the moving targets, we also consider variability related to timing errors (we ignore the fact that the paths are not identical for static targets and targets that move leftwards or rightwards; Figure 13). To estimate the consequences of timing errors, we took the average position and velocity of the pen at the moment that it hit the target (for each of the ten conditions with moving targets) and estimated when and where the pen would hit the target if it arrived at the same position slightly earlier or later. We assumed that the pen was moving at a constant velocity. If we take a standard deviation in timing of 20 ms, we can estimate the spatial error that is introduces in terms of where the pen hits the edge of the target. Combining this with the influence of the curvature of the path (by quadratic summation of the two standard deviations) gives the colored bars in Figure 24. Figure 24 shows that this could be the origin of the errors. Thus, the variations in performance in Experiment 3 are probably not a consequence of a poor choice of path and movement speed in certain conditions. They are probably an inevitable consequence of the combined demands of the task given some temporal uncertainty about one's own movements. 

Conclusions

Our results are consistent with people optimizing their trajectories to the task requirements. We could account for all the differences between the paths that we recorded in terms of the costs and benefits of following a more curved path. In the first experiment, the adjustments when an obstacle was introduced were very modest and had little costs. In the second experiment, subjects did not always approach elongated targets at the angle that would be expected to result in the smallest errors for geometrical reasons. Subjects presumably accepted the consequence of the geometrical disadvantage to avoid taking much longer and moving along much more curved paths. In the third experiment, the task forced subjects to sometimes follow a very curved path. Performance was clearly poorer when the path was very curved, as well as when precise timing was required. 

If our reasoning is correct, then our study shows that subjects are not only aware of the resolution of the outcome of their actions (Trommershäuser, Maloney, & Landy, 2003; Trommershäuser, Mattis, Maloney, & Landy, 2006), but also of the influence of the way the actions are performed (the path and velocity profile) on the resolution of the outcome. Conversely, assuming that there must be a benefit in making the movements in the way that they are made, we can use the choices and errors to help identify the factors that limit performance, such as the temporal uncertainty of the order of 20 ms that we here propose (see also Brenner & Smeets, 2005). Of course it is even safer to make this assumption for over-trained actions under natural conditions: for actions that we all do all the time, or for specific actions by specific people who train those actions extensively. 

Acknowledgments

Commercial relationships: none. 

Corresponding author: Eli Brenner. 

Email: [e-mail]. 

Address: Faculty of Human Movement Sciences, Vrije Universiteit, Van der Boechorststraat 9, Room B-642, NL-1081 BT Amsterdam, The Netherlands.