Catching fly balls in virtual reality: A critical test of the outfielder problem

Abstract

How does a baseball outfielder know where to run to catch a fly ball? The “outfielder problem” remains unresolved, and its solution would provide a window into the visual control of action. It may seem obvious that human action is based on an internal model of the physical world, such that the fielder predicts the landing point based on a mental model of the balls trajectory (TP). However, two alternative theories, Optical Acceleration Cancellation (OAC) and Linear Optical Trajectory (LOT), propose that fielders are led to the right place at the right time by coupling their movements to visual information in a continuous “online” manner. All three theories predict successful catches and similar running paths. We provide a critical test by using virtual reality to perturb the vertical motion of the ball in mid-flight. The results confirm the predictions of OAC but are at odds with LOT and TP.

Introduction

How does an outfielder in the game of baseball know where and when to run to catch a fly ball? The “outfielder problem” is a particularly demanding case of the general problem of interception (Fajen & Warren, 2007). To catch a fly ball, the fielder must not only intercept a moving target but must do so at a specific place and time. Catching involves two phases guided by different processes: moving the body close to the landing point of the ball (e.g., Chapman, 1968) and moving the hand relative to the body to make the final catch (e.g., Peper, Bootsma, Mestre, & Bakker, 1994). The outfielder problem concerns the first phase. Moreover, the fielder must accomplish this with sparse visual information; at an initial distance of over 30 m, most absolute distance cues (binocular disparity, vergence, motion parallax, optical expansion) are not detectable (Cutting & Vishton, 1995), nor are variables such as the balls physical velocity and spin. Approaches to the outfielder problem differ in important ways, such as whether an internal world model or an informational coupling is used to guide behavior. Thus, determining how fielders solve this problem carries implications for general principles of model-based and online control in perceptual-motor tasks. 

The first solution to come to mind is trajectory prediction (TP). Saxberg (1987) proposed that fielders perceive the initial conditions of the balls motion and compute its trajectory to predict where and when the ball will land. TP thus requires accurate perception of the balls distance, speed, and direction of motion, as well as variables such as spin, air density, and gravity that can significantly affect the trajectory (Adair, 1990; McBeath, Nathan, Bahill, & Baldwin, 2008). To predict where the ball will land, these variables must be combined in a sophisticated internal model of projectile motion that takes them all into account. Evidence for such a model is lacking; even skilled baseball players cannot identify correct ball trajectories or predict landing points (Shaffer & McBeath, 2005). 

Alternatively, fielders could use visual information to continuously guide their movements, leading them to the landing position at the right time without explicitly predicting either variable. By directly coupling their movements to optical variables in an online manner, this approach avoids dependence on an internal model and computation of the trajectory. Two theories of this type have been proposed. 

Optical acceleration cancellation (OAC; Chapman, 1968; McLeod & Dienes, 1996, McLeod, Reed, & Dienes, 2006; Michaels & Oudejans, 1992) posits that radial movement toward and away from the ball is controlled by canceling the vertical acceleration of the balls optical projection (i.e., the tangent of the balls elevation angle, tanα, or its projection on a vertical plane, see Figure 1). If the balls optical velocity (dtanα/dt) is increasing, the fielder should accelerate backward, and if it is decreasing, accelerate forward, thereby keeping the optical velocity approximately constant. Tangential movement perpendicular to the direction of the ball is controlled independently by matching the balls lateral position (Chapman, 1968), either by keeping the ball in a constant bearing direction (azimuth ψ) or by holding its horizontal angle to home plate (β) constant (Fajen & Warren, 2007; McLeod et al., 2006; Tresilian, 1995). 

Figure 1

 

Convention for angles and directions used by OAC and LOT. The fielder moves with velocity v, with radial (v_r) and tangential (v_t) components, while moving to catch the ball that originates at home plate (upper left). Angles to the ball are calculated in several planes: elevation angle (α), azimuth angle (ψ), horizontal angle between ball and home plate (β), ball-heading angle (δ), linear optical trajectory angle (γ).

OAC has been criticized on the grounds that the visual system is relatively insensitive to acceleration (Calderone & Kaiser, 1989; Schmerler, 1976). OAC, however, only requires detecting an increase or decrease in velocity with a threshold sufficient to bring the fielder close enough to the landing point to reach the ball (Tresilian, 1995). Visual judgments by seated observers are equivocal on this point (Babler & Dannemiller, 1993; Brouwer, Brenner, & Smeets, 2002; Oudejans, Michaels, Bakker, & Davids, 1999; Zaal & Michaels, 2003) and may depend on task context or fielder skill level. We thus chose to approach the problem by testing whether the theory can account for actual catching behavior. 

In contrast, linear optical trajectory (LOT) theory (McBeath, Shaffer, & Kaiser, 1995) posits that the fielder runs so as to keep the apparent trajectory of the ball linear (from the fielders point of view), such that it appears to move in a constant direction γ relative to the horizontal. LOT has the virtue of not requiring that ball acceleration be detected; instead, LOT only requires that departures from a straight trajectory (i.e., constant γ) be detected. However, LOT does not produce a unique solution or a well-defined mapping from optics to action but rather constrains the fielders movement to a family of paths; an additional constraint, such as that suggested by OAC to control radial movement, must be applied in order to generate a unique path. If such a constraint were applied, though, OAC and LOT would still differ in that LOT theory predicts that the fielders radial and tangential movements are coupled to maintain a constant γ, implying that any factor affecting one would have an impact on the other. In OAC, radial and tangential movements are independent, which provides a means by which the two theories can be tested. 

Distinguishing the three theories (TP, OAC, and LOT) in the real world is complicated by the fact that they all predict successful catches with the same starting and ending positions, and only small differences between predicted paths (e.g., McLeod, Reed, & Dienes, 2002; Shaffer & McBeath, 2002). One way to dissociate them is to use virtual reality to create physically impossible trajectories by perturbing the ball in flight and testing which relations are held constant (Marken, 2005). Specifically, when we change the trajectory of the ball from parabolic to linear in mid-flight, thereby perturbing its vertical motion, TP, LOT, and OAC make different predictions about the fielders response. TP holds that the balls landing point is predicted from initial conditions, and thus a mid-flight perturbation should not affect the fielders movement. OAC predicts that the fielders radial velocity will be adjusted so that dtanα/dt shifts to a new constant value, whereas the tangential velocity will be unaffected so that the horizontal angle β is unchanged. On the other hand, LOT predicts that the fielders radial and tangential velocities will be coupled to maintain γ, preserving a constant ratio between tanα and tanβ over the perturbation. 

Independently, McLeod, Reed, Gilson, & Glennerster (2008) performed a similar manipulation of the balls trajectory and speed, testing four soccer players heading the ball in a 3.5 m × 3.5 m virtual environment. They reported results consistent with a generalized version of the OAC theory, according to which the player keeps α increasing at a steadily decreasing rate (McLeod et al., 2006), and inconsistent with TP and LOT. However, the balls anterior–posterior and lateral landing positions were varied in separate experiments. In the present study, we vary the landing position in both directions simultaneously in a 12 m × 12 m virtual environment, eliciting large radial and tangential movements of the outfielder, to provide a stronger test of the three theories. 

Methods

Participants

Twelve (8 males, 4 females) experienced college baseball and softball players with normal vision took part in the study. 

Apparatus

Data collection took place in the VENLab, a 12 m × 12 m area in which participants were able to move about freely. Participants wore a Cybermind Visette-Pro HMD (60 deg horizontal × 46.8 deg vertical field of view, 640 × 480 pixel resolution for each eye) covered with a black shield, so the surrounding field was dark and also wore a baseball glove on the left hand. Synoptic images of a virtual environment were generated on an Onyx2 graphics workstation (SGI, Mountain View California) using WorldToolKit software. The virtual world consisted of a Julesz-textured ground plane, blue sky, and a white baseball with red stitching (diameter = 7 cm). Head and glove position (4-mm RMS) and orientation (0.1° RMS) were measured using IS-900 hybrid inertial/ultrasonic trackers (InterSense, Burlington, Massachusetts), and head position and orientation were used to update the view of the environment at 60 Hz. 

Task

Participants were asked to catch virtual fly balls presented in the HMD, while moving freely. Balls started at a distance of 36.22 m and landed at one of four locations relative to the participants initial position (Figure 2): ±4.00 m in the forward–backward direction and ±4.00 m in the left–right direction, similar to real-world conditions tested by McLeod, Reed, & Dienes (2001). The task as seen by a participant wearing the HMD is shown in Movie 1, while an external view of the participant is shown in Movie 2. Following 16 practice trials with normal trajectories, 8 normal trials and 8 perturbed trials were tested at each final position, in random order. Normal fly balls followed a parabolic path, with normal gravitational acceleration (9.8 m/s²) and no air resistance. Perturbed fly balls followed a parabolic trajectory until the apogee (25.00 m), at which point the ball descended on a linear path with a constant vertical velocity (11.11 m/s); this altered its vertical optical acceleration but not its lateral optical motion. 

Figure 2

 

Schematic of the experimental conditions. (a) Top view. Crosses show the landing point of the ball for the four final ball positions, located ±4.00 m in the forward–backward and left–right directions. Axis conventions are shown, with +Z being up. The distances are not to scale. (b) Side view of the trajectory of the ball in normal and perturbed conditions.

 

Movie 1

 

The task as seen by a participant wearing the HMD.

 

Movie 2

 

The task as seen by an external viewer.

Analysis

Since we were interested in the initial phase of catching (moving to the landing position), we defined “catches” as trials in which participants brought either the head or the glove within 1 m of the ball. About half of the fly balls in each condition were registered as catches (forward normal 47%, backward normal 44%, forward perturbed 55%, backward perturbed 52%), and only those trials were included in subsequent analyses. The percentage of catches was lower than might be expected in a comparable real-world situation, possibly due to the encumbrance of the HMD and its limited field of view. Nevertheless, we replicated the basic pattern of paths observed by McLeod et al. (2001), in which the fielders path tends to curve slightly toward home plate when running forward and curve in a symmetrical direction when running backward (see Figure 3 for paths from one participant). 

Figure 3

 

Paths for a representative participant, for normal (blue) and perturbed (red) trajectories. Each trial started at X = 0 and Y = 0, with the final ball positions shown with green circles.

Head and glove positions were filtered with a Butterworth filter (5-Hz cutoff) and time series of the angles α, β, γ, and ψ ( Figure 1) and their derivatives were calculated. Linear regressions were performed on tan α, tan β, ψ, and γ vs. time and tan α vs. tan β for each trial. A mean R ² value across all trials was calculated for each experimental condition for each participant. Because no significant differences between fly balls to the right and left of the participant were observed, these trials were grouped together. To avoid transients at the beginning and end of each trial, only data between 1.00 s and 3.47 s were analyzed for the whole trial, while data between 1.00 s and 2.17 s were analyzed for the first half of the trial and between 2.30 s and 3.47 s for the second half. Comparison of R ² values between the first and second halves of trials, and between normal and perturbed fly balls, was made using a z-test following a Fisher z-transformation. 

Results

Optical acceleration cancellation (OAC)

OAC holds that the rate of change of tan α (elevation angle) is constant on normal trials, implying a linear relationship between tan α and time. We thus computed the linear regression of tan α on time for each trial (see Figure 4 for a representative participant; data for all participants may be seen in the auxiliary files). In the normal conditions, mean R ² values exceeded 0.99 for both backward and forward fly balls ( Table 1). In the perturbed condition, however, the slope changed after the perturbation, yielding significantly lower R ² values in that condition ( z(72) = 8.61, p < 0.0001, for forward and z(72) = 6.63, p < 0.0001, for backward fly balls). Because a shift in the slope of tan α to a new constant value would be expected if the fielder cancelled the new optical acceleration caused by the perturbation, we performed separate regressions for the first and second halves of each trial. The mean R ² values were high on both halves of the trial (>0.99 in most conditions), confirming that slopes were also linear following the perturbation. The exception was the second half of the perturbed backward trials, in which the fit ( R ² = 0.84) was significantly lower ( z(32) = 13.52, p < 0.0001) due to the flat slope. The sum of squared error in this condition, indicative of departure from linearity, however, was not significantly different from the backward normal trials ( t(11) = 1.65, p = 0.1272), which suggests that the linear increase in tan α with time held, even in the perturbed condition. This confirms that the balls optical acceleration was approximately zero both before and after the perturbation, consistent with OAC. 

Figure 4

 

Time series of tan( α) for a representative participant, collapsed across left and right fly balls. The time of perturbation is indicated by a tick mark on the x-axis. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see the Model section).

Table 1

 

Mean R ² values across all participants, for the first half of the trial and second half of the trial separately, and for the entire trial. Trials where participants moved to the left and right were grouped together.

	Forward normal	Backward normal	Forward perturbed	Backward perturbed
tan( α) vs. time
First	0.9994	0.9999	0.9993	0.9998
Second	0.9916	0.9933	0.9924	0.8408* ,**
Entire	0.9931	0.9941	0.5914**	0.8494**
tan( β) vs. tan( α)
First	0.9938	0.9961	0.9939	0.9954
Second	0.8594*	0.9553*	0.8847*	0.8347* ,**
Entire	0.8511	0.9408	0.5217**	0.6874**
tan( β) vs. time
First	0.9938	0.9961	0.9939	0.9954
Second	0.9048*	0.9558*	0.9338* ,**	0.9593*
Entire	0.8618	0.9476	0.8766	0.9485
ψ vs. time
First	0.8873	0.9780	0.9011	0.9634
Second	0.9633*	0.9519*	0.9613*	0.9297*
Entire	0.8657	0.8869	0.8746	0.9016
γ vs. time
First	0.8892	0.9786	0.9001	0.9643
Second	0.9639*	0.9527*	0.9626*	0.9316*
Entire	0.8658	0.8871	0.8750	0.9020

 

Note: *Significant difference ( p < 0.05) between R ² for the first and second halves of a trial. **Significant difference ( p < 0.05) between R ² for normal and perturbed fly balls.

Linear optic trajectory (LOT)

LOT (McBeath et al., 1995) posits that fielders run in such a way as to keep the balls optical trajectory straight (i.e., a constant angle γ) throughout a trial, which is equivalent to maintaining a linear relationship between tanα and tanβ (between the balls elevation angle and horizontal angle to home plate). To test this, we performed a linear regression of tanβ on tanα for each trial (Table 1, Figure 5). In the normal condition, the R² values were high, with mean values of 0.8511 and 0.9408 for forward and backward fly balls, respectively. However, the fits were significantly lower in the perturbed condition, with mean R² values of 0.5217 and 0.6874 for forward and backward fly balls (z = 4.0887, p < 0.0001 and z = 5.417, p < 0.0001, respectively). As before, this might be explained by a shift to a new linear slope following the perturbation, but this account appears unlikely for two reasons. First, separate regressions for the second half of the trial yielded significantly lower R² values than for the first half in all conditions (z ≥ 4.92, p < 0.0001). Second, an analysis of the horizontal angle tanβ over time (Table 1, Figure 6) revealed no differences in slope for the whole trial between normal and perturbed trials (z = −0.36, p = 0.7188, for forward and z = 0.05, p = 0.9601, for backward fly balls). Thus, contrary to LOT, the perturbation affected tanα but not tanβ, indicating that the participants radial and tangential movements are not actually coupled. 

Figure 5

 

Plot of tanα vs. tanβ for the same representative participant as in Figure 4. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see the Discussion section).

Figure 6

 

Time series of tan( β) for a representative participant, collapsed across left and right fly balls. The time of perturbation is indicated by a tick mark on the x-axis. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see the Model section).

Tangential movement

Chapman (1968) originally proposed that, to control tangential movement, the fielder simply matched the lateral position of the ball. This might be implemented by nulling the rate of change (or acceleration) of the horizontal angle between the ball and home plate (tanβ) or of the bearing direction of the ball (azimuth ψ). We tested these strategies by regressing tanβ and ψ on time during the first and second halves of each trial (Figures 6 and 7, Table 1). Mean R² values were high for both variables (>0.866), with significant differences between the first and second halves of the trials in both normal and perturbed conditions (∣z∣ ≥ 2.0297, all p ≤ 0.05); R² increased in the backward conditions for ψ and decreased in the other conditions. The strong fits both before and after the perturbation suggest that either variable could be used to control the fielders tangential movement. 

Figure 7

 

Time series of ψ for a representative participant, collapsed across left and right fly balls. The time of perturbation is indicated by a tick mark on the x-axis. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see the Model section).

Discussion

Our results confirm that participants are able to respond to a physically impossible trajectory and catch perturbed fly balls. Surprisingly, they were equally successful at catching both perturbed and normal fly balls, in a manner consistent with the Optical Acceleration Cancellation theory (Chapman, 1968). Participants reacted to the perturbation by recovering a linear increase in tanα (elevation) over time, with a different slope. This linear relationship was quite strong, with R² values exceeding 0.99 in most conditions, both before and after the perturbation. Moreover, participants maintained this relationship while moving both radially and tangentially. The recovery of linearity after the perturbation is surprising because it requires a large correction in running speed to restore a constant optical ball velocity and a varying running speed to maintain it. This finding provides strong evidence that participants cancel the balls optical acceleration to control their radial movements. 

The fielders mid-flight adjustments are inconsistent with the Trajectory Prediction theory (Saxberg, 1987) that the balls trajectory and landing point are predicted from the first part of the balls trajectory; if TP were used to guide fielders to the correct place to catch the ball, there would be no reason to expect the linear increase in tanα. 

Contrary to the Linear Optic Trajectory theory (McBeath et al., 1995), the perturbation in the balls vertical trajectory did not affect the fielders tangential movement. Although tanα (elevation angle) was changed by the perturbation, tanβ (horizontal angle to home plate) was not affected, and a linear relationship in time for tanβ was observed throughout the entire trial, regardless of the perturbation. The change in tanα but not tanβ resulted in a change of γ (trajectory angle) that cannot be accounted for by LOT. Instead, it appears that radial and tangential movements of the fielders are controlled independently, consistent with OAC. 

If tan α and tan β are independent, how are the fielders tangential movements controlled? We tested two versions of Chapmans (1968) hypothesis that the fielder matches the lateral position of the ball, either by nulling the rate of change in the horizontal angle between the ball and home plate (tanβ) or in the bearing direction of the ball (ψ). The high correlations indicate that either variable could be used to control the fielders tangential movement. This is not surprising because they are closely related, but the bearing direction ψ is more general because “home plate” need not be in view to determine the angle. 

These results converge with those of McLeod et al. (2008), who found that soccer players head the ball in a manner consistent with a generalized version of OAC (McLeod et al., 2006). In their version of the theory, the player keeps α increasing at a steadily decreasing rate, whereas in Chapmans (1968) original version, this is accomplished by maintaining a constant rate of increase in tanα. The results in perturbed fly ball conditions seem somewhat at odds with the generalized version of OAC: after the perturbation, α and tanα actually decrease in our experiment (top of Figure 4), or increase at an increasing rate (bottom two panels of Figure 5 in McLeod et al., 2008). In addition, McLeod et al. (2008) needed to introduce a memory component to explain the tendency for dα/dt to return toward its pre-perturbation trajectory, whereas if tanα is used to guide movement, this tendency is simply a consequence of keeping dtanα/dt constant at its post-perturbation value, without having to invoke a memory component. We therefore believe that keeping dtanα/dt constant provides a better control principle for catching fly balls. 

Model

Our results are consistent with OAC for the fielders radial movements, and a constant bearing strategy for tangential movements. To determine whether this theory could actually generate the observed data, we created a dynamical model in which the fielders radial velocity is controlled by nulling the acceleration of tan( α), while simultaneously and independently controlling tangential velocity by nulling the velocity of tan( ψ) similar to the constant bearing model of target interception (Fajen & Warren, 2007). The model treats the fielder as analogous to a damped spring with a fixed point at d²tanα/dt² = 0 and dtanψ/dt = 0. The equations of motion are given as follows: 

$\begin{array}{l}..v_{\mathrm{r}}=-b_r.v_{\mathrm{r}}-k_{\mathrm{r}}*\min \left({\mathrm{d}}^2\tan \right(\alpha )/\mathrm{d}{t}^2,c_{\mathrm{r}})\\ ..{.}_{\mathrm{t}}=-b_{\mathrm{t}}.v_{\mathrm{t}}-k_{\mathrm{t}}*\min (\mathrm{d}\mathrm{t}\mathrm{a}\mathrm{n}(\psi )/\mathrm{d}t,c_{\mathrm{t}})\end{array},$

(1)

where b_r and b_t are damping terms, k_r and k_t are stiffness terms, and c_r and c_t are thresholds that approximate a sigmoidal response to the optical variables. 

Model predictions were simulated using Euler integration with a step size of 1/60 of a second, using the actual ball positions from the experimental data for the ball trajectory. The fielder was assumed to not move for the first 0.5 s of the trial to simulate the delay between the balls launch and the fielders first step. Parameters were fit separately for each participant using NOMAD, a generalized search program using MATLAB script (Abramson, Audet, & Dennis, 2004). This routine minimized the squared error between the x- and z-positions of the model and the data, summed over all time points in all catching trials. The mean R² value was 0.5994 for the x-position and 0.4174 for the z-position (the best possible values were 0.8068 and 0.6836, respectively, determined by calculating R² values between the mean trajectory of each participant and all trials). Fits of the model are shown in Figures 4, 5, 6, and 7 with thick magenta and cyan lines for the normal and perturbed conditions respectively, using the parameters derived for the representative participant (b_r = 19.38 1/s, b_t = 0.19 1/s, k_r = 116.48 m/s, k_t = 223.52 m/s², c_r = 4 rad/s², c_t = 4 rad/s). The model thus approximates the fielders movements when catching a fly ball by directly coupling optical variables to action. 

Conclusion

In sum, the findings demonstrate that OAC can account for the fielders radial movements, even following large perturbations to the vertical motion of the ball. In contrast, the perturbations had no effect on the fielders tangential movements, refuting the key prediction of LOT that they should be linked. Finally, the rapid responses to mid-flight perturbations show that the fielders movements are continuously controlled, contrary to the standard TP theory. The results thus suggest that perception is used to guide action by means of a continuous coupling of visual information to movement, without requiring an internal model of the balls trajectory. 

Supplementary Materials

Supplementary Figure S1a - Supplementary Figure S1a 

Supplementary Figure S1b - Supplementary Figure S1b 

Supplementary Figure S1c - Supplementary Figure S1c 

Supplementary Figure S1d - Supplementary Figure S1d 

Supplementary Figure S1e - Supplementary Figure S1e 

Supplementary Figure S1f - Supplementary Figure S1f 

Supplementary Figure S1g - Supplementary Figure S1g 

Supplementary Figure S1h - Supplementary Figure S1h 

Supplementary Figure S1i - Supplementary Figure S1i 

Supplementary Figure S1j - Supplementary Figure S1j 

Supplementary Figure S1k - Supplementary Figure S1k 

Acknowledgments

This research was supported by a grant from the U.S. National Institutes of Health (R01EY010923). The work was previously reported at the annual meetings of the Psychonomic Society (2003) and the Vision Science Society (2007). 

Commercial relationships: none. 

Corresponding author: Philip Fink. 

Email: [email protected]. 

Address: Institute of Food, Nutrition, and Human Health, Massey University, Palmerston North 4442, New Zealand.