Influence of optic-flow information beyond the velocity field on the active control of heading

Abstract

We examined both the sufficiency of the optic-flow velocity field and the influence of optic-flow information beyond the velocity field on the active control of heading. The display simulated a vehicle traveling on a circular path through a random-dot 3D cloud under a static or a dynamic scene in which dots were periodically redrawn to remove information beyond the velocity field. Participants used a joystick, under either velocity and acceleration control dynamics, to steer and align the vehicle orientation with their perceived heading while experiencing random perturbations to the vehicle orientation. Frequency–response (Bode) plots show reasonably good performance under both display conditions with a decrease in gain and an increase in phase lag for the dynamic scene for both control dynamics. The performance data were fit by a Crossover Model to identify reaction time and lead time constant to determine how much participants anticipated future heading to generate lead control. Reaction time was longer and lead time constant was smaller for the dynamic than the static scene for both control dynamics. We conclude that the velocity field alone is sufficient to support closed-loop heading control, but optic-flow information beyond the velocity field improves visuomotor performance in self-motion control.

Introduction

The ability to perceive and control self-motion to move around in the world is essential for human survival. Ever since Gibson (1950) proposed that humans use the visual image motion of the environment on the retina generated during locomotion (optic flow) to perceive and control self-motion, much research in cognitive psychology and neuroscience has investigated the specific cues from optic flow people use for estimation of self-motion. When traveling on a straight path with no eye, head, or body rotation (pure translation), the focus of expansion (FOE) in the resulting radially expanding retinal flow pattern indicates one's instantaneous direction of self-motion (heading) as well as one's future linear trajectory (path). Under more complex but natural conditions such as when traveling on a curved path or traveling on a straight path with eye, head, or body rotation (translation and rotation), the process of extracting heading and path from optic flow becomes complicated as the rotation disrupts the radial pattern and shifts the FOE in retinal flow away from the heading direction (Regan & Beverly, 1982). Nevertheless, it has been shown mathematically that one can still use information (such as motion parallax) from a single 2D instantaneous velocity field of optic flow to compensate for the rotation in the flow field and recover heading (e.g., Heeger & Jepson, 1990; Longuet-Higgins & Prazdny, 1980), a computation that has been implemented with neurophysiological models of primate extrastriate visual cortex (e.g., Lappe & Rauscheker, 1993; Perrone & Stone, 1994; Royden, 1997; Zemel & Sejnowski, 1998). 

Psychophysical studies have shown that observers can estimate their heading within 1° of visual angle during pure translation (e.g., Warren, Morris, & Kalish, 1988). However, good heading performance during pure translation can be easily achieved by locating the FOE in the 2D flow field without any 3D interpretation of the scene. A better measure of human capability of heading perception is to determine whether humans can recover heading from the combined translational and rotational retinal flow. To address this issue, a number of studies examined heading perception during translation with simulated eye movements using displays generated in such way that the retinal image of the display on a stationary eye was the same as if the eye had moved. While some studies reported poor self-motion estimation from optic flow at high rotation rates (e.g., Banks, Ehrlich, Backus, & Crowell, 1996; Royden, Banks, & Crowell, 1992), several other studies have found that visual cues separate from optic flow such as static or stereoscopic depth cues (van den Berg, 1992; van den Berg & Brenner, 1994a, 1994b), or a cluttered environment with reference objects (Cutting, Vishton, Flückiger, Baumberger, & Gerndt, 1997) can be important for robust self-motion perception during translation and rotation. Note that, among these studies, some used a path, not a heading, judgment task in which participants were asked to judge their perceived future trajectory of locomotion with respect to an environmental reference point (e.g., Cutting et al., 1997; van den Berg, 1996; Warren, Blackwell, Kurtz, Hatsopoulos, & Kalish, 1991). Although the extrapolated future path trajectory and heading are in the same direction when one is traveling on a straight path, they diverge when one is traveling on a curved path as heading becomes the tangent to the curving path trajectory at each moment in time (Li, Chen, & Peng, 2009; Stone & Perrone, 1997). 

To determine whether people can perceive heading during translation and rotation using information from the instantaneous velocity field of optic flow, Stone and Perrone (1997) presented observers with displays consisting of randomly distributed dots that simulated their traveling on a circular path. Observers were asked to judge their heading relative to their virtual line of sight in the scene. Although observers displayed reasonably good heading performance, this finding does not fully support the claim that humans can accurately perceive heading using information from a single velocity field as shown by the mathematical and neurophysiological models cited above. Observers could have used information from the temporally integrated optic-flow field such as the extended trajectories of the dot motions (flow lines, e.g., Kim & Turvey, 1999; Wann & Swapp, 2000) and/or higher order derivatives of optic flow such as dot acceleration (e.g., Rieger, 1983; Royden, 1994) to perceive their path of forward travel, then inferred heading as the tangent to the path. To remove this possibility, Li, Sweet, and Stone (2006b) examined heading perception for travel along a circular path using a dynamic optic-flow display in which environmental dots were periodically redrawn to provide a sequence of velocity fields while removing flow lines, dot acceleration, and other higher order optic-flow cues. They found that with the dynamic display, observers could still accurately perceive heading within 2° of visual angle, thus lending support to the idea that heading is directly available from the optic-flow velocity field for the online control of self-motion. On the other hand, Li et al. (2009) found that when the display did not contain sufficient motion parallax information for observers to remove rotation in the flow field for accurate heading perception, information beyond the velocity field, such as flow lines and/or dot acceleration information, helped improve heading estimation accuracy. Furthermore, Niehorster, Cheng, and Li (2010) also showed that when motion-streak-like form information from the temporally integrated optic-flow field was put in conflict with motion information in the flow field, observers' heading judgment was largely biased toward the FOE direction implied by motion streaks, indicating that optic-flow information beyond the velocity field can affect heading perception from optic flow. 

All the above research studies on human heading perception used a passive judgment task. The extent to which limitations of human heading perception manifest themselves in active control of self-motion has not been determined. In particular, to the best of our knowledge, no study has examined whether humans can control heading during translation and rotation using information from the instantaneous velocity field of optic flow and how optic-flow information beyond the velocity field such as flow lines and/or dot acceleration information affect such active control performance. Active control of self-motion is a closed-loop negative feedback control task and is thus more complicated than simple passive self-motion judgment. The most straightforward way to perform such a task would be to perceive one's current heading and then continuously compare it with the desired direction, making constant adjustments in response to discrepancies between the two. Although it has been clearly established that we can perceive heading from a sequence of velocity fields (Li et al., 2006b), given a demanding active control task, it remains an open question whether optic-flow information beyond the velocity field is necessary for or contribute to the online control of heading, and if so, how they contribute (see Land, 1998). Knowledge of the differential contributions of visual information from and beyond the instantaneous velocity field to active control of heading is critical for vision scientists to understand how the brain processes optic-flow information and for design engineers to optimize the displays and interfaces used by human operators to control self-motion. 

In the current study, to answer the above question, we examined active control of heading during translation and rotation in a feedback-driven closed-loop task. The display simulated an observer steering a vehicle that was traveling on a circular path while facing crosswind perturbations to the vehicle yaw orientation (i.e., the observer virtual gaze direction through the windshield). As in the study by Li et al. (2006b), we tested two display conditions: (a) the static scene in which a 3D cloud composed of 150 random dots was presented, and the dots in the cloud were displayed until they left the field of view, and (b) the dynamic scene in which the lifetime of the 150 random dots in the 3D could was limited to 100 ms (i.e., 6 frames at 60 Hz) to match the integration time of human local motion processing (Burr, 1981; Watson & Turano, 1995). In the case of the static scene, the display provided not only information from the instantaneous velocity field but also visual cues beyond the velocity field such as flow lines and dot acceleration information. In contrast, for the dynamic scene, due to the fact that the dot lifetime was chosen to be as short as possible without degrading motion perception per se, ¹ the display provided a sequence of velocity fields but no behaviorally useful flow lines or dot acceleration information (Calderone & Kaiser, 1989; Snowden & Braddick, 1991; Stone & Ersheid, 2006). ² In each trial, participants were asked to use the joystick to rotate the vehicle orientation (i.e., their virtual gaze direction) to align it with the vehicle's instantaneous direction of travel (i.e., their heading, see Figure 1). In the screen coordinates, this task was equivalent to rotating the misaligned heading direction due to the vehicle orientation perturbation to align it with the center of the screen. Note that the vehicle was constantly traveling on a curved path, thus there was no FOE in the flow field indicating heading. There were, however, many pseudo-FOEs (i.e., the center of a 2D expansion pattern) in the range of 3°–30.3° in the direction of the path curvature for dots spreading the depth range of 6–50 m, which would bias heading control performance to generate inward heading errors if observers were unable to perceive their actual heading from the flow field. 

Figure 1

 

Illustration of the experimental paradigm. A bird's-eye view of the vehicle traveling clockwise showing four time points when the vehicle yaw orientation was randomly perturbed and adjusted by the joystick movement.

Compared with passive perceptual heading judgment, the active control task in the current study is more complex. In passive heading perception judgment, observers need to only indicate their heading at the end of the trial. In the closed-loop active control of heading, however, observers need to continuously respond to the input heading error to adjust the joystick to keep their heading centered on the screen. Given that the controller dynamics influences the effect of control actions on visual cues in the display that observers rely on for the closed-loop control, different control dynamics have been shown to affect the visual cues that people use for the control of object and self-motion (e.g., Fajen, 2008; Li, Sweet, & Stone, 2005, 2006a; Loomis & Beall, 1998). Accordingly, in the current study, we tested two types of joystick control regimes to examine whether observers' reliance on optic-flow cues in the two display conditions changed with control dynamics. In the velocity control condition, the joystick displacement generated a command proportional to the rate of change of the vehicle yaw orientation; in the acceleration control condition, the joystick displacement generated a command proportional to the rate of change of the vehicle yaw rotational velocity. Velocity control is similar to the control of an automobile in which the steering wheel displacement is proportional to the vehicle rotation rate. Acceleration control, such as the control of a spacecraft, is more difficult and less commonly experienced but can still be mastered with practice (Jagacinski & Flach, 2003). 

Previous studies on manual control have shown that human control behavior also changes as the plant control dynamics varies to maintain overall system stability. Specifically, for velocity control dynamics, humans behave like a simple gain controller with a time delay, i.e., they output a scaled, time-delayed version of the input. For acceleration control dynamics, humans behave more like a differentiator, i.e., they attempt to anticipate the input error signal and generate lead control resulting in higher gain at higher frequencies and positively shifted phases at low frequencies (McRuer, Graham, Krendel, & Reisener, 1965; McRuer & Krendel, 1974; see a review in Jagacinski & Flach, 2003; Wickens, 1986). To quantitatively evaluate the effects of optic-flow cues in the two display conditions on active control of heading and to determine the extent to which control behavior was differentially affected by optic-flow cues for velocity and acceleration control dynamics, we fit participants' heading control performance data using a Crossover Model (Li et al., 2005, 2006a; McRuer et al., 1965; McRuer & Krendel, 1974), tailored to assess the response time delay (i.e., reaction time) and the lead time constant indicating how much participants anticipated the input heading error to generate lead control. This analysis enabled us to quantify the amount of predictive/anticipatory visual information about future heading used for heading control under the two control regimes. 

Methods

Participants

Seven staff members and students (four females and three males; four naive to the purpose of the experiment) between the ages of 20 and 38 at the University of Hong Kong participated in the experiment. All had normal or corrected-to-normal vision. 

Visual stimuli and control

The display simulated a participant steering a vehicle that was traveling on a circular path through a 3D cloud (T = 8 m/s, R = ±4°/s; path curvature = ±0.009 m⁻¹, negative value indicates leftward curvature and positive value indicates rightward curvature). The path curvature was well above the threshold for observers to detect traveling on a circular rather than a straight path (Turano & Wang, 1994). During a trial, the vehicle yaw orientation was perturbed by the sum of seven harmonically independent sinusoids, and participants were asked to move a joystick (B&G Systems, JF3) leftward or rightward to align the vehicle orientation (and thus their virtual gaze direction) with its instantaneous direction of travel (Figure 1). In the screen coordinates, this task was equivalent to participants rotating their misaligned heading due to the vehicle rotation to align it with the center of the screen. A simplified block diagram of the whole control system is shown in Figure 2. 

Figure 2

 

Simplified block diagram of the closed-loop heading control task. Human operator transfer function (Y _p) captures the participant's control compensation, and the controller dynamics (Y _c) specifies the joystick control dynamics.

The input perturbation (I) to the vehicle yaw orientation had the following form as a function of time (t): 

$I\left(t\right)=D\sum _{i=1}^7{a}_i\sin (\frac{2\pi k_i}{90}t+{\rho }_i).$

(1)

Table 1 lists the values of a, k, and resulting frequencies (ω _i = k _i /90 Hz). D was set to a value of 2.86°. The phase offset of each sine component (ρ _i ) was randomly varied from −π to π. The average rotation rate of the uncorrected input perturbation to the vehicle orientation was 8.38°/s (peak: 32.01°/s). The use of harmonically independent sum of sines made the vehicle orientation perturbation appear random and allowed for a frequency-based analysis of the linear component of the control response (Li et al., 2005, 2006a; McRuer & Krendel, 1959; Stark, Iida, & Willis, 1961). 

Table 1

 

Magnitudes and frequencies of the seven harmonically independent sinusoids in the input perturbations to the vehicle yaw orientation.

i	a i	k i	ω i (Hz)
1	2	9	0.1
2	2	13	0.14
3	2	22	0.24
4	0.2	37	0.41
5	0.2	67	0.74
6	0.2	115	1.28
7	0.2	197	2.19

Two types of joystick control dynamics were tested: the joystick displacement was proportional either to the rate of change of the vehicle yaw orientation (velocity control) or to the rate of change of the vehicle yaw rotation rate (acceleration control). The control dynamics of the joystick, specified by the controller dynamics (Y _c) in Figure 2, was implemented as 

$Y_{\mathrm{c}}=\frac{1}{s},$

(2)

for velocity control and 

$Y_{\mathrm{c}}=\frac{1}{s(s+0.2)},$

(3)

for acceleration control, where s is the Laplace transform variable. We added a damping factor of 0.2s to acceleration control to reduce the task difficulty, thus our acceleration control is not a perfect acceleration control system of 1/s ². The joystick position was sampled at 60 Hz. The system feedback delay was, hence, 1 frame or 16.67 ms, which is a small fraction of human reaction time. The joystick displacement value ranged from −1 to 1, corresponding to a peak yaw rotation rate of 17.2°/s for velocity control dynamics and a peak yaw acceleration rate of 16.6°/s² for acceleration control dynamics. 

The 3D cloud was composed of 150 white dots (0.4° in diameter, luminance contrast of +99% over a background luminance of 0.1 cd/m²). The dots were generated within a pyramidal frustum subtending the size of the field of view (109°H × 94°V) in the depth range of 6–50 m. The frustum moved with the vehicle orientation controlled by the joystick displacement. The dots were placed in the frustum such that about the same number of dots at each distance in depth was displayed on each frame. This was to ensure that the nearby parts of the frustum contained enough dots, thus the display contained sufficient amount of motion parallax information for accurate heading perception during rotation (Li et al., 2009, 2006b). The number of visible dots per frame was kept relatively constant throughout the trial, i.e., if a certain number of dots moved outside of the frustum in one frame, the same number of dots was regenerated in the frustum with an algorithm to keep the same dot distribution in depth. 

As in the study by Li et al. (2006b), two display conditions were tested: (a) the static 3D cloud scene in which dots were displayed until they left the field of view, thus providing not only a sequence of velocity fields but also optic-flow information beyond the velocity field such as flow lines and dot acceleration information (Movie 1), and (b) the dynamic 3D cloud scene in which dots were set to the limited lifetime of 100 ms (i.e., 6 frames at 60 Hz) to match the integration time of human local motion processing (Burr, 1981; Watson & Turano, 1995). The initial dot lifetime was randomly set from 1 to 6 frames, thus about 25 (1/6 of 150) dots were redrawn per frame. This display provided only a sequence of velocity fields but no useful temporally integrated flow lines or dot acceleration information (Movie 2). Li et al. (2006b) examined human heading perception using the same dynamic and static scene displays and found that both the accuracy and the precision of passive heading judgment for the dynamic scene display (1.8° and 4.2°) were comparable to those for the static scene display (2.1° and 4.1°). Li and Cheng (2011) recently examined human path perception from optic flow using a dynamic random-dot-ground display in which the lifetime of dots on the ground was also limited to 100 ms. Again, they found that path performance was equally accurate and precise for the dynamic and static scene displays. Thus, despite the reduced dot lifetime or spurious motion noise from the 25 dots regenerated per frame in the dynamic scene display, the quality of the local motion estimates in the two display conditions appears to be indistinguishable (see also Footnote 1). 

 

Movie 1

 

A schematic video of the static 3D cloud scene display with the uncorrected input vehicle orientation perturbation. The dot size is optimized for onscreen viewing.

 

Movie 2

 

A schematic video of the dynamic 3D cloud scene display with the uncorrected input vehicle orientation perturbation. The dot size is optimized for onscreen viewing. The dot lifetime approximates 100 ms due to the limitation of the onscreen video recording software.

The visual stimuli were generated on a Dell Precision Workstation 670n with an NVIDIA Quadro FX 1800 graphics card at the frame rate of 60 Hz. They were rear-projected on a large screen (109°H × 94°V) with an Epson EMP-9300 LCD projector (native resolution: 1400 × 1050 pixels, refresh rate: 60 Hz) in a light-excluded viewing environment. The screen edges were covered in matte black cloth to minimize the availability of an artificial frame of reference. To minimize any potential 3D motion conflict with disparity information of the flat screen, participants viewed the visual stimuli monocularly with their dominant eye from a chin rest. The simulated eye height in the display was at 1.51 m corresponding to the average eye height of participants sitting on a high chair at 0.56 m away from the screen. 

Procedure

Participants pulled the trigger of the joystick to start each trial. They were instructed to imagine looking through the windshield of a vehicle that was traveling on a circular path while facing crosswind perturbation to the vehicle yaw orientation (i.e., their virtual gaze direction through the windshield). The vehicle initially rotated leftward or rightward according to the sum-of-sines perturbation input, but this rotation was reduced as participants moved the joystick leftward and rightward to control the vehicle orientation (i.e., their virtual gaze direction) to keep it aligned with their instantaneous direction of travel (i.e., to look straight in the heading direction). The duration of each trial was 95 s. 

Each participant ran two experimental sessions. Each session consisted of 12 trials (3 trials × 2 path curvatures × 2 display conditions) for each control dynamics and typically lasted 30 min. Trials were randomized by left and right path curvatures and were blocked by display condition. The testing order of display condition and control dynamics were both counterbalanced. To ensure participants understood the task and became familiar with the joystick control dynamics, they received practice trials with the static scene displays before the commencement of the experiment. For the practice trials, the dots in the 3D cloud were placed in the depth range of 2–50 m. Participants were informed that by pulling the joystick trigger again, they could view a blue tunnel indicating the circular path that the vehicle was traveling on during practice. Different motion parameters were used for the practice trials (i.e., we randomly varied the translation and rotation rates used for the experiment by 10–15%) to prevent participants from estimating heading using memorized 2D motion cues. No feedback was shown during the actual experiment. The practice continued until their performance stabilized, which, on average, required 12 trials for each control dynamics. In total, the experiment lasted less than 3 h, and participants completed the experiment within a couple of days. 

Data analysis

Time series of heading error, defined as the angle between the vehicle orientation and its heading, the joystick control output, and the input vehicle orientation perturbations were recorded. We analyzed the data beginning 5 s after the start of a trial to ensure that we skipped the initial transient response. To examine different performance aspects revealing the influence of display condition and control dynamics, we took several metrics of the control performance. Total performance error was measured as the average of the time series of the recorded heading error (reflecting overall control accuracy) and the root mean square (RMS) of the time series of the heading error (reflecting overall control precision). To examine the participant's control response specific to the input perturbation frequencies, we performed frequency (Bode) analyses to describe the human operator transfer function (Y _p in Figure 2) from the control performance data. That is, we Fourier transformed both the joystick displacement (in % of maximum displacement) and the heading error (in degree of visual angle) time series to obtain the relevant control response amplitudes and phases. We took the ratios of the amplitudes to compute the gain (in % of max/deg) and the difference between the phases to compute phase lag at each perturbation frequency for each display type. To examine the effects of display condition and control dynamics on the control performance, we then conducted a repeated-measures ANOVA on each of the above performance metrics. 

Modeling

Our closed-loop active control task allowed participants to use the visual feedback of heading error to continuously adjust the joystick to minimize the deviation angle between the vehicle orientation (i.e., their virtual gaze direction) and the vehicle's instantaneous direction of travel (i.e., heading). To perform the task and keep the system stable, participants needed to predict/anticipate future heading error and generate control responses ahead of the input error signal (lead control) as well as to respond to the current heading error, which would lead them to generate control responses lagging the input error signal (lag control). To quantitatively describe participants' overall lead versus lag control behavior in the current closed-loop heading control task and to examine how the anticipatory control behavior varied with display condition and control dynamics, we modeled participants' control performance, captured by the human operator transfer function (Y _p in Figure 2 and Equation 4 below), using a Crossover Model. 

The Crossover Model is a simple linear dynamic model that can describe human control response for a wide variety of closed-loop manual control tasks (McRuer et al., 1965; McRuer & Krendel, 1974). We have previously applied a modified version of the Crossover Model (Sweet, Kaiser, & Davis, 2003) to study the visual cues underlying the active control of a 2D moving line (Li et al., 2005, 2006a). In the model, the human operator transfer function (Y _p) is given by 

$Y_{\mathrm{p}}=\frac{K_{\mathrm{p}}{e}^{-s\tau }(s{T}_{\mathrm{L}}+1)}{s^2/{\omega }_n^2+2s{\zeta }_n/{\omega }_n+1},$

(4)

where K _p represents the overall gain in the control compensation, τ represents the sum of perceptual and neuromotor delays that specify the participant's reaction time, T _L represents a lead time constant indicating the extent to which the participant uses visual cues in the display to anticipate heading error and generate lead control, ω _n and ζ _n represent the fixed second-order response dynamics of the participant independent of the visual stimulus, and s is the Laplace transform variable. 

Model parameters were determined by a best fit to the performance data describing the human operator transfer function (Y _p) with a weighted (by standard error) least-squares procedure (i.e., χ ² fit, see Sweet et al., 2003 for details). We fixed ω _n and ζ _n across the two display conditions and control dynamics such that there were in total 14 parameter values to fit 56 data points for each participant. For all seven participants, the Pearson correlation coefficients between the model estimates and the performance data ranged from 0.96 to 0.998, indicating that between 93% and 99.6% of the variance in the control performance data can be accounted for by the Crossover Model. The reduced χ ² for the model estimates ranged from 0.2 to 0.86 across participants, indicating that although the simple linear Crossover Model provided a good fit to the data, it did not fully account for all aspects of performance. Indeed, it cannot account for any non-linearities or non-stationary characteristics in the participant's control response. 

Results

Overall performance

Before considering the effect of display condition, we first present sample control response data from the static scene display condition. Figure 3 shows the joystick displacement as a function of time from a representative trial for velocity and acceleration control dynamics, respectively. Note that while for velocity control, the joystick output was simply a low-pass-filtered and delayed version of the input heading error signal, for acceleration control, the joystick output had a tendency to lead the input heading error signal especially for slower perturbations. This is consistent with the fact that to maintain overall system stability, humans behave like a simple gain controller with a time delay for velocity control plant dynamics and behave like a differentiator resulting in phase lead at low frequencies for acceleration control plant dynamics (McRuer et al., 1965; McRuer & Krendel, 1974). The pattern of data for the three experienced participants was similar to that of the four naive ones, so the data from all seven participants were analyzed together. 

Figure 3

 

Typical raw performance data of the input heading error (solid line) and the output joystick displacement (dashed line) for (a) velocity control and (b) acceleration control. Note that while for velocity control, the joystick output was a low-pass-filtered and delayed version of the input heading error signal, for acceleration control, the joystick output appeared to lead the input heading error signal especially at low frequencies.

Given symmetrical control performance for left and right path curvatures, we collapsed the heading error data across left and right path curvatures such that positive heading errors were associated with vehicle orientations inside of the path (inward errors or oversteering), and negative heading errors were associated with vehicle orientations outside of the path (outward errors or understeering). The heading bias (overall control accuracy), measured as the mean signed heading error averaged across six trials, was plotted against display condition for each participant in Figure 4. All participants showed a negative heading bias of understeering for the two display conditions with a larger bias for the static scene condition. A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on heading bias revealed that both the main effects of display condition and control dynamics were significant, with F(1,6) = 7.94, p < 0.05 and F(1,6) = 24.58, p < 0.01, respectively. The interaction effect of display condition and control dynamics was not significant. The mean unsigned heading bias for the dynamic scene condition (4.2°) was significantly smaller than that for the static scene condition (6.0°), and that for velocity control (4.0°) was significantly smaller than that for acceleration control (6.1°), consistent with the greater difficulty of acceleration control. Compared with perceptual heading judgment data at a similar path curvature from the study by Li et al. (2006b), the magnitude of the overall heading bias observed in the current study was somewhat larger. However, the observed heading bias was not inconsistent with reported heading thresholds for heading judgment with oscillating optic-flow stimuli (Cutting, Springer, Braren, & Johnson, 1992). Furthermore, participants clearly did not try to align the vehicle orientation toward the pseudo-FOEs in the flow field, which would have resulted in positive heading biases (>3°, see Li et al., 2009, 2006b; Stone & Perrone, 1997). Instead, participants tended to be biased away from the pseudo-FOEs when controlling vehicle orientation. As, in the screen coordinates, the heading control task was equivalent to participants rotating their misaligned heading due to the vehicle orientation perturbation to align it with the center of the screen, the observed outward/understeering heading bias is consistent with an underestimation of the vehicle rotation and is not comparable with the center bias in heading judgment reported by previous heading studies (e.g., Crowell & Banks, 1993). 

Figure 4

 

Heading bias (i.e., mean heading error averaged across six trials) against display condition for each participant for (a) velocity control and (b) acceleration control. Error bars are SEs across seven observers; * indicates the statistical significance from a paired t-test (p < 0.05).

Figure 5 plots the RMS heading error (overall control precision) averaged across six trials against display condition for each participant. A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on the RMS heading errors revealed that only the main effect of control dynamics was significant, with F(1.6) = 115.33, p < 0.0001. The mean RMS error for velocity control (8.5°) was significantly less than that for acceleration control (14.3°), again consistent with the latter's greater difficulty. 

Figure 5

 

Root mean square (RMS) heading error averaged across six trials against display condition for each participant for (a) velocity control and (b) acceleration control. Error bars are SEs across seven observers.

In summary, the above results show that the overall heading control was biased slightly outside of the path, with a smaller bias for the dynamic than the static scene display condition. That is, it appeared that participants understeered the vehicle, and the understeering was greater for the static than the dynamic scene displays. However, the overall heading control precision was similar for the two display conditions. Despite the systematic error (bias) and the random error (RMS error), participants were quite responsive to the induced vehicle rotations and were able, under closed loop, to control their heading reasonably well. Across the two control dynamics, the vehicle rotation command magnitude averaged across participants (mean rotation rate ± SE: 9.8°/s ± 0.6°/s) was close to the uncorrected average vehicle rotation rate of 8.4°/s, indicating that participants were generating a control response with a magnitude appropriately scaled to nullify the overall induced perturbations. Our results are thus generally in agreement with previous findings showing that humans are able to perceive their heading from a dynamic scene display (Li et al., 2006b) and indicate that heading is directly available from the instantaneous velocity field for the effective online control of self-motion. 

The heading bias and the RMS heading error measure the total performance error, both visually and non-visually driven. Performance specific to the perturbation frequencies provides a better measure of the visually driven component of the control responses to the input heading error signal. Below, we therefore show how participants' control behavior changed with the two display conditions at each input vehicle orientation perturbation frequency. Fourier analysis allowed us to segregate response amplitude and delay effects. 

Frequency–response performance

In order to analyze how well participants performed at each of the input perturbation frequencies, we computed the human operator transfer function Y _p (i.e., the ratio of the Fourier transform of the output joystick displacement to that of the input heading error, see Figure 2). Figure 6 plots the gain and phase of the computed human operator transfer function, averaged across six trials and seven participants, as a function of input vehicle orientation perturbation frequency for the two display conditions and control dynamics. The frequency–response (Bode) plots show that, for both velocity and acceleration control dynamics, gain increases with frequency with little phase lag at low frequencies (<0.41 Hz). In fact, for acceleration control, gain increases faster and the joystick response leads the input (positive phase) at low frequencies. This is consistent with the previous findings showing that human operators tend to rely more on velocity information for acceleration control, which allows the operator to anticipate the input error signal to generate lead control (McRuer et al., 1965; McRuer & Krendel, 1974; see a review in Jagacinski & Flach, 2003; Wickens, 1986). For both types of control dynamics, both gain and phase roll off progressively at high frequencies. In addition, gain is larger and phase lag is smaller for the static than the dynamic scene display condition, especially in the high-frequency range. 

Figure 6

 

Frequency–response (Bode) plots for the human operator transfer function (Y _p). The top panels depict mean gain and the bottom panels depict mean phase averaged over seven participants as a function of perturbation frequency for (a) velocity control and (b) acceleration control. The rightmost points in the top panels plot the mean gain averaged across the seven frequencies, and the rightmost points in the bottom panels plot the mean phase averaged across the seven frequencies. Error bars represent SEs across participants (some of them are smaller than the data symbols). Solid curves represent the best-fitting simulations of the Crossover Model, with T _L = 0.74 s, τ = 359 ms, and K _p = 3.43% max/deg for the static scene condition in (a); T _L = 0.69 s, τ = 399 ms, and K _p = 2.6% max/deg for the dynamic scene condition in (a); T _L = 1.62 s, τ = 365 ms, and K _p = 2.20% max/deg for the static scene condition in (b); and T _L = 0.99 s, τ = 410 ms, and K _p = 2.31% max/deg for the dynamic scene condition in (b).

To quantify the change of gain with display condition, we plotted the gain averaged across all frequencies for the two display conditions and the two control dynamics indicated by the rightmost points in the top panels of Figure 6. A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on the gain averaged across frequencies revealed that only the main effect of display condition was significant, with F(1,6) = 68.14, p < 0.001. Overall, the mean gain for the static scene condition was 50% larger than that for the dynamic scene condition (13.6 dB vs. 10.2 dB), indicating better performance for the static than the dynamic scene condition. The decrease in gain for the dynamic scene condition averaged across control dynamics was plotted against perturbation frequency in Figure 7a. A one-way repeated-measures ANOVA showed that the effect of frequency was significant, with F(6,36) = 4.88, p < 0.001. The decrease in gain for the dynamic scene condition was larger at higher frequencies (≥0.41 Hz). 

Figure 7

 

(a) Mean gain decrease from the static to the dynamic scene condition against perturbation frequency and (b) mean relative response delay against perturbation frequency. Error bars represent SEs across seven participants.

To quantify the effect of display condition on response phase, we plotted the phase averaged across all frequencies for the two display conditions and the two control dynamics indicated by the rightmost points in the bottom panels of Figure 6. A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on the phase averaged across frequencies revealed that both the main effects of display condition and control dynamics were significant, with F(1,6) = 8.04, p < 0.05 and F(1,6) = 7.24, p < 0.05, respectively. The mean response phase lag for the dynamic scene condition (124.2°) was larger than that for the static scene condition (112.2°). In addition, consistent with acceleration control dynamics that requires participants to generate more lead control to maintain the system stability, the mean response phase lag (113.9°) for acceleration control was smaller than that for velocity control (122.5°). Then, for each participant, we converted response phase lag to relative response delay by using the response phase of the static scene condition as the reference and subtracting the phase of the dynamic scene condition from that of the static scene condition, then dividing the difference by the corresponding frequency multiplied by 360°. As the interaction effect of display condition and control dynamics was not significant, the relative response delay averaged across control dynamics was plotted against perturbation frequency in Figure 7b. Although the phases at different frequencies are quite different as can be seen from Figure 6, they nonetheless correspond to similar relative response delays. A one-way repeated-measures ANOVA on relative response delays showed that the effect of frequency was not significant (F(6,36) = 0.52, p = 0.79). The overall relative response delay averaged across the seven frequencies was 48.1 ms. 

Modeling

The Crossover Model allows us to perform a quantitative evaluation of the effects of optic-flow cues in the two display conditions on participants' lead versus lag control behavior in the closed-loop heading control task. The lead time constant T _L and the time delay τ capture the visual-stimulus-dependent control characteristics of participants' control performance, and the overall gain K _p captures participants' overall sensitivity to the input heading error. Figure 8 shows the frequency–response plots illustrating how varying T _L, τ, and K _p in the Crossover Model affects the gain and phase of the human operator transfer function Y _p. Specifically, increasing T _L while keeping τ and K _p constant causes gain in the high-frequency range to increase (Figure 8a); increasing τ while keeping T _L and K _p constant causes phase lag in the high-frequency range to increase (Figure 8b); and increasing K _p while keeping T _L and τ constant causes an overall increase in gain (Figure 8c). The solid and dashed curves in Figure 6 show that the model did a good job of describing the observed average control performance. 

Figure 8

 

Frequency response (Bode) plots depicting effects of varying (a) T _L, (b) τ, and (c) K _p in the Crossover Model of the human operator transfer function (Y _p).

By fitting the model to the individual participant performance data, we estimated T _L, τ, and K _p in the Crossover Model of the human operator transfer function Y _p (Figure 2 and Equation 4) for the two display conditions and the two control dynamics for each participant. Figure 9 plots the fitted parameters against display condition for the two control dynamics for each participant. A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on T _L revealed that both main effects of display condition and control dynamics were significant, with F(1,6) = 8.73, p < 0.05 and F(1,6) = 19.99, p < 0.01, respectively. The interaction effect of display condition and control dynamics was not significant. As expected, T _L was significantly larger for acceleration control (1.31 s) than velocity control (0.72 s), consistent with the former requiring observers to rely more on heading movement information to anticipate the input heading error to generate lead control. Furthermore, T _L for the dynamic scene condition (0.85 s) was significantly smaller than that for the static scene condition (1.18 s), indicating that flow lines, dot acceleration, and/or other higher order optic-flow information facilitated participant to use the heading movement information to foresee heading errors to generate lead control. A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on τ revealed that only the main effect of display condition was significant, with F(1,6) = 16.71, p < 0.01. Overall, τ for the dynamic scene condition (405 ms) was significantly larger than that for the static scene condition (362 ms), indicating that optic-flow information beyond the velocity field helped participants initiate faster control responses. 

Figure 9

 

Best-fitting model lead time constant (T _L), response delay (τ), and gain (K _p) parameters against display condition. T _L for (a) velocity control and (b) acceleration control. τ for (c) velocity control and (d) acceleration control. K _p for (e) velocity control and (f) acceleration control. Error bars are SEs across seven observers; * indicates the statistical significance from a paired t-test (p < 0.05).

A 2 (display condition) × 2 (control dynamics) repeated-measures ANOVA on K _p revealed that both the main effect of display condition and the interaction effect of display condition and control dynamics were significant, with F(1,6) = 15.82, p < 0.01 and F(1,6) = 6.74, p < 0.05, respectively. Newman–Keul's tests revealed that while for velocity control, K _p was significantly larger for the static than the dynamic scene condition (3.4 vs. 2.6 % max/deg, p < 0.05), ³ for acceleration control, K _p was similar for the two display conditions (2.2 vs. 2.3 % max/deg). This indicates that participants' sensitivity to the input heading error was larger for the static than the dynamic scene condition for velocity but not for acceleration control dynamics. 

Discussion

To summarize the results, the overall performance measurements indicate that for an average vehicle orientation perturbation speed of 8.4°/s (peak: 32°/s), when the vehicle was traveling on a circular path (path curvature = ±0.009 m⁻¹), the unsigned heading bias for the dynamic scene condition (about 4°) was smaller than that for the static scene condition (about 6°), but the overall heading control precision (RMS error) was similar for the two display conditions. These results indicate that participants were capable of robust heading control during combined translation and rotation using information from the instantaneous velocity field of optic flow alone, consistent with findings from previous heading perception studies showing that when the displays provided a large field of view and/or sufficient motion parallax information, observers could accurately perceive heading during translation and rotation using information from optic-flow velocity fields without access to flow lines, dot acceleration, and other higher order optic-flow variables (Li et al., 2009, 2006b). 

The frequency–response analysis of the human operator transfer function (Y _p in Figure 2) provided greater insight into the visually driven component of the control response, specific to the visual cues in the input optic-flow stimuli, and allowed us to segregate response amplitude and response delay effects. Our data show a larger response gain and a shorter response delay for the static than the dynamic scene condition, indicating that flow lines and other higher order optic-flow cues such as dot acceleration improve the efficiency of heading control, when available. Furthermore, by fitting the frequency–response performance data to the Crossover Model, we were able to determine the amount of lead control that participants could generate showing their anticipatory heading control behavior in the two display conditions. This model-dependent analysis showed that the static scene condition produced a 330-ms increase in the lead time constant and a 43-ms decrease in the control reaction time, averaged across the two control dynamics, compared to performance in the dynamic scene condition. This finding indicates that optic-flow information beyond the velocity field allowed participants to better anticipate future heading in order to generate faster and more effective control responses. Thus, while optic-flow velocity fields support robust heading perception, flow lines, acceleration, and/or other higher order optic-flow variables inform human operators of their changing heading direction such that they can better anticipate and control their future heading. 

As shown by Wann and Swapp (2000), making use of the curvature of flow lines in the flow field, observers can immediately judge whether they are heading to the left or right of their gaze direction without perceiving heading from optic flow. Accordingly, flow lines in the static scene display condition should allow human operators to qualitatively anticipate heading with respect to the vehicle orientation to make quick adjustment of the vehicle orientation in the correct direction. When the anticipation of the heading error (i.e., the deviation angle between heading and the vehicle orientation) is largely qualitative, the control response should not reduce the RMS heading error but should increase lead control and shorten reaction time, as we observed. Alternatively, the instantaneous velocity field in the dynamic scene display does not contain information to specify the source of rotation in the flow field (i.e., whether the rotation is due to the vehicle rotation or path rotation, see Li & Cheng, 2011). In contrast, the higher order derivatives of optic flow such as the dot acceleration in the static scene display can indicate whether the rotation in the flow field is due to the vehicle rotation or path rotation (Li et al., 2006b; Perrone & Stone, 1994; Rieger, 1983; Royden, 1994). Accordingly, participants might have estimated the path curvature more accurately in the static scene condition, which helped them anticipate their future heading to generate greater lead control. This interpretation, however, is at odds with the observed greater heading bias under the static scene condition. 

The shorter reaction time for the static scene condition is also consistent with optic-flow information beyond the velocity field providing “redundancy gain.” That is, the presentation of redundant information (e.g., two or more similar cues) can lead to faster response compared with the presentation of a single cue. The reduced reaction time with redundant stimulus cues has been reported for vision (Corballis, 2002), audition (Schröter, Ulrich, & Miller, 2007), as well as across the visual and auditory modalities (Miller, 1986). The redundant information in the static scene displays thus decreased the reaction time of heading control. 

Conclusion

This paper provides a quantitative description, analysis, and model of human performance in a closed-loop heading control task and of the effect of the availability of optic-flow information beyond the velocity field on heading control performance. We conclude that (1) humans can control heading during translation and rotation using information from the instantaneous velocity field of optic flow alone; (2) the frequency-based analysis shows that optic-flow information beyond the velocity field (such as flow lines and/or dot acceleration information) affects heading control performance, especially at high input perturbation frequencies. This indicates that, when available, human performance benefits from optic-flow information beyond the velocity field for heading control; and (3) for an average heading perturbation speed of 8.4°/s, flow lines, dot acceleration, and/or other higher order optic-flow information can increase lead control by about 40% and reduce reaction time by more than 40 ms, indicating that such information facilitates humans to foresee their future heading in order to generate faster control responses. The findings from the current study reinforce the fact that the visual displays used for active control tasks should be designed to meet the needs and constraints of the human visual system. Specifically, although design engineers could present a sequence of velocity fields alone on a display interface to support human heading control performance, optic-flow information beyond the velocity field allows the controllers to better anticipate their future heading for faster and more effective closed-loop control. 

Acknowledgments

This study was supported by a grant from the Research Grants Council of Hong Kong (HKU 7478/08H) to L. Li. We thank Diederick Niehorster and Joseph Cheng for their assistance in programming and data analysis and two anonymous reviewers for their helpful comments on a previous draft of the article. 

Commercial relationships: none. 

Corresponding author: Li Li. 

Email: [e-mail]. 

Address: Department of Psychology, The University of Hong Kong, Pokfulam, Hong Kong SAR. 

Footnotes

Footnotes

1  To make sure that 2D local motion processing was uncompromised in the dynamic scene display due to the reduced dot lifetime (100 ms, 6 frames at 60 Hz) or spurious motion noise (about 1/6 of 150 dots were redrawn per frame), we conducted a control experiment in which three observers were asked to discriminate the speed of a flow field with pure lateral motion. The pedestal average dot speed (9.5°/s) in the display was equated to that of the translation and rotation flow fields in the current study assuming perfect correction of the input perturbation. We used a standard 2IFC procedure to find the speed discrimination threshold for the dynamic and static scene displays. In each trial, the stimulus presentation time was 400 ms and the interstimulus interval was 500 ms. Across three observers, the motion discrimination thresholds (mean JND ± SE) were 17 ± 2.5% and 18 ± 3.5% for the dynamic and static scene displays, respectively. The indistinguishable thresholds show that the visual system's ability to estimate local motion is similar for the two display conditions, and thus local motion processing is uncompromised by the reduced dot lifetime or spurious motion noise in the dynamic scene display.

Footnotes

2  Fewer than 3% of the extended trajectories of the dot motions (i.e., flow lines) in the dynamic scene subtended a visual angle (maximum 3.7° at 42° eccentricity) that exceeded the diameter of the average V1 receptive field (2–3° at that eccentricity, see Hubel & Wiesel, 1977; Dow, Snyder, Vautin, & Bauer, 1981). As such, a tiny minority of points could theoretically yield a finite psychophysical orientation JND if presented alone (>10°, see Vandenbussche, Vogels, & Orban, 1986), but this sparse and imprecise local orientation signal is unlikely accessible given the sea of masking orientation noise. Lastly, the maximum dot acceleration of the closest points at 6 m (those with the most information about heading) was always less than 2.3°/s² for dot speeds up to 27.5°/s; such levels of acceleration yield random performance in acceleration judgment tasks when presented for only 100 ms (Calderone & Kaiser, 1989; Stone & Ersheid, 2006).

Footnotes

3  We used the ratio of the Fourier transform of the joystick displacement to that of the heading error to compute the human operator transfer function Y _p for the estimation of the model parameters. The units for overall gain K _p are thus % of maximum joystick displacement per degree of visual angle of the heading error on the screen.