Ability to identify scene-relative object movement is not limited by, or yoked to, ability to perceive heading

Abstract

During locomotion humans can judge where they are heading relative to the scene and the movement of objects within the scene. Both judgments rely on identifying global components of optic flow. What is the relationship between the perception of heading, and the identification of object movement during self-movement? Do they rely on a shared mechanism? One way to address these questions is to compare performance on the two tasks. We designed stimuli that allowed direct comparison of the precision of heading and object movement judgments. Across a series of experiments, we found the precision was typically higher when judging scene-relative object movement than when judging heading. We also found that manipulations of the content of the visual scene can change the relative precision of the two judgments. These results demonstrate that the ability to judge scene-relative object movement during self-movement is not limited by, or yoked to, the ability to judge the direction of self-movement.

Introduction

When an observer translates through space, a pattern of optical motion is available at the eye, an “optic flow field” (Gibson, 1958/2009). The flow field contains a point from which motion expands radially. This point is called the “focus of expansion” (FoE) and it indicates the direction of translation. The human brain has a well-documented sensitivity to optical flow fields; when observers are shown patterns of optic flow, they are able to judge the direction of simulated translation (“heading”) to within 1° to 2° (W. H. Warren & Hannon, 1988). It was thought that the primary reason humans are sensitive to optic flow is so they could use estimates of heading to guide locomotion (Gibson, 1958/2009; W. H. Warren & Hannon, 1988). Later an alternative role for optic flow processing was proposed, aiding the identification of object movement during self-movement (e.g., Rushton & Warren, 2005; Royden & Connors, 2010; Calabro, Soto-Faraco, & Vaina, 2011; MacNeilage, Zhang, DeAngelis, & Angelaki, 2012; Fajen, Parade, & Matthis, 2013; Niehorster & Li, 2017); one way to identify optical motion due to the movement of an object in the scene, is to identify the optic flow due to movement of the observer and parse, or filter, it out (i.e., flow parsing). Given that optic flow processing supports both the perception of heading and the identification of object movement during self-movement, the questions that arise include the following: What is the relationship between the perception of heading, and the identification of object movement during self-movement? Does ability to judge heading determine ability to judge object movement? Is a common underlying neural mechanism shared by the two processes? In this study, we investigated these questions by examining the relative precision of judgments of heading and judgments of object movement on matched tasks. We start with a very brief summary of the research on the perception of heading and the identification of object movement during self-movement and then explain how we probe the relationship between the two. 

Optic flow and the perception of heading

During the second world-war, Gibson worked on the problem of selecting potential pilots by examining how they judge approach trajectory. Grindley (as cited in Mollon, 1997) had examined the pattern of motion available at the eye during flying and identified a key feature in optic flow (the FoE) that can be used to discern heading. He realized that if a pilot could identify this point, the pilot could use it to judge, and then control, the direction of flight of the airplane. Gibson recognized the value of Grindley's idea (see Mollon, 1997) and subsequently brought the idea to the attention of the scientific community. Gibson expanded on the original idea and proposed that optic flow can be used for not only flying airplanes but also controlling human locomotion, most critically walking. This proposal became very broadly recognized (see Gibson, 1954/1994; Gibson 1958/2009) and had a substantial impact. Following a seminal paper by W. H. Warren and Hannon (1988) demonstrating observers could judge their direction of heading given only a pure flow field, a vast body of work in psychophysics, neuroimaging, and computational modelling of the “heading perception system” followed. 

Optic flow and the identification of object movement

The identification of the movement of an object in the scene is simple when the eye and the observer are stationary; motion in the retinal images indicates movement of an object in the scene. The problem becomes difficult when the eye or the observer is moving. Retinal motion no longer uniquely identifies object movement; retinal motion is generated by both movement of objects in the scene, and the movement of the eye or the observer through space. Given the eye or the observer is seldom still, even during fixation (Rucci & Victor, 2015), the problem of separating retinal motion due to object movement from retinal motion due to the observer's self-movement is pervasive. 

If we consider the relationship between retinal motion and movement in the world, a solution to the problem is immediately obvious:  

\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\unicode[Times]{x1D6C2}}\)\(\def\bupbeta{\unicode[Times]{x1D6C3}}\)\(\def\bupgamma{\unicode[Times]{x1D6C4}}\)\(\def\bupdelta{\unicode[Times]{x1D6C5}}\)\(\def\bupepsilon{\unicode[Times]{x1D6C6}}\)\(\def\bupvarepsilon{\unicode[Times]{x1D6DC}}\)\(\def\bupzeta{\unicode[Times]{x1D6C7}}\)\(\def\bupeta{\unicode[Times]{x1D6C8}}\)\(\def\buptheta{\unicode[Times]{x1D6C9}}\)\(\def\bupiota{\unicode[Times]{x1D6CA}}\)\(\def\bupkappa{\unicode[Times]{x1D6CB}}\)\(\def\buplambda{\unicode[Times]{x1D6CC}}\)\(\def\bupmu{\unicode[Times]{x1D6CD}}\)\(\def\bupnu{\unicode[Times]{x1D6CE}}\)\(\def\bupxi{\unicode[Times]{x1D6CF}}\)\(\def\bupomicron{\unicode[Times]{x1D6D0}}\)\(\def\buppi{\unicode[Times]{x1D6D1}}\)\(\def\buprho{\unicode[Times]{x1D6D2}}\)\(\def\bupsigma{\unicode[Times]{x1D6D4}}\)\(\def\buptau{\unicode[Times]{x1D6D5}}\)\(\def\bupupsilon{\unicode[Times]{x1D6D6}}\)\(\def\bupphi{\unicode[Times]{x1D6D7}}\)\(\def\bupchi{\unicode[Times]{x1D6D8}}\)\(\def\buppsy{\unicode[Times]{x1D6D9}}\)\(\def\bupomega{\unicode[Times]{x1D6DA}}\)\(\def\bupvartheta{\unicode[Times]{x1D6DD}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bUpsilon{\bf{\Upsilon}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(\def\iGamma{\unicode[Times]{x1D6E4}}\)\(\def\iDelta{\unicode[Times]{x1D6E5}}\)\(\def\iTheta{\unicode[Times]{x1D6E9}}\)\(\def\iLambda{\unicode[Times]{x1D6EC}}\)\(\def\iXi{\unicode[Times]{x1D6EF}}\)\(\def\iPi{\unicode[Times]{x1D6F1}}\)\(\def\iSigma{\unicode[Times]{x1D6F4}}\)\(\def\iUpsilon{\unicode[Times]{x1D6F6}}\)\(\def\iPhi{\unicode[Times]{x1D6F7}}\)\(\def\iPsi{\unicode[Times]{x1D6F9}}\)\(\def\iOmega{\unicode[Times]{x1D6FA}}\)\(\def\biGamma{\unicode[Times]{x1D71E}}\)\(\def\biDelta{\unicode[Times]{x1D71F}}\)\(\def\biTheta{\unicode[Times]{x1D723}}\)\(\def\biLambda{\unicode[Times]{x1D726}}\)\(\def\biXi{\unicode[Times]{x1D729}}\)\(\def\biPi{\unicode[Times]{x1D72B}}\)\(\def\biSigma{\unicode[Times]{x1D72E}}\)\(\def\biUpsilon{\unicode[Times]{x1D730}}\)\(\def\biPhi{\unicode[Times]{x1D731}}\)\(\def\biPsi{\unicode[Times]{x1D733}}\)\(\def\biOmega{\unicode[Times]{x1D734}}\)\begin{equation}\tag{1}{v_{retinal}} = {v_{obj}} + {v_{\it{self}}},\!\end{equation}

where v_retinal indicates overall retinal motion, v_obj indicates retinal motion due to object movement, and v_self indicates retinal motion due to self-movement. Therefore,  

\begin{equation}\tag{2}{v_{obj}} = {v_{retinal}} - {v_{\it{self}}}.\end{equation}

 

Retinal motion due to self-movement, can be estimated using two sources of information, extraretinal information, and retinal information. There are two forms of extraretinal information about self-movement. The first is the copy of the movement commands issued by the brain (efference copy) to change the observer's position to fulfil a goal. The second is the feedback (afference) provided by the vestibular and proprioceptive systems once the movement and change in position occur. Both of these forms of “extraretinal” information can be used to predict (through use of a forward model) the retinal motion that would be expected to result from any given self-movement. Gogel and Tietz (1974) and Wallach, Stanton, and Becker (1974) explored the role of “extraretinal” sources of information in the identification of object movement during self-movement. They showed that extraretinal information makes an important contribution but is not sufficient for accurate and precise estimation of scene-relative object movement (see Wexler, 2003; Tcheang, Gilson, & Glennerster, 2005, for more recent work). 

Global patterns of retinal motion generated during self-movement, optic flow, provide retinal information about self-movement. The role of optic flow in the identification of scene-relative object movement has been explored by a number of investigators (Rushton & Warren, 2005; Rushton, Bradshaw, & Warren, 2007; P. A. Warren & Rushton, 2008; P. A. Warren & Rushton, 2009a; P. A. Warren & Rushton, 2009b; Matsumiya & Ando, 2009; Royden & Connors, 2010; Calabro et al., 2011; Calabro & Vaina, 2012; MacNeilage et al., 2012; Royden & Moore, 2012; P. A. Warren, Rushton, & Foulkes, 2012; Dupin & Wexler, 2013; Fajen & Matthis, 2013; Fajen et al., 2013; Foulkes, Rushton, & Warren, 2013; Royden & Holloway, 2014; Niehorster & Li, 2017; Rogers, Rushton, & Warren, 2017). It has been shown that observers can identify the direction of scene-relative object movement on the basis of retinal information alone (Rushton & Warren, 2005), and insight has been gained into how the “flow-parsing” mechanism works and how different sources of information about self-movement can be combined (e.g., P. A. Warren & Rushton, 2009a; Calabro & Vaina, 2012; MacNeilage et al., 2012; Royden & Holloway, 2014; Layton & Fajen, 2016; Niehorster & Li, 2017). 

Current study

Because the perception of heading and the identification of object movement during self-movement both rely on optic flow, it is natural to hypothesize that they share a common neural mechanism and behavioral performance on the two tasks must be related. Specifically, in circumstances in which an observer is poor at judging heading, the observer would be expected to also be poor at judging scene-relative object movement, because of the relationship described in Equation 2. In the current study, we examined the relationship between the precision of heading judgments and the precision of object movement judgments to address the following two questions: 

•  
  Does the precision with which an observer can judge heading place a limit on the precision with which an observer can judge scene-relative object movement? The way to address this question is to seek evidence that judgments of scene-relative object movement can be more precise than corresponding judgments of heading.
•  
  Does the precision of these two types of judgments covary? That is, does a change that improves the perception of heading produce a yoked improvement in the identification of scene-relative object movement? The way to address this question is to seek evidence that changes to the stimulus can produce differential changes in performance.

In the current study, we addressed the above questions about the relationship between heading and object movement judgments across a series of experiments. In each experiment we simulated self-movement (by moving a virtual scene towards the observer) and used carefully matched tasks that allowed us to compare the precision of heading judgments with the precision of object movement judgments. Figure 1 illustrates the displays, the judgment tasks, and the measurements used in the study. Our experiments are based on measures of precision rather than accuracy. The rationale is that if process A feeds process B, then the precision of the output A constrains the precision of process B. In contrast, the accuracy (bias) of the output A does not necessarily constrain the accuracy of process B due to the fact that bias can be canceled out. 

Figure 1

 

judgments of self and object movement. (a) The visual display. An array of 54 red wireframe scene objects of randomized size and orientation located at randomized distances ahead of the observer. The target sphere was rendered in yellow. The display was rendered with disparity cues to distance, and the observer viewed the scene through shutter glasses. (b) Key to the angles and the motion vectors in panels c and d. α is the initial direction of the target relative to the observer's straight ahead, β is the movement direction of the target relative to α; β was varied across trials. (c) The heading judgment task. Schematic diagram. The target sphere and scene objects all move in the same direction (α + β). Inset shows observer's view, the direction of heading, indicated by the red x, is to the right of the target object. (d) The object movement judgment task. Schematic diagram. Scene objects move in direction α, the target sphere moves in direction (α + β). Inset shows observer's view; target object is moving leftward relative to the scene.

The displays were composed of a 3D cloud of red wireframe objects (see Figure 1). We first calculated α, the initial direction of the target sphere relative to the observer's straight ahead. The target sphere was given a movement direction of α + β, where β is between approximately –2.5° (left) to 2.5° (right) with respect to α in both tasks. In the heading task, the scene objects were given the same movement direction as the target sphere, α + β. In the object-movement task, the scene objects were given the movement direction α. 

In the heading judgment task, β is the difference between the direction of the target and direction of movement of the scene objects (indicated in the flow field by the position of the FoE relative to the target sphere) and is called the target-heading angle. In the object movement judgment task, β is the difference between the direction of movement of the target and the direction of movement of the scene objects and is called the target-scene angle. In both tasks, observers effectively tried to judge whether the angle β was positive or negative. 

Overview of experiments

In Experiment 1, we directly compared the precision of heading and object movement judgments. In Experiments 2.1–2.4, we ran a series of control experiments to rule out possible confounds, specifically local motion contrast, the impact of our choice of disparities and distances, and whether the results generalized when the target object was away from the direction of self-movement. In Experiment 3, we examined the impact on the precision of the two judgments when a simulated gaze rotation was added. In Experiment 4, we examined the impact on the precision of the two judgments when the depth range of objects in the scene was increased. 

Experiment 1: Heading versus flowing parsing

Methods

Participants

Twelve observers participated in these experiments. All (four males, eight females) but one were staff or postgraduate students in the School of Psychology, University of Hong Kong. The remaining observer was the first author. The average age was approximately 26 (aged 19 to 46 years). All had normal, or corrected-to-normal, vision and stereo vision. All but two (the first and last authors) were unaware of the experimental hypotheses. 

The same 12 observers participated in the majority of the experiments reported in this paper (details provided in methods sections). Although the experiments are presented here in a logical order, because of piloting we were able to preplan the majority of the testing and randomize the order in which participants completed the experiments. 

Stimuli

Figure 1 is a schematic representation of the stimuli for the heading and object movement judgment tasks. It shows the angles that were manipulated in the two tasks. In the heading judgment task (left panel), all scene objects including the target sphere were moved in the same direction. In the object movement judgment task (right panel), the target sphere and background objects were moved in different directions. All other parameters are identical in the two tasks. 

The speed of the simulated scene-movement approaching the observer was 1 m/s, a typical walking speed, and objects were placed at a distance of 4.4 to 8.4 m, with 8.4 m chosen as the far distance because most manmade environments such as rooms and corridors do not extend much further than this distance. We chose to place objects no closer than 4.4 m because once objects are a few strides away in peri-personal space, mechanisms concerned with interaction (foot placement relative to the object, planning reaching movements, etc.) would likely come into play. 

The background scene consisted of 55 wireframe objects (Figure 1a). Each object was a wireframe sphere (gluSphere) made of five slices (longitude) and five stacks (latitude) with an initial radius of 8 cm. Objects were placed on a regular 11 × 5 grid with a grid spacing of 28 cm horizontally and vertically. Each object was then given a random orientation and random scaling of ±50% in each dimension. The position of each object was randomly perturbed (±8 cm horizontal, ±9.6 cm vertical, and ±200 cm in depth). The result was an array of randomly shaped objects that were randomly distributed but not overlapping. 

The target sphere (radius: 0.8 cm, shown in yellow) was at 28 cm to the left or right of the middle of the array and replaced the scene object in that location. From the viewpoint of the participant, the initial direction of the target was 2.5° to the left or right of the participant's straight ahead (which was aligned with the middle of the display). 

It is difficult for a participant to fuse large uncrossed disparities on a standard monitor at arm's reach because of the large mismatch with the accommodative (focus) cues provided by the surface of the monitor. Therefore, we reduced the scale of the scene and the self-movement speed by a factor of 8. To give specific examples, the radius of the scene objects was reduced from 8 cm to 1 cm, the self-movement speed was reduced from 1 m/s to 0.125 m/s, and the depth range was reduced from 4.4 to 8.4 m to 0.55 to 1.05 m. The reduction in depth reduced the conflict between the vergence demand and the accommodative demand for near and far objects. Critically, because the same reduction in scale was applied to both the scene and the movement speed, the angular size of the objects and the angular velocities remained consistent with those that would be experienced in the original sized scene and movement speed. 

We used the method of constant stimuli for both tasks. In the heading judgment task, we selected 15 equally spaced target-heading angles ranging from −2.5° (left) to 2.5° (right) of the target (Figure 1b). In the object movement judgment task, we selected 15 comparable equally spaced offsets between the direction of movement of the target and the direction of movement of the rest of the scene objects (Figure 1c). 

Equipment

Antialiased stimuli were rendered, using OpenGL, on an NVIDIA Quadro K2000 graphics card and displayed on an Asus VG278H 27-in. LCD monitor at a resolution of 1,920 × 1,080 at 120 Hz (60 Hz per eye). With their heads stabilized by a chin rest at the viewing distance of 57 cm in a dark room, participants viewed the stimuli through a pair of LCD shutter glasses (NVIDIA 3D Vision 2). The left and right eye stimuli were temporally interleaved and displayed in synchrony with the opening and closing of the left and right eye shutter glass lenses to create a stereoscopic presentation. 

Procedure

On each trial, a static view of the scene appeared for 1 s to allow participants to fuse to the stereo half-images into a 3D scene. The scene then moved and transformed for 1 s to simulate the approach of the observer. At the end of the simulated movement, a black blank screen appeared, and participants pressed a mouse button to indicate whether they were going to pass to the left or right of the target sphere for the heading judgment task, or whether the target sphere was moving leftwards or rightwards relative to the scene for the object movement judgment task. 

Each participant completed 120 trials (8 trials × 15 levels of target-heading angles) for both the self-movement and the object movement judgment tasks. Participants received 5–10 training trials at the beginning of each task. No feedback was given on any trial. Each task typically lasted about 10 min. 

Data analysis

We estimated the precision of the observer's heading and object movement judgments. For each observer, we fitted a cumulative Gaussian function to the data. From the fitted curve, we obtained the slope of the curve, one standard deviation (σ) of the fitted Gaussian function, which provided a measure of precision. 

Results

One participant showed a random pattern of judgments for the heading task. This participant's heading judgment data were thus excluded from the data analysis. For illustrative purposes, we combined the data across the rest of 11 participants to create a “composite observer” and fitted it with cumulative Gaussian functions (Figure 2a). We caution that when data are combined across participants to create a composite observer, there is the danger that differences in accuracy between participants will contaminate estimates of precision. Fortuitously there was little difference in accuracy between observers in this or any other experiments reported, so the composite observer graphs capture the pattern of results quite faithfully. 

Figure 2

 

Heading judgments and scene-relative object movement judgments. (a) Proportion of “rightwards” judgments (open and filled squares) as a function target-heading (for heading judgment) or target-scene (for object-movement judgment) offset angle for the composite observer. Data are fitted with a cumulative Gaussian function (solid lines). (b) Precision of judgment for each individual participant along with the mean for the two tasks. Error bars are SEs across 11 participants. (c) Scatterplot of precision of heading judgments against precision of object-movement judgments for each participant.

A cumulative Gaussian function was fitted to each participant's data, and the precision (slope) was calculated. Figure 2b plots the precision estimates for individual participants along with the mean in the two tasks. This plot confirms the difference between conditions. We tested the significance of the difference in the performance by performing a within-subject sample t test on the data from the two tasks. The difference was statistically significant: Mean ± SE: Heading, 1.39° ± 0.19° versus Object-movement, 0.41° ± 0.09°, t(10) = 5.03, within-subject, two-tailed, p = 0.001. Figure 2c plots the precision of heading judgments against the precision of object -movement judgments for each participant. 

Discussion

The precision of the object movement judgments was higher than the precision of the heading judgments. This indicates that the precision of the heading judgments does not limit the precision of the object movement judgments. This finding suggests that intermediate stages of processing of heading and object movement do not share common mechanisms. Because it is not possible in a single experiment to rule out all potential confounds, such as the presence of local motion cues that could aid the judgments of scene-relative object movement, we conducted a series of control experiments to systematically rule out all the potential confounds that would otherwise qualify the conclusion. 

Experiments 2.1 to 2.4: Control experiments

In Experiment 2.1, we examined whether the performance on the object-movement judgment task was artificially boosted by a 2D target drift cue. In Experiments 2.2 and 2.3, we examined whether performance on the object movement judgment task was artificially boosted by local motion cues due to the presence of scene objects that were close in depth (Experiment 2.2) or in similar egocentric directions (Experiment 2.3). In Experiment 2.4, we investigated whether the disparity range in the display of Experiment 1 had artificially boosted performance in the object movement judgment task. 

Experiment 2.1: Ruling out the potential use of a drift cue when judging object movement

In Experiment 1, due to the geometry of the simulated movement of the scene, an object moving leftwards in the scene would drift leftwards relative to the observer (and also the edges of the screen), and an object moving rightwards relative to the scene would drift rightwards. Consequently, participants could have based their judgments on the drifting direction of the target relative to themselves or the image screen edge, rather than the direction of movement of the object relative to the scene. To rule out the possibility that participants might have used this strategy to artificially boost their performance on the object movement judgment task, we ran the first control experiment. We replicated Experiment 1 but placed the target on the opposite side of the screen to the simulated direction of the scene movement (i.e., the FoE in the optic flow field). This change rendered the target drift cue uninformative. As illustrated in Figure 3a, when the direction of scene movement is to the left of the observer's straight ahead, the target sphere is located right of the straight ahead, and vice-versa. As a consequence, the target sphere will drift rightwards relative to the observer (and the screen edges) whether it is moving leftwards or rightwards relative to the scene. 

Figure 3

 

Scene-relative object movement judgments in the absence of a drift cue. (a) Red arrows and black dotted lines show the (common) direction of movement of the background scene objects. The FoE in the optic flow field is to the left of the observer's straight ahead. The target object (yellow sphere) is located to the right of the observer's straight ahead. The movement direction of the target object (dotted green arrow) is offset slightly from the movement direction of the background scene objects (see inset). In this case, the target object is moving leftward relative to the scene, but drifting rightward relative to the observer and the image screen edge. (b) Composite observer. Proportion of judgments that indicated the object was moving rightwards relative to the scene (filled circles) as a function target-scene angle. Data are fitted with a cumulative Gaussian function (solid line). Data from the object-movement judgment task of Experiment 1 (open squares and dashed line) are included for comparison. (c) Precision of judgment for each individual participant along with the mean for the two experiments. Error bars are SEs across 12 participants.

Methods

Participants

The same 12 observers from Experiment 1 participated in Experiment 2.1. 

All methods were the same as in the object movement judgment task in Experiment 1 except that when scene-movement direction was to the left of the participant's straight-ahead, the target sphere was placed on the right (at a symmetrical location) and vice-versa. 

Results

The data for the composite observer that combines all 12 participants' data are shown for illustrative purposes in Figure 3b. The precision of mean object-movement judgments was similar to that in Experiment 1, i.e., there was no significant difference: Experiment 2.1, 0.63° ± 0.09° versus Experiment 1, 0.64° ± 0.24°, t(11) = 0.037, p = 0.97. 

Experiments 2.2 and 2.3: Local motion and object-movement judgments

One possible explanation of the superior performance in the object movement judgment task in Experiment 1 was that observers made use of local motion. Any difference in motion between the target sphere and scene objects that are at a similar distance or close by, is informative about the movement of the target relative to the scene. In previous work we found that motion within 2° to 3° of a target object contributes to flow parsing (P. A. Warren & Rushton, 2009a).The nearest scene objects were (on average) 2.5° from the target sphere in the current study. We thus ran two control experiments. In Experiment 2.2, we moved scene objects so that none was at a similar distance to the target sphere. In Experiment 2.3, we removed all immediately adjacent (as seen from the viewpoint of the observer) scene objects to the target sphere such that no neighboring scene objects were closer than 5° on average. 

Methods

Participants

The same 12 observers from Experiment 1 participated in Experiments 2.2 and 2.3. 

All methods were the same as in the object movement judgment task in Experiment 1 except for the following details. In Experiment 2.2, we ensured that no background scene objects fell within a volume that covered 1m (12.5 cm, factor 8 scaling; see Experiment 1, stimuli) in front of the target object to 1 m behind. In Experiment 2.3, we removed the 10 background scene objects with the closest grid locations. 

Results

Removal of local objects at a similar depth had no appreciable impact on the precision of scene-relative object movement judgments. The data for the composite observer are shown in Figure 4 (Experiment 2.2, top panels; Experiment 2.3, bottom panels). There were no significant differences between the precision of object-movement judgments in Experiments 2.2 and 2.3 and in Experiment 1: Experiments 2.2 versus Experiment 1, 0.74° ± 0.38° versus 0.64° ± 0.24°, t(11) = 0.58, within-subject, two-tailed, p = 0.57; Experiment 2.3 versus Experiment 1, 0.54° ± 0.27° versus 0.64° ± 0.24°, t(11) = 1.16, within-subject, two-tailed, p = 0.27. 

Figure 4

 

Scene-relative object movement judgments in the absence of local motion. (a) No local motion in same depth plane condition and (c) No local motion in the image plane. (a) and (c) Proportion of “rightwards” object movement judgments (filled circles) relative to the scene as a function target-scene angle. Data for the composite observer are fit with a cumulative Gaussian function (solid line). Data from the object-movement judgment task of Experiment 1 (open squares and dashed lines) are included for comparison. (b) and (d). Precision of judgment for each individual participant along with the mean for the two experiments. Error bars are SEs across 12 participants.

Experiment 2.4: Impact of reduced range of disparities on object-movement judgments

As we explained in the Methods section of Experiment 1, we deliberately scaled down the scene and the simulated self-movement speed to reduce the conflict between the accommodative demand and disparity demand and so make the stimuli easier to fuse. A possible concern is that although the angular velocities in the display of Experiment 1 are equivalent to those found in a scene with a depth of 4.4 to 8.4 m when travelling at a typical walking speed of 1 m/s, the scaling nevertheless increased the binocular disparity range of the scene objects by about 8 times compared with what would be experienced in a 4.4 to 8.4 m scene approached at 1 m/s. In this experiment, we thus examined the possibility that the performance on the object movement judgment task in Experiment 1 was artificially boosted by the increased binocular disparity range of the scene objects. Accordingly, we used the display setup of Experiment 1 but reduced the interocular separation by a factor of 8. The scaling reduced the range of disparities (difference between disparity of far object and disparity of near object) to the range subtended in a scene extending from 4.4 to 8.4 m. Critically the scaling reduced both the uncrossed and crossed disparities to prevent the problem of large uncrossed disparities that we previously sought to avoid. 

Methods

Participants

Twelve observers participated in this experiment. All (eight males, four females) were staff or students in the School of Psychology, University of Hong Kong. Three of these observers (one male, two females) participated in all other experiments reported in this paper. The average age was approximately 26 (range 18 to 36 years). All had normal, or corrected-to-normal, vision and stereo vision. All were unaware of the experimental hypotheses. 

Stimuli and equipment

The stimuli and equipment were identical to that used in Experiment 1, but the interocular separation was reduced by a factor of 8 to reduce the range of binocular disparities for the scene objects as planned. 

Results

The reduction in the binocular disparity range of the scene objects did not reduce the precision of the judgments of object movement. The data for the composite observer are shown in Figure 5. There were no significant differences between the precision of judgments with the reduced range of disparities and the original experiment: 0.20° ± 0.05° versus 0.64° ± 0.24°, between-subjects, two-tailed, t(22) = 1.77, p = 0.091. 

Figure 5

 

Scene-relative object movement judgments with a reduced range of disparities. (a) Proportion of “rightwards” object movement judgments (filled circles) relative to the scene as a function target-scene angle. Data for composite observer are fit with a cumulative Gaussian function (solid line). Data from the object-movement judgment task of Experiment 1 (open squares and dashed line) are included for comparison. (b) Precision of judgment for each individual participant along with the mean for the two experiments. Error bars are SEs across 12 participants.

Discussion of control experiments

In Experiments 2.1–2.4, we examined factors that might have artificially boosted the precision object-movement judgments in Experiment 1. We found no evidence that the stimulus parameters used in Experiment 1, in which we compared judgments of heading and object movement using matched displays, artificially boosted precision on the object-movement judgment task. By running the control experiments, we also incidentally produced a series of replications of the object movement judgment results of Experiment 1. Therefore, we can conclude that the results of Experiment 1 are robust in showing that, on matched tasks, judgments of scene-relative object movement can be more precise that judgments of heading. The ability to identify scene-relative object movement is not constrained by the ability to perceive heading from optic flow. In the next two experiments, we examined whether the relative precision of object-movement and heading judgments is yoked using two different manipulations. 

Experiment 3: Impact of simulated gaze rotation on heading versus flow parsing

When an eye is translating through space, the FoE in the flow field indicates the direction of translation (heading). If the eye rotates as it translates, the FoE in the retinal flow field no longer indicates the heading direction. If the output from the heading perception system feeds flow parsing. any impact of gaze rotation on the precision of heading judgments would be expected to have a knock-on impact on the perception of scene-relative object movement. 

In this experiment, we added a small simulated gaze rotation (≤1°/s) on each trial. The magnitude and sign of simulated gaze rotation was randomly varied from trial to trial. If the heading system provides information for flow parsing, then changes that increase or decrease precision on heading judgments would be expected to produce corresponding changes in the precision of object movement judgments. 

Methods

Participants

The same 12 observers from Experiment 1 participated in this Experiment. 

All methods were the same as in Experiment 1 except that a simulated gaze rotation was imposed in the scene. The gaze rotation was randomly selected from a range of −1° to 1 °/s on each trial. This is a comparatively low rotation rate. Piloting suggested that effects on heading perception would be found in this range. We used this range because at a low rotation rate the simulated gaze rotation was not obvious to naïve observers. 

Results

Two participants showed a random pattern of judgments for the heading task. These two participants' heading judgment data were thus excluded from the data analysis. The heading judgment data for the composite observer that combines the data from the rest of 10 participants are shown in Figure 6a. The simulated gaze rotations decreased the precision of heading judgments. There were significant differences between the precision of heading judgments with simulated gaze rotations in the current experiment and that in Experiment 1: 3.0° ± 0.37° versus 1.40° ± 0.21°, t(9) = 3.66, within-subject, two-tailed, p = 0.005, Figure 6b. 

Figure 6

 

Heading and object-movement judgments in the presence of simulated gaze rotations. (a) and (c) Proportion of “rightwards” judgments (filled circles) as a function target-heading (for heading judgment) or target-scene (for object-movement judgment) offset angle. Data for the composite observer are fit with a cumulative Gaussian function (solid lines). Data from Experiment 1 (open squares and dashed line) are included for comparison. (b) and (d) Precision of judgment for each individual participant along with the mean for the two experiments. Error bars are SEs across 10 participants in (b) and 12 participants in (d). (e) Precision of heading judgments against precision of object-movement judgments for each participant.

In contrast, the simulated gaze rotations did not decrease the precision of judgments of object movement. The object movement judgment data for the composite observer that combines the data from 12 participants are shown in Figure 6c. There were no significant differences between the precision of object-movement judgments with simulated gaze rotations in the current experiment and in Experiment 1: 0.72° ± 0.19° versus 0.64° ± 0.24°, t(11) = 0.38, between-subjects, two-tailed, p = 0.71, Figure 6d. The precision of object movement judgments was also significantly better than that of the equivalent heading judgments in the current experiment: 0.56° ± 0.18° versus 3.0° ± 0.37°, t(9) = 5.99, within-subject, two-tailed, p < 0.001. 

Figure 6e plots the precision of heading judgments against the precision of object movement judgments for each of the 10 participants who had both heading and object movement judgment data. 

Discussion

In this experiment, the addition of a simulated gaze rotation reduced the precision of heading judgments, but it did not produce a corresponding reduction in the precision of object-movement judgments. This again supports the conclusion that the ability to identify scene-relative object movement (i.e., flowing paring) is not constrained by the ability to perceive heading from optic flow and that performance is not yoked on the two tasks. The lack of impact of simulated gaze rotation on object movement judgments also indicates that the drop of precision in the heading judgment task may not be due to the computational complexity of identifying optic flow in the presence of simulated gaze rotations. We discuss this point further in the General discussion. 

Experiment 4: Impact of increased depth range on heading versus flow parsing

Previous studies have shown that the precision of heading judgments increases when the distance of the far objects is increased (van den Berg & Brenner, 1994) and with the increase of motion parallax information in the optic flow field (e.g., Stone & Perrone, 1997; Li & Warren, 2000; Li, Chen, & Peng, 2009). One way to increase motion parallax information in the optic flow field is to increase the depth range of objects in the scene. Accordingly, in this experiment, we increased the depth range of scene objects to about 6 times of that in Experiment 1. We expected an improvement in the precision of heading judgments, and the question was whether we would find a yoked improvement in the precision of object-movement judgments. 

Methods

Participants

The same 12 participants in Experiment 2.4 also participated in this experiment. 

All methods were identical to Experiment 1, but the volume that contained the scene objects was increased from 0.55–1.05 m to 0.5–3.8 m. The new depth range, when scaled to a locomotion speed of 1 m/s (see Methods for Experiment 1), would be equivalent to 4.5 to 30.4 m. An increase in the distance range produces an increase in the range of disparities. To minimize the likelihood of consequent problems with fusion of the stimuli, we reduced the interocular separation by a factor of 2 (the same technique we used in Experiment 2.4). The resulting range of disparities was approximately 0.4°, almost identical to that in Experiment 1. Because the mean distance of the scene objects was increased, we also increased the size of the scene objects by a factor of 2 so that they subtended approximately the same angular size as in Experiment 1. 

Results

The data for composite observers are shown in Figure 7. The extended depth range increased the precision of heading judgments. The increase in the precision of heading judgments with an extended depth range in the current experiment compared with that in Experiment 1 was significant: Experiment 2.4, 0.58° ± 0.08° versus Experiment 1, 1.39° ± 0.19°, t(21) = 3.98, between-subjects, two-tailed, p = 0.00069. 

Figure 7

 

Heading and object-movement judgments with an extended range of distances. (a) Proportion of “rightwards” judgments (open and filled squares) as a function target-heading (for heading judgment) or target-scene (for object-movement judgment) offset angle. Data for the composite observer are fit with a cumulative Gaussian function (solid lines). (b) Precision of judgment for each individual participant along with the mean for the two experiments. Error bars are SEs across 12 participants. (c) Scatterplot of precision of heading judgments against precision of object-movement judgments for each participant.

In contrast, the extended depth range did not increase the precision of object-movement judgments. The difference between the precision of object-movement judgments with an extended depth range in the current experiment compared with that in Experiment 1 was not significant: Experiment 2.4, 0.76° ± 0.14° versus Experiment 1, 0.64° ± 0.24°, t(22) = 0.42, between-subjects, two-tailed, p = 0.68. By extending the depth range of scene objects to increase motion parallax information in the optic flow field, we successfully increased the precision of heading judgments, but the precision of object movement judgments did not increase in a corresponding manner. As a consequence, unlike Experiment 1, there was no significant difference between the precision of heading and object-movement judgments: 0.58° ± 0.08° versus 0.76° ± 0.14°, t(11) = 1.52, within-subject, two-tailed, p = 0.16; Figure 7b. 

General discussion

In Experiment 1 we found that object movement judgments were more precise than heading judgments in matched displays. Across a series of control experiments (2.1–2.4) we looked for, and failed to find, factors that might have artificially boosted performance on the object-movement judgment task. In Experiment 3 we found that simulated gaze rotations decreased the precision of heading judgments but had no effect on the precision of object movement judgments. In Experiment 4 we increased the depth range of the objects in the scene, which produced an increase in the precision of heading judgments but no corresponding change in the precision of object movement judgments. 

What might these results tell us about the underlying neural substrates? Given that the precision of heading judgments is lower than the precision of object movement judgments in Experiments 1–3, it seems very unlikely that the system that supports heading judgments provides the input to the system that identifies scene-relative object movement (i.e., Display Formula\({v_{\it{self}}}\) in Equation 2). This conclusion is in line with Warren, Rushton and Foulkes (2012) who showed that the superimposition of laminar flow on a radial flow display produces a bias in the perceived heading (the “optic flow illusion,” Duffy & Wurtz, 1993) but has no effect on the object movement judgments. 

Foulkes, Rushton, and Warren (2013) suggested that heading and flow parsing rely on a common system at early stages of processing. This conclusion was based on the finding that heading judgments and object movement judgments are affected in a similar way by the statistical qualities of the flow field (the number of flow vectors and the noise added to the flow vectors). The findings of the current study place a limit on how much common processing could be involved. Our results would be compatible with common early stages, perhaps up as far as MT (TO-1) or even MST (TO-2), but the estimation of heading and the identification of object movement must involve some additional independent stages. 

Experiment 4 provides further support for separate later stages. The extended depth range boosted performance on the heading task but not the object movement task. If heading and flow parsing rely on common later stages, then changes in performance would be linked. Furthermore, the results of Experiment 4 also suggest a difference in cue-use between heading and flow parsing processes. It has been reported that an extension of the depth range of scene objects provides enhanced depth cues that are combined with the motion cues at a later stage for heading judgment (van den Berg & Brenner, 1994). This would explain a selective enhancement of heading judgments while the precision of object movement judgments is unchanged. This explanation is compatible with previous research findings showing that observers use both form and motion cues to estimate heading (Niehorster, Cheng, & Li, 2010), but observers use only motion cues when judging object movement (Rushton, Niehorster, Warren, & Li, 2018). 

In summary, the data we reported here, in addition to the previously reported findings, point to common early processing (perhaps a shared processing stage that reaches to MT or MST), followed by a separate processing stages for heading and parsing, the latter reliant on pure optic flow/retinal motion (plus extraretinal inputs), the former integrating other visual cues. 

Differential effects of simulated gaze rotations

Why did the simulated gaze rotation affect heading but not object movement judgments in Experiment 3? The results of Experiment 3 prompt questions about of how heading is perceived. One possibility is that “target drift” has more influence than we commonly recognize. Target drift is the change in the visual direction of a target relative to the observer. The role of target egocentric direction in guiding goal-oriented locomotion such as walking or steering toward a goal, in particular the strategy of keeping a target at a fixed visual direction in egocentric space (Rushton, Harris, Lloyd, & Wann, 1998), is broadly recognized (e.g., Wood, Harvey, Young, Beedie, & Wilson, 2000; M. G. Harris & Carré, 2001; W. H. Warren, Kay, Zosh, Duchon, & Sahuc, 2001; J. M. Harris & Bonas, 2002; Li & Cheng, 2013; Li & Niehorster, 2014). In simulated locomotion and steering tasks, the role of target egocentric direction, is normally not considered when a strong egocentric reference frame is missing. However, rate of change of direction, target drift, remains a potentially potent cue. 

As Llewellyn (1971) pointed out, if you are heading to the left of a near target, it drifts right relative to you, and vice-versa. To reach a near target, you can simply cancel drift (see Rushton & Allison, 2013). Even when the exact egocentric direction is unknown in a simulated locomotion or steering task, the direction of target drift remains an effective cue to judge target-relative heading. The target drift cue was available in Experiments 1 and 2. However, in Experiment 3, the simulated gaze rotation added noise to the target drift cue. The drop in precision in Experiment 3 (Figure 6a) is compatible with target drift playing an important role in the perception of heading. 

The typical heading judgment tasks used in previous studies (e.g., W. H. Warren & Hannon, 1988; Royden et al., 1992; Stone & Perrone, 1997; Li, Sweet, & Stone, 2006; Li et al., 2009; Li & Cheng, 2011; Foulkes et al., 2013) require observers to make heading judgments relative to a homogenous (dot) scene (or the edges of a display) rather than a specific target. The target drift cue cannot be used in such tasks. As a consequence, its role in heading perception has not been systematically studied. The findings reported here should provoke a fuller reconsideration of the role of target drift in future studies. 

The second possibility is based on a reconsideration of the reference frame that used for heading judgments. Gibson (1958) proposed that heading is picked up directly from optic flow in a world-centric reference frame. It is not necessary to know anything about the orientation, position, or movement of body parts or effectors, nor is it necessary to know anything about the vision system which could be an eye or a camera. The information for guiding locomotion is just there in optic flow, which makes it a paradigmatic case for direct perception (see M. G. Harris & Carré, 2001). Under Gibson's proposal, the finding that heading judgments can be impaired in the absence of veridical extraretinal information (such as efference copy and oculomotor signals) about eye movements becomes problematic (hence the extended debate in the 1980s and ‘90s, e.g., W. H. Warren & Hannon, 1988; Royden et al., 1992), because it challenges the notion of a direct pickup of accurate self-movement information from optic flow. In Experiment 3, the low rates of simulated gaze rotation (−1° to 1°/s) should not have posed a problem for the perception of scene-relative heading (W. H. Warren & Hannon, 1988; Li & Warren, 2000). So, why was the precision of heading judgments affected? One potential explanation is that the computation of heading is not performed in a world-centric frame, but rather an egocentric one. 

If we start from an ecological perspective, the utility of perceiving heading in an egocentric reference frame becomes apparent. To illustrate: to walk toward a target, the observer judges the visual direction of the target relative to the body and then generates self-movement in the same direction of the target. Extraretinal information (vestibular cues, proprioceptive cues, gaze rotation signals) provides feedback about the movement, in an egocentric reference frame. Retinal flow also provides information about self-movement in an egocentric reference frame. It would thus be easier for the brain to combine information from retinal and extraretinal sources to estimate heading in an egocentric rather than a world-centric reference frame (see Rushton & Allison, 2013). In Experiments 1 and 2, the observer could compare the egocentric direction of the target and the egocentric direction of heading. This would give the same outcome as judging heading in a world-centric frame. In Experiment 3, we add a simulated gaze rotation on each trial. Simulated gaze rotations provide false information about the rotation of the eye and so would bias estimates of egocentric heading. As the magnitude and direction of the simulated gaze direction varies on each trial, the trial-to-trial differences in bias would lead to a reduction in the estimated precision of heading judgments. 

Which explanation of the results of Experiment 3, if either, is correct, cannot be resolved here but we hope to investigate the issue in future experiments. 

Conclusion

We found that judgments of scene-relative object movement can be more precise than judgments of heading in matched displays. Simulated gaze rotations and an increase in the depth range differentially impact on the precision of judgments of heading and judgments of object movement. It is possible to increase the precision of heading judgments while not producing a corresponding increase in the precision of object movement judgments. Ability to identify scene-relative object movement is not limited by, or yoked to, ability to perceive heading during self-movement. 

Data and code

Data is available from the ReShare research data repository, https://doi.org/10.5255/UKDA-SN-853037. Code is available on request from the same site. 

Acknowledgments

This study was supported by grants from the Research Grants Council of Hong Kong (HKU 746013H), Shanghai Science and Technology Committee (17ZR1420100), and NYU-ECNU Joint Research Institute to LL, a research grant from UK Economic and Social Research Council (ES/M00001X/1) and a Serena Yang travel grant from the Psychology Department at The University of Hong Kong to SKR. We thank Diederick Niehorster for his programming support. 

SKR and LL designed all experiments. SKR programmed the display. RC collected the data. RC, LL, and SKR analyzed the data and generated the figures. SKR, LL, and RC wrote the paper. 

Commercial relationships: none. 

Corresponding authors: Simon K. Rushton; Li Li. 

Email: [email protected]; [email protected]. 

Address: School of Psychology, Cardiff University, Cardiff, Wales, UK; Neural Science Program, NYU-ECNU Institute of Brain and Cognitive Science, New York University Shanghai, Shanghai, PRC.