Possible role for recurrent interactions between expansion and contraction cells in MSTd during self-motion perception in dynamic environments

Abstract

Cortical area MSTd contains cells sensitive to the radial expansion and contraction motion patterns experienced during forward and backward self-motion. We investigated the open question of whether populations of MSTd cells tuned to expansion and contraction interact through recurrent connectivity, which may play important roles in postural control and resolving heading in dynamic environments. We used a neural model of MSTd to generate predictions about the consequences of different types of interactions among MSTd expansion and contraction cells for heading signals produced in the case of self-motion in the presence of a retreating object—a stimulus that recruits both expansion and contraction MSTd cell populations. Human heading judgments from a psychophysical experiment that we conducted were consistent only with the MSTd model that contained recurrent connectivity within and between expansion and contraction cell populations. The model and human heading judgments were biased in the direction of the object motion when the object crossed the observer's future path and biased in the opposite direction when the object did not cross the path. We conclude that recurrent interactions among expansion and contraction cells in MSTd provide a plausible mechanism to support robust self-motion through dynamic environments.

Introduction

Through his observations of fighter pilots during World War II, Gibson noted that linear self-motion generates a radial optic flow field with a singularity point that specifies the direction of travel (or heading) (Gibson, 1950). This point is called the focus of expansion (FoE) when the observer moves forward and focus of contraction (FoC) when the observer moves backward. Neurons in the dorsal medial superior temporal area (MSTd) have been implicated in heading perception because they respond to the radial fields of optical motion (Duffy & Wurtz, 1991a; 1991b; Saito, Yukie, Tanaka, & Hikosaka, 1986; Tanaka, Hikosaka, Saito, & Yukie, 1986) and are selective for the FoE/FoC position (Duffy & Wurtz, 1995). More recently, strong evidence has emerged that causally links MSTd activity to three-dimensional (3D) heading perception in primates (Gu, DeAngelis, & Angelaki, 2012). 

Although the focus is typically on MSTd cells that are sensitive to expansion (expansion cells), MSTd also contains cells that respond to contracting optic flow (contraction cells), which is generated during backward self-motion. The presence of expansion and contraction cells in MSTd raises the question of whether the two populations of neurons interact. Interactions among expansion and contraction cells could serve several important functions, such as improving discrimination between forward and backward self-motion on the basis of visual information. This could be especially useful for postural stability, which demands optic flow sensitivity in all directions (Bardy, Warren, & Kay, 1999; Lee & Aronson, 1974). Inhibitory connections could also play a role in the perception of heading direction in dynamic environments containing moving objects, but the consequences of such interactions are more complex. Consider the scenario in which an observer moves forward in the presence of a moving object that retreats in depth at a faster speed than the observer (Figure 1a). The contracting flow generated by the retreating object occupies a smaller region of the optic flow field and conflicts with the surrounding background expansion (background flow) (Figure 1b). Similar contracting patterns that encompass the entire visual field, however, are also encountered during backwards self-motion through a static environment. Therefore, the object and background flow patterns may activate the same MSTd neurons (Sasaki, Angelaki, & DeAngelis, 2013), which may lead to ambiguity in the heading signal when the object trajectory does not match the observer's heading (Logan & Duffy, 2006). If heading perception depends on MSTd activity, the visual system faces the challenge of reconciling whether the responses of MSTd neurons reflect motion of the observer, an object, or a combination. Inhibitory interactions would perhaps help resolve self-motion from object motion. 

Figure 1

 

(a) Forward self-motion in the presence of a retreating object. (b) Forward self-motion through a static environment produces a radially expanding pattern of optic flow (black). When an object recedes in depth at a faster rate than the observer moves forward, a contracting pattern of motion is generated within the contours of the object (gray).

In the present study, we focus on the scenario in which an observer moves forward in the presence of a retreating object. This scenario is useful because the resulting stimulus contains both expanding and contracting radial flow and therefore might simultaneously engage both expansion and contraction neurons in MSTd. The pattern of heading bias that arises in this scenario may provide insight into the nature of the interactions between expansion and contraction cells. Heading perception would be more accurate in the retreating object scenario if the heading estimate were based entirely on expansion cell activity — that is, if there were no interaction between expansion and contraction cells (no-interaction hypothesis). This is because forward self-motion always generates radial expansion, and when expansion and contraction stimuli are presented separately, individual MSTd neurons are selective to one or the other but not both (Graziano, Andersen, & Snowden, 1994), a finding that is supported by model simulations (Layton & Browning, 2014). On the other hand, such interactions may exist for the reasons explained earlier (i.e., to sharpen the discrimination between forward and backward self-motion). If so, then heading perception may be biased by retreating objects in some circumstances (interaction hypothesis). 

Materials and methods

Our approach to investigate the nature of interactions between expansion and contraction MSTd neurons combines computational modeling and psychophysical experiments. We generated predictions for the interaction and no-interaction hypotheses using a neural model of heading perception in the presence of moving objects. We developed four versions of the model with different connectivities within MSTd and simulated each version to generate predictions. We then compared the predictions of each model against judgments of human observers from a new experiment on heading perception in the presence of retreating objects. The new experiment is needed because past research on self-motion perception in the presence of moving objects has focused on scenarios in which the object either approaches the observer (Layton & Fajen, 2016b, 2016c; Warren & Saunders, 1995) or moves in depth at the same speed as the observer such that it maintains a fixed depth (Royden & Hildreth, 1996). The scenario in which the object retreats from the observer at a speed that exceeds the observer and therefore generates contracting flow has never been studied. 

Specification of neural model

We created a simple neural model to generate predictions about how heading perception should be influenced by retreating objects based on the nature of interactions between expansion and contraction MSTd neurons. The model builds upon that of Layton, Mingolla, and Browning (2012) and upon the Competitive Dynamics model (Layton & Fajen, 2016a), simplified to focus on the nature of the connectivity within MSTd. 

The model consists of two stages, corresponding to motion processing in area MT⁺ and the computation of heading in area MSTd (Figure 2). 

Figure 2

 

Overview of the neural model of primate areas MT⁺/MSTd. Units in model MT⁺ are tuned to motion direction and respond when optic flow appears within the receptive field. Model MT⁺ projects to model MSTd, which contains center-weighted units that are tuned to radial expansion and contraction and are selective to a particular FoE or FoC position within the visual field. We created several versions of the MSTd model, with different types of on-center/off-surround recurrent connections, to generate predictions about how heading signals are affected during self-motion in the presence of retreating moving objects. Units in the no-interaction model integrate bottom-up optic flow signals within their receptive fields, but contain no connections within MSTd. The within-interaction-only model includes recurrent connections among units tuned to the same type of radial motion (expansion units only laterally inhibit other expansion units; contraction units only laterally inhibit other contraction units). The across-interaction-only model includes recurrent connections among units tuned to different types of radial motion (expansion units only laterally inhibit contraction units and vice versa). The full-interaction model contains both types of recurrent connections.

Model MT⁺

MT⁺ units integrate optic flow in the reference frame of the moving observer. The following equation describes the velocity Display Formula of points in the environment Display Formula caused by the observer's self-motion:  

 

 

In Equation 1, t→ = {t_x,t_y,t_z} refers to the translation vector of the observer, r→ = {r_x,r_y,r_z} refers to the rotation vector, and Display Formula indicates the cross-product operator. Points are defined with respect to a right-handed Cartesian coordinate system, in which the X and Y axes span the horizontal (i.e., parallel to the ground) and vertical (i.e., skyward) directions, their origin coincides with the center of the eye, and the Z component aligns with the optical axis. Here, we focus on the movements encountered during self-motion along a flat ground surface without rotation (Display Formula) along the positive Z axis. That is, the observer may move through the XZ axis at eye height, parallel to the ground (Display Formula). To generate the retinal image velocities that serve as the input to model MT⁺, we performed a pin-hole camera projection of points in the world Display Formula onto a planar surface that represents the model retina: Display Formula, where Display Formula refers to the focal length, which we set to 1.7 cm, and Display Formula refers to the horizontal and vertical image components of the projected points. We then took the time derivative Display Formula and plugged in the velocities described by Equation 1 (Longuet-Higgins & Prazdny, 1980):  

 

In Equation 2, Display Formula indicates the horizontal and vertical vectorial motion components, respectively, at each location in the image plane and Display Formula refers to the depth in the environment.  

The magnitude Display Formula and direction Display Formula of each optic flow vector Display Formula are specified by Equations 3 and 4, respectively.  

 

 

 

 

The response of each model MT⁺ unit Display Formula depends on the difference between the direction of the optic flow within the receptive field Display Formula and the unit's preferred motion direction Display Formula,  

 

where Display Formula indicates the half-wave rectification of the operand Display Formula. The cosine in Equation 5 ensures that the MT⁺ unit responds maximally when the motion direction in the receptive field Display Formula matches the preferred direction Display Formula (Figure 3a). The rectified cosine in Equation 5 yields a directional tuning that is consistent with the broad directional bandwidth of many macaque MT cells (Albright, 1984).  

Figure 3

 

Tuning characteristics of model MT⁺ (a) and MSTd (b) cells. (a) Model MT⁺ cells exhibit a broad tuning to motion direction. (b) MSTd units sensitive to radial expansion and contraction are tuned to different FoE and FoC positions, respectively, within the visual field.

In our simulations, we implemented MT⁺ units selective for 24 (Display Formula) directions spaced every degree within the 100° × 100° visual field (240,000 MT⁺ units). For simplicity, we assumed that each unit integrated optic flow (Eq. 5) at a single point Display Formula (i.e., MT⁺ units did not have spatially extended receptive fields). Table 1 summarizes specific parameter values throughout our simulations.  

Table 1

 

Model parameter values.

 

Model MSTd: Receptive fields

We defined the receptive fields of model MSTd units based on sensitivity to one of the two following patterns:  

 

 

 

Equation 6 refers to units tuned to radial expansion (Figure 3b, top row) and Equation 7 refers to units tuned to radial contraction (Figure 3b, bottom row). Units possess heading sensitivities that correspond with the singularity position, which occurs when Display Formula To match the radial patterns A→(i,j,x,y) and B→(i,j,x,y) with the responses generated by MT⁺ cells tuned to 24 directions, we created templates that extract MT directional signals when they appear within the appropriate region of the radial patterns defined by Equations 6 and 7. For example, the template for radial expansion integrates the responses of MT⁺ cells tuned to rightward motion when their receptive fields coincide with the right side of the visual field. The following equations define the radial templates Display Formula for units tuned to expansion Display Formula:  

 

 

 

 

 

Equation 8 gives the angle Display Formula of each vector in the radial template A→ and Equation 9 specifies the range of motion directions (15° in present simulations) that are integrated within each of the Display Formula disjoint spatial subregions that form a MSTd receptive field. We selected this angular granularity so that the spatial subregions of the radial expansion pattern formed by Equation 9 partition the visual field into 24 distinct “wedges” that intersect at the singularity position Display Formula. Each wedge selects MT⁺ motion signals of a common direction when they appear within the appropriate subregion of the radial template. Corresponding contraction templates are obtained by substituting Display Formula in Equations 8 and 9. Receptive fields span the entire visual field, but motion signals nearby the preferred FoE/FoC positioned at Display Formula receive greater weight than distal signals (center-weighting). The parameter Display Formula in Equation 10 controls the extent of the spatial center-weighting and the exponential describes the pattern with which motion weights decrease with distance from the preferred FoE/FoC position (3° SD; see Table 1). In our simulations, we restricted templates to have FoE/FoC selectivities that lie along the horizon that includes the observer's heading direction (Display Formula; see Figure 3b). To refer to expansion and contraction MSTd units together, we use the notation Display Formula. We simulated 100 MSTd units of each template type.  

We denote the similarity or match between each MSTd unit's preferred template Display Formula (Equation 10) and the MT⁺ population activity Display Formula (Equation 5), which is normalized by the energy of the expansion and contraction templates, by Î_±:  

 

Weak matches below the threshold Display Formula (2.5%) are suppressed and the resultant signal is sharpened:Display Formula. Because j remains zero in all simulations, we drop the j index in subsequent equations. To smooth out the distribution of template activations, MSTd units perform an on-center pooling of similar matched motion signals (Display Formula)  

 

where Display Formula indicates the dummy coordinate of the normalized Gaussian filter that spans the space of template singularity positions, and Display Formula refers to the filter's standard deviation.  

Model MSTd: Dynamics

Model MSTd neurons are single-compartment Hodgkin–Huxley membrane equations that obey shunting dynamics.  

 

Equation 13 describes the firing rate Display Formula of the model neuron. The term Display Formula describes the passive decay of the neuron from an activated state to equilibrium, Display Formula refers to the unit's excitatory inputs, and Display Formula refers to the unit's inhibitory inputs. The inputs Display Formula and Display Formula are shunted by the unit activity Display Formula and Display Formula, respectively, which ranges between 0 and 1. Such shunting interactions implement divisive normalization and automatically bound the energy of the network (Grossberg, 1973; Heeger, 1992).  

Model MSTd consists of two populations of units, either tuned to expansion or contraction (Equations 8–10), whose activity Display Formula dynamically evolves over time (Equation 14). For visual clarity, we drop the spatial coordinate notation i, but each equation applies to every spatial position.  

 

 

In Equation 14, Display Formula refers to the input signal from MT⁺ (Equations 11 & 12) and Display Formula and Display Formula refer to binary variables (0 or 1) that control the presence of recurrent connectivity among MSTd units tuned to the same (e.g., expansion-to-expansion) and different (e.g., expansion-to-contraction) pattern types, respectively. As will be described as follows (Equations 16–18), the functions Display Formula and Display Formula define the extent and weight of inhibitory recurrent feedback signals between MSTd units tuned to the same and different pattern type, respectively. The notation Display Formula indicates interactions between MSTd units tuned to the same pattern type and Display Formula indicates interactions between units with opposite singularity sensitivities. For example, the term Display Formula indicates recurrent self-excitation to the unit Display Formula, meaning expansion and contraction cells send positive feedback to themselves (i.e., expansion-to-expansion, contraction-to-contraction). On the other hand, Display Formula refers to the inhibitory signals sent to MSTd units tuned to the opposite pattern (e.g., expansion-to-contraction). The function Display Formula is a sigmoid that describes how feedback activity propagates within the network:  

 

where Display Formula defines to value at which the sigmoid attains one-half its maximal value and Display Formula is the neuron's firing threshold.  

The MSTd units described by Equation 14 interact via on-center/off-surround connections in a recurrent network (see Figure 2 for an overview). The on-center component, Display Formula, consists of two recurrent excitatory interactions. Each unit sends itself an on-center recurrent excitatory signal, Display Formula for expansion cells Display Formula and Display Formula for contraction cells Display Formula. Expansion and contraction units tuned to same FoE/FoC position also receive reciprocal excitatory feedback from one another, Display Formulafor expansion cells Display Formula and Display Formula for contraction cells Display Formula. The excitatory feedback is multiplied by the expansion and contraction input signals, Display Formula and Display Formula, respectively, to preclude expansion cells from indirectly activating quiescent contraction cells and vice versa. In other words, contraction cells cannot modulate the activity of expansion cells by excitatory feedback without a driving bottom-up signal.  

The terms involving Display Formula and Display Formula (e.g. Display Formula in Eq. 14) implement the off-surround recurrent interactions among MSTd units. Each MSTd unit receives inhibitory feedback signals from others tuned to spatially offset centers of motion: Display Formula describes the inhibition a unit receives from the population tuned to the opposite center of motion type and Display Formula describes inhibition a unit receives from the population tuned to the same center of motion type. For example, Display Formula corresponds with the recurrent inhibition an expansion unit receives from contraction units that have neighboring receptive field centers, and the term Display Formula corresponds with the recurrent inhibition the expansion unit receives from other expansion units that have neighboring receptive field centers. Model MSTd units may contain all or a subset of these competitive interactions, depending on the model configuration (see Section Model MSTd: Connectivity). The within- (Display Formula) and across- (Display Formula) MSTd population connections are defined by Equations 16 and 17, respectively.  

 

 

 

The coordinate Display Formula in each sum refers to the dummy index of the Gaussian kernel Display Formula, defined by the following equation, that pools the activity of MSTd units with nearby receptive field centers:  

 

In Equations 16 and 17, Display Formula and Display Formula define the standard deviations of the Gaussian connection kernels between the MSTd units sensitive to the same and different pattern types, respectively (see Table 1).  

The model's heading estimate (Display Formula) was determined by considering the FoE position of the maximally active expansion cell at the end of the trial:  

 

 

Model MSTd: Connectivity

We created four versions of the MSTd model with distinct connectivity patterns. In the no-interaction model, expansion and contraction MSTd units did not interact within (Display Formula) or across (Display Formula) each population. In the within-interaction-only model, MSTd units interacted within (Display Formula), but did not interact across (Display Formula) each population. In the across-interaction-only model, MSTd units did not interact within (Display Formula), but interacted across (Display Formula) each population. In the full-interaction model, MSTd units interacted within (Display Formula) and across (Display Formula) each population. Parameter values used in all simulations are listed in Table 1 and remained the same in all versions of the network.  

Simulation conditions

The way in which retreating objects influence heading estimates in the model depends on the type of connectivity between and within expansion and contraction cells in model MSTd (see Results). To quantify these effects, we simulated the models with optic flow corresponding with self-motion in the presence of a retreating object that moved along different trajectories relative to the observer's heading. The simulated observer moved straight ahead (0° heading) at 200 cm/s for 1.5 s, either along a dot-defined ground plane (6,000 dots) or toward two frontoparallel dot planes (3,000 dots each) initially positioned 800 and 1,000 cm away in depth. The moving object was represented by a 150 Display Formula 150 cm frontoparallel dot plane (1,000 dots) that retreated from the observer along its trajectory at 300 cm/s from its initial depth of 300 cm (16° horizontally and 21° vertically). The object started 200 cm on the left side of observer's heading and its relative trajectory ranged from 5° to 90°, with angles 5°–89° tested in 2° increments. Because our focus is on objects that generate contracting flow patterns, and such patterns arise when objects recede in depth at a faster speed than the observer moves forward, we chose to define object trajectory angles in an observer rather than world centered reference frame. This means that in the largest trajectory angle condition (90°), the object maintained a fixed depth relative to the observer as it moved laterally. In the smallest trajectory angle condition (5°), the object's trajectory was nearly parallel to that of the observer.  

One might wonder why we decided to include the 90° object trajectory condition, in which the object maintains a fixed depth as it moves laterally, given our focus on retreating objects. In a previous study (Layton, Mingolla, & Browning, 2012), Layton et al. demonstrated that an earlier version of our model without contraction cells accounts for the heading bias induced by fixed-depth objects that was reported by Royden & Hildreth (1996). However, contraction cells can be activated (albeit weakly) by laminar flow from moving objects that maintain a fixed depth. As such, augmenting the Layton et al. (2012) model to include contraction cells could, in principle, interfere with its behavior such that it no longer captures the human heading bias induced by fixed-depth objects. Simulations with object trajectory angle set to 90° allowed us to test whether the updated model still accounts for the known pattern of heading bias under these conditions. Similar to Royden and Hildreth (1996), we offset the object's initial position. The object was positioned 500 cm in depth and laterally offset at the beginning of the trial by –275 to 25 cm, in 25 cm increments, where negative and positive values indicate placement to the left and right of the heading direction, respectively. The object moved rightward at a speed of 200 cm/s for 1 s, resulting in a final lateral offset that ranged from –75 to 225 cm. In simulations, we used the frontoparallel plane environment to maintain consistency with the experiments of Royden and Hildreth (1996) and the ground plane for comparison with our experiment. 

Simulations were performed within MATLAB 2016a and Equation 14 was numerically integrated using the built-in function ode15s with a time step of 0.05 s. 

Retreating object heading experiment

Participants

Twelve naïve subjects (seven males, five females) from Rensselaer Polytechnic Institute between the ages of 18 and 21 years participated in the study for course credit. All subjects had a valid driver's license and normal or corrected-to-normal vision. The experimental protocol was approved by the Institutional Review Board at Rensselaer Polytechnic Institute and adheres to the Declaration of Helsinki. All subjects gave informed consent in writing before participating in the experiment. 

Apparatus

The visual displays were generated in the WorldViz Vizard 3.0 environment (Worldviz, Santa Barbara, CA) on an Alienware Area 51 desktop computer (Alienware, Miami, FL) equipped with two NVIDIA Ge-Force GTX 480 graphics cards (NVIDIA, Santa Clara, CA), a 3.2 Ghz Intel Core i7 processor (Intel, Santa Clara, CA), 6GB of memory running Microsoft Windows 7 x64. The displays were projected on a large rear-projection screen using a Barco Ciné 8 projector (1280 × 1024 resolution; 60 hz refresh rate; Barco, Rancho Cordova, CA). Subjects sat in a chair approximately 1m away from the rear-projection screen (100°W × 80°H) and viewed the displays binocularly in a dark room. 

Visual displays

Subjects viewed displays (100°W × 80°H) of simulated self-motion (heading angle h = ±5°, ±15° relative to the central axis) along a straight path. Negative and positive heading angles correspond with simulated self-motion toward the left and right sides of the screen, respectively. The simulated observer had an average human eye height of 1.8 m. Translation of the observer was simulated at 5 m/s. 

The moving object consisted of 500 dots superimposed on a cylinder that matched the black sky and glided along the ground plane. Because the object was opaque, it occluded background dots in the environment behind the object within the observer's field of view. The cylinder had a 1 m radius, and was 3 m tall. 

To test the effect of object trajectory angles in the range used in model simulations, we manipulated the object's rate of retreat (0, 0.75, 1.5, 2.25, and 3 m/s relative to the observer) and lateral speed (0.5, 0.7, 1.15, 1.65, and 2.25 m/s) as independent variables. The combination of these two variables determined the trajectory angle, yielding 21 unique values ranging from 9.5° to 90°. Rate of retreat covaried with the initial depth from the observer (5.5, 4.375, 3.25, 2.125, and 1 m) such that final depth from the observer was constant across trials (5.5 m). At the farthest and nearest initial depths, the object subtended approximately 18° horizontally Display Formula 27° vertically and 35° horizontally 50° vertically, respectively. The objects in the 0 m/s rate of retreat condition maintained their depth relative to the observer and resembled the fixed-depth moving objects studied by Royden and Hildreth (1996). As in model simulations, the initial lateral position of the object was fixed at 200 cm relative to the observer's locomotor axis, so the five final object positions were brought about by varying the lateral speed of the object.  

Procedure

At the beginning of the experiment, subjects completed a short (10 trial) basic heading judgment task that consisted of simulated travel along a straight path toward a backplane defined by dots. The practice block contained no independently moving objects and allowed subjects to become familiar with the experimental instructions. The first frame of the trial appeared for 0.5 s at the outset of each trial. At the end of each 1.5-s trial of simulated self-motion, a blue rectangular postmotion probe (4° in the vertical direction) appeared on the screen at a random horizontal position along with the final frame of the optic flow sequence. Subjects aligned the horizontal position of the probe with their perceived heading by manipulating a steering wheel. Turning the wheel left or right moved the probe in the corresponding direction, and subjects confirmed their probe placement by pressing a button located on the front of the steering wheel. Prior to the main experiment, subjects were notified about the presence of a moving cluster of dots that may appear similar to a moving object. Subjects were instructed to ignore the object as much as possible and base their heading judgment on the direction they are moving through the environment. They were allowed to freely move their eyes during the experiment.¹ No feedback was given during the practice or experimental trials. 

The main experiment consisted of three identical blocks that presented trials in a randomized order. Each block contained 200 trials (4 headings × 2 object starting positions on either side of heading × 5 object rates of retreat × 5 object lateral speeds). For each repetition, the heading angle was randomly jittered around the specified angle (θ + X, X ∼ U(−2°, 2°)). Experimental blocks were counterbalanced across subjects and the whole experiment lasted less than 60 minutes. 

Analyses and bias correction

The data were analyzed using repeated-measures analysis of variance (ANOVA). Mauchly's test was used to confirm sphericity in the data and was not significant for any of the analyses. 

Consistent with other heading experiments (Royden & Hildreth, 1996; Warren & Hannon, 1988), subject responses tended to exhibit a constant bias toward the center of the screen (central axis). We calculated a center screen bias correction for each subject, environment, and heading angle by averaging all subject judgments garnered for a set of trials with the same environment and heading angle, and subtracting that value from the data. That is, we computed the corrected response R̄ 

 

where Display Formula is the response of subject Display Formula to heading Display Formula, object trajectory angle Display Formula, starting position Display Formula, and repetition Display Formula. The Display Formula symbol corresponds to the mean, taken with respect to Display Formula, Display Formula, and Display Formula.  

Results

Heading bias and connectivity within model MSTd

We developed four versions of the neural model to test hypotheses about the nature of interactions within area MSTd. Each model generates predictions about how different patterns of connectivity in MSTd should influence heading perception. The no-interaction model tests the no-interaction hypothesis, which posits that expansion and contraction cells generate independent heading signals. The other three models test the interaction hypothesis, which posits that the two neural populations interact. We considered the following forms of interaction: recurrent on-center/off-surround connectivity between expansion and contraction cell populations plus lateral inhibition within each cell population (full-interaction model), lateral inhibition within each population (within-interaction-only model) and lateral inhibition across the populations (across-interaction-only model). Each model was simulated using optic flow generated during forward self-motion in the presence retreating objects. 

To develop the reader's intuitions about the no-interaction and interaction hypotheses, we begin by considering responses of expansion and contraction cells. Within this section, the responses of each population are depicted schematically using Gaussian curves. This allows us to explain in a more intuitive manner how both cell populations respond to different types of optic flow and how their responses are affected when the two populations interact. Actual model MSTd responses are presented in the subsequent section (Heading bias and object trajectory angle). First, we focus on the situation in which an observer heads forward through a static environment in the direction of the vertical meridian (Figure 4a). In this case, expansion cells tuned to centrally positioned FoE should respond most vigorously and the contraction cell population should remain quiescent (Figure 4b). Conversely, if a moving object recedes into the distance away from a stationary observer (Figure 4c), expansion cells should remain quiescent and contraction cells centered on the receding object should respond most vigorously (Figure 4d). 

Figure 4

 

Schematic model responses that illustrate the qualitative predictions of the no-interaction (b, d) and full-interaction (f–h) models of MSTd in the case of self-motion in the presence of a retreating object. (a) The pattern of radial expansion generated by forward self-motion along a central heading maximally activates expansion cells (vertical black line) in model MSTd tuned to the corresponding FoE (b; thick solid black curve). (c) The pattern of radial contraction generated by a retreating object while the observer is stationary maximally activates contraction cells (d; vertical gray line) in model MSTd (d; thick solid gray curve). Because expansion cells do not respond to the contracting object pattern, the expansion cell response in the no-interaction model resembles that in (b) during self-motion in the presence of a retreating object (e). Therefore, the no-interaction model leads to the prediction that the retreating object should not affect heading estimates. (f–h) Recurrent on-center/off-surround connectivity in the full-interaction MSTd model can lead to heading bias through a peak shift in the MSTd activity distribution. Plots in (f) and (g) depict how lateral inhibition (dashed lines in b, and d) within each expansion and contraction model MSTd population, respectively, affects cells that receive the maximal bottom-up excitatory signal bottom-up responses (vertical lines). On-center connections between expansion and contraction cells tuned to the same FoE/FoC have the effect of adding the two distributions in (f) and (g), which yields the distribution in (h), which has an activity peak that is shifted away from the heading direction. This indicates heading bias in the scenario illustrated in (e).

Now let us consider the patterns of activation that are generated by expansion and contraction cells when the observer moves forward in the presence of the retreating object (Figure 4e). According to the no-interaction hypothesis, expansion and contraction MSTd neurons integrate optic flow signals independently. Therefore, the response of expansion cells should not substantially differ in the case of self-motion in the presence of the retreating object (Figure 4e) as compared to that garnered in the static environment (Figure 4a). That is, expansion cells should signal the observer's central heading (Figure 4b) irrespective of the presence of the moving object because the local contracting pattern inside the contours of the object does not stimulate expansion cells and the MSTd populations do not interact. Likewise, the observer's forward self-motion should not influence the response of contraction cells (Figure 4d). 

On the other hand, if expansion and contraction cells interact by way of on-center/off-surround connections, as posited by the interaction hypothesis, then the heading signal derived from the expansion cell activity in the presence of the moving object should differ compared with that derived in the static environment. To develop the reader's intuition on how the retreating object may influence heading signals under the interaction hypothesis, we focus on an important subset of interactions within the model. We remind the reader that the analysis in this section uses a schematic version of the model and that simulation results based on the actual model will be presented in the next section. 

We first focus on the interplay between the bottom-up excitation, indicated by the solid curves (Figure 4b and 4d), and the off-surround inhibitory feedback sent by the most active expansion and contraction cells to those within the same and other population, respectively, indicated by the dashed curves (Figure 4b and 4d). Figure 4f and 4g depict how the balance between the bottom-up excitation and recurrent inhibition from the most active MSTd units influences neighboring cells: Positive values indicate that the bottom-up excitation outweighs the suppression from the most active expansion and contraction cells and negative values indicate that the suppression from recurrent signals outweighs the excitation from bottom-up signals. Turning our attention to the on-center recurrent connections within and across the MSTd populations, the excitatory feedback has the effect of summing the distributions depicted in Figure 4f and g, yielding that shown in Figure 4h. The combined on-center/off-surround interactions result in a shift in the expansion cell peak toward that of the contraction cells (peak shift), which indicates heading bias in the direction of the retreating object. 

To summarize, the interaction hypothesis leads to the prediction that on-center/off-surround interactions within MSTd should give rise to heading bias during forward self-motion in the presence of retreating objects, whereas the moving object should not influence heading signals according to the no-interaction hypothesis. Quantitative predictions based on the actual model will be presented in the following section. We expect heading bias only in the version of the model with full recurrent on-center/off-surround interactions between expansion and contraction cells (full-interaction model). 

Heading bias and object trajectory angle

The peak shift, and therefore the heading bias, that arises in the full-interaction model (Figure 4h) due to on-center/off-surround interactions between expansion and contraction cells depends on the relative strength and proximity of the driving bottom-up signals to either population (Figure 4b and 4d). To quantify the predictions of the interaction hypothesis about human heading bias based on these two factors, we simulated the full-interaction model with optic flow generated through forward self-motion in the presence of an object that retreats along different relative trajectories (Figure 5). 

Figure 5

 

Simulations of the full-interaction model in which an observer moves forward along a central heading over a ground plane (solid) or toward two frontoparallel planes (dashed) in the presence of an object that retreats along different trajectories. The object started to the left of the observer's heading, moved rightward during the trial, and remained close to the observer's future path near the end of the trial. (top panels) Patterns of activation of expansion (black) and contraction (gray) cells in model MSTd at the end of the trial when the object moved along the ranges of trajectories indicated by the thick lines connecting the top panels to the plot below. In the top activity panels, the x-axis indicates the preferred FoE/FoC tuning of each cell, the y-axis shows the corresponding response, the black vertical line indicates the observer's heading, and the dashed gray line indicates the biased heading estimate produced by expansion cells due to the presence of the retreating object. (bottom panel) Bias in model heading estimates for different object trajectories at the end of the trial, defined in the observer's frame of reference. Negative values correspond to bias in the direction of object motion (rightward) and positive values correspond with bias in the opposite direction (leftward).

To understand how mechanisms within the full-interaction model contribute to the predicted pattern of heading bias, we focus on the qualitatively different activation patterns of expansion and contraction cells when the object retreats along three different trajectory angle ranges. Because results were similar in the two visual environments, we will refer to results from the ground plane simulations (Figure 5; solid curve) and point out differences at the end of the section. Beginning with the 5°–10° trajectory angles (Figure 5; left), the contraction cell population peaks to the left of heading. This may seem counterintuitive because the FoC of the flow within the object is to the right of heading. However, the most active contraction cell is not necessarily that which is tuned to the FoC of the retreating object flow pattern. Due to the center-weighted property of MSTd cell receptive fields, sensitivity decreases with distance from the preferred FoC location (Equation 10). As such, when the retreating object is to the left of heading and the FoC of the flow within the object is to the right of heading, the contraction cell that is aligned with that FoC responds weakly. The maximally active contraction cell is one that is tuned to a FoC in between the object's visuotopic position and the FoC of the object flow pattern. For a 9° trajectory angle, this cell is the one that is slightly to the left of heading (see gray curve in left panel of Figure 5). The contraction cell activity peak and the activity of expansion cells tuned to the same preferred FoE/FoC position summate due to the recurrent excitatory connection between the two MSTd populations. The two signals are sufficiently strong to overcome the response of expansion cells tuned to central headings, which shifts the expansion cell peak leftward toward the contraction cell peak. This results in the 1° heading bias in the direction from which the object came shown in Figure 5 (i.e., toward the left in the conditions depicted in left panel of Figure 5). 

For the middle range of object trajectories in Figure 5 (10°–60°), the object moved close to or crossed over the observer's central heading during the trial, which led to peak contraction cell activity to the right of the observer's heading. The summating effect of the interactions among the populations tuned to the same FoE/FoC position coinciding with the contraction cell peak leads to a shift in the expansion cell peak from the center to the right. The off-surround interactions also play an important role in generating the shift through their suppression of expansion cells tuned to central headings that would otherwise be maximally active. Together, this results in a 1.0°–3.5° bias in the direction of object motion (i.e., toward the right in the condition depicted in middle panel of Figure 5). 

In the final range (60°–90°), the object crossed and cleared the observer's heading. While this activated contraction cells tuned to FoCs positioned predominately to the right of the central heading, the laminar optic flow within the contours of the object weakened the overall population response, which, compared with the middle 10°–60° range, diminished the peak shift and reduced the heading bias. Laminar object flow does not strongly activate expansion or contraction MSTd cells because the response of model MSTd cells depends most crucially on the motion vectors nearby the preferred FoE/FoC position. This is because nearby motion vectors are weighted more heavily than those located further away (Equation 10). More complete, radially symmetric patterns (Figure 1b) should activate model MSTd cells to a greater extent than those that contain more parallel, laminar vectors (Figure 4c), all other factors being equal. 

The pattern of heading bias in the frontoparallel plane environment also can be characterized by three distinct ranges of trajectory angles (Figure 5; dashed curve). Although the 60°–90° range and −3.5° peak bias remained consistent across the two environments, the transition from positive to negative bias occurred at a larger trajectory angle—∼30° in the frontoparallel plane environment compared with ∼10° in the ground plane environment. To understand why the trajectory angle at which this sign change occurred differed across the two visual environments, consider the pattern of MSTd activity above the middle range of trajectory angles (10°–60°), within which contraction cells generate a large activity peak nearby the heading direction. Background optic flow from the frontoparallel planes stimulates a greater area of MSTd receptive fields, resulting in a sharper expansion cell activity peak than in the ground plane environment. The farther apart the activation levels among the two populations are, the more immune the expansion cells are to the influence from contraction cells. Thus, the activity of expansion cells in the frontoparallel environment resists undergoing a peak shift for a larger range of trajectory angles than in the ground plane environment. 

Alternative models

We repeated simulations from Figure 5 with versions of the model with more restricted connectivity within MSTd (no-interaction, within-interaction-only, and across-interaction-only) to determine whether a simpler model yields similar heading bias predictions. Unlike the full-interaction model, the no-interaction, within-interaction-only, and across-interaction-only models only produced weak heading bias for different object trajectory angles (Figure 6). The no-interaction model did not produce any heading bias at all, whereas the bias produced by the within-interaction-only model tended to be in the direction of object motion and the bias produced by the across-interaction-only models tended to be in the opposite direction of object motion. Only the full-interaction model yielded positive heading bias for small object trajectory angles. These simulations suggest that the pattern of heading bias in the full-interaction model is the outcome of balanced excitation and inhibition afforded by recurrent connectivity within and between expansion and contraction populations. 

Figure 6

 

Simulations of the neural model of MSTd with different connectivity configurations among expansion and contraction cells under the same conditions as the ground plane simulations shown in Figure 5. Predictions from the full-interaction model (Figure 5) are overlaid in gray for comparison. The bias reflected model activity at the end of the trial.

In summary, the full-interaction model of MSTd leads to the prediction that heading perception is biased in the presence of retreating objects, whereas versions of the model with more restricted connectivity predict at most a weak bias. 

Heading bias and fixed-depth objects

The 90° object trajectory condition resembles the fixed-depth condition studied by Royden and Hildreth (1996), which is known to bias human heading judgments in the direction of object motion. A model with recurrent on-center/off-surround interactions between expansion cells similar to the one presented here, explained the influence of the fixed-depth object on human heading judgments, but it did not include contraction cells (Layton et al., 2012). Therefore, we performed simulations to confirm that the addition of contraction cells does not interfere with the model's ability to capture the human data. Similar to the conditions within the Royden and Hildreth study, we offset the object's initial position relative to the observer's future path². 

Figure 7 shows the heading bias produced by the full-interaction model when the center of the object started and ended at the positions indicated along the x-axis relative to the observer's future path. Consistent with the findings of Royden and Hildreth (1996), the heading estimates produced by the model are biased when the object occludes the observer's heading for large portions of the trial. As the top panels show, the fixed-depth object does not strongly activate contraction cells in any circumstance. Therefore, the bias generated by the model must arise from recurrent interactions among expansion cells, consistent with previous findings (Layton et al., 2012) and bias obtained by the within-interaction-only model in the case of the 90° object trajectory (Figure 6). 

Figure 7

 

Simulations of the full-interaction model in the case of a fixed-depth object (90° trajectory angle). The center of the object was laterally displaced such that it started and ended in the positions indicated beneath the bottom plot, where negative and positive values correspond to placement to the left and right of the observer's heading. Plot conventions are identical to those in Figure 5. (top panels) Activation patterns of expansion (black) and contraction (gray) cells in model MSTd at the end of the trial when the object started within different ranges of lateral positions, as indicated by the thick lines connecting the panels to the plot below. The model produces heading bias in the direction of object motion when the object moves nearby the observer's heading at the end of the trial (center top panel).

Experimental results

We performed a psychophysical experiment to determine whether human heading judgments are compatible with any versions of model MSTd. Subjects viewed simulated self-motion in the presence of a retreating object under conditions that resembled those used in model simulations. A nonsignificant four-way (heading × starting side × object retreat rate × lateral object speed) repeated-measures ANOVA prompted us to collapse across heading. A follow-up three-way ANOVA revealed no significant main effect, F < 1, of starting side of the object and no significant interactions. Consistent with the predictions of the full-interaction model, there was a significant rate of retreat × lateral speed interaction, F(16, 176) = 3.25, p < 0.01, Display Formula . Because rate of retreat and lateral speed were manipulated in the experiment to test a range of trajectories, we combined these variables into object trajectory angle for subsequent analyses.  

Figure 8 shows the pattern of bias in human heading judgments for different retreating object trajectories. When the object trajectory aligned more closely with the observer's heading (smaller trajectory angles), mean heading judgments were biased in the direction opposite the object's motion (positive in Figure 8) (e.g., leftward if the object moved from left to right). For more oblique object trajectories (larger trajectory angles), the direction of bias in the mean human judgments changed over toward the direction of object motion (negative in Figure 8) (e.g., rightward if the object moved from left to right). To quantify the overall pattern, we fit a simple quadratic model, which yielded the largest absolute bias of −1.23° when the object moved along a 61° trajectory (Display Formula ). The full-interaction model produced a similar pattern of heading bias, resulting in a peak negative bias of −3.5° for a 55° object trajectory, when we simulated conditions that resembled the psychophysical conditions (Figure 5).  

Figure 8

 

Human heading judgments from the psychophysical experiment. Data correspond with the heading bias averaged across subjects and error bars indicate ±1 SE. Conditions in the experiment were similar to those used in simulations (Figure 5). The black curve shows a quadratic fit.

Discussion

In the present article, we investigated the functional outcomes of recurrent connectivity between expansion and contraction cells in primate MSTd. We used a neural model to generate predictions about the consequences that recurrent on-center/off-surround connectivity within and between expansion and contraction cell populations would have on heading signals produced in the case of forward self-motion in the presence of retreating objects. Our simulations demonstrated that full on-center/off-surround recurrent connectivity gives rise to a biased heading signal that depends on the relative trajectory of the retreating object. Heading bias emerged in the model through a peak shift that occurred when the expansion and contraction cells with similarly tuned FoE/FoC positions were both active. Competition between cells within and between each population shifted the heading signal produced by the distribution of expansion cells. The heading bias was greatest when the object trajectory angles relative to heading were small (i.e., object and observer trajectories were nearly parallel)—conditions in which there was a great deal of overlap between the expansion and contraction cell activity distributions in model MSTd. Heading judgments from the psychophysical experiment were only consistent with the pattern of heading bias generated by the full-interaction model of MSTd that included on-center/off-surround interactions within and between expansion and contraction cell populations. 

The peak shift mechanism follows from the lateral inhibition implicated from the model's recurrent connectivity within and between the two MSTd distributions and provides a parsimonious explanation of the pattern of human heading judgments (see Figure 4). A peak shift has been shown in other models to account for a wide range of phenomena, including the transformation of absolute to relative disparity from V1 to V2 (Grossberg, Srinivasan, & Yazdanbakhsh, 2011b), the line neutralization visual illusion (Levine & Grossberg, 1976), and the motion aperture problem (Grossberg, Léveillé, & Versace, 2011a). We emphasize that our analysis identifies the peak shift as a plausible mechanism—other mechanisms may exist and the link between recurrent connections and heading perception should continue to be investigated in future work. 

Recurrent mechanisms and the stability of heading perception

The recurrent mechanisms identified here have implications that extend beyond the interactions between expansion and contraction cells in MSTd—namely the role they play in stabilizing heading perception over time. Most models of primate MSTd that specifically address moving objects do not include recurrent on-center/off-surround interactions and do not consider how heading perception may evolve over time. To capture the pattern of human heading, such models rely on the pooling of MT motion signals, differential motion between the moving object and its background, or a combination—computations that depend on the instantaneous optic flow field. These models predict large heading errors that fluctuate wildly over time as objects occlude the observer's future path (Layton & Fajen, 2016a). In contrast, human heading perception is remarkably stable and robust, even when moving objects occupy large portions of the visual field and cross the future path. As the results of the present psychophysical experiment show, mean human heading errors only reach several degrees of visual angle. When the behavior of different models, including motion pooling (Warren & Saunders, 1995), differential motion (Royden, 1997; 2002), and a variant of the model presented here (Layton & Fajen, 2016a) (Competitive Dynamics model), was tested in a range of dynamic environments, only the Competitive Dynamics model with its recurrent interactions captured the robustness of human heading judgments over time (Layton & Fajen, 2016a). As shown in Figure 6 and in Layton and Fajen (2016a), the Competitive Dynamics model without recurrent connectivity could not account for the pattern of human judgments and heading estimates wildly fluctuated over time. Together, these simulations suggest that recurrent on-center/off-surround interactions may serve as a plausible mechanism by which MSTd stabilizes heading estimates and reduces errors due to transient visual interruptions, such as those caused by large moving objects. 

Performance of other heading models in the presence of retreating objects

Given that other biological models have been developed to address heading perception in the presence of moving objects, it is important to evaluate whether the mechanisms in existing models could account the pattern of human judgments. As mentioned already, models that match motion signals with global motion templates and do not possess interactions within MSTd should generally yield accurate heading estimates in the retreating object scenario (Hatsopoulos & Warren, 1991; Perrone & Stone, 1994). Those that pool motion over regions of the visual field (Warren & Saunders, 1995) may generate heading bias, but in the opposite direction of human judgments (Layton & Fajen, 2016a). Other models perform differential motion computations (Beintema & Van den Berg, 1998; Hildreth, 1992; Royden, 1997; 2002) or decompose the optic flow field into translational and rotational components (Heeger & Jepson, 1992; Lappe & Rauschecker, 1993; 1995), both of which involve subtraction (Royden & Conti, 2003) so we consider their predictions together. A mechanism that subtracts signals from units with nearby receptive fields at or before the level of MSTd may produce heading errors in the direction of object motion for the retreating object scenarios considered here, which is broadly consistent with human judgments. This is because the difference vectors obtained by subtracting object and background motion vectors around the perimeter of the object should intersect at a location offset from the observer's heading in the direction of object motion. Analysis by Royden (2002) showed this to be the case for the fixed-depth object and we have shown elsewhere that the Royden model demonstrates such a bias for objects that retreat at other trajectory angles (Layton & Fajen, 2016a). The overall predicted pattern of bias for a subtractive mechanism may, however, deviate from human judgments in several ways. First, a subtractive mechanism should generate accurate heading estimates when the object FoC aligns with the FoE (0° trajectory angle) and bias in the direction of object motion that increases as the trajectory angle approaches 90°, all other factors remaining equal. Considering that human bias diminishes at large trajectory angles (Figure 8), this prediction was not entirely supported. Second, it is unclear how a subtractive mechanism would account for the reversal in bias direction at small retreating trajectory angles since the bias predicted from the model should grow monotonically from zero. As our analysis shows, the bias drop-off at large trajectory angles and switching bias directions when the FoC and FoE positions are close together could be explained by interactions between expansion and contraction MSTd cells (Figure 4). Additional computational and psychophysical work should be performed to illuminate the involvement of one or a combination of the mechanisms. 

Functional implications of on-center/off-surround connectivity in MSTd

The type of on-center/off-surround recurrent organization proposed by our model MSTd has possible consequences that extend beyond heading perception. On-center/off-surround connections may play an important role in controlling locomotion, for example, when pursuing and attempting to intercept a retreating target. Consider the scenario illustrated in Figure 9 wherein the observer “locks on” the target such that the heading direction coincides with the direction of object retreat (left; Figure 9). The on-center excitatory feedback creates a type of “resonance” between the maximally active expansion and contraction cells in model MSTd, which increases the expansion cell response above the level when the object is absent. Should the target or observer suddenly change course (middle; Figure 9), the signal generated by maximally active expansion sharply drops to or below levels encountered during self-motion without a moving object. This sudden change may serve as a signal that target pursuit is failing and the observer must change his course in order to successfully catch the target. Indeed, once heading realigns with the target's direction of retreat, the expansion response returns to levels encountered prior to the target slip (right; Figure 9). 

Figure 9

 

Scenario in which on-center/off-surround recurrent connectivity between expansion and contraction cells might provide a signal to control locomotion. (left) An observer chases a retreating target and aligns heading with the target's direction of retreat (“target locked on”). The optic flow field (top left) activates the expansion cell tuned to a centrally positioned FoE (solid black curve) to a greater extent than the contraction cell tuned to the FoC in the same visuotopic location (solid gray curve). (center) If the target's trajectory changes (“target slips”), the response of the previously active contraction cell suddenly decreases and the response of the maximally active expansion cell decreases to comparable levels experienced during self-motion without a retreating target (dashed curve). This is because the expansion cell no longer receives excitatory signals from the formerly active contraction cell and the contraction cell that signals the target's shifted trajectory laterally inhibits the expansion cell. (right) If the observer moves to realign the heading direction with the target's trajectory (“target reacquired”), activation levels are restored among the expansion and contraction cells. The sudden decrease in the maximal expansion cell response when the target slips could form the basis of a signal used by other cortical areas to coordinate steering when the pursuit of a target is failing.

When steering based on optic flow, active control (Page & Duffy, 2008) and strategy (Kishore, Hornick, Sato, Page, & Duffy, 2012) may substantially alter MSTd responses. MSTd neurons may trade off FoE/FoC pattern selectivity for enhanced spatial precision in localizing the center of motion, irrespective of whether it is a FoE or FoC, to facilitate performance (Jacob & Duffy, 2015). On-center/off-surround connectivity between expansion and contraction cells may provide a means for top-down signals to flexibly shape the MSTd population activity around the demands of the task (Layton & Browning, 2012). For example, top-down signals may enhance the gain of on-center/off-surround connections within expansion and contraction cell populations when a task demands fine heading discrimination or decrease it when resolving the direction of locomotion is a priority. 

Model MT and MSTd in a broader context

Although the model accounts for human heading judgments in the presence of retreating objects, its parsimony excludes characteristics of MSTd neurons that may play a role in heading perception. Model MSTd contains units that respond to radial expansion/contraction, but MSTd neurons also exhibit tuning to spiral and rotation patterns ((Graziano, Andersen, & Snowden, 1994), which may reflect sensitivity to the broad and complex range of motion encountered during ordinary self-motion. MSTd receptive fields are likely even more complex than such coherent global motion patterns suggest (Mineault, Khawaja, Butts, & Pack, 2012). We did not include tuning to this greater diversity of motion since the self-motion scenario considered here predominately activates cells tuned to radial expansion/contraction, not those tuned to spiral or rotation patterns (Layton & Browning, 2014). This is because linear self-motion in the presence of a linearly moving object does not generate spatial or temporal curvature in the optic flow field (Raudies, Ringbauer, & Neumann, 2013). 

MT neurons also exhibit a considerably greater diversity in their receptive field organization than the cells included within the model that integrate speed and direction throughout the receptive field (Cui, Liu, Khawaja, & Pack, 2013). Many receptive fields contain not only a region that stimulates the neuron when motion in the preferred direction appears, but regions wherein motion suppresses the response (Allman, Miezin, & McGuinness, 1985; Born & Tootell, 1992). Such neurons with antagonistic receptive fields appear to project to the lateral portion of ventral MST (MSTv) (Berezovskii & Born, 2000), which contains neurons that respond to the movement of small moving targets (Tanaka, Sugita, Moriya, & Saito, 1993). The extent to which neurons within MSTd and MSTv interact is unclear, but previous models have hypothesized connectivity for the purposes of tracking objects during pursuit eye movements (Pack, Grossberg, & Mingolla, 2001). 

Heading and multi-sensory signals

Although the model and experiment presented here focus on heading perception from optic flow, there is considerable evidence that heading perception represents a multi-sensory process (Gu et al., 2012). In particular, MSTd neurons tuned to heading derived from optic flow signals exhibit sensitivity to signals present during self-motion from vestibular (Gu, Angelaki, & DeAngelis, 2008) and proprioceptive (Cullen, 2012) systems. The functional redundancy afforded by the presence of heading signals from multiple sensory modalities may mitigate the bias reported here during ordinary locomotion. This may occur by combining heading estimates derived from multiple parallel pathways, such as those involving area V6 in humans (Pitzalis, Fattori, & Galletti, 2013). Signals from nonvisual sensory systems may help disambiguate self- and object motion, but because the object may occlude the heading direction and nonvisual heading tuning is coarse (Froehler & Duffy, 2002), the interactions among MSTd neurons likely still lead to, albeit weaker, biased heading estimates. 

Approximately half of multisensory (visual and vestibular) neurons in dorsal MST (Gu, 2006; Gu et al., 2010) and VIP (Chen, DeAngelis, & Angelaki, 2011) show “congruent” visual and non-visual self-motion directional preferences. The other half exhibit “opposite” directional preferences. An “opposite” cell may fire when the observer moves forward, but experiences optic flow consistent with backwards self-motion. Inconsistent visual and nonvisual self-motion signals naturally occur when moving in the presence of approaching or retreating independently moving objects. Signals generated from opposite cells are ideally suited for determining whether visual optic flow is attributed to observer or object motion, and may decrease heading bias in the presence of moving objects (Sasaki et al., 2013). Indeed, the presence of inertial signals mitigated heading biases that arose when monkeys performed a visual heading discrimination task in the presence of a moving object (Dokka, DeAngelis, & Angelaki, 2015). Future work should investigate how vestibular and proprioceptive signals can facilitate the recovery of heading. 

Acknowledgments

This work was supported by the Office of Naval Research (ONR N00014-14-1-0359). The authors thank Ennio Mingolla for insightful discussions and anonymous reviewers for their helpful feedback on earlier drafts of the manuscript. 

Commercial relationships: none. 

Corresponding author: Oliver W. Layton. 

Email: . 

Address: Department of Cognitive Science, Rensselaer Polytechnic Institute, Troy, NY, USA. 

Footnotes

¹  We restricted eye movements in a previous study (Layton & Fajen, 2016b) and found no evidence that the biases induced by moving objects were due to eye movements generated while tracking the object.

Footnotes

²  Royden and Hildreth (1996) varied the object's starting position and the observer's heading independently from one another, whereas we only varied object's starting position.