Biological motion cues aid identification of self-motion from optic flow but not heading detection

Abstract

When we move through the world, a pattern of expanding optic flow is generated on the retina. In completely rigid environments, this pattern signals one's direction of heading and is an important source of information for navigation. When we walk towards an oncoming person, the optic environment is not rigid, as the motion vectors generated by the other person represent a composite of that person's movement, his or her limb motion, and the observer's self-motion. Though this biological motion obfuscates the optic flow pattern, it also provides cues about the movement of other actors in the environment. It may be the case that the visual system takes advantage of these cues to simplify the decomposition of optic flow in the presence of other moving people. The current study sought to probe this possibility. In four experiments self-motion was simulated through an environment that was empty except for a single, walking point-light biological motion stimulus. We found that by using biological motion cues, observers were able to identify the presence of self-motion despite the lack of stable scene information. However, when estimating heading based on these stimuli, the pattern of observer heading estimates could be approximately reproduced by computing the vector sum of the walker's translation and the stimulated self-motion. This suggests that though biological motion can be used to disentangle self-motion in ambiguous situations, optic flow analysis does not use this information to derive heading estimates.

Introduction

Imagine meeting another person on the street; the person walks towards you while you move towards them. The retinal motion you experience is a combination of two motion components: your own self-motion and the biological motion of the other person. The purpose of the current study is to investigate how the visual system differentiates these two components. 

Locomotion through the world generates a pattern of expanding visual motion on the retina known as optic flow (Gibson, 1950). If the world is entirely stable and does not contain any additional motion, this pattern can be decomposed to provide information about the observer's movement, such as the direction in which one is heading (Longuet-Higgins & Prazdny, 1980). Optic flow is thus an important source of information for visually guided navigation. 

Humans can use optic flow to detect their heading in a number of situations and with a degree of accuracy that allows for safe locomotion (Cutting, Springer, Braren, & Johnson, 1992; Lappe, Bremmer, & van den Berg, 1999). This level of accuracy is maintained even when the visual stimulus contains perturbations induced by eye movements (Li & Warren, 2000; Warren & Hannon, 1990). Warren, Morris, and Kalish (1988) showed that heading estimation was fairly reliable in scenes containing as few as 10 points, but dropped when only two points were visible. Heading estimation also deteriorates when the direction of dots in the flow field is randomized, but remains stable if the speed of individual dots is randomized while keeping the configuration of the flow pattern intact (Warren, Blackwell, Kurtz, Hatsopoulos, & Kalish, 1991). This suggests that the critical information for heading detection lies in the global structure of the optic flow field. 

The importance of the global structure of the flow field for heading estimation is further supported by neurophysiological evidence showing that neurons in middle temporal (MT) and medial superior temporal (MST) areas, which are likely responsible for heading detection (Peuskens, Sunaert, Dupont, Van Hecke, & Orban, 2001), have large receptive fields that are responsive to motion in sizable portions of the visual field (Duffy, 1998; Duffy & Wurtz, 1995; Lappe, Bremmer, Pekel, Thiele, & Hoffmann, 1996; Peuskens et al., 2001; Smith, Wall, Williams, & Singh, 2006; Tanaka & Saito, 1989; Yu, Hou, Spillmann, & Gu, 2017). It is also mirrored by models of optic flow processing, which generally account for heading estimation by pooling motion vectors over large portions of the scene (Bruss & Horn, 1983; Calow, Krüger, Wörgötter, & Lappe, 2005; Lappe & Rauschecker, 1993; Perrone & Stone, 1994). 

The estimation of self-motion from optic flow relies on the stability and rigidity of the global flow pattern (Bruss & Horn, 1983; Longuet-Higgins & Prazdny, 1980). In ecological situations, however, humans often move in conjunction with, or towards other people. In such cases the scene is not rigid, as the movement of another person in the world also produces a characteristic pattern of visual motion. This pattern is referred to as biological motion and consists of both the translation of the other person, as well as the articulation of their limbs (Johansson, 1973). 

From the point of view of heading perception, the nonrigidity produced by the locomotion of an oncoming walker provides a potential source of confusion. Any movement of a point in the environment that is in addition to the self-motion of the observer impairs the usefulness of that point for heading estimation, as the visual motion of this point is an ambiguous combination of two sources of movement. Without knowledge about the movement of the point in the environment, it is not possible to ascertain how much of its retinal motion is due to self-movement. Several studies have shown that even single objects moving externally from an observer affect heading estimation (Layton & Fajen, 2015, 2016; Royden & Hildreth, 1996; Warren & Saunders, 1995), and that the addition of multiple translating objects (Andersen & Saidpour, 2002) or random motion components (van den Berg, 1992; Warren et al., 1991) to a scene is also deleterious to heading estimation. 

In the studies cited above, moving objects were simple shapes, such as squares, points or polyhedrons. Biological motion has the additional level of complexity induced by limb motion, which adds further spurious point motion to the walker's movement in the environment. On the one hand, this limb motion complicates the optic flow pattern. On the other hand, however, the articulated movement of the body in biological motion conveys information about the source of motion, its direction, and its speed. 

Biological motion carries an abundance of information about the movement of actors in the environment. Even when biological motion is reduced to several points attached to the main joints of an actor, a moving person can still be readily recognized (Johansson, 1973). The visual system is highly sensitive to these so-called point light (PL) stimuli, and they can be used to depict a wide range of complex actions (Dittrich, 1993). In addition, the properties of an action such as its speed, direction, or intention can be deduced (Blakemore & Decety, 2001; Lange & Lappe, 2006; Troje & Westhoff, 2006) and future actions can be predicted based on the immediately preceding movements (Diaz, Fajen, & Phillips, 2012). 

These attributes provide information about a person's movement in the environment. For example, Jackson, Warren and Abernathy (2006) showed that rugby players can predict direction changes of other players based on the pattern of their body kinematics and that deceptive body kinematics can adversely affect novice players' judgments. Similarly, studies have shown that there is an intrinsic link between a walker's articulation and its translational motion. Translation biases the perception of a walker's facing and walking direction (Masselink & Lappe, 2015), as well as the perceived action (Thurman & Lu, 2016). Translation also causes PL walkers to appear as animate actors (Thurman & Lu, 2013). Because biological motion cues provide information about a walker's movement through a scene, we suggest that they could potentially be used to facilitate the estimation of heading during locomotion towards other walkers. 

Prior research on both optic flow and biological motion has used PL stimuli as a way to study the purely motion-based processes involved in perceiving self-motion and the motion of other people (Gibson, 1950; Johansson, 1973; Warren & Hannon, 1988). By removing all other features of the stimuli, the signals available to the visual system are clearly defined and constrained. While this does not address the natural, full-cue situation, it allows a precise investigation of the particular mechanisms that contribute to the perception of natural scenes. This research has shown that both self-motion perception (reviewed in Lappe, Bremmer, & van den Berg, 1999) and biological motion perception (reviewed in Blake & Shiffrar, 2007) are supported by the information in point-light stimuli. These stimuli have helped to explain the perceptual mechanisms underpinning biological motion and optic flow processing. Importantly, research has shown that the mechanisms supporting these two motion percepts are quite different. Self-motion perception relies on an analysis of the invariant pattern of motion vectors of points in the environment that is produced by the moving observer (Lappe & Rauschecker, 1993; Longuet-Higgins & Prazdny, 1980; Perrone & Stone, 1994). Conversely, biological motion perception relies on prior knowledge about the structure and movement possibilities of the human body (Beintema & Lappe, 2002; Giese & Poggio, 2000; Lange & Lappe, 2006) and can be supported by the characteristic movement trajectory of even a single foot point (Chang & Troje, 2009a; Mather, Radford, & West, 1992; Troje & Westhoff, 2006). 

How the visual system processes the combination of motion components produced by concurrent biological motion and optic flow can also be studied using PL stimuli. Locomotion towards a PL walker in an otherwise dark environment produces a stimulus that is an ambiguous combination of self-motion and walker motion. That is to say that all motion vectors in the scene correspond to some combination of walker movement and observer translation. In this case, the visual system is faced with the task of disentangling the visual motion produced by the observer, the motion produced by the other person, and the motion produced by that person's appendages. This cannot be achieved without some access to biological motion perception. The question we ask is whether the biological motion perception system provides such information. 

Though one might argue that heading estimation in the presence of biological motion represents an overly specific situation, it is one of the more frequently encountered sources of external motion in natural environments. Consequently, the brain has evolved a specialized visual network for biological motion processing, which is distinct from the network responsible for heading detection. The superior temporal sulcus (STS) is often cited as the key region involved in biological motion processing (Grossman & Blake, 2002; Grossman et al., 2000). In addition, there is evidence that the biological motion network also recruits areas involved in both form processing, such as the extrastriate body area (Downing, Jiang, Shuman, & Kanwisher, 2001; Grossman et al., 2000), the fusiform body area (Michels, Lappe, & Vaina, 2005; Schwarzlose, Baker, & Kanwisher, 2005) and the occipital face area (Grossman & Blake, 2002; Michels et al., 2005), and motion perception, for example MT and the kinetic occipital area (Grossman et al., 2000). 

While optic flow and biological motion stimuli share some common features, the mechanisms responsible for their processing are largely independent. As mentioned previously, heading is derived from the pattern of image motion on the retina, and is invariant to the particular objects in the visual field (Geesaman & Andersen, 1996; Logan & Duffy, 2006), while biological motion relies heavily on the specific form of the human body (Beintema & Lappe, 2002; Giese & Poggio, 2000; Hoffman & Flinchbaugh, 1982; Lange & Lappe, 2006). Given that biological motion and optic flow processing employ largely separate neural networks and operate using distinctly different mechanisms, it is reasonable to suppose that they do not interfere with one another during locomotion. 

In theory, it is possible for biological motion to be processed in parallel with the optic flow field and combined to facilitate optic flow decomposition. We suggest that biological motion could aid in the computation of optic flow in a number of ways. For example, biological motion cues could signal the presence of nonrigid motion in the environment that is independent from the observer's self-motion. This is plausible, given that previous research has shown biological motion is more likely to be perceived as animate than nonbiological motion (Chang & Troje, 2008; Thurman & Lu, 2013), and that the detection human motion is more efficient than the detection of mechanical motion in natural scenes (Mayer, Vuong, & Thornton, 2015, 2017). Another possibility is that knowledge about biological motion in the scene could be used to estimate the translation of the oncoming person and reduce its impact on the estimation of self-motion, consequently improving estimates of heading. 

The current study investigates the perception of self-motion in the presence of oncoming biological motion. In four experiments observers were presented with a stimulus that displayed a PL walker moving towards the observer, while at the same time self-motion of the observer was simulated towards the walker. Other than the PL figure, the scene was empty. Because the scene lacked additional rigid environmental information, the available visual input represented an ambiguous composite of self- and biological motion. As such, heading and self-motion could only be accurately estimated if biological motion cues were used to detect the walker's motion as separate from the observer's translation. Thus, these stimuli provide a means for studying how the visual system decomposes scenes of complex visual motion. 

In the first two experiments, observers were required to determine whether or not the stimulus contained self-motion. In a third task observers were required to report whether they perceived walker motion, self-motion, or a combination of both. The purpose of these experiments was to ascertain whether or not self-motion and walker motion can be separately identified in ambiguous situations based on biological motion cues. In the fourth task, observers were required to estimate their heading direction. The aim of this final experiment was to assess the accuracy with which observers were able to determine heading based on the combined biological and self-motion stimulus. 

General methods

Materials

Stimuli for all experiments were generated on an Apple MacBook Pro equipped with a 512MB Intel HD Graphics 4000 on-board graphics card. A VDC Display Systems Marquee 8500 projector projecting onto a 250 × 200 cm backlit screen was used to present the stimuli. The projector operated at a resolution of 800 × 600 pixels, with a refresh rate of 60 Hz. Stimuli were designed using Matlab (Mathworks) with the Psychophysics Toolbox (version 3) add-on (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). OpenGL libraries (version 2.1, adapted for Matlab) were used for rendering the stimuli. Responses for Experiments 1, 2, and 3 were signaled using a keyboard, whereas responses for Experiment 4 were signaled using a single-button mouse. 

PLW generation

Eleven PL walkers were derived from the 3D motion tracking data of different subjects (four female, seven male) who were instructed to walk at a natural pace during recording (De Lussanet et al., 2008; Kuhlmann, De Lussanet, & Lappe, 2009). Walking speeds ranged from 0.62–1.26m/s, with a mean walking speed of 1.01 m/s (SD = 0.18). During the experiments the original translation speed and articulation rate of the walkers was kept to create natural variation in object speed, making it difficult for observers to base their estimates simply on the final position of the walker. The PLWs themselves consisted of 12 white points corresponding to the left and right ankle, knee, hip, hand, elbow, and shoulder joints. Each point subtended 0.74° of visual angle. The individual points did not expand during the experiments. Rather, the size of the walker, as defined by the distance between individual points, was scaled using perspective projection to create retinal expansion and produce the percept of an approaching figure. 

In the four experiments presented here, three types of walker articulation were presented: forwards walking, backwards walking, and single-posture. Forwards walking PL walkers had articulation patterns that depicted normal forwards walking as described above. To produce backwards walking, the articulation pattern for forwards walking PL walkers was simply reversed. For single-posture walkers, all articulation information was removed, such that walkers maintained a single pose. In all three articulation conditions, walkers always translated through the scene at their natural speed. In Experiment 2 the direction of translation was manipulated, such that walkers translated in the direction of their articulation in some trials and in the opposite direction in others. In the other experiments walkers always translated in their facing/articulation direction. 

Scene

The scene will be described using a coordinate system with its origin located at the position of the virtual “camera” through which the scene was viewed. The z axis extends in depth away from the camera towards the center of the display. The y axis extends vertically while the x axis extends horizontally. Angles given in the following paragraph are measured with respect to the z axis (see Figure 1B for a bird's eye view of the stimulus). 

Figure 1

 

The task used in experiments. (A) Depicts an example frame as seen by the observer with the various possible self-motion and walker motion combinations shown. The blue arrows depict the possible paths of simulated observer motion. The orange arrows depict the possible paths of the walker. The walker's limbs are highlighted in purple to help the reader identify its underlying form. These lines were not present in the experiments. (B) Depicts the task from above. The position of the camera through which the scene was viewed is depicted by the blue character. Dashed arrows show heading directions for the camera and translation paths for the walkers. The center of the display is denoted by the position 0°.

The scene in all experiments consisted of a single, randomly selected PL walker positioned either centrally, or 5° of visual angle left or right of the center of the display. Walkers were projected to appear at an initial depth of 4 m from the observer. The walker was oriented such that it faced opposite to the z axis (i.e., towards the camera, 180° orientation), or 5° right (185° orientation) or left of it (175°orientation). 

In self-motion-present conditions, ego-motion of the observer was simulated by moving the camera through the scene in a particular heading direction. In these conditions, three heading directions were presented: along the z axis (0°), 5° left (355°), or right of it (5°). The camera was moved at a speed of 1 m/s. A diagram of the task is shown in Figure 1. Example self-motion-present and self-motion-absent trials can also be observed in Movie 1. 

 

Movie 1

 

(A) An example trial depicting the self-motion absent condition.

 

Movie 1

 

(B) An example trial depicting walker motion in combination with self-motion.

In Experiments 1 and 4 all possible combinations of walker position, walker orientation, and heading direction were presented an equal number of times, but in a random order for each observer. In Experiment 2 random combinations of walker position and walker orientation were generated for each trial. In Experiment 3 random combinations of walker position, walker orientation, and heading direction were generated for each trial. 

Experiment 1

As described in the Introduction, PL biological motion contains several cues that could potentially be used to disentangle the complicated pattern of optic flow produced when an observer moves towards a walker. The purpose of Experiment 1 was to assess whether observers could identify the presence of self-motion in scenes containing an ambiguous combination of self- and biological motion. This is an important first step for the decomposition of an optic flow pattern and is required for more complex computations such as heading estimation. 

Because the scenes in the current experiment contained no additional rigid background, it would be impossible to identify self-motion without relying on biological motion cues. However, if observers were to take into account biological motion cues, self-motion could be estimated by comparing the observed scene motion with the motion of the walker. If the pattern of a walker's articulation were to account for its translation, one could conclude that all motion in the scene can be attributed to the activity of the PL walker, and that self-motion is therefore not present. On the other hand, if there were a mismatch between the expected motion of the walker and the observed motion, one would conclude that the extra motion in the stimulus is due to self-motion. 

Observers were presented with a scene that contained a single PL walker that walked towards the observer. In half of the trials, self-motion was simulated, whereas in the other half of the trials self-motion was absent. The observer's task was to identify whether or not self-motion was present. It was hypothesized that observers would be able to use discrepancies between the expected walker motion based on biological motion cues and the observed motion to identify self-motion. 

Methods

Observers

Ten students were recruited from the University of Münster to participate in the current study. In addition, both of the authors also participated in the study, resulting in a total of 12 observers (four male, eight female; age M = 27.92, SD = 8.14). All observers had normal or corrected to normal binocular visual acuity. Apart from the authors, all observers were naïve to the aims of the study. Participation was voluntary, and participants were compensated for their time with course credit. Ethical approval for the testing of all subjects was obtained from the ethics board of the Department of Psychology and Sport Science of the Westfälische Willhelms Universität, Münster, Germany prior to testing. 

Procedure

Prior to each experimental trial, observers were presented with a 500 ms blank interval, which was replaced by a centrally presented fixation-cross for 300 ms. After the fixation-cross disappeared, it was replaced by the trial scene with a pseudorandomly selected combination of walker position and walker orientation. On 50% of the trials both self-motion and PL walker motion were present; on the other 50% of trials only walker motion was present. On the self-motion trials, heading directions were also pseudorandomly selected. Movement of the scene began immediately after the stimulus appeared and lasted for 2000 ms. After the movement phase was complete, the scene disappeared again, and observers were required to respond by pressing either the right shift key of the keyboard if they perceived self-motion, or the left shift key if they did not perceive self-motion. No restriction was placed on response times. 

Each of the potential position-orientation-heading condition groupings was presented 20 times, with 10 of these trials containing self-motion and the other 10 containing no self-motion. This resulted in a total of 540 trials. In addition, participants were given 10 randomly selected practice trials prior to beginning the experiment to ensure that they understood the task. In total the experiment took approximately 45 minutes to complete. Observers were seated 100 cm from the display at an eye height of 140 cm. All testing took place in a quiet, darkened room. 

Results and discussion

Prior to analysis the data was screened for outliers; no subjects were excluded from the following analyses. Because of the binomial nature of the responses in the current experiment, binomial generalized linear mixed modelling was used to assess differences between the conditions (Jaeger, 2008). For all generalized linear mixed models reported in this paper Statterwaite approximation (Satterthwaite, 1946) was used to obtain p values for the fixed effects of the models. 

Half of the trials contained self-motion in a certain direction while the other half of the trials contained no self-motion; we therefore first determined whether heading direction had an influence on the number of correct responses in trials containing self-motion. Trials containing no self-motion were excluded for this analysis. A binomial generalized linear mixed model was constructed to analyze the proportion of correct responses in self-motion trials, with heading as a fixed effect and observer as a random effect. There was no significant influence of heading direction on the number of correctly identified self-motion trials (β̂ = 0.02, SE = 0.01, z = 1.62, p = 0.12). As such, for further analyses trials containing self-motion and trials containing no self-motion were combined. 

We were also interested in determining whether the walker's starting position or orientation affected the proportion of correctly identified trials. To assess the influence of walker orientation and starting position, we constructed a second binomial generalized linear mixed model with walker orientation and starting position as fixed effects and observer as a random effect. Trials containing self-motion and those containing no self-motion were combined for this analysis. The number of correct estimates did not differ significantly between orientations (β̂ = −0.08, SE = 0.05, z = 1.69, p = 0.09), nor starting positions (β̂ = −0.02, SE = 0.01, z = −1.59, p = 0.11). There was also no interaction between orientation and start position (β̂ = −0.02, SE = 0.01, z = 1.42, p = 0.16). The proportion of correct responses for each orientation and starting position can be seen in Figure 2. 

Figure 2

 

Average proportion of correct responses for combinations of walker starting position and orientation. Orientations on the x axis indicate the walker's orientation relative to the observer's standpoint (i.e., a walker with the “left” orientation would appear to face slightly left of the observer). Positive values for the starting position reflect positions to the right of the center of the display while negative values indicate values to the left of the center. Vertical bars represent 95% CI.

As no significant differences between conditions were found, data were amalgamated across all conditions. In order to test whether participants could use biological motion cues to discriminate object motion during self-motion, a binomial test was used to compare observer performance averaged over all conditions against chance level (50% of trials identified correctly). Performance was significantly above chance level (z = 92.40, p < 0.01). This shows that observes are able to reliably detect the presence of self-motion using biological motion cues. Further, it does not appear that this ability to disentangle self and object motion is affected by the orientation or position of the walker, or by the observer's heading angle. 

Experiment 2

Experiment 1 revealed that observers are able to disentangle the motion produced by an oncoming walker from that produced by self-motion through the environment. We have argued that biological motion cues are required to disentangle these two sources of motion, as biological articulation provides information about the walker's trajectory that is independent from the optic flow generated by self-motion. Thus, a portion of movement in the scene can be attributed to the walker and any remaining movement is interpreted as self-motion. 

Nonetheless, it is possible that observers may have used some cue other than biological motion to estimate the presence of self-motion in Experiment 1. Notably, the retinal motion in conditions containing a combination of self- and walker motion was faster than in the walker motion only conditions. Though walkers in Experiment 1 had varied translation speeds in an attempt to reduce the usefulness of this cue, it is still possible that observers identified instances of faster and slower retinal expansion and based their estimates of self-motion on this rather than on biological motion cues. 

We conducted a second experiment to rule out this possibility and to provide further evidence for the importance of biological motion cues when discriminating self-motion from walker motion in ambiguous scenes. Observers were shown scenes containing either forwards-articulating walkers or backwards-articulating walkers. Both walkers translated in directions either congruent (i.e., forwards for the forwards articulating and backwards for the backwards articulating conditions) or incongruent (i.e., backwards for the forwards articulating and forwards for the backwards articulating conditions) with their articulation direction. This led to four conditions, which formed two pairs of matched retinal speeds. 

If observers use articulation to identify walker motion in the stimulus, it was hypothesised that they would attribute the translational motion in the scene to self-motion when it was incongruent with the walker's articulation direction. In this case the walker's articulation cannot account for the motion in the scene. An alternative interpretation for the perceived retinal motion is that the observer is moving either towards or away from the walker on a parallel path, thus self-motion is perceived. When the walker's articulation pattern is congruent with its translation direction, no self-motion should be perceived, as the walker's articulation should account for all motion in the scene. Finally, if biological motion cues do not contribute to the disentanglement of self- and walker motion, with observers simply using retinal speed as a cue, responses should be essentially random, or biased towards perceiving either walker or self-motion on the majority of trials. 

Methods

Observers

Seven volunteers were recruited to participate in the current study (four female, three male). The sample had a mean age of 26.14 (SD = 3.67). All observers were naïve to the aims of the experiment. Observers had normal or corrected to normal binocular visual acuity. 

Procedure

In the current experiment two walker articulation conditions were presented, with walkers either articulating forwards or articulating backwards. Articulation is used here to refer to the direction of limb motion of the walker, i.e., forwards articulating walkers displayed a normal walking action, whereas backwards articulating walkers appeared to be walking in reverse (see General methods). In addition, two translation conditions were presented, with walkers either translating forwards (towards the camera) or backwards (away from the camera). Here, translation refers to the walker's physical movement through space. 

Combining these articulation and translation directions produced four conditions: forwards articulation - forwards translation (congruent); backwards articulation - forwards translation (incongruent); forwards articulation - backwards translation (incongruent); and backwards articulation - backwards translation (congruent). Congruent and incongruent translation refers to walkers moving in either the same or opposite direction to their articulation direction. An example of an incongruent condition is shown in Movie 2. Movie 1A depicts an example of a stimulus analogous to the congruent condition. 

 

Movie 2

 

An example trial depicting incongruent walker translation and articulation. In this example the walker articulates backwards but translates forwards.

Random combinations of walker orientations and walker starting positions were generated for each trial. Prior to each experimental trial, observers were presented with a 500-ms blank interval, which was replaced by a centrally presented fixation-cross for 300 ms. After the fixation-cross disappeared it was replaced by the trial scene. Movement of the scene began immediately after the stimulus appeared and lasted for for 2000 ms. Observers were then required to identify whether they perceived the scene as containing self-motion in any direction by pressing the right shift key of the keyboard, the left shift key was pressed if the observer did not perceive self-motion. Fifteen trials were presented for each of the four possible combinations of articulation and translation direction. This resulted in a total of 60 trials, which took approximately 10 minutes to complete. Testing took place in a quiet darkened room, with observers seated at an eye height of 140 cm and 100 cm from the display. 

Results and discussion

A binomial generalized linear mixed model with translation direction and articulation direction as fixed effects and observer as a random effect was used to evaluate whether the pattern of self-motion responses differed for forwards and backwards articulating and translating walkers. Significant effects were found for walker articulation direction (β̂ = 7.51, SE = 0.95, z = 7.92, p < 0.01) and walker translation direction (β̂ = 7.06, SE = 0.91, z = 7.75, p < 0.01). In addition, articulation and translation directions interacted with one another (β̂ = −13.80, SE = 1.64, z = −8.39, p < 0.01). This interaction can be seen in Figure 3, with walkers that translate in a direction congruent with their articulation direction producing fewer self-motion responses than incongruently translating walkers. This was confirmed statistically using Tukey adjusted posthoc tests. Significant differences were found between forwards-articulating/forwards-translating walkers (congruent) and forwards-articulating/backwards-translating walkers (incongruent), as well as backwards-articulating/forwards-translating walkers (incongruent). Differences were also found between backwards-articulating/backwards-translating walkers (congruent) and backwards-articulating/forwards translating walkers (incongruent), and forwards-articulating/backwards-translating walkers (incongruent); all ps < 0.01. 

Figure 3

 

Proportion of self-motion responses for walkers that translated either congruently with their articulation direction or incongruently with their translation direction. Walkers could articulate and translate either forwards or backwards. Thus, for forwards articulating walkers the congruent translation direction would be forwards, whereas for backwards articulating walkers the congruent translation would be backwards. Black vertical bars represent 95% CI.

This suggests that whether or not observers perceive self-motion is determined by the congruency between the walker's translation and its articulation. When translation and articulation directions match, all retinal motion in the scene was attributed to the walker. Conversely, retinal motion was attributed to the observer when a walker's articulation is incongruent with its translation. Importantly, as the rate of retinal expansion was the same for walkers that translated in the same direction, this pattern of results suggests that the percept of self-motion is influenced by the availability of biological motion cues and is not determined by some other factor, such as the retinal speed. 

Experiment 3

The previous experiments have shown that observers are able to discriminate between scenes containing biological motion alone and scenes containing combined biological and self-motion. It has been argued that this ability is driven by cues in the biological motion stimulus. Experiment 2 in particular, showed that perceiving motion in a scene as being produced by either the walker or the observer is dependent on the relationship between the walker's articulation and translation direction and not the retinal speed. The purpose of Experiment 3 was to provide further evidence that biological motion information plays a central role in disentangling the separate motion components in the scene and that some other cue, such as retinal expansion rate cannot account for our results. 

Further, we wanted to show that observers could indeed identify the separate components in the scene simultaneously. Specifically, the current experiment was designed to test whether observers perceived both walker and self-motion in scenes containing the combined motion sources. When walker translation does not match the articulation pattern it is possible that observers perceive the scene motion entirely as self-motion. A second aim of Experiment 3 was to investigate this possibility. 

In this experiment, observers were presented with scenes that contained either articulating walkers or walkers that translated while maintaining a single, non-articulating posture. Observers were required to report whether they perceived the stimuli as depicting walker motion in isolation, self-motion in isolation or a combination of self- and walker motion. If biological motion information facilitates the disentanglement of self-motion and walker motion, we would predict that the combined response occurs most frequently when biological motion information (in the form of articulation) is available, as observers should be able to use the biological motion cues to identify walker motion in the stimulus, with any additional motion being attributed to self-motion. 

When biological motion is unavailable (as is the case for non-articulating, single-posture walkers), observers should be unable to identify combined situations, as there are no cues to signal that there are two sources producing independent motion components in the scene. In single-posture walker conditions we propose that the retinal expansion could be interpreted as being caused by either the object approaching the observer, or the observer approaching the object. As such, in single-posture PL walker conditions, we expect observers to misattribute the scene motion as originating solely from either the walker or ego-motion. 

Methods

Observers

Seven volunteers were recruited to participate in the current study (four female; three male). The sample had a mean age of 26.71 (SD = 3.40). All observers were naïve to the aims of the experiment. Observers had normal or corrected-to-normal binocular visual acuity. 

Procedure

In the current experiment, a single-posture walker condition was used in addition to forwards articulating walkers. In the single-posture walker condition, walkers translated through space, but maintained a single, body posture (see General methods). Consequently, these walkers did not contain any biological motion information and appeared to glide through the scene rather than to “walk” as articulating walkers did. 

Random combinations of walker orientation, walker starting position, and heading direction were generated for each trial. Thirty trials were presented for each articulation condition. In half of the trials walker and self-motion were presented; in the other half only walker motion was present. Although walkers did not appear to “walk” in the single-posture conditions, the retinal expansion rates in walker-only and combined motion trials in these conditions were the same as that in the articulating walker conditions. 

Prior to each experimental trial, observers were presented with a 500-ms blank interval, which was replaced by a centrally presented fixation-cross for 300 ms. After the fixation-cross disappeared, it was replaced by the trial scene. Movement of the scene began immediately and lasted for 2000 ms. At the end of each trial observers were required to report whether they perceived motion in the stimulus as originating from the walker by pressing the right shift key, self-motion by pressing the left shift key, or a combination of walker and self-motion by pressing the space bar. In total 60 trials were presented, with the experiment taking approximately 10 minutes to complete. Observers completed testing in a quiet darkened room and were seated at an eye height of 140 cm at a distance 100 cm from the display. 

Results and discussion

To assess whether the pattern of responses differed for articulating and single-posture walkers, three generalized binomial linear mixed models were fitted for each response type (walker motion, self-motion, combined), with articulation type and self-motion condition as fixed factors and subject as a random factor. The proportions of responses for the three categories in each condition are presented in Figure 4. 

Figure 4

 

Proportion of self-motion (dark green), walker motion (light green) and combined (gold) responses given for scenes containing articulating or single-posture PL walkers in either the presence or absence of simulated self-motion. The four articulation-type/self-motion condition pairings are presented on the x axis. Black vertical bars represent 95% CI.

Significant differences between the articulation conditions were found for the self-motion response (β̂ = 3.90, SE = 0.51, z = 7.61, p < 0.01), with articulating walkers (M = 0.05, SD = 0.10) producing less self-motion responses than single-posture walkers (M = 0.63, SD = 0.23). There was no significant effect of self-motion condition (β̂ = 0.36, SE = 0.61, z = 0.60, p = 0.55) and no significant interaction (β̂ = −0.36, SE = 0.67, z = −0.54, p = 0.59). 

For the walker motion response, significant differences were found between the self-motion and no-self-motion conditions (β̂ = 2.74, SE = 0.34, z = 8.12, p < 0.01). No main effect was found for articulation conditions (β̂ = −0.55, SE = 0.32, z = −1.72, p = 0.08); however, there was a significant interaction between articulation condition and self-motion condition (β̂ = −2.38, SE = 0.47, z = −5.10, p < 0.01). The interaction can be clearly seen in Figure 4, where walker motion responses are higher in the articulating/no-self-motion condition than in other conditions. This was confirmed statistically by Tukey adjusted posthoc tests, which revealed significant differences between the articulating /no-self-motion condition pair (M = 0.83, SD = 0.22) and the articulating/self-motion condition pair (M = 0.28, SD = 0.22, p < 0.01); the single-posture/no-self-motion condition pair (M = 0.25, SD = 0.20, p < 0.01); and the single-posture/self-motion condition pair (M = 0.19, SD = 0.21, p < 0.01). No other significant differences were found (all ps > 0.05). 

For combined responses, significant differences were found between articulation conditions (β̂ = −2.90, SE = 0.36, z = −8.07, p < 0.01) and self-motion conditions (β̂ = −2.34, SE = 0.21, z = −7.39, p < 0.01). In addition, the interaction between articulation condition and motion condition was significant (β̂ = −2.42, SE = 0.52, z = 4.67, p < 0.01). As can be seen in Figure 4, combined responses were most frequent in the articulating/self-motion condition but were seldom given in the other condition pairings. Again, this was confirmed statistically by Tukey adjusted posthoc tests, which showed significant differences between the articulating/self-motion condition pair (M = 0.68, SD = 0.21) and the articulating/no-self-motion condition pair (M = 0.11, SD = 0.14, p > 0.01), the single-posture/self-motion pair (M = 0.18, SD = 0.15, p > 0.01), and the single-posture/no-self-motion pair (M = 0.12, SD = 0.13, p > 0.01). All other contrasts were nonsignificant (all ps > 0.05). 

The current results support the argument that the ability to detect self-motion is driven by biological motion cues. When presented with a single-posture walker that translated through space but did not exhibit any biological motion cues, observers consistently reported seeing self-motion regardless of whether the rate of retinal expansion was equivalent to a combination of walker and self-motion or walker motion in isolation. It should be noted that self-motion was not presented in isolation in any of the conditions. However, because of the ambiguous nature of the stimuli, retinal expansion could theoretically be interpreted as movement of the observer towards the walker or of the walker towards the observer. Evidently, in the absence of biological motion cues, observers were biased towards perceiving the expanding retinal image as self-motion. When walkers articulated, observers were able to discriminate between trials that contained self-motion and trials that contained walker motion only. This implies that there is information in the biological articulation pattern that allows observers to identify and differentiate between translational motion produced by the observer and motion produced by the walker. These results provide further evidence that the ability to discriminate between walker and self-motion in both the current and previous experiments was not based on some additional cue, such the rate of expansion, but relied on biological motion information. 

Experiment 4

Experiments 1, 2, and 3 showed that observers are able to distinguish self- and walker motion based on cues provided by PL walkers. If it is possible to identify self-motion in highly ambiguous scenes, it may also be possible to calculate heading. The identification of heading in the scenes used in the current study would require the exchange of information between biological motion and heading detection mechanisms. Specifically, the spurious motion of the walker would have to be discounted from the optic flow analysis to obtain an accurate estimate of heading. The previous experiments showed that it is possible to identify the motion in the optic flow field produced by walker motion. It has been argued that this is achieved by determining differences between the total retinal motion and the amount of movement accounted for by biological motion cues. The question thus stands as to whether or not the additional motion present in the scene during self-motion can be used to resolve heading. 

To test whether observers can use a walking person to gauge heading, we had observers estimate heading while viewing translation through a scene containing an approaching walker. It was hypothesized that if observers are able to separate self- and biological motion, heading estimates should be fairly accurate in all conditions. Importantly, heading estimates should not be affected by the walker's orientation, as walker motion information should be accounted for beforehand. Conversely, if heading and object motion estimation are subserved by separate systems that do not interact at the level of heading detection, we would predict that heading estimates should represent an approximate average of walker and self-motion, assuming that heading is calculated by pooling all motion vectors in the scene. 

Method

Observers

Eleven student volunteers from the University of Münster, as well as one author participated in the current study (eight female, three male; average age M = 26.67, SD = 8.30). All observers had normal or corrected-to-normal binocular visual acuity. Apart from the author all observers were naïve to the aims of the experiment. Observers were compensated for their time with course credit. 

Procedure

Experiment 4 followed almost the same procedure as Experiment 1 with two exceptions. Firstly, in Experiment 4 both walker translation and self-motion were always present. Secondly, the response phases were different for the two experiments. In Experiment 4, after all motion had finished and the stimulus disappeared, a red probe line appeared at a random location on the horizon of the display. Observers were required to move this probe line using the mouse to indicate their perceived heading. Clicking the mouse signaled a response. No restrictions were placed on response times. 

Each of the potential orientation-position-heading combinations was tested 10 times, resulting in a total of 270 trials. In addition, observers were given 10 randomly selected practice trials beforehand to ensure that they understood the task. In total the experiment took approximately 45 minutes to complete. Observers were seated 100 cm from the display at an eye height of 140 cm. All testing took place in a quiet, darkened room. 

Results and discussion

The data was screened prior to analysis and was not found to contain any outliers, but the assumption of sphericity for ANOVA was violated in some conditions in which appropriate Greenhouse-Geisser corrections were used to account for this violation. 

A three-way repeated measures ANOVA was used to compare the estimated heading directions in all combinations of walker starting position, walker orientation and observer heading direction. Significant main effects were found for the walker starting position F(2, 22) = 7.32, p_{(Greenhouse-Geisser)} = 0.02, η²_p = 0.40, the walker orientation F(2, 22) = 17.95, p_{(Greenhouse-Geisser)} = 0.001, η²_p = 0.62, and the observer heading direction F(2, 22) = 15.75, p_{(Greenhouse-Geisser)} = 0.002, η²_p = 0.59. None of the factors interacted significantly with one another. Heading estimates as a function of heading direction, walker orientation, and walker starting position are shown in Figure 5. 

Figure 5

 

Average estimated heading as a function of actual heading direction for PLWs that were oriented leftwards (yellow), rightwards (blue), or directly forwards (orange). The three graphs depict the estimates for walkers starting at the left, the center, or the right of the display respectively. Positive values on the x and y axes represent positions to the right of the center of the display, whereas negative values represent positions to the left of the display. Vertical bars represent 95% CI.

In order to further examine the main effects, Bonferroni adjusted posthoc tests were used to assess significant differences within the levels of each condition. Posthoc tests between the different starting positions showed that the difference between walkers starting at 5° to the left and walkers starting centrally approached, but did not meet the requirement for statistical significance (p = 0.057). Similarly, the difference between walkers starting at 5° to the left and walkers starting at 5° to the right approached, but did not reach statistical significance (p = 0.057). The difference between centrally and rightwards starting walkers was also nonsignificant (p = 0.10). It appears that whereas heading estimates were shifted slightly towards the walker's starting position, this effect is statistically unreliable. 

Posthoc tests between the different heading directions revealed significant differences between all levels. As can be seen in Figure 5, headings to the right of the display (M = 1.20, SD = 1.55) shifted heading estimates rightwards of estimates in either central (M = −0.53, SD = 0.75, p = 0.005) or leftward (M = −2.23, SD = 2.01, p = 0.006) heading direction conditions. Leftward heading directions also shifted estimates to the left of estimates in the central heading direction condition (p = 0.01). From these results it can be concluded that heading estimates are affected by the direction of the heading in the current experiment. 

All levels of the walker orientation condition also differed significantly from each other. Leftwards oriented walkers displaced heading estimates to the right (M = −1.96, SD = 1.07) compared to rightward oriented (M = 1.06, SD = 1.83, p = 0.003), and centrally oriented (M = −0.67, SD = 1.04, p = 0.02) walkers. Rightward oriented walkers also shifted heading estimates to the left, compared with centrally presented walkers (p = 0.002). This can be seen graphically in Figure 5. These results suggest that heading estimates were biased in the opposite direction of walker translation. Taken together with the results from the previous paragraph, it appears that for the current stimuli, observer heading estimates represent a combination of self-motion and walker motion. 

To further explore this proposition, we calculated heading estimates that would be expected by vector summation of the self-motion and walker translation for each walker orientation and heading direction condition. In this case, we assume that the visual system would simply treat all image motion as resulting from self-motion; thus, the motion of the walker towards the observer would be considered instead as motion of the observer towards the walker. These estimates are depicted in Figure 6 by the dashed lines. The average heading estimates of the observers in Experiment 4 are shown by solid lines. The black dashed line shows the expected heading performance if observers were able to perfectly discount the motion of the walker. The sum of self- and walker motion vectors predicts observer performance fairly well and importantly mirrors certain properties of the data. The vector sum shows that a combination of self- and walker motion vectors should produce biases in heading estimates in the opposite the direction of walker translation. 

Figure 6

 

Colored dashed lines show heading estimates predicted as a sum of the self-motion and walker translation vectors. Solid lines show the heading estimates observed in Experiment 4 for each walker orientation and heading direction condition. The dashed black line shows the expected outcome if observers were to perfectly discount walker motion and estimate heading accurately. Vertical bars represent 95% CI for the observed data.

T tests were used to statistically compare the observer heading estimates for each walker orientation and heading direction condition to the respective estimate predicted by the vector summation. Observer estimates differed significantly from the vector summation estimate when the walker was oriented leftwards and the heading direction was straight ahead (t(11) = −3.01, p = 0.01). No other significant differences were found (all ps > 0.05). 

Differences between the observed data and the vector sum estimates may have occurred either due to random variation in the data, or due to some unaccounted for parameter, such as the increase in the walker's translation speed as its depth decreases, the walker's starting position, or the motion induced by the walker's articulation. On the whole, however, we argue that the pattern of observer heading estimates follow the vector summation better than the true heading. This supports our suggestion that heading estimates in the current experiment reflect a combination of the walker and observer translation vectors. 

General discussion

The current series of experiments investigated the ability of observers to detect self-motion and estimate heading in scenes containing ambiguous combinations of ego-translation and biological motion. Because the available visual input in these scenes represents a combination of both self- and biological motion, it cannot be reliably separated without information about these different sources of motion. We proposed that biological motion cues could provide this information and thus facilitate the decomposition of the optic flow pattern into its respective self- and walker motion components. Ultimately, this should enable observers to detect self-motion and potentially also heading, despite the ambiguity of these stimuli. The purpose of the current series of experiments was to test this hypothesis. 

Experiment 1 showed that the presence of self-motion could be identified in stimuli that were an ambiguous combination of both self- and biological motion. Experiment 2 extended this finding, showing that percept of self-motion is derived from incongruencies between the walker's articulation pattern and its translation or rate of expansion. Experiment 3 further showed that in the absence of biological motion, scenes are rarely interpreted as a combination of self- and walker motion. In combination, these results suggest that biological motion cues play a critical role in disentangling the components of complex optic flow patterns. Experiment 4, however, showed that the visual system is unable to resolve heading based on this information. These findings shed some light on the processes underpinning self-motion detection and heading estimation. 

The most significant finding of the current study is that humans are able to discriminate self-motion from optic flow fields that represent an ambiguous combination of biological and observer-motion. Because no rigid scene information was available, we argue that this discrimination must be based solely on cues provided by biological motion. Experiment 3 tested the validity of this claim explicitly, showing that when biological motion information is unavailable, observers do not perceive combined self- and walker motion. 

We suggest that this detection may have been achieved by identifying discrepancies between the observed retinal flow and the motion that would be expected based on biological motion information alone. We argue that mismatches between the articulation and the translation of a walker provide cues that could potentially facilitate the identification of self-motion in ambiguous situations. This proposal is supported by Experiment 2, in which we showed that when the gait and translation direction of the walker are incongruent, the pattern of optic flow is consistently interpreted as being produced by self-motion. Biological motion processing thus represents the cornerstone of the ability to identify self-motion in the present stimuli. For observers to detect self-motion they must first ascertain the scene motion produced by the walker, which is achieved by processing biological motion cues. Any remaining motion must logically be produced by some external source, with self-motion being a likely candidate. 

In the literature, biological motion processing is often discussed as being driven by either local or global information. In this context, the motion of the single PL walker points, particularly the feet (Troje & Chang, 2013; Troje & Westhoff, 2006), in isolation is considered local information, while the underlying, whole-body form of the walker and its dynamic change over time is considered global information (Beintema & Lappe, 2002; Lange & Lappe, 2006). Both sources of information may contribute to particular aspects of biological motion processing (Chang & Troje, 2009b; Kuhlmann et al., 2009). We suggest that both the local foot motion and global form of biological motion could provide the cues necessary to identify the walker's motion and thus disentangle optic flow in the current situation. 

The form of a PL walker provides cues as to its facing direction (Lange & Lappe, 2006), while changes in form over time enable the perception of the walking direction. Similarly, the foot points can also provide the facing direction, as horizontal asymmetries in the trajectory and acceleration of the feet produce a specific pattern of motion that is orientation-dependent (Chang & Troje, 2009a; Lange, Georg, & Lappe, 2006; Mather et al., 1992; Troje & Westhoff, 2006). Knowing the facing direction of the walker allows us to infer that walkers in the current experiment were highly likely to ambulate in the direction in which they are facing. Mismatches between the walker's facing direction and its apparent translation are therefore indicative of an additional source of motion. We suggest that this additional motion is interpreted as self-motion. 

In addition to the facing direction of the walker, discrepancies between a PL walker's articulation speed and its displacement speed also represent a potential cue that could be used to identify self-motion. There is an explicit link between a walker's articulation rate and the rate at which it translates (Masselink & Lappe, 2015; Thurman & Lu, 2013, 2016). As such, walkers that move faster than expected given their articulation rate would signal the presence of additional observer-generated motion. This cue is signaled by the mismatch between the walker's articulation and translation speed and could be derived from the global motion of the walker, by analyzing changes in the form of the walker over time (e.g., Theusner, De Lussanet, & Lappe, 2014). Alternatively, the local motion of the foot points could be used to determine mismatches between the walker's articulation speed and its translation speed. The forces of friction dictate that a foot contacting the ground during walking is usually stationary. As such, any movement of the foot point during the “stance” phase of a gait pattern is a local cue indicating self-motion. 

During locomotion on a path parallel to the heading of the observer, a PL walker's facing direction is uninformative for obtaining self-motion. Observers in Experiment 1 were able to accurately discriminate self-motion in situations where the path of locomotion was parallel to the walker's path. This suggests that cues provided by mismatches between the walker's articulation rate and its translation speed are adequate for detecting self-motion when facing mismatches are unavailable. 

Though observers are able to infer self-motion from ambiguous situations based on the cues provided by biological motion patterns, Experiment 4 showed that this information does not appear to play a role during heading estimation. A simple vector-sum of the self-motion and walker translation produces a pattern of heading estimates that approximate the data observed in Experiment 4. This suggests that the more complex attributes of the walker's motion, such as the information purveyed by articulation, is not integrated to simplify the problem of heading estimation. These results are consistent with previous research showing that heading is estimated from optic flow fields produced by pooling motion vectors over large portions of the visual scene (e.g., Andersen & Saidpour, 2002; van den Berg, 1992; Warren et al., 1991) and would therefore likely be reproduced by existing models of heading estimation (e.g., Bruss & Horn, 1983; Calow et al., 2005; Lappe & Rauschecker, 1993; Perrone & Stone, 1994). 

It may seem counterintuitive for heading estimation mechanisms not to use the entirety of the information provided by biological motion to produce more reliable heading estimates. However, it is important to remember that both heading and biological motion must be computed rapidly and constantly updated. It could be the case that it is simply more efficient to process both in parallel and combine them at later stages when necessary, such as during obstacle avoidance or interception. Unlike the scenes used in the current experiments, the natural environment is densely textured and contains many points of reference. Thus, though disregarding biological motion information during heading estimation produced errors in the current study, it would be a more viable approach when the remainder of a scene's motion vectors corresponds to self-motion. It may be the case that the abundance of visual information in the real world is adequate to ensure reliable navigation even when walkers are present. This does not detract from the importance of the current study, as we show that biological motion can function as a cue for the disentanglement of ambiguous optic flow, and may be particularly useful for either the identification of self- or biological motion when the scene is noisy or sparse, for example at night or in dense crowds. 

Acknowledgments

This work was supported by DFG La 952-7. 

Commercial relationships: none. 

Corresponding author: Markus Lappe. 

Email: . 

Address: Institute for Psychology, University of Muenster, Muenster, Germany.