Efficiencies for parts and wholes in biological-motion perception

Abstract

People can reliably infer the actions, intentions, and mental states of fellow humans from body movements (Blake & Shiffrar, 2007). Previous research on such biological-motion perception has suggested that the movements of the feet may play a particularly important role in making certain judgments about locomotion (Chang & Troje, 2009; Troje & Westhoff, 2006). One account of this effect is that the human visual system may have evolved specialized processes that are efficient for extracting information carried by the feet (Troje & Westhoff, 2006). Alternatively, the motion of the feet may simply be more discriminable than that of other parts of the body. To dissociate these two possibilities, we measured people's ability to discriminate the walking direction of stimuli in which individual body parts (feet, hands) were removed or shown in isolation. We then compared human performance to that of a statistically optimal observer (Gold, Tadin, Cook, & Blake, 2008), giving us a measure of humans' discriminative ability independent of the information available (a quantity known as efficiency). We found that efficiency was highest when the hands and the feet were shown in isolation. A series of follow-up experiments suggested that observers were relying on a form-based cue with the isolated hands (specifically, the orientation of their path through space) and a motion-based cue with the isolated feet to achieve such high efficiencies. We relate our findings to previous proposals of a distinction between form-based and motion-based mechanisms in biological-motion perception.

Introduction

The ability to perceive the particular kinds of motion exhibited by active biological agents (i.e., biological-motion perception) is something shared by a wide variety of animals, including human beings (Bertenthal & Pinto, 1994; Blake & Shiffrar, 2007; Troje, 2013; Troje & Westhoff, 2006). Johansson (1973) was among the first to show that human observers are able to recognize a variety of human actions based on displays consisting only of moving dots that trace the positions of a small set of locations on the body (i.e., point-light [PL] displays). Subsequently, others have uncovered a wide range of abilities related to the perception of PL displays. For example, human observers are able to reliably determine age (Montepare & Zebrowitz-McArthur, 1988), emotional attributes (Atkinson, Dittrich, Gemmell, & Young, 2004; Atkinson, Tunstall, & Dittrich, 2007), and gender (Mather & Murdoch, 1994) based on the information provided by PL displays. Other abilities include person identification (Troje, Westhoff, & Lavrov, 2005) and estimation of the weight of an object being lifted (Bingham, 1987). Further, the perception of biological motion from PL displays is not limited to humans, with the capacity for biological-motion perception having been demonstrated in animals such as cats (Blake, 1993) and pigeons (Dittrich, Lea, Barrett, & Gurr, 1998). 

Several studies have explored the ontogenetic development of biological-motion perception. A looking preference for ecologically valid PL displays has been demonstrated early in human development (Bertenthal, 1993; Vallortigara & Regolin, 2006; Vallortigara, Regolin, & Marconato, 2005). Specifically, human infants as young as 4 months preferentially attend to intact PL walkers over spatially scrambled versions of the same displays (Bertenthal, 1993). Other work has shown a similar preference in dark-reared chicks (Vallortigara & Regolin, 2006; Vallortigara et al., 2005). Such evidence suggests that the mechanisms involved in biological-motion perception may be evolutionarily old and active early in an organism's development. It has thus been argued that the ability to perceive biological motion may serve as a kind of “life detector” mechanism for conspecifics as well as other species of legged vertebrates (Johnson, 2006; Troje & Westhoff, 2006). 

The idea of a life-detector mechanism has to do with the need for organisms to orient toward and behaviorally respond to predators, prey, and potential mates (Johnson, 2006; Troje & Westhoff, 2006). However, there are many types of motion signals in an organism's environment originating from both animate and inanimate objects. Indeed, some motion in the visual field arises from the organism's own movement, while other motion comes from objects in the environment (Johansson, Vonhofsten, & Jansson, 1980). Because of this inherent ambiguity, having a neural mechanism that renders motion signals from animate objects such as prey and predators more salient than other types of motion would be evolutionarily advantageous; but in order to be useful, such a specialized mechanism would need to be able to separate biological from nonbiological motion, as well as be tuned to the characteristics of the motion generated by a variety of different species. 

The particular motion of the feet has been proposed as the primary signal being used by such a life-detector mechanism (Chang & Troje, 2009; Troje & Westhoff, 2006). Support for this idea comes from experiments designed to pinpoint the parts of the body that are most important for the accurate perception of biological motion. Mather, Radford, and West (1992) found that performance in a PL-walker direction-discrimination task suffered the most when the feet and hands were simultaneously removed, compared to the removal of the elbows and knees or the shoulders and hips. More recently, Troje and Westhoff (2006) report that PL-walker discrimination performance suffered more from inverting the feet alone than from inverting the rest of the walker when the feet were retained in an upright orientation. Chang and Troje (2009) have further shown that the specific acceleration profile along the trajectory traced by the feet was critical to human performance in a PL-walker direction-discrimination task. Specifically, when the points corresponding to the feet were transformed to trace their trajectory at a constant velocity, discrimination performance fell. 

Based on these results, it is clear that the motion of the feet is of critical importance for the human ability to discriminate the direction of PL walking. However, it is not entirely clear why this is the case. One possibility is that the neural mechanisms for processing biological motion have evolved to be especially well tuned to the motion of the feet (i.e., a life-detector mechanism; Troje & Westhoff, 2006). An alternative possibility is that the feet may simply be the most informative aspect of a PL walker for direction discrimination, and thus task performance depends critically upon them not because of specialized neural mechanisms but because of their inherent information content (Johnson, 2006). That is, the feet may not be processed with particular efficiency by the human visual system, but rather their motion may happen to carry relatively more discriminative information than other parts of the body. Definitive evidence for a life detector would need to show that the human visual system is relying on the feet more than would be expected based on how much information is contained in them relative to other parts of the body. The notion of a life detector is that it is uniquely tuned to the characteristics of a particular class of signals arising only from animate entities. A generalized mechanism that uses the most informative aspect of the stimulus would not necessarily be a life detector, but more of an information detector. Being an information detector is not exclusive of being a life detector, but finding a mechanism that is tuned to an aspect of the signal other than that which is most informative would offer compelling evidence of a life detector. In this sense, the patterns of inefficiency in the visual system can be especially informative regarding what the system is tuned to do. 

Thus, the goal of our experiments was to try to dissociate these two possibilities: whether previously demonstrated sensitivity to the motion of the feet is best conceptualized as a life detector or an information detector. We approached this question by using Bayesian ideal-observer analysis to measure the amount of information carried by different parts of human PL stimuli in a standard PL-walker direction-discrimination task. We then compared human to ideal performance in the same task in order to get an estimate of the efficiency (i.e., ideal/human threshold) with which human observers make use of the information carried by different parts of the body (Geisler, 1989; Gold, Tadin, Cook, & Blake, 2008). Although previous studies have provided clear evidence consistent with a performance advantage for the feet in direction discrimination of PL walkers, the measurement of efficiency in this manner allowed us to determine whether this advantage could be explained simply by the physical information inherent to the feet or was due to the ability of the human visual system to make particularly good use of that information, regardless of how much of it is provided. 

Experiment 1

Our first experiment was designed to test whether there is an efficiency advantage for the perception of the motion of feet relative to other parts of the body, by measuring human observers' efficiencies for discriminating the walking direction of PL figures defined by subsets of points on the human form. Specifically, we measured the efficiency for discriminating the walking direction of PL stimuli defined by just the motion of the feet, just the motion of the hands, and the motion of all points except for either the feet or the hands. We reasoned that if there is a processing advantage for the feet over other parts of the body, our human observers should be particularly efficient with the feet shown in isolation and particularly inefficient with the hands shown in isolation. 

Methods

Subjects

Eleven observers (eight women, three men) between the ages of 18 and 22 participated in Experiment 1. All observers were volunteers (unaware of the purpose of the study) and received course credit for participation. 

Stimuli

The data for our PL-walker stimuli were recorded using a VICON MX motion-capture system (Vicon, Oxford, UK). The 3-D coordinates of 13 different body points (one for the head, two each for the feet, knees, hips, shoulders, elbows, and hands) were recorded from 10 different actors and projected onto a two-dimensional plane in order to generate the final PL-walker animations. Each of these animations was also vertically mirror-reflected to create complementary left- and right-walking versions of each actor. Because actors had different stride lengths, the number of gait cycles varied depending on the walker, but each stimulus included at least two and no more than three full gait cycles. For each PL walker we generated six movies, each beginning from a random frame within the gait cycle in order to introduce temporal uncertainty. 

The stimuli were defined in terms of Weber contrast—that is, C_pix = (L_pix − L_bg)/L_bg, where L_pix is the luminance of an individual pixel and L_bg is the luminance of the surrounding background. The points making up each PL walker were positive-contrast squares (3 × 3 pixels) presented in a background set to midgray (33.17 cd/m²). Stimuli were viewed from a distance of 130 cm and subtended 0.96°–1.1° (58–68 pixels) vertically and 0.45°–0.63° (27–38 pixels) horizontally. Each PL movie comprised 85 frames. Stimuli were presented on a Sony Trinitron 520 CRT monitor at a resolution of 1,024 × 768 pixels and a refresh rate of 85 Hz. Thus, each animation was presented for a total duration of 1000 ms. A unique sample of Gaussian, white spatiotemporal pixel noise was added to the stimulus on each trial. The contrast variance of the noise distribution was 0.0625, with a power spectral density (i.e., power per unit bandwidth) of 1.71 × 10⁻⁵ s deg². The noise field covered the entire extent of the PL-walker stimulus region, subtending 3.64° × 3.64° (220 × 220 pixels). 

We wanted to measure how efficiently human observers were able to use the information carried by the feet, compared to another set of points that have similar but different spatiotemporal trajectories—the hands. Beginning with the animations already described, we then generated four additional sets of stimuli that were composed of different subsets of the original 13 points: isolated feet (two points), missing feet (11 points), isolated hands (two points), and missing hands (11 points; see Figure 1 for examples of the stimuli used in each condition). We measured left- versus right-walking discrimination performance in each of these five stimulus conditions by varying the contrast energy (i.e., integrated squared contrast) of the stimuli within the noise across trials using a two-down, one-up adaptive staircase procedure. We also measured thresholds for a statistically optimal or ideal observer in each of the same conditions (see Thresholds and Efficiencies subsections). 

 

Figure 1

 

Left- and right-facing point-light walker stimuli in five stimulus conditions embedded in spatiotemporal white noise. The order of the stimulus conditions in the movie is all points, isolated feet, isolated hands, missing feet, and missing hands. The stimuli presented here are higher in contrast than those presented in our experiments.

Procedure

On each trial, participants were shown a PL walker and asked to decide whether the walker was facing to the left or right using a key-press response. Accuracy feedback was given in the form of a high or low beep; a high beep indicated a correct response, and low an incorrect response. Trials were blocked by condition (i.e., isolated feet, isolated hands, missing feet, missing hands, all points), and block order was randomized for each participant during a given session. Each observer participated in one session of 125 trials per condition (625 trials total). Each trial block was preceded by 10 noiseless, high-contrast practice trials. 

Analysis

We estimated discrimination contrast-energy thresholds in each condition for each observer. Then we ran simulations to estimate thresholds for the ideal observer in each condition. We calculated efficiency as the ratio of the human thresholds to the ideal observer's thresholds. 

Thresholds

Contrast-energy thresholds in each condition were measured by fitting psychometric functions to the staircase data. Specifically, we fitted maximum-likelihood Weibull functions to each subject's staircase data in each condition and defined threshold as the contrast energy yielding 71% correct responses (estimated by interpolation). Threshold error estimates were obtained through 250 bootstrap simulations in each condition (Efron & Tibshirani, 1993). In estimating each threshold, up to 50 initial trials were excluded if the psychometric-function fit was noticeably improved. We examined whether eliminating the first five trials increased the beta (i.e., slope) parameter of the Weibull function. If beta increased by more than 0.1, we eliminated the next five trials and examined beta again, continuing until beta stopped increasing or we had eliminated 50 trials. 

Efficiencies

To estimate the ability of our human observers to make use of available information in each condition, we compared their thresholds to thresholds of a statistically optimal or ideal observer. The ideal observer is a theoretical machine that provides a strict upper bound on performance in a given task. As such, it offers an objective measure of relative difficulty across different stimulus conditions and tasks (Green & Swets, 1966). The ratio of ideal to human threshold is thus a measure of the proportion of information used by the human observer in a given task. This ratio is known as efficiency. 

Ideal-observer thresholds in our experiment were estimated using Monte Carlo simulations. The ideal decision rule for our task–stimulus combination involves choosing the walking direction that has the highest summed a posteriori probability of matching the noisy stimulus presented in a given trial. The a posteriori probability is obtained by comparing each noise-free stimulus template within each walking direction to the noisy trial stimulus point by point and then summing across all the stimulus templates within each of the two walking directions (for specific details on how the ideal observer is implemented in a PL-walker task, see the appendix of Gold et al., 2008). Each ideal threshold was based upon a minimum of 750 trials. The resulting ideal thresholds were used to compute efficiency for each human observer in each condition. 

Results

Thresholds for 10 human observers and the ideal observer in each condition are shown in Figure 2A, and the corresponding human efficiencies are shown in Figure 2B (one female observer was excluded from analysis due to an inability to estimate her thresholds in two of the conditions). These data show that, surprisingly, the ideal observer's performance is virtually unaffected by either the removal or isolation of the hands or feet, indicating that the relative amount of information available was roughly equivalent across all conditions. Human thresholds and efficiencies were similarly unaffected by the removal of either the hands or the feet, suggesting that neither the feet nor the hands played a necessary role in discrimination performance. However, efficiency improved sharply when the feet and hands were shown in isolation, and efficiency was higher for isolated hands than for isolated feet. A one-way repeated-measures ANOVA on the log-transformed efficiencies confirmed that there was a significant difference in efficiency across conditions, F(1, 39) = 49.2, p < 0.0001. Pairwise t tests using the Benjamini–Hochberg correction (Benjamini & Hochberg, 1995) also confirmed that efficiency with isolated feet was significantly higher than for the complete, missing-feet, and missing-hands conditions, and that efficiency for isolated hands was higher than for every other condition (see Table 1 for p values). Mean efficiency did not significantly differ across the complete, missing-feet, and missing-hands conditions. 

Figure 2

 

(A) Contrast-energy thresholds and (B) efficiencies for Experiment 1. The open circles connected by the dashed line in (a) plot thresholds for the ideal observer. Error bars correspond to ±1 standard error of the mean. Black dots correspond to individual subjects.

Table 1

 

Benjamini–Hochberg-corrected p values for all pairwise comparisons in Experiment 1.

 

Discussion

Two patterns are readily apparent from the results of this experiment. First, efficiency was highest when the body parts were shown in isolation, as opposed to either missing or combined with the rest of the points. There are several interesting implications for this result, which we return to in the General discussion. Second, we found that isolated hands showed high overall efficiency, just as high as isolated feet. We were surprised by this result, given the findings from previous experiments on the role of body parts in biological-motion perception (Chang & Troje, 2009; Troje & Westhoff, 2006). Although this result could reflect a genuine processing advantage for the motion of the hands during locomotion, an alternative possibility is that our participants were able to exploit a simple spatial-position cue in the isolated-hands condition that was unrelated to the relative motion of the hands. Close inspection of the hand points shows that their positions are consistently displaced spatially from a meridian through the spine of the PL walker, in the direction that the walker faces, thus providing a reliable spatial cue for the direction of walking apart from the motion information carried by the PL walker (see Figure 1). That is, observers could perform the task by simply answering the question, “Does the bright part (corresponding to the hands) appear more to the left or more to the right of the diffuse mass of points (corresponding to the rest body)?”—a strategy that would have nothing to do with the perception of biological motion per se. 

Experiment 2

In order to test the possibility that our human observers were simply relying on a consistent spatial-position cue to perform the isolated-hands task rather than true biological-motion perception, we repeated our first experiment, only this time we randomly jittered the horizontal position of the stimulus from trial to trial. Introducing spatial uncertainty in this fashion makes the absolute location of the points an unreliable cue yet preserves all of the relationships among points. If observers were simply relying on a spatial cue when the hands were shown in isolation in Experiment 1, the introduction of spatial uncertainty should have a relatively greater impact on efficiency in that condition. 

Methods

Nineteen observers (nine women, 10 men) between the ages of 18 and 21 participated in Experiment 2. Six observers (three women, three men) were excluded from analysis because of an inability to estimate thresholds in at least two conditions. The inability to estimate thresholds arose from the pattern of these subjects' performance. If an observer exhibited a lower proportion correct at a high level of contrast (e.g., the second or third highest level presented), this resulted in either failure to fit the psychometric function or a function with such a poor fit that the estimated threshold was meaningless. Furthermore, psychometric-function fits were not improved with the exclusion of an initial set of trials. The incorrect responses to high-contrast stimuli usually came in the second half of the block, consistent with subjects responding randomly due to being disengaged from the task. Three additional observers were excluded due to equipment malfunction. 

The remaining methods for Experiment 2 were identical to those for Experiment 1, with the exception that on each trial the PL walker could appear at one of seven different horizontally shifted positions with equal probability, ranging from −0.15° to +0.15°. The noise field within which PL stimuli appeared was always presented in the same location on screen. The seven horizontally shifted stimulus positions were defined relative to the noise field. 

Results

The results of Experiment 2 are shown in Figure 3. These data reveal that the introduction of spatial uncertainty did not change the pattern of results across conditions, although it did serve to increase overall efficiency. Close inspection of the data and comparison to the data in Figure 2 reveals that this was due to the uniform increase in ideal thresholds across conditions (a result of the increase in statistical variability introduced by the addition of extra uncertainty) and no corresponding changes in human thresholds. A one-way repeated-measures ANOVA on the log-transformed efficiencies confirmed that there was a significant difference in efficiency across conditions, F(1, 39) = 91.1, p < 0.0001. Pairwise t tests using the Benjamini–Hochberg correction (Benjamini & Hochberg, 1995) yielded the same pattern of results as in Experiment 1 with the exception that the isolated-hands condition exhibited significantly higher efficiency than the isolated-feet condition (see Table 2 for p values). 

Figure 3

 

(A) Contrast-energy thresholds and (B) efficiencies for Experiment 2. The open circles connected by the dashed line in (a) plot thresholds for the ideal observer. Error bars correspond to ±1 standard error of the mean. Black dots correspond to individual subjects.

Table 2

 

The p values for all pairwise comparisons in Experiment 2.

 

Discussion

There are two interesting things to note about these data. First, the fact that human thresholds were unaffected by the introduction of positional uncertainty is exactly what we would expect to find if the amount of positional uncertainty that was introduced to the stimuli did not exceed the amount of intrinsic uncertainty already present in the human visual system. That is, the finding that thresholds were unchanged after adding externally generated spatial uncertainty suggests that the amount of internally generated spatial uncertainty was a greater limiting factor on performance than the externally introduced spatial uncertainty (for an example of this logic used to estimate the amount of intrinsic spatial uncertainty in detection and identification tasks, see Levi, Klein, & Sharma, 1999). 

Second, and more importantly, the fact that the high efficiency exhibited in the isolated-hands condition actually increased relative to the isolated-feet condition with the addition of horizontal spatial uncertainty makes it highly unlikely that observers were relying upon a spatial-position cue to perform the task in the isolated-hands condition. However, an analysis of the path traveled by the hands revealed that an additional spatial-orientation cue remained present in the isolated-hands condition, even with the introduction of horizontal spatial jitter (see the top row of Figure 4). Specifically, for all PL walkers the path traced out by the opponent motion of the hands formed an acute angle relative to horizontal (see Figure 5). Although the angle was different for each PL walker, the range was sufficiently restricted to provide a spatial-orientation cue that reliably indicated the facing direction of the PL walker. The top row of Figure 4 shows that the angle relative to horizontal of a line connecting the hands (θ in Figure 5) in the individual frames of PL animations for all 10 walkers can be used as a reliable signal to distinguish left- from right-facing PL walkers; the only overlap in the distributions of angles in the left-facing and right-facing stimuli occurs in a minority of frames at 0°. 

Figure 4

 

Overlap in orientation of the trajectories created by hands and feet and in left- and right-facing stimuli across rotation conditions. Each plot shows two histograms: left-facing stimuli (solid gray) and right-facing stimuli (black outline). Each histogram depicts the number of frames in which connecting the two points (either hands or feet) creates a particular angle relative to horizontal, as represented by θ in Figure 5. The isolated-hands stimuli show progressively more overlap from no rotation to bidirectional rotation to unidirectional rotation, whereas the feet show the most overlap in the bidirectional-rotation condition and the least in the unidirectional-rotation condition. Additionally, all the frames with an orientation of 0° (horizontal) are overlapping, each stimulus set having the same number of frames that satisfy this criterion in its right- as in its left-facing version. The spike at zero and the gaps immediately adjacent are due to the limited resolution of the frames: One pixel of vertical displacement at 20 pixels of horizontal displacement corresponds to an angle of 1.24°. Each bin is 1° wide.

Figure 5

 

Example of the angle of the path traced out by hands on an individual frame. A left-facing complete point-light walker stimulus is shown on the left side of the figure, and a right-facing complete point-light walker is shown on the right. The solid black line indicates the trajectory formed by connecting the hands with a straight line. Defining the horizontal dotted line as 0°, θ roughly approximates the angle of the path that the hands move along. When the right-facing walker is rotated θ° clockwise, it results in the hands traveling along a horizontal path. The same is true when a left-facing walker is rotated θ° counterclockwise.

Why was efficiency for isolated hands higher than for isolated feet in Experiment 2 with spatial uncertainty, while the difference between the two conditions failed to reach significance in Experiment 1? We do not think the introduction of spatial uncertainty in Experiment 2 is directly responsible for this observed difference in results. The pattern of mean efficiency for isolated feet and isolated hands is the same across the two experiments: In both cases, the mean efficiency is higher for hands than for feet. However, looking at the variance in these two conditions across Experiments 1 and 2, the variance is lower for both conditions in Experiment 2 (see the two rightmost bars in Figures 2 and 3). In Experiment 2, we excluded six observers due to an inability to estimate psychometric thresholds in at least two conditions, while in Experiment 1 we excluded only a single observer for the same reason. The difference in the number of excluded observers offers a relatively straightforward explanation for the smaller variance observed in Experiment 2—namely, excluded observers tended to perform more poorly than nonexcluded observers, whether through apathy or difficulty with the task. For five of the six excluded subjects, the hands condition was among the two or more conditions for which a threshold could not be estimated. 

Experiment 3

In Experiment 3, we examined whether the orientation cue present in the path traced out by the hands could be responsible for the pattern of performance observed in the first two experiments. We introduced picture-plane (i.e., fronto-parallel) rotational uncertainty to our PL-walker stimuli. A rough way of thinking about this stimulus manipulation is that we changed the slope of the ground on which the walker was walking. Conceptualizing our stimulus manipulation this way, a right-facing walker rotated clockwise would appear to be walking downhill if the rotation were less than 90° in magnitude. Similarly, a right-facing walker rotated counterclockwise would appear to be walking uphill if the rotation were less than 90° in magnitude (clearly, a walker actually walking downhill or uphill would exhibit different biomechanics than our stimuli; we only employ the analogy here as an explanatory device). 

Our stimulus manipulation resulted in walkers presented at three orientations: one veridical and two rotated in the x-y or fronto-parallel plane. Example stimuli shown in each condition are presented in Figure 6. We rotated the PL walkers themselves, while the noise field retained the same orientation as in the two preceding experiments. To reduce the hand-based orientation cue most effectively, we applied two successive rotations to each facing direction: Right-facing stimuli were rotated clockwise and left-facing stimuli counterclockwise. The first rotation resulted in the path traced out by the hands being roughly horizontal; the second rotation was of equal magnitude in the same direction. We call this set of three stimulus orientations (veridical, rotated θ°, and rotated 2θ°) the unidirectional-rotation condition (see the left column of Figure 6). Applying this sequence of rotations to the isolated feet, however, introduces an additional spatial cue: The feet systematically appear displaced to the side opposite of their facing direction. To ensure that any differences between the hands and feet conditions were not solely due to elevated efficiency for the feet (driven by the spatial-displacement artifact), we also applied a single rotation in each direction, clockwise and counterclockwise, to the right-facing walkers. We applied the same two rotations to the left-facing walkers. We call this set of stimuli, including veridical orientation, clockwise-rotated, and counterclockwise-rotated, the bidirectional-rotation condition (see the right column of Figure 6). 

 

Figure 6

 

Movie showing left-facing complete stimuli used in the two rotation conditions in Experiment 3. The unidirectional-rotation condition is shown in the left column, and the bidirectional condition is shown in the right column. Point-light walkers could appear at any one of the three orientations depicted for each rotation condition.

Figure 6

 

Movie showing left-facing complete stimuli used in the two rotation conditions in Experiment 3. The unidirectional-rotation condition is shown in the left column, and the bidirectional condition is shown in the right column. Point-light walkers could appear at any one of the three orientations depicted for each rotation condition.

If orientation were the cue that observers had relied upon in the first two experiments to achieve such high efficiencies with the hands, then we would expect the introduction of unidirectional rotational uncertainty to have the largest negative impact on efficiency in the isolated-hands condition. This is because the distribution of orientations overlapped the most between leftward- and rightward-facing PL walkers under these conditions (see Figure 4). Although the bidirectional-rotation condition also makes orientation a less reliable cue for isolated hands, there are still trajectory orientations belonging exclusively to each facing direction. Thus, we would also expect efficiency for isolated hands to be negatively affected in the presence of bidirectional rotational uncertainty, albeit to a lesser extent. In contrast, the isolated feet should be robust to both kinds of rotational uncertainty if humans are not primarily relying on such a cue with respect to the feet (for evidence that they are not, see Hirai, Chang, et al., 2011). In other words, if the elevated efficiencies for the hands observed in Experiments 1 and 2 were due solely to observers' learning to make use of the orientation of the hands' trajectory, we would expect observers to be unable to perform the task in the unidirectional-hands condition. Further, efficiency for bidirectionally rotated hands should be reduced relative to the previous two experiments, whereas efficiency for the feet should be robust to both forms of rotational uncertainty. 

Methods

Eleven observers (nine women, two men) between the ages of 18 and 35 participated in Experiment 3. One male observer was excluded from analysis because of an inability to estimate thresholds in at least three conditions. 

The remaining methods for Experiment 3 were identical to those for Experiment 1, with the exception that we introduced picture-plane rotational uncertainty for the complete, isolated-feet, and isolated-hands conditions. Because we were specifically concerned with the relative performance for the hands and feet when shown in isolation, we generated two sets of rotated stimuli: those rotated bidirectionally and those rotated unidirectionally from the original orientation. For a given facing direction, the bidirectionally rotated set included the stimuli at the original orientation, one orientation rotated clockwise from the original, and another orientation rotated counterclockwise. The unidirectionally rotated set contained stimuli at the original orientation and at two orientations resulting from sequential rotations in a single direction. 

Each stimulus was rotated twice for each rotation manipulation. For example, consider a left-facing PL walker: In the unidirectional-rotation condition, it could appear in the original orientation, rotated 16° counterclockwise, or rotated 32° counterclockwise. Similarly, the corresponding right-facing version could appear in the original orientation, rotated 16° clockwise, or rotated 32° clockwise. In the bidirectional-rotation condition, both the left- and right-facing PL walkers could appear in the original orientation, rotated 16° clockwise, or rotated 16° counterclockwise. Figure 4 demonstrates the impact of the different rotation manipulations on the distribution of orientations of a line connecting the two points visible in the isolated-feet and isolated-hands conditions. In an effort to make the ideal-observer simulations more tractable, we reduced the amount of temporal uncertainty in this experiment by half, retaining only three of the original six temporally shifted versions of the stimuli. 

As in Experiments 1 and 2, the task was blocked by condition with 125 trials per block. However, since there were six conditions (unidirectional and bidirectional rotational uncertainty for each of full, isolated feet, and isolated hands), there were 750 trials overall. We included 20 rather than 10 noise-free, high-contrast practice trials before each block. 

Results

The results for Experiment 3 are shown in Figure 7. These data show that none of the 10 subjects were able to perform the task in the unidirectional-rotation isolated-hands condition (Figure 7A). Therefore, we were unable to estimate thresholds or calculate efficiencies in that condition (i.e., efficiency was zero for all subjects). Additionally, in the bidirectional-rotation isolated-hands condition, the three subjects who performed the task with the unidirectional-rotation hands condition prior to the bidirectional-rotation hands condition were unable to perform the task. Therefore, we dichotomized the outcome measure into a binomial pass/fail measure, based on whether the task could be performed or not for each rotation-by-part condition. A Fisher's exact test revealed that subjects were more often able to perform the task in the complete and isolated-feet conditions relative to the isolated-hands conditions (p = 0.04, see Table 3 for the contingency table). Within the isolated-hands condition itself, a binomial test revealed that subjects failed more often in the unidirectional-rotation than the bidirectional-rotation condition (z = 3.28, p = 0.001, one-tailed). The results of a repeated-measures two-way ANOVA carried out exclusively on the log-transformed complete and isolated-feet efficiencies revealed that there were main effects of part, F(1, 27) = 7.28 , p = 0.012, and rotation, F(1, 27) = 7.73, p = 0.010, and a significant Part × Rotation interaction, F(1, 27) = 6.20, p = 0.020. While efficiency in the isolated-feet condition did not differ across rotation conditions, efficiency in the complete condition was higher under unidirectional than bidirectional rotation. 

Figure 7

 

(A) Contrast-energy thresholds and (B) efficiencies for Experiment 3. No thresholds were able to be estimated in the unidirectional-rotation isolated-hands condition for any subject. This lack of estimate is shown by a bar with infinite height. Three subjects in the bidirectional-rotation isolated-hands condition were unable to provide threshold estimates due to inability to perform the task. Efficiencies for the isolated-hands conditions were not able to be quantitatively compared to those measured in the complete and isolated-feet conditions, and are shown for illustrative purposes only. Black dots correspond to individual subjects, and error bars correspond to ±1 standard error of the mean.

Table 3

 

Contingency table showing dichotomized data in Experiment 3. Notes: Data in the cells are counts, treating each subject in each condition as a single case (10 subjects × 2 rotation conditions = 20 cases per part condition).

 

Discussion

Our results suggest that the efficiency advantage for the hands that we found in our first two experiments was due to observers relying on the orientation of the path traced out by the hands rather than the local opponent motion of the hands themselves. Introducing rotational uncertainty eliminated (in the unidirectional-rotation condition) or impaired (in the bidirectional-rotation condition) observers' ability to perform the left/right discrimination task. Note that for no condition was the task impossible for the ideal observer. This important result means that there was information physically present to perform the left/right discrimination task in every condition, and that the inability of our human observers to perform above chance in some conditions was due to a failure of how information was being processed by the visual system rather than a mere absence of information about PL-walker facing direction. Our results therefore clearly demonstrate that when the relative motion of body parts is rendered as the only reliable source of information about walking direction, the motion of the feet is processed with particular efficiency by the human visual system. 

The significant Part × Rotation interaction for the complete and isolated-feet stimuli suggests a qualitative difference in the signal observers relied upon in the two part conditions. Specifically, the signal that observers relied upon for isolated-feet stimuli was evenly affected regardless of rotation direction, whereas the signal they relied upon for the complete stimuli was less usable in the bidirectional-rotation condition than in the unidirectional-rotation condition. This finding rules out the possibility that observers used the same strategy for both the complete and isolated-feet conditions, even though the feet were present in both stimuli. One possible explanation for this result is that observers may have relied upon a strategy that was based on the relationship between the feet and the rest of the body in the complete condition, whereas this strategy was simply unavailable to them in the isolated-feet condition. Under conditions of unidirectional rotational uncertainty, the feet appear at locations that are unique to each facing direction; however, under conditions of bidirectional rotational uncertainty, the locations of the feet relative to the body are not indicative of facing direction, because the clockwise rotation of one facing direction results in feet in the same general location as the counterclockwise rotation of the other facing direction. Although this is a plausible interpretation of our results, further research into the roles played by various forms of spatial and temporal uncertainty in the perception of biological motion would be needed to definitively tease these factors apart. 

General discussion

Taken together, there are three major findings that stand out from our experiments. First, the higher overall efficiency found in the isolated-parts conditions suggests that the human visual system is adapted for extracting information from hands and feet above and beyond what we would expect based on the information content of the stimuli alone. Second, observers are particularly efficient at using in the information provided by the hands to judge walking direction of human locomotion. And third, the efficiency advantage for the hands appears to be due to the exploitation of form-based orientation information rather than purely motion-based information that observers use to judge walking direction based on the feet. We consider the implication of each of these results below. 

Hands and feet

Our findings extend the results of several previous studies exploring the role of different body parts on the perception of biological motion (Chang & Troje, 2009; Hirai, Saunders, & Troje, 2011; Mather et al., 1992; Thurman, Giese, & Grossman, 2010; Troje & Westhoff, 2006; van Boxtel & Lu, 2015). Mather et al. (1992) found that performance, measured in terms of accuracy, suffered the most with the simultaneous removal of the feet and hands, compared to the elbows and knees or the shoulders and hips, suggesting that the hands or the feet are heavily relied upon in direction discrimination of PL walkers. More recently, Troje and Westhoff (2006) found evidence inconsistent with an exclusively configural interpretation of the inversion effect for PL walkers, revealing the importance of the feet. When PL stimuli were spatially scrambled, effectively eliminating configural cues, accuracy was still lower for inverted than for upright PL walkers. In a subsequent experiment, they narrowed down the source of this difference to the feet, finding that inverting the entire walker reduced accuracy the same amount as inverting just the feet, while inverting the entire walker but leaving the feet in veridical orientation produced no difference in accuracy from walkers in a completely veridical orientation. Importantly, their stimuli included cat and pigeon PL walkers in addition to human PL walkers, which rendered any strategy that relied on any particular body configuration effective for only one third of the trials. While this inclusion of multiple species of PL walker was critical for Troje and Westhoff's interpretation of the mechanism for discriminating local motion of the feet as evolutionarily old and possibly a life detector, it also increases the likelihood of reliance on the local motion as the most informative component across all three stimuli. 

Chang and Troje (2009) followed up on this work by identifying the vertical-acceleration profile of the feet as the aspect of local motion that drives the inversion effect for PL stimuli. In their experiments, although human observers were still able to discriminate walking direction under conditions where this acceleration profile was rendered constant, they found that naturally accelerating feet were discriminated more effectively than the artificial constant-velocity versions of the feet. Although their results are consistent with the idea that human observers use the motion of the feet as a life detector, it does not rule out the possibility that human observers may simply employ a more generalized process that allows them to lock on to the most informative aspects of a stimulus. However, our findings are inconsistent with the generalized-process explanation, providing clear evidence that human observers do indeed employ a mechanism tuned to the motion of the feet. Of course, this reasoning relies upon the assumption that our stimulus manipulations effectively removed all nonmotion cues that observers could have exploited to make use of the feet. Although it is possible that some unspecified nonmotion cue or cues still remained in the feet of our stimuli even after our attempts to carefully eliminate them through systematic stimulus control, it is unclear to us what those cues would be. 

Our findings show that humans are especially efficient at performing direction discrimination of PL walkers using the hands or feet exclusively. Our findings also show that the orientation of the path traced out by the hands is a critical feature for this ability in the case of the hands. The role of the feet in direction discrimination is already well documented (Chang & Troje, 2009; Hirai, Saunders, & Troje, 2011; Mather et al., 1992; Thurman et al., 2010; Troje & Westhoff, 2006). Previous research has implicated both the form and the motion of the upper body in biological-motion perception (Thurman et al., 2010; van Boxtel & Lu, 2015). In a clever application of reverse correlation, van Boxtel and Lu (2015) showed a role for the wrists in action discrimination, noting that the wrists' opponent motion and extent of excursion are second only to those of the feet. Thurman et al. (2010), using bubbles to perform reverse correlation, found that humans mostly relied on the feet and the upper body. Especially in very brief presentations, the shoulders and head were used by humans to discriminate facing direction. At longer stimulus durations (467 ms, still shorter than stimuli presented here), they found that humans also relied on the arms. However, based on their classification images, it seems that the region relied upon is where the arms pass the vertical meridian, and does not extend to the wrists (however, in some classification images it appears that the region including the wrists does approach significance). Mather et al. (1992) found that simultaneous removal of both the hands and feet brought a substantial drop in direction-discrimination performance. This finding suggests a role for the hands but leaves open the possibility that the absence of the feet was the primary factor in the performance drop. Our findings extend these previous results by demonstrating that there is probably an integral characteristic of the hands that humans use to perform direction discrimination. 

Form and motion

Taken together, our experiments provide new evidence suggesting that distinct mechanisms may be responsible for direction discrimination of the hands and feet. This stems from the finding that rotational uncertainty had a very different effect on performance for these two classes of body parts in our experiments. Namely, efficiency for the feet was virtually unaffected by both unidirectional and bidirectional rotational uncertainty, whereas efficiency for the hands was zero with unidirectional rotational uncertainty and greatly reduced with bidirectional rotational uncertainty. Importantly, the ideal observer could still discriminate left-facing from right-facing hands even in the presence of both types of rotational uncertainty, showing that information was always physically present to perform the task. Therefore, humans' difficulty in the bidirectional-rotation hands condition and inability in the unidirectional-rotation hands condition shows that this form of uncertainty uniquely affects the mechanism used by humans to discriminate direction in the isolated-hands condition in Experiments 1 and 2. 

The output of the model used by Thurman et al. (2010) supports this interpretation. That model included separate form and motion pathways; the hands/wrists came out as a diagnostic feature of facing direction in the form pathway but not the motion pathway, while the feet showed the opposite pattern of results. 

van Boxtel and Lu (2015) have reported findings that further support our interpretation. Their design and analysis allowed them to examine not only the contributions of individual points but also the contributions of pairs of points to action discrimination. Of particular relevance is their finding that the hip and the wrist were correlated in their influence on action discrimination. However, the two wrists were not correlated with one another, clearly suggesting that the wrists are important for a form-based mechanism rather than a local-motion-based one that relies on the interaction between body parts. Similarly, the rotation manipulation in our Experiment 3 preserved the local motion of the hands (analogous to the wrists in van Boxtel and Lu's study) while simultaneously altering the relationship between the hands and the hips. Although the hip points were not visible in the isolated-hands conditions of Experiment 3 in our study, in veridical orientation the location of the hips could be inferred from the hands' path possibly using a template-based mechanism similar to that proposed by Lange, Georg, and Lappe (2006). Rotational uncertainty could have disrupted the process altogether, preventing any template match that would provide an inferred location for the hip; or rotation might simply have led to multiple solutions for the hip-location inference, with just as many solutions belonging to right-facing as left-facing walkers. If this were the case, we would expect that including a single point for the hip and a single point for the hand would allow observers to perform direction discrimination even with the unidirectional rotation manipulation. Alternatively, it may be that more than just a hip and a hand are needed for the form-based mechanism to operate accurately. Our experiments do not allow us to decide between these two possibilities, since the only remaining condition includes all the points. Exploring this issue further would be a fruitful direction for future research. 

Higher efficiency for isolated parts

Both of the isolated conditions showed higher efficiency than the complete condition and the two missing conditions in our experiments. Why might this be? Our interpretation is that observers rely primarily on the hands and feet to determine walking direction, and when contrast is distributed to the other points, the information they carry is left largely unused by human observers, resulting in an overall reduction in efficiency. In other words, at a given level of image contrast energy, the amount of contrast at each individual point within the image will be relatively greater if there are fewer total points present within the image. In the case of our isolated conditions, all the image contrast is concentrated into just two points, whereas in the missing and complete conditions, the image contrast is distributed across a greater number of points. If different mechanisms are responsible for computing facing direction from specific points or sets of points, then changing the distribution of contrast across points should change the relative contribution of these mechanisms. More specifically, distributing energy across points that are used less efficiently by the visual system (i.e., points other than the hands and feet) should result in an overall reduction in efficiency, which is exactly what we observed in our experiments. 

We should also note that observers were provided explicit accuracy feedback on each trial in all of our experiments. Thus, it is possible that they learned to make use of the isolated-hands and -feet stimuli based on feedback, and that our experiments might have yielded different results without its inclusion. Experiment 3 may have been particularly susceptible to learning based on feedback, given the inability of three observers to perform the task in the bidirectional-rotation hands condition after having already seen the unidirectional hands condition. On the other hand, previous experiments have convincingly demonstrated that observers still engage in perceptual learning even without the inclusion of explicit feedback, albeit more slowly (e.g., Herzog & Fahle, 1997). Whether feedback played an important role in our experiments is an empirical question worthy of future investigation. 

Relation to neural mechanisms

Recent findings from Vangeneugden, Peelen, Tadin, and Battelli (2014) suggest a possible neural basis for the difference in performance we observed under conditions of rotational uncertainty. Using multivariate pattern analysis and transcranial magnetic stimulation, they showed that an area of occipital cortex called the extrastriate body area is causally implicated in the perception of body configuration (facing-direction discrimination) in PL walkers, whereas a part of temporal cortex (the posterior part of the superior temporal sulcus) was integral for using motion cues (walking-direction discrimination; forward or backward) to distinguish the different forms of PL walkers. Our findings and theirs are complementary and can be thought of as adding noise to different places in the signal-processing path for PL walkers. In our study we added uncertainty to the stimulus, while in their study by using transcranial magnetic stimulation they added uncertainty to the putative neural mechanisms used to process the stimulus. Based on their results we would expect transcranial magnetic stimulation to the extrastriate body area to selectively impair the isolated hands, comparable to adding unidirectional rotational uncertainty. 

Methodological diversity in biological-motion research

A complicating factor when comparing results from studies on biological-motion perception is the fact that a variety of noise types are employed across experimental paradigms. In the current experiments, we used additive luminance noise, following the approaches of Gold et al. (2008) and Lu and Liu (2006). Similarly, Thurman and Grossman (2008) and Thurman et al. (2010) used the bubbles method, which can be understood as multiplicative pixel noise. However, by far the most common form of noise used in experiments on biological-motion perception is a random field of flickering dots that mimic the properties of those used to generate the PL-walker stimulus itself (Bertenthal & Pinto, 1994; Mather et al., 1992; Mather & Murdoch, 1994; Troje & Westhoff, 2006). There were two basic reasons why we chose not to use a more traditional form of random-dot noise in our experiments. First, following the previous work of Gold et al. (2008) on the efficiency of biological-motion perception, we used luminance noise to make the ideal-observer analysis straightforward and easy to compare to previous experiments that have measured the efficiency of information use for complex patterns. Second, white luminance noise is the most feature-agnostic choice one can make for adding variability to a stimulus, because it introduces random variation along all possible stimulus dimensions with equal probability—including those most relevant to the task observers are being asked to perform. By taking this admittedly more conservative approach, we avoid being forced to make strong assumptions about what stimulus dimensions are relevant for performing a particular task. 

Of course, a downside of using white luminance noise is that it makes it difficult to compare our results with those from previous studies that have used more focused forms of noise (e.g., random dots). For example, our estimate of the range of absolute level of efficiencies in our tasks (on the order of 0.5%–2%) differs markedly from that of a recent study by van Boxtel and Lu (2015), who used dot-trajectory noise and found much higher efficiencies (on the order of 20%). There are several possible reasons for this discrepancy, including the use of quite different PL-walker tasks (direction discrimination vs. action discrimination) and different methods for estimating efficiency (response consistency vs. ideal-observer analysis). However, as already mentioned, the use of white luminance noise adds variability to all stimulus dimensions and thus will affect any aspect of the stimulus that an observer is using to perform the task—including the ones they are not particularly efficient at using. As a result, adding broad-spectrum variability like white luminance noise is almost guaranteed to result in lower absolute efficiencies than using a more focused dot-trajectory noise, simply because the ideal observer uses all stimulus dimensions in a statistically optimal manner but a human observer almost never does. 

Beyond the issue of choice of noise, another difference between our experiments and those reported previously in the literature is stimulus size. Our stimuli subtended between 0.96° and 1.1° head-to-foot, making them markedly smaller than those used in previous work (Mather et al., 1992: 1.6° shoulder-to-hip; Troje and Westhoff, 2006: 5.7° head-to-foot, Thurman et al., 2010: 7.6° head-to-foot; but see Gold et al., 2008: 1.6° head-to-foot). We used small stimuli because increased eccentricity is known to be a source of inefficiency for human observers (Or & Elder, 2011). Since the feet are among the most peripheral parts of the stimulus, potential inefficiency due to eccentricity would have made interpreting our results difficult if we had found no efficiency advantage for the feet. 

Lastly, a potential limitation of our application of ideal-observer analysis to the perception of biological motion is that our findings may simply reflect an idiosyncrasy of the specific PL-walker stimuli that we tested in our experiments, rather than a more general property of the perception of PL walkers. While we cannot rule out this possibility, it is worth noting that we chose to test observers with 10 different walkers rather than just one (the minimum number required for direction discrimination), in part to address this specific concern. It is also worth noting that this same limitation is common to most previous experiments involving the application of ideal-observer analysis to the study of complex and natural stimuli, such as letters, objects, and faces (e.g., Gold, Bennett, & Sekuler, 1999; Pelli, Burns, Farell, & Moore-Page, 2006; Tjan, Braje, Legge, & Kersten, 1995). 

Conclusions

Our study has offered compelling new evidence supporting the notion of a processing advantage for hands and feet in perceiving human biological motion. Further, these advantages appear to be attributable to different mechanisms: a form-based mechanism for the hands and a motion-based one for the feet. Although some of the methodological differences mentioned between our experiments and those conducted previously make it somewhat difficult to compare results across studies, it is worth noting that our findings about the privileged roles of the feet and the hands are generally consistent with previous findings. The fact that we have obtained converging results using such different methods and stimulus conditions suggests that our findings likely reflect something relatively deep about the way the visual system processes information related to human biological form and motion. An important future goal is to further explore the generality of these results. 

Acknowledgments

The authors are grateful to Doori Lee for assistance collecting data and to Christopher Taylor for helpful discussions of the initial idea for the first experiment. This research was supported by National Institute of Heath Grant R01-EY019265. 

Commercial relationships: none. 

Corresponding author: Jason M. Gold. 

Email: . 

Address: Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA.