Why are you angry with me? Facial expressions of threat influence perception of gaze direction

Abstract

Gaze direction can influence the processing of facial expressions. Angry faces are judged more angry when displaying a direct gaze compared to an averted gaze. We investigated whether facial expressions have a reciprocal influence on the perception of gaze. Participants judged the gaze of angry, fearful and neutral faces across a range of gaze directions. Angry faces were perceived as looking at the observer over a wider range than were fearful or neutral faces, which did not significantly differ. This effect was eliminated when presenting inverted faces, suggesting these results cannot be accounted for by differences in visible eye information. Our findings suggest the existence of a reciprocal influence between gaze direction and angry expressions.

Introduction

Accurate discrimination of eye gaze direction is an essential skill. The ability to identify whether a predator or conspecific aggressor is looking at you provides a clear evolutionary advantage in evading danger. Recent evidence indicates that gaze direction influences the processing of facial expressions (Adams & Kleck, 2005). Here we addressed whether facial expression exerts a reciprocal influence on the perception of gaze. 

Numerous studies report that angry faces are processed more rapidly, and are perceived as being more angry, when displaying a direct (or mutual) gaze compared to an averted gaze (Adams & Kleck, 2005; Bindemann, Burton, & Langton, 2008; Graham & LaBar, 2007; Sander, Grandjean, Kaiser, Wehrle, & Scherer, 2007). By contrast, there is evidence that the processing of fearful expressions is enhanced when the face displays an averted gaze (Adams & Kleck, 2005; Sander et al., 2007). Adams and Kleck (2003) propose that these findings accord with behavioral nature of these two expressions. Anger and direct gaze both share an approach related behavior, by contrast, fear and averted gaze are associated with avoidance. Thus, when gaze direction matches the behavioral intent signaled by a specific expression, the processing of that emotion is enhanced; known as the ‘shared signal hypothesis’ (Adams & Kleck, 2003). 

However, although subsequent work has confirmed that direct gaze has a consistent influence on the processing of angry faces, the influence of averted gaze on the perception of fearful faces is less robust. A number of studies report that gaze has no effect on the processing of fearful faces or that the processing of fear is enhanced when the face displays a direct gaze (Bindemann et al., 2008; Graham & LaBar, 2007; Hess, Adams, & Kleck, 2007). 

To date, only two studies have examined whether facial expression influences perceived direction of gaze. Martin and Rovira (1982) found that when viewing a video clip of another individual, participants were more likely to infer that the individual was looking at them when displaying a happy expression compared to a non-emotional expression. A more recent study that manipulated gaze direction and head orientation conjointly found a similar ‘direct gaze’ bias for happy faces when compared to angry, fearful or neutral faces (Lobmaier, Tiddeman, & Perrett, 2008). The same study also found that angry faces elicited more direct responses than fearful or neutral faces, which did not significantly differ. However, since gaze was always congruent with head direction in Lobmaier et al.'s study, the latter could have been used as a cue to the direction of social attention, thus it remains to be established whether expression specifically affects perception of gaze per se. To examine the influence of angry and fearful expressions on gaze alone, our current study used only full face images in which gaze was varied while head direction was held constant. 

It is also unclear whether other factors may have accounted for the findings of Lobmaier et al.'s study. When considering differences between angry and fearful faces it is important to note that these two expressions differ with regard to the amount of visible iris and sclera. When expressing anger, changes in facial muscles lead to a narrowing of the eyes, while expressing fear serves to widen the eyes (Susskind et al., 2008). Since the location of the iris within the sclera is thought to be the primary cue in determining gaze direction (Gibson & Pick, 1963), it is important to exclude this interpretation of the results. To address this, we included an inverted face condition, which has been shown to impair facial expression recognition (McKelvie, 1995). This enabled us to measure gaze discrimination based upon the same amount of eye information as upright faces, while disrupting processing of facial expression. Furthermore, to account for any effects of facial expression on gaze perception that may be due to differences in the fundamental affective properties of valence and arousal, we used a set of angry and fearful face images that were matched on ratings of these dimensions. 

Compared to perceptual acuity, our ability to determine whether another's gaze is directed at us appears relatively poor. For example observers show a bias to perceive that they are being looked in the eye even when the looker's gaze is directed at their mouth or nose (Lord & Haith, 1974). Recently, Gamer and Hecht (2007) have measured the range of horizontal gaze directions over which an individual perceives another to be looking at them. They have likened this range to a cone; whereby gaze directions are gradually less likely to be seen as direct as gaze deviates from true direct. For example, at a distance of 1 m any gaze shift within a range of approximately 8° (4° left or right of true direct) is perceived to be directed at the observer (Gamer & Hecht, 2007). In the current study we use the metaphor of ‘cone of gaze’ to refer to the width over which an observer feels another's gaze is directed at them. Determining the width of this cone provides a potentially more sensitive measure of perceptual differences than those used in previous studies (Lobmaier et al., 2008; Martin & Rovira, 1982). In particular, we addressed whether angry faces are associated with a wider cone of gaze (i.e. perceived as looking at the observer over a wider range of gaze directions) than fearful or neutral faces. The shared signal hypothesis (Adams & Kleck, 2003) also predicts that fearful faces should serve to enhance the perception of averted gaze (i.e., narrower cone of gaze than neutral faces). However, behavioral evidence demonstrating a more variable or absent influence of gaze direction on the perception of fearful faces suggests that the cone of gaze for fearful expressions should be marginally wider or not significantly different from that found for neutral faces. 

Materials and methods

Participants

Thirty five participants (18 female) aged between 20 and 43 years (M = 28.49, SD = 7.16) took part in the study which was approved by Cambridgeshire Psychology Research Ethics Committee. All volunteers provided written and informed consent and were paid for participating. 

Stimuli

The face stimuli consisted of grayscale photographs of four male identities, each displaying a neutral, angry and fearful expression. Images were taken from the NimStim Face Stimulus Set (Tottenham et al., 2009) and the Karolinska Directed Emotional Faces (KDEF) image set (Lundqvist & Litton, 1998). We confined our stimuli to male faces only as male and female faces have been shown to differentially influence recognition of angry expressions (Becker, Kenrick, Neuberg, Blackwell, & Smith, 2007). The hair and non-facial areas were removed from the photographs so that only the central face area was visible (Figure 1). Face images subtended a visual angle of approximately 12° × 8°. Gaze direction was manipulated using Adobe Photoshop (http://www.adobe.com/); as used in other studies (Adams, Gordon, Baird, Ambady, & Kleck, 2003; Adams & Kleck, 2005; Graham & LaBar, 2007). The position of the iris of both eyes was shifted to the left or the right at incremental steps of one pixel per image, equivalent to a shift of 0.03 cm, or a visual angle of approximately 5′ minutes of arc viewed from a distance of 50 cm. The images were selected from a set of angry, fearful and neutral faces that had been rated for valence and arousal by a group of 16 participants, who did not take part in the current study. Ratings showed that fearful and angry faces did not differ on measures of arousal (Z = 0.59, p = 0.99) or valence (Z = 1.74, p = 0.18; sidak corrected). Thus, any differences in gaze discrimination cannot be attributed to differences in arousal and valence alone. 

Figure 1

 

Example stimuli. Each facial identity displayed an angry, fearful or neutral expression. An example of one identity posing each of three expressions, is shown for five of the gaze direction used: 7 pixels left, 3 pixels left, direct gaze, 3 pixels right and 7 pixels right.

Design and procedure

Participants sat approximately 50 cm in front of a 15 inch monitor. A chin rest was used to maintain head position and distance from the screen. Faces were presented at the center of a computer screen on a gray background, using E-prime software (Psychology Software Tools Inc., Pittsburgh, PA). Eleven gaze directions were used in total—direct gaze and 2, 3, 4, 5 and 7 pixels left and right of center ( Figure 1). Each face was displayed for 200 ms, followed by a variable ISI between 1.5 and 3 secs (mean = 2.5 secs). Participants were required to press one of three buttons according to whether they considered the face was looking to their left, their right, or directly at them. Participants completed two experimental blocks consisting of 264 trials each. This gave a total of 16 trials for each expression at each gaze direction for each of the upright and inverted conditions. One block contained upright faces the other inverted faces, with the order counterbalanced across participants. Trials were presented in a pseudo-randomized order with each identity displaying each emotion and each gaze direction an equal number of times, with identical conditions repeated over no more than two successive trials. 

Results

For each participant's data, separate psychometric (logistic) functions were fit to the proportion of ‘left’ and ‘right’ responses as a function of gaze direction. The sum of the ‘left’ and ‘right’ fitted functions were subtracted from one in order to define a further function fitting the proportion of ‘direct responses’. The procedure was repeated for each expression (i.e. nine functions in total). Next, we calculated the crossover point between the fitted direct and left functions and the crossover point between the fitted direct and right functions. For each participant this gave us an objective measure of the range of gaze directions that were perceived as looking at them (i.e., their cone of gaze), measured as the total of the absolute values of the crossover points (left/direct-crossover + right/direct-crossover) for each expression ( Figures 2A and 3A). The data from 10 participants were discarded as crossover points could not be calculated for either the upright or inverted condition (i.e. number of ‘direct’ responses were greater than 50% for any gaze direction). This left 25 participants' data (14 female, mean age = 27.04, SD = 6.8) for analysis. 

Figure 2

 

(A) Plot showing mean fitted left, direct and right responses as a function of gaze direction for neutral, angry and fearful faces across all participants in the upright face condition. Dashed lines show cross-over points used to calculate cone of gaze. Arrows represent width of cone. (B) Mean width of cone across all participants for neutral, angry and fearful expressions in the upright face condition. Error bars show ±1 SE. **P < 0.005.

Figure 3

 

(A) Plot showing mean fitted left, direct and right responses as a function of gaze direction for neutral, angry and fearful faces across all participants in the inverted face condition. Dashed lines show cross-over points used to calculate cone of gaze. Arrows represent width of cone. (B) Mean width of cone across all participants for neutral, angry and fearful expressions in the inverted face condition. Error bars show ±1 SE.

Cone of gaze values were entered into a 3 × 2 × 2 repeated measures ANOVA examining emotion (anger, fear, neutral) and orientation (upright, inverted) as repeated measures factors and gender of participant as a between subject factor. Greenhouse–Giesser corrections were applied where appropriate. For all pairwise comparisons p-values were sidak corrected for multiple comparisons. 

The results revealed no main effect of participant gender ( F = 0.31, p = 0.58) and no interactions with gender ( p's > 0.11), thus this factor will not be discussed further. We found no effect of orientation ( F(1,23) = 0.93, p = 0.34) or emotion ( F(1.5,35.5) = 0.48, p = 0.57), however, there was a significant interaction between orientation and emotion ( F(1.5,35.1) = 3.80, p < 0.05, η _p ² = .14). To explore the interaction we entered cone of gaze values into separate repeated measures ANOVAs for upright faces and inverted faces addressing the effect of emotion (anger, fear, neutral). For upright faces, there was a significant main effect of emotion ( F(2,46) = 7.75, p < 0.005, η _p ² = .25) (mean response functions are shown in Figures 2A and 3A). Paired comparisons revealed that participants showed a wider cone of gaze for angry faces compared to fearful ( t(24) = 3.85, p < 0.005) or neutral faces ( t(24) = 2.77, p < 0.05) ( Figure 2B) with no difference between fearful and neutral faces ( t(24) = 0.98, p = 0.70); all comparisons sidak corrected. By contrast, no effect of emotion was found for inverted faces ( F(1.4,46) = 1.58, p = 0.21) ( Figure 3B). 

To determine whether these effects simply reflected a bias for participants to respond ‘direct’ when viewing angry faces we entered the proportion of direct responses in the ‘true’ direct gaze condition into an additional one-way ANOVA with emotion (anger, fear, neutral) as the repeated measures factor. This revealed no effect of emotion for true direct faces ( F(2,48) = 0.57, p = 0.56), providing no support for a response bias. 

Discussion

Our study investigated the influence of threatening facial expressions on the perception of gaze direction. We used the metaphor of ‘cone of gaze’ (Gamer & Hecht, 2007) to refer to the range of gaze directions that are perceived as directed at the observer. Angry expressions were associated with a wider cone than fearful or neutral faces, which did not significantly differ. Inverting the faces eliminated this effect, demonstrating that the influence of expression is unlikely to reflect differences in the amount of visible iris or sclera in the different expressions. 

Our findings accord with previous work showing that for conjointly oriented gaze and heads, faces displaying angry expressions are more likely to be categorized as oriented towards the observer than fearful or neutral faces (Lobmaier et al., 2008). Further to this, we show that expression affects the perception of gaze direction specifically, as opposed to a conjunction of head and gaze or head direction alone, given that they were manipulated in tandem in Lobmaier et al.'s study. By controlling for rated arousal and valence, we also demonstrate that our findings are not attributable to differences in these fundamental affective dimensions, but relate to the specific nature of the expressions themselves. 

As discussed earlier, numerous studies have shown that the perceived intensity of an angry face is increased when displaying a direct gaze. By contrast, the effect of gaze on perception of fearful expressions is less consistent, with some studies showing an enhancement for averted gaze (Adams & Kleck, 2005; Sander et al., 2007), some for direct gaze (Bindemann et al., 2008), and others no effect (Graham & LaBar, 2007; Hess et al., 2007). Here we find an essentially reciprocal influence of expression on gaze—angry expressions widened the range of gaze directions perceived as direct relative to the neutral and fearful face conditions, but fearful faces did not lead to enhanced perception of averted gaze relative to neutral faces (i.e. narrowing of the range of gaze directions perceived as direct). 

To assess whether the effect for angry faces reflected a genuine widening of gaze directions perceived as direct as opposed to a bias to respond ‘direct’ when viewing angry faces, we showed that the number of direct responses for the ‘true’ direct gaze condition did not differ between expressions. This increased tendency to categorize ambiguous gaze directions as direct mirrors the influence of gaze on facial expression, i.e. gaze has a greater influence on the processing of expression when the expression is difficult to discriminate (Bindemann et al., 2008; Graham & LaBar, 2007). However, our finding that participants' gaze perception did not differ between fearful and neutral faces is difficult to reconcile with the shared signal hypothesis, since this would have predicted an increased tendency to categorize ambiguous gaze directions as averted for fearful expressions; see also Lobmaier et al. (2008). 

Instead, we suggest that perceiving the gaze of an angry face as being directed at oneself reflects an adaptive response when faced with a signal of interpersonal threat. The behavioral affordance associated with an angry face is largely dependent upon the direction of gaze. In other words, angry faces afford danger only when their gaze is directed at the observer. Indeed, affordances are said to be more readily communicated over multiple modalities using a configuration of cues (e.g. expression and gaze) rather than single cues in isolation (Zebrowitz & Collins, 1997). When the direction of gaze appears ambiguous, a strategy that infers threat as being directed at the observer may prove more effective than one which ignores the threat. Thus, a false alarm (incorrectly perceiving that the threat is directed at oneself) would be less costly than a miss (ignoring a relevant threat). Fearful expressions infer the presence of danger in the environment, and thus afford danger regardless of gaze direction. Therefore a similar strategy for fearful faces would appear non-advantageous. This may also explain the less robust effect of gaze on perception of fearful expressions (Bindemann et al., 2008; Graham & LaBar, 2007; Hess et al., 2007). 

As noted, a number of participants tended to perceive gaze as direct across all directions and therefore could not be included in the final analysis. This suggests that their cone of gaze extended over a wider range of directions than were used in this study. Variation in the extent of the cone is something we are currently exploring, as well as the issue of whether such variations may be related to individual differences in relevant personality traits. 

In conclusion, we demonstrated that angry faces evoked a greater perception of being looked at than fearful or neutral faces, while the perception of gaze displayed by fearful faces and neutral faces did not differ. Further, we have shown that this cannot be attributed to differences in the amount of information available from the eyes, or the influence of expression on other cues to social attention such as head direction. In addition, these results cannot be accounted for by differences in the rated valence or arousal of the facial expressions. We suggest these results are consistent with a ‘self-referential bias’ when faced with ambiguously directed signals of interpersonal threat, and are difficult to reconcile with the proposed ‘shared signal’ hypothesis. 

Acknowledgments

This research was funded by the UK Medical Research Council under project code U.1055.02.001.0001.01 (Andrew J. Calder). We wish to thank Simon Strangeways for his valuable assistance in adapting the stimuli and Colin Clifford of University of Sydney, Australia for developing the analysis scripts. Development of the MacBrain Face Stimulus Set was overseen by Nim Tottenham and supported by the John D. and Catherine T. MacArthur Foundation Research Network on Early Experience and Brain Development. Please contact Nim Tottenham at [email protected] for more information concerning the stimulus set. 

Commercial relationships: none. 

Corresponding authors: Dr. Michael P. Ewbank or Dr. Andrew J. Calder. 

Email: [email protected]; [email protected]. 

Address: MRC Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge, CB2 7EF, United Kingdom.