Searching for a perceived gaze direction using eye tracking

Abstract

The purpose of the current study was to use eye tracking to better understand the “stare-in-the-crowd effect”—the notion that direct gaze is more easily detected than averted gaze in a crowd of opposite-gaze distractors. Stimuli were displays of four full characters aligned across the monitor (one target and three distractors). Participants completed a visual search task in which they were asked to detect the location of either a direct gaze or an averted gaze target. Reaction time (RT) results indicated faster responses to direct than averted gaze only for characters situated in the far peripheral visual fields. Eye movements confirmed a serial search strategy (definitely ruling out any pop-out effects) and revealed different exploration patterns between hemifields. The latency before the first fixation on target strongly correlated with response RTs. In the LVF, that latency was also faster for direct than averted gaze targets, suggesting that the response asymmetry in favor of direct gaze stemmed from faster direct gaze target detection. In the RVF, however, the response bias to direct gaze seemed not due to a faster visual detection but rather to a different cognitive mechanism. Direct gaze targets were also responded to even faster when their position was congruent with the direction of gaze of distractors. These findings suggest that the detection asymmetry for direct gaze is highly dependent on target position and influenced by social contexts.

Introduction

Human eyes play a fundamental role in social cognition. They are usually the most attended facial feature and a major source of information when processing identity, gender, facial expressions, or age and when inferring attention and intentions of others through gaze direction (for a review, see Itier & Batty, 2009). Evolutionary hypotheses claim that human eye morphology evolved from a necessity for complex social interactions (Emery, 2000). In particular, the ratio of exposed sclera size in the eye outline, which is largest in the human species, seems to have evolved to enhance perception of gaze direction (Kobayashi & Kohshima, 2001). 

Eye contact, also called direct gaze or mutual gaze (for a review, see George & Conty, 2008), is especially important for proper communication because it may be interpreted as a sign of friendliness or intimacy (Kleinke, 1986) or may also be a sign of anger or hostility (Ellsworth & Carlsmith, 1973), especially in the world of nonhuman primates. Being able to detect direct gaze is thus important for survival and social interactions. Direct gaze tells us that we are the focus of the viewer's attention and any other nonverbal communicative behavior is intended toward us (the observer). People are rather accurate at discriminating whether another person is looking directly at them or whether the gaze is averted, from a wide range of distances and head orientations (Gamer & Hecht, 2007). Recent studies, however, suggest that although still accurate, gaze discrimination sensitivity drops for gaze deviated by 5° compared to 10° of visual angle (Jenkins, Beaver, & Calder, 2006), and direct gaze discrimination can be strongly biased by gaze directional adaptation (Jenkins et al., 2006; Kloth & Schweinberger, 2008). 

One theory suggests that evolution has allowed the development of an Eye Direction Detector (EDD) mechanism—a neurocognitive system dedicated to the rapid detection of eyes in the environment and their gaze direction, based on the geometry created by the circular dark iris relative to the white sclera (Baron-Cohen, 1994). Based on cell recordings in monkeys, Perrett and Emery (1994) proposed a more general “Direction of Attention Detector,” which would integrate not only gaze direction but also information from head orientation, body position, and even direction of locomotion. However, due to the importance of mutual gaze, the authors also introduced a specific module that would process eye contact separately. More recently, Senju and Johnson (2009) proposed that eye contact could be processed by the subcortical face-detection system (Johnson, 2005), which is supposedly fast and based on low spatial frequencies, offering a “quick and dirty” route for face and gaze information processing. All of these theories emphasize humans' sensitivity to mutual gaze. Because of its importance for survival and social interactions, it would be sensible that direct gaze also be detected rapidly. 

It has been shown that direct gaze is indeed detected faster than averted gaze. In a seminal paper, von Grünau and Anston (1995) presented participants with pairs of schematic eyes and found that eye targets with a straight gaze were detected faster and more accurately when presented in an array of averted gaze nontargets than an averted gaze target presented in an array of straight gaze nontargets. This detection asymmetry was named the “stare-in-the-crowd effect.” Senju, Hasegawa, and Tojo (2005) replicated this effect using full faces in three-quarter view, suggesting that direct gaze, rather than the visual symmetry of eyes, is the driving element in the stare-in-the-crowd effect (see also Doi & Ueda, 2007; Doi, Ueda, & Shinohara, 2009). Using photographs of eye regions taken from faces in frontal or three-quarter view, Conty, Tijus, Hugueville, Coelho, and George (2006) investigated the interaction of gaze and head orientation on this gaze detection asymmetry. They found that direct gaze was detected faster and more accurately than averted gaze but only under deviated head conditions. Senju and Hasegawa (2006) also found that upright eyes with a direct gaze were detected faster than with averted gaze, even when the rest of the face was inverted. However, the effect vanished when the eyes were inverted. 

This body of research clearly shows that direct gaze is detected faster than averted gaze in many different search paradigms (as long as eyes are upright), when using eyes or faces scattered across the screen. In the current study, we tried to go one step closer to real-life situations by presenting participants with four full individuals displayed side by side, as one could encounter in some occasions like when coming out of an elevator (see Figure 1). The characters were aligned across the screen with front-view faces looking straight ahead (direct gaze) or away (averted gaze). Participants' task was to locate a specific target (the direct gaze character among averted gaze characters or vice versa). The use of these particular stimuli allowed the investigation of possible modulation of the stare-in-the-crowd effect depending on target location, an aspect previously not studied. Based on the literature reviewed above, we expected to see the classic detection asymmetry in favor of direct gaze, as measured by response times (RTs) and error rates. Furthermore, if the stare-in-the-crowd effect was a robust and general phenomenon, we would expect the faster and easier detection of direct gaze to be seen at each of the four positions used in the display. 

Figure 1

 

Example of the stimuli used with, from left to right, characters situated in Positions 1, 2, 3, and 4. (Left) Example of a direct gaze trial (direct gaze target in Position 4, averted gaze distractors in all other positions). (Right) Example of an averted gaze trial (averted gaze target in Position 1, direct gaze distractors in all other positions). The fixation cross, as well as the AOI for each face are overlaid on top of the stimuli; however, these were not shown during the actual stimulus presentation. The displays are presented to scale and represent the entire monitor size.

While previous stare-in-the-crowd studies relied solely on reaction time measures, we used an eye monitoring device to track participants' eye movements and monitor possible visual search strategies employed to detect direct gaze faster than averted gaze. Eye-tracking methodology provides a sensitive and real-time behavioral index of ongoing visual and cognitive processing (Henderson, 2006) and its use in the present experiment gave us two advantages over previous studies. First, it allowed determining whether and how participants' eye movements were biased toward the different target conditions. Second, it ensured participants fixated on the fixation cross before each trial, controlling for the initial position of eye gaze. We hypothesized that participants would explore the visual array with a few fixations in search for the target. Based on the proposed gaze direction or direction of attention detector mechanism (Baron-Cohen, 1994; Perrett & Emery, 1994), whose evolutionary purpose was to detect gaze direction quickly for survival reasons, we expected that the time taken to first detect targets would be shorter for direct than averted gaze targets and would be correlated with response times. This faster detection could result from a more efficient visual search strategy, which would be reflected in an overall pattern of eye movements specific to the direct gaze condition with, for instance, fewer fixations to reach direct than averted gaze targets. Alternatively, the same strategy could be employed for both gaze conditions, but eye movements could be faster for the direct gaze target and/or longer for the averted gaze target (i.e., a timing difference). So far, the literature has not addressed this question. 

We also investigated whether the direction of gaze of the distractor characters in the direct gaze target condition could modulate participants' attention and impact on the target search. Indeed, eyes can direct attention toward specific places or people through averted gaze (Baron-Cohen, Jolliffe, Mortimore, & Robertson, 1997; Emery, 2000), which often results in the perceiver orienting their own attention in the same direction. This orienting of attention by gaze has been demonstrated by numerous studies using Posner-like attention paradigms in which a face cue is centrally presented prior to the onset of a lateral target. Target detection is faster when the gaze of the face is directed toward the side where the target later appears (i.e., congruent targets) and longer when the gaze is looking in the opposite direction (i.e., incongruent targets; Driver et al., 1999; Friesen & Kingstone, 1998; Langton & Bruce, 1999; see Frischen, Bayliss, & Tipper, 2007 for a review). Eye movement studies have also shown that the direction of gaze of characters presented alone or in visual scenes influences participants' eye movements that tend to go in the direction signaled by gaze (Castelhano, Wieth, & Henderson, 2007; Itier, Villate, & Ryan, 2007; Zwickel & Võ, 2010). We thus analyzed the data for the direct gaze condition in terms of the congruency effect the averted gaze distractors may have on target detection. We predicted that targets whose positions were in the direction signaled by averted gaze distractors (congruent targets) would be detected faster than targets whose positions were opposite to distractors' gaze direction (incongruent targets). This would be seen in the form of faster RTs and earlier onset time of first fixation for congruent compared to incongruent targets. 

To summarize, we predicted that (1) direct gaze targets would be detected faster and more accurately than averted gaze targets at every target position, as measured by RTs and error rates, replicating the stare-in-the-crowd effect; (2) this asymmetry in target gaze detection would also be seen in eye movements; (3) the time taken to first detect targets would be shorter for direct than averted gaze targets and would strongly correlate with RTs; and (4) direct gaze targets whose positions were congruent with the direction signaled by averted gaze distractors would be detected faster than targets whose positions were incongruent with the gaze direction of distractors. 

Methods

Participants

Data were collected from 24 undergraduate students (16 females, 8 males; 22 right-handed) from the University of Waterloo (UW), who participated in the study for course credit. Ages ranged from 18 to 22 years (M = 19.7). All participants had normal or corrected-to-normal vision. All participants signed informed written consent letters and the study received full ethics clearance from the UW Research Ethics Board. 

Stimuli

A set of 8 virtual humans, or computer agents, was obtained from Vizard 3.15 software. Each stimulus included 4 female or 4 male agents, displaying their bodies from the knees to the tops of their heads. Agents were standing with their faces and bodies facing forward against a black background (see Figure 1) and all were dressed in business attire. The scenes were displayed at a resolution of 1152 × 864 pixels and subtended 29.2° × 22.2° of visual angle at the viewing distance of 70 cm. Each of the agent's faces subtended 2.1° horizontally by 4.1° vertically (male and female faces were of the same dimensions). The midline of one face to the midline of an adjacent face subtended 7.4° visual angle horizontally. 

Each participant completed two target gaze conditions. In the direct gaze (DG) condition, participants searched for the agent (target) that looked straight ahead while the other agents (distractors) were looking to the left or to the right. The gaze on each of the agent's faces was manipulated using Adobe Photoshop 11.0 and then mirror-reversed to avoid any bias between the right and left sides. In the averted gaze (AG) condition, participants searched for the agent (target) that had an averted gaze among straight gaze distractors. In both DG and AG conditions, half of the trials involved targets with left-averted gaze and half involved targets with right-averted gaze. All faces were of neutral expression. Each of the two condition blocks consisted of 256 trials, which were divided into 4 subblocks of 64 trials each. Condition blocks and subblocks were counterbalanced across participants. In each trial, the 4 motionless agents were presented aligned at four positions on the display (Position 1 = leftmost, whose face center was situated at −11.1° visual angle horizontally, Position 2 = center left, −3.7°, Position 3 = center right, +3.7°, and Position 4 = rightmost, +11.1°; see Figure 1). Half of the trials involved only female agents and half involved only male. Target position and identity were completely counterbalanced across trials. 

Apparatus

The stimuli were presented on a Viewsonic PS790 CRT 19-inch color monitor driven by an Intel Corel 2 Quad CPU Q6700 with a refresh rate of 60 Hz. Eye movements were recorded using a remote EyeLink 1000 eye tracker from SR Research with a sampling rate of 1000 Hz. The eye tracker was calibrated to each participant's dominant eye, but viewing was binocular. Calibration was done using a nine-point calibration accuracy test. Calibration was repeated if the error at any point was more than 1°, or if the average error for all points was greater than 0.5°. Chin and forehead rests maintained participants' viewing position and distance. 

Procedure

Before each condition, participants were given a practice test of 4 trials to get familiar with the stimuli and task. All participants were instructed to detect the target position as quickly and as accurately as possible. Responses were made on a standard keyboard layout, using the keys z (for Position 1), c (for Position 2), number pad 1 (for Position 3), and number pad 3 (for Position 4). Participants used their middle and index fingers of both hands to respond. Stimuli were shown for 2 s regardless of whether a response was made or not. No feedback was given about their behavioral performances. A black screen was presented between each stimulus with a white centered fixation cross. After 1 s, the fixation cross was replaced by a fixation trigger participants must have focused on for 300 ms to activate the next trial. Thus, the starting point of eye movements for each trial was in the center of the screen, near the waist region of the agents between Positions 2 and 3, and subtended 8.4° of visual angle vertically to the center line of the eye level of the agents. It is important to note that none of the agent's faces were in the foveal region of the starting point for each trial. 

Each subblock comprised 64 trials and participants were allowed to take a self-paced break after each one. At the beginning of each subblock, participants' eye movements were recalibrated. Within each gaze condition and subblock, scene presentation was randomized. The entire experiment lasted approximately 70 min. 

Data analyses

Behavioral data

Left and right gaze directions were combined and averaged according to target position in each gaze condition. Mean reaction time (RT) data were computed using correct responses, which comprised 88.7% of the total data. No responses, correct or incorrect, were made in less than 500 ms for any trial, indicating no anticipatory responses. Missed responses comprised 6.1% of the data. RTs and percent error rates were analyzed using a 2 (target gaze condition: direct vs. averted) × 2 (visual field: LVF vs. RVF) × 2 (position: central vs. peripheral) repeated measures ANOVA. The LVF included the two leftmost targets (i.e., Positions 1 and 2) while the RVF included the two rightmost targets (i.e., Positions 3 and 4). Similarly, the peripheral targets included the outer positions (i.e., Positions 1 and 4), while the central targets included the inner positions (i.e., Positions 2 and 3). The Greenhouse–Geisser degrees of freedom correction was used whenever necessary. 

Eye-tracking data

Any fixation of less than 80 ms in duration was removed from the analyses. This eliminated 6.5% of the total data. The areas of interest (AOI) included each of the agents' faces (a circle spanning from the top of their head to the bottom of their neck; see Figure 1); from left to right positions, we labeled these AOI Face 1, Face 2, Face 3, and Face 4. These AOI together comprised 90.1% of the total viewing time across all targets and distractors (18.7% for Face 1, 27.0% for Face 2, 25.4% for Face 3, and 19.0% for Face 4). We also analyzed three separate AOI for the background area in between each of the face areas (i.e., between Faces 1 and 2, between Faces 2 and 3, and between Faces 3 and 4) to examine more carefully the scanning patterns of participants. However, these AOI together only comprised approximately 4% of the total viewing time, in either condition and will thus not be reported or discussed. Lastly, we did not analyze the agents' bodies (i.e., from the neck to their knees), as they do not contain any relevant information for target detection. Indeed, eye-tracking data showed that participants spent less than 6% of their total viewing time looking at all of the agents' bodies combined, in either condition. This equated to less than 0.1 fixations per body on average per trial. 

We analyzed possible eye movement differences for direct compared to averted gaze conditions. This included analyzing, as a function of gaze direction and target position, (i) the average number of fixations made before a response, (ii) the average viewing time before a response, (iii) the average number of fixations to reach the target for the first time, and (iv) the average latency before first fixation on target. The average viewing time before the response included the summed fixation durations before reaching the target (for the last time) on a given trial. This means that the target could have been seen before but the visual search continued and the eyes moved back to the target before the response was made. The average latency before first fixation on target was the average time taken to move the eyes from the fixation cross to the various target AOI for the very first time in each trial. During this time, there could thus be one or several fixations made. Only correct response trials were used and only eye movements made before the (correct) behavioral response were included in the analyses. All of the dependent variables were analyzed according to each target position using a 2 (target gaze condition: direct vs. averted) × 2 (visual field: LVF vs. RVF) × 2 (position: central vs. peripheral) repeated measures ANOVA. The Greenhouse–Geisser degrees of freedom correction was used whenever necessary. 

Results

Behavioral data for direct gaze versus averted gaze

For RT responses, the main effect of visual field approached significance (F(1, 23) = 3.85, p = 0.062, η ² = 0.14; see Figure 2A), with marginally faster RTs in the LVF than in the RVF. A strong effect of position was found (F(1, 23) = 31.75, p < 0.001, η ² = 0.58), indicating faster RTs for central than peripheral targets. There was also a significant main effect of target gaze (F(1, 23) = 10.04, p < 0.005, η ² = 0.30), with faster RTs in the DG than in the AG condition. However, the position × gaze interaction (F(1, 23) = 14.60, p < 0.005, η ² = 0.39) revealed that this response asymmetry in favor of DG was found only for peripheral target positions (t(23) = 4.99, p < 0.001); no effect of gaze was found at central target positions. No other interactions were found. 

Figure 2

 

Results for (A) reaction time responses and (B) percent error rates, as a function of gaze direction and target position. Blue bars represent the Averted Gaze Condition; red bars represent the Direct Gaze Condition. DG vs. AG paired comparison: *p < 0.05.

For percent error rates, a main effect of visual field was found (F(1, 23) = 6.36, p < 0.05, η ² = 0.22; see Figure 2B), indicating a higher error rate in the LVF than in the RVF. A main effect of position was also found (F(1, 23) = 5.02, p < 0.05, η ² = 0.18), due to overall more errors for central than peripheral targets, but this was driven by the interaction with visual field and gaze as described below. There was a main effect of target gaze (F(1, 23) = 8.50, p < 0.01, η ² = 0.27), due to more errors for the DG than the AG condition. The visual field × gaze interaction was significant (F(1, 23) = 26.10, p < 0.001, η ² = 0.53), due to more errors for DG than AG in the LVF (t(23) = 5.30, p < 0.001) while no gaze difference was seen in the RVF. Lastly, there was a visual field × position × gaze interaction (F(1, 23) = 5.23, p < 0.05, η ² = 0.19), indicating higher error rates for the DG than the AG condition in the peripheral LVF (Position 1; t(23) = 6.29, p < 0.001), while the reverse was seen in the peripheral RVF (Position 4; t(23) = 3.78, p < 0.005), with no gaze differences at central Positions 2 and 3. No other interactions were found. When the AG condition was analyzed separately, no effect of visual field or position was found. In contrast, in the DG condition, a main effect of visual field was found (F(1, 23) = 34.06, p < 0.001, η ² = 0.60), as well as a visual field × position interaction (F(1, 23) = 11.35, p < 0.005, η ² = 0.33), due to a linear decrease of error rates from Positions 1 to 4. Paired t-tests revealed that the peripheral LVF (i.e., Position 1) had a marginally higher error rate than the central LVF (i.e., Position 2; t(23) = 1.98, p = 0.059), and the central RVF (i.e., Position 3) had a higher error rate than the peripheral RVF (i.e., Position 4; t(23) = 3.11, p < 0.01). 

Overall gaze exploration strategies between direct and averted gaze conditions

As predicted, participants explored the visual display with a few fixations. On average, 41% of the very first fixation landed on Face 2, 22% in between Faces 2 and 3, and 20% on Face 3, while only 9% landed on Face 1, 4% in between Faces 1 and 2, 2% in between Faces 3 and 4, and 2% on Face 4. Thus, participants almost always explored the central regions first before moving to the periphery, explaining their faster response times at these positions. 

For the average number of fixations made before responding, a main effect of visual field (F(1, 23) = 4.94, p < 0.05, η ² = 0.18; see Figure 3A) was due to less fixations in the LVF than in the RVF. A main effect of position was found (F(1, 23) = 46.54, p < 0.001, η ² = 0.67), indicating more fixations to reach peripheral than central target positions. Additionally, there was a visual field × gaze interaction (F(1, 23) = 5.48, p < 0.05, η ² = 0.19) and a visual field × position × gaze interaction (F(1, 23) = 4.86, p < 0.05, η ² = 0.18), indicating more fixations for the AG than the DG condition at Position 1 (t(23) = 2.24, p < 0.05), while the reverse was seen at Position 4 (t(23) = 2.29, p < 0.05), with no gaze differences at Positions 2 and 3. 

Figure 3

 

Results for (A) average number of fixations made to reach the target before responding, (B) average viewing time before the response, (C) average number of fixations to reach the target for the first time, and (D) average latency before first fixation landing on the target, as a function of gaze direction and target position, before a behavioral response was made. Note the change in scale between (A) and (C) and between (B) and (D). Blue bars represent the AG Condition; red bars represent the DG Condition. DG vs. AG paired comparison: *p < 0.05; ⁺ p < 0.09.

Similar results were found for the average viewing time before the response. A main effect of visual field (F(1, 23) = 13.03, p < 0.005, η ² = 0.36; see Figure 3B) indicated less viewing time for targets in the LVF than in the RVF. A main effect of position was found (F(1, 23) = 9.51, p < 0.01, η ² = 0.29), indicating more viewing time to reach peripheral than central target positions. Additionally, there was a visual field × gaze interaction (F(1, 23) = 8.55, p < 0.01, η ² = 0.27) and a visual field × position × gaze interaction (F(1, 23) = 4.61, p < 0.05, η ² = 0.17), indicating more viewing time for the DG than the AG condition at Position 4 (t(23) = 3.83, p < 0.005), with no gaze differences at the other positions. 

For the average number of fixations to reach the target for the first time, a main effect of visual field (F(1, 23) = 5.05, p < 0.05, η ² = 0.18; see Figure 3C) was due to less fixations in the LVF than in the RVF. A main effect of position was found (F(1, 23) = 90.61, p < 0.001, η ² = 0.80), indicating more fixations to reach peripheral than central target positions. Additionally, there was a visual field × gaze interaction (F(1, 23) = 5.56, p < 0.05, η ² = 0.20) and a visual field × position × gaze interaction (F(1, 23) = 4.67, p < 0.05, η ² = 0.17), indicating more fixations for the DG than the AG condition at Position 4 (t(23) = 3.13, p < 0.01), while the reverse was trending at Position 1 (t(23) = 1.78, p = 0.088), with no gaze differences at Positions 1 and 2. 

For the average latency before first fixation landing on target, the main effect of visual field approached significance (F(1, 23) = 3.39, p = 0.078, η ² = 0.13; see Figure 3D), indicating marginally earlier latencies for targets in the LVF compared to the RVF. A strong effect of position was found (F(1, 23) = 139.08, p < 0.001, η ² = 0.86), due to earlier latencies for central than peripheral target positions. The main effect of target gaze condition approached significance (F(1, 23) = 3.32, p = 0.082, η ² = 0.13), but the significant position × gaze interaction (F(1, 23) = 7.48, p < 0.05, η ² = 0.25) revealed an earlier latency for DG than AG at peripheral target positions (t(23) = 3.05, p < 0.01). However, t-tests revealed that this effect was only significant at Position 1 (t(23) = 2.62, p < 0.05), not at Position 4 (although in the same direction). 

Correlations between RTs, latencies before first fixation on target, and error rates

The RT responses and eye-tracking data revealed earlier detection times at central than peripheral target positions. Across conditions, for both RT responses and first-fixation-on-target latencies, targets in Position 2 were responded to and looked at faster, followed by Position 3, then Position 1, and lastly, Position 4. We thus correlated mean RTs with the time taken to first view the target, across participants. The results revealed strong and highly significant Pearson correlations for each of the four positions: r = 0.91 (p < 0.001, R ² = 0.82) for Position 1, r = 0.59 (p < 0.005, R ² = 0.34) for Position 2, r = 0.83 (p < 0.001, R ² = 0.69) for Position 3, and r = 0.93 (p < 0.001, R ² = 0.87) for Position 4. Thus, more than 80% of the RT variance (expressed as R ²) could be explained by the first-fixation-on-target latencies for Positions 1 and 4, where the stare-in-the-crowd effect was found, and 69% for Position 3. For Position 2, however, latency of the first fixation on target explained only 34% of the RT variance. These results suggest that overall participants responded slower to peripheral positions because they were looking there last, as also suggested by the average number of fixations made before responding. 

We also investigated whether the high proportion of errors at Position 1 in the DG condition could be due to a speed–accuracy trade-off. If this were the case, we should find a negative correlation between RT and first-fixation-on-target latencies and the error rate. In contrast, at Position 1, we found significant positive correlations for RT (r = 0.43, p < 0.05, R ² = 0.19) and for first-fixation-on-target latencies (r = 0.41, p < 0.05, R ² = 0.17), with error rates. No significant correlations were found for the other positions. In other words, at Position 1 only, the longer the RTs and first-fixation-on-target latencies, the larger the error rates. This demonstrates a lack of speed–accuracy trade-off and suggests true errors, possibly reflecting hesitation. 

Congruency effect between target location and averted gaze direction of distractors in the Direct Gaze condition

Since the agents were standing in a horizontal array (see Figure 1), we recoded the data for the DG condition to label each target agent according to whether the other distractor agents were looking in their direction (i.e., Congruent condition) or in the opposite direction (i.e., Incongruent condition). For example, when a DG target was in Position 1, the other 3 agents could have been looking left, toward the target (Congruent condition), or looking right, away from the target (Incongruent condition). This would be similar for Position 4, when the other targets were looking right (toward the target) or looking left (away from the target). For targets in Positions 2 and 3, the congruent and incongruent conditions included 2 distractors (instead of 3) looking toward or away from the target. We analyzed the data using a 2 (congruency condition: Congruent vs. Incongruent) × 2 (visual field: LVF vs. RVF) × 2 (position: central vs. peripheral) repeated measures ANOVA. 

For RT responses, a main effect of visual field was found (F(1, 23) = 4.86, p < 0.05, η ² = 0.17; see Figure 4A), indicating faster RTs in the LVF than in the RVF. A strong effect of position (F(1, 23) = 17.26, p < 0.001, η ² = 0.43) indicated faster RTs for central than peripheral target positions. Importantly, the main effect of congruency condition was significant (F(1, 23) = 9.04, p < 0.01, η ² = 0.28), as congruent targets were responded to faster than incongruent targets. The visual field × position × congruency interaction approached significance (F(1, 23) = 3.88, p = 0.061, η ² = 0.14), due to faster RTs for the Congruent than the Incongruent condition in the peripheral LVF (i.e., Position 1; t(23) = 2.44, p < 0.05), central RVF (i.e., Position 3; t(23) = 2.95, p < 0.01), and marginally for the peripheral RVF (i.e., Position 4; t(23) = 1.90, p = 0.070). No other interactions were found. 

Figure 4

 

Results for (A) reaction time responses, (B) percent error rates, and (C) average latency before first fixation landing on the target, for each target position according to Congruent and Incongruent conditions of the Direct Gaze condition. Purple bars represent the Congruent Condition; green bars represent the Incongruent Condition. Congruent–Incongruent paired comparisons: *p < 0.05; ⁺ p = 0.07.

Since the congruent DG targets had faster RTs overall than the incongruent DG targets, it may be argued that these congruent DG targets are driving the overall gaze difference between direct and averted gaze targets (i.e., in the peripheral target positions). In other words, congruent DG targets may be detected faster than AG targets, but incongruent DG targets may not. We thus analyzed the difference in RTs comparing congruent and incongruent DG targets separately to AG targets. When comparing congruent DG targets to AG targets, a significant position × gaze interaction was found (F(1, 23) = 13.67, p < 0.005, η ² = 0.37), indicating that congruent DG targets had faster RTs than AG targets for the peripheral positions (t(23) = 5.36, p < 0.001), as before. Likewise, when comparing the incongruent DG targets to AG targets, a significant position × gaze interaction was also found (F(1, 23) = 10.88, p < 0.005, η ² = 0.32), indicating faster RTs for incongruent DG targets than AG targets in the peripheral positions (t(23) = 3.62, p < 0.005). No differences were found in the central positions for either comparison, as before. These results indicate that the congruency effect of half of the DG targets was not driving the general gaze difference between direct and averted gazes at peripheral positions. Thus, both incongruent and congruent DG targets yielded faster RTs than AG targets at peripheral positions. 

For percent error rates, a main effect of visual field was found (F(1, 23) = 34.06, p < 0.001, η ² = 0.60; see Figure 4B), indicating more errors in the LVF than in the RVF. There was also a significant main effect of congruency condition (F(1, 23) = 32.06, p < 0.001, η ² = 0.58), indicating more errors in the Congruent than in the Incongruent condition. The visual field × position interaction (F(1, 23) = 11.35, p < 0.005, η ² = 0.33) revealed more errors in the central than peripheral RVF (i.e., Position 3 > Position 4; t(23) = 3.11, p < 0.01), while more errors were found in the peripheral than central LVF (i.e., Position 1 > Position 2; t(23) = 1.98, p = 0.059). There was also a significant visual field × congruency interaction (F(1, 23) = 46.07, p < 0.001, η ² = 0.67), a position × congruency interaction (F(1, 23) = 52.32, p < 0.001, η ² = 0.70), and a visual field × position × congruency interaction (F(1, 23) = 30.01, p < 0.001, η ² = 0.57). These effects were due to higher error rates for the Congruent than Incongruent condition in the peripheral LVF (at Position 1; t(23) = 12.02, p < 0.001), while no effects of congruency were found at the other three positions. Thus, while the total number of DG errors at Positions 2, 3, and 4 were evenly distributed between congruent and incongruent targets, approximately 85% of the DG errors at Position 1 were due to congruency errors. 

For the average latency before first fixation on target, a significant main effect of visual field was found (F(1, 23) = 4.59, p < 0.05, η ² = 0.17; see Figure 4C), indicating earlier latencies for targets in the LVF than in the RVF. A main effect of position (F(1, 23) = 105.84, p < 0.001, η ² = 0.82) indicated earlier latencies in the central than in the peripheral target positions. The main effect of congruency condition was also significant (F(1, 23) = 8.83, p < 0.01, η ² = 0.28), indicating earlier latencies for targets in the Congruent than in the Incongruent conditions. However, the visual field × position × congruency interaction approached significance (F(1, 23) = 3.47, p = 0.075, η ² = 0.13), indicating that onset times in the Congruent condition were earlier than in the Incongruent condition for the peripheral LVF (i.e., Face 1, t(23) = 2.59, p < 0.05) and central RVF (i.e., Face 3, t(23) = 2.36, p < 0.05). 

We also tested for possible correlations between RTs and latencies of first fixation on target and error rates. As for the general analyses reported earlier, a significant positive correlation was found for RT at Position 1 (r = 0.46, p < 0.05, R ² = 0.21) but not at the other positions. Additionally, a positive correlation approached significance for onset latencies at Position 1 (r = 0.39, p = 0.061, R ² = 0.15) but not at the other positions, demonstrating a lack of speed–accuracy trade-off. 

Discussion

The present study investigated whether direct gaze was detected faster than averted gaze in displays containing four full characters. We used eye tracking to extend the results of previous studies on the stare-in-the-crowd effect by examining eye movements in addition to response times. In a target localization task, participants' eye movements were tracked as they searched for a given target whose gaze was different than that of the other three distractors. Our results indicated an asymmetrical bias to respond to direct gaze faster than to averted gaze, as reported previously (e.g., Conty et al., 2006; Doi & Ueda, 2007; Doi et al., 2009; Senju et al., 2005; von Grünau & Anston, 1995). However, this effect was strongly modulated by target position such that direct gaze was responded to faster than averted gaze only in the far peripheral visual fields. Eye movement analyses revealed that the faster responses to DG stemmed from a faster visual detection of DG compared to AG targets at left but not right peripheral positions. Furthermore, the timing of this visual detection, reflected in latencies of first fixation on target, mimicked the behavioral responses and strongly correlated with RTs. Finally, detection of direct gaze was influenced by distractors' gaze direction. We discuss these effects and their implications in more details below, starting with general effects of hemifield and position. 

Targets in the LVF were responded to marginally faster than targets in the RVF, replicating the results of Conty et al. (2006) and Doi et al. (2009). The fact that 41% of the very first fixation landed on Position 2 situated on the left side indicated a trend for leftward bias in initial spontaneous explorations of visual scenes (Ebersbach et al., 1996; Hättig, 1992). This initial bias may have helped the visual search as targets in the LVF were visually detected faster and with fewer fixations than in the RVF. The LVF was also associated with higher error rates than the RVF, as discussed in more detail below. However, for correct response trials, which were those kept for eye movement analyses, less visual exploration was seen in the LVF as indicated by less fixations and shorter viewing times than in the RVF. These results demonstrate a strong visual field asymmetry in this gaze direction search. An LVF advantage for gaze perception has been reported previously (Ricciardelli, Ro, & Driver, 2002) and seems related to the larger involvement of the right hemisphere at the neural level (Calder et al., 2007). This right hemisphere dominance is also seen for various face perceptual processes (Burt & Perrett, 1997) and seems to be a general characteristic of social information processing (Brancucci, Lucci, Mazzatenta, & Tommasi, 2009). 

The response pattern was also strongly modulated by target position, and both eye movements and response times were much faster for central (Positions 2 and 3) than peripheral (Positions 1 and 4) target positions. The characters were presented side by side, like one might see in real-life situations, for example, coming out of an elevator. This specific positioning seems to have influenced participants' general scanning and response pattern to targets. As reflected in the distribution of very first fixations, participants most often looked upward from the centered fixation cross, bringing their gaze on the central positions (Face 2 or 3 or in between) before looking elsewhere. In addition to being seen first, the least exploration time was needed to detect targets in these central positions as seen in the number of fixations and viewing time, explaining the faster responses for targets at these locations. Thus, the pattern of visual search was serial, demonstrating a lack of pop-out effect as previously suggested (e.g., Conty et al., 2006; von Grünau & Anston, 1995). These results, along with the strong correlation patterns between response times and latency of first fixation on target, are in line with recent evidence indicating that gaze direction processing requires focused attention (Burton, Bindemann, Langton, Schweinberger, & Jenkins, 2009). That is, target gaze was only discriminated when participants actually attended to and fixated on the target face. 

The most important finding of the present study concerns the interaction between target gaze and position such that direct gaze yielded faster responses than averted gaze for peripheral targets only. This contrasts majorly with previous stare-in-the-crowd research that did not report target position effects or interactions of target position and gaze direction. Stimuli in these studies consisted of arrays of pictures scattered randomly across the screen, preventing the analysis of specific target locations. In contrast, the characters' positioning in the present study allowed us to investigate more precisely where in the visual field direct gaze might be more efficiently detected. Contrary to our hypothesis and to previous research, the results suggest that the stare-in-the-crowd effect is not a general phenomenon but is heavily dependent on target position and likely on initial gaze position (which was not controlled in previous studies). 

Eye movement analyses confirmed a gaze difference at peripheral positions and a lack thereof at central positions. However, the gaze differences at peripheral positions differed between visual fields. In the LVF (Position 1), the response asymmetry stemmed from a truly faster visual detection of DG than AG targets. This was seen in the time taken to first view the target, which was shorter for DG than AG targets, as well as in the number of fixations made before seeing the target for the first time, which tended to be smaller for the DG than the AG condition. Overall, fewer fixations were made before the actual response to DG than AG targets at this left peripheral position. In contrast, in the RVF (Position 4), participants made surprisingly more fixations before responding in the DG than in the AG condition, which resulted in longer viewing times to detect DG than AG targets. The number of fixations made to reach the target for the first time was also larger in the DG than in the AG condition although the time before first fixation on target did not differ between gaze conditions. Yet, participants were faster to respond to DG targets at Position 4. Thus, we must assume that another mechanism, likely cognitive, must have overcome the longer viewing time for DG in order to respond to DG targets faster than to AG targets. 

It could be argued that at central positions, the lack of stare-in-the-crowd effect was related to a ceiling effect such that the task was so easy it would wash out the experimental manipulation. In contrast, at peripheral sites, gaze targets would need further display exploration and thus more attentional resources to be detected. The increase in task difficulty at these positions would allow the emergence of the effect. The faster RTs and less exploration for central than peripheral positions support this interpretation. However, the generally long RTs recorded suggest that the task was not easy, even at central positions (1231 ms on average for RTs at Position 2, the fastest of all). The pattern of error rates does not support that interpretation either given that error rates were smaller at the right peripheral field compared to central positions for the DG condition (and unchanged across positions for the AG condition). Most importantly, target faces in central positions were not within the foveal region of the starting point for each trial. As mentioned above, participants had to look up from the centered fixation first, bringing their gaze to the top of the monitor on Face 2 or 3 (or in between the two) before starting their search. Thus, gaze information was not directly in front of participants who still needed to move their eyes to get to central positions. For all these reasons, the lack of the stare-in-the-crowd effect at central positions is unlikely related to the task being easier at these positions compared to peripheral positions. 

One possible explanation for the faster detection of direct gaze at peripheral positions may be found in the framework of the putative subcortical face detection system that is thought to be fast and based on low spatial frequencies (LSFs; Johnson, 2005; Senju & Johnson, 2009). According to this theory, the LSF information is carried to the superior colliculus, pulvinar, and amygdala subcortical regions, offering a “quick and dirty” route of visual processing that is best suited to detecting stimuli in the periphery. For example, when faces are viewed in the periphery, LSF information of the face under naturalistic top-lighting conditions yields dark shadowed areas for the eye sockets, surrounded by the illuminated areas of the cheeks, nose, and forehead. This face detection subcortical route has also been proposed to be involved in eye contact detection (Senju & Johnson, 2009), possibly using contrast information between the circular dark iris and the white sclera. This “fast-track modulator” could be the Eye Direction Detector proposed by Baron-Cohen (1994) or the mutual attention detector proposed by Perrett and Emery (1994). Senju and Johnson (2009) proposed that the subcortical route projects onto cortical areas of the social network so as to modulate cortical activation as a function of task demands. Thus, direct gaze targets in the left peripheral visual field (Position 1) might have been detected faster than averted gaze targets as a result of this subcortical face-processing route that would then modulate oculomotor orienting, explaining the earlier latency of first fixation on target in the DG condition. Faces in central positions were also not within the foveal region, but eye movements were made upward from the central fixation cross (i.e., vertically). In contrast, eye movements to the periphery were made mostly laterally from those central positions. This would suggest that the subcortical route works for lateral periphery but not vertical periphery, and direct gaze can be discriminated at about 7.4° of eccentricity (Faces 1 and 4 were at 7.4° from Faces 2 and 3, respectively). The results also suggest a hemifield asymmetry for this subcortical route that does not seem to play the same role for targets in the RVF. The possible involvement of this indirect visual route remains speculative and does not completely fit with the idea that processing gaze requires focused attention (Burton et al., 2009). Thus, although the present data demonstrate a lack of pop-out effect and a serial search strategy to detect gaze, the mechanism behind the faster eye movements for direct than averted gaze in the LVF remains unclear and will have to be addressed by future studies. 

Following previous stare-in-the-crowd studies, we also predicted that participants would be more accurate in the detection of direct than averted gaze. This was the case for the peripheral RVF (Position 4) but interestingly not for any other positions. In fact, participants made more errors overall for the DG condition relative to the AG condition, especially in the peripheral LVF (Position 1). This finding goes against previous research that reported better responses for direct gaze, although again, without exploring possible modulations by target location. One explanation for these results may be related to participants' handedness. The great majority of participants (22 out of 24) were right-handed, and all participants used their right index and middle fingers to respond, respectively, to Positions 3 and 4, whereas they used their left middle and index fingers to respond, respectively, to Positions 1 and 2. Thus, participants' dominant hand may have facilitated responses for targets situated on the right while impairing responses for targets on the left. However, this possibility is ruled out by the fact that no position effects were found for responses to AG targets; the position effects of better accuracy at Position 4 with a concurrent lower accuracy at Position 1 were found only for DG targets and thus cannot be due to handedness. Another possibility for these results may be a speed–accuracy trade-off, especially in the peripheral LVF. Participants looked at, and responded to, targets marginally faster in the LVF relative to the RVF but also made more errors in the LVF, especially at Position 1. The correlation of RTs with error rates demonstrated a lack of speed–accuracy trade-off. In contrast, significant positive correlations for RTs and for latencies before first fixation on target with error rates were found at Position 1, while no significant correlations were found at the other positions. In other words, the longer the response times and latencies of first fixation on target, the larger the error rates, at Position 1 only. This finding suggests that, rather than a speed–accuracy trade-off, errors could reflect hesitation at this position. This is all the more possible as 85% of the errors at Position 1 were congruency errors, i.e., elicited by the gaze direction of distractors. When only the DG Congruent condition was used, a significant positive correlation was found for RTs at Position 1 but not at the other positions, supporting this interpretation. Participants may thus have been confused by the direction of gaze of distractors. In contrast, in the peripheral RVF (Position 4), DG targets yielded the lowest error rates overall, indicating a truly more accurate response to direct gaze. RTs at this position were also the longest, due to participants scanning other target positions first, as also supported by the linear decrease of error rates from Positions 1 to 4. The longer yet more accurate responses at Position 4 most likely resulted from the serial search mentioned previously: if the target was not detected in the first two, and often three, locations, then it had to be in the fourth one (participants knew each trial contained a target). This search process seemed to have facilitated better accuracy at Position 4. 

The analysis of congruency effects of averted gaze distractors on direct gaze targets revealed faster RTs for congruent than incongruent targets. That is, targets were responded to faster when their position was congruent with the direction of gaze of the distractors, suggesting an orientation of attention by distractors' gaze. This interpretation is strongly supported by the literature on gaze orienting, which suggests that we orient our attention in the direction signaled by gaze in an automatic manner (Driver et al., 1999; Friesen & Kingstone, 1998; Frischen et al., 2007; Langton & Bruce, 1999). However, this congruency effect was not the driving force behind the overall gaze difference found between direct and averted gaze targets at peripheral positions as even incongruent DG targets yielded faster RTs than AG targets at these positions. Earlier latencies of first fixation on target were also found for congruent than incongruent direct gaze targets, a result in agreement with recent studies showing that eye movements follow the direction signaled by gaze (Castelhano et al., 2007; Itier et al., 2007; Zwickel & Võ, 2010). However, this result was mostly driven by the peripheral LVF (Face 1) and central RVF (Face 3). Interestingly, congruent targets also yielded more errors than incongruent targets, specifically in Position 1. As noted above, this finding could be due to hesitation errors, whereby participants may have been confused, rather than helped, by the direction of gaze of distractors. 

It is important to note some methodological differences between the current and previous studies. Unlike previous research (e.g., Senju et al., 2005; von Grünau & Anston, 1995), no feedback was given after each trial. Telling participants whether they were right or wrong may have, in those studies, influenced both their response speed and accuracy by influencing their search strategy. Moreover, only four agents were used. It would be interesting to see whether the present results hold for displays containing more characters. For instance, Conty et al. (2006) found larger differences between gaze conditions for RTs and error rates when using display sizes of 8 and 12 stimuli rather than 4. That is, direct gaze was detected more efficiently than averted gaze when many distractors were present, which could be attributed to the greater influence of distractor gaze congruency with distractor number. We also used a different type of search task than previous studies examining the stare-in-the-crowd effect. In previous work, participants detected whether the target was present or absent, not its location as done here. Future studies should examine the effects of feedback, task demands, and distractor number on the stare-in-the-crowd effect and their eye movement correlates. 

In summary, the present study showed that the faster response to direct than averted gaze was found in a localization task, in both visual fields, despite the use of full characters and their bodies, and was not due to (although influenced by) distractors' averted gaze. However, our RTs, error rates, and eye movement results demonstrate that this stare-in-the-crowd effect is dependent on target position and not systematically found. Other studies have also reported instances in which this effect is absent. For example, Conty et al. (2006) found a stare-in-the-crowd effect under deviated head orientations but not in frontal head orientations. Thus, the faster detection of direct over averted gaze is not a systematic phenomenon and is modulated by at least two factors: spatial position and social gaze context. Future studies will have to determine whether other factors can modulate the state-in-the-crowd effect and the perception of gaze in more realistic social contexts. 

Supplementary Materials

Abstract (French version) 

Acknowledgments

This study was funded by CIHR (MOP-87393, MOP-89822, and NSP-94824), CFI, and the CRC Program to R. J. Itier. We would like to specially thank Jessica Hon for running participants and helping in data analyses, as well as Hannah Masterman for creating the stimuli. 

Commercial relationships: none. 

Corresponding author: Adam Palanica. 

Email: [e-mail]. 

Address: Department of Psychology, University of Waterloo, 200 University Avenue, Waterloo, Ontario N2L 3G1, Canada.