Wollaston's effect in infants: Do infants integrate eye and head information in gaze perception?

Abstract

The current study examined infants' sensitivity to Wollaston's effect: When identical eyes are placed in differently angled faces, the perceived gaze direction shifts toward the orientation of the face such that physically, the direct gaze is perceived as averted toward the orientation of the face. Consistent with Wollaston's effect, we found that looking toward direct and averted gaze by 4- to 5- and 7- to 8-month-olds (n = 40) was affected by the head orientation context. These results demonstrate that infants aged 4 to 5 and 7 to 8 months integrate eye and head information to perceive another's gaze direction. In light of recent psychophysical findings, the current results suggest that the visual function supporting constant gaze perception across head rotation is already at work by 4 to 5 months of age.

Introduction

From birth, human infants preferentially attend toward faces that are gazing toward them compared with faces that are gazing elsewhere (Farroni, Csibra, Simion, & Johnson, 2002) and compared with faces with closed eyes (Batki, Baron-Cohen, Wheelwright, Connellan, & Ahluwalia, 2000). Although detection of eye contact by newborn infants seems to be limited to the case of frontally oriented facial context (Farroni et al., 2002; Farroni, Menon, & Johnson, 2006), 4-month-olds seem to detect direct gaze across head orientation as evidenced by similar event-related brain activity across head orientation (Farroni et al., 2002; Farroni, Johnson, & Csibra, 2004). Around this age, infants smile more to faces while they are gazing at them compared with when they are looking away from them (Hains & Muir, 1996). Several studies have also shown that others' gaze direction modulates infants' recognition of faces (Farroni, Massaccesi, Menon, & Johnson, 2007; Yamashita, Kanazawa, & Yamaguchi, 2012) and attention toward peripheral objects (e.g., Farroni, Mansfield, Lai, & Johnson, 2003). However, there has been little investigation of the mechanisms by which infants perceive others' gaze direction. 

In a pioneering study of gaze perception, Gibson and Pick (1963) noted that the simple correspondence between the deviation of the eyes and gaze direction holds only for the special case of a frontally oriented face. For example, direct gaze toward the observer can be expressed as a leftward position of iris and pupil in a rightward-oriented face and vice versa. Despite such changes in image configuration expressing the direction of gaze across head rotation, we can perceive gaze direction effortlessly and relatively accurately (Carlin, Calder, Kriegeskorte, Nili, & Rowe, 2011; Gibson & Pick, 1963; Otsuka, Mareschal, Calder, & Clifford, 2014; Todorovic, 2006). Here, we call this ability to perceive constant gaze direction across various head rotations “gaze constancy.” Recent studies demonstrate that the perceived direction of gaze becomes unreliable when information about head orientation is unavailable, whereas the perceived direction of gaze is mostly constant across head orientation when the whole head is visible (Otsuka et al., 2014; Otsuka, Mareschal, & Clifford, 2015). Based on such findings, Otsuka et al. (2014) suggest that gaze constancy depends on the integration of information from the eye region and information about head rotation. As the head rotates, the relative amount of visible sclera changes as if eye direction shifts opposite to the head rotation, which biases the perceived gaze direction opposite to the head rotation (repulsive bias). The visual system compensates for the repulsive bias and maintains gaze constancy by integrating information from the eyes and information about head orientation, resulting in a tendency for the perceived gaze direction to be attracted toward head orientation (attractive bias). 

An impressive demonstration showing that we integrate information from eyes and information about head orientation in gaze processing has been given by Wollaston (1824; see Figure 1). From the drawing of a face oriented leftward with direct gaze (left), Wollaston produced another face by inserting the same eyes into a drawing of the same individual with his head oriented to the right (right). Although these two faces share identical eyes, the gaze direction of the latter face tends to be attracted toward the head orientation and appears to be looking to the right of the viewer. Such an attractive shift of perceived gaze direction depending on the head orientation context is called “Wollaston's effect” or “Wollaston's illusion” (Langton, Honeyman, & Tessler, 2004; Maruyama & Endo, 1983; Todorovic, 2006, 2009; Wollaston, 1824). Wollaston's effect can be conceptualized as the misapplication of a process that normally maintains gaze constancy despite changes in the visible part of the eyes with head rotation (Otsuka et al., 2015). As Wollaston's faces were created by inserting identical eyes in differentially oriented faces, they lack any change in the visible part of the eyes that induces the repulsive bias. Application of the integration of eye deviation and head orientation, without any repulsive bias to compensate, results in an apparent attractive shift of perceived gaze direction toward head orientation. Given the importance of the operation underlying Wollaston's effect in maintaining the constant perception of gaze direction across head rotation, the current study examines the development of sensitivity to this effect in infants. 

Figure 1

 

Demonstration by Wollaston (from “On the Apparent Direction of Eyes in a Portrait,” by W. H. Wollaston, 1824, Philosophical Transactions of the Royal Society of London, 114, p. 256, in the public domain). From the drawing of a face oriented leftward with direct gaze (left), Wollaston produced another face by inserting the same eyes into a drawing of the same individual with his head oriented to the right (right). Although these two faces share identical eyes, the latter appears to be looking to the right of the viewer.

In a previous study, Nakato et al. (2009) reported that sensitivity to Wollaston's effect develops between 6 and 8 months of age. They used black-and-white drawings of faces, similar to Wollaston's original demonstration, as the stimuli. Infants aged 6, 7, and 8 months were familiarized to a leftward-angled face with leftward averted gaze. After the familiarization, infants were shown the familiarized eyes and a left/right reversed version of the eyes each placed in the context of a rightward-angled face. In this new head orientation context, for adults, the familiarized eyes were perceived to be directly gazing at the observer. On the other hand, the left/right reversed version of the eyes was perceived to be gazing to the right of the observer. Nakato et al. hypothesized that infants would show a novelty preference for familiarized eyes if they are sensitive to Wollaston's effect as it induces a categorical shift of the perceived gaze direction from averted to direct gaze. They found that only 8-month-olds showed such a pattern of preference. 

The Wollaston effect allows us to examine whether infants integrate head orientation as a cue to gaze direction in its own right. Given the importance of such integration in maintaining constant gaze perception across variations in the head orientation (Otsuka et al., 2014, 2015), we further examined infants' sensitivity to Wollaston's effect by using realistic three-dimensional (3D) graphic color images of faces. We chose 4- and 8-month-olds as the target ages. As a previous study suggested that 4-month-olds showed similar neural responses to direct gaze consistently across head orientation (Farroni et al., 2006), we thought that there would be a possibility that even 4-month-olds would show sensitivity to Wollaston's effect. In Farroni et al. (2006), 4-month-olds showed differential event-related potentials (ERPs) to direct gaze and averted gaze in the angled face context similar to what was found in the frontal face context (Farroni et al., 2002). However, it is still unclear from findings of Farroni et al. whether 4-month-olds integrate eye and head orientation information in gaze processing. A close examination of their stimuli (figure 1 in Farroni et al., 2006) shows that their direct gaze stimuli look considerably more direct than their averted gaze stimuli even when seen without head orientation context. Although their results from an inverted stimulus condition show that infants' gaze sensitivity is specific to the upright face context, it is yet to be investigated whether more subtle head orientation information is used in an integrative manner to judge others' gaze direction by 4 months of age. 

In the current study, we used a preferential looking method to examine infants' sensitivity to Wollaston's effect. There were three image conditions: open/closed eyes, frontal face, and Wollaston. In the open/closed eyes condition, a frontally oriented face with open eyes with direct gaze (0°) was paired with the same face with eyes closed. In the frontal face condition, the frontally oriented face with direct gaze (0°) was paired with the same face with averted gaze (±12°). Based on previous studies reporting looking preference for faces with direct gaze over faces with averted gaze (Farroni et al., 2002, 2006) and preference for faces with direct gaze over faces with closed eyes (Batki et al., 2000), we hypothesized that infants would show preference toward direct gaze (0°) in these conditions. In the Wollaston condition, we examined the possibility that infants' gaze perception is influenced by the head orientation context. The images in this condition were created by inserting the eyes from the frontal face condition into a face angled by 20°. Although the eyes are identical between the frontal face condition and the Wollaston condition, for adult observers, the perceived gaze direction shifts toward the head orientation in the Wollaston condition (Langton et al., 2004; Maruyama & Endo, 1983; Todorovic, 2006, 2009; Wollaston, 1824). Consequently, in the Wollaston condition, −12° gaze is perceived to be close to direct gaze in the context of a +20° face, whereas 0° gaze is perceived to be averted toward the head orientation. Thus, we hypothesized that infants would prefer ±12° gaze in the Wollaston condition if they are sensitive to Wollaston's effect. On the contrary, infants would show a consistent pattern of preference toward the 0° gaze if they are insensitive to Wollaston's effect. 

Experiment

Methods

Participants

Sample size was determined on the basis of previous research reporting the direct gaze preference in infants (Farroni et al., 2002, 2006). The final sample consisted of 20 Japanese infants aged 4 and 5 months (11 male, nine female, mean age = 132 days, ranging from 105 to 154 days) and 20 Japanese infants aged 7 and 8 months (13 male, seven female, mean age = 228 days, ranging from 195 to 254 days). An additional 11 infants were tested but were excluded from the analysis because of fussiness (4), because of side bias of greater than 90% (3), or because these infants looked only at one side of the stimuli in one of the trials (4). The experiments adhered to the Declaration of Helsinki guidelines and were approved by the University Human Research Ethics Committee. 

Apparatus

The experiments were conducted in a darkened room, with the cathode ray tube (CRT) monitor for stimulus presentation as the only light source. All stimuli were displayed on a 21-inch CRT monitor (Sony GDM-F520), which was controlled by a computer (Dospara Prime Galleria). The resolution of the CRT was set at 1024 × 768 pixels with an eight-bit color mode and a refresh rate of 60 Hz. There were two loudspeakers, one on either side of the CRT monitor. A charge-coupled device (CCD) camera positioned just below the monitor screen was used to videotape the infant's behavior throughout the experiment. The experimenter could observe the infant's behavior live via a TV monitor connected to the CCD camera. Each infant sat on his or her parent's lap in front of the CRT monitor at the viewing distance of approximately 40 cm. The infant and parent were located inside an enclosure made of iron poles and covered with black light-blocking curtains that covered the apparatus and the experimenters from the infant's view, except for the screen of the CRT monitor and the lens portion of the CCD camera during the experiment. 

Stimuli

Stimuli were full-color images of an Asian female face (see Figure 2). We created a 3D model of the face using the PhotoFit function of FaceGen Modeller 3.5. An average face of four young Asian female faces was used as input. The 3D models of the faces created in FaceGen were imported into Blender 2.70. The original eyes in the faces were replaced with 3D models of eyes created in Blender. Each eye was set to track a fixation target using the “AutoTrack” feature in Blender. The deviation of each eye was controlled by changing the angular position of the fixation target in the horizontal plane. 

Figure 2

 

Example of stimulus pairs in each condition. These image pairs were shown on one of the two test trials. The left/right reversed versions of these images were shown in the other test trial. Images in the Wollaston condition were created by inserting the eyes of frontal face images into the angled face. Although the eyes are identical between the frontal face condition and the Wollaston condition, for adult observers, the perceived gaze direction shifts toward the head orientation in the Wollaston condition. In this example, −12° is perceived to be closer to “direct” and 0° gaze is perceived to be averted to the right of the observer.

First, we rendered images with straight gaze (0°) and leftward gaze (−12°) with the face oriented straight ahead. These images served as the stimuli for the frontal face condition (Figure 2, top row). Second, we rendered an angled face image with the face oriented rightward by 20°. We inserted each pair of eyes with 0° gaze and −12°gaze from the frontal face condition into the angled face image. The angled face image was tilted clockwise by 4° in the image plane to optimize the fitting of eyes without any rotation of the eyes or change in the distance between the eyes. The resultant images with 0° gaze and −12° gaze served as the stimuli for the Wollaston condition (Figure 2, middle row). Finally, by using the morph function of FaceGen, we created a 3D face with closed eyes and rendered an image with closed eyes with the face oriented straight ahead (Figure 2, bottom left). All images were rendered with a camera pointed to the right eye of the face and a single light source from above the camera illuminating the face. The image pairs shown in Figure 2 were used on one of the two test trials for each condition. The left/right reversed versions of these images were used in the other test trial. 

Each facial image subtended about 17° × 26° of visual angle from a distance of approximately 40 cm. All stimulus images were shown against a white background. 

Procedure

Each infant participated in two 20-s trials for each of the three image conditions: frontal face, Wollaston condition, open/closed eyes. The duration of each trial was fixed regardless of the infant's behavior. In the open/closed eyes condition, the face with 0° gaze was paired with that with closed eyes. In the frontal face and the Wollaston conditions, the face with 0° gaze was paired with the face with averted gaze (± 2°). For each condition, the images shown in Figure 1 were shown in one trial, and left-right reversed versions of these images were shown in the other trial. 

Prior to each trial, an animated fixation point with a short beeping sound was presented at the center of the CRT monitor. The experimenter initiated each trial as soon as the infant fixated on the central cartoon image. The position of the face with 0° gaze in the first trial was randomly determined for each infant and was reversed in the next trial. The order of the three image conditions was randomly determined for each infant. 

Analysis

One observer, who was unaware of the identity of the stimulus, measured the infant's looking time at each display on an offline video recorded during the experiments. The experimenters determined which direction the infants were looking based on their overall behavior including eye and head movements. The looking times were summed across the two trials for each image for each condition. To compute the interobserver agreement, a second observer's measurement of the infant's looking time was obtained for 25% of the total data. The interobserver agreement was high (r = 0.96), and the average absolute difference in the preferential looking score between the two coders was 1.48%. 

Results and discussion

Mean total looking time toward either of the two faces across two trials in each condition is shown in Table 1. All infants included in the analysis had a total looking time greater than 20 s for each of the three conditions, and none of the infants was rejected from analysis based on looking time criteria. 

Table 1

 

Mean total looking time in seconds during two trials, together with standard deviations in parentheses for each condition and age group.

 

In each condition, we calculated the preferential looking score by dividing the amount of time spent looking at the target (the face with 0° gaze) by the total time spent looking at either of the two faces during two trials and then multiplying this ratio by 100. As Shapiro-Wilk tests revealed that the preferential looking score for the frontal face condition in young infants deviated from a normal distribution (p < 0.01), we analyzed preferential looking scores using nonparametric methods. None of the Bonferroni-corrected Mann-Whitney U tests revealed a significant difference between younger and older infants on the preferential looking scores in any of the three conditions, p's > 0.05, suggesting a similar pattern of results between the age groups. Therefore, we focused on the differences between the conditions in the following analysis. 

Figure 3 shows the mean preferential score toward target (0° gaze) for each condition across age groups. We performed Friedman's analysis of variance to examine differences between the conditions. The analysis revealed significant variation in the preferential looking scores, χ²(2, N = 40) = 13.65, p < 0.01. We performed a series of Bonferroni-corrected Wilcoxon signed-rank tests comparing the preferential looking scores between the conditions. This analysis revealed that the preferential looking scores for the Wollaston condition were significantly lower than for the other two conditions (vs. frontal face condition: p < 0.05, d = 0.60; open/closed eye condition: p < 0.01, d = 0.94). On the other hand, the difference between the Frontal face condition and the open/closed eye condition failed to reach significance (p = 0.22). To determine whether infants' preference differs from the chance level of 50%, we also performed Bonferroni-corrected one-sample Wilcoxon signed-rank tests on the preference score in each condition. The analysis revealed a significant difference from the chance level only for the preference score in the open/closed eyes condition (p < 0.01, d = 0.64). 

Figure 3

 

Mean preference score for image with 0° gaze in each condition. Error bars represent 1 SE.

Parametric tests proved consistent with the nonparametric test as described above. A two-way analysis of variance with age as between-subject factor and condition as repeated factor revealed a significant main effect of condition only, F(2, 76) = 11.96, p < 0.01, η_p² = 0.24. A series of Bonferroni-corrected two-tailed paired-samples t tests between the conditions revealed that the preference score in the angled face condition was significantly smaller than in the other two conditions: versus frontal face condition, t(39) = 2.70, p < 0.05, d = 0.60; open/closed eye condition, t(39) = 5.14, p < 0.01, d = 0.94, whereas it was similar between the frontal face condition and the open/closed eye condition, t(39) = 2.01, p = 0.15. Bonferroni-corrected two-tailed one-sample t tests against the chance level of 50% on the preference score in each condition revealed a significant difference only for the open/closed eye condition, t(39) = 4.01, p < 0.01, d = 0.63. 

General discussion

We found that infants' looking behavior toward identical eyes was affected by a change in the head orientation context. In the context of a frontally oriented face, infants tended to prefer direct gaze (0° gaze) over gaze averted by ±12° (frontal face condition) and over closed eyes (open/closed eyes condition). These results are consistent with previous studies reporting direct gaze preference (Batki et al., 2000; Farroni et al., 2002, 2006). In the context of an angled face (Wollaston condition), however, infants tended to prefer the face with averted gaze over the face with direct gaze. The observed shift in the preference is consistent with the shift of perceived gaze direction in the Wollaston condition experienced by adults (Langton et al., 2004; Maruyama & Endo, 1983; Todorovic, 2006, 2009; Wollaston, 1824). Thus, our results suggest that 4- to 5- and 7- to 8-month-olds are sensitive to Wollaston's effect and they perceive gaze direction from identical eyes differently depending on the head orientation context. 

The development of the sensitivity to Wollaston's effect seems to coincide with previous reports suggesting development of sensitivity to direct gaze across head orientations (Farroni et al., 2002, 2004). Although it is not clear from the findings from Farroni et al. (2004) whether 4-month-olds integrate subtle facial orientation information in the gaze processing, our results using Wollaston's effect stimuli demonstrate that infants integrate head orientation as a cue to gaze direction in its own right. Recent psychophysical studies in adults suggested that integration of information about head orientation and gaze direction as observed in the Wollaston effect plays an important role in maintaining constant gaze perception across variations in the head orientation (Otsuka et al., 2014, 2015). Our results suggest that such a visual function is already at work by 4 to 5 months of age. 

In a previous study examining Wollaston's effect in infants, Nakato et al. (2009) reported that 8-month-olds but not 6- and 7-month-olds showed sensitivity to Wollaston's effect. Whereas Nakato et al. used black-and-white drawings similar to the original demonstration by Wollaston as the stimuli, we used realistic color images of faces. It is also notable that Nakato et al. (2009) used the familiarization/novelty preference procedure and assessed infants' preference between the novel and familiar images after repeated presentation of the familiarization image. On the other hand, the current study measured infants' spontaneous preference toward the images. Both of these methodological differences may have contributed to the differential results between Nakato et al. and the current study. 

The current results suggest that infants' gaze processing does not depend solely on local eye deviation but that they integrate eye and head orientation information by 4 to 5 months of age. Several previous studies on face recognition suggested that sensitivity to change in featural aspects of faces develops more rapidly than sensitivity to changes in more global or integrative aspects of the face (Bhatt, Bertin, Hayden, & Reed, 2005; Cashon & Cohen, 2004; Schwarzer, Zauner, & Javanovic, 2007). However, some recent studies have demonstrated that integrative processing of faces is functional from early in development (Quinn & Tanaka, 2009; Turati, Di Giorgio, Bardi, & Simion, 2010). For example, Turati et al. (2010) used a modified version of the composite face paradigm and found that an aligned but not misaligned bottom part of the face interfered with recognition of the top part of faces by 3-month-olds. Quinn and Tanaka (2009) reported that infants aged 3 to 4 and 6 to 7 months discriminated a change in the spacing between the facial features (“configural change”) better than a change in local facial feature size. Our results may be consistent with these latter studies. However, this link should be interpreted with caution because, unlike most global or integrative aspects of face processing such as the composite effect and configural processing, Wollaston's effect has been shown to be less susceptible to face inversion (Otsuka et al., 2015; Langton et al., 2004; Maruyama & Endo, 1984). 

Studies with adult participants have revealed that various properties of faces and facial images can influence the perceived direction of gaze. For example, perceived gaze direction is shown to be biased toward “direct” when the eyes are not clearly visible (Mareschal, Calder, & Clifford, 2013; Martin & Jones, 1982; Martin & Rovira, 1981) or when the face is showing a smiling or angry expression (e.g., Ewbank, Jennings, & Calder, 2009; Lobmaier, Tiddeman, & Perrett, 2008; Martin & Rovira, 1982; Slepian, Weisbuch, Adams, & Ambady, 2011). The degree to which adult observers rely on the head orientation in gaze judgment is shown to vary between faces viewed centrally and peripherally (Florey, Clifford, Dakin, & Mareschal, 2015). There are also studies reporting that facial familiarity influences adults' attentional behavior in response to others' gaze (Deaner, Shepherd, & Platt, 2007) and to children's performance on a gaze judgment task (Michel, Hoehl, & Striano, 2014). Future studies could address whether the properties of faces and facial images as mentioned above would affect infants' gaze perception. 

Acknowledgments

This work is supported by a Grant-in-Aid for Scientific Research (A) (21243041 to M. K. Y.) and a Grant-in-Aid for Scientific Research (B) (26285167 to M. K. Y.) from the Japan Society for the Promotion of Science and by the Australian Research Council Discovery Project (DP120102589 to C. W. G. C.). C. W. G. C. is supported by an Australian Research Council Future Fellowship (FT110100150). We thank Jiale Yang, Kazuki Sato, Megumi Kobayashi, Yuiko Sakuta, Yuka Yamazaki, and Yuna Inada for their help in data collection. 

Commercial relationships: none. 

Corresponding author: Yumiko Otsuka. 

Email: [e-mail]. 

Address: Faculty of Law and Letters, Ehime University, Matsuyama, Ehime, Japan.