The role of object affordances and center of gravity in eye movements toward isolated daily-life objects

Abstract

The purpose of the current study was to investigate to what extent low-level versus high-level effects determine where the eyes land on isolated daily-life objects. We operationalized low-level effects as eye movements toward an object's center of gravity (CoG) or the absolute object center (OC) and high-level effects as visuomotor priming by object affordances. In two experiments, we asked participants to make saccades toward peripherally presented photographs of graspable objects (e.g., a hammer) and to either categorize them (Experiment 1) or to discriminate them from visually matched nonobjects (Experiment 2). Objects were rotated such that their graspable part (e.g., the hammer's handle) pointed toward either the left or the right whereas their action-performing part (e.g., the hammer's head) pointed toward the other side. We found that early-triggered saccades were neither biased toward the object's graspable part nor toward its action-performing part. Instead, participants' eyes landed near the CoG/OC. Only longer-latency initial saccades and refixations were subject to high-level influences, being significantly biased toward the object's action-performing part. Our comparison with eye movements toward visually matched nonobjects revealed that the latter was not merely the consequence of a low-level effect of shape, texture, asymmetry, or saliency. Instead, we interpret it as a higher-level, object-based affordance effect that requires time, and to some extent also foveation, in order to build up and to overcome default saccadic-programming mechanisms.

Introduction

Vision is an activity. What you look at determines what you see. But what determines what you look at? On the one hand, you make rapid saccadic eye movements toward parts of a visual scene that are high in contrast or bright in color (Itti & Koch, 2000, 2001; Itti, Koch, & Niebur, 1998). Such saccades are reflexive and depend solely on the scene's low-level properties. On the other hand, you make saccades based on the scene's high-level properties, such as the objects in it (e.g., Nuthmann & Henderson, 2010). The extent to which both factors contribute to eye guidance has been the subject of debate for many years (for reviews, see Henderson, 2003; Rayner, 1998; Rayner, Liversedge, Nuthmann, Kliegl, & Underwood, 2009; Tatler, Hayhoe, Land, & Ballard, 2011). In this debate, the influence of early, low-level properties is typically contrasted with the later-occurring, high-level influence of semantic knowledge (Henderson, Weeks, & Hollingworth, 1999; Loftus & Mackworth, 1978). In the current study, we investigate this decades-old issue in a new way: by contrasting the time course of the effect of low-level stimulus properties with the time course of the effect of visuomotor priming by object affordances. Because visuomotor priming is believed to occur automatically, it may have early effects on eye guidance, perhaps even comparable to the effects of low-level features. 

Visuomotor priming refers to the notion that the mere sight of an action-related object, such as a hammer, immediately activates a motor program associated with it (for behavioral studies, see, e.g., Craighero, Fadiga, Umiltà, & Rizzolatti, 1996; Tucker & Ellis, 1998, 2001; for neuroimaging studies, see, e.g., Chao & Martin, 2000; Grèzes, Tucker, Armony, Ellis, & Passingham, 2003). Visuomotor priming is considered a high-level process because it is not directly related to the low-level properties of visual input. Rather, it requires some form of object recognition, however basic. And yet, visuomotor priming is assumed to occur automatically and nonvoluntarily (e.g., Handy, Grafton, Shroff, Ketay, & Gazzaniga, 2003; Tucker & Ellis, 2001). It has been suggested that object affordances automatically draw the eyes (Myachykov, Ellis, Cangelosi, & Fischer, 2013) and visuospatial attention (Roberts & Humphreys, 2011; see also Handy et al., 2003) toward the action-related part of the object. The purpose of the current study was to test this claim. More precisely, we compared these high-level, object-based effects with the effect of a purely low-level property of the object: its center of gravity (CoG). To this end, we presented participants with isolated photographs of graspable objects and investigated where the eyes landed relative to the objects CoG. 

High-level object-affordance effects

According to Gibson (1979), people perceive objects in terms of their potential usage. He coined the term “affordances” to refer to the action possibilities offered by the environment (Gibson, 1977). In line with this view, a vast amount of research has shown that perceiving an object automatically potentiates an associated motor program (e.g., Craighero et al., 1996). For example, seeing a frying pan with its handle protruded to the right facilitates right-hand responses compared to left-hand responses whereas the reverse is true when the handle protrudes to the left (Tucker & Ellis, 1998). Given this interplay between vision and action, the question arises whether action-related objects facilitate visuomotor transformations by automatically capturing visuospatial attention (Craighero et al., 1996; Jeannerod, Arbib, Rizzolatti, & Sakata, 1995). To investigate this, Handy and colleagues (2003) presented two objects bilaterally (i.e., one on each side of the display), one of which was graspable and one of which was not. Participants indicated over which of the two objects a target was superimposed. The results demonstrated that the event-related potential component P1, which is assumed to reflect enhanced visual processing for attended locations (Clark & Hillyard, 1996), was larger if the target was superimposed over a graspable compared to a nongraspable object. The authors concluded that action-related objects indeed capture attention (Handy et al., 2003), a finding that fits well with Gibson's (1977, 1979) theory of affordances. 

The handle-affordance hypothesis

As described above, most evidence for attentional capture by object affordances comes from studies that have contrasted graspable with nongraspable objects (e.g., Handy et al., 2003). However, following this logic, potentials for action should also capture attention within a single object. At least one study suggests that this is indeed the case. Myachykov and colleagues (2013) measured eye movements while participants viewed and categorized graspable objects. They found that participants spent proportionally more time looking at an object's handle than at other parts even though the (location of the) handle was irrelevant for the task. From these results, the authors concluded that an object's graspable part automatically captures visuospatial attention (Myachykov et al., 2013). We will refer to this line of reasoning as the handle-affordance hypothesis, which predicts that when you make an eye movement toward a graspable object, the eyes should land toward the handle (see Figure 1, orange arrow). 

Figure 1.

 

The handle-affordance hypothesis predicts that the eyes will land toward the handle of a graspable object whereas the action-performing hypothesis predicts that the eyes will go to the other side of the object, that is, in the direction of the action that is implied by the object. The CoG/OC hypothesis predicts that the eyes will land on the object's CoG or OC.

The action-performing hypothesis

In direct contrast to the handle-affordance hypothesis, Roberts and Humphreys (2011) reasoned that action-related objects should bias visuospatial attention in the direction of the action implied by the object. For example, a hammer implies the action “hammering,” which induces an attentional shift toward the hammer's head rather than its handle. After all, in daily life that would be the most probable location to find the (to-be-hammered-on) nail. To test their prediction, Roberts and Humphreys used a Posner-cueing paradigm (Posner, Snyder, & Davidson, 1980) in which graspable objects functioned as central cues. The authors predicted, and found, a cueing effect at the action-performing side of the object (e.g., at the head, but not at the handle, of a hammer) and concluded that visuospatial attention is biased toward the direction of the action implied by the object (Roberts & Humphreys, 2011). We will refer to this line of reasoning as the action-performing hypothesis. As with the handle-affordance hypothesis described above, the action-performing hypothesis is based on Gibson's (1977, 1979) theory of affordances. However, it makes a very different prediction: When you make an eye movement toward a graspable object, the eyes should land toward the action-performing part (see Figure 1, green arrow). Interestingly, Vainio, Ellis, and Tucker (2007) employed a similar paradigm (although to test the handle-affordance hypothesis) and did not find a bias to either side of the object. 

A low-level CoG effect

In visual displays containing two simple shapes, saccades reveal a so-called global effect: Even though participants aim for one of the two stimuli, their eyes deviate toward the other stimulus and land on a location in between the two (Coren & Hoenig, 1972; Findlay, 1982; for reviews, see Vitu, 2008; Van der Stigchel & Nijboer, 2011). This systematic landing-position error is typically interpreted as a tendency for the eyes to land on the CoG of the peripheral visual configuration. For example, Findlay (1982) demonstrated that when two targets differ in size, the eyes do not land exactly at the midpoint between the two but deviate toward the largest target. Likewise, the deviation from the midpoint is stronger for brighter stimuli (Deubel, Wolf, & Hauske, 1984). For the current study, it is important to note that the global effect generalizes to natural objects (Zelinsky, Rao, Hayhoe, & Ballard, 1997). 

Two different accounts for the global-effect phenomenon have been proposed. According to the saccadic-averaging account, the neural basis of the global effect is the superior colliculus, a brainstem region involved in saccade generation. The superior colliculus contains retinotopically organized sensory and motor maps that consist of neurons with large and overlapping receptive/movement fields. As a consequence, activity stemming from two proximally presented visual stimuli combines into one central peak of activity (see, e.g., Vokoun, Huang, Jackson, & Basso, 2014). This peak of activity subsequently triggers a saccade, and the eyes land in between the two stimuli (Findlay & Walker, 1999; Van Opstal & Van Gisbergen, 1989). Such saccadic averaging is considered as the default saccade-programming mode, which can only be overcome if saccadic programming is sufficiently long (Coëffé & O'Regan, 1987; Ottes, Van Gisbergen, & Eggermont, 1985). In line with this idea, the global effect is particularly likely to occur for early-triggered saccades. When their latencies increase, saccades become less susceptible to the global effect (Coëffé & O'Regan, 1987; Vitu, Lancelin, Jean, & Farioli, 2006). In contrast to the saccadic-averaging account, others explain the global effect as a visuomotor strategy (He & Kowler, 1989). According to this strategy account, observers send their eyes toward an intermediate position because this brings the eyes closer to the target. This, in turn, is assumed to optimize subsequent visual information uptake. 

Regardless of which mechanism underlies the global effect, for the current study it is of primary interest whether the eyes are also drawn toward a display's CoG when the display only contains a single stimulus. In this case, on-stimulus landing positions close to the stimulus' CoG would be predicted. Several studies demonstrated that this is indeed the case: When participants were asked to move their eyes toward a line drawing of a simple shape, their eyes landed at the stimulus' CoG (He & Kowler, 1991; Kaufman & Richards, 1969; Kowler & Blaser, 1995; Richards & Kaufman, 1969). 

Research on eye movements toward isolated daily-life objects has shown that the eyes typically land toward the object's center. This phenomenon is referred to as the preferred viewing location (PVL) effect, exactly as the tendency to preferentially land toward the center of words during reading (Rayner, 1979; for reviews, see Vitu, 2008, 2011). For example, Henderson (1993) presented participants with arrays of line drawings of objects and found that landing positions were clustered around the centers of the objects. This was later confirmed by studies using arrays of photographs of real objects instead of line drawings (Foulsham & Underwood, 2009). Even studies using complex natural scenes with objects embedded in them found that observers tend to preferentially make saccades toward the center of objects (Foulsham & Kingstone, 2013; Nuthmann & Henderson, 2010; Pajak & Nuthmann, 2013). Interestingly, the PVL for objects in scenes is modulated by saccade properties (e.g., saccade direction, see Nuthmann & Henderson, 2010; and launch-site distance, see Pajak & Nuthmann, 2013) as well as object properties (e.g., object size, see Pajak & Nuthmann, 2013; and object category, see Yun, Peng, Samaras, Zelinsky, & Berg, 2013). The PVL phenomenon is typically interpreted as a general visuomotor strategy that aims at a location within the stimulus that optimizes its subsequent processing: Fixating at a word or an object's center maximizes the area of the stimulus that benefits from the high visual acuity that foveal vision provides (Henderson, 1993; McConkie, Kerr, Reddix, & Zola, 1988; Nuthmann & Henderson, 2010; Pajak & Nuthmann, 2013). In contrast with this view, Vitu (2008, 2011) proposed that the PVL effect observed for words during reading could also be simply a result of the averaging of the activity of population(s) of neurons with large and overlapping receptive/movement fields, exactly as the global effect with two stimuli. 

To summarize, part of the literature on eye movements toward isolated simple shapes is interpreted as a tendency to move the eyes toward the COG¹ (He & Kowler, 1991; Kaufman & Richards, 1969; Kowler & Blaser, 1995; Richards & Kaufman, 1969). On the other hand, literature on eye movements toward isolated objects (Foulsham & Underwood, 2009; Henderson, 1993) and objects in scenes (Foulsham & Kingstone, 2013; Nuthmann & Henderson, 2010; Pajak & Nuthmann, 2013) is framed in terms of a tendency to move the eyes toward the object center (OC). For real-world objects in general, and for the objects used in the current experiment in particular, CoG and OC are often very close to each other. As a consequence, in the current study we cannot reliably discriminate whether observers moved their eyes toward an object's CoG or the OC. We therefore grouped both predictions and referred to this grouped hypothesis as the CoG/OC hypothesis. We assumed that the tendency to move the eyes toward the CoG/OC can be determined by low-level processes that merely require extracting objects' boundaries. This tendency may prevail over higher-level effects, even those related to objects' affordances. Thus, the CoG/OC hypothesis predicts that the eyes will land close to the CoG/OC of the stimulus. 

Current study

Previous studies on the effect of visuomotor priming have yielded equivocal results when it comes to the distribution of visual attention within graspable daily-life objects. Whereas cueing paradigms demonstrated an attentional shift away from the handle (Roberts & Humphreys, 2011) or no attentional shift at all (Vainio et al., 2007), Myachykov and colleagues (2013) found a bias toward the handle. Importantly, to our knowledge, none of these studies have taken the low-level properties (e.g., the objects' CoG or the distribution of visual saliency) of the stimuli into account. This is crucial because if the eyes are indeed drawn toward the CoG of a visual display (e.g., Findlay, 1982; Vitu, 2008; Zelinsky et al., 1997) the attentional shift toward the action-performing side observed by Roberts and Humphreys (2011) may simply be explained by the fact that, on average, their stimuli were more visually dense on this side or vice versa for the bias toward the handle side observed by Myachykov and colleagues. 

Therefore, the purpose of the current study was to investigate the contribution and time course of low-level CoG/OC effects versus high-level object-affordance effects on where the eyes land on isolated daily-life objects. To this end, we recorded eye movements of participants who viewed simple visual displays containing one isolated graspable object. The object was initially presented in peripheral vision, such that participants' initial saccades brought the object into foveal vision. Before giving a response, participants typically also made one or more refixations within the boundaries of the object. We analyzed the landing positions of both the initial saccades and the refixations in order to examine whether they were biased to the object's handle, the object's action-performing side, or the object's CoG or absolute center. It is of note that the three hypotheses are not mutually exclusive because their effects may come into play with different time courses. More precisely, we predicted that saccades that are executed early in time would be more subject to CoG/OC effects (Coëffé & O'Regan, 1987; Vitu et al., 2006) whereas saccades that are executed later in time would be more subject to object-based, higher-level effects. In Experiment 1, we analyzed eye movements that participants made toward real objects and determined whether the eyes landed on the object's CoG/OC, on the object's handle, or on the object's action-performing part. In Experiment 2, we did the same but added nonobjects that were matched on the low-level properties of the real objects in order to further disentangle the role of low-level versus high-level stimulus properties on eye guidance. 

Experiment 1

Methods

Stimuli and data are available from the first author's website: http://www.cogsci.nl/lvanderlinden/. 

Participants

Eighteen observers participated in Experiment 1. All were right-handed, had normal or corrected-to-normal vision, and were naive as to the purpose of the experiment. They received payment (€10 per hour) in return for their participation and gave their written informed consent. The experimental procedure was in accordance with the Declaration of Helsinki. 

Apparatus

Participants sat in front of a computer screen in a dimly lit room. Stimulus presentation was controlled by OpenSesame (Mathôt, Schreij, & Theeuwes, 2012) in combination with PsychoPy (Peirce, 2007) on a 21-in. CRT monitor with a resolution of 1024 by 768 pixels and a refresh rate of 100 Hz. The distance between the participant's eyes and the monitor was 75 cm and was kept constant by stabilizing the participant's head with a chin rest. Manual responses were collected on a button box. Eye-position data of the right eye were recorded with a remote EyeLink 1000 system (SR Research Ltd., Mississauga, Ontario, Canada) with a sampling rate of 1000 Hz (accuracy: 0.5°; precision: 0.01° RMS). Viewing was binocular. 

Materials

We selected 18 colored photographs of daily-life objects from two standardized stimulus sets (Brodeur, Dionne-Dostie, Montreuil, Lepage, & Op de Beeck, 2010; Moreno-Martínez & Montoro, 2012). Half of the objects were kitchen utensils whereas the other half were garage tools. All objects were relatively long and narrow (width of bounding box around the stimulus: 4.4°–5.7°; height of bounding box around the stimulus: 0.65°–2.02°) and were oriented horizontally. Per category, seven of the nine objects were “handled,” i.e., more graspable on one side than the other (e.g., a knife). The remaining four objects (two from each category) were roughly symmetrical and equally graspable on both sides (e.g., a ruler). These were used as fillers to decrease the chance that participants would notice our handle-orientation manipulation (explained below). The filler trials were not included in the analyses. 

Design

When looking at natural scenes (Dickinson & Intraub, 2009; Foulsham, Gray, Nasiopoulos, & Kingstone, 2013; Nuthmann & Matthias, 2014) or more controlled displays (Williams & Reingold, 2001; Zelinsky, 1996), participants' initial saccades are directed more often leftward than rightward. To prevent this pseudoneglect (Bowers & Heilman, 1980) from influencing our dependent variable (i.e., landing positions on daily-life objects), objects were presented in two different orientations, such that, in a given trial, the handle was pointing either toward the left or toward the right. The here-reported handle-orientation conditions were part of a larger experiment, which also contained two conditions in which stimulus contrast was manipulated. These conditions are not reported in the current paper. 

To investigate the time course of low-level versus high-level effects on saccadic landing positions, a wide range of saccade latencies was needed. To this aim, we used both “step” (i.e., 0-ms gap) and “overlap” trials. The latter are known to result in longer saccade latencies than the former (Saslow, 1967). Thus, in half of the trials, the fixation dot was removed as soon as the object appeared on screen (“step” trials). In the other half of the trials, the fixation dot remained on screen during object presentation (“overlap” trials). Objects were presented either in the upper or in the lower visual field. To prevent saccade amplitude from becoming predictable, we varied stimulus eccentricity randomly between 5° and 7° (M = 5.99°, SD = 0.59°). 

Procedure

The experiment started with a nine-point grid calibration procedure. A typical trial sequence is shown in Figure 2a. Before the start of each trial, a central one-point eye-tracker recalibration (“drift correction”) was performed. The trial proper started with a central black fixation dot (diameter: 0.24°) on a white background. After a random interval (μ = 400 ms, σ = 50 ms, from a Gaussian, min. = 200 ms, max. = 1000 ms) and only when a stable fixation was detected within a 1.5° vertical region centered on the fixation dot, the object appeared in the upper or lower visual field, and the fixation dot either disappeared or stayed on screen (see above). The object's center (i.e., the middle of the bitmap) was aligned with the vertical meridian. 

Figure 2.

 

Trial sequence (a) and dependent variable (b) in Experiment 1.

Participants were instructed to move their eyes toward the object as quickly and accurately as possible. Next, they had to categorize it as either a kitchen utensil or a garage tool by pressing a right- or left-hand button. A button press was effective only when participants gazed at the object (i.e., when fixation position did not deviate more than 1.5° from the vertical center of the object for 50 consecutive samples). If this fixation check took more than 2000 ms to complete, the check was considered as failed. In this case, participants heard a brief warning beep. Trials in which this happened were not analyzed. The object remained on screen until a response was made or a time out of 2500 ms occurred. Finally, a central red or green fixation dot was displayed (500 ms) to inform participants about the correctness of their response (incorrect or correct, respectively). 

The experiment contained six blocks of 96 trials and started with six practice trials. Within blocks, objects were presented once in every condition, resulting in six object repetitions per block. The response rule (e.g., left for kitchen, right for garage) was swapped halfway through the experiment, and the order of response rules was counterbalanced across participants. At the end of every block, participants were informed about their average response time and accuracy on the categorization task. If their accuracy was below 85% correct, they received a warning message asking them to be more accurate. 

Data analysis

Given that our handle-side manipulation only affected horizontal gaze position, we will report only the x-coordinates of saccadic landing positions. More precisely, we normalized these coordinates such that, irrespective of handle orientation (left or right) and the exact size of the closest-fitting rectangular bounding box around the stimulus, landing positions ranged between −0.5 and 0.5. A value of 0.5 meant that the eyes landed at the extreme border of the object's handle side whereas a value of −0.5 indicated that the eyes landed at the extreme border of the object's action-performing side. A value of 0 indicated that the eyes landed exactly at the middle of the bitmap (see Figure 2b). 

Saccades were detected using the built-in EyeLink saccade/fixation-detection algorithm with the default parameters. We found that participants executed at least one (100%) or two (70%) saccades before making a manual response. The first saccade brought the peripherally presented object into foveal vision whereas the second saccade was made within the borders of the already foveated object. This resulted in two dependent variables: the landing positions of initial saccades and the landing positions of the refixations, relative to the object's absolute center. 

Our two main research questions were whether landing positions show a systematic preference for a particular part of the object, and, if so, whether this bias changes over time. To answer these questions, we ran linear mixed effect (LME) models for initial saccades and refixations separately (by using the R package lme4; Bates, Maechler, Bolker, & Walker, 2014). We included the continuous variable saccade latency as a fixed effect. In the current study, for both initial saccades and refixations, we defined “saccade latency” as the time relative to stimulus onset. We centered this variable around its mean. Furthermore, we added random intercepts for participant and object as well as random slopes for participant by saccade latency and for object by saccade latency. We interpreted effects with a value of t > 2 as reliable although we emphasize general patterns over significance of individual tests (Baayen, Davidson, & Bates, 2008). 

As aforementioned, we normalized landing positions relative to handle orientation (such that, for example, positive values always indicate that the eyes landed toward the handle regardless of how the stimulus was oriented on the display). Thus, the intercept of our LME model represents gaze bias relative to the OC when saccade latency is at its reference value. We plotted 95% confidence intervals (CIs; determined on the basis of the function's intercept and corresponding standard error) around the fitted function. Finally, the slope of the relationship between latencies and landing positions indicates whether the direction or the strength of any potential bias changed over time. 

Trials were excluded according to the following criteria: No sufficiently large saccades (with a landing position that deviated more than 2.5° from the central fixation dot on the y-axis) were detected (0.24%), the manual response was incorrect (5.43%), an anticipatory saccade (latency lower than 80 ms) was made (0.59%), or our gaze-contingent fixation checks (see Procedure) failed (1%). Finally, we discarded trials in which landing positions or saccade latencies deviated more than 2.5 SD from the participants' mean (initial saccades: 0.79%, refixations: 1.93%). 

Results

First, we plotted the distribution of landing positions of the initial saccades that participants made toward the peripheral stimulus and the distribution of refixations that participants made within the stimulus. To this end, we first removed the between-subjects variability from the landing positions (Cousineau, 2005). Next, we divided landing positions into 15 equal bins. Figure 3a shows that the distribution of initial saccades appears to be unimodal and that it peaks just to the left of the OC, toward the action-performing side. The distribution of refixations is more skewed. It shows a clear peak toward the action-performing part of the object and a slight tail toward the handle side of the object. 

Figure 3.

 

(a, b) Distributions of initial saccades (blue) and refixations (orange) relative to the object's absolute center (a, gray vertical dotted line) and the object's CoG (b, gray dotted line). The x-axis depicts normalized landing positions. Positive values indicate landing positions on the object's handle side, and negative values indicate landing positions on the object's action-performing side. In order to keep the range on the y-axis constant for both distributions, we normalized absolute frequencies relative to their minimum and maximum frequency within a given distribution. (c) Average gaze bias of initial saccades (orange) and refixations (blue), relative to the object's absolute center (gray horizontal dotted line) as a function of saccade-onset time relative to stimulus onset. The y-axis depicts normalized landing positions. The gray dotted line indicates the reference point (i.e., absolute center); positive values indicate landing positions on the object's handle side, and negative values indicate landing positions on the object's action-performing side. Markers indicate saccade-latency bin means and are plotted for visualization purposes only. Lines indicate linear regressions yielded by the two LME analyses, and shaded areas indicate 95% CIs based on their respective intercepts. Consequently, we interpreted no overlap with the reference point (dashed horizontal line) as a systematic gaze bias. The slope of the relationship between latencies and landing positions indicates whether the direction or the strength of the bias changed over time.

To investigate the time course of these tendencies, we performed the LME analyses as described above (see Data analysis). The results are shown in Figure 3c and Table 1. First, we found that for initial saccades triggered with a mean latency of 175 ms (i.e., the reference value of saccade latency), landing positions approached but did not reach a reliable bias away from the OC (intercept estimate = −0.04, SE = 0.02, t = −1.99). Importantly, however, landing positions of initial saccades did vary as a function of saccade latency, such that the bias toward the action-performing side increased for later-triggered saccades. Furthermore, we found that refixations with a mean latency of 402 ms were directed toward the object's action-performing side (intercept estimate = −2, SE = 0.08, t = −2.48). The 95% CIs indicate that this bias was present throughout the entire range of refixation latencies. This bias was not reliably influenced by saccade latency. 

Table 1.

 

Results for the fixed effects in the LME analyses for the landing positions (relative to the object's center) of initial saccades and refixations in Experiment 1. Notes: For initial saccades, the reference value for the factor saccade latency is 174.89 ms. For refixations, the reference value for the factor saccade latency is 402.11 ms.

 

Correction for CoG

We found that early initial saccades landed near the objects' center whereas later saccades showed a bias toward the objects' action-performing side. The latter is consistent with the action-performing hypothesis (Roberts & Humphreys, 2011). However, an alternative low-level explanation cannot yet be ruled out. After all, action-related tools and utensils tend to be asymmetric in the sense that their handle is longer and narrower than the action-performing part. Consequently, photographs of these objects may contain more visual stimulation (e.g., pixels, contrast with the background, etc.) on the action-performing side (e.g., the head of the hammer in Figure 1) as compared to the handle side. Given this potential asymmetry, the question arises whether the bias toward the action-performing side observed in Experiment 1 was not merely caused by the fact that the CoG of the stimuli was systematically shifted toward the action-performing side. To test this possibility, we first calculated the stimuli's CoG (see Appendix A). Our calculation revealed that the CoG was close but not identical to the stimuli's OC. On average, the CoG of our stimuli was shifted (by about 2% of the object's width) toward the action-performing side (M = −0.10°, min. = −0.5°, max. = 0.5°). For only three out of 14 objects, the CoG was shifted toward the handle side. The resulting CoG for each stimulus is shown in the Supplementary Materials. To “correct” for this small asymmetry, we subtracted the CoG of the relevant object from the (normalized) landing position(s) of the saccade(s) toward that object.² 

The results are shown in Figure 3b and Table 2³ and suggest that the previously observed pattern still holds. First, initial saccades with a mean latency (175 ms) showed a small bias toward the objects' action-performing side (intercept estimate = −0.04, SE = 0.02, t = 2.18). More importantly, however, initial landing positions varied as a function of saccade latency, such that the action-performing bias increased over time. Refixations with a mean latency (403 ms) showed a more-pronounced action-related bias (intercept estimate = −0.2, SE = 0.04, t = −4.84). The 95% CI indicates that this bias was present throughout the entire range of refixation latencies. In sum, early initial saccades landed close to the stimulus' CoG whereas later saccades and especially refixations were biased toward the object's action-related part. 

Table 2.

 

Results for the fixed effects in the LME analyses for the landing positions (relative to the object's CoG) of initial saccades and refixations in Experiment 1. Notes: For initial saccades, the reference value for the factor saccade latency is 174.88 ms. For refixations, the reference value for the factor saccade latency is 402.50 ms.

 

Discussion

In line with the CoG/OC hypothesis, Experiment 1 revealed that when initial saccades toward the peripherally presented object were triggered early, the eyes landed close to the OC/CoG. When initial saccades were launched with longer latencies or when a refixation was generated, the eyes were systematically directed toward the object's action-performing side. We interpreted the latter as a higher-level, object-based effect that takes time to build up. Interestingly, the time that elapsed since stimulus onset could not entirely account for participants' landing positions: For initial saccades and refixations that were initiated with comparable latencies (i.e., between 250 and 280 ms after stimulus onset), the bias was stronger for refixations than for initial saccades (i.e., the curves of the refixations laid below the curves of the initial saccades, thus indicating that the eyes landed farther away from the reference point). 

To exclude alternative, low-level explanations for the action-related bias, we additionally compared landing positions to the stimuli's CoG rather than absolute center. The results of this additional analysis suggested that the visual asymmetry of our stimuli alone is not sufficient to explain the action-related gaze bias. To further exclude this possibility, we conducted a second experiment. 

Experiment 2

The purpose of Experiment 2 was to (a) replicate the low-level CoG/OC effect observed in Experiment 1 and (b) to exclude the possibility that the observed action-related bias was solely due to low-level stimulus properties. To this end, we controlled for the low-level features of our stimuli in several ways. First, and most importantly, we directly compared landing positions on real action-related objects with landing positions on meaningless nonobjects that were matched in shape, texture, and asymmetry. Second, we centered objects on their CoG during stimulus presentation instead of applying a post hoc correction for the CoG as in Experiment 1. Third, we compared observed landing positions with landing positions that were simulated using Itti and colleagues' (1998) saliency model. In addition, compared to Experiment 1, we improved the methodology of Experiment 2 on several other aspects. For example, in Experiment 1, participants categorized objects as either kitchen utensils or garage tools. Arguably, in this task the action-related part of the object is more important than the handle because the handles tend to look alike, but the action-related parts do not. Hence, these parts may have contained the information that was most relevant for the categorization task. Consequently, the bias toward the action-performing part might be explained as a strategy to move the eyes toward the most diagnostic part of the object. Therefore, in Experiment 2, participants simply indicated whether the stimulus was a real object or a nonobject. For this task, there is no reason to think that one part of the object is more crucial for the task than other parts. Moreover, the objects were displayed on various axes around the vertical meridian. We did so to prevent saccade direction from being predictable and hence to ensure that saccades landing on the object's CoG/OC were not merely the result of participants systematically making vertical saccades. 

The main question of Experiment 2 was whether the findings from Experiment 1 would hold when the low-level properties of the stimuli were controlled. Our predictions were as follows: If the CoG/OC effect is indeed an early, low-level effect, it should occur regardless of a stimulus' identity and therefore for both real objects and nonobjects. Thus, landing positions of early saccades should be unimodal, narrow, and peaking around the stimulus' CoG. Conversely, if the action-performing bias is indeed a later, high-level, object-based effect, it should occur only for stimuli that have an action-performing part. Thus, we should observe the action-performing bias⁴ for real objects but not for visually matched nonobjects. More precisely, the distribution of later saccades toward real objects should be asymmetric with a clear peak toward the action-performing side. In contrast, later saccades toward nonobjects should not show any systematic bias, resulting in either a bimodal distribution or a distribution that is still unimodal around the CoG but wider than the initial landing position distribution. Both types of distribution would indicate that, after initially fixating near the CoG, participants moved their eyes randomly to either side of the object. 

Methods

Participants, apparatus

Eighteen different observers participated. As in Experiment 1, all were right-handed, had normal or corrected-to-normal vision, and were naive as to the purpose of the experiment. We recorded eye movements with the same apparatus as in Experiment 1. 

Design

The design of Experiment 2 differed from Experiment 1 in the following aspects. First, we manipulated the factor stimulus type, such that the stimulus could be either a real object or a nonobject. Furthermore, to prevent saccade direction from being predictable, the object was displayed on various axes around the vertical meridian. More precisely, we used a radial arrangement in which the stimulus could appear at an angle of 70°, 90°, 110°, 250°, 270°, or 290° (angles are given relative to the horizontal meridian). Crossed with the factor handle orientation (left or right), this resulted in a repeated-measure 2 × 6 × 2 design. Objects were aligned on their CoG (see Appendix A for how we calculated the CoG) rather than their absolute center. As in Experiment 1, objects were presented with an eccentricity randomly varied between 5° and 7° (M = 6.01°, SD = 0.58°). 

Materials

The real objects were the same as in Experiment 1 except that we did not use the symmetric filler objects anymore. For every object, we generated one visually matched nonobject. The shape, texture, and CoG of the nonobjects were matched to the real objects as much as possible. To generate the shapes of the nonobjects, we first estimated the shape of each real object by measuring radius (i.e., the distance between the outer border and the center) as a function of angle. To give some examples, for a circle, the radius is constant for all angles; for a square, the radius varies as a triangular waveform as a function of angle; for real objects, the relationship between angle and radius is complex and captures many important properties of the object's shape, such as its elongation, whether it has an uneven outline, etc. Next, we made pairs of real objects and calculated the average of their radius-angle functions to create a new shape. This resulted in nonobject shapes that superficially resembled real objects but did not have recognizable properties. Next, we randomly assigned one nonobject shape to each real object. Second, the texture of the nonobject was matched to the texture of the corresponding real object. To this end, we applied a texture synthesis algorithm (Portilla & Simoncelli, 2000) to the real objects⁵ and used its output as the texture for the matched nonobject. Finally, we matched the CoG (on the horizontal axis) of the real object to the CoG of the nonobject. We first calculated the CoG of the real objects (see Appendix A). Next, we adjusted the asymmetry of the nonobject (by making one side thicker than the other side) until the CoG of the nonobject matched the CoG of the real object. 

Procedure

The procedure of Experiment 2 differed from Experiment 1 in the following aspects. Participants had to indicate with a button press whether the stimulus was a real object or a nonobject. The response rule (i.e., which button to press for which response) was varied between participants. Participants performed 14 blocks of 48 trials. Finally, no other online fixation checks than the one-point eye-tracker recalibration at the beginning of each trial was used. 

Data analysis

In Experiment 2, participants executed at least one (96.2%) or two (48.1%) saccades before making their manual response. We normalized the landing positions of these saccades on angle, handle orientation, and the width of a bounding box around the object. We analyzed the x-coordinates of these normalized landing positions relative to the object's CoG. The LME analyses were similar to Experiment 1 except that we added the following effects to the LME models: stimulus type (real object or nonobject) and its interaction with saccade latency as fixed effects and random slopes for participant by stimulus type and object by stimulus type. Furthermore, we added the factor deviation (yes or no) as a fixed effect. “No” indicated that the stimulus appeared at the vertical meridian (as in Experiment 1), and “yes” indicated that the stimulus was presented at a different angle (see Design). We limited ourselves to a two-level factor because we only wanted to test whether the observed results in Experiment 1 generalize to objects presented on a slightly different angle as well. 

Trials were discarded on the basis of the following criteria: No saccades with a y-coordinate that deviated >2.5° from the central fixation dot were detected (3.38%), an anticipatory saccade (latency lower than 80 ms) was made (0.11%), or an erroneous response was given or a time out occurred (5.38%). Finally, trials in which landing positions or saccade latencies deviated more than 2.5 SD from the participants' mean were discarded (first saccade: 1.39%, second saccade: 1.31%). 

Results

First, we plotted the distributions of landing positions relative to the CoG for objects and nonobjects separately. As in Experiment 1, we first removed the between-subjects variability from the landing positions (Cousineau, 2005). Next, we divided landing positions into 15 equal bins. The resulting distributions are shown in Figure 4. As predicted, initial saccades toward nonobjects (Figure 4a, light orange distribution) were unimodally and narrowly distributed. The distribution peaked close to the CoG. Initial saccades toward real objects (Figure 4b, light blue distribution) showed a similar pattern although they appeared to be slightly biased toward the action-performing part of the object. Second, as predicted, the distribution of the landing positions of refixations within nonobjects (Figure 4a, dark orange distribution) was wider compared to that of initial saccades but remained unimodal and peaked around the CoG. In contrast, refixations within real objects (Figure 4b, dark blue distribution) did show a systematic bias, such that the distribution of their landing positions was skewed and peaked toward the action-performing part of the object. 

Figure 4.

 

Experiment 2: Distributions of initial saccades (light colors) and refixations (dark colors) toward nonobjects (a) and real objects (b), relative to the stimulus' CoG (gray vertical dotted line). The x-axis depicts normalized landing positions, such that positive values indicate landing positions on the object's handle side and negative values indicate landing positions on the object's action-performing side. In order to keep the range on the y-axis constant for both distributions, we normalized absolute frequencies relative to their minimum and maximum frequency within a given distribution.

Next, to investigate the time course of these effects, we examined landing positions as a function of stimulus type and saccade latency for initial saccades and refixations separately. The results are shown in Figure 5a and Table 3. For initial saccades, our LME analysis revealed an interaction between saccade latency and stimulus type (estimate = −0.0003, SE = 4.1 × 10⁻⁵, t = −6.000) but no main effects. Indeed, Figure 5a shows that initial saccades toward nonobjects landed approximately on the CoG and that this effect did not appear to change over time. For real objects, we observed a different pattern: If the initial saccade was triggered early, the eyes landed close to the CoG. However, when saccade latencies increased, the eyes started to deviate toward the action-performing part of the object. 

Figure 5.

 

(a) Experiment 2: Average gaze bias of initial saccades (left) and refixations (right) toward real objects (blue) and nonobjects (orange), relative to the stimulus' CoG (gray dotted line) as a function of saccade-onset time relative to stimulus onset. The y-axis depicts normalized landing positions, such that the gray dotted line indicates the reference point (i.e., CoG), positive values indicate landing positions on the object's handle side, and negative values indicate landing positions on the object's action-performing side. Markers indicate saccade latency bin means and are plotted for visualization purposes only. Lines indicate linear regressions yielded by the two LME analyses, and shaded areas indicate 95% CIs based on their respective intercepts. Consequently, we interpreted no overlap with the reference point (gray horizontal line) as a systematic gaze bias. The slope of the relationship between latencies and landing positions indicates whether the direction or the strength of the bias changed over time. (b) Landing positions of saccades simulated based on Itti and colleagues' (1998) model (see text for explanation). The y-axis of (b) is identical to the y-axis of (a), which makes the simulated landing positions directly comparable with the empirical landing positions.

Table 3.

 

Results for the fixed effects in the LME analyses for the landing positions (relative to the object's CoG) of initial saccades and refixations in Experiment 2. Notes: For initial saccades, the reference value for the factor saccade latency is 182.98 ms. For refixations, the reference value for the factor saccade latency is 396.99 ms. Nonobject was the reference value for the factor stimulus type, and “no” was the reference value for the factor deviation.

 

For refixations, we found a main effect of stimulus type, indicating that refixations within real objects showed a stronger bias toward the action-performing part of the stimulus than refixations within nonobjects (estimate = −0.1298, SE = 0.054, t = −2.405). The significant interaction with saccade latency (estimate = −0.0001, SE = 4.1 × 10⁻⁵, t = −2.661) indicates that this difference increased over time. 

For both initial saccades and refixations, normalized landing positions did not vary as a function of the object's deviation from the vertical meridian. This suggests that the observed results generalize to objects that are presented at a slightly different angle. 

Saliency simulation

Experiments 1 and 2 consistently showed that the action-performing bias increased with saccade latency and that it was maximal for refixations. This time course, in combination with the absence of a similar effect for feature-matched nonobjects, suggests that the action-performing bias is the result of higher-level, object-based processing. However, there is still one alternative, low-level explanation that has not yet been ruled out. It is still possible that, despite our effort to match the low-level features of our real objects and nonobjects, the action-performing side of the real objects was more visually salient than the corresponding “action-performing” side of the nonobjects. If so, the dissociation between the gaze bias for both types of stimuli could be explained by low-level saliency only. To rule out this possibility, we used Itti and colleagues' (1998, see Appendix B) saliency model to simulate two saccades toward every display used in Experiment 2. These simulated saccades are a best effort to predict where the eyes would land if eye-movement guidance were purely determined by bottom-up visual saliency. The crucial question was whether the simulated refixation saccades would show a similar pattern as the observed refixations toward objects, for which the action-performing bias was maximal. As can be seen from Figure 5b, this was not the case. Whereas participants tended to refixate the object's action-performing side, simulated refixations did not show this bias. More precisely, irrespective of stimulus type, the average landing position of simulated initial saccades was near the CoG. Later-triggered refixations were slightly biased toward the handle side. This discrepancy between simulated and observed saccades strongly suggests that participants' refixations and long-latency initial saccades were not driven by visual saliency. Furthermore, the fact that simulated saccades were very similar for objects and nonobjects, suggests that our matching procedure was successful. 

Discussion

Experiment 2 revealed that early initial saccades were directed toward the object's CoG whereas later initial saccades as well as refixations were directed toward the object's action-performing part. The early CoG tendency was not due to a bimodal landing-position distribution. Furthermore, we compared eye-movement behavior toward real objects with eye-movement behavior toward visually matched nonobjects and simulated saliency-driven saccades (Itti et al., 1998). These comparisons revealed that the later-occurring action-performing bias was not merely the consequence of a low-level effect of shape, texture, asymmetry, or saliency and thus likely reflects a high-level object-based effect. 

As in Experiment 1, the time that elapsed since stimulus onset could not entirely account for participants' landing positions: For initial saccades and refixations that were initiated with comparable latencies (i.e., between 270 and 290 ms after stimulus onset), the bias was stronger for refixations than for initial saccades (i.e., the curves of the refixations laid below the curves of the initial saccades, thus indicating that the eyes landed farther away from the reference point). We briefly discuss this interesting additional finding in the General discussion. 

General discussion

The current study investigated to what extent low-level versus high-level effects determine where the eyes land on isolated daily-life objects. We operationalized low-level effects as saccades toward the object's CoG or absolute center and high-level effects as visuomotor priming by object affordances. We found that early initial saccades landed toward the CoG/OC. When saccade latency increased, or when a refixation was made, we observed a systematic bias toward the action-performing side of the object. As revealed in Experiment 2, when visually matched nonobjects were also presented, this was not merely the consequence of a low-level effect of shape, texture, asymmetry, or saliency and thus likely reflects a high-level object-based effect. 

A low-level CoG/OC effect

Previous research showed that eye movements toward single simple shapes land near the stimulus' CoG (He & Kowler, 1991; Kaufman & Richards, 1969; Kowler & Blaser, 1995; Richards & Kaufman, 1969). Furthermore, several studies have shown that eye movements toward isolated objects (Foulsham & Underwood, 2009; Henderson, 1993) or objects in scenes (Foulsham & Kingstone, 2013; Nuthmann & Henderson, 2010; Pajak & Nuthmann, 2013), land near the OC. For real-world objects, CoG and OC are typically very close. Therefore, we did not aim at distinguishing between the two but rather to investigate whether this general CoG/OC tendency would prevail in determining where the eyes move. We predicted that the CoG/OC tendency would occur early in time and that it would later be overridden by higher-level factors. In line with this prediction, we found that participants' early initial saccades landed near the CoG/OC of peripherally presented stimuli regardless of whether or not these were real objects. 

This finding is an important complement to the literature as it reveals that the tendency to move the eyes toward an object's CoG/center is unrelated to its semantic content. This might be the result of default, universal mechanisms associated with saccade programming (i.e., saccadic averaging, see Vitu, 2008). Alternatively, it could be that our results reflect visuomotor strategies that aim at bringing the eyes to a position that is optimal for the task (i.e., decide whether the stimulus is a kitchen vs. garage tool or an object vs. nonobject, in Experiments 1 and 2, respectively). 

High-level object-affordance effects

Several studies suggested that visuomotor priming biases visuospatial attention. Intriguingly, however, they were equivocal with regard to the direction of this bias. Whereas Myachykov and colleagues (2013) found that the eyes were automatically drawn toward an object's graspable part (i.e., the handle of a teapot), Roberts and Humphreys (2011) found an attentional shift in the direction of the action that is implied by the object (i.e., toward the pouring part of a teapot). The current results tip the balance in favor of the action-performing hypothesis. We found that when time since stimulus onset elapsed, and most particularly when a refixation was executed, participants' eyes were biased toward the object's action-performing side. Experiment 2 revealed that this was not merely the consequence of a low-level effect of shape, texture, asymmetry, or saliency as the same was not true for refixations within nonobjects; these remained near the CoG/center of the stimulus. Importantly, this action-performing bias takes time to build up. Whereas CoG/OC effects intervened early, the action-performing bias of initial saccades increased over time. Refixations showed the same bias to an even larger extent. 

Time course

The time course that we observed in the present study is consistent with studies on the global effect, which have shown that the contribution of low-level, default mechanisms dissipates over time, thereby making room for more accurate and goal-directed saccades (Coëffé & O'Regan, 1987; Vitu et al., 2006). Similarly, studies on visual search in textured background or the viewing of natural scenes have revealed that early saccades are pulled toward the most visually salient items/regions, and longer-latency saccades and saccades occurring later during scene-viewing time are more likely goal-directed (Siebold & Donk, 2014; Van Zoest, Donk, & Theeuwes, 2004) and influenced by high-level processes associated with object-based semantic processes (De Graef, Christiaens, & d' Ydewalle, 1990; Henderson et al., 1999; Parkhurst, Law, & Niebur, 2002; Van Zoest et al., 2004, but see also Loftus & Mackworth, 1978). As was already envisioned by Pajak and Nuthmann (2013), our findings show that object affordances, just like object semantics, modulate where the eyes move when time elapses since stimulus onset. 

Nevertheless, the time that elapsed since stimulus onset does not fully explain the difference in landing positions between initial saccades and subsequent refixations because, in both experiments, the very latest initial saccades still showed a much smaller action-performing bias than the very earliest refixations. The remaining difference could possibly be explained by the fact that initial saccades were made toward a peripherally presented stimulus (in order to foveate it) whereas refixations were made within an already-foveated stimulus. Probably, it is easier to determine which part of the object is the optimal saccade-target location and to guide the eyes accordingly when the object is already in (para)foveal vision as compared to when it is still in peripheral vision. 

The need to control low-level properties to study object-affordance effects

Importantly, in contrast to what was found by Myachykov and colleagues (2013), in our study, participants did not preferentially look at the object's handle at any point in time. The discrepancy between their and our results is best explained by the different analyses conducted: Whereas we focused on saccadic landing positions, Myachykov and colleagues measured “proportional dwell time.” This was calculated as the total time the eyes remained on a given area of interest (i.e., the handle versus the “body” of the object) divided by the size of the area in pixels. Their results showed that participants spent proportionally more time looking at objects' handles as compared to objects' bodies. However, we believe that using proportional dwell times as a dependent measure is only sound when the object's low-level properties are taken into account. Without doing so, analyses such as the one carried out by Myachykov and colleagues may lead to the reported pattern even when handles and bodies were actually fixated to the same extent. For example, in their stimulus set, the bodies of the object might have contained more pixels than the handles (i.e., the CoG and the OC may not have coincided). Consequently, when participants gazed, for example, 500 ms on an object's body, containing 100 pixels, and another 500 ms on the handle, containing only 10 pixels, proportional dwell time was longer on the latter than on the former area of interest. Therefore, it remains unclear whether attention was really automatically captured by the handles in the study by Myachykov and colleagues. 

The discrepancy of our current results with some previous findings emphasizes how important it is to take a stimulus' low-level features (e.g., CoG, saliency, overall shape, texture) into account. For example, in Myachykov et al.'s (2013) study, eye movements toward real objects could have been compared with eye movements toward nonobjects of which the low-level features were matched as in Experiment 2 of the present paper. 

We believe that studies using real objects as stimuli should convincingly show that a potential higher-level effect (e.g., an affordance effect) is not likely to be explained by the low-level features of the stimuli. Such care should not only be taken when measuring bottom-up-driven oculomotor behavior, but also when measuring other cognitive processes, such as attentional capture by object affordances. Future studies could manipulate a variety of factors, ranging from low-level (e.g., saliency and CoG but also, for example, the availability or visibility of stimulus contours, e.g., Massendari, Tandonnet, & Vitu, 2014) to high-level (e.g., affordances, or semantics, in a visual scene) object properties. Doing so may help to better understand eye guidance in simple visual displays as well as eye guidance during natural scene viewing. 

Conclusions

We investigated to what extent low-level CoG/OC effects versus high-level object-affordance effects determine where the eyes land on isolated daily-life objects. We found that when the programming time of initial saccades was short, the eyes were drawn toward the CoG or absolute center of the object. When saccade latencies increased, and even more so when a refixation was executed, the eyes started to deviate from the CoG/OC and showed a systematic gaze bias toward the object's action-performing part. In line with previous studies (cf. e.g., Henderson et al., 1999; Parkhurst et al., 2002; Van Zoest et al., 2004), we conclude that low-level CoG/OC effects occur early whereas higher-level, object-related effects take time to build up and become more likely when the object is foveated. 

Acknowledgments

L. van der Linden was supported by a grant (“allocation de recherche”) from the French Ministry of Research (2012–2015), and S. Mathôt was supported by ERC grant 230313 attributed to Jonathan Grainger, and Marie Curie Action 622738. We thank Bohdana Zakova for collecting the data of Experiment 2. Experiment 1 was presented at the 2013 European Conference on Eye Movements (ECEM, 2013) in Lund, Sweden. 

Commercial relationships: none. 

Corresponding author: Lotje van der Linden. 

Email: [e-mail]. 

Address: Laboratoire de Psychologie Cognitive (LPC), CNRS, Aix-Marseille Université, UMR 7290, Marseille, France. 

Footnotes

¹  In the current study, we conceptually defined CoG as an index of the object's asymmetry in terms of local contrast. For a technical definition, see Methods and Appendix A.

Footnotes

²  Importantly, we did this at a trial-by-trial level rather than at the condition-mean level. The reasons for doing so were that (a) it allowed us to carry out LME analyses (which are by definition on a trial level), and (b) it appears more legitimate because, after having filtered our data (see Data analysis), not all objects appeared equally often in our data set anymore. Thus, attributing a given object's CoG as much “weight” as the CoG of another object, even if it appears in the data set less often, appeared suboptimal.

Footnotes

³  We do not show the time course separately for landing positions relative to the CoG because the resulting plot was very similar to the time course relative to the OC as shown in Figure 3c.

Footnotes

⁴  We used our real object stimuli as input for the algorithm that generated our nonobject stimuli. Because the real objects have an action-related side and because the nonobjects were generated from these real objects, they obtained a “pseudo-action-performing side.” For example, the shape of the pseudo-action-performing side of a given nonobject could be the combination of the shape of the head of a washing brush and the head of a fork (the exact procedure is explained in the Method section). The fact that the nonobjects obtained a pseudo-action-performing side is important because it allowed us to directly compare normalized landing positions toward real objects with landing positions toward nonobjects.

Footnotes

⁵  We thank G. Zelinsky for introducing this algorithm to us.

Appendix A: CoG calculation

To calculate the CoG per object, we first estimated the “edginess” of the object by applying a Sobel operation to the three-dimensional pixel array (x, y, RGB channel) of the bounding box around the stimulus. This resulted in a new three-dimensional array containing Sobel operators instead of RGB values for each color channel (see Figure A1, upper part). Next, we calculated the weighted average of this array on the horizontal axis of the stimulus and used this weighted average as the CoG (see Figure A1, lower part). 

Figure A1.

 

Example of our CoG calculation for a simplified stimulus consisting of only 5 × 5 black or white pixels. The Sobel operator gives an indication of how much a pixel differs from all its “neighbor” pixels in terms of contrast (see calculations in blue). Because, in the current example, the outer pixels do not have neighboring pixels on all sides, for the current example, we only calculate the Sobel operators of the nine inner pixels. “A” depicts the input array where 0 = black and 1 = white. “G” depicts the array containing Sobel operators. We took the weighted average (see calculations in orange and green) of these Sobel operators as the CoG on the x-axis of the stimulus.

Our CoG calculation ensured that parts of the object where local contrast was high were weighted more heavily than parts of the object where contrast was low. Examples of high-contrast parts are the object's borders (where contrast with the white background is typically high) and “rough” surfaces within the object, such as the hairs of a paintbrush. Examples of low-contrast parts are smoothed, continuous surfaces, such as the blade of a knife. In sum, our CoG indicates the asymmetry of our stimuli in terms of local contrast. 

Appendix B: Saliency maps

We used the NeuroMorphic vision toolkit developed by Itti et al. (1998) to simulate saliency-driven eye movements for all trial displays with the following command: 

ezvision --in=[input image]-T --output-frames=0-4@EVENT --out=png --textlog=[output log] -+ 

In a nutshell, this algorithm first generates a saliency map for the input image. Next, simulated eye movements are determined based on the peak of local contrast of the saliency map combined with a simple inhibition-of-return mechanism. The latter avoids that all simulated eye movements are generated toward the same location (i.e., the location where saliency was highest). Instead, once fixated, the just-fixated location gets temporarily inhibited, such that subsequent saccades are directed elsewhere (i.e., toward the next most salient location).