Free
Research Article  |   January 2009
Differences in fixations between grasping and viewing objects
Author Affiliations
Journal of Vision January 2009, Vol.9, 18. doi:10.1167/9.1.18
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Anne-Marie Brouwer, Volker H. Franz, Karl R. Gegenfurtner; Differences in fixations between grasping and viewing objects. Journal of Vision 2009;9(1):18. doi: 10.1167/9.1.18.

      Download citation file:


      © 2015 Association for Research in Vision and Ophthalmology.

      ×
  • Supplements
Abstract

Where exactly do people look when they grasp an object? An object is usually contacted at two locations, whereas the gaze can only be at one location at the time. We investigated participants' fixation locations when they grasp objects with the contact positions of both index finger and thumb being visible and compared these to fixation locations when they only viewed the objects. Participants grasped with the index finger at the top and the thumb at the bottom of a flat shape. The main difference between grasping and viewing was that after a saccade roughly directed to the object's center of gravity, participants saccaded more upward and more into the direction of a region that was difficult to contact during grasping. A control experiment indicated that it was not the upper part of the shape that attracted fixation, while the results were consistent with an attraction by the index finger. Participants did not try to fixate both contact locations. Fixations were closer to the object's center of gravity in the viewing than in the grasping task. In conclusion, participants adapt their eye movements to the need of the task, such as acquiring information about regions with high required contact precision in grasping, even with small (graspable) objects. We suggest that in grasping, the main function of fixations is to acquire visual feedback of the approaching digits.

Introduction
In the present study, we investigate where exactly people look when performing the everyday act of grasping an object, as opposed to when they are only asked to view the object. While the factors affecting eye movements to objects without grasping are relatively well understood, as are the factors affecting eye movements when pointing 1 to an object, surprisingly little is known about eye movements when grasping an object. In contrast to pointing, there are at least two potential fixation targets in grasping, being the two locations on the object that are contacted by the thumb and index finger. Eye movements could either be attracted to both contact locations successively, or to a compromise position between the two. The eye movements could also be dominated by the index finger, which probably requires relatively precise visual monitoring, or the thumb, which is argued to play an important role in guiding the hand to the object. Finally, properties of the contact locations (e.g., their size) could affect fixations during grasping. Before describing our approach to this problem, we will give a short overview of the relevant literature. 
Effects of visual features on fixation locations
When observers are presented with images of real life situations or artificial stimuli, the regions that they fixate can be predicted to a certain extent by the image's visual content. Features that have been demonstrated to attract fixations are high contrast and high variance (Reinagel & Zador, 1999), distinctive higher-order statistics (Krieger, Rentschler, Hauske, Schill, & Zetzsche, 2000) and high local symmetry (Privitera & Stark, 2000). When observers are asked to look at a spatially extended shape, their gaze typically ends on or near the shape's center of gravity (COG, i.e., the average location of the pixels constituting the shape; He & Kowler, 1991; Kowler & Blaser, 1995; McGowan, Kowler, Sharma, & Chubb, 1998; Melcher & Kowler, 1999; Vishwanath & Kowler, 2003, 2004; Vishwanath, Kowler, & Feldman, 2000). Vishwanath and Kowler (2003) showed that this was especially the case for a visual scanning task, in which participants fixated a number of objects sequentially. Cues to the three-dimensional structure of the shape can influence the gaze location by drawing it to the three dimensional COG (Vishwanath & Kowler, 2004). Salient local features, such as the boundary between two bars constituting an L-shape, have also been shown to attract fixations when observers were asked to make a single saccade to the object or when they were given unlimited time to find the fixation location where they felt to be looking at the object as a whole (Vishwanath & Kowler, 2003). Itti and Koch ( 2000) developed a stimulus driven computational model that predicts human gaze behavior in a search task. 
Effects of task on fixation locations
Obviously, visual features are not the sole determinants of fixation locations. In a famous study, Yarbus (1967) showed that participants fixate different regions of the same picture depending on the question they were asked about the content of the picture. The kind, and thus the location of the most useful information, depends on the task that is performed, which in turn will affect the looking behavior. This is not only true for visual judgment tasks, but also for tasks in which people interact with their environment. Studies about hitting balls in cricket (Land & McLeod, 2000), steering (Land & Tatler, 2001), copying block patterns (Aivar, Hayhoe, Chizk, & Mruczek, 2005; Ballard, Hayhoe, Li, & Whitehead, 1992; Smeets, Hayhoe, & Ballard, 1996), making tea and making peanut butter and jelly sandwiches (Hayhoe, 2000; Land & Hayhoe, 2001; Land, Mennie, & Rusted, 1999) all reflect an anticipation of actions or events rather than simple responses to visual stimuli. For example, batsmen in cricket fixate the place where they expect the ball to bounce (Land & McLeod, 2000). In the studies by Hayhoe and co-workers, participants fixated the jar of peanut butter or the next block they needed in order to copy a particular block pattern, before grasping it. 
Fixation locations during pointing
For pointing tasks, the relation between gaze location and the location that is contacted with the index finger or a pen has been studied in detail. 
Clearly, if participants are not looking at the target, their pointing errors increase (Bekkering, Adam, van den Aarssen, Kingma, & Whiting, 1995; Bock, 1986; Enright, 1995; Henriques, Klier, Smith, Lowy, & Crawford, 1998; Medendorp & Crawford, 2002; Neggers & Bekkering, 1999; Vercher, Magenes, Prablanc, & Gauthier, 1994), demonstrating that at least an approximate match between gazing and pointing location is required for good performance. 
Several findings suggest a close correspondence between gaze and pointing location. Neggers and Bekkering (2000, 2001, 2002) found that the eyes are ‘locked’ to the target until the pointing movement is nearly complete, despite the instruction to saccade to a second fixation target earlier than that. When the target shifts location during the pointing movement, participants naturally move their eyes and hand to the new location. Neggers and Bekkering (2002) conclude that ocular gaze is forced to go to the manual target. Consistent with this, Frens and Erkelens (1991) found that for speeded responses to targets that are presented with a delay, errors were practically always in the same direction for the gaze as for the hand. In a display that altered the perceived direction and thus the perceived extrapolated location of a moving target, gaze and pointing errors were also demonstrated to be correlated (Soechting, Engel, & Flanders, 2001). Admiraal, Keijsers, and Gielen ( 2003) report that in pointing toward remembered targets, the variability in gaze and pointing positions covary. Different potential causes for a correspondence between gaze and pointing location are that the gaze defines the target for pointing movements, that the hand and the eyes are driven by the same information, or that a common motor command is used for both systems. 
However, other findings suggest a looser correspondence between gaze and pointing location. In another condition in the experiment by Frens and Erkelens (1991), in which the speeded response had to be made toward targets presented without a delay, gaze always went into the correct direction whereas the hand did not. A set of experiments about gazing and pointing at the (modified) Müller-Lyer illusion that influences the perceived distance between two locations, shows a spatial independence between gaze and hand under some conditions (Binsted, Chua, Helsen, & Elliott, 2001; Binsted & Elliott, 1999; de Grave, Brenner, & Smeets, 2004; de Grave, Franz, & Gegenfurtner, 2006; de Grave, Smeets, & Brenner, 2006; Mack, Heuer, Villardi, & Chambers, 1985). These discrepancies have been explained by the use of different information by the eyes and the hand, and different strategies or possibilities for adjusting the movement of the eyes and hand online. 
In summary, different degrees of correspondence between gaze and pointing location can be observed under different circumstances. However, it is clear that participants prefer to look at the approximate location at which they are pointing. 
Fixation locations during grasping
In pointing, it may not come as a surprise that gaze is generally directed at the location that is contacted by the hand. There is only one contact location. However, during grasping there are generally two contact positions whereas the gaze can only be at one location at the time. A close correspondence between gaze and contact location as demonstrated by Neggers and Bekkering (2000, 2001, 2002) would not be possible for both contact locations. 
Riek, Tresilian, Mon-Williams, Coppard, and Carson ( 2003) investigated eye movements in a situation that was not grasping, but also involved two contact locations. They asked their participants to point at two targets at the same time, using both hands. Participants appeared to direct their index fingers to a location above the targets, looking at both targets sequentially, and then move the index fingers down simultaneously. The last fixations were usually directed to the left target, or to the smaller of the two targets. The authors suggest that the reason for this is that the left hand and a movement to a small target needed more visual monitoring in (right handed) participants. It remains to be seen whether these findings can be generalized to grasping. Obvious differences are that though in grasping there are also two contact locations, only one hand is guided to the objects. Furthermore, in grasping different digits (always including the thumb) rather than two index fingers are involved. Finally, in grasping, the digits do generally not hover above the contact locations before contact is made, suggesting that a different strategy is used. 
Though it is known that objects are usually fixated before they are grasped, it has never been investigated in detail which locations are fixated in the case that both contact locations are visible. Johansson, Westling, Bäckström, and Flanagan (2001) have thoroughly studied eye movements during grasping with only one visible contact location. They asked their participants to grasp a bar and press it to a target switch, while (in some conditions) avoiding obstacles. The bar was presented just below eye height and participants grasped in such a way that only the contact position of the thumb with the bar was visible, while the contact position of the index finger was at the backside of the bar. Johansson et al. (2001) concluded that people always gaze at the positions with which they want to make contact, which in their study were the grasp location on the bar, the target switch that had to be contacted with the bar and a support surface on which the bar had to be placed back. Regarding the grasp location, Johansson et al. ( 2001) demonstrated a correlation between the (horizontal components of the) grasp site and the fixation location. 
In the present study, we want to see where participants fixate during grasping if they can see both contact locations, and compare these fixations to those made when participants are only looking. 
Thumb or index finger—functions of fixation
It has been proposed that in precision grasping, the thumb and index finger play functionally different roles. More precisely, the thumb is proposed to be guiding the hand to the object, whereas the index finger is mainly responsible for the closing or grasping component of the movement, indicated by a straighter, less variable path of the thumb than the index finger (Galea, Castiello, & Dalwood, 2001; Haggard & Wing, 1997; Wing & Fraser, 1983). From this point of view, the most important function of fixations during grasping will determine whether the eyes are attracted to the contact location of the index finger or the thumb. 
One function of a fixation on the target is to provide visual feedback about the approaching hand to enable online corrections (Binsted et al., 2001; Helsen, Elliott, Starkes, & Ricker, 1998; Lünenburger, Kutz, & Hoffmann, 2000; Riek et al., 2003). If this is the main function of fixations during grasping, we expect fixations to be attracted by the contact location of the index finger, as this finger describes a more variable trajectory and thus requires more visual feedback in order to guide it to its contact location. 
Another function of fixating a target prior to contacting it with the hand, is to provide information about where the contact location is in space to the arm's motor system (Land & Hayhoe, 2001; Soechting et al., 2001). This function becomes apparent by ‘look ahead fixations’ to future targets and obstacles (Hayhoe, Shrivastava, Mruczek, & Pelz, 2003; Johansson et al., 2001; Land et al., 1999; Pelz, Hayhoe, & Loeber, 2001). Extra-retinal information such as efference copies and proprioception of the eyes is proposed to play an important role in localizing a target in space so that an appropriate hand movement to the contact location can be made (Abrams, Meyer, & Kornblum, 1990; Helsen et al., 1998; Pélisson, Prablanc, Goodale, & Jeannerod, 1986). Several experimental findings stress that fixations provide the hand with information about the contact location, rather than stressing the function of providing visual feedback of the approaching hand. For example, the locking of gaze on the target until the pointing movement as found by Neggers and Bekkering (2001) was independent of vision of the hand. Pélisson et al. (1986) showed that despite the absence of visual information about the effector, online corrections to reach a displaced target can be made. In the study by Abrams et al. (1990), wrist movements to a target suffered if participants were not allowed to look at the target, even when the effector was not visible. If the main function of fixations during grasping is to determine the contact location in space, we expect an attraction by the contact location of the thumb, as this is the guiding digit. Consistent with this, Schiegg, Deubel, and Schneider ( 2003) found that in a grasping task without vision of the hand, attention was drawn to the to be grasped branches of an x-shaped object, and especially to the contact location of the thumb. 
Experiment 1
In Experiment 1, we asked our participants to grasp three different flat shapes presented at different orientations in the frontal plane. All shapes were of uniform density and thickness. Besides asking participants to grasp the shapes, we also asked participants to just look at them in order to directly compare eye movements made during grasping to eye movements made to the same shapes when grasping was not required. 
In this first study comparing eye movements toward the same shapes during grasping versus viewing, we chose a shape for which we thought it likely to find a difference (a triangle), a shape to investigate the well established attraction of eye movements by the COG in viewing tasks in the task of grasping (an asymmetric cross), and finally, a symmetric baseline shape (a square). 
The triangle's contact positions, which are its point and its base, differ in size and thus in required contacting accuracy: one has to be more precise in guiding a digit to the triangle's point than to its base. In viewing the triangle's point and base do not have different implications. We will investigate whether this difference between grasping and viewing will be reflected in the gazing behavior. 
The COG is clearly off the center in the asymmetric cross (with the center defined as the average location of the shape's opposite sides in the horizontal and vertical direction; see the small ‘x’ in each shape in Figure 6), which means that the COG's location depends on the orientation of the cross. Whereas we know that the COG is the preferred fixation location for viewing, other parts, such as the digits' contact locations, may be more relevant for grasping. We will investigate whether and if so, to what extent, the COG attracts fixations in grasping as well. 2 
The square serves as a baseline, having no special salient features but a clearly located COG and flat, equally sized grasping locations. 
Methods
Participants
Ten different participants carried out both viewing and grasping tasks. In the grasping task, one of the participants was one of the authors, the rest were students and colleagues from the Justus-Liebig University, naïve to the purpose of the experiment. Half of the participants in each task were women and half were men. Their ages ranged between 21 and 38 years. All participants were self-declared right-handed. 
Stimuli and design
The shapes that the participants were asked to grasp or view were mounted on a Plexiglas frame, which was placed in front of a monitor ( Figure 1A). The shapes were stuck to the frame by two protrusions on the backside of the shape that fitted around the frame's bar so that they could only be removed by a movement perpendicularly away from the frame. The shapes were pushed against a small stop on the left side of the frame, ensuring that they were always presented at the same position. We used three different black plastic objects, depicted in Figure 1B: a square, a triangle (presented in 2 different orientations) and a cross (presented in 4 different orientations). They were 3 mm thick. 
Figure 1
 
(A) Schematic depiction of the setup and the way of grasping in Experiment 1. In the actual experiment, there was no fixation point present during grasping. (B) The shapes that were grasped in Experiment 1: a square, a triangle (respectively ‘upward pointing’ and ‘downward pointing’), and a cross (respectively ‘upward pointing’, ‘rightward pointing’, ‘downward pointing,’ and ‘leftward pointing’). The ellipses and circles indicate the regions where the participants were allowed to contact the objects.
Figure 1
 
(A) Schematic depiction of the setup and the way of grasping in Experiment 1. In the actual experiment, there was no fixation point present during grasping. (B) The shapes that were grasped in Experiment 1: a square, a triangle (respectively ‘upward pointing’ and ‘downward pointing’), and a cross (respectively ‘upward pointing’, ‘rightward pointing’, ‘downward pointing,’ and ‘leftward pointing’). The ellipses and circles indicate the regions where the participants were allowed to contact the objects.
For the grasping task, participants performed 7 shapes * 3 repetitions = 21 practice trials, presented in random order. The actual grasping experiment consisted of six such blocks, resulting in 126 trials for each participant. Due to computer problems in the middle of an experimental block, one participant performed some extra trials so that the total number of recorded trials was 1269. 
For the viewing task, participants practiced only a few trials. As with grasping, each participant was presented with six experimental blocks of 7 shapes * 3 repetitions in random order. The total number of recorded trials was 1260. 
Apparatus
Participants rested their head on a chinrest, 45 cm in front of the monitor. The eyes were approximately at the same height as the horizontal bar of the Plexiglas frame. Eye movements were recorded with a head-mounted, video-based eye tracker (Eyelink II, SR-Research, Osgoode, Ontario, Canada) measuring at 250 Hz. The apparatus was calibrated by the fixation of nine predefined dots, sequentially presented on the monitor. Both eyes were recorded during calibration but only the eye with the least error was recorded during the experiment. After the participants performed half of the total amount of trials, they could rest and the calibration was repeated. The calibration was also repeated when a participant left the chin rest for another reason. At the start of each trial, the participants fixated a dot and pressed a button to indicate that they were fixating. This triggered the eye and hand movement systems to start recording. The fixation point was 11 cm (13.2 deg) to the right of the center of the screen. It was 18 cm (21.6 deg) to the right of the center of the square and the center of the triangle, and 15.5 cm (18.6 deg) to the right of the center of the cross. The dot disappeared after the Eyelink system had performed a drift correction, if necessary. 
For the grasping task, the movement of the hand was recorded with an ultrasonic 3-D tracking device (Zebris, Zebris Medical GmbH, Isny, Germany) measuring at 100 Hz. This system records the location over time of small sound emitting speakers, attached to the nail of the index finger and the side of the thumb. The thin, flexible cables protruding from the small speakers were attached with tape to the participant's hand and arm so that they were not obstructing the movement. 
Procedure
For the grasping task, participants started a trial with the eyes closed and the hands resting in front of them on a button pad. When a beep sounded they opened their eyes and immediately fixated the fixation point that was always presented at the same location to the right of the shape. When they were fixating, they pressed a button. The disappearance of the fixation point was the signal for the participants to reach out and grasp the shape using the thumb and index finger of the right hand. We instructed the participants to grasp the shapes with the index finger on the upper and the thumb on the lower part of the shape (depicted by the ellipses and circles in Figure 1B). If the participants started to move their eyes or hand before the fixation dot disappeared, a warning message appeared and the trial was repeated later. After grasping, they gave the shapes to the experimenter and returned to the starting position. Eye movements were only analyzed from the moment that the fixation dot disappeared until the first of the two digits reached the object. The participants were not instructed regarding eye movements. 
The procedure of the viewing task was the same as of the grasping task, with the exception that instead of being asked to grasp, participants were asked to ‘look at the shape’ after the initial fixation dot disappeared until the moment that a beep sounded, upon which they closed their eyes. The beep was presented 1300 ms after the fixation dot disappeared because in the grasping task, the time between the disappearance of the fixation dot and the moment that the first digit reached the object was approximately 1300 ms. In this way, the time interval over which eye movements were analyzed was approximately the same. 
Analysis
80 of the 1269 grasping trials (6.3%) and 31 of the 1260 viewing trials (2.5%) were rejected due to insufficient quality of eye or hand data. Missing frames of the digit and eye positions were linearly interpolated. In order to smooth irregularities caused by random variations in the signal, each position of each of the digits was averaged over 5 Zebris frames (corresponding to 50 ms). 
For each trial of the grasping task, the fixation locations were determined from the disappearance of the fixation dot until the first of the two digits touched the object. We considered the digit as starting to touch the object when its velocity dropped below a threshold of 3 cm/s above the minimum velocity, determined after the hand reached its maximum velocity in the direction toward the screen. The same velocity threshold was used to determine when the digits started to move to the object. The digits’ movement times were computed from these starting and arrival times. For determining the time that the digits started to move relative to the time that the participant started to fixate the object, we needed to remove extra trials in which the start of the hand's movement was not well recorded as indicated by impossible values of marker location. This was the case for 49 trials (4% of the total number of recorded trials). We used the same set of data for analyzing the end of the digits' movements relative to the time of fixation. 
For the viewing task, we analyzed the fixations made until 1300 ms after the disappearance of the fixation dot (i.e., the time that the participants were signaled to close their eyes). 
A fixation was the interval between saccades. A saccade was defined by subsequent frames in which the eye covered at least 0.6 deg and moved faster than 47.9 deg/s (if there were 4 or less subsequent frames in which the eye moved slower, they were included in the saccade). The start and the end of a saccade (i.e., the end and the start of a fixation) were defined by the first and the last frame of this collection of frames. The first fixation of each trial, which was the fixation on the fixation dot, was discarded. From now on, with 'first fixation' we mean the first fixation after the fixation on the initial fixation dot. 
The coordinates of the fixations were expressed relative to the center of the shape (as indicated by the small ‘x’ in each shape in Figure 6) with negative horizontal values representing values to the left of the center and negative vertical values representing values below the center. 
In order to obtain one value for every trial describing the general location where participants look at, we computed the average location of the fixations made in each trial. Besides the average fixation locations, we investigated the location of the first and the second fixation for each trial as well as the horizontal and vertical difference between them (that is, the horizontal and vertical component of the second saccade in the trial). For grasping trials with more than one fixation, we computed the maximal absolute vertical distance between any two fixations in that trial. If participants looked at both contact locations within a trial, this value would be close to 7.8 cm (the actual vertical distance between the contact locations of the triangles) or 9 cm (the actual vertical distance between the contact locations of all other shapes). If participants fixate one of the contact locations more often than the other one, the maximal absolute vertical distance would still indicate that both locations were being fixated. In addition to these variables describing fixation location, we will describe the general characteristics of the data by reporting on a) when the digits started to move and arrive at the object relative to the time that the participant started fixating, b) the number of fixations, and c) when fixations started relative to the disappearance of the fixation dot. We will also have a closer look at trials in which only one fixation was made. 
We compare the results of the grasping and viewing tasks mainly by using repeated measures ANOVAs, with shape as within-subjects variable and task as between-subjects variable (if we use another test, this is explicitly mentioned). For almost all of these ANOVAs, there turned out to be a main effect of shape—we focus on the effects of task and only mention the absence of an effect of shape. For all statistical tests, we used 0.05 as the level of significance. 
Results
Eye–hand timing in the grasping task
Figure 2 shows the trajectories of eye, index finger and thumb in an example grasping trial. Fixations are indicated by crosses, the average of the fixations in that trial by a star. In all analyzed trials, participants made at least one saccade in the direction of the shape before grasping it, although the objects were clearly visible in the periphery when fixating the fixation point. 
Figure 2
 
Example trial of the trajectories of eye (gray), thumb and index finger (black) when grasping an upward pointing triangle. Fixations are depicted as crosses, the average fixation in this trial as a star. The circles indicate the positions of the markers that were attached to the digits at the moment that the object was reached.
Figure 2
 
Example trial of the trajectories of eye (gray), thumb and index finger (black) when grasping an upward pointing triangle. Fixations are depicted as crosses, the average fixation in this trial as a star. The circles indicate the positions of the markers that were attached to the digits at the moment that the object was reached.
The gray symbols in Figure 3A show when the index finger and thumb start to move relative to the start of the first fixation. On average, the thumb starts to move 49 ms (standard error of the mean or SEM = 30) before the eye arrives, and the index finger 62 ms ( SEM = 28). A repeated measures ANOVA with digit (thumb or index finger) and shape as within-subjects variables showed that the relative starting time of the hand depended on shape ( F (6,54) = 3.02, p = 0.01). The starting times of index finger and thumb were not significantly different from each other ( F (1,9) = 0.99, p = 0.35) and there was no interaction between digit and shape ( F (6,54) = 0.90, p = 0.50). The black symbols in Figure 3A indicate when the index finger and thumb arrive at the shape relative to the start of the first fixation. On average, the thumb arrived 944 ms ( SEM = 61) after the first fixation started and the index finger 860 ms ( SEM = 61). The relative arrival time of the hand depended on shape ( F (6,54) = 2.71, p = 0.02) and on digit ( F (1,9) = 48.61, p < 0.01), with the index finger arriving before the thumb. There was also an interaction between digit and shape ( F (6,54) = 2.35, p = 0.04). Clearly, the hand arrived at the object long after the participant made their first fixation. It also arrived considerably long after the last fixation started: for trials in which there were more than one fixation, this was on average 434 ms ( SEM = 31) for the thumb and 346 ms ( SEM = 31) for the index finger (data not shown). 
Figure 3
 
Eye–hand timing and general fixation characteristics in Experiment 1. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in the grasping task of Experiment 1, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the grasping (solid circles) and the viewing (empty squares) task. (C) Average number of fixations per trial for each shape, separately for the grasping (solid circles) and the viewing (empty squares) task. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape. For trials with only one fixation (data points to the far right), the data is collapsed over shapes. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data for grasping, empty symbols represent the data for the viewing task. The error bars in all panels are standard errors of the mean. Note that the standard error of the mean reflects within- as well as between-subjects variance and therefore should not be interpreted as being indicative for significance in our repeated measure design.
Figure 3
 
Eye–hand timing and general fixation characteristics in Experiment 1. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in the grasping task of Experiment 1, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the grasping (solid circles) and the viewing (empty squares) task. (C) Average number of fixations per trial for each shape, separately for the grasping (solid circles) and the viewing (empty squares) task. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape. For trials with only one fixation (data points to the far right), the data is collapsed over shapes. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data for grasping, empty symbols represent the data for the viewing task. The error bars in all panels are standard errors of the mean. Note that the standard error of the mean reflects within- as well as between-subjects variance and therefore should not be interpreted as being indicative for significance in our repeated measure design.
General fixation characteristics
The solid symbols in Figure 3B show the proportion of trials in which a certain number of fixations (1 up to 5) were made when performing the grasping task; the empty symbols refer to the viewing task. In Figure 3C, the average number of fixations per trial is shown for each shape and task separately. On average, there were 2.39 ( SEM = 0.17) fixations per grasping trial and 2.41 ( SEM = 0.21) fixations per viewing trial. There was neither a main effect of task on the number of fixations per trial ( F (1,18) < 0.01, p = 0.96), nor did shape and task interact ( F (6,108) = 1.42, p = 0.22). 
Figure 3B shows when the first (gray symbols) and the second (black symbols) fixations started relative to the disappearing of the initial fixation dot, separately for each shape and task. The gray data points to the far right are the average starting times of the first fixation for grasping and viewing trials in which only one fixation was made. These are collapsed over shape because of the low number of trials with only one fixation. The first fixation started later in the grasping task than in the viewing task (on average 73 ms, main effect task: F (1,18) = 6.12, p = 0.02). This was more strongly so for some shapes than others (interaction between shape and task: F (6,108) = 2.97, p = 0.01). There was no effect of task and no interaction with shape on the start time of the second fixation (respectively F (1,18) = 2.69, p = 0.12 and F (6,108) = 1.92, p = 0.08). The start times of fixations in trials with only one fixation did not differ between the tasks (independent sample t-test: t 18 = 0.56, p = 0.58). We compared the start times of single fixations to the start times of first fixations for trials with more than one fixation, collapsed over shape, by using a repeated measures ANOVA with task as between-subjects factor and ‘single or more fixations’ as within-subjects factor. There was no effect of task ( F (1,18) = 1.36, p = 0.26) and no interaction between task and single or multi fixation trials ( F (1,18) = 0.21, p = 0.65). However, the first fixation started significantly later in single fixation trials compared to trials with more than one fixation ( F (1,18) = 5.49, p = 0.03). 
Linear regressions on average values for participants and shapes in the grasping task show that faster movements correlate positively with fewer fixations, as measured by the movement time (R 2 = 0.58) and the time of the digits' arrival relative to the first fixation (for both thumb and index finger: R 2 = 0.65, p-values of all three regressions <0.01). 
Fixation locations
Figures 4 and 5 give impressions of the spread of the fixation locations by showing all fixations of all participants in respectively the grasping and viewing task. Figure 6 presents the averaged fixation location data. The center of each shape is indicated by an ‘x’. The COG is indicated by an ‘o’. The stars refers to the mean of the average fixation locations (black for grasping, gray for viewing), with the horizontal and vertical lines representing the standard errors of the mean in horizontal and vertical direction. For the trials in which more than one fixation was made, the mean of the first fixation is indicated by a square and the mean of the second fixation by a circle, again with error bars indicating the standard error of the mean in the two directions. An arrow connects the first to the second mean fixation. 
Figure 4
 
The locations of all fixations made by all participants in the grasping task of Experiment 1, separately for each shape.
Figure 4
 
The locations of all fixations made by all participants in the grasping task of Experiment 1, separately for each shape.
Figure 5
 
The locations of all fixations made by all participants in the viewing task of Experiment 1, separately for each shape.
Figure 5
 
The locations of all fixations made by all participants in the viewing task of Experiment 1, separately for each shape.
Figure 6
 
Fixation locations for each shape and each task (grasping in black, viewing in gray) in Experiment 1. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects.
Figure 6
 
Fixation locations for each shape and each task (grasping in black, viewing in gray) in Experiment 1. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects.
Average fixations
Figure 6 shows that for all shapes, the average fixation location in the grasping task is higher than in the viewing task. A repeated measures ANOVA on the vertical location of the average fixation with shape as within-subjects factor and task as between-subjects factor indicated that this was significant (main effect of task: F (1,18) = 6.11, p = 0.02), but the effect was stronger for some shapes than for others (interaction between task and shape: F (6,108) = 8.46, p < 0.01). Overall, the horizontal location of the average fixation did not differ between grasping and viewing ( F (1,18) = 1.00, p = 0.33), but there was an interaction between task and shape ( F (6,108) = 4.66, p < 0.01). 
Figure 6 suggests that participants were fixating closer to the COG when viewing than when grasping. To test this, we computed the absolute distance between the average fixation location and the COG for every participant and every shape in both tasks. The mean absolute distance for grasping was 2.54 cm ( SEM = 0.35) and for viewing 1.11 cm ( SEM = 0.13). A repeated measures ANOVA with shape as within-subjects factor and task as between-subjects factor indeed showed a significant effect of task ( F (1,18) = 15.04, p < 0.01), as well as an interaction with shape ( F (6,108) = 2.29, p = 0.04). 
An attraction of fixations to the COG when viewing is also suggested by comparisons between average fixations of different shapes. Paired t-tests indicate that participants look more to the right for the right pointing cross compared to the left pointing cross ( t 9 = 14.98, p < 0.01), more upward for the upward pointing cross than for the downward pointing cross ( t 9 = 9.20, p < 0.01) and more downward for the upward pointing triangle compared to the downward pointing triangle ( t 9 = 16.22, p < 0.01). All of these results are consistent with an attraction by the COG. Though the comparison between tasks suggests that COG is less important in grasping than in viewing (see above), gaze still seems to be attracted by the COG when grasping the crosses. Comparing the horizontal fixation locations between the right and left pointing cross in grasping trials indicate that participants fixate more to the right for the right than for the left pointing cross (paired t-test: t 9 = 27.76, p < 0.01). This suggests that their gaze is drawn to the COG or to the contact positions. A significant higher fixation location for the cross right side up compared to the cross upside down (paired t-test, t 9 = 4.91, p < 0.01) is consistent with an attraction to the COG rather than to the contact positions, which are identical in both cases. When grasping, there is no difference in average vertical fixation position between the upward and downward pointing triangles ( t 9 = −0.22, p = 0.83), which may have been expected if gaze would have been attracted by the COG or the point of the triangle. 
First and second fixations
In 78% of the grasping trials and 80% of the viewing trials, participants made more than one fixation ( Figure 3B). The horizontal location of the first fixation was not affected by task (main effect: F (1,18) = 1.74, p = 0.20, interaction between task and shape: F (6,108) = 1.39, p = 0.23). The overall vertical location of the first fixation was also not affected by task ( F (1,18) = 2.07, p = 0.17) though there was an interaction between shape and task ( F (6,108) = 2.80, p = 0.01). 
In contrast, task clearly affected the location of the second fixation: it was higher when grasping than when viewing (main effect of task on the vertical location: F (1,18) = 6.19, p = 0.02, interaction with shape: F (6,108) = 10.27, p = 0.01). For the horizontal location of the second fixation, there was no overall effect of task ( F (1,18) = 0.70, p = 0.41), but there was an interaction between task and shape ( F (6,108) = 2.96, p = 0.01). Thus, there are no main effects of task on the first fixation whereas the second fixation is located higher when grasping than when only viewing. The effects of task as described for the average fixation seem to be mainly due to the fixations after the first one. 
Consistent with the effect of task on the vertical location of the second fixation, a repeated measures ANOVA on the difference between the vertical location of the first and the second fixation revealed a main effect of task ( F (1,18) = 8.39, p < 0.01), i.e. participants saccade more upward when grasping than when viewing. There was also an interaction between task and shape ( F (6,108) = 5.20, p < 0.01). 
Figure 6 and Tables 1 and 2 illustrate the difference in vertical movement of the eyes between the tasks. Figure 6 shows that within a grasping trial, the eyes move upward for all shapes except for the downward pointing triangle, whereas there are no consistent eye movements in the vertical direction in the viewing task. The first three data columns in Table 1 show the results of one sample t-tests on the vertical difference between the first and the second fixation for every shape in the grasping task. The upward movement is significant for most shapes. In contrast to what the average fixation data suggested, the difference between the first and the second fixation within a grasping trial provides evidence for an attraction of the eye toward the point of the triangle: the vertical difference is more upward for the upward pointing triangle than for the downward pointing triangle (paired t-test on the vertical difference for the two triangles: t 9 = 4.29, p < 0.01). To test whether participants saccaded more strongly upward for the upward compared to the downward pointing triangle in the grasping compared to the viewing task, we performed an independent sample t-test on the difference of the vertical component of the saccade between the triangles. Indeed, there was a significant effect of task ( t 18 = 3.79, p < 0.01), confirming that this was the case. 
Table 1
 
Experiment 1—grasping task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Table 1
 
Experiment 1—grasping task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Shape Vertical difference (cm) t 9 p Horizontal difference (cm) t 9 p
Square 1.02 3.46 <0.01* −2.21 −5.81 <0.01*
Up cross 0.21 1.86 0.10 −1.77 −6.22 <0.01*
Right cross 0.58 1.91 0.09 −1.55 −3.02 0.01*
Down cross 1.91 3.41 <0.01* −1.37 −3.59 <0.01*
Left cross 0.85 3.87 <0.01* −1.73 −4.16 <0.01*
Up triangle 1.84 5.01 <0.01* −1.59 −5.11 <0.01*
Down triangle −0.61 −1.86 0.10 −1.39 −4.99 <0.01*
Table 2
 
Experiment 1—viewing task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Table 2
 
Experiment 1—viewing task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Shape Vertical difference (cm) t 9 p Horizontal difference (cm) t 9 p
Square −0.10 −0.35 0.73 −1.50 −4.85 <0.01*
Up cross −0.18 −1.06 0.32 −0.99 −3.68 <0.01*
Right cross 0.04 0.11 0.92 −1.35 −3.25 0.01*
Down cross 0.78 4.41 <0.01* −0.95 −3.62 <0.01*
Left cross −0.11 −1.33 0.22 −1.08 −5.40 <0.01*
Up triangle 0.28 1.97 0.08 −1.35 −4.13 <0.01*
Down triangle 0.24 1.20 0.26 −1.19 −3.91 <0.01*
Figure 6 and Tables 1 and 2 show that for both tasks and all shapes, the second fixation was to the left of the first one. The task did not affect the extent to which the eyes moved leftward (repeated measures ANOVA on the difference between the horizontal location of the first and the second fixation, main effect task: F (1,18) = 1.81, p = 0.20, interaction with shape: F (6,108) = 0.47, p = 0.83). This ANOVA did not indicate an effect of shape ( F (6,108) = 1.36, p = 0.24). 
The absolute distance between the average first fixation and the COG was smaller during viewing than during grasping (1.57 cm ( SEM = 0.26) versus 2.76 cm ( SEM = 0.43), main effect task: F (1,18) = 5.74, p = 0.03, no interaction with shape: F (6,108) = 1.38, p = 0.23). This was also true for the second fixation (1.43 cm ( SEM = 0.12) versus 2.95 cm ( SEM = 0.39), main effect task: F (1,18) = 13.78, p < 0.01, no interaction with shape: F (6,108) = 2.01, p = 0.07). 
Single fixations
If participants made only one fixation in a trial, this fixation was shifted into the direction of the average location of second fixations in trials containing more fixations, relative to the average location of the first fixations in those trials. For grasping, single fixations were on average 1.36 cm ( SEM = 0.07) to the left of and 0.35 cm ( SEM = 0.11) above the first fixation in multi-fixation trials ( t 6 = −15.30, p < 0.01 and t 6 = 2.72, p = 0.03 respectively). 
For viewing, single fixations were on average 0.42 cm to the left of the first fixation in multi-fixation trials ( SEM = 0.09, t 6 = −4.10, p < 0.01). As the vertical location of first and second fixations were about the same, it is not surprising that the vertical location of single fixations did not differ from that of first fixations in multi-fixation trials (on average 0.14 cm, SEM = 0.08, t 6 = 1.34, p = 0.23). These results suggests that compared to first fixations in multi-fixation trials, single fixations are closer to the desired final location so that in contrast to multi-fixation trials, additional, corrective fixations are not necessary. 
Maximal distance
For the grasping task, the average maximal vertical distance between two fixations in one grasping trial was 2.14 cm ( SEM = 0.21). The values are plotted for each shape separately in Figure 7
Figure 7
 
The maximal absolute vertical distance between any two fixations within one trial in the grasping task of Experiment 1, separately for each shape. Error bars are standard errors of the mean.
Figure 7
 
The maximal absolute vertical distance between any two fixations within one trial in the grasping task of Experiment 1, separately for each shape. Error bars are standard errors of the mean.
Discussion
Fixation locations when grasping
When our participants were planning to grasp an object, they saccaded more upward after the first saccade to the object and in general fixated a higher location compared to when they were only viewing. The latter effect is mainly due to the fixations after the first one as indicated by the fact that the significant main effect of task on vertical fixation location only appeared for the second and not for the first fixation. This suggests that first, participants make a general saccade toward the shape that is not strongly influenced by task, but rather by visual features like the COG. The average fixation locations when grasping the differently oriented crosses provide evidence for an attraction by the COG. 
Also, the participant's gaze was attracted by a region where the digits had to be precisely guided to, which we had operationalized by the point of a triangle. Although the average vertical fixation location did not differ between the upward and downward pointing triangle, the direction of eye movements were more upward for the upward pointing triangle than for the downward pointing triangle (the only shape in which participants tended to saccade downward rather than upward). 
Thus, after the first fixation, participants start to tailor their fixations to the specific needs of the task. In the case of grasping, this first experiment suggests these needs to be obtaining information around the area of the index finger's contact location and the contact location that is relatively difficult to guide the digits to. This is best illustrated by the upward pointing triangle and downward pointing cross. The first fixation during grasping seems to be attracted downward to the COG, so that a particularly large upward saccade was needed to get closer to the contact location of the index finger or the region of high required accuracy. 
Another finding is that after their first fixation, participants saccaded to the left, that is, continuing to move their eyes in the direction from the initial fixation dot to the shape. Participants seem to prefer undershooting the desired fixation location slightly and then saccading further than overshooting and having to saccade back ( Figure 6). 
The maximal absolute vertical distance between two fixations within one grasping trial was small (2.14 cm) compared to the vertical distance between the contact locations (7.8 cm for the triangles and 9 cm for the other shapes). This indicates that participants did not first look at the thumb and then at the index finger, or the other way around. This is confirmed by Figure 4, which does not show two clouds of fixations at the contact position. For all shapes, the fixations are clearly shifted to the upper part of the shape, except for the downward pointing cross and the downward pointing triangle. These exceptions respectively suggest that both a low COG and a small contact position at the bottom of the object prevents the gaze from going upward to the index finger. 
The fact that the number of fixations decreases with quicker hand movements is not surprising as there is simply less time for making more fixations. However, together with our finding that single fixations are ‘better aimed’ than first fixations in trials with more fixations, it demonstrates a coordination between hand and eyes and points out that proper fixations for grasping are necessary, even for these repeatedly presented small objects. Our results do not show whether the reason for the decreasing number of fixations with quicker hand movements is planned or post-hoc. That is, participants could vary in planning the precision of eye movements concurrently with planning the speed of the hand, or they could slow down the hand when their gaze landed at a less appropriate position on the shape. 
Fixation locations when only viewing
When participants looked at the shapes without grasping them, the saccade after the one toward the shape was not generally upward and the eyes were not attracted to the point of the triangle. Our results show a stronger attraction of the eyes to the COG in the viewing task than in the grasping task. This is consistent with the findings of Vishwanath and Kowler ( 2003) that in a viewing task participants fixate close to the COG of shapes. Whereas in the viewing task, participants only have to visually explore the object, in the grasping task participants both look at the object for visual exploration and for the purpose of grasping it. As discussed before, in the grasping task the eyes are drawn to the COG as well, presumably reflecting the visual exploration part of the task. However, the grasping part is visible in the attraction being weaker than in the viewing task. The other fixation location results are similar to the results of the grasping task. As in grasping, the eyes saccaded to the left during a trial. 
Fixation timing
The number of fixations per trial was the same for grasping and for only viewing. The first fixations occurred later in grasping than in viewing. This finding contrasts with the findings of Lünenburger et al. (2000) who reported lower saccadic reaction times3 toward illuminated LEDs if accompanying pointing movements were made than without these movements (though the difference was very small). For drawing movements with a stylus to a target, Bekkering, Adam, Kingma, Huson, & Whiting (1994) and Bekkering et al. ( 1995) found longer saccadic reaction times compared to when no drawing movement needed to be made. It seems that different movements can affect saccadic reaction times in different ways. Compared to pointing to a single target, our grasping movement and the drawing movement as used by Bekkering and co-workers may require more planning time and thus delay the initial saccade. 
For both tasks, the first fixation started later if only one fixation was made compared to when more fixations were made, and they were relatively close to the final desired fixation location (as measured by the average second fixation location in other trials). This better aiming could indicate a more careful planning of that first fixation so that additional fixations were not needed. It does not seem likely that first fixations in single fixation trials started too late for more fixations being possible ( Figure 3D). 
Experiment 2
In Experiment 1 participants saccaded more in the direction of the index finger's contact location when they had to grasp the shape compared to when they only had to look at it. However, in Experiment 1 the contact location of the index finger was always at the upper part of the shape. The question arises whether the second fixation was attracted by the index finger or whether participants prefer to look at the upper part of the shape during grasping for another reason, for example, because of differences between the upper and lower visual field in information processing (Danckert & Goodale, 2001; Krigolson & Heath, 2006; Previc, 1990). In Experiment 2, we will differentiate between these two possibilities by moving the contact position of the index finger from the upper part of the shape toward the sides of the shape and having the participants grasp the objects with left and right hand. If the second fixation is attracted by the index finger, this manipulation should result in different fixations locations than in Experiment 1. If, on the other hand, the second fixation is generally directed more to the upper part of the shape, then this manipulation should have little effect. In addition, we tried to replicate the attraction by a contact point that requires high precision and therefore we have used the square as well as the triangular shapes. 
Methods
Participants
Ten participants participated. One of them was one of the authors, the rest were students and colleagues from the Justus-Liebig University, naïve to the purpose of the experiment. The author and one of the other participants had also participated in the grasping part of Experiment 1. Half of the participants were women and half were men. Their ages ranged between 21 and 38 years. All participants were self-declared right-handed. 
Stimuli and design
We asked the participants to grasp the same square and triangle that we had used in Experiment 1. The triangle was now presented in a right pointing and left pointing orientation. We used another Plexiglas frame to mount the shapes on than in Experiment 1. In one half of the experiment, we presented the shapes on the left side ( Figure 8). For the other half of the experiment, we flipped the frame to present the shapes on the right side. The fixation dot was always presented on the opposite side of the shape. It was 17.0 cm (20.4 deg) away from the center of the square and the center of the triangle. Participants were asked to contact the sides of the shape (and the point of the triangle) using their index finger and thumb. The shapes presented on the left were grasped with the left hand and the ones presented on the right with the right hand. With this design, we kept the visual stimuli basically constant, while varying the location of the contact positions of index finger and thumb: for the shapes presented on the left, the index finger's contact position was on the left side of the shape, whereas the index finger's contact position was on the right side of the shape for shapes presented on the right. 
Figure 8
 
Schematic depiction of the setup and the way of grasping in Experiment 2, when the shapes were presented on the left. The mirror image of this depiction would represent the situation when shapes were presented on the right. In the actual experiment, there was no fixation point present during grasping.
Figure 8
 
Schematic depiction of the setup and the way of grasping in Experiment 2, when the shapes were presented on the left. The mirror image of this depiction would represent the situation when shapes were presented on the right. In the actual experiment, there was no fixation point present during grasping.
Participants performed 3 blocks consisting of 3 shapes * 6 repetitions = 18 trials (in random order) with the shapes presented on the right side, and 3 blocks with the shapes presented on the left side, resulting in a total of 108 trials per participant. Half of the participants started with grasping shapes on the right side, the other half started with the shapes on the left side. Before each half of the experiment (right side and left side), participants performed one block of practice trials. 
Apparatus and procedure
The apparatus and procedure were the same as for the grasping task in Experiment 1
Analysis
We ran a total of 973 experimental trials. Out of these, 72 trials (7.4%) were rejected, due to insufficient quality of eye or hand data. For analyzing the eye–hand timing, 11 extra trials in Experiment 2 (1% of the total number of recorded trials) had to be discarded due to insufficient quality of the start of the hand's movement. The same kinds of analyses were performed as in Experiment 1, except for that now, we have presentation side as an extra variable. 
Results
Eye–hand timing
The gray symbols in Figure 9A show when the index finger and thumb start to move relative to the start of the first fixation. On average the thumb starts to move 59 ms ( SEM = 27) before the eye arrives, and the index finger 70 ms ( SEM = 28). A repeated measures ANOVA with digit, shape and presentation side as within-subjects variables showed that there were no main effects and no interactions of any of the variables (all p-values > 0.08). The black symbols in Figure 9A indicate when the index finger and thumb arrive at the shape relative to the start of the first fixation. On average, the thumb arrives 935 ms ( SEM = 51) after the first fixation started, and the index finger 873 ms ( SEM = 50). The relative arrival time of the hand depended on shape ( F (2,18) = 6.48, p = 0.01) and on digit ( F (1,9) = 76.22, p < 0.01), with the index finger arriving before the thumb. There was also a marginally significant interaction between digit and shape ( F (2,18) = 3.54, p = 0.05) and an interaction between digit and side ( F (1,9) = 16.59, p < 0.01). The latter interaction indicated that the difference in relative arrival time between the two digits was smaller when the shape was presented on the left side (on average 40 ms) compared to the right side (on average 85 ms). For trials in which there were more than one fixation, the thumb arrived 381 ms ( SEM = 33) and the index finger 319 ms ( SEM = 36) after the last fixation had started. 
Figure 9
 
Eye–hand timing and general fixation characteristics in Experiment 2. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in Experiment 2, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the right (solid circles) and left (empty squares) presentation side. (C) Average number of fixations per trial for each shape, separately for the right (solid circles) and left (empty squares) presentation side. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape and presentation side. For trials with only one fixation (data point to the far right), the data is collapsed over shapes and presentation sides. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data when the shape was presented on the right, empty symbols represent the data when the shape was presented on the left. The error bars in all panels are standard errors of the mean.
Figure 9
 
Eye–hand timing and general fixation characteristics in Experiment 2. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in Experiment 2, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the right (solid circles) and left (empty squares) presentation side. (C) Average number of fixations per trial for each shape, separately for the right (solid circles) and left (empty squares) presentation side. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape and presentation side. For trials with only one fixation (data point to the far right), the data is collapsed over shapes and presentation sides. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data when the shape was presented on the right, empty symbols represent the data when the shape was presented on the left. The error bars in all panels are standard errors of the mean.
General fixation characteristics
Figure 9B shows the proportion of trials in which 1 to 5 saccades were made, separately for the shapes presented on the right side (solid symbols) and the shapes presented on the left side (empty symbols). On average, there were 3.17 ( SEM = 0.22) fixations per trial for the shapes presented on the right and 2.78 ( SEM = 0.30) for the shapes presented on the left. The average number of fixations per trial is presented separately for shape and presentation side in Figure 9C. A repeated measures ANOVA on the average number of fixations with shape and side as within-subjects factors showed a significant effect of shape ( F (2,18) = 4.23, p = 0.03), but no effect of presentation side ( F (1,9) = 3.23, p = 0.11) and no interaction ( F (2,18) = 2.54, p = 0.11). Figure 9D shows the start times for the first (gray symbols) and second (black symbols) fixations, separately for each shape and presentation side. The data point to the far right represents trials in which only one fixation was made, collapsed over all conditions. Repeated measures ANOVAs on the start times indicated an effect of shape for the second fixation ( F (2,18) = 4.79, p = 0.02), but not for the first fixation ( F (2,18) = 0.19, p = 0.82). There were no effects of presentation side for the first and for the second fixation (respectively F (1,9) = 3.61, p = 0.09 and F (1,9) = 3.97, p = 0.08) and no interactions between shape and side (respectively F (2,18) = 1.26, p = 0.31, and F (2,18) = 1.40, p = 0.27). We did not replicate the effect found in the previous experiment of a longer starting time for the first fixation if only one fixation was made compared to trials in which more fixations were made ( t 8 = 0.02, p = 0.99). This could be due to the overall later start times. 
Fixations
Figure 10 shows all fixation locations made by all participants for the shapes presented on the left and the shapes presented on the right separately. Figure 11 presents the averaged fixation locations for the shapes presented on the right (black symbols) and the shapes presented on the left (gray symbols). Participants made more than one fixation in 85% of the trials. 
Figure 10
 
The locations of all fixations made by all participants in Experiment 2, separately for each shape.
Figure 10
 
The locations of all fixations made by all participants in Experiment 2, separately for each shape.
Figure 11
 
Fixation locations for each shape and presentation side: right in black, left in gray. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by the squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects. The hands and dots outside the shapes represent the hand that was used and the side of the initial fixation dot when the shape was presented on the right (black) and on the left (gray).
Figure 11
 
Fixation locations for each shape and presentation side: right in black, left in gray. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by the squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects. The hands and dots outside the shapes represent the hand that was used and the side of the initial fixation dot when the shape was presented on the right (black) and on the left (gray).
Figure 11 and Table 3 do not show a consistent difference between the first two fixations in the vertical direction. Only for the left pointing triangle presented on the left, there was a significant upward movement. An independent samples t-test on the difference in vertical fixation position between the first and the second fixation indicated that participants saccaded significantly more upward for the square in Experiment 1 compared to the square presented on the left ( t 18 = 2.56, p = 0.02) and compared to the square presented on the right ( t 18 = 2.85, p = 0.01) in the present experiment. 
Table 3
 
Experiment 2—mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Table 3
 
Experiment 2—mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Side Shape Vertical difference (cm) t 9 p Horizontal difference (cm) t 9 p
Right Square −0.05 −0.21 0.84 0.98 2.37 0.04*
Right Right triangle −0.31 −0.97 0.36 2.59 4.59 <0.01*
Right Center triangle −0.15 −0.42 0.68 0.47 1.17 0.27
Left Square −0.10 −0.32 0.76 −0.65 −0.94 0.37
Left Right triangle 0.54 2.05 0.07 −0.33 −0.69 0.51
Left Center triangle 0.60 2.66 0.03* −2.07 −3.74 <0.01*
On average, the horizontal component of the second saccade was in the main movement direction of the eyes for all shapes, with the first (undershooting) fixation landing at the side of the shape closest to the initial fixation dot. However, for the square presented on the left, the right pointing triangle presented on the left, and the left pointing triangle presented on the right, the horizontal difference between the first and the second fixation was very small and not significantly different from zero. The latter two results indicate that the eyes do not move away from the point of the triangle when they are already close to it ( Figure 11). The horizontal differences for the triangles pointing away from the initial fixation point (the right pointing triangle presented on the right and the left pointing triangle on the left) are large compared to the square, also indicating an attraction to the point of the triangle. 
As in the first experiment, single fixations were made further into the direction of the shape than the first fixations in multi-fixation trials (0.76 cm, SEM = 0.12, t 5 = −5.00, p < 0.01). The vertical location was (not surprisingly) statistically the same (on average 0.31 cm above the first fixation in multi-fixation trials, SEM = 0.12, t 5 = 2.06, p = 0.09). In this experiment, we did not find correlations between number of fixation and the hand's movement time. 
Discussion
In the present experiment, participants did not saccade upward as they did in the previous experimental grasping task. Thus, the attraction of fixations to the upper part of the shape during grasping as found in the previous experiment is related to where the thumb and index finger contact the object rather than that the upper part of the shape is especially important for another reason. The results are consistent with an attraction by the contact location of the index finger. However, we should note that in Experiment 2, the attraction seemed to be weaker than in Experiment 1, indicating that there may be an interaction between the location of the index finger and the contact location on the object. We need further experiments to investigate more precisely whether, and if so how, the attraction of fixations to the index finger varies as a function of the contact locations of thumb and finger (while maintaining a natural grasp) or other variables. 
In Experiment 2 we again found evidence for an attraction of fixations to the smaller contact location, where the digit needed to contact the object more precisely. The first fixation landed at the side of the triangle closest to the location of the initial fixation dot (as observed in the previous experiment in which participants also undershot the targets). When the point of the triangle was in this region, the second fixation stayed close to the first. When the point of the triangle was further away from the initial fixation location, participants saccaded in that direction. 
Summary and general discussion
We investigated where people look at during grasping objects in a setup where the contact positions of thumb and index finger with the to-be-grasped object were both visible. Experiment 1 showed that although the distance between the contact positions was only about 11.5 degrees, participants did not show the same fixation patterns when grasping as when only viewing. Below we give our view of the events during a gaze guided grasp with the two contact locations visible. 
Overview of events during gaze guided grasping
In grasping, the eyes start to move to the object later than when the object is only viewed, suggesting that the eyes wait for the hand's motor planning. The average time that the hand leaves before the first fixation starts is 60 ms. Since saccades in our experiment lasted about 40 ms, this means that the eyes and the hand generally leave for the target at around the same time. This is consistent with results in other tasks involving grasping (e.g. Hayhoe et al., 2003; Pelz et al., 2001). 
At first, the eyes are attracted by the object's COG but undershoot the target. The first fixation locations during grasping are similar to the ones made during only viewing. This suggests that participants use the same mechanism, which could be largely based on visual features of the shape, to plan and perform the first general saccade toward the shape. Saccadic undershoots of targets have been reported before (Lünenburger et al., 2000; see for overviews Harris, 1995 and Kowler & Blaser, 1995). Especially large visual targets, such as the ones used in the present experiments, are prone to undershoot (Ploner, Ostendorf, & Dick, 2004). 
In the case that only one fixation is made, the saccade tends to be executed late and better aimed. The accompanying movement of the hand tends to be fast. However, in by far the most cases, a second fixation follows the first, still long before the hand arrives. The second fixation differs particularly between grasping and only viewing. Whereas during viewing, the second fixation mainly makes up for the undershoot, during grasping the second fixation seems to be attracted by regions that need to be contacted precisely, and to the contact location of the index finger. Possible reasons for this will be discussed later. 
Finally, the hand reaches the object, with the index finger arriving before the thumb. 
Determinants of fixation location in grasping
At least for the shapes and setup we used, we can reject the hypothesis that participants were looking at the contact position of the thumb. The spread of the individual fixations, which were not centered around the two contact locations, and the relatively small maximal vertical distance between two fixations within a trial, indicated that participants were not alternating fixations at the contact position of the thumb and of the index finger between or within trials. Thus, our results differ from those found by Riek et al. (2003), who asked their participants to point at two targets at the same time, using the index fingers of both hands. Their participants directed their index fingers to a position above the targets, looking at both targets sequentially, and then move the index fingers down simultaneously. Our task also involved two contact locations, but participants did not need to look at both contact locations. There was also a rather long time interval between the last fixation and the arrival of the hand. Apparently, it is not necessary to monitor grasping as extensively as monitoring the hands in the bi-manual task of Riek et al. ( 2003). Clearly, grasping is a well-practiced task and, although multiple digits are attached to it, it involves the guiding of only one hand to the target. 
Although participants fixated closer to the COG during the viewing than during the grasping task, the COG also appeared to affect fixation locations during grasping. This is most clearly demonstrated by the upward and downward pointing cross, for which the grasping locations were exactly the same, but the eye movements very different. For the downward pointing cross, the first fixation seemed to be attracted by the COG, so that a large upward saccade was needed to reach the desired gaze location near the index finger, whereas for the upward pointing cross, the eyes initially landed near the COG and stayed there as that location was close to the contact position of the index finger. Thus, the results are consistent with both the index finger's contact position and the visual features of the object (the COG and perhaps salient feature, see footnote i) determining the fixation locations in grasping objects, where the visual information especially affects the first fixation and the index finger's contact position especially affects the second fixation. A similar result was found by Stritzke and Trommershäuser ( 2007). Their participants pointed to targets while avoiding penalty regions. The visual saliency of the target and penalty regions were manipulated. Whereas the first fixation was attracted to the salient region, the second one shifted closer toward the contact position of the index finger. These results suggest a more low-level mechanism for planning the first general saccade and a higher level, more task dependent mechanism for the saccades after that. 
Why the index finger?
Our study points at the contact location of the index finger as one of the attractors of fixation during grasping. Possibly, this is related to the finding that the index finger was the first to arrive at the object. However, we could not support this hypothesis by positive correlations between the extent that the eyes saccaded upward and the extent that the index finger arrived before the thumb, neither within nor between participants. 
As we discussed in Introduction, two important functions of fixations during goal directed movements can be distinguished: to provide goal position information and to provide visual feedback about the approaching effector so that the movement can be adjusted online. The goal position function of fixations has been demonstrated amongst other findings by fixations to future grasping targets and obstacles (Hayhoe et al., 2003; Johansson et al., 2001; Land et al., 1999; Pelz et al., 2001) and by the importance of gazing at a pointing target, even when the effector is not visible (Abrams et al., 1990). As there is evidence that the thumb guides the hand to the object (Galea et al., 2001; Haggard & Wing, 1997; Wing & Fraser, 1983), the goal position function of fixation in grasping should be apparent through fixation of the thumb's contact location. As the index finger is the digit following a more variable, curved trajectory (Galea et al., 2001; Haggard & Wing, 1997; Wing & Fraser, 1983), the visual guidance function should be apparent through fixation of the index finger. Our results resemble the latter. Consistent with this, a recent study by de Grave, Hesse, Brouwer, and Franz ( 2008) showed that occluding the contact location of the index finger diminished its attraction of gaze. This is not to say that fixations during grasping are not used for locating the goal, but rather that the system seems to prefer to fixate the location where visual feedback is needed more, at the cost of fixating the location that would probably be the most informative in terms of goal position. 
The relative importance of visual feedback would be consistent with Binsted et al. (2001), who stress that the eyes need to get somewhere near the target area to provide the manual system with visual information about the relative positions of the hand and the target, and that extra retinal position information is only important when this visual information is severely degraded. Our findings are also consistent with the grasping model of Smeets and Brenner (1999, 2001, 2002). In their model, the fingertips and the tip of the thumb are independently guided toward the planned contact positions on the object. Their specific trajectories are only determined by the starting locations of the digits, the contact locations, and the distance across which the digits are planned to move perpendicularly to the surface of the grasping locations (which varies with required precision). Without assigning specific roles to the thumb and index finger, Smeets and Brenner can predict typical observations of the digits' movements, such as the more curved and variable path of the index finger compared to the thumb. Following their model, there would not be any particular reason to look at the thumb's contact location, and it would make sense to look at the index finger to acquire visual feedback. 
Necessity of fixating the target during grasping
We argue that fixations in grasping are mainly important for visual online guidance of the approaching digits. Other actions, such as placing an object on a table, do not always require accompanying fixations (Hayhoe et al., 2003). Several studies, in which participants needed to perform additional tasks after grasping one object, show that visual feedback is not needed until the very end of the movement as the eyes often leave the target object just before the grasp is completed (Johansson et al., 2001; Land & Hayhoe, 2001). Of course, at the very last part of the grasp, the digits' movements cannot be adjusted anyway because of the visuo-motor delay, and the gaze may just as well be sent to locations that are relevant for the follow-up action. Johansson et al. (2001) downplay the importance of eye movements during their grasping and moving task by reporting that the time needed to perform the movement was not increased when the participants had to fixate a fixation light. However, the obstacle was hit more frequently and the participants said that they needed to concentrate during grasping when fixating elsewhere. Also, Ballard et al. (1992) and Ballard, Hayhoe, and Pelz ( 1995) report that for a block building task, participants rather refixate objects than use memory. 
In sum, we conclude that in normal situations, people will use their gaze in grasping rather specifically, ultimately aiming for the spot where online visual feedback is most useful, such as a contact location that is difficult to reach and the contact location of the index finger. 
Acknowledgments
Thanks to Tim Klucken for collecting data. Thanks to Dirk Kerzel for his Eyelink C-library that we used in our experimental programs. Supported by the European Commission (Grant HPRN-CT-2002-00226) and the Deutsche Forschungsgemeinschaft (DFG Forschergruppe FOR 560 “Perception and Action”). 
Commercial relationships: none. 
Corresponding author: Anne-Marie Brouwer. 
Email: anne-marie.brouwer@tno.nl. 
Address: Kampweg 5, P.O. Box 23, 3769 ZG Soesterberg, The Netherlands. 
Footnotes
Footnotes
1  Here, we use ‘pointing’ in the sense of touching the target.
Footnotes
2  The intersection of the bars making up the cross could also be considered as comprising a salient feature, similar to the intersection of the two limbs in an L-shape that has been shown to attract gaze in a viewing task (Vishwanath & Kowler, 2003). As the COG is close to the intersection of the bars, any attraction to the COG for this shape could also be considered as attraction to the salient feature. In any case, similar to the COG, the salient feature has been shown to play a special role in viewing whereas, a priori, it does not play a specific role in grasping.
Footnotes
3  Saccadic reaction time is not exactly the same as the start time of the fixation because the latter includes the movement time of the eyes. However, the difference would be relatively small, and if anything, the saccadic reaction time would be more in the direction of being shorter for grasping compared to only looking as saccades tend to be faster when accompanied by an arm movement in the same direction than when not (Epelboim et al., 1997; Snyder, Calton, Dickinson, & Lawrence, 2002).
References
Abrams, R. A. Meyer, D. E. Kornblum, S. (1990). Eye–hand coordination: Oculomotor control in rapid aimed limb movements. Journal of Experimental Psychology: Human Perception and Performance, 16, 248–267. [PubMed] [CrossRef] [PubMed]
Admiraal, M. A. Keijsers, N. L. Gielen, C. C. (2003). Interaction between gaze and pointing toward remembered visual targets. Journal of Neurophysiology, 90, 2136–2148. [PubMed] [Article] [CrossRef] [PubMed]
Aivar, M. P. Hayhoe, M. M. Chizk, C. L. Mruczek, R. E. (2005). Spatial memory and saccadic targeting in a natural task. Journal of Vision, 5, (3):3, 177–193, http://journalofvision.org/5/3/3/, doi:10.1167/5.3.3. [PubMed] [Article] [CrossRef]
Ballard, D. H. Hayhoe, M. M. Li, F. Whitehead, S. D. (1992). Hand–eye coordination during sequential tasks. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 337, 331–339. [PubMed] [Article] [CrossRef]
Ballard, D. H. Hayhoe, M. M. Pelz, J. B. (1995). Memory representations in natural tasks. Journal of Cognitive Neuroscience, 7, 66–80. [CrossRef] [PubMed]
Bekkering, H. Adam, J. J. Kingma, H. Huson, A. Whiting, H. T. (1994). Reaction time latencies of eye and hand movements in single- and dual-task conditions. Experimental Brain Research, 97, 471–476. [PubMed] [CrossRef] [PubMed]
Bekkering, H. Adam, J. J. van der Aarssen, A. Kingma, H. Whiting, H. T. (1995). Interference between saccadic eye and goal-directed hand movements. Experimental Brain Research, 106, 475–484. [PubMed] [CrossRef] [PubMed]
Binsted, G. Chua, R. Helsen, W. Elliott, D. (2001). Eye–hand coordination in goal-directed aiming. Human Movement Science, 20, 563–585. [PubMed] [CrossRef] [PubMed]
Binsted, G. Elliott, D. (1999). Ocular perturbations and retinal/extraretinal information: The coordination of saccadic and manual movements. Experimental Brain Research, 127, 193–206. [PubMed] [CrossRef] [PubMed]
Bock, O. (1986). Contribution of retinal versus extraretinal signals towards visual localization in goal-directed movements. Experimental Brain Research, 64, 476–482. [PubMed] [CrossRef] [PubMed]
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Earlbaum Associates.
Danckert, J. Goodale, M. A. (2001). Superior performance for visually guided pointing in the lower visual field. Experimental Brain Research, 137, 303–308. [PubMed] [CrossRef] [PubMed]
de Grave, D. D. Brenner, E. Smeets, J. B. (2004). Illusions as a tool to study the coding of pointing movements. Experimental Brain Research, 155, 56–62. [PubMed] [CrossRef] [PubMed]
de Grave, D. D. Smeets, J. B. Brenner, E. (2006). Why are saccades influenced by the Brentano illusion? Experimental Brain Research, 175, 177–182. [PubMed] [CrossRef] [PubMed]
de Grave, D. D. Franz, V. H. Gegenfurtner, K. R. (2006). The influence of the Brentano illusion on eye and hand movements. Journal of Vision, 6, (7):5, 727–738, http://journalofvision.org/6/7/5/, doi:10.1167/6.7.5. [PubMed] [Article] [CrossRef]
de Grave, D. D. J. Hesse, C. Brouwer, A. M. Franz, V. H. (2008). Fixation locations when grasping partly occluded objects. Journal of Vision, 8, (7):5, 1–11, http://journalofvision.org/8/7/5/, doi:10.1167/8.7.5. [PubMed] [Article] [CrossRef] [PubMed]
Enright, J. T. (1995). The non-visual impact of eye orientation on eye–hand coordination. Vision Research, 35, 1611–1618. [PubMed] [CrossRef] [PubMed]
Epelboim, J. Steinman, R. M. Kowler, E. Pizlo, Z. Erkelens, C. J. Collewijn, H. (1997). Gaze-shift dynamics in two kinds of sequential looking tasks. Vision Research, 37, 2597–2607. [PubMed] [CrossRef] [PubMed]
Frens, M. A. Erkelens, C. J. (1991). Coordination of hand movements and saccades: Evidence for a common and separate pathway. Experimental Brain Research, 85, 682–690. [PubMed] [CrossRef] [PubMed]
Galea, M. P. Castiello, U. Dalwood, N. (2001). Thumb invariance during prehension movement: Effects of object orientation. Neuroreport, 12, 2185–2187. [PubMed] [CrossRef] [PubMed]
Haggard, P. Wing, A. (1997). On the hand transport component of prehensile movements. Journal of Motor Behavior, 29, 282–287. [PubMed] [CrossRef] [PubMed]
Harris, C. M. (1995). Does saccadic undershoot minimize saccadic flight‐time A Monte-Carlo study. Vision Research, 35, 691–701. [PubMed] [CrossRef] [PubMed]
Hayhoe, M. (2000). Vision using routines: A functional account of vision. Visual Cognition, 7, 43–64. [CrossRef]
Hayhoe, M. M. Shrivastava, A. Mruczek, R. Pelz, J. B. (2003). Visual memory and motor planning in a natural task. Journal of Vision, 3, (1):6, 49–63, http://journalofvision.org/3/1/6/, doi:10.1167/3.1.6. [PubMed] [Article] [CrossRef] [PubMed]
He, P. Y. Kowler, E. (1991). Saccadic localization of eccentric forms. Journal of the Optical Society of America A, Optics and Image Science, 8, 440–449. [PubMed] [CrossRef] [PubMed]
Helsen, W. F. Elliott, D. Starkes, J. L. Ricker, K. L. (1998). Temporal and spatial coupling of point of gaze and hand movements in aiming. Journal of Motor Behavior, 30, 249–259. [CrossRef] [PubMed]
Henriques, D. Y. Klier, E. M. Smith, M. A. Lowy, D. Crawford, J. D. (1998). Gaze-centered remapping of remembered visual space in an open-loop pointing task. Journal of Neuroscience, 18, 1583–1594. [PubMed] [Article] [PubMed]
Itti, L. Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, 1489–1506. [PubMed] [CrossRef] [PubMed]
Johansson, R. S. Westling, G. Bäckström, A. Flanagan, J. R. (2001). Eye–hand coordination in object manipulation. Journal of Neuroscience, 21, 6917–6932. [PubMed] [Article] [PubMed]
Kowler, E. Blaser, E. (1995). The accuracy and precision of saccades to small and large targets. Vision Research, 35, 1741–1754. [PubMed] [CrossRef] [PubMed]
Krieger, G. Rentschler, I. Hauske, G. Schill, K. Zetzsche, C. (2000). Object and scene analysis by saccadic eye-movements: An investigation with higher-order statistics. Spatial Vision, 13, 201–214. [PubMed] [CrossRef] [PubMed]
Krigolson, O. Heath, M. (2006). A lower visual field advantage for endpoint stability but no advantage for online movement precision. Experimental Brain Research, 170, 127–135. [PubMed] [CrossRef] [PubMed]
Land, M. F. Hayhoe, M. (2001). In what ways do eye movements contribute to everyday activities? Vision Research, 41, 3559–3565. [PubMed] [CrossRef] [PubMed]
Land, M. F. McLeod, P. (2000). From eye movements to actions: How batsmen hit the ball. Nature Neuroscience, 3, 1340–1345. [PubMed] [CrossRef] [PubMed]
Land, M. Mennie, N. Rusted, J. (1999). The roles of vision and eye movements in the control of activities of daily living. Perception, 28, 1311–1328. [PubMed] [CrossRef] [PubMed]
Land, M. F. Tatler, B. W. (2001). Steering with the head The visual strategy of a racing driver. Current Biology, 11, 1215–1220. [PubMed] [Article] [CrossRef] [PubMed]
Lünenburger, L. Kutz, D. F. Hoffmann, K. P. (2000). Influence of arm movements on saccades in humans. European Journal of Neuroscience, 12, 4107–4116. [PubMed] [CrossRef] [PubMed]
Mack, A. Heuer, F. Villardi, K. Chambers, D. (1985). The dissociation of position and extent in Müller-Lyer figures. Perception & Psychophysics, 37, 335–344. [PubMed] [CrossRef] [PubMed]
McGowan, J. W. Kowler, E. Sharma, A. Chubb, C. (1998). Saccadic localization of random dot targets. Vision Research, 38, 895–909. [PubMed] [CrossRef] [PubMed]
Medendorp, W. P. Crawford, J. D. (2002). Visuospatial updating of reaching targets in near and far space. Neuroreport, 13, 633–636. [PubMed] [CrossRef] [PubMed]
Melcher, D. Kowler, E. (1999). Shapes, surfaces and saccades. Vision Research, 39, 2929–2946. [PubMed] [CrossRef] [PubMed]
Neggers, S. F. Bekkering, H. (1999). Integration of visual and somatosensory target information in goal-directed eye and arm movements. Experimental Brain Research, 125, 97–107. [PubMed] [CrossRef] [PubMed]
Neggers, S. F. Bekkering, H. (2000). Ocular gaze is anchored to the target of an ongoing pointing movement. Journal of Neurophysiology, 83, 639–651. [PubMed] [Article] [PubMed]
Neggers, S. F. Bekkering, H. (2001). Gaze anchoring to a pointing target is present during the entire pointing movement and is driven by a non-visual signal. Journal of Neurophysiology, 86, 961–970. [PubMed] [Article] [PubMed]
Neggers, S. F. Bekkering, H. (2002). Coordinated control of eye and hand movements in dynamic reaching. Human Movement Science, 21, 349–376. [PubMed] [CrossRef] [PubMed]
Pélisson, D. Prablanc, C. Goodale, M. A. Jeannerod, M. (1986). Visual control of reaching movements without vision of the limb II Evidence of fast unconscious processes correcting the trajectory of the hand to the final position of a double‐step stimulus. Experimental Brain Research, 62, 303–311. [PubMed] [CrossRef] [PubMed]
Pelz, J. Hayhoe, M. Loeber, R. (2001). The coordination of eye, head and hand movements in a natural task. Experimental Brain Research, 139, 266–277. [PubMed] [CrossRef] [PubMed]
Ploner, C. J. Ostendorf, F. Dick, S. (2004). Target size modulates saccadic eye movements in humans. Behavioral Neuroscience, 118, 237–242. [PubMed] [CrossRef] [PubMed]
Previc, F. H. (1990). Functional specialisation in the lower and upper visual fields in humans: Its ecological origins and neurophysiological implications. Behavioral and Brain Sciences, 13, 519–575. [CrossRef]
Privitera, C. Stark, L. (2000). Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 970–982. [CrossRef]
Reinagel, P. Zador, A. M. (1999). Natural scene statistics at the centre of gaze. Network, 10, 341–350. [PubMed] [CrossRef] [PubMed]
Riek, S. Tresilian, J. R. Mon-Williams, M. Coppard, V. L. Carson, R. G. (2003). Bimanual aiming and overt attention: One law for two hands. Experimental Brain Research, 153, 59–75. [PubMed] [CrossRef] [PubMed]
Schiegg, A. Deubel, H. Schneider, W. X. (2003). Attentional selection during preparation of prehension movements. Visual Cognition, 10, 409–431. [CrossRef]
Smeets, J. B. Brenner, E. (1999). A new view on grasping. Motor Control, 3, 237–271. [PubMed] [PubMed]
Smeets, J. B. Brenner, E. (2001). Independent movements of the digits in grasping. Experimental Brain Research, 139, 92–100. [PubMed] [CrossRef] [PubMed]
Smeets, J. B. Brenner, E. (2002). Does a complex model help to understand grasping? Experimental Brain Research, 144, 132–135. [PubMed] [CrossRef] [PubMed]
Smeets, J. B. Hayhoe, M. M. Ballard, D. H. (1996). Goal-directed arm movements change eye–head coordination. Experimental Brain Research, 109, 434–440. [PubMed] [CrossRef] [PubMed]
Snyder, L. H. Calton, J. L. Dickinson, A. R. Lawrence, B. M. (2002). Eye–hand coordination: Saccades are faster when accompanied by a coordinated arm movement. Journal of Neurophysiology, 87, 2279–2286. [PubMed] [Article] [PubMed]
Soechting, J. F. Engel, K. C. Flanders, M. (2001). The Duncker illusion and eye–hand coordination. Journal of Neurophysiology, 85, 843–854. [PubMed] [Article] [PubMed]
Stritzke, M. Trommershäuser, J. (2007). Eye movements during rapid pointing under risk. Vision Research, 47, 2000–2009. [PubMed] [CrossRef] [PubMed]
Vercher, J. L. Magenes, G. Prablanc, C. Gauthier, G. M. (1994). Eye–head–hand coordination in pointing at visual targets: Spatial and temporal analysis. Experimental Brain Research, 99, 507–523. [PubMed] [CrossRef] [PubMed]
Vishwanath, D. Kowler, E. (2003). Localization of shapes: Eye movements and perception compared. Vision Research, 43, 1637–1653. [PubMed] [CrossRef] [PubMed]
Vishwanath, D. Kowler, E. (2004). Saccadic localization in the presence of cues to three-dimensional shape. Journal of Vision, 4, (6):4, 445–458, http://journalofvision.org/4/6/4/, doi:10.1167/4.6.4. [PubMed] [Article] [CrossRef]
Vishwanath, D. Kowler, E. Feldman, J. (2000). Saccadic localization of occluded targets. Vision Research, 40, 2797–2811. [PubMed] [CrossRef] [PubMed]
Wing, A. M. Fraser, C. (1983). The contribution of the thumb to reaching movements. Quarterly Journal of Experimental Psychology, 35, 297–309. [PubMed] [CrossRef] [PubMed]
Yarbus, A. (1967). Movements of the eyes. New York: Plenum.
Figure 1
 
(A) Schematic depiction of the setup and the way of grasping in Experiment 1. In the actual experiment, there was no fixation point present during grasping. (B) The shapes that were grasped in Experiment 1: a square, a triangle (respectively ‘upward pointing’ and ‘downward pointing’), and a cross (respectively ‘upward pointing’, ‘rightward pointing’, ‘downward pointing,’ and ‘leftward pointing’). The ellipses and circles indicate the regions where the participants were allowed to contact the objects.
Figure 1
 
(A) Schematic depiction of the setup and the way of grasping in Experiment 1. In the actual experiment, there was no fixation point present during grasping. (B) The shapes that were grasped in Experiment 1: a square, a triangle (respectively ‘upward pointing’ and ‘downward pointing’), and a cross (respectively ‘upward pointing’, ‘rightward pointing’, ‘downward pointing,’ and ‘leftward pointing’). The ellipses and circles indicate the regions where the participants were allowed to contact the objects.
Figure 2
 
Example trial of the trajectories of eye (gray), thumb and index finger (black) when grasping an upward pointing triangle. Fixations are depicted as crosses, the average fixation in this trial as a star. The circles indicate the positions of the markers that were attached to the digits at the moment that the object was reached.
Figure 2
 
Example trial of the trajectories of eye (gray), thumb and index finger (black) when grasping an upward pointing triangle. Fixations are depicted as crosses, the average fixation in this trial as a star. The circles indicate the positions of the markers that were attached to the digits at the moment that the object was reached.
Figure 3
 
Eye–hand timing and general fixation characteristics in Experiment 1. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in the grasping task of Experiment 1, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the grasping (solid circles) and the viewing (empty squares) task. (C) Average number of fixations per trial for each shape, separately for the grasping (solid circles) and the viewing (empty squares) task. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape. For trials with only one fixation (data points to the far right), the data is collapsed over shapes. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data for grasping, empty symbols represent the data for the viewing task. The error bars in all panels are standard errors of the mean. Note that the standard error of the mean reflects within- as well as between-subjects variance and therefore should not be interpreted as being indicative for significance in our repeated measure design.
Figure 3
 
Eye–hand timing and general fixation characteristics in Experiment 1. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in the grasping task of Experiment 1, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the grasping (solid circles) and the viewing (empty squares) task. (C) Average number of fixations per trial for each shape, separately for the grasping (solid circles) and the viewing (empty squares) task. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape. For trials with only one fixation (data points to the far right), the data is collapsed over shapes. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data for grasping, empty symbols represent the data for the viewing task. The error bars in all panels are standard errors of the mean. Note that the standard error of the mean reflects within- as well as between-subjects variance and therefore should not be interpreted as being indicative for significance in our repeated measure design.
Figure 4
 
The locations of all fixations made by all participants in the grasping task of Experiment 1, separately for each shape.
Figure 4
 
The locations of all fixations made by all participants in the grasping task of Experiment 1, separately for each shape.
Figure 5
 
The locations of all fixations made by all participants in the viewing task of Experiment 1, separately for each shape.
Figure 5
 
The locations of all fixations made by all participants in the viewing task of Experiment 1, separately for each shape.
Figure 6
 
Fixation locations for each shape and each task (grasping in black, viewing in gray) in Experiment 1. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects.
Figure 6
 
Fixation locations for each shape and each task (grasping in black, viewing in gray) in Experiment 1. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects.
Figure 7
 
The maximal absolute vertical distance between any two fixations within one trial in the grasping task of Experiment 1, separately for each shape. Error bars are standard errors of the mean.
Figure 7
 
The maximal absolute vertical distance between any two fixations within one trial in the grasping task of Experiment 1, separately for each shape. Error bars are standard errors of the mean.
Figure 8
 
Schematic depiction of the setup and the way of grasping in Experiment 2, when the shapes were presented on the left. The mirror image of this depiction would represent the situation when shapes were presented on the right. In the actual experiment, there was no fixation point present during grasping.
Figure 8
 
Schematic depiction of the setup and the way of grasping in Experiment 2, when the shapes were presented on the left. The mirror image of this depiction would represent the situation when shapes were presented on the right. In the actual experiment, there was no fixation point present during grasping.
Figure 9
 
Eye–hand timing and general fixation characteristics in Experiment 2. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in Experiment 2, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the right (solid circles) and left (empty squares) presentation side. (C) Average number of fixations per trial for each shape, separately for the right (solid circles) and left (empty squares) presentation side. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape and presentation side. For trials with only one fixation (data point to the far right), the data is collapsed over shapes and presentation sides. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data when the shape was presented on the right, empty symbols represent the data when the shape was presented on the left. The error bars in all panels are standard errors of the mean.
Figure 9
 
Eye–hand timing and general fixation characteristics in Experiment 2. (A) Starting times (gray) and arrival times (black) of thumb (solid circles) and index finger (empty squares) relative to the start of the first fixation in Experiment 2, separately for each shape. (B) Proportion of trials in which a certain number of fixations (1 to 5) were made, separately for the right (solid circles) and left (empty squares) presentation side. (C) Average number of fixations per trial for each shape, separately for the right (solid circles) and left (empty squares) presentation side. (D) Starting times of fixations relative to the disappearance of the fixation dot. For trials in which more than one fixation were made, the data are separately presented for each shape and presentation side. For trials with only one fixation (data point to the far right), the data is collapsed over shapes and presentation sides. The upper row of (black) data points are the starting times for the second fixations, the lower row (gray) for the first fixations. Solid symbols represent the data when the shape was presented on the right, empty symbols represent the data when the shape was presented on the left. The error bars in all panels are standard errors of the mean.
Figure 10
 
The locations of all fixations made by all participants in Experiment 2, separately for each shape.
Figure 10
 
The locations of all fixations made by all participants in Experiment 2, separately for each shape.
Figure 11
 
Fixation locations for each shape and presentation side: right in black, left in gray. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by the squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects. The hands and dots outside the shapes represent the hand that was used and the side of the initial fixation dot when the shape was presented on the right (black) and on the left (gray).
Figure 11
 
Fixation locations for each shape and presentation side: right in black, left in gray. The star is the mean average fixation location with error bars representing the standard error of the mean in horizontal and vertical direction. The data points indicated by the squares and circles represent trials in which more than one fixation was made with the former representing the location of the first fixation and the latter the location of the second fixation. An arrow connecting the two emphasizes the average direction and length of the saccade. The ‘x’ indicates the center position of each shape and the (0,0) point in our fixation reference frame (defined as the average location of the shape's opposite sides in the horizontal and vertical direction). The ‘o’ indicates the location of the COG (the average location of the pixels constituting the shape). These symbols were not actually present on the objects. The hands and dots outside the shapes represent the hand that was used and the side of the initial fixation dot when the shape was presented on the right (black) and on the left (gray).
Table 1
 
Experiment 1—grasping task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Table 1
 
Experiment 1—grasping task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Shape Vertical difference (cm) t 9 p Horizontal difference (cm) t 9 p
Square 1.02 3.46 <0.01* −2.21 −5.81 <0.01*
Up cross 0.21 1.86 0.10 −1.77 −6.22 <0.01*
Right cross 0.58 1.91 0.09 −1.55 −3.02 0.01*
Down cross 1.91 3.41 <0.01* −1.37 −3.59 <0.01*
Left cross 0.85 3.87 <0.01* −1.73 −4.16 <0.01*
Up triangle 1.84 5.01 <0.01* −1.59 −5.11 <0.01*
Down triangle −0.61 −1.86 0.10 −1.39 −4.99 <0.01*
Table 2
 
Experiment 1—viewing task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Table 2
 
Experiment 1—viewing task: mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Shape Vertical difference (cm) t 9 p Horizontal difference (cm) t 9 p
Square −0.10 −0.35 0.73 −1.50 −4.85 <0.01*
Up cross −0.18 −1.06 0.32 −0.99 −3.68 <0.01*
Right cross 0.04 0.11 0.92 −1.35 −3.25 0.01*
Down cross 0.78 4.41 <0.01* −0.95 −3.62 <0.01*
Left cross −0.11 −1.33 0.22 −1.08 −5.40 <0.01*
Up triangle 0.28 1.97 0.08 −1.35 −4.13 <0.01*
Down triangle 0.24 1.20 0.26 −1.19 −3.91 <0.01*
Table 3
 
Experiment 2—mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Table 3
 
Experiment 2—mean vertical and horizontal differences between the first and second fixations (positive values indicate upward and rightward movements) and results of one sample t-tests.
Side Shape Vertical difference (cm) t 9 p Horizontal difference (cm) t 9 p
Right Square −0.05 −0.21 0.84 0.98 2.37 0.04*
Right Right triangle −0.31 −0.97 0.36 2.59 4.59 <0.01*
Right Center triangle −0.15 −0.42 0.68 0.47 1.17 0.27
Left Square −0.10 −0.32 0.76 −0.65 −0.94 0.37
Left Right triangle 0.54 2.05 0.07 −0.33 −0.69 0.51
Left Center triangle 0.60 2.66 0.03* −2.07 −3.74 <0.01*
© 2009 ARVO
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×