Gaze when reaching to grasp a glass

Abstract

People have often been reported to look near their index finger's contact point when grasping. They have only been reported to look near the thumb's contact point when grasping an opaque object at eye height with a horizontal grip—thus when the region near the index finger's contact point is occluded. To examine to what extent being able to see the digits' final trajectories influences where people look, we compared gaze when reaching to grasp a glass of water or milk that was placed at eye or hip height. Participants grasped the glass and poured its contents into another glass on their left. Surprisingly, most participants looked nearer to their thumb's contact point. To examine whether this was because gaze was biased toward the position of the subsequent action, which was to the left, we asked participants in a second experiment to grasp a glass and either place it or pour its contents into another glass either to their left or right. Most participants' gaze was biased to some extent toward the position of the next action, but gaze was not influenced consistently across participants. Gaze was also not influenced consistently across the experiments for individual participants—even for those who participated in both experiments. We conclude that gaze is not simply determined by the identity of the digit or by details of the contact points, such as their visibility, but that gaze is just as sensitive to other factors, such as where one will manipulate the object after grasping.

Introduction

People typically use visual information to guide their movements. The deployment of gaze has been investigated during many activities, such as driving (Wilkie & Wann, 2003), performing sports (Mann, Spratford, & Abernethy, 2013), or preparing breakfast (Hayhoe, Shrivastava, Mruczek, & Pelz, 2003; Land, Mennie, & Rusted, 1999). In general, people display fixation patterns that optimize information uptake for the planning and control of the upcoming action. When performing arm movements toward objects, gaze is shifted to the object of interest before the hand starts moving toward it (Johansson, Westling, Backstrom, & Flanagan, 2001; Smeets, Hayhoe, & Ballard, 1996), presumably because obtaining visual information about the circumstances near where one is going to act is beneficial for the control of goal-directed hand movements (Schlicht & Schrater, 2007; Volcic & Domini, 2014; Voudouris, Smeets, & Brenner, 2012). When moving a single finger to a specified position, one's gaze remains anchored to that position (Neggers & Bekkering, 2000). 

However, sometimes there is more than one relevant position, such as when simultaneously reaching with both hands to two different targets. In that situation, people typically look at the two targets sequentially with the fixation just before contact being toward the more difficult target: either the one that will be reached with the nondominant hand or the smaller target (Riek, Tresilian, Mon-Williams, Coppard, & Carson, 2003). We, therefore, consider gaze at the moment of contact to be most critical. When grasping an object, people need to choose appropriate contact points for two or more digits (Voudouris, Brenner, Schot, & Smeets, 2010) and bring their digits to those points (Cuijpers et al., 2004; Smeets & Brenner, 1999). When using a precision grip, the thumb and index finger make contact with the object almost synchronously (Voudouris et al., 2012; Voudouris, Smeets, & Brenner, 2016), but gaze can obviously only be directed to one position at a time. Where do people look when their digits make contact with the object? 

Given the strong tendency to look at the target in single-digit pointing, it is surprising that, when grasping, people do not appear to be particularly keen on their chosen contact points being visible (Voudouris et al., 2012), and they seldom choose to look at one of the contact points around the time of the grasp even if both contact points are visible (de Grave, Hesse, Brouwer, & Franz, 2008). Instead, they rather look somewhere between those points (de Grave et al., 2008), presumably because doing so allows them to guide both of the digits to their respective contact points. People often fixate closer to the position that will be contacted by the index finger. The critical gaze bias toward the index finger does not seem to be due to biomechanical reasons as it has been found for a variety of grasping postures (Brouwer, Franz, & Gegenfurtner, 2009; Bulloch, Prime, & Marotta, 2015; Cavina-Pratesi & Hesse, 2013; de Grave et al., 2008; Prime & Marotta, 2013; Voudouris et al., 2016). What might be the reason for such gaze biases? 

The bias to fixate near the index finger's contact point becomes weaker when accuracy requirements are increased for the thumb (Brouwer et al., 2009), when the object's shape is asymmetric (Desanghere & Marotta, 2015), and when one is initially looking closer to the thumb's contact point (Voudouris et al., 2016). However, such circumstances do not induce a clear bias toward the thumb's contact point. The bias toward the index finger's contact point has been suggested to be related to the index finger being the first digit to make contact with the object (Cavina-Pratesi & Hesse, 2013), but we have shown that this cannot be the main reason (Voudouris et al., 2016). Here, we explore whether we can explain these biases by the (lack of) visibility of the contact points. 

In many daily activities, such as picking up a cup from a table, the final part of the index finger's trajectory is hidden from sight by the object itself. When grasping in this configuration, people tend to look closer to the index finger's contact point throughout the grasping movement (Belardinelli, Stepper, & Butz, 2016) despite the index finger's contact point not being visible. When people grasp an object with the index fingers of both hands and the object is oriented in such a way that one of the surfaces by which it is grasped is occluded by the object itself, they fixate near the contact point that is on the object's occluded surface (Voudouris et al., 2016). The similarity with gaze when grasping with finger and thumb, with the finger making contact with a surface that is occluded by the object, suggests that occlusion, rather than the specific digit, might be critical when picking an object off a table. 

Why might the occlusion of the contact point make one look in that direction? One possibility is that occlusion makes it more likely that one will discover some unexpected constraint for the finger during the action. Another possibility is that, because one lacks direct visual information to guide the tip of the finger, the last part must be extrapolated, which requires more precise information. Either of these explanations is consistent with occlusion of the finger's contact point making one look closer to that contact point unless the index finger is completely occluded by the object, in which case constraints are invisible and extrapolation is impossible. This might explain why people seem to fixate near their thumb's contact point at the time of their grasp when the target object is at eye height and is grasped with the index finger on the far side of the object so that it completely occludes the last part of the index finger's path (Grant, 2015; Johansson et al., 2001). Changes in how the hand occludes parts of the object might also be one of the reasons why gaze differs between normal grasping at eye height and awkward, upside-down grasping of the same objects in order to turn them around (Belardinelli et al., 2016). Before trying to distinguish between the different reasons why occlusion of the region near a contact point may be critical, we set out to evaluate whether occlusion is really a factor that determines where people fixate an object during grasping. 

To investigate whether people only fixate their thumb's contact point when the whole region surrounding their index finger's contact point is completely occluded, we asked participants to grasp a glass filled with water (transparent) or milk (opaque) and which was placed at eye height (seen from the side) or approximately at hip height (seen more or less from above). To encourage participants to grasp carefully, we asked them to pour the contents of the glass into another glass. The glass was oriented such that the required grip would place the thumb and index fingers in positions that corresponded with different gaze directions. This was particularly important for the glasses at eye height because we had to be able to distinguish between fixations near the thumb's and the index finger's contact points despite them being at about the same height in the visual field. 

When the glass is at eye height, the area around the index finger's contact point is visible through the glass of water but not through the glass of milk. The deformations of what one sees through the glass of water may even make it especially important to look carefully to interpret what one sees. The deformations might alternatively attract gaze because the visual feedback does not match one's expectations. Thus, if the extent of the occlusion is an important factor, people might look near the index finger's contact point around the time of the grasp when the glass is full of water. When it is full of milk, we expect people to look near the thumb's contact point in accordance with earlier findings (Grant, 2015). When the glass is at hip height, the index finger's contact point is close to the occluding edge because the glass is largely seen from above, so we expect participants to fixate near their index finger's contact point irrespective of whether the glass is full of water or milk. 

Of course, we may be mistaken in assuming that the bias to look closer to the index finger's contact point has something to do with the visibility of the contact points. Moreover, it is possible that the deformations of the image as seen through the glass when it is at eye height make it difficult to interpret the visual information. In this case, rather than trying to obtain information by looking through the glass, such information may be ignored so that whether or not the liquid in the glass is transparent becomes irrelevant. In either of these cases, gaze will not be affected by the kind of liquid. Therefore, we will only see an influence of where the glass is placed with gaze being closer to the thumb's contact point for glasses at eye height and closer to the index finger's contact point for glasses at hip height. 

Experiment 1

Methods

Participants

Ten participants took part in the experiment (P1–P10; six women, four men; mean age: 28 years; range: 21–33 years). Participants were right-handed by self-report, had normal or corrected-to-normal vision, and were naïve as to the purpose of the experiment. Prior to the experiment, participants gave their written informed consent as approved by the ethics committee of the Faculty of Human Movement Sciences of the Vrije Universiteit Amsterdam. 

Apparatus and experimental setup

The experiment was performed in a normally illuminated room. Participants stood in front of a height-adjustable table (122 cm wide, 60 cm deep, adjustable height) on top of which a pedestal (27 cm tall, 3 cm diameter) was placed. The pedestal, attached to a thin wooden board, was approximately 25 cm in front of the participant's right shoulder. We used a glass that encouraged participants to place their digits at certain places: it had a square base of 6 × 6 cm with corners that rounded off slightly as the sides diverged toward the top of the glass. The glass was 9 cm in height, and the opening at the top varied in “diameter” between 8 and 8.5 cm as one moved from the centers of the surfaces to the “corners.” It weighed about 460 g when filled with liquid. No one reported any difficulties grasping this glass with a precision grip. 

The advantage of using a glass of this shape is that we could place and orient it in a manner that encouraged participants to pick contact points that corresponded to different gaze directions. This is important because it allowed us to determine whether gaze is oriented closer to the thumb's or the index finger's contact point. The glass was either filled with water or with milk. It was filled until approximately 1 cm below the rim. The glass was carefully placed on the pedestal. A second, identical, empty glass was placed on the table approximately 20 cm to the left of the pedestal. A small mark on the table, 25 cm to the right of the target glass, indicated the hand's starting position. A schematic depiction of the setup is shown in Figure 1. 

Figure 1

 

Setup of Experiment 1 as seen from (a) the side and (b) above. The target glass is depicted at eye height. The red part of the base of the pedestal represents the starting LED, which had to be fixated at the beginning of each trial. The height of the target glass and the starting LED are not visible in panel b. The black point at the right indicates the hand's starting position. The empty glass to the left of the participant is on the surface of the table.

The position and orientation of the participant's head, the positions of her or his right thumb and index finger, and the position of one fixed point in space were all recorded at 100 Hz with an Optotrak 3020 motion-tracking system (Northern Digital, Waterloo, ON, Canada) that determined the positions of infrared markers with two cameras (resolution of about 0.1 mm). Three infrared markers were attached to an individual dental-impression bite-board that the participant held in her or his mouth. These three markers were used to determine the position and orientation of the head. Single infrared markers were attached to the nails of the participant's right thumb and index finger. An additional single marker was attached to the lower part of the pedestal and was aligned along the participant's line of sight with the starting LED (a light-emitting diode that participants had to fixate at the beginning of each trial and that was also placed at the bottom of the pedestal). Eye movements of both eyes were recorded at 500 Hz (resolution of about 0.2°) with an Eyelink II eye tracker (SR Research Ltd.). Three additional LEDs were used for the calibration of the eye-movement recordings, each of which was aligned with an infrared marker along the participant's line of sight. 

Calibration procedure

We used the same calibration procedure as in our previous study (Voudouris et al., 2016) and restrict ourselves here to a rough description. In order to know where people were looking while allowing them to make head movements, we first related measured positions of the markers on the bite-board to positions of the participant's eyes. This was done for every participant before starting the experiment. Having done this, we could determine the positions of the eyes and the orientation of the head throughout the experiment by measuring the positions of the markers on the bite-board. 

In a second step, we related the output of the eye tracker to orientations of the eye relative to the head. Four calibration LEDs were placed on the table in a rectangular configuration. Each LED was accompanied by an infrared marker that was aligned with the LED along the participant's line of sight. The eye tracker was attached to the participant's head while she or he held the bite-board in her or his mouth. The participant stood in front of the table, and the eye tracker cameras were adjusted so that they did not occlude critical parts of the work space. One of the LEDs was illuminated, and the participant had to fixate it. Once a fixation was detected, a tone was presented, the LED turned off, and the next LED was illuminated, prompting the participant to make a saccade to fixate the illuminated LED. The calibration continued until six cycles of fixations of the four LEDs were performed (24 fixations in total). The camera coordinates of each pupil as determined by the eye tracker were combined with the orientation of the head and the directions from the eye to the LEDs to determine how the eye-tracker data corresponds with eye orientations relative to the head. After the calibration, the experimental session began. 

Experimental procedure

The height of the table was adjusted so that the target glass was either at the participant's eye height so that the participant could see the glass from the side or at hip height so that the participant could see the glass more or less from above. In both cases, the target glass was placed on the pedestal. Care was taken that the cables of the infrared markers on the bite-board, thumb, and index finger did not hinder the participant's movements. Each trial started with the participant bringing the hand to the starting position and fixating the illuminated starting LED. The experimenter placed the glass on the pedestal and started the recording with a button press. The starting LED then turned off, indicating that the participant could start moving her or his eyes and arm in order to grasp the glass between thumb and index finger. Once the glass was grasped, the participant had to carefully pour its contents into the empty glass that was located to the participant's left. Once its contents were poured, the participant placed the target glass next to the other glass and brought the hand and gaze back to their respective starting positions in anticipation of the next trial. 

Participants were instructed to start the trial with their digits at the starting position and fixating the starting LED and to grasp the target glass with their thumb and index finger within 5 s after the starting LED turned off. Participants' vision was not restricted at any time during the experiment, and they received no further instructions about where to look, how to grasp the glass, or how soon after the cue or how quickly they should move. We presented the two height conditions (eye height, hip height) in two separate blocks of trials. Within each of these blocks, each of the other two conditions (water, milk) was presented 22 times, resulting in 44 trials that were presented in a pseudo-randomized order. The order of the blocks was counterbalanced across participants. The experiment lasted approximately 30 min. 

Data analysis

To determine where the participants' digits made contact with the glass, we first determined the onset of the grasping movement: This was the first moment at which the average position of the markers on the two digits exceeded a velocity threshold of 10 cm/s. We then calculated the end of the movement separately for each digit based on the multiple sources of information method (Schot, Brenner, & Smeets, 2010): The moment of contact had to be after the maximal three-dimensional opening of the grip, and the likelihood of a frame being the moment of contact decreased with the digit's tangential velocity (likelihood ∼ 1/velocity) and increased with the absolute value of its tangential acceleration (likelihood ∼ acceleration; there was a distinct deceleration peak due to the contact with the glass). For each digit, the moment at which the product of these likelihoods was greatest was considered to be the end of this digit's movement and the position of the digit at that moment to be the digit's contact point. We expressed the distance between the contact points of the two digits (grip size) as a visual angle to be able to relate the angular position of gaze to the contact points of the digits. 

In order to analyze gaze, we first related the raw output of the eye tracker to visual directions with respect to the head on the basis of the calibration. The visual directions of the two eyes were then averaged. Next, we synchronized the eye-tracker values (measured at 500 Hz) with the positions of the Optotrak markers (measured at 100 Hz) to obtain a combined 100 Hz signal by choosing the eye-tracker value that was closest in time to each set of Optotrak values. We did not try to estimate where participants were looking in depth, but used the positions of the eyes as derived from the markers on the bite-board to define a polar reference frame with its origin halfway between the two eyes (“cyclopean eye”). We considered the direction connecting the eyes to be horizontal. Both the gaze direction and the positions of the Optotrak markers on the nails of the thumb and index finger were expressed in this head-centered polar coordinate system. 

We were interested in where people were looking when they grasped the glass (hereafter referred to as critical fixation). For this, we took the moment at which the first digit made contact with the glass. If the eyes were moving at more than 35°/s at that moment, indicating that a saccade was being made, we took the last sample before the saccade (i.e., before the eyes moved at more than 35°/s). We determined the visual angle between the critical fixation and each of the two contact points. The difference between these two visual angles indicates whether participants' critical fixations were closer to the thumb's or to the index finger's contact point. We refer to this difference as the critical gaze bias. A critical gaze bias of zero indicates that the critical fixation was exactly equidistant from the two contact points. A positive value indicates fixating closer to the index finger's than the thumb's contact point. We also calculated the contact asynchrony. This is the difference between the time the index finger and thumb made contact with the glass with positive asynchronies indicating that the index finger made contact first. 

The critical gaze bias was calculated for each trial and then averaged across repetitions for each condition and participant. The influence of the glass position (eye height, hip height) and the glass contents (water, milk) on these average values was evaluated with a two-way, repeated-measures ANOVA. If people look near their index finger's contact point unless the whole region near the contact point is hidden from view, we expect an interaction between position and content. If there is some other reason for looking closer to the thumb when grasping an object at eye height, we expect a main effect of position. If the reason for looking closer to the thumb in previous studies with objects at eye height was unrelated to the object's position, we do not expect any effects. We evaluated the possible relationship between contact asynchrony and gaze bias by determining the correlation between the participant's contact asynchrony and her or his gaze bias across all trials. We did this separately for each experimental session. 

Results

For individual participants, both the digits' trajectories and the gaze trajectories are reproducible across trials (the trajectories of one participant are depicted in Figure 2). Participants always grasped the glass with their index finger to the right of their thumb. For this example, participant P4, the critical fixation position depended on the position of the glass. When the glass was at eye height (Figure 2a and b), gaze was directed somewhere below the thumb's contact point irrespective of the content of the glass. When the glass was at hip height (Figure 2c and d), gaze was usually close to the index finger's contact point. 

Figure 2

 

Gaze and digit trajectories of participant P4 in Experiment 1 for a glass at (a, b) eye height and (c, d) hip height, filled (a, c) with water and (b, d) with milk. The trajectories are represented within a head-centric polar coordinate system (rather than in a Cartesian one) with a fixed scale for all four panels. Therefore, the dimension and orientation of the drawn glass differ systematically between the two heights: The drawn glass is smaller when at hip height because it is then further away from the eyes. Red and cyan curves represent the trajectories of the thumb and the index finger, respectively. Green curves represent the trajectories of gaze until the moment of the critical fixation with black dots at the end representing the positions of the critical fixations.

Although all participants showed systematic fixations within a task (small error bars in Figure 3), the fixation behavior differed considerably across participants. For both positions of the glass and for glasses with both water and milk, critical fixations were closer to the thumb's than to the index finger's contact point for most participants (Figure 3). This did not differ systematically between the two glass positions, F(1, 9) = 0.43, p = 0.84, but the critical fixations were further toward the thumb's contact point when the glass was filled with milk than when it was filled with water, F(1, 9) = 9.61, p = 0.013, without a significant interaction between position and content, F(1, 9) = 3.79, p = 0.08. On average, the critical gaze bias was −1.2° ± 2.3° (mean ± standard deviation across participants) when the glass was filled with water and −1.8° ± 2.3° when it was filled with milk. Although this difference is significant, it is negligible in comparison with the differences between participants. Importantly, the tendency to look closer to the thumb was certainly not only present for glasses of milk at eye height as we had expected, but also for glasses at hip height. Filling the glass with milk rather than water even biased the critical fixations toward the thumb for more participants when the glass was at hip height than when the glass was at eye height. 

Figure 3

 

Results of Experiment 1 for when the glass was placed at (a) eye height and (b) hip height. Critical gaze bias (how much closer gaze was to the thumb's than to the index finger's contact point at the moment of contact) for each glass contents and for each participant (with the standard error across each participant's trials). Each color represents a participant. Negative values indicate that the critical fixation is closer to the thumb's contact point.

When the glass was at eye height, the participant whose data are depicted in Figure 2 fixated considerably below the digits' contact points. Figure 4 illustrates the distance between the critical fixation and each of the two contact points for each trial performed by each participant. When the glass was at eye height, the sum of the distances between the critical fixation and each of the contact points was often much larger than the distance between the digits at the moment of grasp (points often far from the gray area, Figure 4a). This is because, for glasses at eye height, participants frequently fixated considerably below instead of between their contact points. Thus, the bias was not really toward looking at the thumb's contact point. 

Figure 4

 

Critical fixation locations in Experiment 1 when the glass was placed at (a) eye height and (b) hip height. Distance between the critical fixation and each of the two contact points for each trial performed by each participant, normalized by grip size. For the sum of the distances to the two contact points to be much larger than the distance between those points (points far from the gray area; sum of normalized distances considerably larger than one), participants must have fixated far from their contact points. This often happens when the glass is at eye height (panel a). It is much less common when the glass is at hip height (panel b). Color-coding as in Figure 3.

When the glass was at hip height, for most trials the sum of the distances between the critical fixation and each contact point approximates the grip size (points near gray area, Figure 4b), implying that the critical fixation was near the line segment connecting the contact points. This means that the negative critical gaze bias that was seen for most participants when grasping at hip height (Figure 3b) really corresponds to fixating near the thumb's contact point. 

When the glass was at eye height, the index finger touched the object 40 ms earlier than the thumb (median contact asynchrony). There was a significant negative correlation between this contact asynchrony and gaze bias (r = −0.31, p < 0.001): Participants looked closer to the thumb's contact point when the index finger touched the glass earlier. This is the opposite of what Cavina-Pratesi and Hesse (2013) found. When the glass was at hip height, the median contact asymmetry was 45 ms and was not correlated with the gaze bias (r = −0.02, p = 0.6). 

Discussion

We asked participants to grasp a glass of water or milk that was placed either at eye or hip height and to empty its contents into another glass on their left. We intentionally used a glass filled with a liquid and introduced a subsequent action with that glass to encourage a careful selection of contact points. We find no evidence that not being able to see the region near the index finger's contact point (glass with milk at eye height) specifically biases participants to look at the thumb's contact point. 

Looking through the glass of water shifts the visual position of the index finger as it approaches the glass, changing the relation between the seen and the felt position of the finger. It also shifts visually perceived details about the far side of the glass. If we had mainly found differences between gaze biases when grasping at eye height, we would have had to determine the precise reason. It could have been the result of the space behind the glass only being visible when the glass is full of water or of the glass of water optically displacing images of structures such as the approaching finger to the right or of gaze being attracted by the visual–haptic mismatch that needs to be resolved. However, the influence of the contents of the glass is certainly not stronger when the glass is at eye than hip height. We do not know why many of our participants look slightly closer to their thumb when the glass is full of milk than when it is full of water irrespective of the position of the glass (or possibly even more so when the glass is at hip height). 

The most surprising finding is that we did not find the usual tendency to look close to the index finger's contact point when the glass was at hip height, a condition in which we and others have previously observed such behavior (Brouwer et al., 2009; Cavina-Pratesi & Hesse, 2013; Voudouris et al., 2016). Instead, we found an overall tendency to look closer to the thumb's contact point. This bias was not related to the thumb contacting the glass earlier than the index finger in this study because the index finger generally made contact first. Moreover, we did not find a positive correlation between contact asynchrony and gaze bias. When the glass was at eye height, participants frequently looked considerably below both contact points, closer to the base of the glass. The tendency to look closer to the thumb may, therefore, not be related to the digits' contact points at all. 

When only considering glasses at eye height, one might propose that participants were biased toward directing gaze at the glass's centers of mass, which would be consistent with previous findings (Brouwer et al., 2009; de Grave et al., 2008). People generally grasp objects above their centers of mass (Voudouris et al., 2010), so a bias toward fixating the glass's center of mass would direct gaze below the contact points as we observe. Fixating between the contact points and the base of the glass's support might also be an effective strategy to obtain visual information at all relevant locations (keeping both the contact points and the small base of the glass's support at an acceptable retinal eccentricity). However, for most participants, gaze was biased toward the thumb also when the glass was at hip height, in which case the glass's center of mass was between the two contact points in terms of visual angle, and the base of the glass's support was not visible at all (at least when the glass was full of milk). 

Another possible explanation for the unexpected tendency to look closer to or even below the thumb's contact point is that, after grasping the glass, the participants had to pour its contents into a second glass that was always to their left. In most previous studies, participants only had to lift the target object. We added the complexity of pouring the liquid into another glass in order to encourage the participants to grasp carefully. Might the fact that a careful action (pouring) had to be performed on the left have made participants fixate further to the left, which coincides with fixating closer to the position of their thumb's contact point? It is less likely that gaze is biased toward the thumb as a result of the thumb's contact point being more important for pouring because that would not explain why participants looked lower than the thumb when the glass was at eye height. When the glass with milk or water was at eye height, the glass into which the liquid had to be poured was considerably lower in the visual field. If the subsequent task determines the fixation bias, the original bias found in previous studies might primarily have been caused by the lifting task because the index finger is usually higher in the visual field than the thumb when grasping (when grasping an object from a table, the finger and thumb are usually at about the same height, but the index finger is usually further away from the body so that it is higher in the visual field). 

Experiment 2

In this experiment, we examined whether the critical fixation is biased toward where participants will act after grasping the glass. We also examined whether the precision of the subsequent action is important. Eight participants (P1–P4 and P11–P14; five women, three men; mean age: 29 years; range: 25–33 years) took part in this experiment. Four of them (P1–P4) had also participated in Experiment 1. Except for the details mentioned below, the apparatus, procedure, and data analysis were identical to those of Experiment 1. We always filled the target glass with water and placed it at hip height directly on the table. Because the target glass was placed directly on the table and not on the pedestal, the starting LED could not be placed at the same location as in Experiment 1. It was moved to the hand's starting position (30 cm right of the target glass). In each trial, the experimenter placed either a coaster or an empty glass at a specific position on the left or right of the table (Figure 5). After grasping the glass, participants either had to simply place the target glass on the coaster or to pour the water into the empty glass. Participants could see the experimenter place the coaster or glass, so they knew which task they would be performing and where they would be performing it before the trial started. The two lateral positions at which the next action (place or pour) occurred were 50 cm from the original position of the target glass. The two tasks and two positions were presented in pseudo-random order within one block of 44 trials. The effects of the task (place vs. pour) and the position (left vs. right) on the critical gaze bias were evaluated with a 2 × 2 repeated-measures ANOVA. 

Figure 5

 

Setup of Experiment 2 as seen from (a) the side and (b) above. The target glass (large diamond) is always at hip height. The small black and red symbols in panel b indicate the hand's starting position and the starting LED, respectively. In addition, an empty glass (small diamond) or coaster (disk) was placed on the left or right side of the table. Details as in Figure 1.

If the critical fixations were closer to the thumb's contact point in Experiment 1 because the next action was always performed on the left, which was always closer to the thumb's contact point within the participants' visual field, we should find a main effect of position with a rightward bias (closer to the index finger's contact point) when the subsequent task was to be performed on the right and a leftward bias when it was to be performed on the left. If, contrary to our previous argument, the critical fixations were closer to the thumb's contact point because the thumb had to be placed more precisely in order to rotate the glass around the thumb when pouring the liquid, we should find a main effect of task with fixations closer to the thumb's contact point when pouring but not when placing the glass irrespective of the side at which these tasks were performed. 

Results

As anticipated, participants always grasped the glass with the index finger to the right of the thumb, so a positive bias always corresponds with looking farther to the right. The side at which the subsequent action took place did not have a consistent influence on where gaze was directed at the moment of contact with the glass, F(1, 7) = 4.26, p = 0.07. The fixation biases varied considerably across participants. The participants who were influenced most by the position of the subsequent action looked closer to their index finger (i.e., farther to the right) when the subsequent task was performed on the right than when it was performed on the left (open symbols below filled symbols in Figure 6a). However, some participants' gaze was not influenced by the position of the upcoming action. 

Figure 6

 

Results of Experiment 2. Different symbols indicate placing or pouring on the left or right. (a) Critical gaze bias for each task and position. (b) Distance between the critical fixation and each of the two contact points on all trials, normalized for grip size. Color-coding as in panel a. Details as in Figures 3 and 4.

There was no main effect of task (place or pour), F(1, 7) = 0.04, p = 0.84, or interaction between position and task, F(1, 7) = 0.03, p = 0.87. As the only two participants who appear to show a different gaze bias for subsequently placing or pouring (P1 and P2) have their biases in opposite directions, the difficulty of the subsequent task is probably not important in determining fixation locations. As was the case for the glasses at hip height in Experiment 1, gaze was directed usually more or less between the two digits' contact points (Figure 6b). We found a tendency to contact the glass with the index finger before doing so with the thumb (median asynchrony of 20 ms) and, similarly to Experiment 1, no correlation between the contact asynchrony and the gaze bias (r = −0.04, p = 0.2). 

Participants P1–P4 participated in both Experiments 1 and 2. In both experiments, there was a condition in which the task was to grasp a glass filled with water that was at hip height and pour its contents into another glass that was on the left. These participants' results for this task in the two experiments are shown in Figure 7. It is clear that even the participants have clear individual gaze biases in each experiment, but these differ in size or (for P4) even in sign under very similar conditions. 

Figure 7

 

Gaze biases of participants P1 to P4 for similar conditions in Experiments 1 and 2: grasping a glass of water at hip height and pouring its contents into a glass on the left. Details as in Figures 3 and 6.

Discussion

For several participants, the critical gaze was clearly closer to the thumb's contact point when the subsequent action was performed on the right than when it was performed on the left. However, the magnitude of the effect varied across participants (Figure 6a). There was no overall tendency for participants' critical fixations to be closer to their index finger's contact point as had been found in other studies (Brouwer et al., 2009; Bulloch et al., 2015; Cavina-Pratesi & Hesse, 2013; Voudouris et al., 2016) or closer to the thumb's contact point as we had found in Experiment 1. 

In Experiment 2, we positioned the starting LED at the hand's starting position. This was to the right of the participants. There are reports of a critical gaze bias toward the eyes' starting point (Brouwer et al., 2009; Voudouris et al., 2016). This is presumably a strategy to obtain new visual information sooner because the greater the eye movement amplitude, the longer the duration that the eyes are moving so fast that no useful information can be obtained (Harris, 1995). The reduction in the tendency to look closer to the thumb's contact point in Experiment 2, relative to Experiment 1, might be the result of gaze initially being directed to the right. However, it might also partly be due to the subsequent action not always being on the left or to other differences between the two experiments, such as the lower precision needed to grasp the glass when it is on the table rather than on a pedestal. Whatever the reason for this reduction, it confirms that critical fixation locations are sensitive to the exact nature of the task, so there is probably no single factor that can explain gaze behavior when grasping. 

General discussion

Our aim was to better understand fixation biases in grasping. Most previous studies show that people fixate near their index finger's contact point irrespective of the grasp configuration (Belardinelli et al., 2016; Brouwer et al., 2009; Cavina-Pratesi & Hesse, 2013; Voudouris et al., 2016). In two experiments, we examined to what extent fixation is biased by the visibility near the contact points and by the action performed after grasping. We found that visibility might matter but that not all participants look closer to the less visible contact point or closer to the more visible contact point. We also found that some participants' critical fixations are biased in the direction of the subsequent action that one will perform with the glass, but others' fixations are not. 

In Experiment 1, we found that for most participants critical fixations were closer to the thumb's than to the index finger's contact point (Figure 3). For glasses at eye height, such fixations were not really toward the thumb's contact point but were often well below both contact points, bringing gaze closer to the glass's narrow base of support and to where the upcoming pouring action would take place after the grasp. As critical fixations for glasses at hip height when the base of support was not visible were also closer to the thumb's contact point, we believe that the fixations for glasses at eye height were primarily influenced by the position of the upcoming action and not by the accuracy requirements imposed by the narrow base of support. In either case, it seems evident that gaze is not just directed toward a certain contact point but that other aspects, such as the requirements after the grasp, are also considered. This extends previous findings showing that people shift their gaze to where they will act in the future (Johansson et al., 2001; Land et al., 1999; Smeets et al., 1996). 

The suggestion that fixations are biased toward where the subsequent action will take place is supported to some extent by Experiment 2. In this experiment, some participants had a clear tendency to direct gaze toward the position of the upcoming placing or pouring action (open symbols below filled symbols in Figure 6a). However, not all participants had this tendency. Considering that the fixation locations are not likely to be arbitrary (Belardinelli, Herbort, & Butz, 2015), the idiosyncratic gaze patterns found in Experiment 2 (as well as Experiment 1) suggest that gaze, when grasping an object, is an individual compromise between looking near positions that are relevant for the grasp itself and looking at positions that are advantageous for other reasons, such as being close to where one was looking before (Voudouris et al., 2016) or where one anticipates to be looking next. The position itself rather than the specific action at that position seems to be the important factor because gaze when grasping to pour was very similar to gaze when grasping to simply place the glass elsewhere (open symbols often grouped and separated from filled symbols in Figure 6a). 

Previous studies have reported fixations toward the index finger's contact point when participants have been asked to grasp an object off a table and move it upward (Cavina-Pratesi & Hesse, 2013; Voudouris et al., 2016), detach an object from a vertical screen and hand it to the experimenter (Brouwer et al., 2009; de Grave et al., 2008) or place it on a table (Desanghere & Marotta, 2015), grasp an object and hold it without manipulating it at all (Prime & Marotta, 2013), or grasp an object with an upside-down grasp and pretend to drink from it or hand it to the experimenter (Belardinelli et al., 2016). Finding a bias toward the index finger in such diverse circumstances provides direct evidence that biomechanical constraints are not responsible for the choice of fixations (Cavina-Pratesi & Hesse, 2013). It makes it all the more surprising that we found a bias toward looking at the thumb in Experiment 1. 

When grasping and lifting an object, people may fixate higher on the object in order to direct their gaze closer to where the object will be in the future. This position might often coincide with the index finger's contact point, which, in such cases, is often higher in the visual field. However, it is not clear whether the actions reported in some of the studies involved moving the object upward in the visual field (Brouwer et al., 2009; de Grave et al., 2008). Moreover, optimizing gaze for future actions cannot explain biases when the task is simply to grasp and hold symmetrical objects without manipulating them at all (Bulloch et al., 2015; Prime & Marotta, 2013). Furthermore, although a bias related to the position of the subsequent action could account for some of our findings, such as some participants' tendency to look below the digits when the glass was at eye height in Experiment 1, Experiment 2, which was specifically designed to test this hypothesis, showed that this was not an important factor for all participants. The bias in the direction of the subsequent action was not consistent across participants, and it was quite small for most participants who did show a bias in that direction (Figure 6a). Thus, such a bias probably contributes to the selection of fixation locations to some extent but not more so than the other factors that have been identified (Voudouris et al., 2016). 

Four participants (P1–P4) took part in both experiments. In each experiment, they poured the contents of a glass of water positioned at hip height into an empty glass on the left. Although their critical fixations were very consistent within each experiment, the four participants' fixations were different in the two experiments (Figure 7). The most extreme example is P4, who clearly looked closer to the index finger in Experiment 1 and clearly looked closer to the thumb in Experiment 2. Because the gaze biases were quite consistent across each participant's trials within each experiment as shown by the small error bars, the differences between these participants' gaze in the two experiments suggest that the biases depend on the exact nature of the task. Multiple factors must be involved because the effects are not consistent across participants: P3 looks closer to the index finger's contact point in Experiment 2 than in Experiment 1, whereas the other three look closer to the thumb's contact point. 

One potential factor is that different participants are influenced to different extents by the initial gaze location (which differed between the two experiments). Another is that where one looks might be influenced by the other conditions in the experiment: In Experiment 1, participants always poured the contents of the glass on their left, whereas in Experiment 2, they only did so in a quarter of the trials (because only half the trials involved pouring, of which only half involved doing so on the left). Performing the same action repeatedly at the same position may let participants anticipate that visual information at that position will be needed. Alternatively, always performing the same action at the same position may make people more familiar with the position, so looking there as soon as one has grasped the glass may not be as important. We checked that participants' average gaze bias in the first six trials of each condition did not differ from the average bias in the remaining trials in either of the experiments and that the gaze biases in Experiment 2 did not depend on the previous site of the action and found no evidence that the trials cannot be treated independently. There are undoubtedly other factors that can contribute to where one looks. The large differences that we find between participants suggest that searching for the critical factor that determines gaze when grasping may be a futile endeavor. 

Choosing appropriate contact points is an important aspect of planning a grasping movement. People choose unusual contact points if doing so facilitates subsequent hand movements (Rosenbaum & Jorgensen, 1992; Sartori, Straulino, & Castiello, 2011), showing that people take future requirements into account. People do not opt for looking directly at one of the chosen contact points (de Grave et al., 2008; Voudouris et al., 2016); neither do they choose different contact points if their usual contact points are hidden from view (Voudouris et al., 2012). Both where one looks and how one grasps an object are evidently influenced by many factors other than the ability to guide the digits to their contact points. Here, we show not only that the position at which a future action with the grasped object will take place may be one of those factors, but also that such factors' influence on where people look when reaching to grasp an object differ considerably across individuals. Finally, the present study shows that gaze is probably not primarily directed to one of the digits' contact points as has often been assumed. 

Acknowledgments

This work was supported by the German Research Foundation (DFG) TRR 135. 

The data presented in this paper are available at: https://doi.org/10.5281/zenodo.1401845. 

Commercial relationships: none. 

Corresponding author: Dimitris Voudouris. 

Email: . 

Address: Experimental Psychology, Justus-Liebig University, Giessen, Germany.