Looking away from a moving target does not disrupt the way in which the movement toward the target is guided

Abstract

People usually follow a moving object with their gaze if they intend to interact with it. What would happen if they did not? We recorded eye and finger movements while participants moved a cursor toward a moving target. An unpredictable delay in updating the position of the cursor on the basis of that of the invisible finger made it essential to use visual information to guide the finger's ongoing movement. Decreasing the contrast between the cursor and the background from trial to trial made it difficult to see the cursor without looking at it. In separate experiments, either participants were free to hit the target anywhere along its trajectory or they had to move along a specified path. In the two experiments, participants tracked the cursor rather than the target with their gaze on 13% and 32% of the trials, respectively. They hit fewer targets when the contrast was low or a path was imposed. Not looking at the target did not disrupt the visual guidance that was required to deal with the delays that we imposed. Our results suggest that peripheral vision can be used to guide one item to another, irrespective of which item one is looking at.

Introduction

When performing daily life tasks, we normally direct our gaze at objects that we are interacting with or that we intend to interact with in the near future (Hayhoe & Ballard, 2005; Land, 2006; Land & Hayhoe, 2001; Neggers & Bekkering, 2000; Neggers & Bekkering, 2001; Smeets, Hayhoe, & Ballard, 1996; Voudouris, Smeets, & Brenner, 2012; Voudouris, Smeets, & Brenner, 2016). Looking at the target of a goal-directed arm movement is beneficial (Carson, Chua, Elliott, & Goodman, 1990; Carson, Goodman, Chua, & Elliott, 1993; Prablanc, Echallier, Komilis, & Jeannerod, 1979; Prablanc, Pélisson, & Goodale, 1986; Soechting & Flanders, 1989), probably in part because the spatial resolution at the position of the target is enhanced (Schütz, Braun, & Gegenfurtner, 2009). A higher resolution helps one to precisely localize the target and to adjust the ongoing movement if the target is displaced (Brenner & Smeets, 2011; Brenner & Smeets, 1997; Elliott, Binsted, & Heath, 1999; Oostwoud-Wijdenes, Brenner, & Smeets, 2011; Prablanc et al., 1986). If the target is moving, tracking it with one's eyes improves judgments regarding the future trajectory of the target (Spering, Schütz, Braun, & Gegenfurtner, 2011) and of its velocity (Goettker, Braun, Schütz, & Gegenfurtner, 2018; Goettker, Brenner, Gegenfurtner, & de la Malla 2019), especially under circumstances in which the velocity may otherwise be misjudged such as when the target and the background are isoluminant (Braun et al., 2008) or when the target contains local pattern motion that is different from the motion of the object as a whole, as is the case when a patterned ball is rolling across a surface (de la Malla, Smeets, & Brenner, 2017). 

Interestingly, even though manual interactions with static or moving objects involve our hand as well as the target, we almost never look at our hand when moving it to an object. We may not need to look at our hand because we have access to various sources of (efferent and afferent) proprioceptive information about the position and motion of the hand (Ernst & Banks, 2002; Kuling, Brenner, & Smeets, 2016; Rincon-Gonzalez, Warren, Meller, & Tillery, 2011; Sittig, Denier van der Gon, & Gielen, 1987; Sober & Sabes, 2005; van Beers, Sittig, & Denier van der Gon, 1996; van Beers, Sittig, & Denier van der Gon, 1999). Information about the target can only be obtained through vision. That proprioceptive information about the position and movement of the hand contributes to how ongoing arm movements are guided can be inferred from the fact that manipulating visual information about the target during a movement has a stronger effect than manipulating visual information about the hand (Berkinblit, Fookson, Smetanin, Adamovich, & Poizner, 1995; Elliott, 1988; Sarlegna et al., 2003). However, the fact that manipulating visual information about the hand does influence movements shows that people use visual information about the hand, as well. People react to changes in visual feedback about their hand's position or motion (Brenner & Smeets, 2003; Saunders & Knill, 2003; Saunders & Knill, 2004; Saunders & Knill, 2005; van Beers, Wolpert, & Haggard, 2002; van der Kooij, Overvliet, & Smeets, 2016). Moreover, the extent to which people can see their hand, or a representation of their hand, during goal-directed movements determines the accuracy of their movements (Bozzacchi, Brenner, Smeets, Volcic, & Domini, 2018; de la Malla, López-Moliner, & Brenner, 2012). Furthermore, when a visual representation tracks the unseen hand with a delay, as happens when using a teleoperation system or a computer mouse, the movement of the hand is adjusted to ensure that the delayed representation reaches the target or follows the correct trajectory (Cunningham, Billock, & Tsou, 2001a; Cunningham, Chatziastros, Von der Heyde, & Bülthoff, 2001b; de la Malla, López-Moliner, & Brenner, 2014; Rohde, van Dam, & Ernst, 2014). In a previous study (Cámara, de la Malla, López-Moliner, & Brenner, 2018), we found that this was true even when the delay between the movement of the occluded hand and that of the visual representation of the hand was unpredictable. In that case, the hand movements toward moving targets were adjusted to the delay in the trial in question, despite the delays being interleaved so that participants could not adapt to them. When participants look at a moving target while moving a delayed cursor toward it (Cámara et al., 2018) or while trying to track it with a cursor (Gouirand, Mathew, Brenner, & Danion, 2019), they must be using peripheral vision to guide the ongoing movement of the hand. 

The pervasive finding that people direct their gaze at the target of a goal-directed movement, even when there are inconsistencies between proprioception and visual information about one's own movement, made us wonder whether doing so is somehow essential for guiding the hand to the target. We suspected that this might not be the case, despite the many studies showing that people look at the target, because in daily life it is not uncommon to be confronted with additional simultaneous requirements, such as attending to what other people are doing. This is particularly evident in team sports involving a ball and when negotiating traffic. Our previous study (Cámara et al., 2018) has already shown that participants do not simply rely on guiding their hand toward where they are looking, because participants compensated for delays between the hand and the cursor. Here, we set out to examine how visual guidance of a goal-directed movement would change if participants were not looking at the target. 

As in one of the conditions of our previous study, we introduced an unpredictable delay between the hand and the cursor to force participants to adjust their movements. This allowed us to evaluate the extent to which participants use visual information to guide their ongoing movement. The participants’ task was to slide their unseen finger over a horizontal surface so that a cursor representing the finger would pass through a moving target. We know that participants look at the target rather than at the cursor when performing this task (Cámara et al., 2018). To encourage participants to look at the cursor rather than at the target, we gradually reduced the contrast between the cursor and the background during each session. We began with a high contrast so that participants would become accustomed to adjusting their movements to the varying delays. We reduced the contrast gradually because the contrast at which peripheral vision would no longer be sufficient for guiding the movement was not evident. The contrast was not reduced to zero because the cursor has to be visible for participants to look at it. The assumption was that making it more difficult to localize the cursor would entice participants to look at it. In a second experiment, we asked participants to move the cursor along a specified path toward similar targets. In this case, participants might want to look at the cursor to ensure that it remains on the visible path. The targets were quite large and moved in a completely predictable manner, minimizing the need to visually track them to know where they were and how they would move. We compared performance on trials in which participants spontaneously exhibited different kinds of gaze behavior, rather than explicitly instructing participants to look where they normally would not on some trials, so that participants would not have to prioritize between complying with the gaze instructions and hitting the targets. 

Methods

Participants

A total of 13 participants (7 females and 6 males between the ages of 21 and 61 years) took part in the experiments. Three of the participants were authors. Only the authors knew the purpose of the experiment. Four of the remaining participants were familiar with the task and setup because they had taken part in a former study of ours (Cámara et al., 2018), but they did not know the purpose of the current study. The six remaining participants did not know the purpose of the study and did not have previous experience with the task or with the setup. All participants had normal or corrected-to-normal vision (three participants wore contact lenses and five wore glasses). Three participants reported being left handed. None had evident motor abnormalities. All participants gave their written, informed consent before taking part in the experiment. The experiments were part of a program that was approved by the ethical committee of the Vrije Universiteit Amsterdam. The experiments were carried out in accordance with approved guidelines. 

Apparatus and calibration

Figure 1 illustrates the setup used in this study. Participants sat in front of the setup and looked into a half-silvered mirror while performing the task on a horizontal surface beneath the mirror. Stimuli were projected from above (CP-X325 LCD Projector; Hitachi, Tokyo, Japan) onto a horizontal back-projection screen at a frame rate of 60 Hz and with a resolution of 1024 × 768 pixels. The half-silvered mirror was below the back-projection screen, so images were reflected by the half-silvered mirror, giving participants the illusion that the display was on the same horizontal plane as the one in which they were moving. The distance between the back-projection screen and the half-silvered mirror was the same as the distance between the mirror and the surface on which participants were performing the task. Participants had to slide the index finger of their dominant hand across the surface below the half-silvered mirror to try to pass through visually presented targets that moved across the screen. Lights beneath the half-silvered mirror allowed us to control the visibility of the hand (participants could not see their hands when the lights were off). 

Figure 1.

 

Schematic representation of the setup. Images were projected from overhead onto a screen positioned above a half-silvered mirror, creating the illusion that the stimuli moved on the surface on which the participants performed the interceptive movement. Lights beneath the mirror were turned on during the calibration so participants could align their fingers with the calibration targets. Otherwise, the lights were off so participants could not see their hand but saw a cursor that followed their index finger with a certain delay.

We recorded participants’ index finger and eye movements as they tried to intercept the targets. We did so using two Optotrak 3020 cameras (Northern Digital Inc., Waterloo, Ontario, Canada), recording at 250 Hz. One of the cameras was placed behind the setup facing the participant. This camera recorded the position of an infrared marker that was attached to the nail of the index finger that the participant used to perform the task. The second camera was placed to the left of the participant and tracked the position of three infrared markers attached to a biteboard that participants held with their teeth. The biteboard was not attached to anything else, so participants could move their heads in any way they wanted while performing the experiments. Eye movements were recorded with an Eyelink II (SR Research Ltd., Kanata, Ontario, Canada) at 500 Hz. 

We needed to calibrate our setup in order to be able to relate gaze measurements with the projected images on the screen. To do so, we first needed to know the spatial coordinates of the images on the screen. This was determined with the help of a pointer consisting of a rod with a tapered end and three infrared markers attached to it. The pointer was calibrated by placing an additional marker at the tapered end and determining its position with respect to the three markers. With the calibrated pointer we determined the positions of five consecutively presented dots on the screen by holding the rod so that the tip of the tapered end was at the dot. These positions were used to determine how to render objects at desired positions in subsequent images. To determine the positions of the eyes with respect to the biteboard, we placed the pointer with the tapered end between the participant and the screen. Twenty different points were presented consecutively on the screen, and participants had to move their head to align the tip of the pointer with each of the dots, first when looking only with the left eye and then when looking only with the right eye. Whenever a participant pressed a button to indicate that the tip of the pointer was aligned with the dot, a measurement was taken and the next dot appeared. Each measurement provided us with a line passing through the eye: the line through the tip of the rod and the point on the screen. This line was expressed in a coordinate system fixed to the markers attached to the biteboard. The position at which the 20 lines for each eye come closest together was taken as the position of that eye with respect to the biteboard. To calibrate the eye movements, we presented a dot in the middle of the screen and asked participants to move their head while maintaining fixation on the dot. We combined the coordinates of the pupil with respect to the head with the position of the dot relative to the head to determine the transformation of the Eyelink coordinates that minimized the deviations in calculated gaze positions throughout this period. The calibration was verified by rendering dots where we estimated participants were looking and asking participants whether the dots were where they were looking. If not, the calibration procedure was repeated. Finally, the position of the finger was calibrated by having participants place their finger at four indicated positions on the screen. For more details about the calibration, see de la Malla et al. (2017) and de la Malla, Rushton, Clark, Smeets, & Brenner (2019). 

Stimuli and procedure

Stimulus items were presented on a white background. Each trial began when participants placed their index finger on a 1.4-cm-diameter green disk (the starting position). The starting position was located in front of the participant, 10 cm closer to the participant than the screen center. When the participant's finger had been at the starting position for a random period of time between 600 and 1200 ms, the target appeared. The target was a 2-cm-diameter black disk that appeared 8 cm to the left or to the right of the screen center and 10 cm farther away from the participant than the screen center. The target moved horizontally across the screen at 10 cm/s. It moved to the right if it appeared on the left, and to the left if it appeared on the right. 

Participants had to try to move through the target with a single continuous sliding movement. They were explicitly instructed not to stop at the target but instead to cross it without lifting their finger off the surface. After each interceptive movement, participants had to move their finger back to the starting position to start the next trial. To help them find this position without providing information about the delay, a cursor was shown at the position of the finger, but only when the finger was static (moved less than 0.04 mm in 4 ms; de la Malla et al., 2012). Participants could have a break any time they wanted simply by not placing their finger at the starting position. 

Participants performed two experiments in a fixed order. Except during the eye tracker calibration, participants received no instructions regarding eye movements. They were free to look anywhere they wanted. Each of the two experiments had 280 trials in total. During the first 20 trials, participants did not receive any feedback about their movement (No Feedback condition). These trials were intended to get participants accustomed to the task. In the following 20 trials, participants saw a cursor consisting of an 0.8-cm-diameter black disk that indicated the position of their index finger with a delay of about 59 ms (D59 condition). This delay between the cursor and the finger is the minimal delay that our system allowed us to use due to the time it takes to acquire the Optotrak data and render the images. These trials were intended to allow participants to become accustomed to the cursor as a representation of their finger. For the remaining 240 trials, the cursor was delayed with respect to the finger by either 59, 100, 150, or 200 ms (Random Delay condition). The delay was chosen randomly on each trial, with a different random order for each participant. In the Random Delay condition, we also gradually increased the brightness of the cursor so that its contrast with the white background decreased systematically across trials until the cursor was almost invisible. 

We defined the contrast of the cursor as the difference in luminance between the background and the target divided by the background luminance. The change in contrast between the cursor and the background followed the equation:  

\begin{equation}\textit{Contrast}\;\left( t \right) = {C_{min}} + \left( {{C_{max}} - {C_{min}}} \right)\left(1 - \frac{t}{T}\right)^n\end{equation}

(1)

where C_min and C_max are the minimum and maximum contrast, respectively; t is the trial number in the Random Delay condition; T is the total number of trials in this condition (240), and n determines the speed at which the contrast changes. We used values of 0.2, 1, and 2 for C_min, C_max, and n, respectively. 

The design of the second experiment was exactly the same as the first one, with the exception that a red line (0.1 cm wide) was shown along the midline of the screen to indicate the path that the cursor (finger) should take to intercept the target (Figure 1). Note that specifying the path also fixed the moment at which one had to try to hit the target. It took about 40 minutes to complete the two experiments. During a break between the two experiments, which was as long as the participants wanted, participants could take off the Eyelink headset, remove the biteboard from their mouth, and detach the marker from the nail of their index finger. Adding the line in the second experiment put more pressure on guiding the cursor. We anticipated that this might make participants direct their gaze at the cursor more frequently. It might also result in participants fixating the specified interception point: the position at which the target crosses the path that participants had to follow. Introducing a second task does make interpreting differences in interception performance more complicated, because participants might differ in the extent to which they attribute resources to the two tasks (intercepting the target and keeping the cursor on the path), so effects that are only found in this second experiment should be interpreted with caution. 

Analyses

Trials in which participants lifted their finger off the surface during the interceptive movement or did not reach the path of the target were excluded from the analysis (seven trials in Experiment 1 and two trials in Experiment 2). We used the measured area of the pupil to identify cases where the eye was not well detected. If the median measured area within a trial was less than 20% of the usual value (the overall median measured area across all participants in that experiment) or if the pupil size decreased to less than 2/3 of the median pupil size on a given trial, the recording of that eye was not considered valid for that trial. Following this procedure, we excluded eight trials in Experiment 1 because neither of the eyes was well detected. We also excluded the recording of one of the eyes of one of the participants from both experiments because the position of the pupil was clearly jittering, but we included the trials in the analyses using information from the other eye. An additional 16 trials in Experiment 1 and 18 in Experiment 2 were based on information of only one eye. Thus, overall, we excluded only 15 trials from further analyses in Experiment 1 (<0.5% of the data) and two trials in Experiment 2. Whenever we had information from both eyes, we determined where participants were looking (gaze) by averaging the estimates of where the lines of sight of the two eyes intersected the screen (see Apparatus and Calibration section). When only information from one of the eyes was available, gaze values corresponded to the measures obtained from that eye. 

In order to determine how where people look when they make interceptive movements influences the extent to which the movements are guided by visual information from the cursor, we needed to characterize patterns of gaze for each trial. An obvious category to consider is following the moving target with one's gaze (Target). A category that we might expect to see in the second experiment, in which the interception location is specified in advance because it is where the line crosses the path of the target, is maintaining gaze at the interception location (Fixating). In the current study, we manipulated the contrast of the cursor with respect to the background to try to make people look at the cursor rather than at the target. An additional category to consider, therefore, is following the cursor with one's gaze (Cursor). Any pattern that did not fit into these three categories was assigned to a fourth category (Other). 

To determine which category each trial belonged to, we first computed the median gaze velocity in the sagittal and in the lateral direction for each trial and combined these values to give a gaze movement direction and speed for that trial. Only data from 50 ms after the cursor had begun moving (or after the finger had begun moving for the first 20 trials with no cursor) until the finger crossed the path of the target were considered because we were interested in gaze during the cursor or finger movement. If the median gaze movement speed was below 3 cm/s we considered participants to be Fixating. If the gaze movement direction (obtained from combining the median lateral and sagittal gaze velocities for the trial in question) was within 45 degrees of the direction of the target's motion (rightwards or leftwards on different trials), we considered participants to be looking at the Target. If the gaze movement direction was upward in the visual field (away from the participant) and differed by more than 45 degrees from the direction of the path of the target we considered participants to be looking at the Cursor. All other cases were categorized as Other. To illustrate the effectiveness of this way of classifying trials, Figure 2 shows how gaze changed during the last 400 ms of trials within each gaze category. Each panel in this figure shows the median gaze positions at various times before the finger crossed the path of the target. The median was determined across all trials and participants included in each category, after aligning all gaze traces so that they end at the same position. 

Figure 2.

 

Median gaze positions at various times before the finger crossed the path of the target. Positions are defined with respect to the position at which the finger crossed the path of the target. Separate columns represent the gaze categories, and rows represent the two experiments. The brightness of the trajectories gradually increases with the remaining time to cross the path of the target. In order to be able to combine the two directions of target motion, we mirrored the paths for leftward-moving targets. The plots confirm that gaze followed the target (moving from [–4, 0] to [0, 0] during the same time) for trials assigned to the Target category. Gaze followed the cursor for trials assigned to the Cursor category (the finger usually reached the target after the target had crossed the midline in Experiment 1). Gaze barely changed for trials assigned to the Fixating category. Gaze was difficult to interpret for the trials assigned to the Other category, which included very few trials.

We used gaze velocity for our categorization because gaze signals from the Eyelink are prone to drift, probably largely due to the cameras moving slightly with respect to the eyes as a result of the way they are attached to the head, so relying on velocity signals rather than position signals makes the analysis more robust. Moreover, determining whether one is looking at the target or at the cursor when the two are close to each other is difficult when relying on their relative positions rather than on the direction in which the gaze is moving. We computed the velocity of both gaze and finger (or cursor) movement, in the sagittal and lateral directions, by dividing the distance between the positions 15 ms before and after each moment by the corresponding time interval of 30 ms. The finger and cursor were considered to have begun moving when their sagittal velocity was greater than 3 cm/s. Of course, the difference between the moments at which they began moving depended on the delay in that trial. 

Figure 3 illustrates how gaze positions were converted into velocity heat maps. We use heat maps to represent gaze behavior rather than relying on the representation in Figure 2 because such plots also provide an impression of the variability. In Figure 3, one can see how the lateral (Figure 3A) and sagittal (Figure 3B) gaze positions changed as a function of the remaining time to hit the target. This is shown for a representative trial of the Target (blue), Cursor (brown), and Fixating (gray) categories. The trials did not begin at the same time because we considered only data from 50 ms after the cursor began moving (until when the finger crossed the path of the target). Figures 3C and 3D show how the lateral and sagittal velocity changed across time on the same trials. Figure 3E presents the combination of the lateral and sagittal velocities at each moment. This representation is clearly not suitable for combining large numbers of trials, so we use heat maps where the color indicates the relative frequency of occurrence of a given combination of velocities. Figure 3F shows an example of such a heat map based on 25 trials of the Cursor category. In order to simplify the presentation of gaze data, we considered lateral velocities to be positive when they were in the same direction as the target is moving and sagittal velocities to be positive when they were away from the participant. This is equivalent to mirroring the gaze (and target and finger or cursor) movements laterally whenever the target was moving to the left, so that all data can be combined as if the target were always moving to the right. 

Figure 3.

 

Illustration of the gaze analysis. Lateral (A) and sagittal (B) gaze positions as a function of the time left before the attempt to hit the target for a representative trial that has been assigned to the Target (blue), Cursor (brown), and Fixating (gray) categories. The black line represents the position of the center of the target across time. Note that the target had a diameter of 2 cm, so whenever gaze positions were within 1 cm of the black line the eyes were directed at the target. The corresponding gaze velocities are shown both as a function of time (C, D) and relative to each other (E). The black straight lines in these panels represent the lateral and sagittal velocity of the target. Gaze velocities relative to each other (E) can be plotted as a heat map. (F) Heat map for 25 trials in which gaze was categorized as following the cursor. The density in this plot was normalized so that red represents the most frequent pair of velocities and blue represents any combination of velocities that did not happen. The intersection of the purple lines indicates the velocity of the target.

After categorizing the trials on the basis of gaze, we examined to what extent participants used visual feedback from the cursor to guide the cursor to the target. We did so by calculating the temporal error between the unseen finger and the target (de la Malla et al., 2012; de la Malla et al., 2014; Cámara et al., 2018) at the moment the finger crossed the path of the target. This error was obtained by dividing the distance between the finger and the target at the moment the finger crossed the path of the target by the velocity of the target. If participants were trying to hit the target with the cursor, the temporal error between the finger and the target would be equal to the imposed delay between the finger and the cursor. Alternatively, if they were not using visual information about the cursor to guide their movement at all, the temporal error between the finger and the target would not depend on the delay. 

Results

Experiment 1

The first question was whether reducing the visibility of the cursor by decreasing its contrast with respect to the background influenced participants’ gaze patterns. Randomizing the delay across trials makes it essential to use visual information from the moving cursor to adjust the finger movement, because participants cannot know the delay until the cursor begins moving. Reducing the contrast of the cursor across trials makes it more difficult to see the cursor, so we expected participants to direct their gaze at the cursor when the contrast reached some level. Figure 4 shows the category to which each trial was assigned for the 13 individual participants. The pattern of gaze differed among participants. Participants 4, 9, and 13 almost always looked at the target. Participants 1, 3, and 5 appear to have switched to looking at the cursor when the latter was difficult to see and then back to looking at the target again when contrast decreased further. Participant 2 also switched to looking at the cursor when its contrast decreased but did so at a lower contrast and did not switch back to looking at the target. Participant 10 seemed to look at the cursor rather than at the target. The remaining participants mainly looked at the target, with the instances of looking at the cursor being less clearly related to the contrast of the cursor (i.e., to the later trials). 

Figure 4.

 

Gaze category per trial and participant in Experiment 1. Gaze median movement direction (zero is in the same direction as the target; positive is away from the body) and category (coded by color and shape) per trial (horizontal axis) and participant (different panels). Participants 1 to 3 are three of the authors. The vertical gray lines indicate the transitions between experimental conditions (No Feedback, D59, and Random Delay conditions, respectively).

As was to be expected, for the Target category (which includes 85% of the trials) gaze velocity was close to 10 cm/s laterally (the velocity at which the target moved; intercept of purple lines in Figure 5A). For the Cursor category (13% of the trials), the movement was mainly in the sagittal direction, but the velocity was lower than that of the cursor, indicating that participants were not tracking the cursor very well (or at least not very smoothly). For the Fixating category (1% of the trials), the velocity is obviously close to zero. The Other category (0.6% of the trials) clearly includes some strange gaze patterns (as we saw in Figure 2), but the velocity is mostly close to zero so these might be trials in which participants looked for the cursor by making several fixations or something similar. The heat maps showing the lateral and sagittal velocities of the cursor (Figure 5B) reveal that participants not only moved their finger forward (in the sagittal direction) but also moved it in the direction of the target motion (laterally, as is also visible in Figure 2). That there are no velocity values near zero for the cursor is due to the heat maps only including data from 50 ms after the cursor began moving. In the No Feedback condition, where no cursor was shown, we took data from 50 ms after the finger began moving (and we considered the movements of the finger in the cursor velocity plots). 

Figure 5.

 

Summary of gaze and cursor motion in Experiment 1. Heat maps of the occurrence of lateral and sagittal gaze (A) and cursor (B) velocities (or finger velocities in the No Feedback condition). Each column corresponds to a different gaze category. The intersections of the purple lines indicate the velocity of the target. The density scale is the fraction of instances within each category.

To determine whether looking at the cursor influenced the way the cursor was guided toward the target, we compared the temporal errors that participants made in intercepting the targets on trials in which gaze followed the target (Figure 6A) with those on trials in which gaze followed the cursor (Figure 6B). In the Random Delay condition (trials 41–280), using visual information to guide the cursor to the target would give rise to systematic differences in temporal errors between trials with different delays. Because the delays were presented in random order and we nevertheless wanted to average across participants, we first averaged the temporal errors for each delay between the finger and the cursor within blocks of 20 trials. We did so for each participant and then determined the mean and standard error across participants. The temporal errors clearly depended on the delay, confirming that participants use the provided visual feedback to intercept the target with the cursor rather than with the finger (as in Cámara et al., 2018). The standard errors within Figure 6A (and Figure 6B) were not systematically larger for longer delays, so the participants’ performance was not more variable for the longer delays. The fact that the mean temporal errors are more variable and the standard errors larger in the Cursor category (Figure 6B) than in the Target category (Figure 6A) is a consequence of there being much fewer trials in this category (compare the number of blue dots and brown squares in Figure 4). The difference in the number of trials included in each category can also be seen by comparing the height of the colored bars of Figures 6C and 6D. The colored bars in Figure 6 represent the percentage of trials in the Target (Figure 6C) and Cursor (Figure 6D) categories. The four colors correspond to the four different delays. Importantly, the separation between the errors for the different delays (the differently colored and shaped symbols in Figures 6A and 6B) and the percentage of targets that were hit (black diamonds in Figures 6C and 6D) did not differ systematically between the two categories (Target and Cursor). The contrast did modulate the degree of success in intercepting the target. Unsurprisingly, participants hit fewer targets when the contrast was low. At the lowest contrasts (last two bins of trial numbers), participants were no longer able to compensate for the delays (the separation between the different symbols in Figures 6A and 6B is no longer systematic), presumably because they could no longer use visual information to guide the cursor irrespective of where they were looking. 

Figure 6.

 

Interceptive behavior for the Target and Cursor categories in Experiment 1. Temporal error (time between the moment the finger crossed the path of the target and the moment the target crossed the position at which it did so) when gaze followed the target (A) and when it followed the cursor (B). The vertical lines separate the three conditions of No Feedback, D59, and Random Delay. The colored horizontal lines represent errors of 59 ms (red), 100 ms (green), 150 ms (blue), and 200 ms (violet), which are the errors one would expect if the cursor (rather than the finger) intercepted the target. Each point represents the average temporal error of trials with a certain delay within a block of 20 trials. Error bars are standard errors across participants’ mean values. The panels in the middle column show the percentage of trials that were assigned to the Target (C) and Cursor (D) categories. The bars are divided into four color-coded parts to show the percentage of trials for each temporal delay. The black diamonds indicate the percentage of trials that were hit in each block for the gaze category in question. The rightmost panels show the mean reaction times (triangles) and movement times (circles) of the finger for the same blocks of 20 trials for the Target (E) and Cursor (F) categories (with standard errors across participants).

Randomizing the delay on each trial ensured that there was no way to know its value before the finger began moving, so all adjustments to the different delays had to take place during the movement. As a consequence of not moving along the shortest possible path to intercept the target (not only in the sagittal direction; see Figure 5B), participants hit the target after it had passed the midline. Consequently, the sum of the reaction and movement times presented in Figures 6E and 6F is often longer than 0.8 s (which is the time it took for the target to reach the midline). The reaction time of the finger was quite consistent during the course of the experiment (across blocks of trials) and was similar when participants looked at the target (triangles in Figure 6E) and at the cursor (triangles in Figure 6F). Movement times were also quite consistent when participants looked at the target (circles in Figure 6E). The movement time was slightly shorter when there was no cursor (No Feedback condition) and slightly longer when the cursor was difficult to see (last blocks of 20 trials, including the blocks in which participants could no longer compensate for the differences in delays, as shown in Figure 6A). The mean movement times when participants looked at the cursor (circles in Figure 6F) are more variable due to the smaller number of trials and to different participants contributing to different blocks, so the visible changes should be interpreted with caution. 

Experiment 2

The design of the second experiment was exactly the same as Experiment 1, except that a red line, 0.1 cm wide, connected the starting point with the closest point on the path of the target (see Figure 1) and that participants were instructed to try to move along this line to intercept the target. It is important to realize that this additional task also had implications for the interception task, because it specified when and where the target should be hit, which was not specified in the first experiment. We expected participants to more frequently direct their gaze at the cursor in this experiment and wanted to see how doing so influenced how well they could guide the cursor to the target. 

As we anticipated, adding the requirement of moving along a specified line increased the number of trials in which participants primarily directed their gaze at the cursor (compare the number of brown squares in Figure 7 to the number in Figure 4). There were still large differences among participants. There was less of a tendency to mainly look at the cursor when its contrast was low, suggesting that the increased tendency to look at the cursor was related to having to move along the line in general, rather than to difficulties in moving the cursor along the line when it was difficult to see in peripheral vision due to the reduced contrast. Some participants also kept their eyes static quite often (Fixating, gray triangles in Figure 7; participants 3, 8, and especially 13). 

Figure 7.

 

Gaze category per trial and participant in Experiment 2. Details are as in Figure 4.

The peaks of the density distributions of the gaze velocities (Figure 8A) were very similar to those of Experiment 1 (Figure 5A). The Target category was again the largest, with 46% of the trials. Of the remaining trials, 32% fell within the Cursor category, 14% within the Fixating category, and 7% within the Other category. The density distributions of the cursor velocities (or finger velocities for the No Feedback condition) are quite different from those of Experiment 1 (compare Figure 8B with Figure 5B). This is because participants were now required to move their finger in a specific manner (avoiding lateral motion; this difference is also very clear when comparing gaze for the Cursor category in the two experiments in Figure 2). Because participants had to follow the path while trying to intercept the target, it is not surprising that the cursor movements did not depend on the gaze category. 

Figure 8.

 

Summary of gaze and cursor motion in Experiment 2. Heat maps of the occurrence of lateral and sagittal gaze (A) and cursor (B) velocities (or finger velocity in the No Feedback condition) for the various gaze categories (in different columns). Details are as in Figure 5.

In order to see whether differences in gaze behavior influenced interception, we determined the temporal interception error for blocks of 20 trials, as we had done for Experiment 1. As in Figure 6, the central column of panels of Figure 9 shows the percentage of trials that fell within each category. In accordance with our impression from Figure 7, the number of trials within each category barely depended on the contrast of the cursor (Figures 9D–9F). In agreement with what we observed in Experiment 1, the pattern of temporal errors (systematic differences between the different symbols in Figures 9A–9C) was similar for the Target (Figure 9A) and Cursor (Figure 9B) trials. In both cases, participants reached the path of the target earlier if the delay was longer (the order was more or less correct), although the extent to which they did so was smaller than in Experiment 1. Moreover, for both categories of trials the systematic pattern of temporal errors was absent when the contrast was low. Thus, having to move along a specified path mainly reduced the extent to which participants adjusted their behavior, possibly because they also had to keep the cursor on the line, but more likely simply because they had to move faster so they had less time to make the necessary adjustments (as will be explained in more detail below). The systematic pattern of temporal errors, which indicates that the path of the cursor is adjusted on the basis of visual information from the ongoing movement, appears to be absent for Fixating (Figure 9C). Despite this, the percentage of trials in which the target was hit (black diamonds in Figures 9D–9F) did not clearly depend on the gaze category. It did decrease when the target contrast was low, but it was not evidently lower for trials in which participants fixated (Figure 9F). For all categories, the number of hits appears to be lower than in Experiment 1, so perhaps the adjustments were too small to make a difference in the number of hits in this experiment. 

Figure 9.

 

Interceptive behavior for Target, Cursor, and Fixating categories in Experiment 2. Performance measures for trials in which gaze was categorized as tracking the target (A, D, G) or the cursor (B, E, H), and when it was categorized as fixating (C, F, I). (A–C) Time difference between when the finger crossed the path of the target and when the target was at the position at which it did so. (D–F) Percentage of trials assigned to the category in question (bars) and percentage of trials hit (symbols). (G–I) Mean movement time (MT) and reaction time (RT). Details are as in Figure 6.

As already mentioned, having to move along a specified path reduced the extent to which participants adjusted their behavior to the delay on the trial in question. This is not surprising given that specifying the path in this manner meant that participants had to make the cursor hit the target after 0.8 s (the target moved at 10 cm/s and appeared at a distance of 8 cm from where it crossed the red line). In Experiment 1 participants generally took longer to hit the target (Figures 6E–6F; sum of reaction time and movement time). Having more time, especially more time from when the finger began to move, obviously increased the opportunity to adjust the movement to the delay. In Experiment 2 participants took about 0.68 s to complete their movements (Figures 9G–9I; sum of reaction time and movement time). Taking into account that the average delay between the finger and the cursor was about 0.13 s, the cursor crossed the path of the target after about 0.81 s. This means that, on average, the finger reached the path of the target about 10 ms later than it would need to do to compensate for the average delay. This systematic error is inconsequential considering that the target diameter was 2 cm and its velocity was 10 cm/s so that participants had a 200-ms time window to intercept the target. The large time window actually meant that adjusting the timing of the movement to the average delay was enough to obtain a reasonable number of hits. Participants did appear to move in accordance with the average delay (Figures 9A–9C). Doing so could be based on the outcome of previous trials (de la Malla et al., 2012); nevertheless, participants also appeared to make adjustments when there was enough time to do so. They may even have begun moving sooner (smaller reaction time) in the second experiment than in the first (compare triangles in Figures 9G–9I with those in Figures 6E and 6F) to allow more time to make such adjustments to the ongoing movements. 

We evaluated the extent to which participants followed the instruction of moving along the red line by determining the largest deviation of the finger from the line for each movement to the target. We averaged these deviations across trials within each gaze category for each block of trials and then averaged across participants (Figure 10). Seeing the cursor clearly helped participants move along the line, as the maximal distance from the line was largest when no feedback was provided (first block of trials) and also increased when the contrast of the cursor became very low (last blocks of trials). There was a slight tendency to remain closer to the line when gaze was directed at the cursor than when it was directed at the target, but in all cases the deviations were quite modest, indicating that participants did comply with our instructions. When cursor contrast was high, the deviation was only about 5 mm, despite the fact that we took the largest deviation on each trial regardless of where it was along the path. 

Figure 10.

 

Average maximal distance between the finger and the red line within blocks of 20 trials. Symbols and error bars are means and standard errors across participants. Colors and shapes differentiate among the gaze categories. Vertical lines separate the three conditions.

Discussion

The results of the present study show that it makes very little difference for how precisely an interception is timed whether participants direct their gaze at the target or at the cursor (Figures 6A, 6B; Figures 9A, 9B). Furthermore, we confirmed that people can guide their hand, or a representation of their hand (the cursor), to the target using peripheral vision. Although we succeeded in getting some participants to look at the cursor rather than at the target under some circumstances, participants still usually looked at the target, despite its motion being completely predictable and the motion of the cursor being unpredictable due to the randomly chosen delay between their finger and the cursor. When participants were asked to follow a certain path to the target (Experiment 2), they also occasionally fixated on the point at which they were expected to intercept the target rather than tracking either the target or the cursor with their gaze. In this case, they appeared not to be able to guide the cursor on the basis of visual information from the ongoing movement (no systematic pattern related to the delays in Figure 9C). The number of trials that fell within this category was too small to allow us to be completely certain of this, but if it is true it might mean that people can only reliably monitor the progress of one item at a time with peripheral vision. 

The randomized delays in the current experiments encouraged participants to control the cursor on the basis of visual information provided during the trial, as in our previous study (Cámara et al., 2018). Most studies investigating the effect of temporal delays on performance have used constant delays (Cattan, Perrier, Bérard, Gerber, & Rochet-Capellan, 2018; Cunningham et al., 2001b; Foulkes & Miall, 2000; Miall & Jackson, 2006; Rohde et al., 2014; Vercher & Gauthier, 1992) or delays that changed in a predictable way (de la Malla et al., 2014; Knelange & López-Moliner, 2019). In such cases, participants can adjust their actions across consecutive trials (adapt to the delay) in addition to adjusting ongoing movements. In our case, by varying the delay between the finger and the cursor randomly across trials (in the Random Delay condition) participants could only know the delay between the cursor and the finger well after the finger began to move (when the cursor did so). Thus, although participants were able to see whether they were successful on each trial, they could not benefit from feedback from the previous trial to adjust their movements to the delays on individual trials. 

Picking a random delay for each trial forced participants to respond directly to how they saw the cursor move in order to compensate for the delay, rather than relying on a more gradual adaptation to the delay. Nevertheless, the temporal errors were clearly positive when the contrast was very low by the end of the experiment (Figure 6), which could indicate that participants had adapted to the average delay. However, the temporal errors were also clearly positive during the first 20 trials, which was before any delayed cursor had been shown, as was also found in previous studies (de la Malla et al., 2012; de la Malla et al., 2014). Some adaptation due to being exposed to delays might have occurred, because the error during the first trials does appear to be slightly smaller than during the last trials. Moreover, performance was less accurate here than in previous studies involving delays (e.g., de la Malla et al., 2012; de la Malla et al., 2014) in which participants could adapt, although this could be due to the target being considerably larger in this study (2-cm diameter) compared to the other studies (0.8-cm diameter), so accuracy was less important. We plotted the timing errors of the finger rather than of the cursor, although participants clearly interpreted the task to be avoiding timing errors of the cursor rather than of the finger (hitting the target with the cursor rather than the finger). By plotting interception errors in this manner, any systematic differences among errors on trials with different delays can be attributed to adjustment to the ongoing movement on the basis of visual information from the cursor. 

Our participants were less proficient at adjusting their ongoing movements to the delay in Experiment 2 than in Experiment 1. This is undoubtedly due partly to the fact that the time available to intercept the target was shorter in the second experiment than in the first one because of the imposed path that participants had to follow to intercept the target. Another possible reason for participants being less proficient at making adjustments in Experiment 2 could be that adding the requirement of moving along a specified path means asking participants to perform two tasks at the same time: hitting the target and moving along the line. Several studies have shown that completing two tasks at the same time comes at a cost (Fagot & Pashler, 1992; Hommel, 1998; Levy & Pashler, 2001; Pashler, 1994). Another possible reason for participants being less proficient at adjusting their ongoing movements to the delays in Experiment 2 is that imposing an interception point prevented participants from making spatial corrections (Brenner & Smeets, 2015). Imposing a path as in the present study (and in Cattan et al., 2018), physically restricting the movement to a certain path (Tresilian & Houseman, 2005; Tresilian & Lonergan, 2002), or even just specifying the position at which participants have to hit a target (Brenner & Smeets, 2011; de la Malla et al., 2017) restricts participants to changing their movement speed when they need to adjust their movements, which may be less effective than adjusting the interception point because the latency to do so is longer (Brenner & Smeets, 2015). Irrespective of why the additional requirement influenced performance, it was successful in making participants regularly direct their gaze toward the cursor (Figure 7), even when the cursor was not very difficult to see (before contrast was very low), presumably because doing so helped participants achieve the goal of moving along the line. Importantly, even in this case, interception performance was very similar when participants directed their gaze at the target and when they directed it at the cursor (Figures 9A, 9B, 9D, 9E). 

The fact that directing gaze at the cursor did not affect performance under circumstances that were specially designed to make it beneficial to gather information about the motion of the cursor probably explains why people normally do not look at the cursor. Even if the cursor only responds after a delay, its motion is the consequence of the person's action. The person has no influence on the motion of the target; therefore, people are usually justified in being more confident about their anticipation of what the cursor will do (it will follow the same path as their finger). Even if the finger does not follow the anticipated path, its deviation from the path can be judged from proprioceptive as well as visual information (Ernst & Banks, 2002; van Beers et al., 1996; van Beers, Sittig, & Denier van der Gon, 1998; van Beers et al., 1999; van Beers et al., 2002). That people can move their hand quite reliably along an intended path, especially after some practice (van Beers, 2009), is evident from the modest increase in the maximal deviations from the line during the last blocks of Experiment 2 (Figure 10). Thus, it is not unreasonable to rely on less precise visual information about the position of the finger obtained through peripheral vision, while maintaining gaze on the target to be sure to know where it is and how it moves. This is consistent with the tendency to track the target with one's eyes not being restricted to interception tasks. Participants also track the target rather than the cursor when the task is to track an unpredictably moving target as accurately as possible with a cursor, with no instructions about gaze (Danion & Flanagan, 2018). It is also consistent with evidence that precise ocular pursuit of a predictably moving target is not necessary for successfully interacting with the target (Cesqui, Mezzetti, Lacquaniti, & d'Avella, 2015; de la Malla et al., 2017; López-Moliner & Brenner, 2016). Thus, directing gaze at the target is not essential for intercepting moving targets, but because judging the motion of a target relies primarily on vision it is logical to direct gaze toward the target whenever there is no clear reason to direct it elsewhere. Moreover, studies with immediate feedback, in which performance is much better, suggest that tracking the target with one's eyes has some additional advantages (reviewed in Brenner and Smeets, 2018). 

On a final note, we would like to point out that there were large differences among participants with regard to the extent to which they directed their gaze at positions other than the target and in the circumstances under which they did so. In the second experiment, the differences among participants might be due to them prioritizing the two tasks (moving along the path and hitting the target) differently. However, participants may differ in the extent to which they rely on proprioceptive information or in how well they can extract regularities from the task such as the constant target velocity or the time it takes for the target to reach the center of the screen. In this context, we want to direct the reader's attention to the gaze strategy followed by participant 10 (Figures 4 and 7). This participant clearly moved her eyes to follow the cursor rather than the target. The same participant took part in our earlier study involving a delayed cursor (participant 6 in Cámara et al., 2018). Although most of the participants in the previous study followed the target with their gaze to try to intercept it, this participant was prone to pursue the cursor. In the present study, we monitored this participant's eye movements especially carefully during the experimental session to make sure that the measurements were correct. At the end of the present experiments, we asked her whether she had any specific strategy to succeed in the task. She specified that she looked at the cursor while trying to intercept the target. Even though she intentionally followed a different gaze strategy, she hit 86% and 69% of the targets in the first and second experiments, respectively, which is close to the average performance (see dark gray diamonds in Figures 6 and 9). Therefore, her behavior provides a clear example of our claim that looking at the cursor while tracking the target with peripheral vision is not necessarily detrimental for performance. 

Acknowledgments

Supported by Grants BES-2014-069289 and PSI2017-83493-R (AEI/FEDER, UE) and 2017SGR48 from the Catalan government. CM and EB were supported by Grant NWO 464-13-169 from the Dutch Organization for Scientific Research. 

Data supporting these findings are available through the Open Science Framework following this link: https://osf.io/24q5g/. 

Commercial relationships: none. 

Corresponding author: Cristina de la Malla. 

Email: [e-mail]. 

Address: Vision and Control of Action Group, Department of Cognition, Development and Psychology of Education, Institut de Neurociències, Universitat de Barcelona, Barcelona, Spain.