Preparing coordinated eye and hand movements: Dual-task costs are not attentional

Abstract

Dual-task costs are observed when people perform two tasks at the same time. It has been suggested that these costs arise from limitations of movement goal selection when multiple goal-directed movements are made simultaneously. To investigate this, we asked participants to reach and look at different locations while we varied the time between the cues to start the eye and the hand movement between 150 ms and 900 ms. In Experiment 1, participants executed the reach first, and the saccade second, in Experiment 2 the order of the movements was reversed. We observed dual-task costs—participants were slower to start the eye or hand movement if they were planning another movement at that time. In Experiment 3, we investigated whether these dual-task costs were due to limited attentional resources needed to select saccade and reach goal locations. We found that the discrimination of a probe improved at both saccade and reach locations, indicating that attention shifted to both movement goals. Importantly, while we again observed the expected dual-task costs as reflected in movement latencies, there was no apparent delay of the associated attention shifts. Our results rule out attentional goal selection as the causal factor leading to the dual-task costs occurring in eye–hand movements.

Introduction

In everyday situations, we frequently reach for objects—be it a simple task like picking up a cup of coffee or a complex task like clearing an office table. However since we usually look at the object we reach for, most reaching movements actually require doing two things at the same time, that is, planning and executing an eye and a hand movement simultaneously (Horstmann & Hoffmann, 2005; Johansson, Westling, Backstrom, & Flanagan, 2001; Land & Hayhoe, 2001; Pelz, Hayhoe, & Loeber, 2001). It might seem trivial to plan both eye and hand movements together, but it constitutes an instance of cognitive multitasking. 

It is known that doing two tasks simultaneously bring costs, since both error rates and reaction times typically increase as compared to doing only one task at a time. These are typically referred to as dual-task costs (Pashler, 1994; Schubert, 2008), which arise when two different tasks compete for limited cognitive resources. In such a scenario, the limited resources could either be shared between the two tasks, leading to a slowing of both (Kahneman, 1973), or else execution of one of the tasks could be postponed until critical processing in the other is finished (Pashler, 1994; Schubert, 1999). While much is known about dual-task costs and the situations in which they arise, it remains debated whether these do occur in the case of simultaneous eye and hand movements, and if so, which particular processing stage(s) between early stimulus processing and final execution of the movement might be specifically involved in the processing bottleneck. 

A number of studies have shown that whether there is interference between eye and hand movements depends on a variety of factors, such as on how the saccade is elicited and on what type of manual response is required. Pashler, Carrier, and Hoffman (1993) have demonstrated that there are almost no dual-task costs when simple button presses and eye movements to an abrupt onset are prepared together, suggesting that reflexive saccades directed toward an onset stimulus can be possibly executed without interference. Similarly, no dual-task costs have been reported when people made reflexive saccades to a peripheral location and simultaneously performed a rhythmic manual tapping task (Sharikadze, Cong, Staude, Deubel, & Wolf, 2008). In contrast, dual-task interference was found to occur even with simple button presses when non-reflexive saccades had to be performed to a location indicated by a central cue (Pashler et al., 1993); obviously, the planning of these saccades required an intentional selection of the movement goal. 

Dual-task interference becomes more prominent when, instead of a simple button press, a manual reaching movement is required. It has been observed that latencies of saccades directed to peripheral onsets are longer if, simultaneous to saccade preparation, a reaching movement has to be planned to the same target (Bekkering, Adam, Kingma, Huson, & Whiting, 1994; Bekkering, Adam, van den Aarssen, Kingma, & Whiting, 1995). This suggests that dual-task costs for saccades arise when a reach must be directed to a spatial target, but not when the movement involves just a simple (non-spatial) button press. In other words, it seems that dual-task costs do arise when both eye and hand movements rely on the selection of a spatial movement goal. They also arise when saccades and button press responses have the same or a different directional component (e.g., to make a saccade to the right and press the left button; Huestegge & Koch, 2009). These findings make it likely that the mutual interference between the two tasks occurs in the movement planning phase, for instance, during the selection of the movement target (Bekkering et al., 1995), rather than in movement execution. Movement goal selection (“I will reach for this apple”) occurs at an early stage of movement planning during which object parameters such as target location in space and object size are specified (Andersen & Buneo, 2002; Milner & Goodale, 1995). 

While at least some of the dual-task costs can be explained by assuming that the two effectors compete for resources to select the movement goal, not all findings suit this pattern. Some studies reported even shorter saccade or reach latencies if participants made simultaneous eye and hand movement to the same object as compared to making single eye or hand movements (Lünenburger, Kutz, & Hoffmann, 2000; Niechwiej-Szwedo, McIlroy, Green, & Verrier, 2005). However, these observations do not necessarily contradict the hypothesis that movement goal selection leads to dual-task costs, since in all of the above-mentioned studies movement goal selection was limited by the fact that participants were asked to make eye, hand, or both movements to only one common target present on the screen (or to one of two targets present in separate visual hemifields). This raises the question as to which degree saccade or hand movement goal selection was activated, since in some cases movements might have been purely reflexive, toward a single target present within one visual hemifield. 

It is important to note that none of these studies analyzed movement goal selection directly but instead relied on indirect measures such as reaction times or movement endpoint errors. Thus, it is possible that while movement execution was delayed in a dual-task situation, movement goal selection was not affected by the need to perform a second task. 

It is now well established that the selection of a stimulus as the goal of a movement is related to attention shift to the movement target. A number of studies have shown that these attention shifts precede the initiation of goal-directed saccades, reaching movements and grasping (Baldauf & Deubel, 2010; Deubel & Schneider, 1996; Deubel, Schneider, & Paprotta, 1998; Montagnini & Castet, 2007; Schiegg, Deubel, & Schneider, 2003). Hence, spatial attention can be used as an index of movement goal selection before movement onset. 

By measuring both movement latencies and spatial attention, we investigated whether movement goal selection is the causal factor that leads to the costs observed in these dual-task situations. Participants performed conjoint saccades and manual reaching movements while we manipulated the temporal overlap between the planning of these movements. In three experiments, two central movement cues were presented sequentially, with a variable stimulus-onset asynchrony (SOA) between the presentations. The movement cues could indicate either the same spatial location or spatially separate locations. The range of SOAs was selected such that in the short SOA conditions planning of saccade and reaches would overlap, whereas in the long SOA conditions those tasks would not overlap. If dual-task costs would occur, they should be largest at the shortest SOAs and smallest at the longest SOAs. In Experiment 1, the first cue specified the reach goal, and the second cue indicated the saccade goal. In Experiment 2, we measured whether dual-task costs are observed also when the movement order was reversed—the first movement cue indicated the saccade and the second cue specified the reach. Finally, in Experiment 3, we measured movement goal selection by using spatial attention as its index. Participants had to reach and look at two different locations while we presented a perceptual probe (a letter) at randomly chosen times during movement planning. This probe could be presented at cued saccade or reach locations, or at locations that were not relevant for the action. It is established that probe discrimination at exogenously or endogenously (as is the case with movement planning) cued locations can index attention at that location (for a review, see Carrasco, 2006). Thus, we could measure whether attention shifted to saccade or reach locations, and whether this shift was delayed when saccade and reach planning overlapped. Combined, the three experiments should reveal (1) whether there are dual-task costs for combined eye and hand movements as reflected in movement latencies, and (2) whether these costs would arise due to movement goal selection as measured in probe discrimination at the saccade and reach goals. 

Experiment 1

In Experiment 1, we determined the dual-task costs arising in a situation in which participants first made a reach, and then a saccade. We varied the time interval (SOA) between two arrow cues instructing to start each movement. If dual-task costs occur, the costs should be largest at the shortest SOAs and smallest at the longest SOAs (Schubert, 1999), since under the first conditions saccade and reach planning are temporally more overlapping than under the latter in which saccade planning starts long after the reaching onset. Additionally, we manipulated whether eye and hand movement goals were shared or not: on half of the trials, participants made saccades and reaches to the same location, and on the other half of trials to two different locations. If eye and hand movement planning shares a common goal selection process, then for the short SOAs there should be a crosstalk between these two systems, resulting in faster saccades and reaches when the two movements were directed to the same goal. On the other hand, if the goals for eye and hand movements are selected independently, there should be no benefit to plan saccades and reaches to the same location. 

Methods

Participants

Twenty-two participants (mean age 23 years, 10 females) participated in this study. All participants had normal or corrected-to-normal vision. Informed consent was obtained from all participants. 

Apparatus

Participants sat in a dimly illuminated room. They placed their right hand on a slightly inclined pointing plane, under a one-way mirror. Stimuli for pointing and saccades were projected from a monitor above onto the mirror. This setup allowed the visual stimuli to appear on the pointing plane, while the participants could not see their hand. In order to provide visual feedback about the hand position, an LED fixed to the tip of the right index finger could be illuminated during the experiment. LED was lit up in the beginning of the trial for participants to arrange their finger with visual stimulus and was illuminated at the end of the trials to provide feedback about reaching accuracy. Stimuli were presented on a 21-inch Conrac 7550 C21 display with frame frequency of 100 Hz, at a display resolution of 1024 * 768 pixels. Visual stimuli were shown on a gray background with a mean luminance of 5.1 cd/m². 

Reaching movements were recorded with a Fastrack electromagnetic position and orientation measuring system (Polhemus, 1993), consisting of a central transmitter unit and a small receiver, mounted on the tip of the index finger of the participant's right hand. The sender unit was placed 60 cm in front of the participant. The device allows for a maximal translation range of 10 ft, with an accuracy of 0.03 in RMS. The frequency bandwidth of the system is 120 Hz; the time delay is 4 ms. Eye movements were recorded with a video-based eye tracking system (SensoMotoric Instruments, Eyelink-I), which provides an accuracy better than 0.1 degree, at a recording frequency of 240 Hz. Head movements were minimized by means of an adjustable chin rest. 

Procedure and stimuli

Figure 1 depicts the stimulus sequence. During each trial, a central fixation cross and twelve mask elements (size 0.9 × 1.4 deg, composed of randomly generated lines) were presented on the uniform background, arranged on an imaginary circle with a radius of 6.5 deg. Participants first directed the index finger of the right hand and their gaze to the central cross; 580 to 880 ms later, the first movement cue—an arrow that pointed toward one of the mask stimuli—was presented at the central fixation. The mask elements were arranged on the circle as if forming a clock face, and the arrow could point toward 2, 4, 8, or 10 o'clock. The arrow was presented for 100 ms, and participants were instructed to reach with the right index finger to the object indicated by this cue. After a variable time (SOA) from the first cue onset, a second movement cue was presented, again for 100 ms. Participants were instructed to saccade to the location indicated by the second arrow. On 50% of the trials, the second cue indicated the same target as the first cue (thus participants had to reach and look at the same location); on the other 50%, the second movement cue indicated a different target than the first cue. In those trials where the cues indicated different targets, the distance between the first and second movement targets was either three items in the clockwise direction or three items in the anti-clockwise direction (for example, if the first cue indicated a reach target at 2 o'clock, then the second cue would indicate (with equal probability) a saccade target at 5 or 11 o'clock, which amounts to an angular distance of 90 degrees from the first cued location). The SOA between the two movement cue onsets was 150, 200, 300, 350, 400, 450, 500, 600, 700, 800, or 900 ms. We chose this wide interval of SOAs in order to precisely measure at which cue delay dual-task costs would appear for saccades and reaches. Since the reaching latencies were typically 200–300 ms, the interval covered the time when reaches were still planned, when the hand was in motion, and when the finger was already at the object. 

Figure 1

 

Experimental procedure. In Experiment 1, participants were asked to quickly reach to the location indicated by the first arrow and then quickly saccade to the location indicated by the second arrow. The time interval between the arrow onsets (SOA) was varied. In Experiment 2, the arrow appearing first instructed the saccade, while the second appearing arrow instructed the reaching. In Experiment 3, participants again reached and looked at two objects indicated by the two subsequent cues. Additionally, a probe display appeared at −50 to 650 ms with respect to the first movement cue onset. The probe was a digital letter “E” or “3,” embedded in a circular array of distractors. Participants reported the probe identity after completing the movement task.

Participants were instructed to reach and look as quickly as possible when the respective movement cues appeared, without delaying their movements or trying to group them. Each participant performed 4 experimental blocks of 144 trials each. All participants had a practice block before starting the experimental task. Six of the participants performed 12 experimental blocks in order to investigate possible practice effects. 

Movement data analysis

Saccade and reach movement data were stored for offline analysis and saccades and reaches were detected using custom software. Reach onsets were defined as points in time when the vectorial velocity reached a threshold of 1°/s. Saccade onsets were defined as points in time when eye velocity threshold exceeded 150°/s. We further defined a 2° radius circle around central fixation as a maximum window within which saccade and reach movement starting position could vary. We removed all trials in which saccades smaller than 2° in size appeared before saccade cue onset. We accepted reach or saccade endpoint as correct if it fell closer to the reach/saccade goal than to any other irrelevant location besides the goal, and if the movement had a minimum latency of 100 ms after the movement cue appeared. Additionally, all trials with saccade or reach latencies longer or shorter than 3 standard deviations from the mean of each subject were rejected. 

Results

We analyzed whether there were dual-task costs when participants made combined eye and hand movements. If there were no dual-task costs, then neither the reaction time of the first task (the reaching) nor the reaction times of the second task (the saccade) should be influenced by the SOA manipulation. Typical dual-task costs would be reflected in an effect of SOA on the reaction times of the second task (the saccade)—these should be longer for the short SOA conditions than for the long SOA conditions. For the reaction time of the first task, there should be either no effect of SOA or an effect that should also depend on SOA. 

The data indeed revealed that the SOA manipulation did not affect the reaction times of the first task—reaching latencies for the shortest SOA of 150 ms were 336 ± 14 ms (mean and standard error of the mean) and were 337 ± 15 ms for the longest SOA of 900 ms (repeated measures ANOVA, F(10, 210) = 0.79, p > 0.6). This means that participants started the reach movement immediately after the first movement cue appeared and did not try to postpone their response until the second movement cue was shown. 

The SOA manipulation had a markedly different effect on the saccade latencies. Saccade latencies decreased with increasing SOA (repeated measures ANOVA, F(10, 210) = 53.03, p < 0.01), indicating that in the short SOA condition participants were not able to initiate their saccade immediately after the saccade cue appeared. The observed dual-task costs were about 100 ms—saccade latency decreased from 384 ± 14 ms for the 150-ms SOA condition to 280 ± 9 ms for the 900-ms SOA condition. Thus, typical dual-task costs did occur under these conditions, with participants being unable to perform the eye movement before they finished preparing the reaching movement. 

We next analyzed whether there were any costs or benefits when saccades and reaches were directed to the same location or to different locations. First, we analyzed reaching movements, as it has been shown that in dual-task situation the task that is performed first (here, the reach) is completed faster if the second task shares a common response code (here, the saccade made to the same location as the reach), compared to a situation with different responses in the two tasks (Hommel, 1998; Lien & Proctor, 2002). Unexpectedly, we did not observe this effect—reaching latencies were not shorter when saccades and reaches were directed to the same location (Figure 2A; none of the planned one-tailed repeated measures t-tests comparing each time bin was significant, all ps > 0.5). This indicates that planning saccade and reach to the same location did not facilitate the preparation of the reach. One possibility of explaining this discrepancy is that we used a larger number of potential target locations (targets could appear at 8 different locations), unlike other studies (e.g., Hommel, 1998; Lien & Proctor, 2002) that used mostly two opposing response categories (e.g., left vs. right motor response). Furthermore, our task required precise spatial location coding—to reach to one of the multiple locations on the screen while making a saccade to a different location—instead of button presses. Note that the need to plan spatially directed movements and the number of potential reach locations could also interact, as reaches to displays with multiple objects are executed faster than reaches to displays with fewer objects (Song & Nakayama, 2006). 

Figure 2

 

Dual-task interference in Experiment 1. (A) Latencies of the reaching movements as a function of SOA. (B) Saccade latencies as a function of SOA. Data are shown separately for trials when saccades and reaches were directed to the same location (red curves) or to different locations (blue curves). Symbols in (B): *p < 0.05, ^p = 0.08. Vertical bars indicate ±SE. Data are slightly translated horizontally to increase the visibility of different conditions.

Next, we analyzed whether there were benefits when the saccade was directed to the same location as the reach. It has been demonstrated that movement planning leads to a shift of attention to the movement goal location (Deubel & Schneider, 1996; Linnell, Humphreys, McIntyre, Laitinen, & Wing, 2005); thus, planning a movement to one location is likely to be helpful as a cue in planning a subsequent movement to the same location. This leads to the prediction that latencies of saccades when they are directed to the same location as the reaching should be shorter than latencies of saccades directed to different locations than reaches. A two-way ANOVA with the first factor SOA and the second factor specifying saccade/reach location agreement did not show significant effect of the second factor (F(10,210) = 1.51; p = 0.2). However, interaction between the second and SOA and saccade/reach location agreement was significant (F(10,210) = 2.42; p < 0.01). We looked in more detail at short and long SOA conditions by performing separate t-tests. Our planned comparisons also showed that for SOAs less than or equal to 600 ms saccade latencies were not shorter if the saccades were directed to the same location as reaching, not even for the shortest 150-ms SOA condition (Figure 2B; at this time bin, mean latency of the saccades directed to the same direction as reaching was 384 ± 13 ms; mean latency of the saccades made to a different location than reaching was 390 ± 17 ms, repeated measures t-test p = 0.60). 

For the long SOA conditions starting at 700 ms, saccade latencies were found to be even longer if saccades were directed to the same as compared to a different location than the reaching movement (Figure 2B, last three SOA conditions). A two-way ANOVA over these 3 last SOA conditions with second factor specifying saccade/reach location agreement was significant for the second factor (F(1,21) = 13.21, p < 0.01). This effect seemed to persist over all three SOA conditions (SOA 700, 800, and 900 ms; individual repeated measures t-tests), and individual subject data showed that majority of the subjects demonstrated this effect. This effect can possibly be attributed to Inhibition of Return (Klein, 2000), which we will discuss in more detail later. 

Our findings demonstrate that the second (saccade) task was delayed while the first (reach) task was processed. In order to provide further evidence that reach planning indeed delayed saccades, we analyzed whether on trials with longer reach latencies the saccades also exhibited longer delays. For this purpose, reach latency in each trial was assigned to one of four quartiles (movement latencies increased from 260 ms in the first quartile to 380 ms in last quartile). Then, saccade latencies were separated into trials where the reaching latencies belonged to the 1st, 2nd, 3rd, and 4th quartiles. If reaching movements were delayed, then saccade latencies should be delayed as well. Thus, for short SOAs saccade latencies should be shorter if reaching latencies were shorter and longer if reaching latencies were longer. For long SOA conditions, this effect should disappear, as reaches would have already started or even finished. For this analysis, we again split the data according to reach/saccade location agreement. Figure 3 shows the result of this analysis, depicting saccade latencies for all SOA conditions. Figure 3A depicts the results for trials where reaches and saccades were directed to the same location, and Figure 3B depicts those for trials where the movements were aimed to different locations. The data show that longer reaching latencies indeed resulted in longer saccade latencies. This effect was most pronounced for the shortest SOA conditions. For SOA 150 ms, the saccades were about 50 ms slower for the longest as compared to the shortest reaching latency quartile, whereas in the SOA 800-ms condition this difference was only 20 ms. 

Figure 3

 

Reaching movements delay saccades. Data are shown for trials where the saccade was directed to (A) the same location as the reaching or to (B) a different location. Reaching latencies were divided into quartiles; the higher the quartile number, the longer the reaching latency. Four sample SOA conditions are shown (see figure legend). Other SOA conditions are plotted as light blue lines.

We computed repeated measures two-way ANOVAs with quartile and SOA as main factors. We split this analysis for trials with saccades and reaches to the same location and trials with saccades and reaches to different locations. When reaches and saccades were directed to the same location, the main effect of SOA was significant, indicating that saccade latencies decreased with increasing SOA, F(10, 150) = 7.93, p < 0.01. The main effect of quartile was also significant, showing that longer reaching latencies lead to longer saccade latencies (F(3, 150) = 18.12, p < 0.01). The interaction between these two factors was also significant, F(30, 150) = 1.61, p < 0.05. 

An equivalent analysis performed for trials when the saccade and reaches were directed to different locations revealed similar results. Again, saccade latencies decreased with increasing SOA (main effect of SOA was significant, F(10, 150) = 8.88, p < 0.01), and longer reaching latencies led to longer saccade latencies (main effect of quartile, F(3, 150) = 27.65, p < 0.01). The interaction between SOA and quartile was also significant (F(30, 150) = 1.60 p < 0. 05), again meaning that longer reaching latencies delayed saccades most in the shortest SOA conditions. 

We also analyzed whether longer reach latencies delayed saccades more or less, if saccades were directed to the same or different location as reaches. We found no significant differences between those conditions (paired samples t-test comparisons for saccades directed to the same versus saccades directed to different location than reaching for each reaching latency quartile were not significant, p > 0.05). 

Last, we analyzed movement endpoint errors. When making saccades and reaches to two different locations, participants sometimes made movement errors by either looking at the reach goal (15% of trials in this condition) or by reaching to the saccade goal (14% of trials), implying a crosstalk between the movement planning for the hand and for the eye. We propose that these errors may result from the difficulty of our task in which two types of trials were interleaved—eye and hand movements directed to the same location or to different locations. Participants may have sometimes failed to switch to the less preferred type of task (eye and hand movements directed to different locations) and instead looked and reached to the same target. 

Experiment 2

In the second experiment, we asked whether similar dual-task costs could be observed when the participants first made a saccade, and then a reach. 

Methods

Seven participants (mean age 25 years, 3 females) participated in the study. All participants had normal or corrected-to-normal vision. Informed consent was obtained from all participants. 

The procedure was the same as in Experiment 1, with the following exceptions. The first movement cue now directed the saccade, while the second movement cue directed the reaching movement. SOA between the cues varied between 150 and 600 ms (150, 200, 250, 300, 350, 400, 450, 500, and 600 ms). Each participant completed 3 blocks of 144 trials. 

Results

After the first movement cue appeared, a saccade was initiated with a similar latency for all SOA conditions (repeated measures ANOVA, main effect of SOA not significant, F(8, 48) = 0.37, p > 0.9). Thus for the SOA 150-ms condition, i.e., the shortest SOA, mean saccade latencies were 314 ± 33 ms, which was not different from the longest SOA, the SOA 600-ms condition, in which saccade latencies were 334 ± 46 ms. In contrast, reaching latencies showed pronounced dual-task costs—as SOA increased, reaching latencies decreased (F(8,48 = 8.05, p < 0.01)). For the SOA 150-ms condition, mean reaching latency was 499 ± 27 ms, which was longer than for the SOA 600-ms condition in which mean reaching latency was 417 ± 25 ms (t(6) = 6.06, p < 0.01). Thus, in the present task the reach latencies revealed dual-task costs of around 80 ms (mean RT at SOA 150 ms − mean RT at SOA 600 ms). Figure 4 shows saccade and reach latencies as a function of SOA for trials when saccades and reaches were directed to the same location or to different locations. Again, saccade latencies were not shorter when saccades and reaches were directed to the same location (repeated measures t-test, all ps > 0.05). On the other hand, reach latencies were affected by saccade target location. When a two-way ANOVA was performed, with SOA and saccade/reach location agreement as factors, the main effect of saccade/reach location agreement was not significant (F(8,48) = 1.09, p > 0.30), but interaction between the two factors was (F(8,48) = 2.44; p < 0.05). For the SOA 150-ms condition, reaches directed to the saccade location started after 488 ms ± 26 ms; these latencies were 63 ms shorter than when the reaches were directed to a different location (551 ms ± 42 ms; repeated measures t-test, t(6) = −2.46, p < 0.05); none of the other SOA conditions showed significant differences. The benefit observed at the SOA 150-ms condition could be explained by previous observations that people are faster to reach to the objects they are allowed to look at (Prablanc, Echallier, Komilis, & Jeannerod, 1979). 

Figure 4

 

Dual-task interference in Experiment 2. The first movement cue directed the saccade target; the second movement cue instructed the reach target. Data are shown for trials when saccades and reaches were directed to the same location (red line) or to different locations (blue line). Symbol in (B): *p < 0.05. Vertical bars indicate ±SE. Data are slightly translated horizontally to increase the visibility of different conditions.

Finally, we analyzed movement errors. On trials when saccades and reaches were directed to different locations, participants made 23% of errors by looking at the location they were supposed to reach, and on 4% of trials they reached to the location they were supposed to look at. The proportion of errors did not vary as a function of SOA (ANOVA for saccade errors, with SOA as the main factor—F(8,48) = 0.38, p > 0.9; F(8,48) = 0.99, p > 0.4 for reach errors). This demonstrates that there is some crosstalk when saccade and reach targets have to be selected. It is not clear, though, whether these saccade and reach errors are due to participants being used to look and reach to the same locations in everyday situations. 

In sum, these results show that dual-task costs arise for reaches when the saccade was executed first. 

Experiment 3

In two parts of Experiment 3, we tested directly whether movement goal selection (in contrast to movement execution) is affected by the need to do two tasks simultaneously. As it has been shown that attention may shift to saccade and reach locations early during movement planning (Deubel & Schneider, 1996; Rizzolatti, Riggio, & Sheliga, 1994), we measured attention at saccade and reach locations by presenting an attentional probe—a briefly shown letter that participants had to report at the end of the trial. During this task, the first cue indicated a reach target and a subsequent cue indicated the saccade target (like in Experiment 1), and a probe could appear sometime during saccade or reach planning at different locations on the screen. If participants shifted their attention to saccade or reach location, probe discrimination should be better at those locations than at other locations, to which no movement was planned. During the experiment, we also varied probe presentation time, which allowed us to determine at which point in time attention shifted to saccade or (and) reach locations. For example, it could be that attention deployment associated with saccade planning is delayed as long as the reaching does not start, leading to the prolongation of the saccade latencies as demonstrated in Experiment 1. Alternatively, it is possible that there are no dual-task costs to select a saccade goal even when the selection occurs during reach planning—this should be reflected in a parallel attention allocation to both saccade and reach targets before reaching onset. 

Methods

Participants

Eight participants (mean age 23 years, 3 females) participated in Experiment 3. Ten participants took part in the “Saccade-only” control task (mean age 25, 4 females). They had normal or corrected-to-normal vision. Informed consent was obtained from each participant. 

Apparatus and procedure

The apparatus and procedure of the experiment were the same as in Experiment 1, with the following exceptions. After the first movement cue appeared, participants had to reach to the object indicated by the cue. There were only 4 possible reaching locations (at 2, 5, 7, and 10 o'clock). With an SOA of 150 or 400 ms after the first cue, a second arrow cue was shown indicating the saccade goal (see Figure 1). The saccade goal could be located 3 or 5 items clockwise or anti-clockwise from the reaching location. Saccade and reaching movement goal selection was measured by presenting a probe stimulus. For this purpose, the display containing the mask elements changed into a display containing 11 distractor digits (digital “2” and “5”) and one target character (digital letter “E” or digital “3”). This probe display was presented for 80 ms and was then masked. Given the short presentation time of the probe display, the probes could be detected only if participants attended to the probe location at the time when the probe was presented. The probe display could appear randomly in a time interval ranging from 50 ms before to 650 ms after the onset of the first movement cue. In other words, the mask elements could change into probe and distractors at any point of time, before the appearance of the first movement cue, up to a point in time when both movements were already finished. The probe was presented either at the saccade goal (33% of trials), at the reach goal (33% of trials), or at one of the other, movement-irrelevant locations (33% of trials). Participants were asked to indicate the probe identity (“E” or “3”) at the end of each trial. We analyzed only trials where the probe appeared before eye movement onset. 

Each block consisted of 144 trials. Participants completed at least 6 blocks of the task. 

“Saccade-only” control experiment

In order to provide a baseline on how attention shifted to saccade goals when no simultaneous reach were to be made, we additionally performed a control experiment in which participants only looked at the object, without executing any reach movement (Saccade-only task). The design of this experiment was identical to Experiment 3, except that only one movement cue was presented. Participants had to saccade to the location indicated by the cue. The probe could be presented at the saccade target (50% of trials) or at a randomly selected, movement-irrelevant location (50% of trials). Each participant performed at least 4 experimental blocks of 192 trials each. 

Results

As in the previous experiments, we observed dual-task costs when the planning processes for the two movements overlapped in time. Saccade latencies decreased with increasing SOA (316 ± 22 ms for SOA 150 ms as compared to 239 ± 13 ms for SOA 400 ms, repeated measures t-test, t(7) = 6.53, p < 0.01). On the other hand, reaching latencies were not affected by the SOA condition (286 ± 15 ms for SOA 150 ms and 286 ± 15 ms for SOA 400-ms conditions, repeated measures t-test, p > 0.05). Thus, saccade initiation was delayed if the reach was still being planned at the time of saccade cue presentation (SOA 150 ms). Saccade initiation was not delayed if the reach had already started, which was the case for the SOA 400-ms condition. 

Next we analyzed whether participants were able to select movement goals during the preparation of the movements. For this purpose, we used probe discrimination rate as a measure of movement goal selection. Since the probe was presented at variable times, we were able to analyze the time course of attentional deployment to the probe locations. For each time point (every 50 ms), we calculated the proportion of trials in which participants correctly discriminated the probe. As we were interested in the shift of attention to saccade and reach goals before the movement onset, we excluded all trials in which probes were presented either after saccade or reach onset. The results are depicted in Figure 5. It can be seen that after the reach cue appeared, participants were at chance to discriminate the probes if they were presented at movement-irrelevant locations (probe discrimination was not different from chance level, p > 0.05). In contrast, probe discrimination at the reach goal increased gradually following the presentation of the respective cue. Further data analysis revealed that 50 ms after presentation of the reaching cue, participants became better than chance to discriminate probes presented at the reaching location (t-test comparing discrimination at reaching location versus 50% chance level, t(7) = 2.56, p < 0.05). This demonstrates that participants shifted their attention to the reach goal before the start of the reaching movement. 

Figure 5

 

Probe discrimination rate at saccade and reach goals in the dual task of Experiment 3 (results do not include data from the Saccade-only task), as a function of time after reach cue presentation. Vertical dashed lines indicate the onsets of the cues for respective movements (e.g., blue dashed line—reaching cue presentation). Vertical arrows indicate the average movement latencies (e.g., blue arrow—reaching onset). Vertical error bars indicate ±SE. Data are slightly translated horizontally to increase the visibility of different conditions.

For the SOA 150-ms condition, and about 100 ms after the saccade cue appeared, probe discrimination became significantly better than chance also at the saccade goal (t(7) = 5.13, p < 0.01). After this point in time, i.e., already relatively long before saccade onset and also before the onset of the reach movement, participants were consistently better than chance to discriminate probes presented at the location of the saccade goal (all ps < 0.05). This shows that the attentional shift to the saccade goal started well before saccade onset. These results are in line with previous demonstrations showing increased probe discrimination at the saccade locations (Baldauf & Deubel, 2008; Deubel & Schneider, 1996; Godijn & Theeuwes, 2003). 

Two important conclusions can be drawn from these results. First, improvement of probe discrimination at the saccade target was better than chance already before the reaching movement started. So, even though these saccades were markedly delayed due to the dual-task conditions, participants did not delay the selection of the saccade goal until after they started their reaching—the saccade goal was obviously selected before the start of the reaching movement. Second, the data demonstrate that attention was allocated to the two target locations simultaneously, as participants were better than chance to discriminate probes presented at both the saccade and the reaching goal before the reach started. 

Somewhat unexpectedly, we found that for the SOA 400-ms condition, discrimination rate at the saccade goal increased already 150 ms before the saccade cue appeared (t(7) = 2.88, p < 0.05); from that time onward, participants were better than chance to discriminate probes presented at the saccade location. Note that after this initial increase in accuracy, discrimination rate at the saccade goal did not change over time until the appearance of the saccade cue. Only then, discrimination performance improved further. The predictive increase in probe discrimination accuracy suggests that participants tried to anticipate where they would have to make a saccade. If we assume that participants split their attention evenly between 4 possible saccade target locations, and given that probability to guess the probe identity correctly was 0.5 in our two-alternative forced choice task, then probe discrimination at possible saccade target should be 63% (1/4 + (1 − 1/4) * 0.5), which was similar to what we observed. 

It should be noted here that, given the similarity of the initial increase of discrimination performance for both SOA conditions (red and green curves in Figure 5), we cannot exclude that anticipatory effects may also be involved in the SOA 150-ms condition. However, the assumption that the early attention shifts to the saccade target in the SOA 150-ms condition are elicited by the presentation of the saccade cue seems to be more parsimonious. 

Further converging evidence for this assumption comes from the results of a parallel study in which we used a different combination of SOAs (SOA 150 ms and SOA 200 ms). In this study, we also observed that probe discrimination increased at the saccade location before the reach onset for the SOA 150-ms condition, while attention shifts were accordingly delayed for the SOA 200-ms condition (Jonikaitis & Deubel, in press, cf. Figure 4). Importantly, there were no anticipatory attentional shifts apparent for the SOA 200-ms condition in this study, which further confirms that saccade targets can indeed be selected during reach planning. 

Thus combined we found that 250 ms after the reach cue appeared (time when green and red curves start rising in Figure 5)—and still before the reach onset—probe discrimination was already better than chance at either the already specified saccade goal (SOA 150-ms condition) or at the yet to be specified saccade goal (SOA 400-ms condition). These two observations strongly argue that reach movement planning did not prevent the attentional shift to specified or potential saccade locations; thus, attentional selection of saccade locations was not delayed in time. 

Even though participants shifted their attention to saccade locations during reach planning, it could be that this was an effect observed by mixing two different groups of trials in our design—on some trials, saccade and reach locations were close by, and on other trials, those locations were further away. Participants could have shifted their attention only to saccade locations further away or to saccade locations in the different visual hemifield than the reaches (for example, Alvarez & Cavanagh, 2005). To assess this possibility, we split the data by trials with saccades made to the opposite hemifield than the reaches and trials with saccades made to the same hemifield. We observed no effect on probe discrimination due to this data split (all ps > 0.7). We also split trials by whether saccade location was close or far from the reach location (3 or 5 items away from the reach object on the display). Again, we observed no discernible differences (all ps > 0.5; Figure 6). 

Figure 6

 

Probe discrimination at saccade goal in Experiment 3 as a function of the distance between eye and hand movement goal locations. Dashed gray line indicates saccade onset. Vertical error bars indicate ±SE. Data are slightly translated horizontally to increase the visibility of different conditions.

Even though participants were able to select the saccade target before reaching onset, it is still possible that participants would have selected the target faster if there were no need to perform simultaneous reaching. In other words, the observed dual-task costs may have partly arisen because saccade target selection was somewhat delayed (even though it started before the reaching onset). To investigate this possibility, we compared the discrimination performance from the dual-task conditions with performance in the Saccade-only task, which did not include a reaching movement. Figure 7 shows discrimination performance, aligned to the time of saccade cue presentation, for the dual-task conditions and for the Saccade-only task. It can be seen that probe discrimination increased at about the same time after saccade cue onset in both the dual‐task and the Saccade-only task. We calculated at which time probe discrimination after the saccade cue onset was better than performance 50 ms before the saccade cue onset (in order to equate for baseline differences in discrimination before cue onset in the SOA 400-ms task). This analysis shows that 100 ms after saccade cue onset in the Saccade-only task probe discrimination was better than baseline (t(9) = 3.21, p < 0.05); the same time value was found for the SOA 150-ms task (t(7) = 4.86, p < 0.01) and for the SOA 400-ms task (t(7) = 2.85, p < 0.01). Figure 7 includes also the data where the probe appeared after saccade onset. Note that in the Saccade-only task probe discrimination reached a certain level before the saccade and improved after saccade onset, as participants were then looking at the target directly. Interestingly, in the SOA 150-ms condition, probe discrimination at the saccade goal kept improving as long as the saccade did not start. The similarity of the temporal dynamics between the two conditions (SOA 150 ms and Saccade-only task) is striking, even though the saccade started in one condition considerably earlier than in the other. So, although saccade execution was considerably delayed in the SOA 150-ms condition, the attention shifts to the saccade target were not delayed. In other words, reach planning and execution did not delay saccade goal selection in the dual-task condition, as probe discrimination was not different from that in the Saccade-only task. 

Figure 7

 

Probe discrimination at saccade goal during the dual task of Experiment 3 and the Saccade-only task. Dashed gray line indicates saccade cue presentation. Vertical arrows indicate average saccade latencies (i.e., saccade onset times) in the different conditions. Note that for both dual- and single-task conditions attention allocation to the saccade goal follows the same pattern, even after the onset of the saccade in the control Saccade-only task. Vertical error bars indicate ±SE. Data are slightly translated horizontally to increase the visibility of different conditions.

Further evidence that the delay of the saccades in the SOA 150-ms task was not related to the timing of the presaccadic attention shifts was provided by an analysis of the temporal relation between presaccadic attention shift and saccade onset. In a different study, we observed that saccades with shorter latencies are normally preceded by an earlier attention shift to the saccade target (Jonikaitis & Deubel, in press)—the faster the participants shifted their attention to the saccade goal, the shorter were the saccade latencies. For the dual-task conditions of the present experiment, we expected to find this tight temporal coupling between attention shift and saccade onset for the SOA 400-ms condition, where reaching movement and saccade processing no longer interfered. For the SOA 150-ms condition, however, the coupling should disappear, given that the dual-task costs as reflected in the saccade delay were unrelated to the presaccadic attention shift. In order to test this prediction, we split, for each SOA condition and each participant, saccade latencies by median into short latency saccades and long latency saccades. 

As we had expected, saccadic reaction times were not related to speed of attention deployment in the SOA 150-ms condition. While the median split in this condition leads to a mean latency of 270 ± 16 ms for the faster saccades and 363 ± 29 ms for the slower saccades, this difference is not reflected in the attentional allocation for longer or shorter latency saccades (Figure 8, left panel, all repeated measures comparisons not significant). 

Figure 8

 

Probe discrimination before short and long latency saccades. Saccade cue appeared at time 0. Vertical arrows—average short and long saccade latencies. Color curves—probe discrimination rate at saccade target when saccades latencies were short (red line) or long (blue line). Vertical error bars indicate ±SE.

In the SOA 400-ms condition, the trials with faster saccades had an average saccade latency of 209 ± 16 ms; the trials with slower saccades had an average latency of 270 ± 17 ms. As can be seen from the right graph of Figure 8, probe discrimination at the saccade goal increased earlier for the trials when saccades had shorter latencies than on trials with longer latency saccades. At 150 ms after the saccade cue, discrimination was better at the saccade target if the saccade latencies were shorter (t(7) = 2.56, p < 0.05), while attention deployment occurred considerably later for the slower saccades. Thus, at the time when there were no dual-task costs observed, earlier attention shifts were associated with shorter saccade latencies. This finding suggests that the dual-task costs in the SOA 150-ms condition arise at a processing stage different from—and probably later than—the attentional selection of the saccade goal. 

Together the findings show that the participants were able to shift their attention to saccade and reaching goals before reaching onset, and that there was no delay in saccade goal selection while the reaching was planned. Thus, while the saccade onset became markedly delayed due to the dual-task condition, this delay was not reflected in the time course of attentional allocation before the saccade. 

Discussion

Our experiments investigated whether dual-task costs in the simultaneous planning of eye and hand movements result from a competition for attentional resources. Movement latencies showed large dual-task costs when saccades had to be planned during reaching preparation. However, these costs did not arise from the attentional selection of the movement goals. The results show that participants can shift their attention to a saccade target even while the reaching movement is being planned and has not yet started. 

Dual-task costs in the planning of saccades and reaches

We found that there were large dual-task costs when the planning of goal-directed saccades and reaches overlapped in time. Our results are comparable to findings reported by Pashler et al. (1993). In their study, participants were not able to elicit a saccade if the central cue instructing the saccade appeared while the participants performed a tone discrimination task requiring a manual button press. The magnitude of the effects found in our study (about 100-ms dual-task cost for saccades made in the SOA 150-ms condition) was about equal to the effect observed in Pashler et al.'s study (also 100-ms cost for SOA 150 ms). The main difference between these two studies is that in our experiments participants had to plan two movements directed to different locations, whereas in the study of Pashler et al. the first task was a button press and the second task was a goal-directed saccade. 

The dual-task interference observed in our experiments could result from various stages of movement planning. In our task, participants had to interpret each cue, select an appropriate response (to make an eye or a hand movement), and to plan the movement itself. Movement planning consists of selecting an appropriate target for the movement and specifying all movement parameters. Additionally, factors such as uncertainty about when the second cue appears (Gottsdanker, 1980), impaired timing judgments during dual tasks (Brown, 1997), and confusability regarding the direction of motion of the effector (Huestegge & Koch, 2009) may also play a role. Our results suggest that one of the most important processes involved in the task, namely the selection of the movement goals, did not cause the dual-task interference. It remains to be investigated at which stage during movement planning the interference actually occurs. 

It is difficult to directly compare our results to some of the other studies that investigated saccade-reaching dual-task costs, since these did not systematically manipulate the overlap between saccade and reach planning (Bekkering et al., 1994, 1995; Lünenburger et al., 2000). Although it has been reported that saccade latencies are shorter if concurrent reaches to the same object are planned (Lünenburger et al., 2000), the opposite pattern of results was found in a different set of studies (Bekkering et al., 1994, 1995). It is possible that the requirement to make two movements to the same object simultaneously might evoke a pattern of eye–hand coordination that is “hard-wired.” For example, both Lünenburger et al. and Bekkering et al. have suggested that the superior colliculus might mediate the observed coupling between the eye and hand, as some of the neurons in intermediate and deep layers of superior colliculus are known to fire before arm movements (Werner, Dannenberg, & Hoffmann, 1997). The assumption that simultaneous eye and hand movements might be coordinated in a special way is also supported by the finding that saccade durations decrease if saccades are made simultaneously with hand movements (Snyder, Calton, Dickinson, & Lawrence, 2002). 

Movement goal selection for eye and hand

Another matter of debate in eye–hand movement studies has been whether the target representation for movement planning is shared between both systems or is separate. We did not find a saccadic latency benefit when the saccade was planned to the same location as the reach. Thus, even though participants selected a target for the reach, they were not faster to saccade to that same target than to saccade to a different target. This indicates that movement goal selection for the eye and the hand movements is relatively independent. In other words, when the reach is planned, the saccade does not have to be planned to the same target (for a similar observation, see also Stritzke & Trommershäuser, 2007). 

Our results argue against some findings that were interpreted as showing that eye and hand movement goal selection is shared. Neggers and Bekkering (2000, 2001), for example, reported that if participants are making a saccade and a reach to the same object, they are unable to move their eyes to a different location while the hand is still moving. In addition, it has been observed that saccade latencies are longer if a simultaneous hand movement is planned to the same location (Bekkering et al., 1994, 1995). 

We think that those studies could be interpreted in a different way—it might be advantageous to keep the eyes stable while the hand movement is planned or executed. A number of studies, behavioral and neurophysiological, show that eye position influences the planning for reaching and pointing (Batista, Buneo, Snyder, & Andersen, 1999; Medendorp & Crawford, 2002; Medendorp, Goltz, Vilis, & Crawford, 2003). This indicates that the visual system keeps track of where the hand and the reach goals are relative to the eye and suggests that every eye movement requires the recalculation of the hand movement goal position with respect to the new eye position. Thus, keeping the eyes stable might be advantageous for fast hand movement planning, but this coupling does not necessarily mean that movement goal selection is shared for eye and hand movements. Further research needs to be carried out to clarify whether targets for eye and hand are selected independently. 

In Experiment 3, we demonstrated that two targets, one for the saccade and one for the reach, can be selected in parallel, before reaching movement onset. In other words, before reaches started, participants were attending simultaneously to both saccade and reach locations. In addition, attention was allocated to the saccade goal immediately after the saccade cue onset—regardless of SOA. Thus, it did not matter whether the hand movement was planned at that time or not—participants selected the saccade target immediately after saccade cue onset. This demonstrates that saccade goal selection was independent of whether the reach goal was selected at that time or not. The finding further supports the conjecture that the mechanisms selecting the goals for eye and hand movements are dynamically independent (Jonikaitis & Deubel, in press). 

Split attention

We also demonstrate that attention can be split to multiple locations, as illustrated by our finding that probe discrimination was better than chance at saccade and reach goal locations before reach onset. That attention can be split has been proposed in a number of studies (e.g., Adamo, Pun, Pratt, & Ferber, 2008; Alvarez & Cavanagh, 2005; Awh & Pashler, 2000; Bichot, Cave, & Pashler, 1999); however, this view has also been vigorously objected (e.g., Dubois, Hamker, & VaRullen, 2009; Jans, Peters, & Weerd, 2010). Our data clearly support the view that attention can be split to parallel locations in a task involving the preparation of eye and hand movements, in line with further recent evidence (Jonikaitis & Deubel, in press). One interesting question concerns how this split is achieved. Our task, contrary to typical tasks investigating parallel attention foci, did not explicitly instruct attention to shift to any location. The main task was the movement task, and we observed that probe discrimination increased at the movement goal locations. The shift of attention to the movement goals seems to be involuntary to some degree, as probe discrimination at movement goal locations increases even when participants are explicitly informed that probe is more likely to appear at other locations (Deubel & Schneider, 1996; Jonikaitis & Deubel, in press; Tibber, Grant, & Morgan, 2009; Wilder, Kowler, Schnitzer, Gersch, & Dosher, 2009). This seems to be true also in cases where no discrimination task is present, but attention is measured using ERPs (Baldauf & Deubel, 2009). Moreover, attention was found to shift to multiple locations when a sequence of eye or hand movements to multiple targets is prepared (Baldauf & Deubel, 2008, 2010; Baldauf, Wolf, & Deubel, 2006; Godijn & Theeuwes, 2003). All these evidences suggest that attentional resources can be distributed to multiple targets during the planning of combined eye and hand movements as shown here, as well as during the preparation of movement sequences. 

The question still remains as to the relationship between automatic attention allocation to movement goals as studied here and the intentional, simultaneous attention allocation to multiple stimuli. It could be that different attentional resources exist for the shifting attention before movement onset and the intentional attending to other locations (Montagnini & Castet, 2007). While this question remains to be investigated, our data support the view that attention can be transiently split. 

Inhibition of return

We also observed that saccades were delayed when participants already reached to that location. This effect occurred late, at an SOA of around 700–900 ms and thus was within the time frame when Inhibition of Return (IOR) is known to occur (Klein, 1988, 2000). IOR is regarded as a mechanism that discourages attentional (or saccadic) revisiting of previously attended locations. Our results show that targets selected for hand movements can inhibit saccadic orienting to those targets. In other words, within the IOR time frame, participants tended not to direct saccades to the locations they already reached at. 

It has been suggested that IOR originates from either attentional or saccadic systems. A possible attentional explanation of our findings is that participants shifted their attention to the hand movement target when they planned the hand movement. Later, when the saccade had to be planned to that same target, the shift of attention to this location was delayed, resulting in the observed IOR effect. 

Another possible explanation is that the observed IOR is a saccadic effect (Theeuwes & Godijn, 2002). It could be argued that participants planned a saccade to every reach target—without executing the saccade, which resulted in an IOR effect. However, if this were the case, then at short SOAs saccades directed to the reaching goal should have been faster than saccades directed elsewhere, a result that we did not observe (see Figure 2). Our findings thus argue for an attentional origin of IOR. 

Reaction time is not attention

A striking observation of this study is that while saccades showed large dual-task costs as measured in saccadic latencies, there were no attentional target selection costs, i.e., the attention shift preceding the saccade showed no delay. This is surprising given the common assumption that attention and saccades are closely coupled when people are asked to make speeded responses while eye or hand movements are planned. The clear dissociation between saccadic reaction time and attentional selection indicates that caution should be taken in using saccade or hand movement latencies as a measure of target selection or attentional allocation. Instead of attentional processing, the latencies may merely reflect dual-task constraints occurring at later stages of sensorimotor processing. 

Acknowledgments

This study was supported by the Deutsche Forschungsgemeinschaft (GRK 1091) and by the 7th Framework Program of the European Community (Project “GRASP,” ICT-215821). 

Commercial relationships: none. 

Corresponding author: Donatas Jonikaitis. 

Email: [e-mail]. 

Address: Allgemeine und Experimentelle Psychologie, Ludwig-Maximilians-Universität, München, Germany.