The spatiotemporal characteristics of the attentional shift relative to a reach

Abstract

While the attentional shift preceding a saccadic eye movement has been well documented, the mechanisms surrounding the attentional shift preceding a reach are not well understood. It is unknown whether these mechanisms may be the same as those used in perceptual tasks, or those used in the planning of a saccade. We mapped the spatiotemporal properties of attention relative to a reach to determine the time course of attentional facilitation for hand movements alone. Participants had to reach toward a target and during the reach a perceptual probe could appear at one of six locations around the target, and at nine temporal offsets relative to the cue. Results showed a consistent pattern of facilitation in the planning stages of the reach, with attention increasing and then reaching a plateau during the completion of the movement before dropping off. These results demonstrate that planning a hand movement necessitates a shift in attention across the visual field around 150 ms before the onset of a reach. While these results are broadly consistent with the results of experiments mapping attentional shifts for saccades, the spatiotemporal profile of facilitation found shows that reaching without a concurrent eye movement also causes shifts in attention across the visual field. These results also suggest that the profile of the attentional shift preceding and during a hand movement is different at different locations across the visual field.

Introduction

In everyday life we are confronted with an overwhelming amount of visual information, some of which is behaviorally relevant, but most of which is not. Visual attention allows us to choose which areas are relevant, and can lead to a perceptual facilitation at those areas (Carrasco, 2006; Carrasco, Ling, & Read, 2004; Liu, Abrams, & Carrasco, 2009; Treue, 2004). Specifically, attention seems to play an important role in the planning and execution of saccades (Deubel, 2008; Deubel & Schneider, 1996; Kowler, Anderson, Dosher, & Blaser, 1995; Zhao, Gersch, Schnitzer, Dosher, & Kowler, 2012), and studies have suggested that the same might be true of hand movements (Baldauf, Wolf, & Deubel, 2006; Jonikaitis & Deubel, 2011; Song & Nakayama, 2006), with visual attention providing guidance for movements. 

While the attentional shift surrounding hand movements has been explored relative to saccadic endpoint versus reach endpoint (Deubel & Schneider, 2003), or relative to the same or opposite pointing location (Rolfs, Lawrence, & Carrasco, 2013), the specific spatial and temporal properties of this shift are still unknown. In particular, the precise perceptual benefit gained from a shift of attention preceding or during a reach is unclear. We measured the spatial and temporal spread of attention before and during a goal-directed movement made without a concurrent eye movement. 

Attention and movement planning

Much research has been conducted to determine how a shift in attention may relate to a movement, and the mechanisms underlying this allocation of spatial attention. Theories such as the premotor theory of attention argue that the mechanisms responsible for spatial attention and the programming of saccades are the same; that is, programming a saccade causes a shift in attention (Rizzolatti, Riggio, & Sheliga, 1994). This link between attention and an impending movement has been primarily studied in relation to saccades, and some studies have suggested that there is a close link between attention and saccades (Deubel & Schneider, 1996, 2003). Contrary to these studies, there is some evidence to suggest that, while attention plays an important part in saccadic programming, attention and saccades can be dissociated to some extent (Kowler et al., 1995). 

Similarly, the evidence surrounding the link between attention, saccades, and hand movements has not yet provided a definitive answer about whether the mechanisms underlying the attentional shift are the same for perception, saccade, and reach. Given the close link between saccades and reaches, it's important to note that measuring the spatiotemporal profile of attention during a reach may give a clue as to how attentional resources are shared between reaches and saccades. While this study focuses on reaches alone, much research has been conducted to try to discern how attention may shift for these different, but closely linked, movements. 

One of the ways in which researchers have attempted to dissociate the mechanisms underlying the shift for both saccades and reaches is by looking at the temporal profile of the attentional shift for these movements. If a difference in the timing of the attentional shift between these movements is found (e.g., Jonikaitis & Deubel, 2011), then it is taken as evidence for a different mechanism driving the attentional shift for each movement type. While many studies have examined the spatial properties of attention relative to a movement (Deubel, 2008; Deubel & Schneider, 2003), looking at the temporal properties may provide a clearer answer as to whether attention shifts on the same time-scale for eye and hand movements. If the time course is similar, this would suggest a unified mechanism for these movements. If the time course is different, it would suggest that the mechanisms driving the attentional shift may be separated. 

The importance of vision in planning a reach

Visual information is very important in the planning of a precise and accurate hand movement (Sober & Sabes, 2005), both at the outset and during the completion of the reach (Ma-Wyatt & McKee, 2006, 2007). It is logical, therefore, to suggest that the visuomotor system would benefit from the increased perceptual sensitivity that attending to the reach location could provide. 

This shift in attention relative to a reach has primarily been studied in relation to a concurrent saccade and reach, and it is not well understood how attention may shift when a reach is being completed in isolation. The link between attention and reaching has been studied in relation to the location of the attentional benefit accompanying both a saccade and a reach, and this research suggests that attention is tied to the location of the impending reach (Deubel & Schneider, 2003). Moreover, when multiple sequences of movements are being planned, attentional resources can be allocated to each of these upcoming reach locations in parallel (Baldauf et al., 2006). These studies show that the locus of attentional benefit is related to the locus of the planned reach. It is suggested, therefore, that attention is necessary for the planning of a hand movement. It has been postulated that the trajectory of a reaching movement over the course of the reach could reflect the shifting locus of attentional focus, which also reflects the motor system's internal target selection process (Song & Nakayama, 2006). This provides evidence for the idea that the locus of attention may also shift during the execution of a reach as the motor plan for the reach is updated throughout the movement. 

Studies that have looked at sequences of hand movements (Baldauf et al., 2006) show that when movements are being planned, there is a certain spatial “window” of attention that accompanies the movements. Baldauf et al. (2006) found that all impending reach locations showed an attentional benefit compared to nontarget locations. It has also been observed that when preparing a reaching movement to a sequence of locations, attention spreads to all action-relevant goals in parallel, with the amount of the attentional enhancement reflecting the serial order of the impending movement goals (Baldauf & Deubel, 2010). Additionally, there is evidence to suggest that when preparing a bimanual reaching task, attentional facilitation is observed at both intended movement goals (Baldauf & Deubel, 2008), and that when planning a sequence of actions, the whole sequence is planned in advance of movement initiation (Hesse & Deubel, 2010). This parallel allocation of attention is also supported by ERP evidence, showing that multiple movement goals are attended to with the same level of attentional resource (Baldauf & Deubel, 2009). 

This attentional selection of movement locations gives a broad idea of how large the attentional window might be, and how it might behave in relation to different movements. Indeed, it has been suggested that preparing a manual movement creates an “attentional landscape,” which takes into account all locations that are relevant for planned, goal-directed movements (Baldauf & Deubel, 2010). If attention is allocated only to the location of a reach, one would predict that locations close to the reaching target would show an attentional benefit, whereas locations further away would not. If, however, the findings of a dissociation between saccade and attention along multiple paths hold true for reaches as well, one might expect a broader pattern of attentional facilitation across the visual field. The previous work in this area has not mapped in detail the manner in which attention may spread across the visual field relative to a reaching goal. While researchers have mapped attention in relation to a small number of sequential pointing locations, this does not indicate how attention spreads, and the temporal manner in which it may shift across the visual field relative to a reaching movement. 

This does not, however, fully solve the problem of whether the mechanism that underlies attentional modulation for saccadic and perceptual tasks is the same as the mechanism used during the planning and execution of a hand movement. We are interested in the attentional mechanisms underlying both saccade and reach, even though this study aims to solely explore reaches. However, since saccades and hand movements are tightly linked, patterns of the time course and spread of the attentional shift relative to a saccade may also hold true when looking at the attentional shift relative to a hand movement alone. In particular, if the mechanisms underlying both saccades and reaches are similar, we would expect a similar pattern of attentional facilitation for hand movements as that which has previously been seen for saccades. A number of studies have attempted to determine whether the mechanisms underlying saccade and reach preparation are shared, or whether they can be dissociated. While this paper does not directly compare these mechanisms, we believe that it is an important problem to note, and puts the results of this study into the context of a greater overall problem. 

The evidence provides divergent results, with some studies suggesting a tight coupling between saccades and hand movements, and some suggesting that separate resources may be responsible. Evidence for dissociated resources can be seen in dual-task paradigms in which a manual movement could be planned without an effect on a concurrent saccade task (Jonikaitis, Schubert, & Deubel, 2010; Sharikadze, Cong, Staude, Deubel, & Wolf, 2009). Brain lesion studies also provide compelling evidence for separate resources for different attentional tasks: in a study comparing covert attention and presaccadic attention, it was found that covert attention shifts were impaired but presaccadic perceptual facilitation was not impaired (Blangero et al., 2010). If this is the case then we would expect the spatiotemporal pattern of attention relative to a reach to be different to the profile that has been measured relative to saccades in these previous studies. 

There is, however, evidence to the contrary, suggesting that attentional resources underlying reaches and saccades are tightly coupled. For example, in a visual search task the final saccade in each trial was always directed to the goal of a concurrent reaching movement, suggesting that the movements are coupled (Song & McPeek, 2009). Additionally, evidence for shared resources can be seen when the eye and hand must be directed to different goals—in this case the majority of the attentional resources are allocated to the saccadic task, leaving little attention to be directed to the goal of the hand movement (Khan, Song, & McPeek, 2011). 

This divergent evidence makes it hard to determine whether the attentional shift preceding a saccade is indeed comparable to the shift that occurs when a hand movement is being planned and executed. One method that can be implemented to dissociate the attentional mechanisms behind reaches and saccades is examining the temporal profile of this attentional shift. This may prove useful in attempting to determine the mechanisms behind presaccadic and prereach attentional shifts, and give some insight into how these attentional resources may or may not be shared. This study provides a basis for future comparisons of saccades and reaches using this technique. 

The temporal profile of attention relative to a reach has been studied to a far lesser extent than the time course of the attentional shift preceding saccades alone. However, the benefit seems to be similarly demonstrated on the order of 100–150 ms before the reach onset (Jonikaitis & Deubel, 2011; Rolfs et al., 2013). Research by Jonikaitis and Deubel (2011) has also provided evidence that the time course of attentional facilitation was different for saccades and hand movements: delaying an eye movement caused a delay in the attentional shift to the saccade location. However, the deployment of attention to the reach location was unaffected. This dissociation in the timing of the attentional shift provides support for the notion that the attentional control mechanisms for eye and hand are temporally independent. 

While this may provide an estimate of the temporal parameters surrounding this prereach attentional shift, there is no evidence suggesting exactly how and where this shift may occur. As the comparisons of attentional facilitation have primarily been tested by looking at perceptual performance at the reach location compared with a location on the opposite side of the visual field (e.g., Rolfs et al., 2013), this approach does not provide insight into the magnitude of the perceptual facilitation around the reach end point. We would expect to see this attentional facilitation at the reach end point specifically based on the previous literature, which has shown that attention does enhance perceptual performance at the location of a planned reach (Deubel & Schneider, 2003). 

This study aims to provide a comprehensive spatiotemporal map of the attentional facilitation preceding a reach. By mapping the attentional facilitation surrounding a hand movement without a concurrent saccade, we will provide a thorough overview of the spatiotemporal characteristics of the attentional shift relative to a reach alone. This quantification of the spread around the reach target, before the eyes move, is aimed to investigate covert attention at the reach target rather than the enhancement related to the increased resolution brought about by an eye movement. This is important as it will provide an understanding of how the planning and execution of a hand movement changes our visual representation of the world. While we are not testing the attentional shift that occurs with a concurrent eye and hand movement, this experiment will provide a solid basis for future mapping of the attentional shift with a concurrent eye and hand movement, which will then allow us to investigate the mechanisms which may be implicated in this shift. 

By probing six locations around the reaching target on each side of the screen over nine time points, this study aims to resolve some of the ambiguity surrounding the prereach attentional shift, and aims to map this shift with both a finer temporal and spatial resolution than has been attempted by previous research in the area. By mapping this spatiotemporal shift, it is hoped that the link between attention and the planning of a hand movement will become clear. Consistent with previous studies, it is expected that there will be a buildup of attention before the onset of the reach, and this will stabilize during the time course of the reach. We also wish to quantify the difference between the peak attentional facilitation at different locations across the visual field during the reach and the attentional facilitation at other time points during the reach. This will allow us to determine which time points during the reach attract the most amount of attentional resources. Additionally, this study includes a secondary paradigm that aims to compare the magnitude of attentional facilitation seen when a reach is being prepared with the magnitude of perceptual facilitation seen when a perceptual task is being completed with no concurrent movement. 

Methods

Participants

Six participants completed the study; two were authors, four were naïve as to the purposes of the study. Four of the participants were experienced psychophysical observers (two of the naïve participants and the two authors). All had normal or corrected-to-normal vision. All participants comfortably used their right hand to point, with ages ranging from 22 to 41 years. Ethics approval was obtained from the School of Psychology, and all participants signed informed consent forms prior to participation. Participants completed approximately 15 hr of data collection in 1-hr blocks. All participants were free to withdraw without penalty at any time. 

Equipment

Stimuli were presented on a 17-in. ELO touchscreen monitor, with a resolution of 1024 × 768 pixels and a screen refresh rate of 85 Hz. The monitor was calibrated prior to every session to ensure that contrast levels remained consistent across sessions, and to ensure that the monitor's nonlinear gamma function was corrected to be linear. Eye movements were measured using an EyeLink 1000 eye tracker (SR Research, Ottawa, ON, Canada) to record eye position during the task. Eye position was sampled at a rate of 1000 Hz with a spatial precision of 0.25°. The experiment was run using custom software written in MATLAB (MathWorks, Natick, MA) using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). The equipment was set up such that a mouse was located centered to the screen, and a keyboard was located to the left of the mouse to collect perceptual responses. 

Experimental design

The experimental design was a novel paradigm that aimed to measure the perceptual facilitation on a perceptual task while a reach was being planned and executed. This methodology is based on studies such as Castet, Jeanjean, Montagnini, Laugier, and Masson (2006), Rolfs and Carrasco (2012), and Rolfs et al. (2013), which use a predetermined threshold-level probe as a basis for determining the benefit in performance resulting from an attentional shift across the visual field. 

The experiment was a fully repeated measures design. All participants first completed a contrast threshold detection task, followed by interleaved blocks of a reaching and a perceptual task. All participants completed between 30 and 45 blocks of data collection, depending on how many trials were excluded. Each block consisted of 216 trials. 

The experiment comprised three separate tasks. The first task was a preliminary task that served to set individual contrast threshold levels for the perceptual probes that would appear in subsequent tasks. The second task was a reaching task measuring perceptual facilitation across the visual field while fixation was maintained and a reach was executed. The third task was a perceptual-only condition that used the threshold-adjusted probes to determine the perceptual facilitation that occurs covertly, when no reach is being made. 

Contrast threshold task

This preliminary task aimed to set the contrast value for the perceptual probe in the reaching and perceptual-only conditions by measuring the contrast threshold for each observer for each possible probe location. 

Individual contrast threshold measurements were taken for each participant at each of the 12 probe locations, with each location being tested separately. The participant maintained central fixation, and upon depressing a key the probe appeared at the location being tested. The probe was an oriented line (of the same length and orientation as the perceptual probes), which appeared for 20 ms. The participant reported the orientation of the line using the left or right arrows on the keyboard. Using a QUEST paradigm set to 82% threshold level (Watson & Pelli, 1983), the luminance of the subsequent probe underwent an increment if the previous response was incorrect, and a decrement if the response was correct. The background luminance of the screen was 27.5 cd/m². Forty trials were used to obtain the threshold measurement, and each location was tested three times, with the final threshold being the average of the three values. These values were then used such that each probe was presented at this threshold value for each participant. 

Pointing task

At the start of each trial a grey screen appeared with a central fixation point, which was 0.25° in size and 21% Michelson contrast from the background. Two touch targets located at 10° eccentricity from the centre also appeared. These targets were circles 0.75° in diameter, and 8% contrast from the background. Each circle contained a cross of 9% contrast, which formed the basis of the secondary discrimination task at the probe location (Figure 1). 

Figure 1.

 

Pointing task events. After a key press, a cue indicated the side of the screen to which the participant had to point. At a variable SOA after the cue, a perceptual probe (oriented line) appeared at one of six locations surrounding the reach target. A perceptual discrimination task also appeared at the touch location to ensure participants were directing their attention to that location.

As shown in Figure 1, the reaching trial started when the participant depressed a key on the mouse, which was affixed to the desk 40 cm from the screen, central to the screen and the participant's body. After a key press by the participant, a cue appeared at the center of the screen, signaling the side of the screen where the probe would appear (with 100% validity), and the side to which the participant had to point. 

The cue was a triangular arrow of 0.5° size and 17% contrast, which pointed to either the left or right side. After the cue had been displayed for 50 ms, a beep sounded to signal that the participant could start their reach to the target on the cue side. This beep also signaled the start of the stimulus onset asynchrony (SOA), after which a perceptual probe would appear at one of six locations on the cued side, and a discrimination task at the touch target would occur. 

Left and right cues occurred with equal probability, so perceptual probes appeared at any one of 12 locations around the screen with equal probability (see Figure 2). These locations were 4°, 8°, 12°, and 14° from the fixation point along the central horizontal axis, and two locations 3° and 6° above the touch target at 10° from fixation point. These specific probe locations were chosen so that they would not be occluded by the hand during the end of the reach, and thus ensured that the reach did not affect performance on the perceptual discrimination task. The oriented lines were 0.5° in length, oriented at an angle of 45° left or right, and were at a contrast level determined by an individual threshold task for each participant. Either the top or bottom section of the cross inside the touch target would have an increase in contrast for 20 ms, which formed the basis of the secondary discrimination task. 

Figure 2.

 

Possible locations where the perceptual probe could appear. The probe could appear at any of the probe locations on the cued side of the screen.

The SOA could be one of nine different lengths: 0, 59, 118, 189, 248, 307, 354, 425, or 496 ms, measured from 50 ms after cue onset. This range was chosen with the aim of capturing the attentional facilitation, which may accompany both the preparation and the completion of a reach. After the reach was completed, a post cue indicated whether the subject had to report the orientation of the perceptual probe (90% of trials), or the discrimination task at the touch target (10% of trials) using the arrow buttons on the keyboard (left/right for perceptual probe, up/down for target discrimination task). Auditory feedback was given for correct and incorrect answers. Feedback was also given if the participant reached too slowly (>600 ms) or if they started their reach too early (<100 ms), to try to constrain movements so that the probe appeared during the period of the planning and execution of the reach. For each trial touch location, reach latency, reach time, reach accuracy, and the perceptual response were recorded. 

The secondary discrimination task was used to ensure that participants were not directing attention covertly to possible probe locations across the visual field. Although they were instructed to direct attention to the reach target, this secondary task aimed to ensure this was the case. The results of this secondary discrimination task were not analyzed due to the occlusion of the target by the hand, making the results unreliable. 

Perceptual task

This task was exactly the same as the pointing task, except that no reach was made. This task aimed to compare the performance when a reach was being made, compared with performance when no reach occurred. As with the pointing task, each trial started with a gray screen and central fixation point. Even though there was no pointing in this task, the pointing targets were maintained for accurate comparison of the tasks. To start a trial, participants pressed a key, and after a variable delay (calculated as in the pointing task) the central cue appeared, signaling the side on which the perceptual probe would appear (Figure 3). The discrimination task at the touch target appeared on 40% of the trials, compared with the reaching task appearing on 10% of the trials. This increase aimed to ensure that similar attentional priority was placed on the touch target as when a reach was being planned to that location. 

Figure 3.

 

Perceptual task events. As with the reaching task (Figure 1), a cue indicated the side of the screen at which the perceptual probe (oriented line) would appear. This probe could appear at a variable SOA at any one of six locations. A perceptual discrimination task also appeared at the equivalent of the touch location in the reaching task to ensure participants were directing their attention to that location.

As with the reaching task, after the cue had been displayed for 50 ms, both the perceptual task and the discrimination task at the targets would appear after a variable SOA from the same range as the pointing task. The responses and feedback were collected and presented as above. Eye movements were monitored at all times to ensure that the participants were maintaining fixations in both the pointing and perceptual tasks. 

Results

Preliminary analyses

Data exclusions

Trials in which the participant failed to maintain central fixation were excluded. Based on the data recorded by the EyeLink, movements of >1° were excluded from further analysis. The remaining data were then filtered according to reach time and reach latency, such that ±2 standard deviations from the mean of these measures were excluded. Individual reach dynamics are shown in Table 1. Reach latency was measured as the time from the “go” signal beep until the time the participant lifted their finger from the mouse button. Reach time was measured as the time from the lifting of the finger from the mouse to the time the screen was touched. 

Table 1.

 

Reach dynamics for each participant.

 

Data sorting

Data was sorted according to the time the perceptual probe appeared relative to the onset of the reach. This was done by sorting the data into 50 ms bins according to the time the perceptual probe appeared relative to the reach onset time; for example, if the perceptual probe appeared 50 ms before the onset of the reach, the results of that trial would be put in the −50 ms bin. Data were sorted into 12 bins that spanned the range −200 to 300 ms in 50 ms intervals. All data were analyzed on a trial-by-trial basis to ensure that data were put into the appropriate bin for each trial. This was done in order to see how the onset and planning of a reaching movement affects the profile of attentional facilitation. 

Perceptual performance—pointing task

Data were weighted according to the number of trials per condition for each participant. Data were also collapsed across left and right sides such that only six locations were analyzed. Figure 4 shows mean performance across participants for each time bin and each location. 

Figure 4.

 

Perceptual performance per bin per location. Gray areas indicate time bins before reach onset; white areas indicate time bins after reach onset. The dotted vertical line indicates reach onset time. Error bars are 95% confidence intervals calculated using Jeffreys interval, which uses a Bayesian approach to calculate confidence intervals in binomial data. The 82% mark used to calculate baseline threshold values is indicated by the green horizontal line.

Percent correct varied as a function of time bin, for all locations. As can be seen from Figure 4, the pattern of variation was somewhat different across different locations. We used a generalized linear mixed model to analyze significant effects in these data. The model used fixed effects of time bin and location, and a random effect of participant. There was a significant main effect for time bin, F(10, 330) = 14.33, p = 0.00, and location, F(5, 330) = 54.60, p = 0.00. There was also a significant interaction between time bin and location, F(50, 330) = 2.59, p = 0.00, on perceptual performance. 

Attentional facilitation is the enhancement in the percentage correct performance that is above the 82% threshold level used to determine the baseline contrast threshold. The overall pattern of facilitation suggests that facilitation improves across the visual field before reach onset, and stabilizes during the early part of the reach. The locations along the horizontal axis relative to the reach target show a marked drop off in performance later in the reach, while the locations above reach target remain stable. This suggests that the profile of attention along the horizontal and vertical axis relative to the reach target is different. 

Maximum and minimum performance

From Figure 4, it is clear that attention builds up (that is, performance improves), and then generally drops off. This pattern is consistent with earlier findings that attention builds up over time at a location and that a peak in attentional facilitation can be identified relative to the onset of an eye movement or reach (e.g., Jonikaitis & Deubel, 2011; Rolfs & Carrasco, 2012). To quantify the magnitude of this attentional facilitation during the reach, we compared maximum and minimum performance at each location (see Figure 5). This difference gives a summary of the quantitative change in perceptual performance across the visual field, using the same data as shown in Figure 4. 

Figure 5.

 

Maximum and minimum performance per location, across all participants. Gray areas indicate time bins before reach onset; white areas indicate time bins after reach onset. The dotted vertical line indicates reach onset time. Error bars are standard error of the mean.

The maximum and minimum values for each participant were collated. Paired samples t tests were conducted to determine if the differences between the maximum and minimum performance at each location were significant. There were significant differences between the maximum and minimum values for location 1, t(5) = 5.83, p = 0.002; location 2, t(5) = 9.63, p = 0.000; location 3, t(5) = 20.71, p = 0.000; location 4, t(5) = 5.89, p = 0.002; location 5, t(5) = 12.252, p = 0.000; and location 6, t(5) = 6.65, p = 0.001. All locations except location 4 showed that the maximum facilitation occurred earlier in the reach, with the minimum performance being observed late in the reach. 

Reaching performance versus perceptual performance

The difference between reach accuracy and performance on the perceptual probe task was compared to determine whether there was a trade-off of perceptual performance for reach performance at different points during the reach. This was calculated by normalizing the values for all data for both reach accuracy and perceptual performance for all participants, such that each measure was calculated as a ratio of the maximum value for that measure. These measures were compared for each time bin to determine whether performance was weighted in favor of perceptual performance or reaching performance for each location within that time bin. 

In Figure 6, each data point represents the averaged data across all participants for each location. Each point therefore represents approximately 1,000 trials across participants, and approximately 150 data points per participant. The trade-off between reaching and perceptual performance in each time bin is shown in Figure 6. For each time bin, the locations where reaching performance was prioritized appear above the diagonal line; for those locations where perceptual performance was prioritized, the markers appear below the horizontal line. This figure provides an overview of how the priority changes from reaching performance to perceptual performance across time, and that this priority change is different across locations. 

Figure 6.

 

Perceptual performance versus reaching performance. Points above the diagonal line indicate that perceptual performance is better than reaching performance for those conditions.

Figure 7.

 

Heat map showing perceptual performance across all locations, per time bin. Reaching target location is indicated by a circle. Heat map shows the visual field from 4° from fixation to 14° from fixation, as these were the range of locations tested. The 82% baseline threshold level is indicated by the color green (the color bar is equivalent to the percentage indicated).

The trade-off between reaching and perceptual performance can be seen as the data points shift from reaching performance being better than perceptual performance, to perceptual performance being generally prioritized at a later stage during the reach. For time bins from −200 ms to +100 ms, the weighting is generally in favor of reaching performance, showing that during the preparation of the reach and the first stages of the reach, reaching performance was prioritized over perceptual performance. After this time, the trade-off between reaching and perception is more evenly distributed across locations, with the final time bin of +300 being in favor of perceptual performance for all locations apart from locations 3 and 4, which are above the reach target. This is intuitive as the reach would then be complete, so there is no need to favor reaching performance over perceptual performance across the visual field; however, the probe locations that are the same eccentricity as the reaching target may still show a residual reaching benefit when compared with probe locations that are further away. 

Heat map of spatiotemporal profile of performance

A heat map of spatiotemporal performance was created to visualize the spread of attentional facilitation relative to the reach. Values for the heat map were calculated by using perceptual performance across the different locations tested in the visual field during the reach task. Values for locations in time and space between these values were weighted and calculated using the distance between the desired point in the visual field and the experimental location. In addition to the required reliance on the nearest points and the grain size of the points to be estimated, this distance was used to estimate the values between the experimental locations. 

Perceptual performance without pointing

A secondary experiment was conducted (see Figure 3) to determine the magnitude of the spatiotemporal facilitation when there is no reach being prepared. This experiment aimed to compare the facilitation that accompanies a reach with the facilitation observed in a purely perceptual paradigm. 

We tested only four time points, which were calculated using the average reach latency and reach time for each participant. As the reach latencies and reach times differed across participants, the timing of the perceptual probe differed slightly for each. There were four possible probe onset times for each participant, which were calculated using the mean reach onset time for the reaching task. This meant that there were two possible probe onset times before the mean reach latency and two after. These points were then binned as being before or after the hypothetical reach onset time. Only four time bins were chosen, as we wanted to test the overall magnitude of facilitation when no reach was being prepared, without comparing the fine-grain changes measured in the reaching task. 

In Figure 8, the timing of the perceptual probe relative to the average reach onset was estimated by averaging the data across participants and using the average reach latency to calculate when the probe would have appeared relative to the reach, had there been a reach in that experiment. This means that, even though there was no reach in this condition, using the average reach time for each participant, it can be estimated when the probe would have occurred relative to this averaged reach latency. It can be seen that, at some locations, the magnitude of facilitation is fairly similar (locations 2 and 6), whereas locations 1 and 5 see better performance in the reaching task, and locations 3 and 4 see better perceptual performance compared to early time bins in the reaching condition and no difference at times after the onset of the reach. 

Figure 8.

 

Perceptual-only performance is shown for all participants in blue. Error bars are 95% confidence intervals calculated using Jeffreys interval as in Figure 4. Percentage correct data for the reach condition (per Figure 4) is replotted in gray for comparison.

Discussion

Attentional facilitation at different locations

We measured the spatiotemporal profile of attention during a rapid goal-directed movement. We found that there was a shift in space and time, and this profile was different (though qualitatively similar) for each location, and markedly different for the horizontal and vertical axes tested. For locations between the central fixation and touch location (that is, along the horizontal axis), attentional facilitation was greatest approximately 50 ms before reach onset. For locations above the touch location (in the vertical axis), attentional facilitation was greatest between 100–300 ms after reach onset. The six spatial locations and nine time points allowed a finer-grained analysis of this shift in attention than has previously been measured. We also measured pointing performance and found the touch points to be quite consistent across probe times. 

The buildup before reach onset observed for targets along the same direction between the touch location and central fixation is consistent with previous results. Rolfs et al. (2013) reported an increase in performance from 150 ms before reach onset, and Jonikaitis and Deubel (2011) saw an increase in perceptual performance from about 150 ms before the reach, and then a stabilization of performance during the reach itself. The broad pattern of attentional facilitation across the visual field supports the idea that the locus of attention can be spread to multiple locations (Baldauf & Deubel, 2008, 2010), although this study involved only one movement instead of a series of movements. The nature of the attentional facilitation that we observed is in line with the idea of an attentional landscape, which highlights behaviorally relevant locations differentially (Baldauf & Deubel, 2010). This idea that it is not only the reach location that shows attentional facilitation is not only supported by studies showing multiple loci of attention during sequences of reaches, but by studies showing that in purely perceptual tasks, attention can also be spread broadly beyond the immediate behaviorally relevant location (Koenig-Robert & VanRullen, 2011). 

Although this experiment examined the attentional shift relative to a reach while fixation was being maintained, the results can be compared to previous studies that have examined the attentional shift with both saccades and hand movements. Since saccades and reaching are linked, one can also compare the time course of these two movements to try to discern if the attentional shift is due to a single mechanism that drives both the presaccadic shift and the prereach shift, or if the attentional resources for these movements are different. This shift has been studied to a greater extent in relation to saccades rather than relative to a reach, and one can compare these studies to see if the time course is similar. 

The time course of the presaccadic attention shift has been studied in multiple experiments, with many different paradigms. However, a discrepancy arises in the research surrounding the timing of the presaccadic attentional shift, with some studies claiming the shift accompanying a saccade can happen as early as 50–100 ms after a cue (Deubel, 2008), and others finding that this shift happens on a slower time scale of around 150–200 ms after cue (i.e., Castet et al., 2006; Jonikaitis & Deubel, 2011; Montagnini & Castet, 2007). Nakayama and Mackeben (1989) found the peak attentional facilitation in a covert attention task without any saccades to be about 200 ms after the cue, which would seem to be slower than the attention shift accompanying a saccade. 

There are several key differences between our paradigm and those used by previous studies. Other studies have measured the attentional benefit relative to cue onset/offset (Castet et al., 2006; Deubel, 2008; Montagnini & Castet, 2007), and others have measured the attentional benefit relative to reach or saccade onset (Jonikaitis & Deubel, 2011; Rolfs & Carrasco, 2012). The paradigms differ in both stimuli and the timing of cue presentation, precue presentation, and the time points at which the attentional shift was actually recorded. It is, therefore, difficult to compare results directly. 

One pattern that is observed at the locations horizontally in line with the reach target was the marked drop-off in performance that occurred from 150–200 ms after reach onset. While there is no definitive answer as to why this occurred, a number of explanations may be plausible. The first is that the drop-off may be due to the occlusion of the perceptual probes by the hand as it neared the end of the reach. The locations of the probes were chosen to avoid this problem; however, there is a possibility that occlusion could have still occurred. Another explanation that could account for this pattern is inhibition of return, which could result in the inhibition of performance at those previously attended locations. This effect is seen to act on a similar time course to the inhibition shown here (Klein, 2000). However, a more likely explanation is that that the drop-off is due to a general decay of attention that occurs after a certain time point. Nakayama and Mackeben (1989) observed that attention seems to drop off from about 300 ms after the cue for “transient” forms of attention, which is consistent with the pattern observed here. Similarly, Koenig-Robert and VanRullen (2011) found that there were inhibitory areas across the visual field as well as facilitation. It seems logical in this case that the drop-off in attention would occur once the reach is nearing completion and the need for enhanced attentional resources is lower. It is, however, difficult to make a direct comparison to previous studies investigating the time course of attention relative to a reach, as they focus on the attentional buildup prior to the reach as opposed to the pattern of attention for the entire reach, until the hand is very close to the target. 

While performance at location 5 shows a change in percentage correct over the time course of the reach that is similar to that observed for other horizontal locations, the overall percentage correct appears shifted down. Performance at location 5 seems, by comparison, to be inhibited. This is a curious pattern and it could be due to a crowding effect from the neighboring touch target and location 6 (Whitney & Levi, 2011). This seems unlikely however, as only one probe location is visible at a time, so the crowding effect would have to come from the touch target alone. Another possible explanation is that participants overshot their reach, thus obscuring the location 5 probe. As mentioned earlier, however, the paradigm was designed specifically to avoid this. Further experiments investigating this pattern may provide a more parsimonious explanation of these aspects of the dataset. 

The trade-off between perceptual performance and reach performance

In the time bins before reach onset, performance seems weighted almost exclusively in favor of the perceptual task for each of the probe locations, and it is only toward the end of the reach that the reaching performance seems prioritized. This result suggests there is a trade-off between reaching accuracy and performance on the perceptual task, which benefits performance on the perceptual probes in the planning stages of the reach. When the reach is well underway, the priority shifts to reach accuracy being higher than the accuracy on the perceptual task; this is in line with the pattern shown in Figure 4, which shows a drop-off in perceptual performance in the later stages of the reach. When regarded in conjunction with the results in Figure 6, this drop-off could reflect the shift in priority from perceptual performance across the visual field to completing the reach accurately. 

Performance on the reaching task versus the perceptual-only task

Figure 8 shows the performance on the secondary task, which was a perceptual task with no reaching. This task sampled only four time points, which were calculated for each individual, based on the average reach latency for that participant. Thus, it is difficult to directly compare the time points with the data in Figure 4 as the data are averaged across participants, and the timing of the probe appearance is slightly different for each participant. While not directly comparable, the performance on this secondary task seems to show a fairly similar pattern of attentional facilitation as the reaching task, in both the temporal profile and the magnitude of facilitation seen at each individual location. This could indicate that the attentional mechanisms underlying perception and reaching may be similar. Alternatively, it could indicate that the paradigm was measuring attentional facilitation driven by perceptual mechanisms rather than the attentional facilitation driven by planning a hand movement. The nature of this paradigm, however, does not allow us to dissociate performance driven by perception and performance driven by action. Comparing performance on the reaching and perceptual task with the perceptual-only task is the only indicator of the potential dissociation of these different mechanisms. 

Mechanisms underlying the attention shift relative to a movement

The premotor theory of attention (Rizzolatti et al., 1994) postulates that different types of movements have their own neurologically localized mechanisms controlling the orienting of attention to an impending movement location. There is evidence to suggest a dissociation between attention for perception and attention for action. Blangero et al. (2010) reported a dissociation between covert attention and presaccadic attention in patients with right posterior parietal damage. The idea that the parietal lobe is important for integrating attentional mechanisms was also suggested by Khan et al. (2009), who suggested that while there may be different mechanisms for perception and action, there could be a functional coupling of saccade preparation and attention in the parietal cortex. This suggests that, while the mechanisms underlying attention for perception and attention for action might be different, the parietal cortex brings these mechanisms together. 

Gregoriou, Gotts, and Desimone (2012) have also demonstrated that different patterns of activation are found in frontal eye fields (FEF) when attending to a target or planning a saccade to that target. It is possible that the attentional effects observed in the FEF may also be due to the connections from the FEF to other areas that are implicated in attention, such at the lateral intraparietal area (LIP) and superior colliculus (SC) (Moore & Fallah, 2004). Indeed, it has been seen that microstimulation of the SC also increases perceptual performance at the relevant area of space, suggesting that the SC might also play a role in the orienting of attention (Muller et al., 2005). 

Evidence suggests that there is a neural circuit that mediates the coordination of eye and hand movements (Crawford et al., 2004), so it is possible that areas such as the FEF and SC, which are implicated in saccade-related attention, may also be used to guide the attentional shift that results from the planning and execution of a hand movement. 

Gherri and Eimer (2010) found that the preparation of a manual response could significantly modulate N1 components between 150 and 190 ms after stimulus onset. In an EEG study, Baldauf and Deubel (2009) found that perceptual probes presented at the location of a reaching goal elicited a response from the N1 component, indicating the presence of attention at the reach target. While there may be a mandatory link between the locus of a movement and attention, it is still not clear whether this is due to shared or independent neural resources. The secondary condition revealed that there was a difference in performance between the conditions, and this difference varied across times and locations. The pattern of performance, however, was not markedly different from the pattern observed for the reaching condition, which may indicate that the mechanisms between perceptual and motor performance are shared. 

While these studies provide evidence that mechanisms underlying saccades may be connected with the orienting of attention, it is still unclear how this may relate to the planning of other movements such as hand movements, and whether the attentional mechanisms for hand movements are the same as those which are involved in the presaccadic attention shift. 

Priority maps

What purpose might this attentional allocation play? It has been suggested that attentional shifts contribute to a priority map of relevant stimuli that is built up over time. This concept of a priority map proposes to integrate both top-down and bottom-up input to rank the behavioral relevance of locations across the visual field, and determines how visual attention should be directed, that is, to the item with the highest attentional priority (Bisley & Goldberg, 2010; Bisley, Ipata, Krishna, Gee, & Goldberg, 2009; Fecteau & Munoz, 2006). It may be the case that the attentional shift seen in this study has dual stages. First, the priority map selects the behaviorally relevant location (in this case the reaching target), and this behavioral selectivity subsequently allows attention to be directed to the relevant location. Indeed, there is evidence to suggest that salience map models can account for not only the guidance of attention and eye movements, but also that of manual pointing movements (Zehetleitner, Hegenloh, & Muller, 2011). 

This idea may also fit with previous experiments that have found that, when there are multiple impending reach locations, these locations can be selected in parallel rather than serially (Baldauf et al., 2006). It may be the case that, in this experiment, both the reach target and the perceptual probe locations could have been selected as being behaviorally relevant by the priority map, and attention could have been directed to these locations in parallel. An inherent problem with probing locations in this manner is that, although the attentional facilitation caused by the planning of a reach may be measured across the visual field, it is hard to dissociate this from any attentional resources that may be allocated to the probe location as a result of the behavioral relevance of these locations. 

Attentional oscillations and the attentional blink

It has been shown that when attention is “reset” using a visual or aural cue, there is a subsequent oscillation in attentional performance for the preceding second, with performance cycling at a rate of approximately 8 Hz (Landau & Fries, 2012). These patterns of oscillation performance are similar to the patterns shown in Figure 4, but the period of the oscillations are much slower than 125 ms, as observed in Landau and Fries (2012). We have sampled up to 400 ms after the cue, so we would expect about three cycles of oscillations during this period. However, we only see a period of increase followed by a decrease. Thus, it is unlikely that our sensitivity modulation is due to attentional oscillations as described by Landau and Fries (2012). 

In the attentional blink, attentional resources for subsequently presented stimuli are temporarily unavailable and, therefore, a downturn in performance is observed after the presentation of an attended stimulus (for review see Dux & Marois, 2009). Similarly, it could be the case that the downturn observed for the horizontal locations is due to attentional resources being temporarily exhausted. This explanation does not, however, hold for the vertical locations, which show little downturn in attention over time. 

Conclusion

These results have demonstrated that there is an attentional shift which builds up before the onset of a reach, and that the temporal profile of this shift differs across the visual field. The results show a buildup of attention around 150 ms before the onset of a reach, which is in line with previous research. This study, however, also measures the profile of attention throughout the reach, and shows that once the movement is well under way, attention narrows around the touch target while the surrounding locations are suppressed. The profile of attention also seems to be different for locations along the horizontal and vertical axes, which is consistent with the idea that attention is restricted to the reach target later in the reach, when reach performance is prioritized. 

Overall, the results provide support for the notion that planning and executing a reach causes a shift in attention across the visual field, and specifically around the reaching target. These results also provide a solid basis for future comparisons of the mechanisms underlying both reaches and saccades. 

Acknowledgments

This project was supported by an Australian Postgraduate Award scholarship to E. E. M. S. We would like to thank Preeti Verghese for her generous advice and feedback during the project and Concetta Morrone, Laura Walker, and Dan Navarro for helpful discussions. 

Commercial relationships: none. 

Corresponding author: Anna Ma-Wyatt. 

Email: [email protected]. 

Address: School of Psychology, University of Adelaide, Australia.