Space, color, and direction of movement: How do they affect attention?

Abstract

Abstract

Abstract:

Abstract  Paying attention improves performance, but is this improvement regardless of what we attend to? We explored the differences in performance between attending to a location and attending to a feature when perceiving global motion. Attention was first cued to one of four locations that had coherently moving dots, while the remaining three had randomly moving distracter dots. Participants then viewed a colored display, wherein the color of the coherently moving dots was cued instead of location. In the third task, participants identified the location that had a particular cued direction of motion. Most observers reported reductions of motion threshold in all three tasks compared to when no cue was provided. However, the attentional bias generated by location cues was significantly larger than the bias resulting from feature cues of direction or color. This effect is consistent with the idea that attention is largely controlled by a fronto-parietal network where spatial relations are preferentially processed. On the other hand, color could not be used as a cue to focus attention and integrate motion. This finding suggests that color relies heavily on processing by ventral temporal cortical areas, which may have little control over the global motion areas in the dorsal part of the brain.

Introduction

We rely on our ability to pay attention in order to perform several day-to-day tasks ranging from finding a pair of socks to navigating traffic. This is because attention allows us to selectively process only relevant information from what could often be a cluttered environment. The selection of relevant information can be based on the spatial locations of objects (location-based attention) (Pestilli & Carrasco, 2005; Posner, Snyder, & Davidson, 1980; Theeuwes & Van der Burg, 2007) as well as on other attributes such as color or direction of motion (feature-based attention) (Saenz, Buracas, & Boynton, 2002; White & Carrasco, 2011). It has been shown that both these types of attentive selection modulate neural activity in cortical areas belonging to two interconnected but distinct pathways (Kravitz, Saleem, Baker, & Mishkin, 2011; Ungerleider & Mishkin, 1982). Location-based attention is mediated primarily by the dorsal pathway, which projects from the primary visual cortex (V1) and extends into the parietal lobe (Bisley, 2011; Bisley & Goldberg, 2003; Saalmann, Pigarev, & Vidyasagar, 2007). Feature-based attention is subserved mostly by the ventral pathway, which connects V1 to the temporal lobe (Chelazzi, Miller, Duncan, & Desimone, 1993; Motter, 1994). However, the pattern of interaction between these two pathways, which in turn determines the mechanism by which attention operates, is still a matter of debate. One idea that has been proposed is that the dorsal pathway uses its spatial saliency map to spotlight a location and, via top-down feedback channels, gates what is subsequently processed by the ventral stream (Bullier, 2001; Vidyasagar, 1999). Such a mechanism can potentially also solve the binding problem, since the features that are associated with a single object would be bound together by temporal coincidence within the spotlight, despite being processed in different cortical areas. This raises the possibility that location-based attention may be more effective in aiding performance for a given task than feature-based attention. 

We tested this possibility using a set of three psychophysical motion-perception tasks. A proportion of dots in one of four squares on the screen moved coherently in one of four directions (upward/downward/leftward/rightward). In each task, we compared the minimum proportion of coherently moving dots required to bring performance to the 62.5% threshold in the presence or absence of an endogenous cue (symbolic arrows or names of colors). The first task (Figure 1a and see Supplementary Video 1¹ for corresponding animation) tested location-based attention; the second, color-based attention (Figure 1b and Supplementary Video 2); and the third task probed our ability to use a direction of motion cue in order to enhance performance (Figure 1c and Supplementary Video 3). Each task was also performed without any cue to provide a baseline for comparison. 

Figure 1

 

Task design. Each of the three tasks shown here began with the relevant task instruction (top row) followed by the pre-cue and then the motion stimulus. In the example here, the target patch “T” contained the coherently moving dots. The other patches had random motion “N”. The cues (second row) refer to either the location of the target patch (top-left in a), the color of the coherently moving dots (red bold typeface “T” in b), or the direction of motion of the target dots (upwards in c). Each task was also performed without any cue to establish baseline performance.

Materials and methods

Participants

Five people (aged between 18 and 35 years, two females) participated in this experiment. The participants (including one of the authors, AV) were either students or staff members at the Department of Optometry and Vision Sciences, University of Melbourne. They had normal or corrected-to-normal visual acuities, and normal color vision. All participants except the author were naive to the purposes of the experiment. The study conformed to the Code of Ethics (Declaration of Helsinki) and was approved by the institutional ethics committee. 

Apparatus

Stimuli were generated using a ViSaGe stimulus generator (Cambridge Research Systems, Rochester, UK) and displayed on an EIZO Flexscan F980 monitor (21 inches, resolution of 800 × 600 pixels, 100 Hz refresh rate). Participants viewed the monitor at a distance of 57 cm. Gamma correction of the monitor was performed prior to each session with an OptiCAL photometer (Cambridge Research Systems). The CIE1931 coordinates of the monitor were x = 0.610, y = 0.340 for the red phosphor, x = 0.280, y = 0.594 for the green phosphor, and x = 0.142 and y = 0.070 for the blue phosphor. 

The stimulus comprised four patches with moving dots called random dot kinematograms (Newsome & Pare, 1988). Within one of these patches (target), a proportion of dots (signal) moved coherently in one of four directions (upward, downward, rightward, leftward). In the remaining three nontarget fields, noise dots moved incoherently in random directions. The proportion of signal dots was varied dynamically between 50% and 1% coherence using the ZEST (Zippy Estimation by Sequential Testing) routine (King-Smith, Grigsby, Vingrys, Benes, & Supowit, 1994) which was designed to converge on the 62.5% correct threshold level. As the ZEST routine employs increments on a log scale, the coherence levels displayed were converted to a linear scale for plotting purposes. 

The dot fields were centered 10° eccentrically, along an oblique axis from the central fixation spot (top-left, bottom-left, top-right, and bottom-right fields). In our display, each field (6° × 6° at 57 cm) consisted of an array of 88 randomly positioned Gaussian blobs with a diameter of 0.53° and deviation of 0.15°. These blobs minimized the effect of chromatic aberration by removing high spatial frequency information present in hard-edged uniform discs. The luminance of the dots was 15 cd/m², and the dots were presented with a mask of 10% luminance noise against a background of 7 cd/m². The dot density of each field was 1.85 dots per square degree. For the second color-based attention task, heterochromatic flicker photometry was first performed for each observer to minimize luminance differences between the red and green dots. However, the contrast level of the dots to the background was the same as in the first task (36%). The color of the signal dots in the target patch was either red or green, assigned with equal probability. Importantly, there were equal numbers of red and green dots in every patch regardless of whether they were coherent or incoherent so as to avoid any pop-out cues (Li & Kingdom, 2001). This meant that if the proportion of coherently moving dots was, for instance, 20%, and the cued color was red, amongst the 44 red dots in the target patch, 18 dots moved coherently while the remaining 26 red dots as well as the 44 green dots moved randomly. In the nontarget patches, all 44 red dots and the 44 green dots moved in random directions. 

The moving signal dots were randomly chosen in each apparent-motion frame and displaced 0.3° in the signal direction (upward, downward, rightward, leftward), thus resulting in a dot speed of 6°/s. This random selection of signal dots in every alternate frame minimized possible artifacts caused by tracking of individual dots. Hence, global motion processing was essential for determining the coherent motion direction. The noise dots had a fixed displacement (0.3°) but moved in random directions from frame to frame. If a dot fell outside the target square, it was wrapped around to the opposite side. The entire stimulus consisted of 16 apparent-motion frames displayed for a total of 800 ms. 

General procedure

Observers were instructed to maintain fixation on a 0.2° central spot presented on the screen. Eye movements were monitored using an infra-red eye tracker (Dr. Bouis Infrared Oculometer, spatial resolution 0.1° and 1000 Hz sampling rate). The trial was initiated by pressing a button. After a delay of 800 ms, the target field and the nontarget fields were presented in the four quadrants. In the first and the second tasks, there were four stimulus conditions (target moving in one of the four directions). Participants were asked to indicate the direction of motion (upward, downward, leftward, rightward) by pressing a button. In the third task there was another set of four stimulus conditions (target in one of four quadrants in the visual field). Here, participants had to detect which of four locations contained coherent motion (top-left, bottom-left, top-right, bottom-right). In the cued conditions, depending on the task either a location cue (arrow indicating target field position) (Figure 1a) or a feature cue (colored fixation spot and the corresponding color name in the color-based task and arrows indicating direction of motion of the signal dots in the direction-based task) was presented for 800 ms (see Figure 1b and c). Following the cue, the motion stimulus (target and nontarget fields) appeared in the four quadrants. There were 512 trials in each session with 128 randomized presentations of each stimulus condition. These 128 trials were split into four blocks of 32 trials. Auditory feedback (high-pitched beep for correct and low-pitched one for incorrect responses) was provided for all trials. 

Data analysis

Psychometric thresholds for 62.5% correct performance 4-alternative-forced-choice, was obtained from ZEST for each condition at each of the four locations and for each direction. In order to compare performance in a particular task with and without a cue, paired t tests were conducted. The magnitude of the attention bias obtained with each cue was computed as the ratio of threshold obtained without a cue to that obtained in the presence of a cue. A repeated-measures ANOVA was then carried out to determine whether there were differences in attentional weighting at any particular location or for any direction. 

Results

Repeated-measures ANOVA was done to test for differences in attention biases produced by the individual cue (location, color, and direction) at each of the target locations and directions. No significant differences were found using either of the cues, and so the data could be collapsed across both locations and directions. When location was cued, as illustrated in Figure 2a, the minimum coherence needed for threshold global motion perception was lower (mean with standard deviation = 13 ± 2%) than without a cue (17 ± 3%), (Paired t test, mean difference = 4.2 [95% CI, 2.3–6.2], p = 0.004). This attention-related advantage was, however, absent when color was cued (mean difference = 1.4 [−1.3–4], p = 0.2) (Figure 2b,); and when direction was cued, this difference was again relatively small (mean difference = 1.4 [0.2–2.5], p = 0.02) (Figure 2c). To further examine the relative size of this attention effect in the three cueing conditions, a ratio of the threshold (THR) obtained without a cue to that obtained in the presence of a cue was computed. Nonparametric ANOVA followed by posthoc tests on these ratios confirmed that location cues had a larger effect on improving performance than either color or direction of motion cues (Figure 2d, p = 0.009). 

Figure 2

 

Effect of location, color and motion direction cues on performance. Paired two-tailed t tests were conducted. (a) In the first task, performance was better with the location cue (mean 13 ± 2%) than without (mean 17 ± 3%). (b) In the second task, there were no significant differences in performance with (mean 20 ± 4%) and without the color cue (mean 21 ± 6%). (c) In the third task, performance was better with the direction of motion cue (mean 18 ± 5%) than without (mean 20 ± 4%). (d) Overall attentional bias was calculated as a ratio of thresholds obtained with and without each cue. Kruskal-Wallis ANOVA and Dunn's post-hoc multiple comparison test indicated that there were significant differences in the overall size of the attention effect (p = 0.009) and that these differences were significant between location versus direction cueing and location versus color cueing. The error bars indicate standard deviations for each observer over 16 blocks of trials.

It was surprising to find that despite the long presentation time (800 ms) in our study, color cues did not have any significant effect on task performance. On the other hand, previous studies incorporating similar motion perception tasks have shown that color cues did aid in motion segregation (Li & Kingdom, 2001; Snowden & Edmunds, 1999). In these studies however, the effect of the cue was significant only when the coherently moving dots (signal) were of a different color from the randomly moving ones. This suggests that when the coherently moving dots are of a different color, they form a minority dot set that stand out (pop-out) from the remaining randomly moving dots. In our color-based task, however, there were always 44 red and 44 green dots in each patch, regardless of motion coherence, thereby eliminating such pop out effects. 

In addition to the possibility of a pop-out cue for color in previous studies, there are two further explanations that could account for the discrepancy between the present findings and those in the literature. Firstly, binding of color and motion is poor at the spatial scale used in our experiment. Secondly, luminance cues override any potential benefit from color cues. In order to test these possibilities, we designed two additional control experiments. 

Control experiment 1

The control experiments were similar to the main experiments but with some modifications. The purpose of the first control experiment was to determine thresholds for color identification when perceiving motion and also to determine whether these thresholds varied with the type of cue provided. By using a color identification task, it was possible to explore the presence of binding inaccuracies between color and motion. In this experiment, the same target and nontarget patches were presented, but this time a pop-out cue was provided in the target patch (i.e., the coherently moving signal was differently colored from the noise dots). The proportion of red and green dots in the nontarget patches was identical to that in the target. Hence, if the target had 20% red dots, the nontarget patches also had 20% red dots except that these moved randomly rather than coherently. The same five participants from the main experiment participated in this control experiment. Initial runs with coherences capped at 50% as in the main task revealed that the tasks were impossible for the observer to perform and the thresholds estimated were all above 50%. Coherences were then varied between 100% and 1% to further aid performance. This was a two-alternative-forced-choice experiment, and hence thresholds for 80% accuracy were recorded. 

Consistent with pilot runs, threshold performances for all three tasks (no cue, location cue, and direction of motion cue) were achieved only at coherences above 50% (Figure 3a) (mean 53 ± 0.5%). Repeated-measures ANOVA for thresholds obtained in the three conditions—no cue, location cue, and direction cue—did not vary significantly (p = 0.27). Since performance did not vary with the type of cue, this experiment was repeated for patches at an eccentricity of 4° from fixation and with just the location cue (Figure 3a, last bar). Performance again did not vary between the two eccentricities (Paired t test, mean difference = −0.03, p = 0.9). 

Figure 3

 

Results of control experiments 1 and 2. The tasks were similar to those in the main experiment. (a) Comparison of threshold motion coherences obtained when observers were asked to report whether the color of the coherently moving dots was red or green. Repeated-measures ANOVA revealed no significant differences. This was also true for patches that were closer to fixation (from 10° eccentricity to 4° eccentricity in the last bar). (b) In the second control experiment, an increased luminance noise mask (30%) compared to the 10% luminance mask used in the main experiment did not significantly change the effect of the color cue.

Control experiment 2

The second control experiment was a repetition of the second color-based attention task in the main experiment, except that the 10% luminance noise which was used to mask the influence of chromatic aberrations was now raised to 30%. This adjustment was done in an attempt to abolish any residual luminance cues and hence force the attention system to rely only on color cues. The effect of the cue was quantified as the ratio of performance without the color cue to that with the color cue (Figure 3b). Three of the five observers participated in this experiment. Mann-Whitney tests revealed no significant differences between the two (low and high luminance) noise groups (p = 0.4). Together, the results from the two control experiments suggested that only when the coherently moving dots formed a cluster (i.e., at high coherences) could color be accurately identified. This inability to identify color was present even when the patches were centered closer to fixation than in the original task (Figure 3a, last bar), thereby arguing against any effects related to the spatial scale of the dots used. Finally, the effect of the color cue was negligible even with the additional luminance noise in the second control experiment (p > 0.05) (Figure 3b). 

Discussion

The results from our main experiment are consistent with the hypothesis that location cues are indeed more facilitative than feature-based cues of color or direction of motion. This idea is in line with the guided search model (Wolfe, 1993) as well as a neural model for attention (Vidyasagar, 1999) wherein the parietal areas that process spatial information gate what object features are processed by the temporal areas at any given time. When location is cued, only the dots within the spotlight are preferentially processed, and hence performance is better than in the no-cue condition. In contrast, when a feature such as direction of motion is cued, the spotlight has to be sequentially focused on each location until the target is detected. The idea of multiple (namely, dual) spotlights as demonstrated by McMains and Somers (2004) is highly unlikely in our task design because the spatial arrangement of our display would require four spotlights (not just two), a suggestion that has not been demonstrated in the literature. It is noted that, while the unitary spotlight process is still quite effective in processing motion direction (the dorsal pathway is specialized to process such motion), the size of the bias evoked here is relatively small when compared to location cues. 

One could criticize, however, that our task design was biased because of differences in the number of dots that were attended in each task. When locations were cued, the dots that needed to be attended were restricted to only those within a single patch i.e., 25% the number of dots in the no-cue condition. On the other hand, attending to features like color and direction reduced this number only by half (for color) or not at all (for direction). One could attempt to resolve this issue by changing the configuration of the stimuli in the feature-based tasks (i.e., use two instead of four patches in the color task and one out of four in the direction task, or reduce the dot density in each patch), but such adjustments would again introduce other, much worse, confounds. If the difference in the number of dots attended had indeed affected our results, then performance would have improved with color cues (which halved the number of dots attended), and this improvement would also have been more than with direction cues (which did not reduce the number of dots attended), but this was not the case. Our results instead demonstrate that location cues improved performance significantly whereas feature cues failed to have a consistent effect, with, if anything, direction cues leading to slightly, but significantly, better performance than color cues. 

Results of previous psychophysical studies that used visual search tasks are also consistent with our findings (Meeter & Theeuwes, 2006; Theeuwes & Van der Burg, 2007; Vierck & Miller, 2008). However, these studies were not designed to compare location versus feature-based effects and instead strongly argued that feature-based attention has no effect on the top-down selection process. Our study compared and contrasted both feature- and location-based attention using the same stimulus paradigm, thereby avoiding confounding effects of having to compare results between studies that use different methodologies. 

In this study, we estimated thresholds using the ZEST procedure rather than measuring psychometric functions using a method of constant stimuli. The latter would have consumed an inordinate amount of time (Konsevitch and Tyler, 1999) with possible fatigue and learning effects confounding the results. It is, however, possible that a difference in the slope of the psychometric function between tasks could have affected the degree of attentional shift, particularly if the slopes were shallower for the feature-based attention tasks. Nevertheless, there is no evidence in the literature of such shallower psychometric functions for feature-based tasks and even if this were so, that color or direction cues would fail to provide a discernible attentional advantage owing to the nature of the slope. 

An important finding that emerged from our study is that the binding of color and motion is relatively poor unless the colored dots appear as one large surface as is the case when coherences are greater than 50%. This outcome is consistent with previous motion-perception studies that used color cues and where the stimuli were overlapping motion arrays with two layers of 100% coherent dots (Andersen, Fuchs, & Müller, 2011; Katzner, Busse, & Treue, 2006; Saenz, Buracas, & Boynton, 2003). In these studies, color was found to play a beneficial role by aiding behavioral performance. On the other hand, in studies that employed random-dot kinematograms similar to those in our main experiment (Barbur, 2004; Li & Kingdom, 2001), color could not be used as a cue to segregate coherent and incoherent motion. 

The question then arises as to why we do not have accurate color-motion binding for small segregated objects. One possibility is that having this ability would not serve any ecological purpose. For instance, if a swarm of bees were attacking us, discerning whether they are black bumble bees or yellow honeybees would not aid us in our flight. On the other hand, perhaps being able to localize where the bees are and the general direction in which they are flying might enable us to avoid a few stings. Focal spatial attention would not only be a very effective way of processing just the item or items of interest in a cluttered field, but would also aid in solving the binding problem. 

Conclusions

Our psychophysical cueing results support the view that attentional selection is primarily a location-based system, at least in tasks defined by motion. Although features such as direction of motion weakly facilitate motion processing, attending to a location in the visual field results in a marked improvement in motion processing and hence appears to dominate attention selection. 

Supplementary Materials

mp4 S1 

mp4 S2 

mp4 S3 

Acknowledgments

This work was supported by an Australian National Health & Medical Research Council grant to T. R. V., an International Postgraduate Research Scholarship from the Commonwealth Government of Australia, and a scholarship from the Faculty of Science, University of Melbourne to A. V. 

Commercial relationships: none. 

Corresponding author: Trichur Raman Vidyasagar. 

Email: [e-mail]. 

Address: Department of Optometry & Vision Sciences, The University of Melbourne, Parkville, Australia. 

Footnotes

¹   The supplementary videos are only for illustrative purposes and hence do not represent the true spatio-temporal profile of the dots or the stimulus as a whole.