Exogenous visual orienting by reward

Abstract

Abstract

Abstract:

Abstract  Classic spatial cueing experiments have demonstrated that salient cues have the ability to summon attention as evidenced by performance benefits when the cue validly indicates the target location and costs when the cue is invalid. Here we show that nonsalient cues that are associated with reward also have the ability to capture attention. We demonstrate performance costs and benefits in attentional orienting towards a nonsalient cue that acquired value through reward learning. The present study provides direct evidence that stimuli associated with reward have the ability to exogenously capture spatial attention independent of task-set, goals and salience.

Introduction

The expectation of a reward is known to be the driving force behind adaptive behavior and learning. A lot is known about the motivational effect of reward (e.g., Engelmann & Pessoa, 2007), but only recently the effects of reward on attention and perception have been explored. It has been suggested that reward may lead to a sustained change in visual perception even in the absence of an explicit reward (Anderson, 2013; Awh, Belopolsky, & Theeuwes, 2012; Chelazzi, Perlato, Santandrea, & Della Libera, 2013; Seitz, Kim, & Watanabe, 2009). A rapidly growing body of evidence suggests that reward-induced changes in perception and attention might be automatic in nature (Anderson, Laurent, & Yantis, 2011b; Hickey, Chelazzi, & Theeuwes, 2010) and may not fall within any of the two traditional domains of visual perception as they seem to be neither entirely bottom-up nor top-down driven (Awh et al., 2012). 

Among the first who studied sustained effects of learned stimulus-reward contingencies in visual perception were Della Libera and Chelazzi (2009). They trained stimulus-reward contingencies between shapes and payment schedules and asked participants to perform the same task five days later but without giving them any monetary reward. The results showed that the reward training had established long-term attentional biases in relation to the specifically rewarded shapes. Not only were responses towards the high rewarded shapes speeded, but also detection was improved when participants engaged in a masked search task. 

In a subsequent series of experiments by Anderson, Laurent, and Yantis (2011a, 2011b), participants were also exposed to a training session. Their task was to search for a red or green target circle among differently colored nontarget circles. For different participants, either the red or green color was associated with a high or low monetary reward probabilistically delivered following each trial. In a subsequent test session in which no reward was given, participants were exposed to a variant of the additional singleton task of Theeuwes (1991, 1992, 2010) in which they searched for a unique shape in an array of randomly colored elements. On half of the trials, one of the nontarget circles was rendered red or green constituting the to-be-ignored distractor. The results showed that the time to find the target (the unique shape) was significantly longer when a distractor in a color associated with high reward was presented relative to a distractor in a color associated with low reward. The authors explained the increase in search time in terms of increased attentional capture by the color that was associated with high reward relative to the capture by the distractor color associated with low reward. It was concluded that stimuli with increased learned value are prioritized even when they are not relevant for the current task. A similar conclusion was drawn by Hickey et al. (2010) who argued that rewarding a particular color on one trial results in an automatic bias on the next trial. 

Even though these studies clearly demonstrate that search for a target is slowed by the presence of a distractor associated with a high reward, they do not unequivocally show that this slowing in search for the target is the result of spatial attention being captured by the high rewarded distractor Indeed, it is possible that the presence of a stimulus associated with a high reward biases search without capturing attention. For example, Folk and Remington (1998) argued that the increase in search times due to the presence of a singleton distractor in the well-known additional singleton paradigm of Theeuwes (1991, 1992) has nothing to do with attentional capture but instead is due to what Kahneman, Treisman and Burkell (1983) called filtering costs. According to this notion, filtering costs can occur when task-irrelevant but salient stimuli compete for attention and have to be filtered out (see also Folk, Remington, & Wu, 2009). As this filtering operation is effortful and time consuming, search efficiency is affected (Folk & Remington, 1998). It is important to note that filtering does not involve a shift of spatial attention to the location of the distractor as this effect is believed to be completely nonspatial. In line with the notion of filtering costs, it is possible that the presence of a stimulus associated with a high reward affects search without capturing spatial attention. Because all previous studies suggested attentional capture by the high rewarded stimulus without providing direct evidence for it, the present study was designed to determine the allocation of spatial attention in the presence of a rewarded stimulus. 

In the classic exogenous spatial cueing paradigm of Posner (1980), a cue (typically an abrupt onset) precedes the presentation of the target. This cue is either presented at the location of the upcoming target or at a location where the target will not appear. The typical finding is that when the onset cue is valid (i.e., the target appears at the cued location), response times (RT) are fast and accuracy is high compared to a condition in which the cue is invalid (i.e., the target appears at the uncued location), indicating that the cue captured spatial attention, i.e., attention was drawn to its location. Typically, a cue results in performance costs and benefits: Relative to a neutral condition, participants are faster and more accurate for valid trials and slower and more inaccurate for invalid trials. Finding costs and benefits in a spatial cueing paradigm in which the cue has no predictive value is taken as direct evidence that the cue grabbed attention automatically, directing spatial attention to the location of the cue (Folk & Remington, 1998; Yantis & Jonides, 1990). 

The present study was designed to determine whether learned reward value has a direct effect on attentional orienting. To that end during training, participants searched for a target letter that was consistently placed inside a colored circle. For one color, responding to the target was followed by reward; for another color, responding to the target was not followed by reward. After the training phase, participants engaged in a test phase, which was a variant of the exogenous Posner spatial cueing task. As cues we employed the colored circles that were used during training. Because of the training, one color was associated with the target and reward, one was associated with both the target and no reward, and other colors were neither associated with the target nor with reward. We examined RT costs and benefits in relation to these cues. If a cue with trained reward value grabs spatial attention, we expect benefits when the target appears within this circle. Accordingly, we expect costs when the target appears within another circle while the cue with trained reward value is present. If, however, the presence of the cue trained with reward only needs to be filtered out (cf. filtering costs) then one does not expect RT costs and benefits as spatial attention is equally likely to be allocated to either one of the two locations. If anything, filtering costs would result in an overall increase in RT, yet no spatial cueing effects in terms of RT costs and benefits. A direct effect of a stimulus associated with reward on attentional orienting also predicts that a stronger learning effect is related to a stronger attentional capture even when the stimulus is no longer indicative for reward. 

Material and methods

Participants

Seventeen healthy individuals (nine female, mean age ±24) with normal or corrected-to-normal eye-sight gave written informed consent for participation. Participants were provided monetary compensation of between 8€ and 11.50€ (mean = 9.75€) based on their performance. Data from one participant were removed from analysis due to performance at near chance-level. 

Apparatus and stimuli

Stimuli were created in OpenSesame (Mathot, Schreij, & Theeuwes, 2012) and presented on a Samsung SyncMaster 2233RZ monitor (1680 × 1050 resolution, 120 Hz). The display consisted of two outline circles (3.4° diameter), each having a unique color (red, green, yellow, or blue). The circles were presented simultaneously along the horizontal meridian to the left and right (9° center-to-center) of a central black fixation dot. The experiment was divided into a training and test phase. During training, two of four different letters (“S”, “P”, “H”, “E”; 2.3° × 1.1°, made up of equally-sized line segments) were placed within the colored circles. During the test phase, figure-eight placeholders preceded the display that had similar circles and letters as during training. All stimuli were presented on a uniform gray background. 

Procedure and design

Participants were seated in a sound-attenuated, dimly lit room at a distance of 70 cm to the screen. Each trial started with a randomly determined fixation period of 400–600 ms. During training two differently colored circles, each containing a different letter, were presented for 800 ms. Participants had to indicate which target letter (“S” or “P”) was present in the display by pressing the appropriate key with their index fingers. The other circle always contained a randomly chosen distractor letter (“H” or “E”). After the response, the fixation dot disappeared and feedback was immediately presented in the center. Following a correct response a “+” sign and a number appeared for 1000 ms, indicating how many points participants had earned during that trial. For incorrect or too slow responses “−10” appeared on the screen, indicating a loss of points. At the start of the experiment, participants were informed that the points they would earn for correct responses corresponded with up to 12€ paid out at the end of the experiment. No information was given about how many points corresponded to how much money. 

Two design features were important for the training phase: First, target letters consistently occurred in only two of the four colored circles (henceforth called “trained circles”). All colors were equally often presented and counterbalanced over participants. Second, each of the two trained circles and accordingly their specific color was bound to a fixed reward schedule that was beyond control of the participant. One color was rewarded with 10 points in 80% and with 0 points in the remaining 20% of the trials (“rewarded circle”), while the other was rewarded with 0 points on all trials (“non-rewarded circle”). The two remaining colors were non-rewarded as well but crucially never contained a target letter during the training phase (“non-trained circles”). All participants completed one practice block and eight experimental blocks of 80 trials each, yielding a total of 720 trials. 

During the test phase which took place on the same day as the training, the fixation period was followed by the presentation of two circles each containing a figure-eight premask. After 200 ms two line segments of the figure-eight premasks were extinguished, thereby revealing the letters without any abrupt onset (Figure 1). The circles and letters remained on the screen for another 800 ms or until response was given. Participants performed the same letter discrimination task as during the training phase and were explicitly informed that during this phase they would not receive any points or money. Moreover, participants were told that the target letter would equally often appear in either circle during the test phase, emphasizing that the color of the circles was task-irrelevant. All participants completed one block of practice and six experimental blocks of 72 trials each, yielding a total of 504 trials. 

Figure 1

 

The trial sequence in the test phase. There were in total four different cue display types: trained and rewarded, trained but not rewarded, and nontrained. In one condition, both trained circles were presented, one of which was rewarded and the other was not rewarded during training. In this example, the red circle corresponds to the trained and rewarded circle, the green to the trained but not rewarded circle, and the blue and yellow to the nontrained circles.

In the test phase the stimulus-reward contingencies were explicitly tested. The target letter was equally likely to occur in any of the colored circles. All four colored circles were equally often presented (never the same color within a display). The location of the circles and accordingly of the letter, as well as the letters themselves, were randomly chosen on a trial-by-trial basis throughout the whole experiment. 

Data analysis

RTs below 100 ms were excluded from further analyses leaving 99.9% of the data. Only correct responses were used for RT analyses. 

Results

Training phase

To assess whether rewarding a color affected RT and how its effect modulated performance over time, we conducted a within-subjects analysis of variance (ANOVA) on mean RT with reward and block as factors. There was no main effect of block, F(7, 105) = 1.365, p = 0.256, yet reward was reliable, F(1, 15) = 10.551, p = 0.005, η_p² = 0.413. Participants were faster when the target appeared in the rewarded (M = 479 ms ± SD = 36.46) than when the target was in the non-rewarded circle (487 ms ± 31.83), indicating that reward speeded performance. Moreover, there was an interaction of reward and block, F(7, 105) = 2.110, p = 0.049, η_p² = 0.123, indicating that the effect of reward changed over blocks. Figure 2 shows that during training the effect of reward was particularly strong in the later blocks, suggesting that it took some training to learn the stimulus-reward contingency. To investigate the learning more thoroughly we split the data in half and compared the reward effect in both halves. An ANOVA on the mean RT with reward and block (block 1–4 vs. block 5–8) revealed a significant main effect of reward, F(1, 15) = 10.551, p = 0.005, η_p² = 0.413, but no effect of block, F(1, 15) = 2.890, p = 0.110. Importantly, there was an interaction of reward and block, F(1, 15) = 17.133, p = 0.001, η_p² = 0.533, indicating that participants were indeed faster when the target appeared in the rewarded circle compared to when it appeared in the non-rewarded circle but only during the last half of the training phase, t(15) = 4.427, p < 0.001, 95% CI (18.77, 6.57). There were no significant changes in errors over time. 

Figure 2

 

Mean RT for rewarded and nonrewarded trials of the training phase over time. Bars here and in all other figures represent within-subject 95% confidence intervals (Morey, 2008).

Test phase

We wanted to determine how the trained stimulus-reward contingencies affected performance over time when they were no longer predictive for reward. For that purpose, we first ran an ANOVA on reward and block on mean RT for trials in which the target appeared in the colored circle previously associated with reward (trained reward condition) versus when it appeared in a circle that had never contained the target during training (non-trained condition). There was a highly reliable main effect of reward, F(1, 15) = 16.863, p = 0.001, η_p² = 0.529, but no main effect of block, F(5, 75) = 0.820, p = 0.482, or interaction, F(5, 75 = 0.790, p = 0.560. The error rate mirrored RT as there was a main effect of reward, F(1, 15) = 20.705, p < 0.001, η_p² = 0.580), but no main effect of block and interaction of reward and block. These results indicate that participants were not only faster (499 ms ± 35.42 vs. 524 ms ± 30.45) but also made less errors (7.1% ± 4.3 vs. 13.1% ± 8.4%; see Table 1 for all error rates) when the target appeared inside the trained reward circle (see Figure 3; red lines). A similar analysis was performed on trials in which the target appeared in the circle previously trained but not rewarded (trained non-reward condition) versus when it appeared in the circle that had never contained the target during training (non-trained condition). There was no effect of training, F(1, 15) = 2.356, p = 0.146); block, F(5, 75) = 0.606, p = 0.696, or an interaction, F(5, 75) = 0.570, p = 0.723), these results indicate no difference for the trained non-reward circle with respect to a non-trained circle (see Figure 3; green lines). Similar results were obtained for error rates. 

Figure 3

 

Time course of the reward effect on mean RT during the test phase. For each subject the mean RT for each block and condition was calculated and averaged across all subjects.

Table 1

 

Mean error rates (SD) in the test phase of the experiment in percent. Note that for the both nontrained condition the target location corresponds solely to the colors of both nontrained circles.

Target location	Condition
	Trained reward	Trained non-reward	Both trained	Both non-trained
Target in trained circle	7.1 (4.3)	8.4 (7.1)	6.4 (5.0)	6.1 (5.1)
Target in nontrained circle	13.1 (8.4)	8.0 (6.1)	15.1 (8.2)	7.6 (7.6)

In a subsequent analysis, we directly assessed the effect of the trained stimulus-reward contingency relative to mere training. By analyzing trials in which both trained circles were present (both trained circles present condition), we directly assessed the effect of the trained stimulus-reward contingency relative to mere training. An ANOVA showed a significant main effect of reward, F(1, 15) = 7.781, p = 0.014, η_p² = 0.342, but no effect of block, F(5, 75) = 1.368, p = 0.246, and no interaction, F(5, 75) = 0.966, p = 0.444. This outcome indicates that when both trained circles were present, participants were biased towards selecting the trained circle associated with reward (499 ms ± 36.06) relative to the trained circle associated with no reward (520 ms ± 22.99). Figure 4 also suggests that this bias remained stable over time. The error rate difference as a result of the reward effect (6.4% ± 5 % vs. 15.1% ± 8.2%) did not change over time as there was only a main effect of reward, F(1, 15) = 20.973, p < 0.001, η_p² = 0.583, but not block, F(5, 75) = 0.626, p = 0.681, and no interaction, F(5, 75) = 0.372, p = 0.867, showing that the results can again not be explained in terms of a speed-accuracy trade-off. 

Figure 4

 

Time course of the reward effect on mean RT for trials in which both trained circles were present. For each subject the mean RT for each block and condition was calculated and averaged across all subjects

Taken together these results suggest that a previously trained stimulus-reward contingency leads to a sustained bias of attention. However, training a stimulus-target association alone (i.e., without reward) is not enough to obtain such an effect. Crucially, the results suggest that the attentional bias remained stable over time during the course of the test phase (see Figures 3 and 4). 

Attentional orienting

Having established that the paradigm is sensitive to a reward-induced bias in attentional orienting, we investigated whether participants also showed RT costs and benefits due to associated reward relative to a neutral baseline. For that purpose, we calculated mean RT when both non-trained circles were present in the display (both non-trained circle present condition). A previously performed t test on target location revealed neither an effect on RT, t(15) = 0.190, p = 0.852, 95% CI (−10.86, 12.54), nor on error rate, t(15) = 0.940, p = 0.362, 95% CI (−0.0104, 0.0503), suggesting that this condition was not affected by reward or training and hence can be considered to be a proper neutral baseline. Relative to this baseline, there were RT benefits when comparing it to trials in which the target appeared in the circle associated with reward, t(15) = 4.007, p = 0.001, 95% CI (20.68, 7.60), and there were costs when the trained reward circle was present but the target appeared at the other location, t(15) = 2.602, p = 0.020, 95% CI (−18.00, −2.53). Consistent with the notion of spatial attentional capture, attention was always oriented towards the trained reward circle: If it contained the target, participants were faster than baseline; if not, attention needed to be reoriented away from the trained reward circle, causing significant costs relative to baseline (see Figure 5). Note that there were no costs and benefits for the condition in which a color was associated with the target but never rewarded (all comparisons t < 1). 

Figure 5

 

Analysis on performance costs and benefits as evident in the test phase.

Correlation of training and test phase

We also quantified the relationship between training and test phase by correlating the participants' individual difference scores for the target location in the rewarded versus non-rewarded condition of the training with the difference scores for the target location in the trained reward versus trained non-reward condition of the test phase. Note that this comparison is not trivial, as the trained reward versus trained non-reward condition is a stimulus configuration that was never presented during the training. The Fisher transformed Pearson correlation coefficient revealed a reliable correlation between the difference scores, r = 0.458, p (one-tailed) = 0.043, 95% CI (0.10, 0.73). This result indicates that participants who showed the stronger reward effect during training also showed a stronger reward effect during the test phase in which immediate reward was completely absent (Figure 6). 

Figure 6

 

Correlation of training and test performance. Higher values correspond with more attentional capture. †p value represents one-tailed t test.

Discussion

The present results demonstrate that previously learned stimulus-reward contingencies affect the speed and accuracy of attentional orienting. Relative to a neutral baseline, there were RT benefits when the target appeared at the same location as the previously rewarded circle (e.g., “valid” trials in the classical Posner task), and RT costs when the target appeared at the other location having a colored circle that was associated with no reward (“invalid” trials; see Figure 5). These costs and benefits for attentional orienting were exclusive for the stimulus that was trained with explicit reward and were stronger for participants that showed a stronger reward effect during training. 

The present findings are consistent with recent studies demonstrating that a stimulus associated with high reward affects attentional selection priority in visual search (Anderson, 2013; Hickey et al., 2010). For example, Anderson et al. (2011a) showed that when searching for a shape singleton, a distractor with a color associated with a high reward caused more slowing in search than a distractor associated with a low reward. This increase in search time was attributed to an increased involuntary shift of spatial attention to the location of the high value distractor (see also Hickey et al., 2010). Even though viable, these studies did not show that the increase in search time was indeed due to increased shifts of spatial attention towards the location of the high value distractor. These results could likewise be explained by filtering costs, as the presence of the high reward distractor competes for attention and needs to be filtered out, causing the increase in search time. The present study, however, clearly demonstrates that learned value does have a direct effect on the orienting of spatial attention by showing costs and benefits in attentional orienting for the previously rewarded circle. 

The present findings are also consistent with oculomotor studies which showed differential effects of actual learned value on the saccadic eye movement system (Anderson & Yantis, 2012; Hickey & van Zoest, 2012; Theeuwes & Belopolsky, 2012). For instance, in Theeuwes and Belopolsky (2012) participants performed an oculomotor task during a separate training phase in which they had to make a saccade to one of two target objects (either a horizontal or a vertical bar) that was presented among five other objects. The two target objects were associated with different reward schedules (high or low). In the test phase, participants performed a color singleton search task and were asked to make a saccade only to the target singleton, ignoring all other stimuli. Importantly, two-thirds of the trials also contained one of the previously rewarded objects as a distractor in the display. The results showed that participants' eyes were significantly more often captured by the onset of the distractor that was associated with high reward compared to the one associated with low reward. Although saccadic latency was not affected by the associated reward, the presence of a distractor significantly slowed saccades towards the target as compared to when it was absent. These findings provide evidence that when participants have the instruction to make a saccadic eye movement, stimuli that are associated with high reward can capture the eyes and disrupt on-going, goal-directed behavior even when they are no longer rewarded. A comparable conclusion was drawn by Anderson and Yantis (2012) providing evidence for oculomotor capture under more naturalistic conditions of an unconstrained viewing adaptation of their visual search task (Anderson et al., 2011b). Even though these studies clearly show that reward contingencies during the training of saccadic eye movements may result in increased oculomotor capture during testing, such a finding does not necessarily mean that spatial attention is also captured in tasks (such as the current one) which do not require eye movements. Indeed, several studies have shown that there may be a dissociation between attentional and oculomotor capture (Belopolsky & Theeuwes, 2009; Theeuwes, De Vries, & Godijn, 2003). Also, rewarding an overt action, such as the execution of a saccadic eye movement towards a particular object, may have a much stronger and more lasting effect on subsequent motoric action than the training of a covert shift of attention as rewarded in the present study. 

The spatial orienting of attention towards the color associated with the reward in the present study should be considered to be stimulus-driven (bottom-up) in origin as the colors of the stimuli were completely irrelevant for the task during the test phase. As argued by Yantis and Egeth (1999), one can only speak of attentional capture in a purely stimulus-driven or bottom-up fashion when the stimulus feature in question is completely task-irrelevant and when there is no incentive for the participant to deliberately attend to it. Clearly, during the test phase the colored circles were task-irrelevant and there was no incentive to attend to any specific color as doing so was neither rewarding nor helping task performance. Yet, the results show that attention was first oriented to the color associated with reward. It should also be stressed that the effect is not only present in the beginning of the test phase (for which one may claim that some transfer for training must have occurred) but notably remains present during all blocks of testing. Indeed, in the last block of trials, after a total of 432 trials in which no reward was ever given, the effect on attentional orienting was equally strong as in the beginning of testing (see Figures 3 and 4). 

The present study shows that a color that was associated with merely finding the target during training does not obtain the same status as a color that was associated with finding the target and was rewarded during training. In fact, during the test phase there was no difference between the color that was associated with finding the target and a color that was never associated with finding a target. Consistent with Anderson et al. (2011a), this suggests that the mere training of a stimulus-target relationship alone is not sufficient to produce a bias in visual orienting. 

It is important to note that the training procedure that we used in the current study has similarities to the training procedure of Anderson et al. (2011a, 2011b). However, there are also important differences: In Anderson et al.'s experiments, participants were explicitly told to search for either a red or a green target circle among a number of other colored nontarget circles. In other words, participants had an explicit search template indicating which color was associated with the high reward and which was associated with a low reward. This explicit instruction may have biased search. In the present study, however, participants were not explicitly told to search for any particular color. Instead, they had to find a particular target letter which was place inside a colored circle. Participants had to find out which color was associated with the reward and which color was not if they wanted to obtain more reward. It is feasible that without an explicit instruction the association between the stimulus and reward becomes stronger as a strategic search template may play a lesser role. A stronger association between the stimuus and reward could explain why in the present experiment during testing the effect of reward on attentional selection remained equally strong across trials, an effect that was not seen in the Anderson et al. (2011a), in which the effect of reward on attentional priority was no longer present in the later blocks of the test phase (see also Theeuwes & Belopolsky, 2012). Note, however, that other studies have found longer and persisting attentional bias effects when longer intervals between training and test phase were used (Anderson & Yantis, 2013; Della Libera & Chelazzi, 2009). 

Until not very long ago, changes in selective visual attention have been exclusively assigned to two rather separate domains, top-down and bottom-up control. Within this framework, physically measurable perceptual salience is integrated in a perceptual salience map to form exogenous attention (e.g., Itti & Koch, 2001). Here we show that a physically identical and task-irrelevant stimulus can elicit costs and benefits in exogenous attentional orienting which are subject to a previously trained stimulus-reward contingency. The present study hence adds to the growing body of literature (Anderson, 2013; Chelazzi et al., 2013) suggesting that not all aspects of exogenous attention can be explained in terms of the traditional concept of salience. The findings support the notion of a priority map integrating both top-down goals and physical salience as well as selection history (Awh et al., 2012). 

Acknowledgments

This research was supported by an ERC advanced grant [ERC-2012-AdG – 323413 Jan Theeuwes]. We would like to thank the anonymous reviewers for valuable comments on an earlier version of the paper. 

Commercial relationships: none. 

Corresponding author: Michel Fabian Failing. 

Email: [e-mail]. 

Address: VU University, Department of Cognitive Psychology, Amsterdam, The Netherlands.