Neuronal activity in superior colliculus signals both stimulus identity and saccade goals during visual conjunction search

Abstract

Although we know that the process of saccade target selection is reflected in the activity of sensory–motor neurons within saccade executive centers, the description of this process at the neural level has yet to fully account for all selection outcomes. The current study sought to determine how neuronal activity in the intermediate layers of the superior colliculus (SC) determines correct saccade target selection by examining the activity of visuomovement neurons during both correct and error trials of monkeys performing a relatively difficult visual conjunction search task. We found that a stimulus presented in a neuron's response field, but not foveated, was associated with greater activity if it was the search target instead of a distractor, indicating that SC neurons could represent stimulus identity. Nevertheless, activity was greater when a saccade was made to a stimulus than when it was not, further implicating these neurons in selecting the saccade goal. Together with the related observation that, when the target fell in their response fields, SC neurons discharged significantly more if the monkey correctly selected it instead of a distractor, these results suggest that visual stimuli are selected when these neurons reach a critical activation level. Our findings show that the outcome of all visual search trials, regardless of the stimulus being selected, is predicted by SC neuronal activity.

Introduction

The sensory guidance of movement entails a sequence of processing stages, which, in the presence of several alternatives, must include the discrimination of the target stimulus as well as the programming of the correct response. In the case of visual behavior, studies using the visual search paradigm have suggested the existence of such a two-stage selection process, in which a visual/attentional analysis selects the search target from distractor stimuli before a motor program is initiated to eventually produce a saccadic eye movement that brings the target image onto the fovea (see, for a review, Schall & Thompson, 1999). Neurophysiological studies have shown that these two processes are reflected in the activity of sensory–motor neurons in both the frontal eye fields (FEFs; Sato & Schall, 2003; Thompson, Hanes, Bichot, & Schall, 1996) and the intermediate layers of the superior colliculus (SC; McPeek & Keller, 2002a) during visual feature search tasks, in which the target and distractors differ by a single visual feature. Neurons in these saccade executive centers initially respond to the presentation of a stimulus in their response fields, regardless of the stimulus' feature or identity (i.e., target or distractor), but their activity eventually evolves to signal the location of the search target before saccade initiation. These previous studies have been generally concerned with neural activation during trials in which monkeys made a single correct saccade to the target stimulus. Behavior and performance in these tasks, however, are not perfect. This is especially true for more difficult conjunction search tasks in which stimuli are defined by a conjunction of multiple features (Bichot & Schall, 1999a; Valero & Paré, 2003). We therefore set out to determine how the brain participates in the guidance of attention in all trial outcomes. In other words, does sensory–motor activity predict whether a trial will be correct or incorrect? 

Models of visual attention have proposed that the brain represents the visual world in a salience map that is responsible for the deployment of all covert and overt shifts of attention (Cave & Wolfe, 1990; Findlay & Walker, 1999; Itti & Koch, 2001; Olshausen, Anderson, & Van Essen, 1993; Treisman, 1988; Wolfe, 1994; for a review, see Thompson & Bichot, 2005). Created from converging bottom–up and top–down information, the salience map contains representations of different sensory stimuli, whose magnitudes are related to the relative importance of each stimulus and the probability of directing a saccade to them (Logan, 1996; Wolfe, 1994). These peaks of activity are resolved in a winner-take-all competition on a trial-by-trial basis such that the highest peak is selected as the saccade goal (Cave & Wolfe, 1990; Findlay & Walker, 1999; Itti & Koch, 2001). A search target that is highly discriminable from distractors would have a far greater representation than all other stimuli and would therefore be easily selected. In the case of difficult search, target and distractor representations have more similar dimensions and, on any given trial, there is a greater probability that the representation of a distractor exceeds that of the target and leads to the incorrect selection of that distractor. Logan's (1996) CODE Theory of Visual Attention describes how the probability of each behavioral choice is a function of the proportion of its representation above a criterion threshold applied across the surface of the salience map. With low target discriminability, the target can be correctly selected with an elevated criterion threshold set above the level of all distractor representations, but this comes at the cost of speed. Applying a lower criterion threshold speeds up the selection process, but it reduces accuracy by as much as the proportion of distractor representations above the criterion threshold. Here, we examined whether the SC participates in such a selection process by testing the hypothesis that a certain level of neuronal activity must be reached before the search target, or any other stimulus, is selected as the saccade goal. 

There is already evidence that FEF neuronal activity predicts the saccade goal for both correct and incorrect trials in visual feature search tasks (Thompson, Bichot, & Sato, 2005). In the present study, we recorded single visuomovement neurons from the SC intermediate layers while two monkeys performed a visual conjunction search task, which captures more closely the complexity of most visual situations and thus provides us with a more general account of the selection mechanisms within the salience map. As this task is generally more difficult to perform than feature search tasks (Shen & Paré, 2006a), it reliably yielded a significant number of error trials from which neuronal activity could be examined. Besides the bottom–up information that the salience map receives from multiple feature maps, performance in visual conjunction search also requires the integration of top–down information, that is, the representation of the target identity in short-term memory. We therefore also examined whether SC neuronal activity reflects this top–down influence by representing the target differentially (representing the stimulus identity) or only the location of the upcoming saccade (i.e., the saccade goal). Preliminary reports of these data were previously presented in abstract form (Shen & Paré, 2005, 2006b). 

Methods

Data were collected from two female rhesus monkeys ( Macaca mulatta, 4.5–6.0 kg) cared for under experimental protocols approved by the Queen's University Animal Care Committee and in accordance with the Canadian Council on Animal Care guidelines and the U.S. Public Health Service Policy on Humane Care and Use of Laboratory Animals. 

The surgical procedure, stimulus presentation, data acquisition, and behavioral data analyses have been previously described (Shen & Paré, 2006a). A plastic recording cylinder was placed in the dental acrylic implant, directed toward the SC (15 mm above and 0 mm anterior/posterior of stereotaxic zero), and centered on the midline with the top tilted 40° posterior of vertical. During the postsurgery recovery period, monkeys received both antibiotics and analgesic medications. They were then trained with operant conditioning and positive reinforcement to perform fixation and saccade tasks on a daily basis for a liquid reward until satiation. The extracellular activity of single SC neurons was recorded using previously described techniques (Paré & Wurtz, 2001). Spike occurrences were sampled at 1 kHz. 

Behavioral paradigms

Neurons were first characterized while monkeys performed a delayed saccade task to temporally dissociate visual stimulation from saccade execution (Paré & Wurtz, 2001). The main data of this report were collected while monkeys performed a subsequent visual conjunction search task (Shen & Paré, 2006a). For each trial, a central fixation stimulus initially appeared and acted as a cue for the identity of the target stimulus. The animals were required to look at the fixation stimulus within 1,000 ms of its appearance and remain fixated for 500–800 ms. Following the fixation interval, the fixation stimulus disappeared simultaneously with the appearance of a concentric array of one target stimulus and seven distractor stimuli on a dark background. During each trial, either a target or a distractor appeared randomly in the center of the neuron's response field and all other stimuli were randomly positioned equidistant from the fixation stimulus and from each other. The target was a combination of a color (green or red) and a form (circle or square). The monkeys were given 500 ms after stimulus presentation to foveate the target for 200–300 ms. If the target was successfully foveated, the animals received a maximal liquid reward amount along with a reinforcement tone. If the first saccade was incorrectly directed at a distractor, the monkeys were given an additional 2,000 ms to foveate the target; a minimal amount of liquid reward was delivered (<0.33 of the maximal amount along with the reinforcement tone) on eventual fixation of the target. 

A simple detection task preceded each block of conjunction search trials, in which the intended target of the search first appeared as the fixation stimulus and then stepped to one of the eight positions used in the conjunction search task. This task was performed in a single block of 160 trials to familiarize the monkey with the search target in the visual conjunction search trials. 

Data analysis

Trials were identified as correct if the monkey successfully foveated the target after a single saccade and as incorrect if more than one saccade was made before target foveation. Trials in which the animals failed to initiate fixation or to make a stimulus-directed saccade were discarded from analysis. The probability of making the initial saccade to the target stimulus (saccade probability) for correct trials was calculated simply as the proportion of correct trials in each session, whereas the probability of an incorrect saccade was the proportion of incorrect trials in each session. Response time (RT) was measured as the time taken to initiate the first saccade following target presentation and is reported as the median of the distribution of RT because the distributions were skewed. 

Trials were sorted offline into four outcomes ( Figure 1) based on the visual stimulus presented in the neuron's response field and the monkey's initial response:

•  
  T _INSAC _IN ( Figure 1A) included all trials in which the monkey made a single correct saccade to the target presented in the neuron's response field;
   
  D _INSAC _IN ( Figure 1B) included all trials in which the monkey made an incorrect initial saccade to a distractor presented in the neuron's response field while the target was in one of five opposing positions;
•  
  D _INSAC _OUT ( Figure 1C) included all trials in which a distractor was in the response field and the monkey made a single correct saccade to the target presented in one of three opposing positions; and
•  
  T _INSAC _OUT ( Figure 1D) included all trials in which the target was presented in the neuron's response field and the monkey made an incorrect initial saccade to a distractor stimulus in one of five opposing positions.

Comparison between activity recorded in the different types of trials examined the following processes: target selection in correct trials ( Figure 1A vs. Figure 1C), distractor selection in incorrect trials ( Figure 1B vs. Figure 1D), saccade selection ( Figure 1A vs. Figure 1B), target selection ( Figure 1A vs. Figure 1D), and representation of stimulus identity ( Figure 1C vs. Figure 1D).

Figure 1

 

Conjunction search task trial outcomes. (A) Correct trials in which monkeys made a single saccade to the target stimulus in the neuron's response field (T _INSAC _IN); (B) an initial saccade directed at a distractor in the response field but eventual fixation of the target in an opposing position (D _INSAC _IN); (C) a single correct saccade to the target in an opposing position (D _INSAC _OUT); (D) an initial incorrect saccade to an opposing distractor but eventual fixation of the target in the neuron's response field (T _INSAC _OUT). In all of the sample displays, the search target is the unique red circle.

A neuron was included in the data analysis if it had at least five trials in whichever of the above outcomes considered. As the response fields of SC neurons are large and broadly tuned (Edelman & Keller, 1998; Goldberg & Wurtz, 1972; Sparks, Holland, & Guthrie, 1976; Stanford & Sparks, 1994), trials in which the initial saccade was directed to stimuli flanking the response field were excluded from analysis. 

The methods used to analyze the neural data have been previously described (Thomas & Paré, 2007). Briefly, rasters of neuronal discharge and continuously varying spike density functions were aligned on either the time of visual stimulus presentation (stimulus aligned) or the onset of the first saccade (saccade aligned). Spike density functions were constructed by convolving spike trains with a combination of growth (1-ms time constant) and decay (20-ms time constant) exponential functions that resembled a postsynaptic potential (Thompson et al., 1996). 

Signal detection theory (Green & Swets, 1966) was used to estimate how well an ideal observer of SC activity can distinguish between two sets of activity (during successive 5-ms intervals) by calculating receiver operating characteristic (ROC) curves (Thompson et al., 1996). The area under each of the ROC curves was then plotted as a function of time to describe the time course of neuronal choice probability. These probability values were fit with a Weibull function: W(t) = γ − (γ − δ)·exp[−(t / α)^β], where t is the time after stimulus onset, α is the time at which the function reaches 64% of its full growth, β is the slope, γ is the upper asymptote of the function, and δ is the lower asymptote of the function. A chance probability of .5 reflects the complete overlap of activity distributions, whereas a probability of 1.0 indicates perfect discrimination between the two stimuli. The time at which SC neurons could determine which stimulus fell in their response fields (discrimination time [DT]) was taken as the time at which the function reached the criterion value of 0.75. The initial discrimination bias probability was taken as the function's first asymptote, whereas discrimination magnitude (DM) was defined as the second asymptote. The initial discrimination probability indicated whether neurons were already discriminating at the beginning of a trial, whereas DM reflected the degree to which the ideal observer could accurately distinguish between the two distributions just before saccade initiation. Weibull functions were calculated only with spikes occurring before saccade onset and terminated as soon as there were less than five trials in one set of trials. The RT ranges for the sets of trials were matched across all comparisons. 

We used a 25-ms epoch centered on the DT calculated from the comparison of correct target and distractor trials (T _INSAC _IN vs. D _INSAC _OUT) to quantify a neuron's activation in advance of saccade initiation for each trial outcome (early selection activation). This arbitrary epoch represents the first instant that a neuron can correctly signal the identity of the stimulus in its response field, giving us a comparative measure of the neuron's earliest selectivity for that stimulus. Unlike DM, this epoch always ended well in advance of saccade initiation. In addition, an ideal observer analysis compared the early selection activation in correct target trials (T _INSAC _IN) to that measured in the other trial outcomes (T _INSAC _OUT, D _INSAC _OUT, and D _INSAC _IN) to quantify each neuron's selectivity for the search target in advance of saccade initiation (early target selectivity). 

Statistical significance was set at p < .05. All values reported are mean ± standard error unless otherwise stated. 

Results

Behavioral and neuronal database

The behavioral performance of two monkeys was examined in a total of 31,937 trials over 48 experimental sessions. The target was foveated with a single correct saccade with a probability well above chance (Monkey G: 0.665; Monkey H: 0.716). Median RT averaged 164 ± 2 ms for correct target trials (T _INSAC _IN), and it did not differ significantly across trial types ( Figure 2A; T _INSAC _OUT: 160 ± 2 ms, D _INSAC _OUT: 163 ± 2 ms, D _INSAC _IN: 167 ± 3 ms; ANOVA on ranks, H = 2.70, df = 3, p = .44). A speed/accuracy trade-off did, however, exist across sessions, as accuracy increased with RT ( Figure 2B; Spearman rank correlation test, r = .33, p < .05). 

Figure 2

 

(A) Cumulative probability distributions of RTs for all trial outcomes. (B) Accuracy as a function of median RTs in correct trials. Chance performance in this task was 12.5%.

A total of 48 neurons were recorded from within the SC intermediate layers (14 from Monkey G and 34 from Monkey H) and identified as visuomovement neurons while monkeys performed the delayed saccade task. Their activity before saccades (last 100 ms before saccade onset) was significantly greater (rank sum test, p < .001) than the corresponding delay activity (last 300-ms epoch before fixation stimulus disappearance), and they had reliable visually evoked responses within 100 ms of stimulus onset. We calculated a visuomovement index (VMI) to quantify the relative magnitude of visually evoked and saccade-related activity measured in the delayed saccade task and to identify where each neuron was situated along the visuomovement axis: VMI = (vis − mov) / (vis + mov), where vis is the peak activity within 100 ms of stimulus onset ( M = 169 ± 11 sp/s, range = 31–427 sp/s) and mov is the peak activity within ±40 ms of saccade initiation ( M = 443 ± 26 sp/s, range = 78–746 sp/s). Thus, a neuron with a much stronger visually evoked response than saccade-related activity would have a VMI close to 1.0, whereas a neuron with only saccade-related activity would have a VMI of −1.0. The VMI ranged from −0.85 to +0.58 ( M = −0.37 ± 0.29), with the great majority (41/48, 85%) having saccade-related activity greater than visually evoked responses. 

Neuronal discrimination

The role of SC in saccade target selection during visual feature (McPeek & Keller, 2002a) and conjunction (Valero & Paré, 2003) search has been previously investigated in studies in which correct target trials were compared to correct distractor trials. To determine whether the neurons sampled in this study similarly participated in the discrimination of target from distractors, we performed an ideal observer analysis to estimate how distinct the activation was between correct target (T_INSAC_IN) and distractor (D_INSAC_OUT) trials. Figure 3A illustrates how the initial activation of a sample SC neuron was not different for a target (black curve) or distractor (gray curve). Its initial discrimination probability was near chance (Figure 3B, gray curve), but over time, the activity grew to signal the presence of the target in the response field: Activity associated with a target became enhanced and that associated with a distractor was suppressed. This neuron reached the criterion threshold of discrimination 77 ms after stimulus onset and well before saccade initiation; its DM was 0.98. Across the neuronal sample, the initial discrimination probability in response to the presentation of a target versus a distractor was not significantly different from chance (Table 1). Similarly, the neurons' visually evoked responses (first 25 ms of activation) were not different for the target or any distractor (one-way ANOVA, F = 0.15, df = 3, p = .93). The activity of all neurons, however, evolved to significantly discriminate the target from distractors before saccades were initiated (Figure 4B, black histogram; Table 1). The average lead time obtained from stimulus-aligned data was not significantly different from the average DT measured using saccade-aligned data (Table 1; paired t test, p = .88). 

Table 1

 

Neural choice probability parameters (M ± SE, range) for stimulus-aligned and saccade-aligned data.

 

Note: *Not significantly different from .5 ( p > .05). ^†Significantly different from .5 ( p < .05). ^‡Significantly different from each other ( p < .05).

Figure 3

 

(A) Mean (± SE) activation of a representative neuron for correct target (T _INSAC _IN), incorrect target (T _INSAC _OUT), incorrect distractor (D _INSAC _IN), and correct distractor (D _INSAC _OUT) trials, aligned on stimulus presentation. The shaded area shows the 25-ms epoch centered on the DT calculated from the T _INSAC _IN versus D _INSAC _OUT comparison used to quantify a neuron's activation in advance of saccade initiation for each trial outcome (early selection activation). (B) Neuronal choice probability functions of correct target trials compared to correct distractor trials (gray), incorrect target trials (blue), and incorrect distractor trials (red).

Figure 4

 

Early target selectivity (A and C) and DM (B and D) when comparing correct target trials to correct distractor trials (T _INSAC _IN vs. D _INSAC _OUT, black), incorrect target trials (T _INSAC _IN vs. T _INSAC _OUT, blue), or incorrect distractor trials (T _INSAC _IN vs. D _INSAC _IN, red).

In summary, SC neurons signal the selection of a targeting saccade in advance of its initiation, consistent with previous studies (McPeek & Keller, 2002a; Valero & Paré, 2003). By examining only correct trial outcomes, these studies have not addressed whether these neurons signal the identity of the stimulus in their response fields (i.e., target vs. distractor) or, simply, the upcoming saccade goal. Furthermore, what happens when the target is not initially selected? How does stimulus representation within SC lead to either correct target selection or incorrect distractor selection? We explored these issues by comparing directly the four possible trial outcomes (see Figure 1). If a neuron merely selects the goal of the impending saccade, then stimuli that are not selected should be similarly represented, regardless of identity. Moreover, if a stimulus is selected as the saccade goal only if its representation exceeds a certain level, then we would predict that, on error trials, the target activation was lower than this criterion. The first set of analyses will test this prediction by comparing the activity during trials in which the target was presented in a neuron's response field and the monkey either correctly selected it (T_INSAC_IN) or failed to do so (T_INSAC_OUT). The second set of analyses will determine whether a distractor becomes selected (D_INSAC_IN) or not (D_INSAC_OUT) if the neuronal activity exceeds the selection threshold estimated from target trials. 

Target selection

What leads to the failure of target selection in error trials? To answer this question, we compared the activity of neurons when a target in their response fields was either correctly selected (T _INSAC _IN) or not (T _INSAC _OUT). Figure 3A shows how our sample neuron's initial activation did not predict the outcome of a trial, but its activation evolved to select the saccade goal in correct trials: Target-related activity in correct trials (black curve) became more enhanced than in error trials (blue curve). For this neuron, when data were aligned on stimulus onset, the probability that an ideal observer could predict whether the trial would be correct or incorrect was initially near chance ( Figure 3B, blue curve) but reached the discrimination threshold at 92 ms; its DM was 0.98. Across the sample, the mean initial discrimination probability was near chance ( Table 1), and the function remained at this level past the visually evoked responses. All neurons had DMs >0.5, and for 81% of neurons (39/48; Figure 4B, blue histogram), this function grew to reach the criterion threshold of discrimination before saccade initiation ( Table 1). This was significantly later than the DT associated with target selection in correct trials (T _INSAC _IN vs. D _INSAC _OUT; paired t test, p < .05). The average DM was also significantly less than that found when comparing correct target and distractor trials ( p < .001). Similar results were obtained when data were aligned on saccade onset ( Table 1). These observations suggest that reduced activation associated with the target leads to that target not being correctly selected. That the ability of the ideal observer to discriminate a correct target trial from an incorrect target trial is less reliable than its discrimination from a correct distractor trial (D _INSAC _OUT) implies that SC neurons represent some aspect of stimulus identity, that is, whether the stimulus is a target or a distractor. 

Could trial outcome be predicted well in advance of saccades? To answer this question, we examined the early selection activation for each trial outcome (see Figure 3A and the Methods section). If SC neuronal activity predicts whether a correct selection will be made, then the activity in advance of saccade initiation should be greater for a correct selection than for an incorrect one. Figure 5B shows how, for our sample neuron, the early selection activation in correct target trials was greater than in error trials (107 vs. 90 sp/s; t test, p < .05). Across the sample, the average activation was significantly greater for correct than incorrect trials (124 ± 7 vs. 102 ± 7 sp/s; paired t test, p < .001), suggesting that SC neurons are involved in correctly selecting the target as the saccade goal. 

Figure 5

 

Selection threshold. (A) ROC curve of T _INSAC _IN versus T _INSAC _OUT distributions for a sample neuron. The ideal criterion is defined as the point on the ROC curve for which the sum of its x and y values is closest to 1. (B) Distributions of early selection activation for the same sample neuron for correct (T _INSAC _IN, black) and incorrect target (T _INSAC _OUT, blue) trials as well as incorrect (D _INSAC _IN, red) and correct (D _INSAC _OUT, gray) distractor trials. (C) Mean (± SE) early selection activation of all trial outcomes for 35 neurons. The criterion activation falls in between SAC _IN and SAC _OUT outcomes.

SC neurons are not known to be color selective (McPeek & Keller, 2002a; Ottes, Van Gisbergen, & Eggermont, 1987), and we showed above that their visually evoked responses do not reflect stimulus feature or identity. If these neurons were only involved in selecting a stimulus as the saccade goal, then their activation should be independent of the feature of the stimulus within their response fields when it is not selected. For our sample neuron (Figure 5B), early selection activation in incorrect target trials was still greater than that in correct distractor trials (90 vs. 68 sp/s; t test, p < .01). The same was observed for 85% of neurons (41/48) as well as for the averaged activation across the sample (102 ± 7 vs. 88 ± 6 sp/s; paired t test, p < .001). An ideal observer analysis comparing the early selection activation in correct target trials and incorrect target trials shows how SC neural choice probability predicts trial outcome in advance of saccade initiation (early target selectivity; Figure 4A, blue histogram). This early target selectivity was above chance for all but one neuron, and 31% (15/48) of neurons reached the criterion threshold of discrimination. Moreover, its distribution was significantly greater than chance (t test, p < .001) but less than the early target selectivity obtained from comparing correct target and distractor trials (Figure 4A, black histogram; p < .001). 

Along with the differences in DM found in the preceding temporal analysis, this evidence suggests that SC neurons are involved not only in correctly selecting the saccade goal but also in representing stimulus identity before the initiation of saccades. In incorrect trials, SC activity reflected the identity of the visual search target but was presumably not significant enough to select the saccade goal. It is, however, possible that the enhanced activation in incorrect target trials (T _INSAC _OUT) as compared to correct distractor trials (D _INSAC _OUT) is due to the planning of a subsequent saccade landing on the search target. Incorrect trials inherently include more than one saccade, and the enhanced activation during these trials may be evidence of concurrent processing. McPeek and Keller (2002b) found that, during feature search trials in which monkeys made two saccades to locate the search target, SC neurons showed activity related to the goal of the second saccade before and during the execution of the first. To control for this possibility, we analyzed only the trials in which the second saccade occurred significantly after the first. A minimum intersaccade interval (ISI) was determined for each session by subtracting the average saccade duration from the average RT. This eliminated 44% of incorrect target trials and excluded eight neurons from analysis. The remaining trials with two saccades had an average ISI of 174 ± 4 ms. For the remaining 40 neurons, average early selection activation in incorrect target trials was still greater than that in correct distractor trials (103 ± 9 vs. 87 ± 7 sp/s; paired t test, p < .001). Moreover, when we included only trials with more than two saccades, the effect persisted (103 ± 12 vs. 84 ± 10 sp/s; p < .001; N = 27). 

Selection threshold

The significantly greater activation observed in correct target trials than in error trials suggests that a critical activity threshold must be reached for a stimulus to be selected as the saccade goal. This predicts that a distractor would be selected if its representation exceeds this threshold. 

What level of activity determines whether a given trial will be correct or incorrect? To estimate a selection threshold for each individual neuron, we used the ideal observer analysis to compare the early selection activation in correct (T _INSAC _IN) and incorrect (T _INSAC _OUT) target trials (e.g., see Figures 5A and 5B). The selection threshold was defined as the ideal criterion, that is, the point on the ROC curve that maximized the probability that correct activation was greater than the criterion (tending to a value of 1) and simultaneously minimized the probability that incorrect activation was greater than that same criterion (tending to a value of 0). Across the sample of 48 neurons, criterion activation averaged 111 ± 7 sp/s, a level significantly less than the early selection activation in correct trials (T _INSAC _IN: 124 sp/s; paired t test p < .001) but still greater than that in incorrect trials (T _INSAC _OUT: 102 sp/s; p < .001). 

Next, we compared each neuron's early selection activation in trials in which a distractor was incorrectly selected (D _INSAC _IN) to their estimated selection thresholds ( Figure 5C). Sufficient data were available in 35 neurons (13 from Monkey G and 22 from Monkey H). In support of our hypothesis that a stimulus is selected if its representation exceeds the threshold, the average incorrect distractor activation (119 ± 9 sp/s) significantly surpassed the average ideal criterion (110 ± 8 sp/s; p < .01) but was not different from correct target activation (T _INSAC _IN: 124 ± 8 sp/s; p = .64). Conversely, the activation associated with a distractor that was not selected (D _INSAC _OUT: 87 ± 7 sp/s) was significantly less than both the estimated threshold ( p < .001) and the activation in incorrect target trials (T _INSAC _OUT: 99 ± 8 sp/s; p < .001). 

Consistent with the previous analyses of target selection, a target is correctly selected if its representation exceeds a threshold. When, instead, a distractor's representation surpasses the selection threshold, that distractor becomes incorrectly selected as the saccade goal. 

Saccade selection

Another case for SC's role in selecting the saccade goal can be made if we compare incorrect distractor (D _INSAC _IN) trials to those when the target was correctly selected (T _INSAC _IN). Figure 3A illustrates how, for our sample neuron, the activation profiles for these two sets of trials were very similar over time and the early selection activations associated with the target and distractor were not statistically different ( Figure 5B; 110 vs. 107 sp/s; t test, p = .52). For all neurons, the activation associated with an incorrect saccade to a distractor was not different from correct saccades made to the target (119 vs. 124 sp/s; paired t test, p = .12). For the sample neuron, the ideal observer initially performed near chance ( Figure 3B), and its DM was 0.38. On average, initial discrimination probabilities were not different from chance, whereas DMs were significantly greater than chance only when data were aligned on stimulus onset ( p = .03; p = .28 when data were aligned on saccade onset; Table 1). These DMs, however, were significantly less than those associated with correct trials (T _INSAC _IN vs. D _INSAC _OUT; Figure 4D, p < .001). An analysis of early selection activation distributions of the two SAC _IN outcomes revealed that early target selectivity was near chance ( Figure 4C, red histogram), whereas the comparison of correct trials (T _INSAC _IN vs. D _INSAC _OUT) yielded significantly greater selectivity ( p < .001). In summary, the early selection activation associated with a target or a distractor was not different if the responses were both saccades into the response field. 

The conclusions reached so far implicate SC in the role of both representing the stimulus identity and selecting the saccade goal, but to what extent does SC neuronal activity predict each of these? We addressed this question by relating each neuron's DM to saccade probability for trials that were either correct (T _INSAC _IN vs. D _INSAC _OUT) or incorrect (D _INSAC _IN vs. T _INSAC _OUT). Neurons that are exclusively concerned with where the saccade goes irrespective of stimulus identity should yield a DM of 1.0 regardless of saccade probability. Alternatively, neurons that only represent the identity of the target stimulus should show perfect discrimination for the target in both correct (DM = 1.0) and incorrect (DM = 0) trials. The same subset of 35 neurons was used for this analysis. As seen in Figure 6, the ensemble of neurons fall between these two alternatives, and at the level of the population, the evidence suggests that the SC spans the range between stimulus representation and the selection of saccade goals. 

Figure 6

 

SC activity reflects both stimulus identity and the selection of saccade goals. DMs from ideal observer analysis of correct (T _INSAC _IN vs. D _INSAC _OUT, black) and incorrect (D _INSAC _IN vs. T _INSAC _OUT, red) trials as a function of the probability of making a saccade to the stimuli. Each neuron contributes a pair of values to this plot.

The difference between DMs of the correct and incorrect comparisons describes each neuron's position within the two alternative processes of stimulus representation and saccade goal selection. Does this difference predict the ability of neurons to distinguish correct and incorrect target trials? We found that DM differences were related to a neuron's early target selectivity for correct (T _INSAC _IN) as opposed to incorrect (T _INSAC _OUT) target trials (see Figure 4A, blue histogram; Spearman rank correlation test, r = −.46, p < .01). Neurons that were more concerned with representing stimulus identity than selecting the saccade goal (i.e., large difference in DMs) were also those that were not good at discriminating correct from incorrect target trials (early target selectivity ∼0.5). In other words, these neurons had similar responses to a target in their response fields, regardless of the subsequent behavioral response. In contrast, neurons that were concerned with selecting the saccade goal (small differences in DMs) were better at separating correct from incorrect activations (early target selectivity >0.75) as they had far greater activation in advance of saccades made into their response fields. 

If a neuron's discharge characteristics contribute to its functional properties, then it is expected that neurons with greater saccade-related activation (or small VMIs) would have smaller differences in DM. There was no significant correlation between DM difference and VMI ( r = .18, p = .29), nor could DM differences be accounted for by the magnitude of the visually evoked responses ( r = .11, p = .52) or saccade-related activity ( r = −0.18, p = .29) measured in the delayed saccade task. A neuron's position between the two alternatives of stimulus representation and saccade goal selection was a product neither of its discharge characteristics nor of its position along the visuomovement axis. 

Discussion

We showed that, during a visual conjunction search task, the activity of SC visuomovement neurons predicted trial outcome in advance of saccade initiation. Enhanced target activation accompanied correct selection of the target as the saccade goal, whereas significantly lower target activation preceded an incorrect selection of a distractor stimulus. This observation supported the hypothesis that a critical threshold must be reached for a correct target selection. Moreover, that target activation in error trials was still greater than distractor activation suggests that these neurons represent stimulus identity. They also appear to play a role in selecting the saccade goal as their activation was greater than the estimated selection threshold when a saccade was made into their response fields than when it was made away, regardless of stimulus identity. Altogether, these results indicate that the SC is involved in the formation of a salience map of visual space that guides the selection of saccades. 

Selection mechanisms within the salience map

The representation of the visual world as a salience map has previously been supported by the activity of neurons from a network of sensory–motor brain regions while monkeys performed correct visual search trials (Bichot & Schall, 1999a; Ipata, Gee, Goldberg, & Bisley, 2006; McPeek & Keller, 2002a; Schall & Hanes, 1993; Thomas & Paré, 2007; see also Thompson, Bichot, et al., 2005). The data presented here are the first to explore SC neuronal activity during erroneous visual search trials, and they help to further incorporate this structure into the framework of the visual salience map. What this study adds to this literature is how SC neural activity predicts behavioral choice. Consistent with the salience map hypothesis, SC representations of visual stimuli were found to predict whether these stimuli would be selected as saccade goals. In addition, we showed evidence for a threshold of selection and estimated it by computing the ideal observer's optimal criterion distinguishing SC activity at the time of neural discrimination. Other evidence for the existence of a threshold of selection comes from a study using feature search tasks (Thompson, Bichot, et al., 2005), in which FEF neuronal activity related to a target in a neuron's response field without a saccade being initiated never reached the level that was reached for trials that resulted in a targeting saccade. McPeek and Keller (2004) also showed that inactivating the target representation within the SC during feature search led to the incorrect selection of distractor stimuli with equal probability, as if each distractor representation could equally reach the threshold of selection in the absence of a competitive target representation. The present study is the first to estimate the threshold of selection during visual conjunction search. By determining how well an ideal observer could distinguish SC neuronal activity associated with correct and incorrect target trials, we showed that the SC output could reliably predict behavioral choice in all trials, including distractor trials. 

The salience map is probabilistic, and on a given trial, especially in a difficult search task such as in this study, target and distractor representations compete for selection. Both the short RTs and the significant proportion of errors of our monkeys suggest that they used a relatively low criterion threshold to select their saccade goals from all stimulus representations. This search strategy resembles that observed in several human studies that did not constrain visual behavior (e.g., Findlay, 1997; Maioli, Benaglio, Siri, Sosta, & Cappa, 2001; Williams, Reingold, Moscovitch, & Behrmann, 1997). It also suggests that our search task promoted a more natural behavior than in previous monkey studies, which have reported better search accuracy and longer RT to visual search displays (Bichot & Schall, 1999b; McPeek & Keller, 2002a; Sato, Murthy, Thompson, & Schall, 2001; Sato & Schall, 2003; Schall, Hanes, Thompson, & King, 1995; Thompson et al., 1996). By allowing monkeys to make only a single saccade to foveate the target stimulus, the longer RT, along with the delayed discrimination timing, suggests that monkeys in these previous studies set a higher criterion threshold of selection. In this view, differences in the selection threshold underlie different search strategies and possibly account for intersession variability in behavioral responses (see Figure 2B). 

The evidence that SC activity accumulates over time toward a threshold of selection that dictates behavioral choice supports the integration-to-threshold mechanism hypothesized to underlie decision making (e.g., Mazurek, Roitman, Ditterich, & Shadlen, 2003). Our observation that the accuracy of the first saccades improves with the lengthening of their initiation (Figure 2) is consistent with the main prediction of such models. However, the somewhat low accuracy and narrow range of RT across sessions, the relatively weak correlation between accuracy and RT, and the lack of significant RT difference between trials with correct and incorrect stimulus selection suggest an integration process that is imperfect or perhaps time limited. Ludwig, Gilchrist, McSorley, and Baddeley (2005) have proposed a temporal filter model, in which only the earliest visual information (first 100 ms) following stimulus presentation contributed to the activity's rise to threshold, resulting in saccade RTs in visual search that are independent of task difficulty. Our data thus seem to be equally in line with this model, which harkens back to our previous report that the initial responses to our visual search tasks are best considered as triggered automatically (Shen & Paré, 2006a). At the present time, either mechanism could account for our data because integration-to-threshold models also predict the small RT variation we observed when subjects, like our monkeys, use a low criterion threshold. 

Target and saccade selection processes

The activity of most SC visuomovement neurons was found to be concerned with both the representation of stimulus identity and the selection of the saccade goal, suggesting that SC neuronal activity could be associated with both target and saccade selection processes, but to what extent are these distinct processes? In this respect, it is noteworthy that the basic discharge characteristics of our sample of visuomovement neurons failed to predict their ability to participate in the saccade target selection process. Within the SC, saccades are encoded by neuronal ensembles, which are topographically organized and dynamically interactive (Day & Paré, 2005; Munoz & Istvan, 1998). It is therefore likely that cooperation within ensembles underlies the process of stimulus identity representation and helps coordinate the selection of the saccade goal. That there appears to be no functional distinction during visual search between neurons with different basic discharge properties suggests that SC ensembles of visuomovement neurons are the operational elements of an integrated system in which the two processes of target and saccade selection become closely intertwined, at least under our experimental conditions. Functional distinctions based on discharge characteristics have, however, been found within FEF and SC neuronal populations in two recent studies on covert orienting, which may underlie the process of target selection: Whereas neurons with visually evoked activity had their activity modulated by attention, neurons with only saccade-related activity were unaffected (FEF; Thompson, Biscoe, & Sato, 2005; SC: Ignashchenkova, Dicke, Haarmeier, & Thier, 2004; see also Horwitz & Newsome, 1999). Although we sampled a wide range of visuomovement neurons in this study, we cannot rule out functional distinctions between extreme classes of neurons. Nevertheless, the hypothesis that movement-only neurons are a class apart because they are exclusively associated with saccade production is at odds with the demonstration that their activity can be observed in the absence of saccades (Hanes, Patterson, & Schall, 1998; Paré & Hanes, 2003). It may be that these neurons are recruited beyond saccade processing in overt visual search but that increased inhibitory (fixation) control during covert attention tasks gates their activation selectively. 

The representation of stimulus identity by SC neurons seems to be incompatible with the widely accepted view that these neurons lack feature-selective responses (for a review, see Sparks, 1986). This apparent conflict can be reconciled by considering that stimulus features are reflected in SC neuronal activity during visual search because they are behaviorally relevant. Indeed, SC neurons have been shown to display activity predictive of an upcoming saccade when a stimulus is made behaviorally relevant, that is, as the saccade target (Glimcher & Sparks, 1992; Goldberg & Wurtz, 1972; Paré & Wurtz, 2001). Salience maps may thus contain units that can display feature selectivity as an adaptive response to the demands of the task. Such an adaptive property is best exemplified by the observations that neurons in both SC (Horwitz, Batista, & Newsome, 2004) and FEF (Bichot, Schall, & Thompson, 1996) can develop strong feature selectivity following long-term exposure to that feature. Similar to these findings, top–down influences on the neuronal activity in FEF (Bichot & Schall, 2002) and SC (Day, Valero, & Paré, 2003) are also observed in trial-to-trial feature priming of a visual search target. 

Using the same analysis relating neuronal DM and saccade probability (see Figure 6), Thompson, Bichot, et al. (2005) also reported that the activity of FEF neurons was concerned with both the representation of stimulus identity and the selection of the saccade goal during feature search tasks. However, their results indicate that the activity of these neurons signaled the saccade goal much more so than it reflected the identity of the stimulus in their response fields. It is possible that this greater emphasis on saccade goal selection is related to differences in tasks and strategies. Required to foveate the target after only a single saccade, the monkeys in this previous study may have adopted a higher criterion threshold to select their saccade goals than in our study—an explanation consistent with their longer RT and greater accuracy (see above). On the other hand, the lesser emphasis on representing the stimulus identity could be simply related to the larger discrepancy between target and distractor representations in the feature search task on the saliency map, thereby rendering target discrimination relatively easy. 

Beyond promoting more natural gaze behavior, this study strived to move beyond the feature search task to investigate the selection mechanisms within the visual salience map. The saccade target selection process in natural visual scenes involves the selection of items with multiple descriptive features from other similar items. The activation of multiple feature maps and their combination in a salience map is thus more consistent with the complexity of most visual situations, and our approach provides a useful tool to understand both the associated visual behavior (Shen & Paré, 2006a) and its neural basis. 

Acknowledgments

The Canadian Institutes of Health Research, the J.P. Bickell Foundation, and the EJLB Foundation supported this work. K. Shen holds an Ontario Graduate Scholarship and a postgraduate scholarship from the Natural Sciences and Engineering Council of Canada. M. Paré holds a New Investigator Award from the Canadian Institutes of Health Research and an Early Researcher Award from the Ontario Ministry of Research and Innovation. 

We are grateful to W. Clarence for expert assistance with the training and preparation of the animals. We thank G. Day and J. Valero for their contributions to the data collection. We also thank the following individuals for their help in this work: T. Battle, L. Ekstrom, I. Everdell, R. Pinkerton, and M. Yurick for data analysis and visual display programs; S. Hickman, F. Paquin, R. Pengelly, and C. Wellstood for machining, electronics, and computing support; A. Lablans, K. Moore, and J. Scott for additional animal care assistance; B. Beresford, D. Harrington, and M. McMurray for veterinary care; and N. Bains for statistics. We appreciate the useful comments that E. Brunamonti, I. Everdell, and N. Thomas made on this article. 

Commercial relationships: none. 

Corresponding author: Martin Paré. 

Email: [e-mail]. 

Address: Department of Physiology, Queen's University, Botterell Hall, Rm. 438, Kingston, Ontario, K7L3N6, Canada.