Visually guided pointing movements are driven by the salience map

Abstract

Visual salience maps are assumed to mediate target selection decisions in a motor-unspecific manner; accordingly, modulations of salience influence yes/no target detection or left/right localization responses in manual key-press search tasks, as well as ocular or skeletal movements to the target. Although widely accepted, this core assumption is based on little psychophysical evidence. At least four modulations of salience are known to influence the speed of visual search for feature singletons: (i) feature contrast, (ii) cross-trial dimension sequence and (iii) semantic pre-cueing of the target dimension, and (iv) dimensional target redundancy. If salience guides also manual pointing movements, their initiation latencies (and durations) should be affected by the same four manipulations of salience. Four experiments, each examining one of these manipulations, revealed this to be the case. Thus, these effects are seen independently of the motor response required to signal the perceptual decision (e.g., directed manual pointing as well as simple yes/no detection responses). This supports the notion of a motor-unspecific salience map, which guides covert attention as well as overt eye and hand movements.

Introduction

The notion of a salience map is a core concept in current theories of visual search (e.g., Bisley & Goldberg, 2003; Itti & Koch, 2001; Wolfe, 1994). On these accounts, the salience map has several important characteristics: (i) it is a topographical representation of the visual scene; (ii) it signals local “distinctiveness” for each position in the scene, (iii) rather than information about the visual features that make a given position distinct; and (iv) it is motor-unspecific, that is, it can guide covert attention, eye, and body or limb movements. The latter assumption forms the question at issue in the present study: Although it is widely accepted that modulations of salience do equally affect yes/no detection, left/right localization, perceptual discrimination, and saccadic or manual reaching actions, there is actually little empirical evidence to substantiate this assumption with regard to the time course of selecting the first item. 

Functionally, the core effect of salience is to determine the priority and, thus, the sequence of further processing: locations are selected in the order of their salience, with the most salient location being selected first, followed by the second most salient location, etc. This functional effect of salience, namely, determining selection probability, has been used to explain performance in search tasks involving both artificial, laboratory stimuli and natural scenes. As for laboratory stimuli, there exist two broad search phenomena: a feature singleton, that is, an item that has a unique feature compared to all other items on the screen (e.g., a red among green disks) and is found rapidly and independently of how many items there are in the search array (e.g., Treisman & Gelade, 1980; Wolfe, 1994); by contrast, search for a conjunction target, such as the (line conjunction) letter T among variably (orthogonally) oriented letter Ls, becomes slower the more items there are presented. Itti and Koch (2000) explained this difference in performance in terms of the functional effect of salience: a feature singleton target always produces the highest activation on the salience map and thus is always selected first; by contrast, a letter T among letter Ls does not lead to a higher activation on the salience map, so that letters have to be serially selected and attentionally checked until the target is found, yielding an increase in search time the more items are present in the array. 

However, there is a second functional property of salience: Even if a target is the most salient item in the visual scene, further increasing its salience can still enhance performance (e.g., Found & Müller, 1996; Maljkovic & Nakayama, 1994; Verghese & Nakayama, 1994). This enhancement cannot be due to a modulation of selection probability, because the target is already the first item to be selected. Rather, it must result from a modulation of the time course of selection: when a target has reached the critical level of salience to be selected first, further increasing its salience leads to faster attentional selection. 

These two functional properties of salience have been investigated using different types of stimulation and responses. Whereas investigations of selection probability as a function of salience have mostly used natural images and ecologically valid, saccadic responses (e.g., Bruce & Tsotsos, 2009; Seo & Milanfar, 2009), the time course of selection has been examined using mainly laboratory stimuli and artificial, manual button-press responses (e.g., Found & Müller, 1996; Maljkovic & Nakayama, 1994). While there are virtually no studies investigating the time course of selection using natural stimuli, psychophysical studies employing natural responses to laboratory stimuli, such as saccades or pointing movements, are sparse (e.g., Becker, 2008a, 2008b; McPeek, Maljkovic, & Nakayama, 1999; Song & Nakayama, 2006). The basic assumption why it is justified to focus on manual key-press responses when investigating the time course of attentional selection is based on property (iv) of the salience map concept: salience affects detection as well as covert attentional selection, saccadic selection, and selection for manual reaching. Although this assumption is widely accepted, it is nearly unsubstantiated by psychophysical evidence. Arguably, however, such an empirical substantiation is crucial for extrapolating findings from studies using button-press responses to salience in general, irrespective of the action or effector by which responses are indicated. 

Investigating the motor unspecificity—cognitive construct—of the salience map psychophysically gains in importance as this issue is currently debated at the neuronal level as well. As pointed out by Gottlieb, Balan, Oristaglio, and Schneider (2009), there are two theoretical alternatives of how target selection decisions for, for instance, directed eye and limb movements are made. The salience map account assumes that target selection decisions are made based on activity of a salience map and the resulting target coordinates can be used to direct attention, the eyes, or limbs to the corresponding location. To this approach, Gottlieb et al. (2009) juxtapose an alternative account that (they understand) has been advanced by Gold and Shadlen (2007) and Mazurek, Roitman, Ditterich, and Shadlen (2003) and assume that sensory evidence is mapped directly onto a motor response:

 

“Given a source of sensory evidence, the accumulation of evidence toward a decision is postulated to occur in premotor networks that also plan the motor response through which the decision is expressed. If the decision is signaled with a saccade, saccade premotor areas (which would include LIP) are postulated to integrate the evidence toward the decision. If the decision is signaled with a limb movement, this integration is performed in limb premotor areas” (Gottlieb et al., 2009, pp. 1224–1225).

 

In other words, Gold and Shadlen's (2007) framing of the question implies that target selection decisions are made by motor-specific processing areas (see Figure 1 for an illustration). ¹  

Figure 1

 

Two alternative architectures underlying perceptual decisions indicated by either saccadic or limb movements. (a) The concept of a motor-unspecific salience map, where target selection decisions can be used to guide ocular and skeletal movements, and (b) that of motor-specific decision mechanisms, where target selection decisions are computed in premotor areas (see Gottlieb et al., 2009, for a similar depiction).

The aim of the present study was to contribute hitherto missing psychophysical evidence from visual search performance to the central assumption that perceptual decisions are mediated by a motor-unspecific salience map. In standard visual search paradigms, participants are required to indicate their decisions by speeded key presses—typically one key press for “target-present” and another for “target-absent” decisions; and reaction time (RT) differences are attributed to differences in search-related processes, especially the speed of attentional selection (or, respectively, to differences in response-related processes). In the standard search literature, a number of effects have been described that are assumed to reflect variations in attentional selection speed based on variations of target saliency. The strategy pursued in the present study was to reinvestigate these well-documented effects—however, with requiring observers to perform a manual reaching (rather than some arbitrary key press) response, that is, observers had to point to the target location as fast as possible. Salience map models predict that the same manipulations of target saliency that affect attentional selection in standard visual search tasks should also influence the initiation latencies of manual pointing movements. Failure to find such effects on manual pointing reactions would provide evidence against the notion of a motor-unspecific salience map. 

Hitherto, there have been few studies on manual pointing movements in visual search for feature singleton, “pop-out” targets (Song & Nakayama, 2006, 2007; see Song & Nakayama, 2009 for a review). Song and Nakayama (2006) reinvestigated the “priming of pop-out” (PoP) effect, originally established by Maljkovic and Nakayama (1994), in a paradigm in which observers had to find an odd-colored diamond and report which edge of the target diamond was chipped off (“left” vs. “right” decision). Importantly, in this paradigm, the assignment of target and distracter color can change from trial to trial; for example, the target may be red among green distracters on one trial, and green among red distracters on the next. The crucial finding is an intertrial “priming” effect: RTs are faster on a given trial when the target–distracter (color) assignment remains the same as on the previous trial(s), compared to when it changes. Song and Nakayama (2006) used the same paradigm but asked observers to manually point to the target. Based on the finding that pointing movements became more accurate and faster the more items were present in the display, Song and Nakayama (2006) concluded that manual pointing movements, as well as saccades and perceptual discrimination tasks, require the deployment of focal attention. Within the present context, this finding can be taken to indicate that manual, oculomotor and attentional tasks show a similar dependency on the number of items in the display (i.e., essentially, item density) and thus could all be based on a motor-unspecific salience map. However, the effect of display density could also be viewed as a rather special case. This is because in standard visual search paradigms (with manual key-press responses), interitem density has been reported to have an influence only in discrimination, but not detection, tasks (Bravo & Nakayama, 1992)—whereas frequent performance of detection tasks is likewise assumed to be driven by salience (e.g., Found & Müller, 1996; Nothdurft, 2002; Wolfe, 1994). 

Thus, arguably, to substantiate the notion that a motor-unspecific saliency map underlies attentional selection irrespective of the particular response system, it is necessary to examine the effects of a wider range of saliency modulations on, for example, manual pointing movements—in particular, manipulations that unequivocally influence “bottom-up” salience in different standard visual search paradigms, including yes/no detection tasks. To do so, the present study investigated the effects of a range of manipulations known to influence performance in visual search detection and discrimination tasks, in particular: (i) feature contrast, (ii) dimensional intertrial transitions, (iii) dimensional top-down cueing, and (iv) dimensional redundancy of target definitions—where manipulations (i) and (iv) are unequivocally varying “bottom-up” saliency. 

Feature contrast is known to modulate the strength of salience signals (Bruce & Tsotsos, 2009; Wolfe & Horowitz, 2004; see also Duncan & Humphreys, 1989): the higher the featural similarity of the target to the distracters, the less salient it is. Consequently, the time necessary to select the target increases with decreasing salience (e.g., Nothdurft, 1992; Sato, Murthy, Thompson, & Schall, 2001). 

Intertrial sequences of target-defining dimensions also influence search for feature singletons targets: search is facilitated if the target-defining dimension repeats (e.g., color-defined target on trial n preceded by color-defined target on n − 1) rather than changes (e.g., color-defined target on trial n preceded by orientation-defined target on n − 1), even if the target feature changes within the repeated dimension (e.g., red target on trial n preceded by blue target on trial n − 1; Found & Müller, 1996). An electrophysiological correlate of these sequence effects is found in the N2pc event-related component, which is considered to be a marker for attentional selection (e.g., Eimer, 1996; Luck & Hillyard, 1994): N2pc latencies are shorter for dimension repetition trials, compared to change trials (and this is the case even if the behavioral response requires discrimination of a target feature dissociated from the selection-critical feature; Töllner, Gramann, Kiss, Müller, & Eimer, 2008). These findings substantiate the original account of dimensional intertrial effects (Müller, Heller, & Ziegler, 1995), namely, that dimensional weights modulate salience computations. Thus, for example, if a color target is presented on a given trial, the weight for the color dimension is automatically increased and that for all other dimensions decreased. Consequently, when a color target is presented on the next trial, it can be singled out faster due to the increased dimensional weight assigned to the target dimension. 

Furthermore, dimensional weights and, thus, salience signal computations can be top-down modulated if observers intentionally prepare for one specific target dimension (e.g., Müller, Reimann, & Krummenacher, 2003). For example, search for a color target on a given trial is expedited if observers receive a valid symbolic pre-cue indicating that the next target is likely to be color-defined, compared to an uninformative (neutral) or invalid (in the example: orientation) pre-cue. This dimensional cueing effect has also an electrophysiological correlate in the N2pc component: N2pc latencies are shorter for valid than for invalid cues (Töllner, Zehetleitner, Gramann, & Müller, 2010). 

Finally, salience models assume that feature contrast signals generated within different dimensions (e.g., color and orientation) are integrated/summed in the computation of the attention-guiding overall salience map. Salience summation predicts that if a target is redundantly defined in two dimensions (e.g., differing from distracters in both color and orientation), rather than just in one dimension (e.g., differing from distracters only in color or only in orientation), search performance is facilitated. While such redundancy gains have been observed frequently (e.g., Koene & Zhaoping, 2007; Krummenacher, Müller, & Heller, 2001, 2002), Zehetleitner, Krummenacher, and Müller (2009) were able to demonstrate that indeed co-activation (i.e., salience summation)—rather than a mere parallel race, parallel interactive processing, or serial exhaustive processing of feature contrast signals from the two dimensions—accounts for the observed (RT distribution) pattern of the redundancy gains. Consistent with salience summation, the N2pc latencies are shorter for dimensionally redundant targets compared to targets defined in one dimension only (Töllner, Zehetleitner, Krummenacher, & Müller, 2011). 

In sum, intradimensional feature contrast, dimensional intertrial sequences, symbolic dimension pre-cues, and dimensional target redundancy are known to modulate salience signals in singleton search tasks. The purpose of the present study was to investigate whether the latencies of manual pointing movements to search targets would be modulated by the same manipulations of salience. Onset latencies were examined because these reflect processing prior to the overt pointing movement (Glover, 2004; see also Song & Nakayama, 2006, 2007), though we also analyzed the movement durations. Changes in movement time can stem from the (preparatory) planning stage of movement (Fitts, 1954; Glover, 2004), from processes that control and modulate the movement during its execution (Ma-Wyatt & McKee, 2007), and/or from modulations of movement trajectories (Song & Nakayama, 2006). 

Experiment 1

As pointed out in the Introduction section, salience is assumed to have two functional effects: determining (i) the probability of selecting a location and (ii) the time necessary for the first selection. Varying feature contrast, or target distracter similarity, is the most direct way to manipulate salience, as salience is basically a measure of how much a location differs featurally from its surround. Regarding the probability of selection, there have been many demonstrations that the more similar the target becomes to the distracters (i.e., in terms of features), the larger the rise in search time when increasing the number of items in the display (e.g., Duncan & Humphreys, 1989; Nagy, Sanchez, & Hughes, 1990; Nagy & Sanchez, 1990; Wolfe & Horowitz, 2004). However, it has also been demonstrated that even at high levels of feature contrast at which the target “pops out,” further modulations of feature contrast can still modulate response times (e.g., Verghese & Nakayama, 1994; Zehetleitner, Krummenacher et al., 2009; Zehetleitner, Proulx, & Müller, 2009). That is, for these levels of feature contrast, the slopes of the functions relating search times to the number of display items were effectively zero (i.e., the target “popped out”), but the intercept of the functions was influenced by salience. Hitherto, however, this modulation of intercept (rather than slope) has only been investigated in detection paradigms. Thus, demonstrating such a modulation in another paradigm—in particular, a manual pointing task—will permit strong conclusions to be drawn about the motor unspecificity of salience maps. 

Therefore, in Experiment 1, saliency was modulated by manipulating feature contrast in two dimensions (luminance and orientation), with five levels of feature contrast per dimension. 

Methods

Participants

Twelve observers participated in Experiment 1 (4 males; 2 left-handed; normal or corrected-to-normal vision; median age 25 years) for payment (8 Euro, about $11, per hour). Observers were naive as to the hypotheses of this and all other experiments of the present study. 

Apparatus

Observers viewed the stimuli on an ELO (ELO Touch Systems, Elo Entuitive Systems, Fremont, CA) CRT Open-Frame 17′ touch monitor driven by a personal computer (PC) with Windows XP operating system. The CRT touch screen was mounted into the cut-out surface of a desk, at an angle of approximately 30° to the surface level (Figure 2a). The whole setup was placed in a sound-isolated cabin with dim background lighting. The viewing distance to the center of the monitor was about 50 cm, and observers were instructed to maintain this distance throughout the experiment. Their task was to point as fast as possible to the (feature singleton) target in the search array presented on the monitor, using the index finger of their dominant hand. The monitor had a touch-sensitive layer converting pressure into electrical signals. These signals were transformed by the company driver into mouse clicks. That is, when the finger touched a location on the screen, the driver issued a mouse button-down event to the operating system with spatial coordinates corresponding to the touched location. This mouse click event was captured by the response recording software (purpose written in C++). 

Figure 2

 

(a) Experimental setup for this set of experiments. The touch screen monitor was built into the surface of the desk with a tilt of approximately 30° to the surface. (b) A sample display with all possible target locations being circled for illustration purposes (circles were not present during the experiment). (c) Distracter (uppermost, leftmost bar) and possible target types (other bars). Targets could also be tilted to the left to the same levels of degree as to the right (not shown). The first column of bars illustrates the different types of orientation levels used in the experiments, the second column depicts the different luminance levels, and the third column shows the redundant conditions of Experiment 4.

Stimuli and timing

The display consisted of gray, upright rectangles (bars), each 0.6° of visual angle wide and 2.7° high, that were arranged on three concentric (imaginary) circles around a white fixation point on a black background (0.6 cd/m²). The (invisible) circles were 4.5°, 8.5°, and 12.5° in radius, with six, twelve, and sixteen equidistant item locations, respectively. One target was placed randomly at one of the seven possible positions on the upper half of the middle circle (see Figure 2b); each of these positions was equally to contain a target in a trial block. There were five absolute levels of target orientation contrast relative to the vertical (0°) distracters. At each level, the target could be tilted to the left or right of the vertical, both resulting in the same absolute level of orientation contrast: 5°, 10°, 15°, 20°, and 45°. The target was tilted equally often left- and rightward in a block; the tilt direction was randomized by the stimulus presentation software and not recorded (so that it was not possible to analyze for any differences between left and right tilts). Luminance targets had several levels of brightness (38, 19, 14, 13, or 10 cd/m²), all higher than the distracters (5.9 cd/m²). ² Trials started with the simultaneous onset of all stimuli, which remained visible until the observer responded. The intertrial interval was 900 ms, with a temporal jitter of 200 ms. The first block of trials was used for practice (data not analyzed). 

Design and procedure

Experiment 1 consisted of one session of approximately 45 min. All participants performed a total of 1080 trials in blocks of 60 trials. After pointing to the target, the index finger had to be moved back to the starting position at the bottom (near) end of the screen, which was marked by a gray disk (1.5° in diameter). The next trial would start only after the finger touched this position. After each trial block, participants were informed about their mean total time (i.e., movement initiation latency plus movement duration) and error rate in the just completed block. 

Data analysis

Data analysis was carried out using R (R Development Core Team, 2006). An error was defined when the landing position of the finger deviated by more than 1 degree of visual angle from the center of the target. Initiation latency was defined as the interval between stimulus and movement onset. Movement duration was the interval between movement onset and offset. Total time was the sum of the initiation latency and the movement duration. Trials on which latencies were faster than 150 ms or total times longer than 1800 ms were eliminated as outliers. Less than 2% of the trials were excluded due to response errors or (latency or total) time criteria. Error rates, initiation latencies, movement durations, and total times were examined by repeated-measures analyses of variance (ANOVAs) with the factors target dimension (luminance, orientation) and feature contrast (low, high). 

Results

Initiation latency

The ANOVA of initiation latencies revealed significant main effects of dimension, F(1, 11) = 10.49, p < 0.01, and feature contrast, F(1, 11) = 20.09, p < 0.001, as well as a significant interaction between dimension and feature contrast, F(1, 11) = 19.67, p < 0.01. ³ High-feature-contrast targets were responded to faster than low-contrast targets (407 vs. 440 ms), and orientation-defined targets were responded to faster than luminance-defined targets (417 vs. 429 ms). 

Total time

The ANOVA of total times revealed a similar pattern of effects with significant main effects of dimension, F(1, 11) = 22.99, p < 0.001, and feature contrast, F(1, 11) = 63.33, p < 0.001, as well as a significant interaction between feature contrast and dimension F(1, 11) = 33.61, p < 0.001. Responses took less time with high-feature-contrast targets compared to low-contrast targets (589 ms vs. 650 ms) and less time with orientation targets than with luminance targets (610 vs. 629 ms). Figure 3 presents the three dependent variables (total time, initiation latency, movement duration) in relation to feature contrast. 

Figure 3

 

The three dependent variables (total time, initiation latency, and movement duration) in relation to feature contrast of Experiment 1. Bars indicate standard error of the mean. The five different feature contrast levels for luminance- and orientation-defined targets are plotted and fitted to an exponential function.

Movement duration

The ANOVA of movement duration also revealed both main effects, dimension, F(1, 11) = 16.05, p < 0.01, feature contrast, F(1, 11) = 39.28, p < 0.001, and the interaction between dimension and feature contrast, F(1, 11) = 13.87, p < 0.01, to be significant. Movement durations were shorter with high-feature-contrast targets than with low-contrast targets (182 vs. 211 ms) and shorter with orientation targets than with luminance targets (192 vs. 200 ms). 

Error rates

Error rates were low overall (0.8%). Participants performed significantly better when feature contrast was high rather than low (0.3% vs. 1.1% errors), F(1, 11) = 5.58, p < 0.05. In addition, there was a tendency for performance to be more accurate with orientation-defined targets than with luminance-defined targets (0.6% vs. 0.9% errors), F(1, 11) = 3.49, p < 0.1. There was no interaction between dimension and feature contrast, F(1, 11) = 2.59, p = 0.14. 

Discussion

In summary, for all three components of the movement times (total time, initiation latency, and movement duration) and pointing accuracy, performance was better for high- than for low-feature-contrast targets. That is, an up-modulation of saliency via the degree of feature contrast expedited initiation, execution, total duration, and accuracy of manual pointing movements. Consequently, feature contrast affects visual search performance whether measured by “target-present” (vs. “target-absent”) key presses or by target-directed pointing movements. In all further experiments, (only) two levels of feature contrast for each dimension were introduced along with manipulations of dimension repetition/change (Experiment 2), semantic dimension cueing (Experiment 3), and dimensional redundancy of target definition (Experiment 4). 

However, in Experiment 1 (as well as the subsequent experiments), we only instructed observers to maintain fixation but did not actually monitor eye movements. Thus, it is conceivable that they did make eye movements and these were responsible for the observed salience effects. In order to rule this out, we carried out a control experiment similar to Experiment 1, in which we assured observers' maintenance of fixation by monitoring their eye movements. To do so, for technical reasons, the distance of the touch screen to the observer had to be increased. Similar to Experiment 1 (and the subsequent experiments), participants were instructed to maintain eye fixation while pointing to the target. Eight observers took part in this control experiment. The display was the same as in Experiment 1, except that only target defined by orientation were presented (i.e., there were no luminance-defined targets), with a tilt to the right or the left by 6° or 45° (i.e., high- or low-salience orientation targets). The experiment consisted of 336 trials, presented in blocks of 84 trials. The first block was considered practice and not included in the analysis. Eye movements were recorded at a sampling rate of 1000 Hz by an SR Research Tower-Mount EyeLink 1000 (SR-Research, Mississauga, Ontario, Canada) eye tracker with a chin and forehead rest. Trials on which the amplitude of horizontal or vertical eye shift components exceeded 2.5° from the fixation cross were excluded from further analysis (4.9% of all trials). Only correct trials were analyzed (12% were discarded as error trials). 

An ANOVA of total time revealed the main effects of feature contrast to be significant, F(1, 7) = 13.24, p < 0.01: high-contrast targets were responded to faster than low-contrast targets (710 vs. 733 ms). An analogous ANOVA for initiation latencies also revealed the main effect of feature contrast to be significant, F(1, 7) = 9.16, p < 0.05: movements to high-contrast started earlier than those to low-contrast targets (452 vs. 461 ms). For movement duration, the ANOVA revealed the main effect of feature contrast to be marginally significant, F(1, 7) = 3.62, p < 0.1: durations tended to be shorter for high-contrast than for low-contrast targets (258 vs. 271 ms). This pattern is qualitatively similar to that observed in Experiment 1 (although the effects appear to be somewhat smaller in size). Given this, we are reasonably sure that the skeletal motor effects reported in the present study are unlikely to be confounded by systematic eye movements. 

Experiment 2

Experiment 2 was designed to examine the effect of dimension repetitions versus changes on pointing movements. Based on the assumption that the salience map is motor-unspecific, Experiment 2 was expected to replicate the effects of dimension repetition/change—which have previously been demonstrated in yes/no detection (e.g., Found & Müller, 1996) and perceptual discrimination tasks (e.g., Töllner et al., 2008)—in the latencies of manual pointing movements. The dimension repetition effect (DRE) is the difference in reaction times for trials n on which the target dimension repeated (from trial n − 1) versus trials on which the dimension changed. 

Methods

Participants

Twelve observers participated in Experiment 2 (5 males; all right-handed; normal or corrected-to-normal vision; median age 24 years). Three participants were excluded from the analysis and substituted by new participants because their mean initiation latency was below 100 ms, and one was excluded due to an error rate above 15%. Six observers who took part in Experiment 2 had also participated in Experiment 1. 

Apparatus

The apparatus was the same as in Experiment 1. 

Stimuli and timing

The display was the same as in Experiment 1, except for the targets: Orientation targets could differ from the vertical distracters by a tilt to the left or the right by 5° or 20°. Luminance targets could be more or less bright (21.1 and 10.4 cd/m²) relative to the dark gray distracters (5.9 cd/m²). 

Design and procedure

Experiment 2 consisted of one 45-min session. All participants performed a total of 1140 trials in blocks of 60 trials. Initiation latencies, total movement times, motion durations, and error rates were examined by ANOVAs with the factors feature contrast (low vs. high), dimension (orientation vs. luminance), and intertrial dimension transition (repetition vs. change). 

Results

Initiation latency

The ANOVA of initiation latencies revealed the main effect of feature contrast to be significant, F(1, 11) = 11.89, p < 0.01, and those of dimension, F(1, 11) = 3.92, p < 0.1, and dimension transition, F(1, 11) = 4.44, p < 0.1, to be marginally significant. In addition, the interaction feature contrast × dimension transition was significant, F(1, 11) = 5.69, p < 0.05. High-contrast targets were responded to faster than low-contrast targets (440 vs. 471 ms) and orientation-defined targets faster than luminance-defined targets (453 vs. 459 ms). Planned comparisons revealed the dimension repetition effect to be greater than zero for low-feature-contrast targets (6 ms), t(11) = −2.39, p < 0.05, but not for high-contrast targets (1 ms), t(11) = 0.66, p = 0.53. Figure 4 presents the three dependent variables (total time, initiation latency, movement duration) in relation to the mean DRE for luminance- and orientation-defined targets of high and low feature contrasts. 

Figure 4

 

Mean DRE (i.e., difference of times for trials with dimension repetition and with change) of Experiment 2 for luminance- and orientation-defined targets of high and low feature contrasts for the three dependent variables (total time, initiation latency, and movement duration).

Total time

The ANOVA of total time revealed the main effects of feature contrast, F(1, 11) = 40.91, p < 0.001, and of dimension transition, F(1, 11) = 11.72, p < 0.01, and the interaction feature contrast × dimension transition, F(1, 11) = 8.67, p < 0.05, to be significant. Total movement time was shorter with high- than with low-contrast targets (670 vs. 742 ms) and shorter when the target-defining dimension was the repeated across trials than when it changed (703 vs. 709 ms). Planned comparisons revealed the dimension repetition effects to be greater than zero for low-contrast targets (12 ms), t(11) = −3.88, p < 0.01, but not for high-contrast targets (1 ms), t(11) = 0.30, p = 0.77. 

Movement duration

The ANOVA of movement durations revealed the same pattern of effects as total time: High-contrast targets were associated with shorter movement times than low-contrast targets (230 vs. 270 ms), F(1, 11) = 15.04, p < 0.01. Movement times were shorter with dimension repetitions compared to changes (248 vs. 252 ms), F(1, 11) = 8.59, p < 0.05. The interaction feature contrast × dimension transition was also significant, F(1, 11) = 7.83, p < 0.05, due to dimension repetition effects being greater than zero for low-contrast targets (6 ms) t(11) = −2.86, p < 0.05, but not for high-contrast targets (1 ms), t(11) = −0.85, p = 0.41. 

Error rates

Error rates were low overall (0.9%). The ANOVA of the error rates revealed performance to be more accurate when feature contrast was high rather than low (0.2% vs. 1.6% errors), F(1, 11) = 7.94, p < 0.05. Besides this main effect of feature contrast, the interaction dimension and dimension transition was significant, F(1, 11) = 6.85, p < 0.05. 

Discussion

In summary, for all three components of movement time (total time, initiation latency, and movement duration), performance was better for high- than for low-contrast targets. Dimensional intertrial transition influenced initiation latencies, total time, and movement duration with low-contrast targets (though not with high-contrast targets). Thus, dimension repetition effects were observable in manual pointing movements, as in standard visual search (detection) and compound search tasks. 

Experiment 3

Experiment 3 was designed to examine the effect of a further modulation of salience on visually guided pointing movements: that of semantic (i.e., “top-down”) dimensional pre-cues. Assuming that the salience map is motor-unspecific, one would expect to find dimensional cueing effects in the latencies of manual pointing movements as well as in yes/no detection (e.g., Müller et al., 2003) and perceptual discrimination tasks (e.g., Töllner et al., 2010). 

Methods

Participants

Thirteen observers participated in Experiment 3 (4 males; all right-handed; normal or corrected-to-normal vision; median age 24 years). Four observers who participated in Experiment 3 had already taken part in Experiments 1 and 2. 

Apparatus

The apparatus was the same as in Experiment 1. 

Stimuli and timing

The stimulus display and arrangement was the same as in Experiment 2 (see Figure 2). The only difference to Experiment 2 was the presentation of a semantic cue (i.e., a word) indicating the likely target-defining dimension on the upcoming trial. There were neutral cues, which provided no information about the defining dimension of upcoming target, and dimensional cues, which were 100% predictive as to the target dimension (thus, if the next target was, e.g., orientation-defined, the cue was invariably “ORIENTATION”). In more detail, each trial started with a central white fixation dot presented for 900 ms, followed by a dimensional or neutral cue for 1000 ms, and then again a fixation dot for 900 ms. The orientation and luminance cues—the German words “SCHRAEG” and “HELL” for tilted and bright, respectively—indicated the dimension of the upcoming target with 100% validity; by contrast, a neutral cue (the word “NEUTRAL”) provided no information about the target dimension; 1900 ms after cue onset, the search display appeared. The proportion of dimensional to neutral pre-cues was 80% (40% orientation and 40% luminance cues) to 20%, with the various types of cue being presented randomized within blocks. 

Design and procedure

Experiment 3 consisted of one session of approximately 20 min, with a total of 240 trials presented in blocks of 60 trials. The instruction was essentially the same as in Experiment 2, except that observers were asked to actively set themselves for a target defined in the cued dimension. Initiation latencies, total times, movement durations, and error rates were subjected to ANOVAs with the factors dimension (orientation vs. luminance), feature contrast (low vs. high), and cue validity (neutral vs. valid). 

Results

Figure 5 presents the different components of the movement times as a function of cue validity (valid, neutral). 

Figure 5

 

Mean DCE (i.e., difference of times for trials with neutral and with valid cues) of Experiment 3 for the three dependent measures (total time, initiation latency, and movement duration) for targets defined of high and low feature contrasts.

Initiation latency

The ANOVA of initiation latencies revealed significant main effects of dimension, F(1, 12) = 9.74, p < 0.01, feature contrast, F(1, 12) = 69.79, p < 0.001, and cue validity, F(1, 12) = 10.22, p < 0.01. The interaction feature contrast × cue validity approached significance, F(1, 12) = 3.57, p < 0.1. Orientation-defined targets were responded to faster than luminance-defined targets (570 vs. 595 ms), high-contrast targets faster than low-contrast targets (519 vs. 646 ms), and importantly, validly cued targets were responded to faster than neutrally cued targets (575 vs. 590 ms). The dimension cueing effect tended to be more marked for low- than for high-contrast targets. In fact, as revealed by planned comparisons, the cueing effect was significant (greater than zero) only for low-contrast targets (19 ms), t(11) = 2.21, p < 0.05, but not for high-contrast targets (6 ms), t(11) = 1.07, p = 0.31. 

Total time

The ANOVA of total times also revealed all main effects to be significant: dimension, F(1, 12) = 5.08, p < 0.05, feature contrast, F(1, 12) = 121.66, p < 0.001, and cue validity, F(1, 12) = 8.15, p < 0.05; moreover, the interaction feature contrast × cue validity was significant, F(1, 12) = 5.35, p < 0.05. Total time was shorter with orientation- than with luminance-defined targets (802 vs. 824 ms), with high- than with low-contrast targets (736 vs. 890 ms), and following valid rather than neutral cues (805 vs. 821 ms). Planned comparisons revealed the dimension cueing effect to be greater than zero with low-contrast targets (27 ms), t(11) = 2.40, p < 0.05, but not with high-contrast targets (2 ms), t(11) = 0.41, p = 0.69. 

Movement duration

The ANOVA of movement times revealed only a significant main effect of feature contrast, F(1, 12) = 15.33, p < 0.01. High-contrast targets were associated with shorter movement times than low-contrast targets (217 vs. 245 ms). 

Error rates

Error rates were low overall (1.3%). The ANOVA of error rates revealed more accurate performance when feature contrast was high rather than low (0.5% vs. 2.2% errors), F(1, 12) = 5.39, p < 0.05. There were no further significant effects (all F < 1.5, p > 0.25). 

Discussion

In summary, Experiment 3 revealed faster total movement times and initiation latencies for valid dimensional pre-cues relative to neutral cues. The failure for this (numerical) effect to reach significance for movement duration may have to do with movement duration effects being always smaller than initiation latency effects in the present study. Generally, however, the data of Experiment 3 indicate that dimensional pre-cueing affects manual pointing tasks in a similar way to standard visual search (detection) and compound search tasks. 

Experiment 4

Experiment 4 was designed to examine the salience effect of dimensionally redundant target definition on pointing movements. Participants had to perform the same task as in the previous experiments, except that targets could be defined by feature contrast in both the orientation and the luminance dimension (redundant targets), rather than contrast in just the orientation or the just the luminance dimension (as had been the case in the previous experiments). 

Redundantly defined targets are responded to faster than singly defined targets (redundant signal effect, RSE), for several possible reasons. For instance, Raab (1962) showed that statistical facilitation in an independent parallel race of the two target-defining features present on redundant signal trials would translate into an RSE. That is, if there is a race between two (redundant) signals for triggering a response, with overlapping distributions of triggering times, then one of the two signals is likely to trigger the response faster than the other—thus yielding a mean RSE. Miller (1982) demonstrated that the parallel independent race model of Raab has an upper boundary for how large the RSE can become. This upper boundary was formalized by the so-called race model inequality (RMI): when this inequality is violated, the observed RSE is larger than parallel race models would allow, so that a different model would be required to explain the effect. Miller proposed a co-activation model, that is, both response-relevant signals are integrated before triggering a response. Accordingly, salience summation models (e.g., Bruce & Tsotsos, 2009; Itti & Koch, 2001) are co-activation models: (redundant) feature contrast signals in two dimensions (originating from the same location in the visual array) are summed to yield the attention-guiding salience map signal. Violations of the RMI have been found in several visual search studies (e.g., Koene & Zhaoping, 2007; Krummenacher et al., 2001, 2002; Töllner et al., 2011; Turatto, Mazza, Savazzi, & Marzi, 2004; for a review, see Zehetleitner, Krummenacher, & Müller, 2008). Additionally, Zehetleitner, Krummenacher et al. (2009) found the size of the RSE to interact with feature contrast: the RSE was larger for low- than for high-feature-contrast targets. This finding provides evidence against alternative processing architectures that could theoretically also lead to violations of the RMI (in particular, serial-exhaustive models; see Townsend & Nozawa, 1995). If the modulation of salience by dimensional target redundancy is specific to attention and the oculomotor system, no RSE or at least no violations of the RMI should be observed in visual pointing movements. By contrast, if dimensional target redundancy influences salience regardless of the effector system used for response, RSEs and violations of the RMI should occur also with manual pointing movements. 

Methods

Participants

Twenty-six observers participated in Experiment 4 (11 males; one left-handed; normal or corrected-to-normal vision; median age 24 years). Two participants were excluded from the analysis and substituted by new participants because their initiation latency was below 100 ms. Four observers who participated in Experiment 4 had already taken part in the previous experiments. 

Apparatus

The apparatus was the same as in Experiment 1. 

Stimuli and timing

The stimulus display and arrangement were the same as in Experiment 2. In addition to the four types of single targets already used in Experiments 2 and 3 (orientation- and luminance-defined, each with high or low feature contrast), in Experiment 4 there were four redundant signal conditions (in which the target was both orientation- and luminance-defined): dim–shallow, dim–steep, bright–shallow, and bright–steep. 

Design and procedure

Experiment 4 consisted of two sessions of approximately 45 min each. Each session started with a practice block, which were not included in the analysis. Participants performed a total of 2592 trials in blocks of 72 trials. 

Data analysis

To examine the RSE, RTs for redundantly defined targets were subtracted from the RTs to singly defined targets in the “faster” of the two dimensions. For example, for dim–steep redundant targets, the corresponding single targets were dim and steep, respectively. The RSE was then calculated as the difference in RT between dim–steep redundant targets and the faster of both single targets (i.e., luminance dim and, respectively, orientation steep). Violations of the RMI (Miller, 1982) were examined as proposed by Colonius and Diederich (2006). According to Colonius and Diederich, the RMI can be formalized as d(t) = F(t) − min(1, G1(t) + G2(t)) ≤ 0, where d(t) denotes the Kolmogorov distance between the two distributions F(t) (i.e., the distribution based on the redundant signal RTs) and min(1, G1(t) + G2(t)) (i.e., the distribution corresponding to the maximum possible benefit for redundant signals compared to single signals under the race model assumption). The race model predicts the distance d(t) to be smaller than zero for all times t. To test whether d(t) differs significantly from zero for a particular point in time t, we employed the method of vincentization (e.g., Kiesel, Miller, & Ulrich, 2007; Miller, 1982). Accordingly, the group distribution is calculated by evaluating d(t) for each observer at a defined number of quantiles. In order to avoid an overestimation of RMI violations due to multiple testing (if the RMI is violated at a particular quantile, violations at neighboring points are likely to occur as well; Kiesel et al., 2007; van Zandt, 2002), we tested the RMI within a limited range of quantiles (between 0.05 and 0.20) only (see Kiesel et al., 2007, for details). 

Results

Singly defined targets in Experiment 4 were the same as those in Experiments 2 and 3. As the results for these targets mirror the previously reported data patterns, this part of the analysis will not be reported in detail (for the sake of brevity). The result section is divided into three sections: (i) analysis of redundant target trials, (ii) analysis of the RSE, and (iii) violations of the RMI. Error rates were low overall (0.8%), and an ANOVA revealed only a main effect of target type, F(1, 25) = 18.21, p < 0.001: performance was more accurate when the target was redundantly defined rather than singly defined (0.4% vs. 1.2% errors). 

Redundant target trials

The RTs for redundant signal trials were examined by repeated-measures ANOVA with factors feature contrast of the orientation component of the redundant signal (steep vs. shallow) and feature contrast of the luminance component (dim vs. bright) of the redundant target. For total time, both main effects and the interaction were significant: orientation component, F(1, 25) = 131.07, p < 0.001; luminance component, F(1, 25) = 66.96, p < 0.001; interaction, F(1, 25) = 43.04, p < 0.001. The same was true for initiation latency [orientation component, F(1, 25) = 53.64, p < 0.001; luminance component, F(1, 25) = 27.75, p < 0.001; interaction, F(1, 25) = 9.48, p < 0.01] and total time [orientation component, F(1, 25) = 27.40, p < 0.001; luminance component, F(1, 25) = 24.72, p < 0.001; interaction, F(1, 25) = 21.30, p < 0.001]. Redundant target RTs were faster with high relative to low feature contrast in the orientation component, for total time (615 vs. 640 ms), initiation latency (412 vs. 431 ms), and movement duration (202 vs. 209 ms); the same pattern was evident for the luminance component: total time (620 vs. 635 ms), initiation latency (417 vs. 426 ms), and movement duration (203 vs. 209 ms). ⁴  

RSE

Figure 6 presents the RSEs for all three movement components: total times, initiation latencies, and movement durations. The magnitude of RSE was analyzed by an ANOVA with the factors orientation feature contrast (steep vs. shallow) and luminance feature contrast (bright vs. dim). For total time, both main effects and the interaction were significant: orientation component, F(1, 25) = 63.16, p < 0.001; luminance component, F(1, 25) = 60.34, p < 0.001; interaction, F(1, 25) = 106.05, p < 0.001. The same was true for the initiation latency: orientation component, F(1, 25) = 12.71, p < 0.01; luminance component, F(1, 25) = 12.91, p < 0.01; interaction, F(1, 25) = 35.91, p < 0.001; and for movement duration: orientation component, F(1, 25) = 73.71, p < 0.001; luminance component, F(1, 25) = 58.84, p < 0.001; the interaction, F(1, 25) = 88.18, p < 0.001. For all movement components, the RSE was greater than zero (see Figure 6) in all conditions [all t > 2.18, p < 0.05], except for the condition “steep–dim” in the movement durations [t(25) = 1.45, p = 0.16]. 

Figure 6

 

The size of the RSE (in ms) for the four possible combinations of redundant targets with low (shallow/dim) and high (steep–bright) intensity for orientation and luminance for the three different dependent variables (total time, initiation latency, and movement duration) of Experiment 4.

Violations of the RMI

Significant violations of the RMI were observed only for the total time of the pointing movement. With low feature contrast in both dimensions (6°/dim), the RMI was significantly violated at the 0.05 quantile [t(25) = 2.00, p < 0.05]; and with high contrast in orientation and low contrast in luminance (20°/dim), the RMI was significantly violated in the quantile range between 0.05 and 0.10 [t(25) > 2.02, p < 0.05]. Given these RMI violations, parallel race models can be ruled out in general for target-directed pointing movements to feature singletons. Note that violations of the RMI provide a conservative “indication” of co-activation (e.g., salience summation) effects. In the previous experiments of the present study (as well as in Experiment 4), the effects of the various salience manipulations in total time were the sum of the effects in initiation latency and movement duration. Possibly, violations of the RMI become demonstrable only in this more encompassing measure. 

Discussion

Experiment 4 replicated the results of Zehetleitner, Krummenacher et al. (2009; visual search detection task) for target-directed pointing movements: there were significant redundancy gains, which were larger with low than with high feature contrast. Furthermore, there were significant violations of the RMI and an interaction in the RTs between the two feature contrast components in redundant targets. This pattern indicates that feature contrast signals in the two dimensions were integrated within a co-activation-type functional architecture, rather than being processed independently in parallel or in an exhaustive serial search. Thus, modulations of salience induced by dimensional target redundancy are effective also in visually guided pointing movements. 

General discussion

Motor specificity of salience maps

The present experiments were designed to examine whether modulations of salience known to influence performance in visual search tasks also are effective when observers have to perform a manual pointing movement to the target location. If salience summation models (e.g., Itti & Koch, 2001) not only apply to the guidance of covert attention and overt eye movements, but also to manual pointing movements (see Gottlieb et al., 2009), it can be concluded that salience maps offer a general target selection decision resulting in target coordinates that can be used for attentional selection and programming of ocular and skeletal movements. In general, initiation latencies for pointing movements were slower for low- than for high-feature-contrast targets, for cross-trial changes than for repetitions of the target-defining dimension, for neutral than for valid dimensional pre-cues, and for targets defined in a single dimension rather than redundantly in two dimensions. That is, all the modulations known to affect standard performance measures (simple detection and compound task responses) in pop-out search tasks also influenced the initiation latencies of manual pointing movements. This supports the conclusion that salience map models account not only for the guidance of attention and eye movements but also that of manual pointing movements (see Figure 1). 

Thus far, the discussion has focused on the cognitive concept of a salience map, independently of its neuronal implementation. Concerning the latter, there seems to be no single area in the brain that embodies “the” salience map. Rather, there are several areas that exhibit properties attributed to a salience map, mostly within the oculomotor network (Fecteau & Munoz, 2006) and including the superior colliculus (McPeek & Keller, 2002), the pulvinar thalamus (Robinson & Petersen, 1992), the frontal eye fields (FEF; Sato & Schall, 2003), and the lateral intraparietal sulcus (LIP; Bisley & Goldberg, 2003). 

Neurons in LIP have been reported to signal salience independently of the specific motor response required; for instance, independently of whether a saccade had to be made to the target or a to-be-discriminated target feature had to be indicated by a manual response (Gottlieb et al., 2009). In the latter task, a monkey had to search for a left- or right-facing letter “E” (while maintaining fixation) and to indicate the “orientation” of the target by releasing one of two bars. Despite the search being performed covertly, LIP was strongly active, even though the perceptual decision was mapped onto a manual choice response. 

Similarly, Thompson and Bichot (2005) have recently argued that FEF activity, rather than being tied to eye movements, reflects visual salience in a motor-unspecific manner. Their conclusion was based on three findings: (i) the time required for discriminating between a feature singleton target and a non-target presented in FEF neurons' receptive fields was independent of saccadic reaction time; (ii) FEF activity discriminated between targets and non-targets even in the absence of a saccadic response; and (iii) FEF activity indicated the location of a feature singleton independently of the saccade goal. 

Thus, taken together, the current psychophysical and neurophysiological evidence converges on the view that salience signals are motor-unspecific and can guide target selection decisions independently of whether further action requires overt or covert shifts of attention or visually guided pointing movements. 

In addition to initiation latencies, movement durations were also influenced by target feature contrast, dimensional cross-trial transition, and redundancy of target definition. The effect of salience on movement durations can, in principle, result from two processing stages: movement planning or execution. According to Glover (2004), trajectories of hand movements are first planned and then executed and both processing stages can influence movement durations. For instance, with small movement targets (Fitts, 1954), it is possible to plan a slower but more accurate movement. However, visual feedback can also be used to control and change the movement trajectory during movement execution (Ma-Wyatt & McKee, 2007). Thus, it is possible that low-salience targets give rise to the planning of slower movements (compared to high-salience targets), and/or high-salience targets permit improved online control of the movement and thus faster movement execution. 

Implications for current debates in visual search

The present findings indicate that dimension-based attention, which is assumed to modulate saliency computations (e.g., Müller & Krummenacher, 2006), is also effective in manual pointing tasks—consistent with the notion of an overall salience map guiding task performance in general, rather than being tied to attention or one specific (e.g., the oculo-) motor system. The fact that dimensional (and other salience-based) effects can be seen in attention-based search tasks as well as manual pointing tasks is important, given that several competing accounts of dimension-based effects have been advanced in the literature. 

Two alternatives to the dimension-weighting account (dual-route models and response-based accounts) assume that dimensional repetition/change and cueing effects are specific to detection tasks only. Although dual-route models (e.g., Chan & Hayward, 2009; Mortier, van Zoest, Meeter, & Theeuwes, 2010) and response-based accounts (e.g., Becker, 2008a, 2008b; Cohen & Magen, 1999; Mortier, Theeuwes, & Starreveld, 2005; Theeuwes, Reimann, & Mortier, 2006) differ with regard to the processing stage to which they attribute dimensional effects in detection tasks, both types of account agree in their prediction that dimension-based effects are bound to visual detection tasks exclusively and are prevented to occur (by virtue of the underlying functional architecture) in tasks involving the localization of targets, such as compound search tasks or saccadic or manual localization tasks. Dual-route models assume that the salience map is involved in all tasks except for detection—which is solved via non-spatial detection modules that signal only the presence of feature contrast within a given dimension (e.g., color or orientation) but not the location from which this contrast originates (see also Treisman & Gelade, 1980). Dimension weighting, according to dual-route models, influences only these dimensional modules (i.e., the detection route) but not salience map computations. Consequently, tasks requiring localization of the target should not yield dimension repetition/change or cueing effects. Response-based accounts attribute dimension-based effects to mechanisms involved in response selection, for example, the “dimension-action units” proposed by Cohen and Shoup (2000). Given this, dimension-based effects can become evident if dimension repetitions always co-occur with response repetitions (in which case the same action unit would be used on consecutive trials), as is the case in “target-present/absent” detection paradigms. By contrast, no dimension-based effects should be observed when the target-defining attributes are dissociated from the response-defining attributes, that is, when dimension repetitions/changes occur independently of response repetitions/changes (in which case a repetition of the target dimension would lead to a repetition of the dimension action unit only for response repetitions but not for dimension repetitions associated with a change of response). 

Although both models have already been challenged by findings of dimension repetition (Müller & Krummenacher, 2006; Olivers & Meeter, 2006; Theeuwes et al., 2006; Töllner et al., 2008) and cueing effects (Müller & Krummenacher, 2006; Töllner et al., 2010), the present findings further contribute evidence against dual-route and response-based accounts (regarding the latter: at least as exclusive accounts) of dimensional cueing and repetition effects—because these effects were observed in the initiation latencies of manual pointing movements. If dual-route or response-based models were correct, such effects should not have materialized under the present task conditions. Furthermore, the present data hint at a reason why some studies failed to find dimensional effects in, for example, localization (Chan & Hayward, 2009; Mortier et al., 2010) or saccadic compound tasks (Becker, 2008a, 2008b): for high-contrast targets (as exclusively used in the other, cited studies), dimensional effects were difficult to observe. However, when feature contrast was reduced while maintaining efficient search, such effects were readily seen in the present study. 

The fact that salience-based effects become greater for low- than for high-contrast targets has recently been reported in several studies. Redundancy gains were greater for low- than for high-contrast targets in a go/no-go detection task (Zehetleitner, Krummenacher et al., 2009). A reanalysis of the same study published in Zehetleitner and Müller (2010) also revealed DREs to be substantially increased for low- compared to high-contrast targets. Zehetleitner, Krummenacher, Geyer, Hegenloh, and Müller (2010) reported DREs as well as dimensional cueing effects for low-contrast, but not for high-contrast, targets in a left/right localization task. In summary, salience-based effects of redundancy, dimension repetition, and dimension cueing are greater for low- than for high-contrast targets, irrespective of whether the task is to detect, roughly localize (left/right), or point to the target. To explain this phenomenon, Zehetleitner and Müller (2010) applied the Ratcliff Diffusion Model (RDM; Ratcliff, 1978) logic to visual search decisions. According to RDMs, differences in stimulus quality lead to differences in decision times. These differences become greater the longer the decision takes (see also Ratcliff, Thapar, & McKoon, 2003). Applied to the present experiments, the difference in stimulus quality induced by dimensional redundancy, repetition, and cueing lead to relatively greater modulations in decision times for low- than for high-contrast targets. The dependency of salience-based effects on decision times is further supported by the fact that in Zehetleitner, Krummenacher et al. (2009) redundancy gain and dimension repetition effects (as reanalyzed in Zehetleitner & Müller, 2010) were not only modulated by feature contrast but also by speed–accuracy trade-off: both effects were greater for slow decision times in the accuracy condition than for fast decision times in the speed condition. As to the question why effects of dimensional redundancy, repetition, and cueing are apparent for high-contrast targets in detection, but not in (left/right) localization, pointing, or compound tasks, Zehetleitner and Müller (2010) proposed a computational model that could explain this fact at least for detection and (left/right) localization: they found out that decision times were faster for localization tasks than for detection tasks—and faster decisions lead to smaller salience-based effects. It is thus possible that in the present pointing task, too, target localization decisions were faster than detection decisions, leading to smaller (even undetectable) effects for high-contrast targets. 

Conclusion

In summary, we investigated whether salience maps (e.g., Bruce & Tsotsos, 2009; Itti & Koch, 2001) are motor-unspecific or whether there are motor-specific processing paths for target selection decisions (see Figure 1). To this end, we examined four manipulations that are generally assumed to modulate salience. If salience maps were (motor-) specific to the allocation of spatial attention and/or eye movements, such modulations of salience should leave visually guided pointing movements unaffected. By contrast, if salience maps are general and guide several motor systems, salience-based effects were expected to be seen also in manual pointing responses. We found that initiation latencies of pointing movements were faster for high than for low feature contrast, for cross-trial repetitions rather than changes of the target-defining dimension, for valid rather than neutral (semantic) dimensional pre-cues, and for targets redundantly defined in two dimensions, rather than just one. These findings support the notion that salience maps do not only guide attention and eye movements but also manual pointing movements. 

Acknowledgments

This research was supported by grants from the DFG (Excellence Cluster Cognition for Technical Systems (EC 142)) to HJM and MZ. 

Commercial relationships: none. 

Corresponding author: Michael Zehetleitner. 

Email: [e-mail]. 

Address: Leopoldstr. 13, 80802 München, Germany. 

Footnotes

Footnotes

1  Note, though, that Mazurek et al. (2003) are quite careful, with respect to the saccadic decisions investigated in their study, when they state: “Finally, neither our data nor our model address the question of whether integration actually occurs in LIP or elsewhere, only to be relayed to LIP” (p. 1267). Similarly, Gold and Shadlen (2007) seem to leave the answer open. On the other hand, they state that the common approach, in the relevant studies looking for the neural correlates of a decision, of “focus[ing] on parts of the brain known to select and prepare the associated movement … leaves open the question of how and where the brain forms decisions that are not used to select a particular movement” (p. 562). This would appear to tally with the reading of Gottlieb et al. (2009) and shows at least that the two alternatives are open in the neurophysiological literature.

Footnotes

2  In a control experiment, we verified that the low-feature-contrast targets were found “efficiently,” that is, the 95% confidence intervals of the search slopes (i.e., the slopes of the functions relating RT to the number of possible target locations in the search array, either 7 or 19) in a go/no-go detection task ranged from −4.6 to 3.0 ms/item.

Footnotes

3  The interactions (in this as well as the subsequent experiments) between dimension and feature contrast were due to the selected levels of feature contrast in both dimensions: the five (or, respectively, two) levels of feature contrast in orientation did not directly match the five (or two) levels in luminance.

Footnotes

4  The significant interaction between the two levels of contrast in the redundant signal paradigm indicates that the underlying processing architecture cannot be based on serial checking of both dimensions (Townsend & Nozawa, 1995; see Zehetleitner, Krummenacher et al., 2009, for an elaboration of this argument).