Feature-based attentional integration of color and visual motion

Abstract

In four variants of a speeded target detection task, we investigated the processing of color and motion signals in the human visual system. Participants were required to attend to both a particular color and direction of motion in moving random dot patterns (RDPs) and to report the appearance of the designated targets. Throughout, reaction times (RTs) to simultaneous presentations of color and direction targets were too fast to be reconciled with models proposing separate and independent processing of such stimulus dimensions. Thus, the data provide behavioral evidence for an integration of color and motion signals. This integration occurred even across superimposed surfaces in a transparent motion stimulus and also across spatial locations, arguing against object- and location-based accounts of attentional selection in such a task. Overall, the pattern of results can be best explained by feature-based mechanisms of visual attention.

Introduction

Functional specialization is one of the hallmarks of the primate visual cortex. Different attributes of a visual stimulus, like motion, depth, form, and color, are known to be processed in separate areas or even pathways of the visual cortex (Felleman & Van Essen, 1991). A number of such specialized cortical areas have been identified, and they seem to form at least two processing streams (Ungerleider & Mishkin, 1982). The ventral pathway (mainly involving areas V1, V2, V4, TEO, and IT) shows specialization for the processing of color and shape. In contrast, areas in the dorsal pathway (V1, V2, V3, MT/MST) analyze information about motion and spatial relations. This notion of spatially separate and functionally independent, parallel processing streams represents an important conceptualization of visual information processing. However, the functional separation is far from complete (Ferrera, Nealey, & Maunsell, 1992), and a large number of anatomical connections between these two pathways have been demonstrated, providing a neural substrate for interactions. 

In reaction time (RT) research, the combined processing of separable sensory signals has been studied with the redundant-target paradigm (Miller, 1982, 1986; Mordkoff & Yantis, 1993). In a typical experimental situation, two different sensory signals are defined as targets and participants are required to make speeded responses if either of the two targets is detected. Of special interest is a condition in which both targets are presented simultaneously (redundant-target trials). This condition is then compared with those in which either of the targets is presented alone (single-target trials). It is typically found that RTs to redundant targets are faster than RTs to single targets, and this finding is commonly referred to as redundancy gain (Giray & Ulrich, 1993; Iacoboni & Zaidel, 2003; Krummenacher, Müller, & Heller, 2001, 2002; Miller, 1982, 1986, 2004; Miller, Ulrich, & Lamarre, 2001; Miniussi, Girelli, & Marzi, 1998; Mordkoff, Miller, & Roch, 1996; Mordkoff & Yantis, 1991, 1993; Turatto, Mazza, Savazzi, & Marzi, 2004). 

Essentially, two classes of models have been advanced to explain this redundancy gain. The first class consists of race models. They are based on the idea that the two concurrent sensory signals are processed separately and independently and that responses can be initiated as soon as one of the two signals is detected. Sensory information for the two signals is not combined to initiate a response. Responses to redundant targets are particularly fast because they are produced by the faster of the two detection processes. The term race model illustrates the fact that responses are thought to be initiated by the winner of a race between the two separate detection processes. If one assumes that processing time randomly varies from trial to trial, and that the distributions of processing times for the two signals overlap, it follows that, on average, the time needed by the winner will be less than the time needed by either racer. For this reason, race models have also been characterized as producing statistical facilitation (Raab, 1962). The second class of models consists of coactivation models (Miller, 1982). In contrast to race models, sensory information for the two signals is combined to reach some decision criterion based on which responses are initiated. Here, responses to redundant targets are particularly fast because such a criterion will be reached faster if two processes combine in satisfying it. The term coactivation reflects the fact that two processes combine in activating a response. 

The race-model inequality

Miller (1982) has provided a formal test to decide between these two classes of models with experimental RT data. In brief, he showed that all race models have to fulfill the following inequality: 

$P(\mathrm{R}\mathrm{T}<t|S_1\mathrm{a}\mathrm{n}\mathrm{d}{S}_2)\le P(\mathrm{R}\mathrm{T}<t\left|S_1\right)x+P(\mathrm{R}\mathrm{T}<t|S_2),$

(1)

where t is the time needed to respond to a signal and S₁ and S₂ are the two targets. Intuitively, this inequality formalizes an implicit constraint applying to all variants of race models: Responses to redundant targets (S₁ and S₂, presented together) cannot be faster than the fastest response to either of the single targets (S₁ or S₂, presented alone) of the single targets. Note that the three terms in this inequality represent the cumulative probability density function (CDF) of RT on redundant- and single-target trials, respectively. The CDFs obtained in a speeded detection task just need to be evaluated at the different values of t to decide between race and coactivation models: If the inequality is violated for any of them, all variants of race models can be rejected. 

In this study, we apply this logic to the processing of color and motion signals. We consider performance that is consistent with predictions of race models as evidence for separate and independent processing of these two stimulus dimensions. Conversely, performance inconsistent with predictions of race models would argue against separate and independent processing and rather support cross-dimensional integration of color and motion signals. 

In Experiment 1, we establish that redundancy gains can be observed for the stimulus dimensions color and direction of motion. We further demonstrate that explanations based on race models can safely be rejected, favoring cross-dimensional integration of color and motion signals in speeded target detection tasks. In three subsequent experiments, we investigate in more detail whether the observed integration depends on specific stimulus attributes or task demands. Throughout, we find robust redundancy gains, as well as strong evidence for cross-dimensional processing of color and motion signals. 

Experiments

Experiment 1

The purpose of Experiment 1 was to investigate whether redundancy gains can be observed for targets defined by a change in color or in the direction of motion. We used moving random dot patterns (RDPs) in a go/no-go target detection task, requiring speeded responses if a particular color (single color target), a particular direction of motion (single direction target), or both (redundant target) were presented and no response if the stimulus did not contain any of the designated targets. 

Methods

The stimulus (Figure 1a) was composed of dots moving within a virtual circular aperture with a diameter of 5 deg. The dot density was 2 dots/deg² of visual angle. Each dot subtended 0.1 deg of visual angle. The RDP was plotted against a black background and centered on a yellow fixation point that was presented in the middle of the screen. Each trial started with an RDP that rotated around the fixation point at an angular speed of 2.9 deg/s; all the dots were gray. In case of a color change, the dots changed to red, green, or blue. The four colors were equiluminant (25 cd/m²). In case of a direction change, 70% of all dots started to translate into one of three linear motion directions: either upward, to the left, or to the right. The remaining 30% of the dots moved in random directions. This was done to prevent participants from solving the task by tracking a single dot. The stimulus was presented on a VGA monitor (Lacie, Electron22 Blue IV) operating at a refresh rate of 85 Hz and a resolution of 80 pixels/deg of visual angle. Presentation of the stimulus and recording of the responses were controlled by an Apple Power Mac G4 computer. 

Figure 1

 

Stimuli used in Experiments 1–4. (a) Experiment 1. A single RDP rotating around the fixation square. (b) Experiment 2. Two superimposed RDPs rotating against each other. (c) Experiments 3 and 4. Two rotating RDPs centered 3.75 and 7.5 deg above and below fixation in Experiments 3 and 4, respectively.

At the beginning of an experiment, participants were told which color and direction of motion were defined as targets. They were instructed to respond with a keypress on a computer keyboard (“H”) if the target color, the target direction, or both were presented (go trials) and to withhold their response if no target was presented (no-go trials). The participants were told to respond as quickly as possible without making too many errors. The trial sequence is illustrated in Figure 2. A stationary dot pattern was presented, together with the fixation point, until participants triggered the beginning of a trial by pressing the space bar on the keyboard. Following this, the RDP rotated around the fixation point for 1,000–1,500 ms. Then, one of the following events occurred: a change in color, a change in direction of motion, or both. The changes lasted for 175 ms before the original stimulus properties were restored. Following the participants' response, the stimulus was removed. If no response had been given within 1,000 ms after onset of the event, the trial was terminated and the response was considered a no-go. Auditory feedback was given at the end of each trial. The participants triggered the beginning of the next trial when they were ready to proceed. 

Figure 2

 

Trial sequence. A stationary pattern of white dots was present at the beginning of each trial. It then rotated around the fixation square for 1,000–1,500 ms before changing its color, direction of motion, or both. After another period of 175 ms, the RDP changed back to its initial rotation and color.

Any redundancy gain would result in shorter RTs to redundant targets as compared to single targets. This, however, would be a comparison between a condition with two sensory events (redundant-target trials, with simultaneous changes in direction of motion and color) against a condition with only a single sensory event (single-target trials, with changes either in direction of motion or in color). Because two sensory events represent a much stronger signal compared with a single sensory event, this difference by itself could speed RTs in the redundant-target condition. To make sure that potential redundancy gains do not result from a difference in the number of sensory events per se, we introduced single-target control conditions having two sensory events. In these control conditions, the single targets were combined with neutral events in the other stimulus dimension (i.e., a single color target was combined with a neutral change in direction, and a single direction target was combined with a neutral change in color). A neutral event is “neutral” in the sense that it appeared equally often in combination with go signals as well as with no-go signals. Consequently, the appearance of a neutral event was uninformative as to whether the participant should respond, whereas the appearance of a no-go event always signaled to withhold the response. Experiment 1 was divided into six blocks of 100 trials each. In a single block, there were 10 signal conditions (5 go and 5 no-go conditions, listed in that order): redundant targets, single color targets, single direction targets (the latter two will be referred to as “single target alone”), single color targets combined with neutral direction changes, single direction targets combined with neutral color changes (“single target + neutral”), redundant no-gos, single color no-gos, single direction no-gos, single color no-gos combined with neutral direction changes, and single direction no-gos combined with neutral color changes. Overall, there were 50% go and 50% no-go trials. The 10 signal conditions were presented in a pseudorandomized order until 10 correct responses had occurred in each condition. Within each condition, the initial rotation of the RDP was clockwise for five trials and counterclockwise for the remaining five trials. Between blocks, participants were given a break of 5 min. The assignment of three particular colors to go, no-go, and neutral conditions was counterbalanced across participants and remained unchanged throughout the entire experiment. To achieve the same level of difficulty for the go and no-go motion tasks, we always treated upward motion as the neutral direction for all participants. For half of the participants, rightward motion represented the go direction, whereas leftward motion represented the no-go direction; for the other half, this assignment was reversed. Table 1 summarizes one such combination of go, no-go, and neutral events forming the 10 signal conditions. 

Table 1

 

Experiment 1. Combination of go, no-go, and neutral events for participants instructed to detect rightward motion (direction target) or the color red (color target). The color green and leftward motion did not require a response. In the “single + neutral” condition, single-color signals were accompanied by neutral direction changes (upward motion), whereas single-direction signals were accompanied by neutral color changes (blue).

	Go signals			No-go signals
Redundant	Red + right			Green + left
	Go signals			No-go signals
		Color		Direction		Color	Direction
Single alone	Red		Right		Green	Left
Single + neutral	Red + up		Right + blue		Green + up	Left + blue

Participants

Each participant was tested in two sessions that were performed on separate days. A single session was divided into three blocks of approximately 10 min each. Each participant was instructed to respond to a particular color and a particular direction of motion at the beginning of the experiment. During the first session, participants completed a practice block to become familiar with the task. These practice blocks were not included in the analyses. Twelve participants (four men and eight women; ages 22–36, M = 27.1, SD = 5.3) were tested in Experiment 1 and were paid for their participation. All had normal or corrected-to-normal vision. They gave informed written consent and were naive as to the purpose of the experiment. All sessions were conducted in a dimly illuminated, quiet testing booth. Participants were comfortably seated with their head resting in a chin and forehead rest at a distance of 114 cm from the computer monitor. 

Data analyses

Two analyses were performed. The first analysis addressed the redundancy gain and examined whether RTs to redundant targets were faster than RTs to single targets. Mean RTs were determined for each combination of participant and signal condition (redundant target, single color alone, single direction alone, single color + neutral direction, single direction + neutral color). For each participant, we selected the faster of the two mean RTs from the “single target alone” conditions (“fastest single alone”). Analogously, we determined the faster of the two mean RTs in the “single target + neutral” condition (“fastest single + neutral”; see Miller, 1982). Statistical significance was evaluated with a one-way ANOVA involving the within-subjects factor of target type (redundant target vs. fastest single alone vs. fastest single + neutral). 

The second analysis compared RT distributions between conditions to test for violations of the race-model inequality. Following Miller (1982), we compared the sum of the CDFs for the single-target conditions to the CDF for the redundant-target condition. First, we determined the CDFs for each combination of participant, block, and signal condition (redundant target, single color alone, single direction alone, single color + neutral direction, single direction + neutral color) by rank ordering the 10 RTs, which then represent an estimate of the unknown, true CDF at 10 percentiles (.05–.95). We then computed the sum of the CDFs in the “single target alone” condition (“sum of single alone”) and, analogously, the sum of the CDFs in the “single target + neutral” condition (“sum of single + neutral”). The obtained CDFs were then averaged across blocks for a given participant and finally across participants. To test for violations of the race-model inequality, we compared the CDF for the redundant-target condition to the sum of the CDFs for the “single target alone” condition. Analogously, we compared the CDF for the redundant-target condition to the sum of the CDFs for the “single target + neutral” condition. The race-model inequality would be violated if RTs from the redundant-target CDF were faster than corresponding RTs from the summed CDF at any of the percentiles. Statistical significance was evaluated by conducting paired t tests across participants at each of the 10 percentile pairs (redundant vs. “sum of single”). 

Results and discussion

Redundancy gain

Across participants, performance reached 89% correct for the no-go trials and varied between 99% and 100% correct for the five go conditions. Mean RTs across participants are shown in Figure 3. The ANOVA revealed a highly significant difference between mean RTs, F(2, 22) = 33.73, MSE = 126.39, p < .001, Greenhouse–Geyser corrected. Post hoc analyses confirmed that RTs for redundant targets (328 ms) were faster than RTs for both “fastest single alone” (358 ms, p < .001) and “fastest single + neutral” (363 ms, p < .001). However, RTs for “fastest single alone” were not different from RTs for “fastest single + neutral” (p = .28). A 95% confidence interval for the main effect of target type indicates that redundancy gains are in the range of 26–40 ms under the present conditions. 

Figure 3

 

Experiment 1. RTs as a function of target type (redundant target vs. fastest single alone vs. fastest single + neutral). RTs to redundant targets are lower than RTs in the other two conditions (p < .001 for both comparisons). There is no statistical difference between fastest single alone and fastest single + neutral. Error bars indicate the 95% confidence interval for the main effect of target type (see Loftus & Masson, 1994).

Race-model inequality

Mean CDFs across participants are shown in Figure 4. Comparing the redundant-target CDF (pentagrams) to the sum of single alone CDF (circles) reveals a clear violation of the race-model inequality because the redundant-target CDF lies above and to the left of the “sum of single alone” CDF (upper panel in Figure 4). 

Figure 4

 

Experiment 1. Violations of the race-model inequality are revealed by comparing the redundant-target CDF (pentagrams) with the sum of the single-target CDFs (circles). Inconsistent with predictions of all race models, the redundant-target CDF lies above and to the left of the CDF for the sum of the single targets. This is the case for both sum of single alone (upper panel) and sum of single + neutral (lower panel). It shows that the fastest response to redundant targets can be faster than the fastest responses to single targets. Significance markers along the Y-axis indicate the percentiles in which RTs to redundant targets were reliably faster than corresponding RTs from the sum of the single CDF curve (.001 < p < .02).

Paired t tests across participants at each of the 10 percentile points confirmed that RTs from the redundant signal CDF were reliably faster than RTs from the “sum of single alone” CDF at the 5th, 15th, 25th, and 35th percentiles (.001 < p < .02). This shows that the fastest responses to redundant targets can in fact be faster than the fastest response to single targets, which is inconsistent with predictions of all race models. Analogously, we compared the redundant-target CDF to the sum of the CDFs in the “single target + neutral” condition to test for a violation of the race-model inequality when single targets are combined with neutral events (lower panel in Figure 4). As is apparent from this figure, the race-model inequality was again considerably violated. RTs were reliably faster for the redundant-target CDF at percentiles 0.05, 0.15, 0.25, 0.35, 0.45, 0.55, and 0.65 (.001 < p < .005). 

The results argue against separate and independent processing of color and motion signals in a speeded target detection task with moving RDPs. We interpret this finding as evidence for cross-dimensional integration of color and motion information. In Experiment 1, neither the examination of redundancy gains nor the test for violations of the race-model inequality revealed any difference depending on whether we used single signals alone or single signals combined with neutral events. This finding is inconsistent with the idea that redundancy gains in the “single target + neutral” condition are simply due to response interference or cognitive inhibition effects. For instance, one could conceive that participants internally define the stimuli such that the designated color and direction were labeled as “go signals” and all others were considered “no-gos.” Resolving this conflicting information might have slowed down responses in the “single target + neutral” condition, thereby artificially producing a redundant-target effect (RTE), because redundant targets never contained conflicting information. However, because there is no difference between responses to single targets and single targets combined with neutral events (Figure 3), we can safely reject this possibility. Having demonstrated that adding a neutral signal to a single target has no inhibitory effect, we did not include the “single target alone” condition in all the succeeding experiments (i.e., we only used single targets that were combined with neutral events). 

Experiment 2

The purpose of Experiment 2 was to investigate whether the observed cross-dimensional integration of color and motion signals requires that the two target signals belong to the same object. We rotated two populations of dots against each other, resulting in the percept of two superimposed surfaces (i.e., objects). This enabled us to present redundant targets always at the same location but either on the same surface or on different surfaces. If color and motion signals are integrated only if they belong to the same object, then we should observe violations of the race-model inequality only if redundant targets are presented on the same surface but not if they are presented on different surfaces. 

Methods

The basic go/no-go target detection paradigm was identical to Experiment 1 except for the following changes. First, two RDPs were rotated against each other (Figure 1b). Second, because of superimposing two RDPs, there were twice as many dots in the transparent motion stimulus (4 dots/deg² of visual angle) as compared with the single surface used in Experiment 1. Third, single targets in a given stimulus dimension were always combined with neutral events in the other dimension. Fourth, six blocks consisting of 120 trials each were run on two consecutive days. In a single block, there were 12 signal conditions: 6 go conditions comprising redundant targets with both signals in the same surface or in different surfaces, single color target + neutral direction event in the same surface or in different surfaces, and single direction target + neutral color event in the same surface or in different surfaces and also 6 no-go conditions comprising redundant no-gos in the same surface or in different surfaces, single color no-go + neutral direction event in the same surface or in different surfaces, and single direction no-go + neutral color event in the same surface or in different surfaces. See Table 2 for an overview of a possible combination of go, no-go, and neutral events in Experiment 2. Twelve participants (five men and seven women; ages 20–33, M = 24.9, SD = 3.4) participated in Experiment 2. They were informed that the changes in color and direction of motion could either occur in the same surface or in different surfaces. 

Table 2

 

Experiment 2. One possible combination of go-, no-go, and neutral events for participants instructed to detect rightward motion (direction target) or the color red (color target). Single-color signals were always accompanied by a neutral change in direction (upward motion); single-direction targets were always presented together with a neutral change in color (blue). All events occurred either on the same surface or on different surfaces.

		Go signals			No-go signals
		Redundant	Single color	Single direction	Redundant	Single color	Single direction
Same surface	Surface 1	Red + right	Red + up	Right + blue	Green + left	Green + up	Left + blue
	Surface 2	No change	No change	No change	No change	No change	No change
Different surfaces	Surface 1	Red	Red	Right	Green	Green	Left
	Surface 2	Right	Up	Blue	Left	Up	Blue

Data analyses

As before, redundancy gains were examined by comparing RTs to redundant targets with the fastest RTs to single targets. Mean RTs were computed for each combination of participant, target type (redundant target vs. single target), and target surface (same vs. different). The average of the faster of the two single-target conditions was calculated across participants, separately for each target surface condition. Statistical significance was evaluated with a two-way ANOVA involving the within-subjects factors target type (redundant target vs. “fastest single target”) and target surface (same vs. different). Violations of the race-model inequality were evaluated as in Experiment 1. 

Results and discussion

Redundancy gain

Across participants, performance reached 92% correct for the no-go trials and varied between 98% and 100% correct for the five go-conditions. Mean RTs across participants are shown in Figure 5. The ANOVA revealed a highly significant main effect of target type, F(1, 11) = 53.68, MSE = 192.67, p < .001, indicating that responses to redundant targets were reliably faster than responses to the single targets (326 vs. 355 ms). A 95% confidence interval for the main effect of target type indicates that redundancy gains are in the range of 20–38 ms. Most importantly, neither the main effect of target surface (p = .17) nor the interaction between target type and target surface (p = .87) reached significance. This clearly shows that there is no difference in redundancy gains between changes occurring in the same surface (29 ms) and changes occurring in different surfaces (30 ms) in a transparent motion stimulus. 

Figure 5

 

Experiment 2. RTs as a function of target type (redundant target vs. fastest single target) and target surface (same surface vs. different surfaces). RTs to redundant targets are faster than RTs to single targets. This effect does not depend on whether redundant targets are presented on the same surface or on different surfaces. Error bars indicate the 95% confidence interval for the main effect of target type (see Loftus & Masson, 1994).

Race-model inequality

Mean CDFs across participants are shown in Figure 6. The upper panel represents conditions in which changes in color, direction of motion, or both occur in the same surface of a transparent motion stimulus. Contrasting the redundant-target CDF (pentagrams) with the “sum of single targets” CDF (circles) reveals a clear violation of the race-model inequality because the redundant-target CDF lies above and to the left of the “sum of single targets” CDF. Paired t tests across participants at each of the 10 percentile pairs confirmed that RTs from the redundant-target CDF were reliably faster than RTs from the “sum of single targets” CDF between the 5th and 55th percentiles (.001 < p < .002). Analogously, we compared the redundant-target CDF to the “sum of single targets” CDF in the condition where changes in color, direction of motion, or both occurred in different surfaces (lower panel in Figure 6). The race-model inequality was again considerably violated. RTs were reliably faster for the redundant-target CDF between percentiles 0.05 and 0.35 (.01 < p < .02). 

Figure 6

 

Experiment 2. Violations of the race-model inequality are revealed by comparing the redundant-target CDF (pentagrams) with the sum of the single-target CDFs (circles). Inconsistent with predictions of all race models, the redundant-target CDF lies above and to the left of the CDF for the sum of the single targets. This is the case not only if redundant targets occur on the same surface (upper panel) but also if they appear on different surfaces (lower panel). Significance markers along the Y-axis indicate the percentiles in which RTs to redundant targets were reliably faster than corresponding RTs from the sum of the single CDF curves (.01 < p < .02).

The results of Experiment 2 show that color and motion signals are integrated, even if they occur in different surfaces of a transparent motion paradigm. This is evident from the fact that the magnitude of redundancy gains does not depend on whether two target signals occur in the same surface or in different surfaces. In addition, violations of the race-model inequality are prominent in both situations, suggesting integration of color and motion information across overlapping surface borders. Alternatively, one might argue that these two signals are integrated simply because they occur at the same location. Next, we tested whether integration of color and motion signals persists even if the two surfaces are positioned at different spatial locations. 

Experiment 3

The purpose of Experiment 3 was to investigate whether the integration of color and motion signals across surfaces requires that the two surfaces are spatially overlapping. Instead of superimposing the two RDPs, we separated them spatially (Figure 1). If integration is restricted to a common spatial location, we should observe violations of the race-model inequality only if the target signals are presented at the same location but not if they are presented at different locations. 

Methods

The basic go/no-go target detection paradigm was identical to Experiment 2 except for the following changes. The two RDPs were presented at spatially separate locations, centered 3.75 deg above and below fixation. Each of the 12 signal conditions was presented 12 times (6 go conditions: redundant targets with both signals in the same location or in different locations, single color target + neutral direction event in the same location or in different locations, and single direction target + neutral color event in the same location or in different locations; 6 no-go conditions: redundant no-go in the same location or in different locations, single color no-go + neutral direction event in the same location or in different locations, and single direction no-go + neutral color event in the same location or in different locations). Within each condition, three trials were used for each possible combination of the upper RDP's initial direction of rotation (clockwise vs. counterclockwise) and the location of the target event (upper vs. lower RDP). See Table 3 for a possible combination of go, no-go, and neutral events. Twelve participants (six men and six women; ages 20–27, M = 23, SD = 2.6) were tested in Experiment 3. They were instructed to fixate on the central fixation square during the trials. Throughout each session, the experimenter monitored eye fixation with an infrared camera connected to a monitor outside the testing booth. 

Table 3

 

Experiment 3. One possible combination of go-, no-go, and neutral events for participants instructed to detect rightward motion (direction target) or the color red (color target).

		Go signals				No-go signals
		Redundant	Single color	Single direction	Redundant	Single color	Single direction
Same location	Location 1	Red + right	Red + up	Right + blue	Green + left	Green + up	Left + blue
	Location 2	No change	No change	No change	No change	No change	No change
Different locations	Location 1	Red	Red	Right	Green	Green	Left
	Location 2	Right	Up	Blue	Left	Up	Blue

Data analyses

Redundancy gains were examined as before. Mean RTs were determined for each combination of participant, target type (redundant target vs. single target), and target location (same vs. different). For each participant, the faster of the two mean RTs in the single-target conditions was selected (“fastest single same” and “fastest single different,” respectively). Statistical significance was evaluated with a two-way ANOVA involving the within-subjects factors target type (redundant target vs. fastest single target) and target location (same vs. different). 

Results and discussion

Redundancy gain

Across participants, performance reached 97% correct for the no-go trials and varied between 96% and 100% correct for the five go-conditions. Mean RTs across participants are shown in Figure 7. The ANOVA revealed a highly significant main effect of target type, F(1, 11) = 125.74, MSE = 165.08, p < .001, indicating that responses to redundant targets (372 ms) were reliably faster than responses to the single targets (414 ms). A 95% confidence interval for the main effect of target type indicates that redundancy gains are in the range of 34–50 ms. The main effect of target location also reached significance, F(1, 11) = 7.15, MSE = 118.51, p < .05, indicating that responses to targets presented at the same location (389 ms) were faster than to targets presented at different locations (397 ms). Most important, however, was the absence of an interaction between target type and target location (p = .8). This clearly shows that redundancy gains do not depend on whether the target events are presented at the same location (41 ms) or at different locations (42 ms). 

Figure 7

 

Experiment 3. RTs as a function of target type and target location. RTs to redundant targets are faster than RTs to single targets. RTs were also faster for events occurring at the same location than those occurring at different locations. The absence of a significant interaction shows that the RTE does not depend on whether redundant targets are presented at the same location or at different locations. Error bars indicate the 95% confidence interval for the main effect of target type (see Loftus & Masson, 1994).

Race-model inequality

Mean CDFs across participants are shown in Figure 8. The upper panel represents conditions in which changes in color, direction of motion, or both occur at the same location. Paired t tests across participants at each of the 10 percentile pairs confirmed that RTs from the redundant-target CDF (pentagrams) were reliably faster than RTs from the “sum of single targets” CDF (circles) everywhere between the 5th and 55th percentiles (.001 < p < .05). The same comparison for the condition in which the target events occurred at different locations (lower panel in Figure 7) also revealed faster RTs for the redundant-target CDF everywhere between percentiles 0.05 and 0.65 (.001 < p < .05). Hence, in both cases, strong violations of the race-model inequality were observed. 

Figure 8

 

Experiment 3. Violations of the race-model inequality are revealed by comparing the redundant-target CDF (pentagrams) with the sum of the single-target CDFs (circles). Inconsistent with predictions of all race models, the redundant-target CDF lies above and to the left of the CDF for the sum of the single targets. This is evident irrespective of whether redundant targets occur at the same location (upper panel) or at different locations (lower panel). Significance markers along the Y-axis indicate the 10 percentile points in which RTs were reliably faster for redundant targets than for the sum of single targets (.001 < p < .05).

These results show that redundancy gains and violations of the race-model inequality are prominent even if the two target events occur at separate spatial locations. It provides evidence for integration of color and motion signals across the visual field. To further assess the generality of this conclusion, we next tested whether the observed effects depend on the extent of spatial separation between the two stimuli. 

Experiment 4

The purpose of Experiment 4 was to investigate whether the cross-dimensional integration observed in Experiment 3 is affected by the spatial distance between two RDPs. 

Methods

Methods and design were identical to Experiment 3 except for the spatial distance between the two RDPs, which were centered 7.5 deg above and below fixation here (Figure 1c). To achieve this, we reduced the distance between participant and monitor to 57 cm, resulting in a monitor resolution of 40 pixels/deg of visual angle. All stimulus properties were adjusted to ensure identity of the retinal image. Twelve participants (five men and seven women; ages 22–32, M = 24.2, SD = 2.9) were tested in Experiment 4. Redundancy gains and violations of the race-model inequality were evaluated as in the preceding experiments. 

Results and discussion

Redundancy gain

Across participants, performance reached 97% correct for the no-go trials and varied between 92% and 100% correct for the five go-conditions. Mean RTs across participants are shown in Figure 9. 

Figure 9

 

Experiment 4. RTs as a function of target type and target location. RTs to redundant targets are faster than RTs to single targets. RTs were also faster for events occurring in the same location than those occurring in different locations. The absence of a significant interaction shows that the RTE does not depend on whether redundant targets are presented in the same location or in different locations. Error bars indicate the 95% confidence interval for the main effect of target type (see Loftus & Masson, 1994).

The ANOVA revealed a highly significant main effect of target type, F(1, 11) = 159.18, MSE = 100.50, p < .001, indicating that responses to redundant targets were reliably faster than responses to the single targets (412 vs. 448 ms). A 95% confidence interval for the main effect of target type indicates that redundancy gains are in the range of 30–42 ms. The main effect of target presentation also reached significance, F(1, 11) = 14.58, MSE = 178.27, p < .01, indicating that responses to targets presented in the same location were faster than to targets presented in different locations (423 vs. 438 ms). However, as in Experiment 3, the interaction between target type and target location did not reach significance (p = .14), indicating that there is no difference in redundancy gains between changes in the same location (38 ms) and changes in different locations (31 ms). It clearly shows that the magnitude of the redundancy gains is not affected by the spatial distance between the two stimuli. This conclusion is further supported by Miller (1982), who also reported the absence of distance effects in a different variant of the redundant-target paradigm. 

Race-model inequality

Mean CDFs across participants are shown in Figure 10. Conditions in which changes in color, direction of motion, or both occur at the same location are shown in the upper panel. Paired t tests across participants at each of the 10 percentile pairs confirmed that RTs from the redundant-target CDF (pentagrams) were reliably faster than RTs from the “sum of single targets” CDF (circles) everywhere between the 5th and 65th percentiles (.001 < p < .008). The lower panel of Figure 10 shows the condition in which the target events occurred at different locations. RTs for the redundant-target CDF were significantly faster between percentiles 0.05 and 0.25 (.001 < p < .01), with the difference at the 35th percentile just closely failing to reach statistical significance (p = .057). Hence, for targets at the same and at distant locations, strong violations of the race-model inequality were observed. 

Figure 10

 

Experiment 4. Violations of the race-model inequality revealed by comparing the redundant-target CDF (pentagrams) with the sum of the single-target CDFs (circles). Inconsistent with predictions of all race models, the redundant-target CDF lies above and to the left of the CDF for the sum of the single targets. This is evident irrespective of whether redundant targets occur at the same location (upper panel) or at different locations (lower panel). It shows that the fastest response to redundant targets can be faster than the fastest responses to single targets. Significance markers along the Y-axis indicate the 10 percentile points in which RTs were reliably faster for redundant targets than for the sum of single targets (.001 < p < .05).

The outcome of Experiment 4 further supports the notion that color and motion information is integrated across stimuli presented at different locations in the visual field. Even when the stimuli are separated by 15 deg (center-to-center distance), strong redundancy gains and reliable violations of the race-model inequality are observed. 

General discussion

In four experiments, we investigated the cross-dimensional integration of color and visual motion signals by using the redundant-target paradigm. We presented moving RDPs in a speeded go/no-go target detection task, in which participants were required to respond to changes in the direction of motion (single target), color (single target), or both (redundant target). Experiment 1 established that redundancy gains are present for targets defined by their direction of visual motion and color: Responses to redundant targets were faster than responses to single targets. There was no difference between single targets presented alone and single targets that were combined with neutral events in the other stimulus dimension, which argues against the possibility that our redundancy gains are caused by inhibitory influences due to the presentation of a neutral event. Furthermore, the redundancy gains were inconsistent with predictions based on race models. We consider this as evidence that color and motion signals are integrated rather than processed separately and independently in this particular task. Experiment 2 shows that such integration is not restricted to a single surface in a transparent motion paradigm. Whereas redundant targets were presented either in the same surface or in different surfaces, integration of color and motion signals was revealed in both cases. Finally, Experiments 3 and 4 show that the integration of target information is independent of the spatial separation between the stimuli: Integration was observed irrespective of whether redundant targets were presented at the same location or at different locations, even if the spatial distance between the stimuli was as large as 15 deg. Taken together, this series of experiments demonstrates the integration of visual features that are represented in distinct visual areas across different stimulus constellations and task demands. 

Our data allow us to draw conclusions about potential mechanisms of attention underlying performance in the redundant-target paradigm. Essentially, three different types of attentional mechanisms have been reported. Spatial attention refers to the ability to attend to a particular location in the visual field. All sensory information presented at the attended location is processed more efficiently than information outside the spatial focus of attention (Eriksen & St. James, 1986; Posner, 1980). Feature-based attention, on the other hand, describes a mechanism by which attending to a particular feature (i.e., the color “red”) enhances processing of only the attended feature, independent of the spatial focus of attention. Finally, object-based attention means that attention can be directed to perceptual groups or “objects” and that features of the same object are processed more efficiently than features belonging to different objects (“same-object advantage”; Blaser, Pylyshyn, & Holcombe, 2000; Duncan, 1984; O'Craven, Downing, & Kanwisher, 1999; for reviews, see Driver & Baylis, 1998; Scholl, 2001). Strong evidence for object-based attentional mechanisms comes from studies in which two objects are superimposed, such that spatial location by itself cannot be used to orient attention. As for visual motion, superimposed objects are created by overlaying two RDPs moving coherently in opposite directions, thereby generating the percept of two surfaces sliding across each other. By assuming an object-based mechanism of visual attention, one would predict that redundancy gains and violations of the race-model inequality should be obtained only for redundant targets occurring on the same but not on different surfaces (same-object advantage). In contrast, we find strong redundancy gains and reliable violations of the race-model inequality in both conditions (Experiment 2), ruling out an object-based account of the effect. By assuming a spatial attentional mechanism, on the other hand, one would expect that redundancy gains and race-model violations should become evident for redundant targets occurring at the same location but not for different locations. However, the effects were consistently independent of spatial location (Experiments 3 and 4). Therefore, we propose that a feature-based attentional mechanism can best account for our findings. Several studies have demonstrated that observers can effectively attend to nonspatial stimulus features, and this has also been shown for color (Anllo-Vento & Hillyard, 1996; Corbetta, Miezin, Dobmeyer, Shulman, & Petersen, 1990; Sàenz, Buracas, & Boynton, 2003) and direction of motion (Martínez-Trujillo & Treue, 2004; Sàenz, Buracas, & Boynton, 2002; Treue & Martínez-Trujillo, 1999; Treue & Maunsell, 1996). In our task, participants have divided their attention between the target color and target direction, resulting in an enhanced processing of the designated target features throughout the visual field. The RTE has already been related to feature-based attention in the study by Mordkoff and Yantis (1993). In their variant of the redundant-target paradigm, they investigated integration of color and shape information and found violations of the race-model inequality not only if the two target elements were part of the same object (i.e., a colored letter) but also if they occurred at different spatial locations (i.e., a colored frame around a white letter or a colored patch below the letter). Yet, there are two important differences between these experiments and our approach. First, using moving RDPs allowed us to superimpose two objects and randomly present same-object and different-object conditions without any change in stimulus attributes or perceptual task. Second, whereas Mordkoff and Yantis have demonstrated violations of the race-model inequality for the stimulus dimensions color and shape, we show such violations for the dimensions color and direction of motion. 

At first sight, our results might seem to be difficult to reconcile with recent behavioral studies underscoring the importance of location, on the one hand, or objects, on the other hand, for performance in visual tasks. In particular, there are two different lines of research that both arrive at apparently discrepant conclusions: investigations of feature binding and work on object-based visual attention. Investigating feature binding, Holcombe and Cavanagh (2001), for example, have demonstrated that the stimulus dimensions color and orientation are correctly bound at very high rates of presentation but only if they were superimposed (i.e., presented at the same location). Why does location matter in the case of feature binding but not in our paradigm? We think that this discrepancy can be explained by differences in task demands. As Roskies (1999) has framed it, for feature binding, “one sort of visual feature […] must be correctly associated with another feature […] to provide a unified representation of that object.” Such an explicit association is not a requirement in the type of divided-attention paradigm we have employed. In fact, because the two target features are not necessarily presented together in a given trial (single targets), an explicit association of the target features is impossible in those trials. Instead, detection of either target feature (direction of motion or color) is sufficient to initiate a response: As soon as, for example, the color red appears, a speeded response is required, regardless of the direction the red dots are moving. Likewise, the presence of, for example, rightward motion calls for an immediate response, irrespective of the color of the rightward moving dots. If we had intended to investigate the binding of color and motion, we would have asked the participants to indicate, for example, in which direction the red dots were moving. In such a situation, we would certainly expect location-based effects, that is, better performance if these two features were presented in the same location as compared with different locations. In our task, however, examining in which direction the red dots are moving might even have detrimental effects on RT. In the second line of research, transparent motion paradigms have been used to investigate object-based mechanisms of visual attention. Simultaneous judgments about speed and direction of motion are more accurate if they concern the same surface as opposed to different surfaces (Valdés-Sosa, Cobo, & Pinilla, 1998). Furthermore, when a cue directs attention to one of two superimposed surfaces, subsequent changes in the direction of motion are discriminated less accurately in the uncued surface compared with the cued one (Mitchell, Stoner, Fallah, & Reynolds, 2003; Reynolds, Alborzian, & Stoner, 2003; Valdés-Sosa, Cobo, & Pinilla, 2000). The underlying electrophysiological mechanism seems to be a selective reduction of visual neuronal responses to changes in the unattended surface, as evidenced by reduced early components of the event-related potential (ERP) (P1, N1; Valdés-Sosa, Bobes, Rodríguez, & Pinilla, 1998). Given the considerable evidence for object-based attentional mechanisms with transparent motion stimuli, one might expect to find an indication of such mechanisms with the redundant-target paradigm as well. Just as in the case of feature binding, the object-based transparent motion studies we have mentioned place different demands on the visual system as compared with our task. Valdés-Sosa, Cobo, et al. (1998, 2000), Reynolds et al. (2003), and Mitchell et al. (2003) used designs that drew or directed attention to a given surface, resulting in prioritized processing of one surface over the other. Our design, in contrast, required participants to divide attention between the stimulus dimensions color and direction of motion, and not between two objects or surfaces. Because our targets were equally likely to appear in one surface or the other, participants would not benefit from allocating resources to one surface at the expense of the second one. Furthermore, as soon as the color turned, for example, red, a speeded response was required, regardless which surface was involved. To perform optimally in our task, participants would be well advised to just focus on, for example, the color red or some dots moving to the right and not to segregate the superimposed surfaces. Here as well, examining to which surface the target color belongs might be disadvantageous in terms of RT performance. Taken together, it seems justified to assume that differences in task demands can account for the apparent discrepancies between the various paradigms. 

Alternatively, one could argue that these discrepancies might stem from the fact that we used speeded responses and compared RT distributions, whereas the other paradigms used threshold measurements (Holcombe & Cavanagh, 2001) or percentage of correct responses (Reynolds et al., 2003; Valdés-Sosa et al., 2000) as their dependent variable. In particular, one might propose that feature binding (showing location-based effects) and attentional discrimination paradigms (showing object-based effects) directly probe mechanisms at early stages of visual processing, whereas RTs are influenced by many stages of processing between the retina and the motor cortex. However, it is not the case that RTs are insensitive to effects of perceptual integration and attention in early visual areas. For example, recent works on electrophysiological (Womelsdorf, Fries, Mitra, & Desimone, 2005) and functional MRI (Weissman, Roberts, Visscher, & Woldorff, 2005) have revealed a trial-by-trial correlation between stimulus-evoked activity in visual cortical areas and RT. Moreover, it has been shown that RTs also provide a signature for the presence or absence of feature binding, for example, in classic visual search studies (Treisman & Gelade, 1980). 

It is important for our approach, however, that the RTE is, at least partly, a perceptual effect and does not arise entirely at a premotor or motor level of processing. The locus of the RTE has been discussed controversially, with some studies advocating a premotor or motor contribution (e.g., Diederich & Colonius, 1987; Giray & Ulrich, 1993) but others arguing against it (Miller et al., 2001; Mordkoff et al., 1996). For instance, Giray and Ulrich (1993) measured response force in addition to RT. Response force, which is regulated in the motor cortex (Scott, 2003), was largest in redundant-target trials, leading the authors to propose contributions of motor areas to the RTE. However, Miller et al. (2001) provided direct evidence against a motor locus by analyzing single-cell recordings from primary motor cortex in nonhuman primates. Briefly, if the RTE originated during perceptual processing, the input signals to the motor cortex should already be speeded in redundant-target trials. This would be evident in shorter latencies (i.e., time differences between stimulus onset and onset of neuronal activity) of primary motor neurons in response to redundant targets compared with single targets. Alternatively, if the RTE arose at late motor levels of processing, the motor cortex output signals should show an additional redundancy gain. In this case, the difference between response latencies of primary motor neurons and corresponding RTs should be smaller for redundant trials compared with single signal trials. Miller et al. found a reduction in neuronal response latencies to redundant targets, although there was no difference between neuronal latencies and RTs directly disconfirming the hypothesis that late motor areas constitute the origin of the RTE. Moreover, evidence in favor of perceptual contributions to the RTE has been reported consistently by a number of studies. ERP recordings have demonstrated influences of redundant targets on components associated with early visual processing (N1, P1; Miniussi et al., 1998) or target selection (P2, N2p; Reimann, Müller, & Krummenacher, 2004). Behaviorally, early sensory or perceptual contributions to the RTE have also been reported by Turatto et al. (2004) and Krummenacher et al. (2002), respectively. Taken together, these studies confirm that there is a substantial perceptual component to the RTE. 

The functional integration of color and motion information has been investigated in a number of other paradigms, using behavioral, neurophysiological, and neuroimaging methods. Evidence for mostly independent processing of color and motion comes from psychophysical studies on temporal asynchronies in visual perception (Arnold & Clifford, 2002; Arnold, Clifford, & Wenderoth, 2001; Moutoussis & Zeki, 1997; Nishida & Johnston, 2002; Viviani & Aymoz, 2001). Moutoussis and Zeki (1997) used moving objects that change their direction of motion and color. Although these changes would occur in perfect synchrony in some trials, they were shifted by different time lags with respect to each other in other trials. Psychophysical measures of the point of subjective synchrony revealed that motion changes have to happen 70–80 ms earlier than color changes for them to be perceived as occurring simultaneously. Exploiting the color-contingent motion aftereffect as an indirect measure of perceptual synchrony, Arnold et al. (2001) demonstrated a similar amount of processing lag for color and motion attributes of a stimulus. Such asynchronies in perception are taken as evidence for independent processing of color and motion signals and for a functional specialization of the visual brain areas (Zeki & Bartels, 1998, but see also Bedell, Chung, Ogmen, & Patel, 2003). In contrast, a number of studies have demonstrated functional interactions between the color and motion processing systems (for reviews, see Croner & Albright, 1999a; Dobkins & Albright, 1993a). If the two streams were functionally separate, the perception of a moving object should be impossible if object and background are isoluminant. This prediction has been rejected in behavioral experiments (Dobkins & Albright, 1993b; Hawken, Gegenfurtner, & Tang, 1994). Furthermore, corresponding neurophysiological studies have shown that neurons in the medial temporal area (MT), which is strongly implicated in the perception of visual motion (Salzman, Britten, & Newsome, 1990; Salzman, Murasugi, Britten, & Newsome, 1992), continue to signal the direction of motion of heterochromatic stimuli even under conditions of isoluminance (Dobkins & Albright, 1994; Gegenfurtner et al., 1994; Saito, Tanaka, Isono, Yasuda, & Mikami, 1989). Similarly, it has been demonstrated that color information can improve perceptual performance: Psychometric and neurometric detection thresholds in coherent motion displays are strongly reduced if dots carrying the motion signal and random noise can be segmented based on different but isoluminant colors (Croner & Albright, 1997, 1999b). The chromatic influence on motion processing can be strong, particularly under conditions of low luminance contrast (Thiele, Dobkins, & Albright, 1999, 2001), and is independent of attentional load (Thiele, Rezec, & Dobkins, 2002). Whereas some studies have shown a more prominent contribution of color information to motion processing for stimuli modulated along the red–green cardinal axis in color space (i.e., with L- and M-cone input; Gegenfurtner et al., 1994; Ruppertsberg, Wuerger, & Bertamini, 1993, but see also Lu, Lesmes, & Sperling, 1999 for an alternative account), others have also documented reliable effects for stimuli modulated along the yellowish-violet axis (i.e., with S-cone input; Seidemann & Newsome, 1999; Wandell et al., 1999). Taken together, these studies demonstrate that color information is available to the visual motion processing system. Likewise, it has been shown that some neurons in the ventral stream area V4, which is mainly specialized for the processing of orientation and color, are direction selective (Desimone & Schein, 1987; Ferrera, Rudolph, & Maunsell, 1994; Mountcastle, Motter, Steinmetz, & Sestokas, 1987; Tolias, Keliris, Smirnakis, & Logothetis, 2005). Together, these findings provide strong evidence for shared neuronal resources for color and motion processing across the two visual streams, potentially representing a neural substrate for the perceptual integration of color and visual motion signals. 

Possibly, one could object that perceptual integration of color and motion signals is achieved entirely by dorsal stream processing in the present experiments. Although the colors used in our experiments were objectively isoluminant (25 cd/m²), there might have been differences in perceived, subjective isoluminance, which might have been driving dorsal stream neurons. We argue that such an effect cannot explain the integration observed. In all experiments, four different colors were used (gray, red, green, and blue), of which only one was the designated target color. Thus, participants could not simply respond to changes in chromatic contrast or possible differences in subjective isoluminance. Considering the extremely low error rate, it seems highly unlikely that participants based their perceptual decisions on differences in subjective luminance, which, presumably, would have been much smaller than the differences in chromatic contrast. Moreover, if the dorsal stream simply integrated changes in direction of motion with changes in subjective isoluminance, no difference between redundant targets and direction targets combined with neutral color changes would be expected. In fact, our data demonstrate highly significant differences between these two conditions, ruling out such an interpretation of the results. 

In conclusion, we demonstrate the integration of color and motion information in a speeded detection task using the redundant-target paradigm. All variants of race models proposing independent and separate processing of color and motion signals in such a task can be ruled out. This is in line with recent psychophysical and neurophysiological evidence for substantial interactions between the color and motion processing systems. We show that this integration persists throughout different stimulus constellations and task demands. With transparent motion stimuli, integration of color and motion signals occurs across overlapping object borders, ruling out object-based selection in such a design. Spatial separation of color and motion signals does not constrain their integration either, rejecting location-based accounts of the effect. Feature-based theories of attentional selection, on the other hand, propose enhanced processing of the attended features throughout the visual field. Because color and motion signals neither have to be assigned to the same object nor to the same location to be integrated in the redundant-target paradigm, performance in this task can be best characterized by a feature-based mechanism of divided visual attention. 

Acknowledgments

We thank Jeff Miller for valuable discussions and advice, Christoph Nothdurft for helpful comments on an earlier version of the manuscript, and Çaglar Akçay for testing the participants in Experiments 1 and 2. 

Commercial relationships: none. 

Corresponding author: Steffen Katzner. 

Email: [e-mail]. 

Address: German Primate Center, Göttingen, Germany.