Combined effects of feature-based working memory and feature-based attention on the perception of visual motion direction

Abstract

We investigated whether human subjects' ability to identify the direction of a brief pulse of coherent motion in a random-dot pattern (RDP) was influenced by: (a) maintaining in working memory the direction of motion of an RDP previously presented far from the pulse (feature-based working memory or FBWM, Experiment 1), (b) attending to the direction of an RDP co-occurring with but far from the pulse (feature-based attention or FBA, Experiment 2), and (c) both FBWM and FBA acting simultaneously (Experiment 3). In the first two experiments, pulse direction identification performance was higher when the remembered direction (FBWM) or the direction of the concurrently attended RDP (FBA) matched the pulse direction than when it was opposite. In Experiment 3, performance was highest when both the remembered and the attended directions matched the pulse direction (combined effects of FBWM and FBA), it was intermediate when only one of them matched the pulse direction, and it was lowest when neither matched the pulse direction. Our results demonstrate that both feature-based working memory and feature-based attention can individually modulate the perception of motion direction and that when acting together they produce an even larger modulation.

Introduction

The human visual system faces at least two major challenges concerning information processing. First, the amount of information that enters the retina at any given moment greatly exceeds the system's processing capacity. Second, visual information often becomes unavailable while it is still relevant to behavior. Two cognitive processes provide solutions to these challenges. The first, attention, allows us to select behaviorally relevant visual signals and enhance their processing, while filtering out irrelevant ones (Boynton, 2005; Posner, 1980). The second, working memory (WM), allows us to temporarily maintain representations of behaviorally relevant stimuli that are no longer visible (Baddeley, 1996; Goldmanrakic, 1995; Postle, 2006). After decades of research in vision, it remains largely elusive how these two processes interact (Awh, Vogel, & Oh, 2006). 

A widespread conception in cognitive science is that the attentional control system is one of the key elements of WM (Baddeley, 1996). At the same time, it has been hypothesized that WM representations are linked to the origin of attentional signals. For example, it has been proposed that top-down attentional filtering might result from the effect of WM representations of behaviorally relevant features (e.g., spatial location, color, shape) modulating the processing of retinal signals in visual cortical areas (Desimone & Duncan, 1995). Strong evidence in favor of this proposition has come from neurophysiological studies of visual search in monkeys (Bichot & Desimone, 2006; Chelazzi, Duncan, Miller, & Desimone, 1998; Chelazzi, Miller, Duncan, & Desimone, 1993, 2001). Visual search tasks, however, require a template item in WM to intentionally guide the search toward matching items on a display. A generalization of the concept of WM-guided attention has further led to the hypothesis that the maintenance of visual representations in WM automatically biases attention toward visual stimuli that match these representations (Downing, 2000). 

Supporting this view, Downing (2000) showed that during the maintenance of a visual stimulus representation in WM, briefly presenting an irrelevant matching stimulus automatically shifted attention toward its location. Similarly, while subjects maintained a geometric shape in WM, reaction times in a concurrent visual search task were shorter when the target item was located within a matching geometric shape than when it was located within other shapes (Soto, Heinke, Humphreys, & Blanco, 2005). These studies show that the content of WM can bias the allocation of spatial attention. 

A more controversial issue is whether the maintenance of non-spatial visual features in WM could influence perceptual processing of those features (feature-based WM, FBWM). For example, Turatto et al. reported that in contrast to previous studies showing that attention can alter the perceived value of visual features such as motion coherence (Liu, Larsson, & Carrasco, 2007) and motion speed (Turatto, Vescovi, & Valsecchi, 2007), the automatic shift of attention toward a stimulus matching the content of WM did not alter its perceived speed (Turatto, Vescovi, & Valsecchi, 2008). They suggested that although FBWM influences the allocation of spatial attention, it does not necessarily alter the perception of stimulus features. 

Another set of studies has focused on isolating the effects of attending to a visual feature when WM representations of that feature are not required (feature-based attention or FBA). They have demonstrated that attending to a stimulus feature presented at a given location on a display can modulate the processing of that feature elsewhere, in monkeys (Martinez-Trujillo & Treue, 2004; Maunsell & Treue, 2006; Treue & Martinez-Trujillo, 1999; Treue & Maunsell, 1996) and humans (Andersen, Hillyard, & Muller, 2008; Andersen, Muller, & Hillyard, 2009; Liu et al., 2007; Muller et al., 2006; Saenz, Buracas, & Boynton, 2002; Serences & Boynton, 2007; Stojanoski & Niemeier, 2007). 

In the present study, we examined the effects of FBWM, FBA and their combination on motion direction perception. We conducted two experiments to isolate the individual effects of FBWM and FBA on performance in a task that required perceiving the direction of a coherent motion pulse and an additional experiment aimed at investigating their simultaneous effects on performance in the same task. 

Experiment 1: Effect of feature-based working memory on the perception of a coherent motion pulse

The purpose of Experiment 1 was to test whether the maintenance of a direction of motion in WM modulates the perception of motion direction. In a delayed match-to-sample task, subjects were required to remember the direction of motion of a sample random-dot pattern (RDP) and compare it to the direction of test RDPs. Concurrently, subjects performed a pulse identification task requiring them to identify the direction of a brief pulse of coherent motion (Figure 1). We measured pulse identification performance when the direction held in WM was either the same as or opposite to the direction of the pulse. We hypothesized that subjects would be better at perceiving pulses in the same direction as the one held in WM than pulses in the opposite direction (effect of FBWM). 

Figure 1

 

Visual stimuli presented in Experiment 1. (a) Depiction of stimulus display with the location of the three RDPs and the cue. (b) Diagram showing the time course of stimuli presentation in one sample trial. The three rows show the stimuli presented over time at the three locations. Each circle with a single arrow represents an RDP with coherent motion in the direction of the arrow. Each circle with multiple arrows represents an RDP with 0% coherent motion. During periods illustrated in gray, no stimulus was presented at a particular location. Subjects performed the delayed match-to-sample and pulse identification tasks simultaneously.

Methods

Subjects

Seven subjects (2 females, 5 males) aged between 20 and 40 years (median = 27 years) with normal or corrected-to-normal vision participated in the study. Four of them were naïve to the purpose of the experiment. Each subject gave written consent prior to the experiments. All procedures were preapproved by the Ethics Committee of the Faculty of Medicine at McGill University. 

Each of the seven subjects completed Experiments 1, 2, 3, and Controls in three sessions of approximately 45 min each. Every subject performed a total of approximately 300 trials: 30 in Experiment 1, 40 in Experiment 2, 60 in Experiment 3, 60 in Control 2, 50 for the estimation of the psychometric curve of pulse coherence, and between 15 and 35 practice trials for each experiment. Five of these subjects also participated in Experiment 4. Two additional subjects participated in Experiments 3 and 4. 

Setup and stimuli

During the experiments, each subject sat in a dark room 57 cm away from a CRT computer monitor (LaCie, Oregon, USA, 75-Hz refresh rate, 1280 × 1024 resolution, size 21 inches). The subject's head was stabilized using a head–chin rest. The experiments and stimuli were programmed using the Matlab Psychophysics Toolbox software (Mathworks, Naticks, MA), running on an Apple G4 PowerPC computer (Apple, Cupertino, CA). Stimuli were displayed on a dark background (luminance < 0.001 cd/m²) and consisted of a 0.25°-by-0.25° fixation square located in the middle of the screen, a 0.93°-by-0.13° horizontal bar (spatial cue) located 2.80° to the left or right of the fixation square, and random-dot patterns (RDPs; Figure 1). Each RDP was composed of 300 white dots (luminance of 39.75 cd/m²). The dots were 0.09° in diameter and moved at a speed of 4.7°/s in linear trajectories within a stationary virtual circular aperture of 6.25° in diameter. RDPs were presented centered at 3 locations: sample, pulse, and test. The sample location was 5.63° above the fixation square. The test location was 9.38° toward one side of the fixation square. The pulse location was 9.38° away from the fixation square, on the side opposite to the test location. In each trial, the test location was randomly chosen as right or left. In order to indicate this location, the spatial cue was displayed during the entire trial. The setup and stimuli properties were the same in all the experiments. 

The motion coherence of an RDP was defined as the percentage of dots moving in the same direction (e.g. 50% coherence corresponds to half of the dots moving in the same direction and the remaining dots moving in randomly chosen directions from 0° to 360°). An RDP is referred to as having 0% coherence if all the dots move in randomly chosen directions (from 0° to 360°). Direction of motion is described relative to the screen's vertical meridian (i.e., upward motion = 0°, rightward = 90°, downward = 180°, and leftward = 270°). 

Task

Each trial consisted of two tasks: a delayed match-to-sample task and a pulse identification task (Figure 1). In the delayed match-to-sample task, subjects fixated on the central square at the beginning of each trial, and then pressed the spacebar key to start the trial. One thousand milliseconds later, an RDP appeared at the sample location moving with 100% coherence in one of the following directions: 45°, 135°, 225°, or 315°. After 1000 ms, the sample disappeared and subjects continued fixating for a variable delay period of 1800 to 2250 ms. Following that period, four test RDPs with 100% coherent motion were serially presented at the test location during 250 ms each, separated by intervals lasting from 450 to 750 ms (inter-test intervals). During these intervals, RDPs with 0% coherent motion were presented, as well as before the first and after the last test RDP. Each test RDP could move in a 45°, 135°, 225°, or 315° direction. Subjects were required to count how many of the test directions matched the sample direction. The number of matches could be zero, one, or two and was randomly chosen from trial to trial. Subjects indicated the number of matches at the end of each trial. 

In the pulse identification task, subjects viewed an RDP presented at the pulse location from the end of the delayed match-to-sample task delay period until the end of the trial (Figure 1). The RDP contained motion with 0% coherence throughout this entire period except for a 250-ms pulse of coherent motion in a direction of 45°, 135°, 225°, or 315°. The coherence of this pulse was chosen individually for each subject as described in the Data analysis section. Subjects were instructed to identify the pulse direction. 

At the end of the trial, all the stimuli were removed and subjects were no longer required to fixate. Four arrows appeared at the screen center depicting the four possible directions of the coherent motion pulse (45°, 135°, 225°, and 315°). Subjects were required to press one of four keys, each corresponding to one pulse direction. Immediately after a response was given, the arrows disappeared and the following text was displayed: “How many matches of sample & test? 0, 1, or 2.” Subjects pressed one of three keys to report the number of matches. 

An important detail in this experiment is that the pulse of coherent motion was presented during one of the three inter-test intervals, when only 0% coherent motion was shown at the test location, and while subjects maintained the sample direction in WM in order to perform the delayed match-to-sample task (Figure 1b). This allowed us to test whether the representation of a given direction in WM influenced subjects' ability to identify the direction of the pulse. In half of the trials, the sample direction was the same as the pulse direction (sample-same condition); in the other half, the sample direction was opposite to the pulse direction (sample-opposite condition). 

Randomization procedures

In each trial, we randomly chose the pulse direction, the condition, the directions of the four tests, and the inter-test interval during which the pulse occurred (1st, 2nd, or 3rd). In order to make sure that during the pulse presentation subjects maintained the sample direction in WM, we never presented the pulse after two test directions had already matched the sample (maximum number of matches). 

Data analysis

Performance in the delayed match-to-sample task was quantified as the percentage of trials in which a subject correctly reported the number of tests whose direction matched the sample direction. Mean performance across subjects in this task was 92% (88% among sample-same trials and 95% among sample-opposite trials). We excluded from analysis all trials with incorrect responses in the delayed match-to-sample task, since in these trials we could not guarantee that the subjects remembered the sample direction. From the remaining trials, performance in the pulse identification task was measured in each of the two conditions as the percentage of trials in which a subject correctly reported the direction of the coherent motion pulse. In each condition, all subjects performed the pulse identification task above chance. 

To correct for deviations from normality and symmetry that may occur in distributions of percentages, we applied a rationalized arcsine transformation on the performance scores. We then compared performance between the two conditions across all subjects by using a paired-samples t-test on the transformed data. In all other experiments, statistical tests were also performed on rationalized arcsine-transformed performance values. From now on, we will refer to this transformed data as performance scores or simply performance. 

To graph the results across subjects (Figure 3), we normalized the performance of each subject in each condition to the mean performance across conditions. Then, for each condition, we averaged the normalized performances across subjects. The same procedure was used in all other experiments to obtain estimates of mean normalized performance (Figures 5, 7, 8, and 9). 

Before starting the experiments, we obtained baseline measurements of each subject's performance in the pulse identification task alone while they were presented with the same pattern of stimuli used in Experiment 1 but were told to ignore the sample and test stimuli. We used five different levels of pulse coherence: 10, 25, 45, 65, and 95%. Subjects performed a total of 50 trials (10 for each coherence level). Performance data for all five coherence levels were then fitted with the following sigmoid function: 

$P=100-75\left(\frac{h^s}{h^s+C^s}\right),$

(1)

where C is the coherence level, P is the performance, the parameter h is the coherence value at which half of the curve's performance range is reached (half-maximum), and the parameter s determines the curve's slope at the point of inflection. 

We aimed at obtaining from each subject, during Experiments 1, 2, and 3, an average pulse identification performance intermediate between saturation (100%) and chance (25%), ideally in the range of 60% to 70%. We anticipated that performance values in this range would be modulated by FBWM and FBA, while values at saturation or chance may be poorly or not modulated at all by these two factors. We used the pulse coherence level at which subjects perform at 85% during the pulse identification task alone, taking into account that during the dual task performance shall drop at least 15–25% with respect to the single task (Niebergall, Huang, & Martinez-Trujillo, 2010). Thus, for each subject, we used the psychometric curve's fit parameters h and s to estimate the pulse coherence (C) corresponding to 85% identification performance (Figure 2). In all experiments, we set the pulse coherence to this value. Across subjects, the mean coherence value at 85% performance was 0.51. As anticipated, pulse identification performance during the experiments fell around the desired range, averaging 68% in Experiment 1, 62% in Experiment 2, and 70% in Experiment 3 across subjects. 

Figure 2

 

Pulse identification performance of an example subject as a function of motion coherence. The performance data (dots) were fitted with a sigmoid curve. The value of coherence corresponding to 85% performance was calculated from the curve's estimated parameters (dashed lines and X).

Results and discussion

We compared pulse identification performance in the two experimental conditions: when the sample direction was the same as the pulse direction (sample-same) vs. when it was opposite (sample-opposite). Performance was higher in the sample-same than in the sample-opposite condition (paired-samples t-test, t(6) = 2.69, p = 0.02, one-tailed; Figure 3). On average, performance values differed by 11% between the two conditions. 

Figure 3

 

Results from Experiment 1. Normalized performance averaged across all subjects in the sample-same and sample-opposite conditions. Error bars indicate standard errors across subjects. The relative direction of the sample with respect to the pulse in each condition is depicted. For illustration purposes, in this and all other figures, the pulse direction is always 45°.

These results show that the maintenance of a direction of motion in WM facilitated the perception of coherent motion in that direction relative to the opposite direction (FBWM). The experiment required the comparison of a sample motion direction with the directions of test stimuli. Importantly, both the sample and the tests were located at known positions but far from the pulse. Nonetheless, the maintenance of the sample direction in WM influenced pulse identification, suggesting that the effect of FBWM propagated to the pulse location. We should note that because the pulse location was not systematically changed from trial to trial to more than two positions, it is not possible to definitively conclude that the effect would be present at any randomly chosen visual location and that it is therefore “space-independent.” 

Our findings agree with a recent report by Silvanto and Cattaneo (2010) investigating the influence of FBWM on visual perception. They induced moving phosphenes by transcranial magnetic stimulation (TMS) of area V5/MT while subjects maintained a representation of a moving stimulus in WM. Phosphene perception was enhanced when the motion direction of the remembered stimulus matched that of the phosphene and attenuated when the remembered stimulus and the phosphene had opposite directions. One possibility is that the mechanisms underlying phosphene modulation by FBWM are similar to those underlying the effect reported here. One difference with the mentioned study was that while WM-based phosphene modulation was present only when the phosphene had the same location as the sample stimulus, the effect observed in our study was present at a location different from that of the sample. 

Experiment 2: Effect of feature-based attention on the perception of a coherent motion pulse

We tested whether directing attention to the motion direction of an RDP can modulate the perception of a coherent motion pulse in a second RDP positioned far away. As in Experiment 1, subjects performed the pulse identification task. However, instead of a delayed match-to-sample task, they simultaneously performed a direction change detection task, which required them to report the occurrence of direction changes in a test RDP co-occurring with but far from the pulse. If attention to the test direction (FBA) selectively modulates the perception of motion direction at distant locations in the visual field, then we anticipate that pulse identification performance will vary depending on the relationship between the pulse and the co-occurring test direction. 

Methods

Subjects

The same seven subjects from Experiment 1 participated in this experiment. The setup and stimuli properties were as described in Experiment 1. 

Task

The stimulus sequence was identical to that of Experiment 1, with two exceptions: (1) the pulse was presented at the same time as one of the test stimuli, and (2) either zero, one, or two of the test RDPs were randomly chosen to change their direction of motion by 60° (clockwise or counterclockwise) after 125 ms of presentation (Figure 4). Two tasks were performed simultaneously in each trial: the direction change detection and the pulse identification task. In the former, subjects had to count how many of the four test RDPs changed direction. As in Experiment 1, a “sample” RDP was presented, but it was irrelevant to the task. 

Figure 4

 

Visual stimuli presented in Experiment 2. (a) Depiction of stimulus display. (b) Diagram showing the time course of stimuli presentation in one sample trial. Subjects performed the direction change detection and the pulse identification tasks simultaneously. Details are as described in Figure 1. Angled arrows represent RDPs that changed motion direction.

The pulse identification task was the same as that described in Experiment 1. Subjects were required to identify the direction of a 250-ms pulse of coherent motion. The pulse could be presented simultaneously with any of the test RDPs, except with those that changed direction. This avoided exogenous shifts of attention (Yantis & Jonides, 1990) toward the test stimulus that may influence the amount of attention allocated to the pulse detection task. At the end of each trial, all visual stimuli disappeared and subjects were no longer required to fixate. As in Experiment 1, subjects used key presses to report their choices. 

The direction change detection task required subjects to attend to the direction of motion of each of the four test RDPs in order to count the number of direction changes. The pulse RDP occurred while subjects attended to the direction of a test stimulus, allowing us to test whether attending to a particular test direction influenced subjects' ability to identify the pulse. In 50% of the trials, the pulse and the test had the same direction (test-same condition). In the other 50%, they had opposite directions (test-opposite condition). If FBA—to the motion direction of the test—enhances the perception of motion in that direction, then pulse identification performance in the test-same condition will be higher than in the test-opposite condition. 

Randomization procedures

In each trial, we randomly chose the condition, the motion directions of the pulse and the tests, the test with which the pulse co-occurred, the number and order of tests that changed direction, and the direction of the irrelevant sample. Given that subjects could potentially stop attending to the tests after perceiving two direction changes, the pulse never occurred after a second direction change. 

Data analysis

We quantified performance in the direction change detection task as the percentage of trials in which a subject correctly reported the number of tests that changed motion direction. Across subjects, mean performance was 86%. Our analysis excluded trials with an incorrect response to the direction change detection task, given that in these trials we could not guarantee that subjects attended to the test stimuli. From the remaining trials, performance in the pulse identification task was independently measured for each of the two experimental conditions (see Methods section of Experiment 1). 

Results and discussion

We compared pulse identification performance between trials of the test-same and test-opposite conditions. The mean normalized performance across subjects was higher when the pulse and the test had the same direction than when they had opposite directions (paired-samples t-test, t(6) = 1.90, p = 0.05, one-tailed; Figure 5). Across subjects, performance differed by an average of 13% between the two conditions. Interestingly, this difference was somewhat similar to the difference in performance between the two conditions of Experiment 1 (11%). 

Figure 5

 

Results from Experiment 2. Mean normalized performance across subjects in the test-same and test-opposite conditions. Error bars indicate standard errors. The illustration depicts the relationship between the directions of the test and the pulse in each condition.

The results show that attention to the test direction facilitated the identification of pulses in the same direction relative to pulses in the opposite direction. Because the test and the pulse were located far from each other in opposite hemifields, we attribute this effect to global FBA. Alternatively, it could be argued that the effect is due to a “Gestalt” grouping effect (Baylis & Driver, 1993), whereby pulse perception is facilitated by its binding or grouping with the test when the directions of the two stimuli are the same. This possibility was discarded during an additional control experiment (see Control 2). 

Global FBA effects on motion perception have been previously observed. Sàenz, Buracas, and Boynton (2003) showed that attention to one of two spatially overlapping RDP surfaces moving in opposite motion directions facilitated the perception of speed in a distant surface moving in the same attended direction. Interestingly, this effect was absent when the attended RDP surface was not accompanied by an overlapping surface moving in the opposite direction. In our experiment, the test RDP co-occurring with the pulse was presented in the absence of an overlapping motion surface, and yet it was able to cause an FBA effect on pulse identification. It is possible that our sequentially but randomly presented test stimuli with different motion directions at the same location require high attentional demands comparable to those needed when viewing overlapping surfaces of transparent motion but not when viewing a single motion surface. 

In general, our findings are consistent with previous reports that FBA modulates signal strength in human (Kamitani & Tong, 2006; Saenz et al., 2002) and macaque (Martinez-Trujillo & Treue, 2004; Treue & Martinez-Trujillo, 1999) motion processing areas. Furthermore, these modulatory effects may underlie the behavioral effects reported here. 

Experiment 3: Simultaneous effects of feature-based working memory and feature-based attention on the perception of a coherent motion pulse

In Experiment 1, we found that the maintenance of a direction of motion in WM can influence subjects' ability to identify the direction of a coherent motion pulse (FBWM). We then showed in Experiment 2 that attending to a direction of motion can also influence pulse identification (FBA). The purpose of Experiment 3 was to investigate whether and how FBWM and FBA simultaneously modulate pulse identification. In this experiment, subjects performed the delayed match-to-sample and pulse identification tasks at the same time. Importantly, in contrast with Experiment 1, the pulse of coherent motion co-occurred with one of the test stimuli of the delayed match-to-sample task. Thus, during the presentation of the pulse, subjects held the representation of the sample direction in WM and at the same time attended to the direction of a test stimulus. We tested how pulse identification performance varied as a function of the sample (remembered) and test (attended) directions with respect to the pulse direction. 

Methods

Subjects

Nine subjects (3 females, 6 males) participated in this experiment. Seven of them also participated in Experiments 1 and 2, and the other two participated in Experiment 4. The setup and stimuli were as described in Experiment 1. 

Task

Subjects simultaneously performed the delayed match-to-sample and pulse identification tasks. The pulse of coherent motion co-occurred with one of the four test RDPs in the delayed match-to-sample task (Figure 6). 

Figure 6

 

Stimuli in Experiment 3. (a) Depiction of stimulus display. (b) Diagram showing the time course of stimuli presentation in one sample trial. Subjects performed the delayed match-to-sample and pulse identification tasks simultaneously. Details are as described in Figure 1.

During the presentation of the pulse, subjects were required to maintain the sample direction in WM and attend to the direction of the co-occurring test (Figure 6b). To investigate the simultaneous effects of the sample direction (FBWM) and the test direction (FBA) on pulse identification, we designed different trial types based on a two-factorial design. One factor was the direction of the sample with respect to the pulse, which had two levels: sample-same and sample-opposite. The other factor was the direction of the co-occurring test with respect to the pulse, which also had two levels: test-same and test-opposite. The two factors were crossed, resulting in four conditions: sample-same/test-same, sample-same/test-opposite, sample-opposite/test-same, and sample-opposite/test-opposite. 

Randomization procedures

In each trial, we randomly chose the pulse direction, the condition, the number of tests with a direction matching the sample, the test directions, and the test with which the pulse co-occurred. The pulse never occurred after two tests matched the sample direction, to make sure that during the presentation of the pulse, subjects were still required to attend to the test and remember the sample direction. 

Data analysis

Trials were grouped by condition. For each of the four conditions, the performance of each subject in the pulse identification task was calculated as described in the previous experiments. We did not include trials in which subjects' response to the delayed match-to-sample task was incorrect, since there was no guarantee that in these trials subjects remembered the sample direction and attended to the tests throughout the trial. Across subjects, mean performance in the delayed match-to-sample task was 88% (87% among sample-same/test-same trials, 91% among sample-same/test-opposite trials, 89% among sample-opposite/test-same trials, and 85% among sample-opposite/test-opposite trials). 

Results and discussion

We tested whether pulse identification performance differed across the two levels of each of the factors: the sample direction with respect to the pulse (FBWM) and the co-occurring test direction with respect to the pulse (FBA). The mean normalized performance across subjects was highest in the sample-same/test-same condition, intermediate in the sample-same/test-opposite and sample-opposite/test-same conditions, and lowest in the sample-opposite/test-opposite condition (Figure 7). A two-factor repeated-measures ANOVA across subjects (sample and test directions as factors) revealed a significant main effect of both sample direction (F(1,8) = 6.43, p = 0.018, one-tailed) and test direction (F(1,8) = 9.61, p < 0.01, one-tailed) on pulse identification. The interaction effect between the two factors was not significant (F(1,8) = 1.70, p = 0.23). 

Figure 7

 

Results from Experiment 3. Normalized performance averaged across all subjects in each condition: sample-same/test-same, sample-same/test-opposite, sample-opposite/test-same, and sample-opposite/test-opposite. The illustration depicts the relationship between the directions of the sample, test, and pulse in each condition.

The average difference in performance between sample-same and sample-opposite trials (pooled across test directions) was 12%, an estimate of the effect of FBWM on pulse identification. This magnitude was similar to the difference in performance between sample-same and sample-opposite trials in Experiment 1 (11%). The average difference in performance between test-same and test-opposite trials (pooled across sample directions) was 14%, an estimate of the magnitude of the effect of FBA on pulse identification. Again, this magnitude was similar to the difference in performance between test-same and test-opposite trials in Experiment 2 (13%). The average difference in performance between the sample-same/test-same and sample-opposite/test-opposite conditions was 26%, an estimate of the simultaneous effects of FBWM and FBA on pulse identification. 

Experiment 4

Experiments 1 and 3 investigated the effects of FBWM on motion perception using a task in which subjects had to hold the direction of a sample RDP in WM. However, because there were only four possible directions of the sample and the tests, it may be argued that subjects could have performed the task by simply using categorical representations of the four sample directions rather than a true visual WM representation of the sample. In order to investigate this possibility, we conducted an additional experiment using a different task with a larger number of sample and test directions. 

Methods

Subjects

Seven subjects (4 females, 3 males) participated in this experiment. Two of them also participated in Experiment 3. 

Task

Subjects simultaneously performed a delayed match-to-sample task and a pulse localization task. The delayed match-to-sample task instructions were identical to those in the previous experiments: subjects were required to count how many test RDPs (0, 1, or 2) moved in the same direction as the sample. However, the task had two modifications: first, the motion direction of the sample was chosen randomly in each trial from the entire 360° range of directions, rather than from 4 possible directions. Second, the test RDPs could take any of 8 possible directions separated by 45°, one of them the same as the sample direction. 

Simultaneously with the delayed match-to-sample task, subjects performed a pulse localization task. They were presented with two RDPs located equidistant from the fixation point one above the other and on the side opposite to the test (Figure 8). The RDPs were presented from the end of the delay period of the delayed match-to-sample task until the end of the trial. Both contained 0% coherent motion, but in one of them (randomly chosen in each trial), a 250-ms pulse of coherent motion occurred during one of the three inter-test intervals, while subjects held the sample direction in WM. The task was to report whether the pulse occurred in the upper or lower RDP. At the end of the trial, two circles outlining the two alternative pulse locations and the text “Pulse location” appeared in the middle of the screen. They pressed one of two keys to report the pulse location. Immediately after, they reported the number of test directions that matched the sample direction, as in Experiments 1 and 3. 

Figure 8

 

Stimuli in Experiment 4. (a) Depiction of stimulus display. (b) Diagram showing the stimulus presentation time course in one sample trial. Subjects simultaneously performed the delayed match-to-sample and pulse localization tasks. Details are as described in Figure 1.

The pulse localization task allows measuring the subjects' ability to perceive coherent motion. However, different from the pulse identification task, subjects indicated in which one of the two possible locations the coherent motion pulse occurred. As in Experiment 1, the sample direction was the same as the pulse direction in half of the trials (sample-same condition) and opposite in the remaining half of the trials (sample-opposite condition). Subjects performed a total of 120 trials. Based on the results of Experiment 1, we anticipated higher performance at localizing the pulse in the sample-same than in the sample-opposite condition. 

Data analysis

Performance in each one of the two conditions was measured as the percentage of trials in which a subject correctly reported the pulse location. Trials with an incorrect response in the delayed match-to-sample task were not included in the analysis. Mean performance across subjects in the delayed match-to-sample task was 85% in both sample-same and sample-opposite trials. Before performing the experiment, we chose a level of motion coherence of the pulse for each subject using the method described in the Data analysis section of Experiment 1, in order to obtain pulse localization performance values intermediate between saturation (100%) and chance (50%). Mean performance across subjects was 76%. 

Results and discussion

Performance in the pulse localization task was higher in the sample-same than in the sample-opposite trials (paired t-test, t(6) = 2.50, p = 0.023, one-tailed; Figure 9). These results show that the maintenance of a motion direction in WM facilitated subjects' ability to detect and localize the presence of coherent motion in the same direction relative to the opposite one (FBWM). It is important to emphasize that in this experiment, the direction that subjects had to remember was randomly chosen from a complete 360° range. This prevented the subjects from using a fixed set of categorical representations of the sample and challenged them to maintain a visual representation of that stimulus direction during the trial. 

Figure 9

 

Results from Experiment 4. Mean normalized performance across subjects in the sample-same and sample-opposite conditions. Error bars indicate standard errors. The illustration depicts the relationship between the sample and pulse directions in each condition.

Furthermore, the current results allow us to control for a potential confounder that may have been present in Experiments 1 and 3. In both experiments, subjects had to keep in WM the directions of the sample and the pulse until the end of the trial. When both directions were the same, the WM load was one. When they were opposite, the WM load was two. Lower load in the former situation could yield better performance due to limitations in the subjects’ WM capacity (Baddeley, 1996; Cowan, 2001). The results of Experiment 4 rule out this possibility since here subjects did not report the pulse direction but its location, and therefore, they only needed to maintain the sample direction in WM. Under those circumstances, pulse performance was higher in sample-same relative to sample-opposite trials, arguing against differences in WM load between conditions as an explanation for the observed effects. 

General discussion

Our results show that the perception of motion direction can be modulated by FBWM and FBA individually (Experiments 1 and 2) and simultaneously (Experiment 3). Interestingly, FBWM and FBA had similar effects on pulse identification performance when comparing the results of Experiments 1 and 2 or in Experiment 3. The combined effects were larger than each of the individual effects and approximately equal to their sum, suggesting that they can additively combine. One question arising from our results is whether FBWM and FBA act through different and independent mechanisms or neural pathways. With our behavioral data set, it is difficult to answer this question. However, at least one study has reported that in the dorsolateral prefrontal cortex of macaques, different neurons carry signals related to WM and attention (Lebedev, Messinger, Kralik, & Wise, 2004). One may speculate that in our experiments, different pools of neurons were differentially activated by FBWM and FBA. If these neurons independently carry signals that through top-down projections modulate sensory processing in visual areas, that may explain the additive effect of FBWM and FBA. 

The similarities in the magnitude of the FBWM and FBA effects we are reporting here may have implications for current theories proposing that WM is an emergent brain property due to the coordinated recruitment, via attention, of brain systems that have evolved to accomplish sensory-, representation-, and action-related functions (Postle, 2006). It may well be that during the maintenance of a stimulus feature in WM, the same modulatory mechanisms recruited when “visually” attending to that feature are active. This would explain why FBWM and FBA effects on motion processing seem to be behaviorally (this study) and physiologically indistinguishable (Bisley, Zaksas, Droll, & Pasternak, 2004; Treue & Martinez-Trujillo, 1999; Zaksas & Pasternak, 2006). Our finding that the modulation caused by the combined action of FBWM and FBA is larger than the individual effects could be explained by an increase in the strength of the same modulatory mechanism when both remembered and attended features coincide. 

What are the mechanisms by which WM for motion direction influences the processing of visual motion? One hypothesis is that during the presentation of a visual stimulus, the responses of early visual neurons selective for a particular feature of the stimulus are modulated by the maintenance of that feature in WM. Previous studies have shown that the content of WM correlate with patterns of fMRI signals in early visual brain areas (Ester, Serences, & Awh, 2009; Harrison & Tong, 2009), suggesting that neurons involved in the processing of sensory features may also play a role in the maintenance of such features in WM (Pasternak & Greenlee, 2005). On the other hand, single unit electrophysiological recordings have shown that during the maintenance of a motion direction in WM and in the absence of sensory input, the spiking activity of direction-selective neurons in area MT does not consistently encode the remembered direction during the entire memory period (Bisley et al., 2004; Zaksas & Pasternak, 2006). One possible explanation for this inconsistency is that fMRI measurements do not directly correlate with the spiking activity but with other signals such as local field potentials that carry information about modulatory inputs from other areas (Khayat, Niebergall, & Martinez-Trujillo, 2010; Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001). This issue, however, needs further investigation. 

Although our experiments were not aimed at testing whether WM representations automatically evoke shifts of attention, they do demonstrate at least a degree of automaticity in the effects of FBWM and FBA. This is because the pulse identification task was independent from the delayed match-to-sample and direction change detection tasks. This result agrees with reports of WM representations influencing the allocation of spatial attention, not only when such influence does not represent an overall advantage or detriment to the performance of the task (Downing, 2000), but also when it interferes with performance (Olivers, Meijer, & Theeuwes, 2006; Pashler & Shiu, 1999; Soto et al., 2005). On the other hand, Downing and Dodds (2004) showed that WM-based attention shifts can also be avoided when they are detrimental: performance in a visual search task was not affected by whether or not one of the distractor items in the search array matched a representation held in WM. Furthermore, Woodman and Luck (2007) showed that when subjects knew that an item held in WM was never the target in a visual search task, performance was higher when that item was part of the search array than when it was not. They concluded that subjects were capable of strategically directing attention away from items similar to those held in WM. Taken together, the results of the aforementioned studies and ours suggest that WM representations automatically facilitate the processing of matching stimuli. This automaticity, however, can be voluntarily overridden depending on the task demands (Han & Kim, 2009). 

In sum, we have demonstrated individual modulatory effects of FBWM and FBA on motion direction perception, as well as a larger modulation when they act simultaneously. These two effects share at least two common attributes: they are both global, and they are feature-based. An issue that remains as a challenge for future studies is to fully characterize the neural mechanisms and pathways underlying their effects on perception and behavior. 

Controls

In Experiments 1, 2, and 3, we showed that the ability of subjects to identify the pulse direction was affected by the direction of the sample and by the direction of the test co-occurring with the pulse. From these results, we concluded that maintaining a direction of motion in WM, as well as attending to a motion direction, can modulate motion perception. However, it could be argued that the observed effects were not necessarily due to the maintenance of the sample direction in WM or attending to the test, but by mere exposure to the sample or the test (e.g., sensory priming by the sample, or perceptual grouping of the test and the pulse). To rule out this possibility, we carried out two additional procedures, referred to as Control 1 and Control 2. 

Control 1

In Experiment 2, we presented a sample RDP identical to the one in Experiments 1 and 3 (Figure 4b). However, this stimulus was not relevant for any of the tasks. This allowed us to test whether the direction of the sample influences pulse identification when subjects are exposed to it but do not maintain it in WM. 

As described earlier, there were two types of trials in Experiment 2: test-same and test-opposite. Within each type, trials were randomly divided into: (a) those in which the sample and the pulse directions were the same (sample-same), and (b) those in which these directions were opposite (sample-opposite). Half of the trials were sample-same and half were sample-opposite. Performance in the pulse identification task was measured as described in the Data analysis section of Experiment 1. 

Across subjects, mean normalized performance was similar between sample-same and sample-opposite trials (Figure 10; t(6) = 0.072, p = 0.47, one-tailed paired-samples t-test). We conclude that the effect of the sample direction on pulse identification performance observed in Experiment 1 was not due to exposure to the sample per se, but rather to the maintenance of the sample direction in WM. 

Figure 10

 

Results from Control 1. Mean normalized performance across all subjects in sample-same and sample-opposite trials of Experiment 2. Error bars indicate standard errors. The illustration depicts the relationship between the directions of the sample and the pulse in each condition.

Control 2

In Experiments 2 and 3, subjects were better at identifying the direction of the pulse when it was the same as the one of the co-occurring test than when it was opposite. As proposed above, it can be argued that this effect might not be due to attention to the test direction but rather to mere exposure to the test during the presentation of the pulse. To discard this possibility, we performed an additional experiment. This experiment also served to corroborate that pulse identification performance was not affected by presentation of the sample itself, but rather by the maintenance of the sample direction in WM. 

The procedure was identical to that of Experiment 3, except that subjects were instructed to perform the pulse identification task only, while ignoring the sample and the tests. We carried out a two-factor repeated-measures ANOVA across all subjects using sample direction and test direction as the two factors. We found no significant effect of sample direction (F(1,6) = 0.09, p = 0.78) or test direction (F(1,6) = 0.20, p = 0.67), on performance (Figure 11). The interaction between the two factors was not significant (F(1,6) = 0.60, p = 0.47). Thus, when subjects were exposed to the sample and the test but neither held the sample direction in WM nor attended to the test direction, pulse identification performance was unaffected by the directions of the sample and the test, in agreement with previous studies using similar controls (Downing, 2000; Olivers et al., 2006; Soto et al., 2005). 

Figure 11

 

Results from Control 2. Normalized performance averaged across all subjects in each condition: sample-same/test-same, sample-same/test-opposite, sample-opposite/test-same, and sample-opposite/test-opposite. The illustration depicts the relationship between the directions of the sample, test, and pulse.

Response bias

In Experiments 1 and 3, performance was higher when the sample and the pulse had the same direction than when they had opposite directions. One possible explanation for these results is that in trials in which subjects did not perceive the pulse direction, they could have had a tendency to use the direction of the remembered sample as a response to the pulse identification task. A response bias of this sort would cause subjects to correctly guess the pulse direction more often when it was the same as the sample direction than when it was opposite. In order to avoid this confounder, we explicitly told the subjects that the pulse identification and delayed match-to-sample tasks were independent of each other. Nevertheless, it is possible that subjects ignored our instructions and still biased their responses. 

To test this possibility, we analyzed the responses to the pulse identification task in error trials of Experiment 3. If subjects were biased to respond in the direction of the sample when they did not perceive the pulse, then the proportion of error types in the sample-same and sample-opposite trials would differ, i.e., the proportion of incorrect sample-same trials with a response opposite to the sample direction would be significantly less than the proportion of incorrect sample-opposite trials with a response equal to the sample direction. Among error trials, the mean proportions of these two error types across subjects were 0.34 and 0.37, respectively. We compared these values within subjects using a paired-sample t-test and found no significant difference (t(6) = 0.50, p = 0.32, one-tailed), ruling out the possibility that our results were due to a response bias. 

Acknowledgments

The experiments were supported by grants from the EJLB Foundation and the Canada Research Chair Program awarded to JMT. DM is supported by a grant from the CIHR. CK was supported by a UCL Bogue Research Fellowship and is currently holding a Feodor Lynen Award from the Alexander von Humboldt Foundation. MS was supported by an NSERC undergraduate student research award. 

Commercial relationships: none. 

Corresponding author: Julio Martinez‐Trujillo. 

Email: [e-mail]. 

Address: 3655 Prom. Sir William Osler, Room 1223, McGill University, Montreal, QC H3G 1Y4, Canada.