Perceptual learning of motion discrimination by mental imagery

Abstract

Abstract

Abstract:

Abstract  Perceptual learning can occur when stimuli are only imagined, i.e., without proper stimulus presentation. For example, perceptual learning improved bisection discrimination when only the two outer lines of the bisection stimulus were presented and the central line had to be imagined. Performance improved also with other static stimuli. In non-learning imagery experiments, imagining static stimuli is different from imagining motion stimuli. We hypothesized that those differences also affect imagery perceptual learning. Here, we show that imagery training also improves motion direction discrimination. Learning occurs when no stimulus at all is presented during training, whereas no learning occurs when only noise is presented. The interference between noise and mental imagery possibly hinders learning. For static bisection stimuli, the pattern is just the opposite. Learning occurs when presented with the two outer lines of the bisection stimulus, i.e., with only a part of the visual stimulus, while no learning occurs when no stimulus at all is presented.

Introduction

Perceptual learning is the ability to improve perception through practice (for reviews see Fahle & Poggio, 2003; Fahle, 2005). It is usually assumed that perceptual learning is stimulus-driven, i.e., the synaptic modifications underlying learning are triggered by the presentation of the stimuli. However, we have recently shown that visual perceptual learning can also occur in the absence of proper stimulus presentation when observers imagined the relevant part of a bisection stimulus (Tartaglia et al., 2009b; see also Amitay, Irwin, & Moore, 2006; Dupuis-Roy & Gosselin, 2007). A bisection stimulus consists of two outer lines, the distance between which is bisected by a central line in two unequal components (Supplementary Material Figure S1). In standard perceptual learning experiments, observers train to discriminate whether this central line is closer to the left or right outer line. Observers' performance in this task significantly improves (Crist, Kapadia, Westheimer, & Gilbert, 1997; Otto, Herzog, Fahle, & Zhaoping, 2006; Tartaglia, Aberg, & Herzog, 2009a). In a mental imagery experiment, observers were presented only with the two outer lines of the bisection stimulus and were asked to imagine the central line offset either to the right or to the left. After such imagery training, observers' performance, measured with the complete bisection stimulus, improved, i.e., observers were better in discriminating the direction of the central line (Tartaglia, Bamert, Mast, & Herzog, 2009b). 

Mental imagery is a composite of—at least partially—distinct subabilities (see for example Mast & Kosslyn, 2002). Exposure to parts of static stimuli often improves mental imagery of the entire static stimuli. For example, mentally completing a fragmented figure is better than imagining the entire figure (e.g., Intons-Peterson, 1981; see also Nielsen & Smith, 1973; Tversky, 1975; Thompson & Klatzky, 1978). The presentation of a visual grid has also been shown to support mental imagery tasks (Spivey & Geng, 2001). Hence, a perceptual assistance can aid mental imagery of static stimuli (e.g., Mast, Berthoz, & Kosslyn, 2001). Unlike mental imagery of static stimuli, mental imagery of motion stimuli can be disrupted by the presentation of incongruent stimuli. For example, a motion aftereffect induced by moving stimuli interferes with the mental rotation of a letter (Corballis & McLaren, 1982). Similarly, motion adaptation can interfere with imagery of a moving dot (Gilden, Blake, & Hurst, 1995). Here, we show that imagery perceptual learning of static and moving stimuli shows similarly different characteristics. 

General methods

General setup

Stimuli were presented on a Philips 201B4 CRT monitor driven by a RadeOn 9200 SE graphics card. The display was linearized through look-up tables and had a luminance resolution of 10 bits. The screen was refreshed at 100 Hz and had a spatial resolution of 1024 × 768 pixels. The room was dimly illuminated (0.5 lux) and background luminance measured on the screen was below 1 cd/m². Participants observed the stimuli from a distance of 2 m. 

Observers

30 naive students from the EPFL (Ecole Polytechnique Federale de Lausanne) and UNIL (University of Lausanne) participated in this study (age range 18–30). Students were paid for their participation. The Freiburg visual acuity test was used to determine observers' visual acuity (Bach, 1996). Observers had to reach a value of 1.0 (corresponding to 20/20) for at least one eye. Observers signed informed consent and were told they could quit the experiment at any time they wished. Observers were naive as to the goals of the experiment. 

Stimuli and task

The motion stimulus used in all experiments to determine pre- and post-training baseline performance consisted of 300 dots randomly distributed within a square field (10° width × 10° height of visual angle at a viewing distance of 2 m). Each dot subtended 5 min of arc. Dots were white (20 cd/m²) on a dark background and moved with a constant speed of 5.0°/s. Dot lifetime was 100 ms. The direction of motion of each dot was kept constant during its lifetime. When the lifetime of a dot expired, a new dot appeared in the subsequent frame at a random position. The age of each dot was randomly chosen to avoid the dots all expiring simultaneously. The independent variable was the percentage of dots moving in the same direction either horizontally (right/left) or vertically (upwards/downwards). Each trial started with a red fixation dot (8 min of arc, 10 cd/m²) presented in the center of the screen, followed after 500 ms by the stimulus (300 moving dots) for 5 s. The fixation point did not disappear during stimulus presentation. Between response and subsequent trial, only the fixation point was presented on the screen. A block consisted of 80 trials. In a binary task, observers had to discriminate the direction of dot motion (right/left or up/down). Responses were given by pressing one of two buttons. Auditory feedback was given for incorrect responses and omissions. 

Procedure

The procedure was the same for all six experiments, except for the training phase. 

1.  
   Familiarization. Observers were first familiarized with the experimental setup by performing 80 trials of a vertical chevron discrimination task.
2.  
   Adaptive procedure for threshold determination. We ran 2 blocks of 80 trials (one for the horizontal motion direction and one for the vertical motion direction) using an adaptive procedure (PEST, Taylor & Creelman, 1967) to find the individual motion coherence level for which observers performed at 75% correct (threshold).
3.  
   Pretraining baseline measurements. We determined baseline performance for the horizontal and vertical motion direction. A fixed motion coherence level was used for each observer individually. The level was set to the threshold as determined in the previous step. This method ensures that performance is not at ceiling or floor in the prebaseline measurements. The two conditions were measured twice for each observer in a randomized order to reduce the influence of learning effects in the average data.
4.  
   Extra payment. To keep observers alert during the training, we offered them an extra payment of 5 CHF for every 2% of improvement. We informed observers about the extra payment after having measured the pretraining baselines. The extra payment was offered to all participants in all six experiments.
5.  
   Training phase. In Experiment 1, observers trained to discriminate the direction of dot motion (rightward vs. leftward) embedded in randomly moving dots. This was the only experiment in which observers trained with the proper motion stimuli. The training phase comprised a total number of 4160 trials (52 blocks of 80 trials each) performed in 10 separate daily sessions. On the first and last day of training, observers performed just two blocks of training, i.e., 160 trials; in addition, the baseline measurements were conducted. Starting from the second day of training up to the ninth, observers performed six blocks of training per session, i.e., 480 trials per day. Experiments 2, 3, 4, and 5 (the mental imagery experiments) involved different mental imagery training conditions in which none or only parts of the motion stimulus of >Experiment 1 were presented on the screen.
    
   Before imagery training, we presented a random dot motion display in which the percentage of coherent motion to the right or to the left was suprathreshold, yielding more than 95% correct responses. A high frequency tone (1 kHz) was associated with motion direction to the right while a low frequency tone (700 Hz) was associated with motion to the left. We told observers about the association which was rehearsed in 80 trials (there was no association phase in Experiment 5). During imagery training, the high (1 kHz) or the low (700 Hz) frequency tone indicated to imagine motion to the right (high tone) or to the left (low tone). Observers were instructed to imagine a small percentage of dots moving and to push one out of two buttons accordingly. The training phase comprised the same number of trials as in Experiment 1 (in Experiment 1 no tones were presented). Experiment 6 was identical to Experiment 5, but observers were not asked to imagine any stimulus.
6.  
   Post-training baseline measurements. In all experiments, post-training baseline measurements, identical to the pretraining baseline measurements, were determined. These measurements were conducted between 9 and 15 days after the pretraining baselines.

Data analysis

In all experiments, we compared post- and pretraining d′ for the horizontal and the vertical direction of motion. To assess learning, we computed (two-tailed, paired) t-tests (alpha = 0.05) with the null hypothesis that there is no difference between pre- and post-training d′. In addition, in Experiment 1, regression lines were fitted to the training data of the observers (not shown). The slopes of the regression lines were compared with the hypothesis of no improvement of performance (a slope of 0.0). 

General results

Experiment 1: perceptual learning of motion direction discrimination

In a standard perceptual learning experiment, observers trained with a standard coherent motion discrimination task (Ball & Sekuler, 1982; Ball & Sekuler, 1987; Zohary, Celebrini, Britten, & Newsome, 1994; Vaina, Sundareswaran, & Harris, 1995; Liu & Vaina, 1998; Liu, 1999; Liu & Weinshall, 2000; Seitz & Watanabe, 2003; Pilly & Seitz, 2009). 

Methods

Baseline performance was measured for the horizontal (right/left) motion discrimination task as well as for the vertical (upwards/downwards) motion direction (Figure 1a). Following the baseline measurements, observers trained with the horizontal motion discrimination stimulus for 52 blocks, i.e., 4160 total trials (Figure 1b). After training, baselines were measured again for both directions of motion (Figure 1c). Four naive observers participated in the experiment. 

Figure 1

 

Experiment 1. (a) Pretraining baseline measurements. In the horizontal motion discrimination task, observers had to discriminate the direction of coherent motion (right or left) (a.a). The light grey arrows indicate the direction of the coherently moving dots, while the dark grey arrows indicate random motion of the remaining dots (arrows were not presented on the real display). In the vertical motion discrimination task, observers had to discriminate the direction of coherent motion (upwards or downwards) and push a button accordingly (a.b). (b) Training phase. Observers trained with the horizontal motion discrimination task for 4160 trials. (c) Post-training baseline measurements (same as in a). (d) We report results in terms of d′, a criterion free measure of sensitivity. Higher values of d′ indicate better performance. Post-training baselines were significantly higher than pretraining baselines for the trained horizontal motion. Error bars indicate the standard error of the mean (SEM) for four observers.

Results

We found a significant improvement of performance for the trained motion direction (Figure 1d; horizontal motion direction: post-pre baseline performance = 1.4; p = 0.039; t = 3.495; df = 3). There was no improvement for the untrained direction of motion (Figure 1d; vertical motion direction: Post-pre baseline performance = 0.6; p = 0.2; t = 1.645; df = 3). In the training phase, slopes of regression lines for the horizontal motion direction were significantly different from zero (mean slope: 0.3, p-value: 0.04; results not shown). Individual data are shown in the Supplementary Material Figures S2, S3. 

Experiment 2: imagery training—blank screen

We investigated whether perceptual learning of motion direction discrimination could occur via mental imagery. 

Methods

We measured baseline performance for the horizontal motion discrimination as well as for the vertical motion direction (Figure 1a). During the training phase (Figure 2a), observers were presented with a blank screen only. In each trial, a high or a low frequency tone indicated to imagine approximately 30 dots, i.e., 10% of the total number of dots, moving to the right or left, respectively. At the end of the training session, we measured baseline performance again for both motion directions (Figure 1c). Four naive observers participated in the experiment. 

Figure 2

 

Experiment 2. (a) Training phase. A blank screen was presented during the imagery training. Observers were asked to imagine a small number of dots moving coherently to the left or right, depending on a tone (arrows and dots were not shown in the actual display). The pre- and post- baselines conditions were exactly the same as in Experiment 1 (not shown). (b) Baseline performance significantly improved for the trained horizontal motion but not for the untrained vertical motion. Error bars indicate the standard error of the mean (SEM) for four observers.

Results

Sensitivity increased for the trained motion direction (Figure 2b; horizontal motion direction: post-pre baseline performance = 0.67; p = 0.04; t = 3.365; df = 3). We did not find an increase in sensitivity for the vertical direction of motion (Figure 2b; vertical motion direction: post-pre baseline performance = −0.06; p = 0.8; t = 0.212; df = 3). Individual data are shown in the Supplementary Material Figures S2, S3. 

Experiment 3: imagery training—noise

Experiment 2 showed that perceptual learning of motion stimuli can occur via imagery when only a blank screen is presented. Previous studies have shown that the presence of incongruent motion interferes with mental imagery. Here, we show that no improvement of performance occurs when noise is presented instead of a blank screen during the imagery training. 

Methods

We measured baseline performance for both the horizontal and the vertical direction of motion (Figure 1a). During the training phase, randomly moving dots were presented. Observers were asked to imagine a subset of these dots (approximately 10%, i.e., 30 dots) to move coherently to the right or left (Figure 3a). After training, baseline measurements were repeated for both directions of motion (Figure 1c). Five naive observers participated in the experiment. 

Figure 3

 

Experiment 3. (a) Training phase. Randomly moving dots were presented during the imagery training. Observers were asked to imagine a small proportion of the dots moving coherently to the left or right, depending on a tone. (b) Baseline performance did not improve for both the trained (horizontal) and untrained (vertical) motion directions. Error bars indicate the standard error of the mean (SEM) for five observers.

Results

Sensitivity improved neither for the trained horizontal motion direction, nor for the untrained vertical one (Figure 3b, horizontal motion direction: Post-pre = −0.071; p = 0.7; t = 0.498; df = 4. Vertical motion direction: post-pre = −0.13; p = 0.6; t = 0.537; df = 4). Individual data are shown in the Supplementary Material Figures S2,S3. 

Experiment 4: imagery training—static dots

Experiment 3 showed that random motion obstructed imagery perceptual learning. To investigate whether this interference is related to the noise, we asked observers to imagine coherent motion while presented with a pattern of static dots. We expected that the processing of static dots interfered less with imagery. 

Methods

During the imagery training phase, observers were asked to imagine a small part of the static dots (approximately 10%, i.e., 30 dots) moving coherently to the right or to the left, according to the tones (Figure 4a). Nine naive observers participated in the experiment. 

Figure 4

 

Experiment 4. (a) Training phase. A number of fixed dots were presented during imagery training. Observers were asked to imagine a small proportion of the dots moving coherently to the left or to the right, depending on a tone. Pre- and post- baselines were exactly the same as in the experiments before. (b) Baseline performance did not improve for either the horizontal or for the untrained, vertical motion direction. Error bars indicate the standard error of the mean (SEM) for nine observers.

Results

No increase in sensitivity after training with both directions of motion was found (Figure 4b, horizontal motion direction: post-pre = 0.25; p = 0.26; t = 1.206; df = 8, vertical motion direction: post-pre = 0.22; p = 0.1; t = 1.890; df = 8). Individual data are shown in the Supplementary Material Figures S2, S3. 

Experiment 5: imagery training—congruent moving dots

It has been shown that a stimulus compatible with the stimulus that has to be imagined can serve as a perceptual assistance for imagery. For example, Mast et al. (2001) have shown that a mental image of static dots was more easily formed when some of the dots were presented during imagery. Here, we tested whether also imagery learning benefits from a perceptual assistance. 

Methods

In the imagery training, observers were presented with three dots moving to the right or left randomly positioned on the screen. The direction of motion was randomized across trials and was associated with a high or a low frequency tone. Observers were instructed to imagine a small proportion of dots (approximately 30 dots) moving in the same direction and at the same speed as the three dots (Figure 5a). Four naive observers participated in this experiment. 

Figure 5

 

Experiment 5. (a) Training phase. Three coherently moving dots were presented during the imagery training. Observers were asked to imagine a small proportion of dots moving in the same direction as the three dots. (b) Baseline performance significantly improved for the horizontal motion direction but not for the untrained vertical direction. Error bars indicate the standard error of the mean (SEM) for four observers.

Results

Sensitivity significantly improved for the trained motion direction, indicating that the visual cues efficiently guided mental imagery (Figure 5b, horizontal motion direction: post-pre = 0.35; p = 0.045; t = 3.189; df = 3). This improvement did not transfer to the untrained motion direction (Figure 5b, vertical motion direction: post-pre = 0.4; p = 0.5; t = 0.644; df = 3). Individual data are shown in the Supplementary Material Figures S2, S3. 

Experiment 6: no mental imagery—moving dots

In the previous experiment, three coherently moving dots guided imagery learning. Here, as a control experiment, we show that motion discrimination did not improve when observers did not imagine motion. 

Methods

The training phase was identical to that of Experiment 5, except that observers did not have to imagine moving dots. Observers had to discriminate the motion direction of the three dots (Figure 6a). Four naive observers participated in the experiment. 

Figure 6

 

Experiment 6. (a) Training phase. As in Experiment 5, three coherently moving dots were presented during training. Observers indicated the motion direction. No imagery was involved in this experiment. (b) Baseline performance did not improve through training neither for the horizontal nor for the untrained, vertical motion direction. Error bars indicate the standard error of the mean (SEM) for four observers.

Results

Performance was above 95% correct for the motion direction task during training (results not shown). We did not find an increase in sensitivity for the trained as well as the untrained motion direction (Figure 6b, horizontal motion direction: post-pre = 0.03; p = 0.7; t = 0.361; df = 3; vertical motion direction: post-pre = 0.4; p = 0.2; t = 1.740; df = 3). Individual data are shown in the Supplementary Material Figures S2, S3. 

Discussion

Most models of perceptual learning assume that the synaptic changes underlying perceptual learning are driven by the presentation of stimuli. Without stimulus presentation learning is impossible. Other factors, such as attention, motivation, or task instructions can only modulate learning but they cannot drive it. However, we have previously shown that perceptual learning can occur in the absence of (proper) stimulus presentation when observers were asked to imagine the stimuli or parts of them (Tartaglia et al., 2009b). 

Imagery of static and motion stimuli shows different characteristics. While imagery of static stimuli activates the primary visual cortex (Kosslyn, Thompson, Kim, & Alpert, 1995; Slotnick, Thompson, & Kosslyn, 2005, Thompson, Kosslyn, Sukel, & Alpert, 2001), imagery of motion stimuli activates motion areas like MT, nearly as strongly as real motion, but not V1 (Goebel, Khorram-Sefat, Muckli, Hacker, & Singer, 1998; Grossman & Blake, 2001; Seurinck, de Lange, Achten, & Vingerhoets, 2011). This is consistent with the idea that the activation of specific brain regions depends on the content of the mental image (O'Craven & Kanwisher, 2000). For imagery of motion stimuli, stimuli that are incompatible with the direction of the imagined motion cause disruption of imagery (Corballis & McLaren, 1982; Gilden et al., 1995). Given these differences between the imagery of static and motion stimuli, we hypothesized and found that imagery perceptual learning of static stimuli is different from imagery perceptual learning of motion stimuli. 

We found that motion discrimination improved when observers imagined left-right moving dots on a blank background during training (Figure 2). To the contrary, in the case of mental imagery of bisection stimuli no improvement of performance occurred when only a blank screen was provided (see Supplementary Material Figure S1 and Tartaglia et al., 2009b). Possibly this discrepancy comes from the fact that high spatial resolution mental images, required for bisection stimuli, need a spatial reference. On the other hand, imagery perceptual learning of Gabor patch detection is possible with no stimulus presentation during training because no spatial reference is needed (Tartaglia et al., 2009b). Such a reference seems also not to be necessary for motion imagery learning. However, motion discrimination failed to improve when the imagined motion was incompatible with motion of real stimuli (Figure 3). Such disruptive interference was observed also when static dots were presented during motion imagery (Figure 4). Interestingly, Experiment 3 was the only experiment in which observers reported difficulties in performing the mental imagery task, indicating that, indeed, the randomly moving dots were impeding imagery. Accordingly, Experiment 5 shows that when stimuli were congruent with the mental image, they could serve as a guide or a perceptual assistance to mental imagery (Mast et al., 2001), yielding improvement of performance (Figure 5). 

As mentioned above, imagery bisection learning works only if the outer lines are presented. It may be argued that similarly to motion imagery, static imagery needs perceptual assistance. However, while motion imagery benefits from the perceptual assistance of a small signal which varies from trial to trial, static imagery rather needs a spatial reference which remains constant from trial to trial and does not contain any informative signal. 

All together, the current results provide yet another demonstration that mental imagery can stand in for the visual stimulus, and that imagery training improves visual perception. 

Data for the trained motion direction are rather clear and consistent for conditions where performance improved (Experiments 1, 2, and 5) and where performance remained virtually unchanged (Experiments 3 and 6). An ANOVA with a single factor (factor Experiments) showed that Experiment 3 was significantly different from imagery learning with a blank screen (p = 0.039). Results for the untrained motion direction are less clear. Data for these conditions are in general noisy and the associated p-values vary greatly. The scatter plots reported in the Supplementary Material (Figure S2) clearly show that high p-values come from high interobserver variances. We have determined performance for the orthogonal motion direction because it is standard in perceptual learning experiments to investigate transfer of learning to untrained stimulus features, and in a previous manuscript on imagery learning of static stimuli (Tartaglia et al., 2009b), contrary to most findings in standard perceptual learning, there was transfer to the untrained orientation of bisection and Gabor stimuli. The reasons for this transfer remain unclear for the moment. It might be that the small number of trials per training session causes transfer, as shown in Aberg, Tartaglia, & Herzog (2009). Also the present data are not conclusive. It might be argued that the improvement of performance in the untrained motion direction is an indication of rather unspecific learning effects which are not related to imagery. In our previous manuscript, we showed, however, that learning is indeed caused by imagery. For example, no learning occurred when observers were presented with the two outer lines of the bisection stimuli, as during imagery training, but were required to perform a different task which did not involve imagery (Tartaglia et al., 2009b). In this contribution, we also showed that learning occurred when observers imagined motion on top of a very small signal (Experiment 5). No learning occurred when observers were presented with the very same small signal but were not required to imagine (Experiment 6). 

The improvement of performance and the lack thereof do not have a straightforward interpretation in terms of initial threshold or initial pre-training performance, i.e., we found no consistent pattern of correlation between learning and initial threshold or initial performance (see Supplementary Material Figures S4, S5). 

In summary, imagery perceptual learning of motion stimuli is possible, does not require any visual input, and is disrupted by the presence of incompatible noise. Imagery of static and motion stimuli shows different characteristics and so does imagery learning of static and motion stimuli. Accordingly, it is very likely that imagery learning relies on the same mechanisms active during imagery. 

Supplementary Materials

PDF S1 

Acknowledgments

We would like to thank Bilge Sayim for helpful discussions and Marc Repnow for technical support. Elisa Tartaglia is funded by the Sinergia project “State representation in reward based learning in human healthy observers, schizophrenic patients, and models of perceptual learning” and by the "Perspective Researcher fellowship" (project number PBELP3-135838) of the Swiss National Science Foundation (SNF). 

Commercial relationships: none. 

Corresponding author: Elisa Tartaglia. 

Email: [e-mail]. 

Address: Laboratory of Neurophysics and Physiology, UMR 8119 CNRS, Université René Descartes, Paris, France.