Linking perceptual learning with identical stimuli to imagery perceptual learning

Abstract

Perceptual learning is usually thought to be exclusively driven by the stimuli presented during training (and the underlying synaptic learning rules). In some way, we are slaves of our visual experiences. However, learning can occur even when no stimuli are presented at all. For example, Gabor contrast detection improves when only a blank screen is presented and observers are asked to imagine Gabor patches. Likewise, performance improves when observers are asked to imagine the nonexisting central line of a bisection stimulus to be offset either to the right or left. Hence, performance can improve without stimulus presentation. As shown in the auditory domain, performance can also improve when the very same stimulus is presented in all learning trials and observers were asked to discriminate differences which do not exist (observers were not told about the set up). Classic models of perceptual learning cannot handle these situations since they need proper stimulus presentation, i.e., variance in the stimuli, such as a left versus right offset in the bisection stimulus. Here, we show that perceptual learning with identical stimuli occurs in the visual domain, too. Second, we linked the two paradigms by telling observers that only the very same bisection stimulus was presented in all trials and asked them to imagine the central line to be offset either to the left or right. As in imagery learning, performance improved.

Introduction

Perceptual learning is the ability to improve perception. For example in a bisection task, three parallel lines are presented with the central line slightly offset either towards the right or left outer line. Participants indicate the offset direction. Training gradually improves performance. Visual perceptual learning improves for most basic visual features and is specific for the trained stimulus and task (e.g., Crist, Kapadia, Westheimer, & Gilbert, 1997; Fahle & Edelman, 1993; Fahle & Morgan, 1996; Karni & Sagi, 1993; for a review, see Fahle & Poggio, 2003). Perceptual learning is usually thought to be determined by synaptic changes depending on a learning rule such as backpropagation or unsupervised learning (Hebb, 1949; Hinton & Sejnowski, 1999). In most learning models, learning is fully determined by the learning algorithm and the stimulus presentation (Tsodyks & Gilbert, 2004). For example in perceptrons (Rosenblatt, 1958), the presentation of a stimulus μ activates neurons in the input layer. This activation is propagated to the output layer via synaptic connections ω_j. If the output activity y^μ does not match the desired output t^μ for stimulus μ, an error t^μ − y^μ is computed. The synaptic weights ω_j are adjusted depending on this error:  

 

where is the activity of the j-th input neuron in response to stimulus μ (η controls the learning speed). In unsupervised learning algorithms, as for example in unsupervised learning (Poggio, Fahle, & Edelman, 1992), synaptic changes depend only on the concurrent activation of neurons:  

 

 

In both types of learning rules, if no stimulus is presented, = 0 for all j, because there is no activation of the input layer. Thus, Δω_j = 0, for all j. Hence, trivially, perceptual learning cannot occur in the absence of stimulus presentation according to these models. However, humans can learn in the absence of proper physical stimulation (Tartaglia, Bamert, Mast, & Herzog, 2009). First, performance in a bisection task was determined (pretraining baseline). During training, the bisection stimulus was presented without the central line; i.e., only the very same outer lines were presented in all trials. Participants were asked to imagine the central line. After training, performance was determined with the “normal” bisection stimulus again. Compared to the pretraining baseline condition, performance improved. Learning with these imagined stimuli occurred with half of the gain as training with the “normal” bisection stimuli, where the central line was presented also during training. We were able to rule out unspecific effects and attribute the gains in performance to mental imagery. In another experiment, contrast detection improved when observers were asked to imagine a Gabor patch on a blank screen during training (Tartaglia et al., 2009).  

Interestingly, human performance can also improve if the very same stimulus is presented repeatedly during training (Amitay, Halliday, Taylor, Sohoglu, & Moore, 2010; Amitay, Irwin, & Moore, 2006; Roth, Refael-Taub, Sharvit, & Kishon-Rabin, 2006). During training, the very same tone was presented in three subsequent intervals. Participants were not told about the identical tones, but asked to indicate which of three tones was higher in frequency. Frequency discrimination with different tones was measured before and after training. Training with identical tones improved performance in the posttraining measurements. Authors suggested that learning with identical stimuli occurs because external feedback enhances motivation as there were no improvements in the no-feedback condition (Amitay et al., 2010). 

Both in learning with imagined and identical stimuli, the very same stimulus is presented during training. In learning with imagined stimuli, the crucial part of the stimulus is omitted, such as the central line or even the entire Gabor, and observers are asked to imagine it. In learning with identical stimuli, the crucial part is presented and observers are told that there is variation even though there is not. 

Perceptual learning with identical stimuli challenges computational models as much as perceptual learning without proper stimulation because in both paradigms stimuli are identical during training. Classic neural networks can neither learn to discriminate identical stimuli nor learn with imagined stimuli. Here, we link the two paradigms suggesting that learning with identical stimuli is driven by mental imagery (Tartaglia, Bamert, Herzog, & Mast, 2012; Tartaglia et al., 2009). 

General methods

General setup

Stimuli were presented in the center of X-Y monitors (Tektronix 608, Beaverton, OR; HP-1332A, Hewlett-Packard, Palo Alto, CA) controlled by a PC via fast 16-bit DA converters (1 MHz pixel rate). Refresh rate was 200 Hz. Lines were composed of dots drawn with a dot pitch of 200 μm at a dot rate of 1 MHz. The dot pitch was selected to make the dots slightly overlap; i.e., the dot size (or line width) was of the same magnitude as the dot pitch. Luminance was measured with a Minolta LS-100 luminance meter. Luminance of the stimuli was 80 cd/m². The room was dimly illuminated (0.5 lux) and the background luminance of the screen was below 1 cd/m². The viewing distance was 2 m. Chin rests with forehead bars were used to reduce head movements. 

Observers

Sixty-one naïve students from the École Polytechnique Fédérale de Lausanne (EPFL) or the University of Lausanne (UNIL) took part in the study. Ten observers participated in Experiments 1, 2, 3, 4, and 4a (Supplementary Material). Eleven observers participated in the Experiment 4b. Each participant took part in only one experiment. Participants signed informed consent and had to reach a value of 1.0 with at least one eye with the Freiburg visual acuity test (Bach, 1996; corresponding to a Snellen fraction of 20/20). Participants were paid 20 Swiss Francs per hour. Procedures were conducted in accordance with the Declaration of Helsinki and were approved by the local ethics committee. 

Stimuli

Bisection stimuli consisted of two outer lines making up a spatial interval bisected by a middle line (Figure 1a). Lines were 20′ (arc minutes) long. The interval delineated by the two outer lines was 26.6′. Vertical bisection stimuli were used in the training phase, and horizontal and vertical bisection stimuli in pre- and posttraining blocks. Chevron stimuli were used for familiarization and were composed of two 10′ long lines. Chevrons were 45° counterclockwise oriented. No fixation point was presented. 

Figure 1

 

Stimuli and procedure. Spatial layout of a vertical bisection stimulus used in the pre- and the posttraining blocks (a). The central line was either offset to the left or right during the pre- and the posttraining blocks but not during training where it was exactly in the center in all trials. The procedure was identical for all experiments (b), except for the Experiment 4b (Supplementary Material 1.2). Observers performed 10 training sessions. Pre- and posttraining were determined in the first and the last session.

Procedure

Each trial started with four markers at the corners of the screen presented for 500 ms, followed by the bisection stimulus for 1000 ms (Figure 1b). Observers had 3 s to indicate the offset direction of the central line by pressing one of two buttons. After button press, the screen remained blank for 500 ms between the button press and the subsequent trial. Each block was composed of 80 trials. 

On the first day, observers were first familiarized with the experimental setup by performing one block of the chevron discrimination task. Observers indicated the direction towards which the chevron was pointing by pressing one of two push buttons. We used chevron stimuli to minimize procedural and fast learning effects in the subsequent bisection tasks. The choice of that familiarization task was motivated by the fact that it does not transfer to the bisection discrimination (Kramer & Fahle, 1996). Thereafter, we measured thresholds of 75% correct responses for both horizontal and vertical bisection stimuli in one block each. To do this, we used an adaptive staircase method and maximum likelihood estimation of the parameters of the psychometric function (parameter estimation by sequential tracking [PEST], starting value: 100″; 80 trials; Taylor & Creelman, 1967). 

In the pretraining, observers performed two blocks of vertical and two blocks of horizontal bisection task with fixed offset values equal to their threshold values (previously measured with PEST procedure). These blocks were randomized; i.e., either vertical or horizontal blocks were performed first. After that, observers performed two blocks of training. 

From the second until the ninth session, observers performed six blocks of training per session (i.e., 480 trials per session). In the tenth session, observers performed two last training blocks followed by the posttraining measurements which were identical to the pretraining measurements. Overall, there were 52 training blocks during training, i.e., 4,160 trials in total. Training never exceeded more than two weeks. During pre- and posttraining, auditory feedback was given for incorrect responses. 

Data analysis

We conducted a two-way, repeated measures ANOVA with the factors Orientation (vertical, horizontal) and Pre-Post testing (pretest, posttest). Post hoc analysis was performed using Scheffé tests to determine simple main effects. 

Experiment 1: Training with the identical visual stimulus

Training with identical stimuli improved performance in auditory perceptual learning (Amitay et al., 2006, 2010; Roth et al., 2006). In Experiment 1, a similar procedure was adapted for visual bisection stimuli. Observers were presented with the very same vertical bisection stimulus in each training trial, where the center line was always in the middle. Observers were told that there was an offset. The task was to discriminate the (nonexisting) offset of the central line. 

Methods

Fake error feedback was provided. We decreased the percentage of feedback tones across sessions to mimic improvement. The percentage of the amount of the feedback per block was chosen randomly from a uniform distribution defined by the mean and its lower and upper limits. Thus, in the first three training sessions (14 blocks), 25% ± 3% of error tones were provided. From the fourth to the sixth training session (18 blocks), the percentage of error tones was decreased to 22% ± 3%. In the last four sessions (20 blocks), the amount of feedback was decreased to 19% ± 3%. 

Results

Sensitivity improved significantly for the vertical bisection stimulus through training: Figure 1, first column; posttest–pretest = 1.14 ± 0.29, F(1, 9) = 12.34, p = 0.007. The improvement for the horizontal bisection stimulus was not significant, but there was a trend: Posttest–pretest = 0.59 ± 0.24, F(1, 9) = 3.31, p = 0.102. Hence, performance can improve by training with identical visual stimuli, confirming earlier results in the acoustical domain (Amitay et al., 2006, 2010; Roth et al., 2006). Interactions were not significant, F(1, 9) = 4.89, p = 0.054. 

Experiment 2: Training with identical stimuli and no feedback

Methods

The procedure was the same as in Experiment 1, except that no feedback was provided. 

Results

Sensitivity for both the vertical and the horizontal bisection stimuli did not improve significantly: Figure 2, second column; posttest–pretest = 0.64 ± 0.29, F(1, 9) = 4.16, p = 0.072; posttest–pretest = 0.32 ± 0.22, F(1, 9) = 1.03, p = 0.337, although there was a trend for the vertical stimulus. Hence, results are similar as in the auditory domain (Amitay et al., 2006, 2010; Roth et al., 2006). Interactions were not significant, F(1, 9) = 1.90, p = 0.20. 

Figure 2

 

Pre- (blue bars) and posttraining (red bars) measurements were identical in the five experiments. Experiments differed only in the training type where the central line was always exactly in the center. Observers indicated the nonexistent offset direction in Experiments 1 and 2, imagined the central line to be slightly offset (Experiment 3), or discriminated the longer horizontal line (Experiment 4). Improvements for vertical bisection stimulus were observed in Experiments 1 and 3. Despite a strong trend, the training without feedback did not generate a significant improvement for the vertical bisection stimulus. Mean ± SEM for 10 observers.

Experiment 3: Identical stimuli and mental imagery

Experiments 1 and 2 reproduce the experiments from the auditory in the visual domain. Significant learning with identical stimuli occurred only with feedback. Amitay et al. (2010) argued that improvements with identical stimuli occur because external feedback maintains motivation. Feedback and motivation may be important for learning but do not explain learning per se (Fahle, Edelman, & Poggio, 1995; Mckee & Westheimer, 1978). Here, we tested whether mental imagery can explain learning with identical stimuli. 

Methods

After the pretraining measurements, observers performed a block with a vertical bisection stimulus, in which the displacement of the central line (right or left) was suprathreshold, yielding more than 95% correct responses. A high frequency tone (1 kHz) was associated with a right offset and a low frequency tone (700 Hz) with a left offset. Observers were told about the association. During the training phase, we presented vertical bisection stimuli without offset, i.e., the central line bisected the spatial interval into two equal parts, as in Experiment 1. We told observers about the zero offset and asked them to imagine the line to be offset. In addition, we presented either a 1000 Hz or a 700 Hz tone. For a 1000 Hz tone, observers were asked to imagine the central line to be closer to the right outer line and to press the correct button. When presented with a 700 Hz tone, observers imagined the central line to be closer to the left outer line and pressed the left button. In rare cases when observers pressed the wrong button, an error feedback tone was provided. 

Results

Sensitivity improved significantly for the vertical bisection stimulus: Figure 2, third column; posttest–pretest = 0.68 ± 0.20, F(1, 9) = 10.85, p = 0.009. But sensitivity did not improve for the horizontal bisection stimulus: Posttest–pretest = 0.40 ± 0.30, F(1, 9) = 1.96, p = 0.086. Interactions were not significant, F(1, 9) = 1.14, p = 0.313. 

Experiment 4: Line length discrimination

Here, we tested whether factors other than imagery (e.g., procedural learning) could have caused the improvements observed in the previous experiments. Two horizontal lines were added above and below the bisection stimulus. Observers indicated the longer one of the two. 

Methods

Only in the training trials, the vertical bisection stimulus was presented together with two flanking, horizontal lines parallel to each other (Figure 2, fourth column). One of the horizontal lines (the reference) varied randomly from trial to trial within an interval of 45′–55′ while the second line was always longer. The distance between both horizontal lines was 50′, and the lines were vertically centered on the screen. The position of both horizontal lines was shuffled randomly; i.e., the reference could appear either in the upper or lower position. Auditory feedback was provided for incorrect responses. Before training, thresholds of 75% correct responses for the line length difference were determined with the PEST procedure (starting value = 140″ of length difference between the two horizontal lines). During training, the nonreference line was longer by the threshold length from the reference. 

Results

The post hoc analysis revealed that the training did not improve the sensitivity either for the vertical, posttest–pretest = 0.18 ± 0.15, F(1, 9) = 1.89, p = 0.203, or for the horizontal, posttest–pretest = 0.21 ± 0.16, F(1, 9) = 2.59, p = 0.142, bisection stimuli. This result suggests that improvements in the bisection task observed in the previous experiments were not due to simple exposure to the stimuli or unspecific learning (see Discussion). Interactions were not significant, F(1, 9) = 0.03, p = 0.867. 

Discussion

First, perceptual learning with identical stimuli occurs in the visual domain similarly as in the auditory domain (Amitay et al., 2006, 2010; Roth et al., 2006). Second, feedback led to significant improvements whereas there was only a trend toward improvement without feedback. It seems that, as with “normal” perceptual learning, feedback speeds up learning for identical stimuli (Herzog & Fahle, 1997) but is not necessary for it. Hence, feedback-mediated increases in motivation may play an important role in perceptual learning (Amitay et al., 2010; Herzog & Fahle, 1997, 1998; Lukaszewski & Elliott, 1962; Shibata, Yamagishi, Ishii, & Kawato, 2009); however, they do not explain why learning occurs at all (Amitay et al., 2010; Fahle et al., 1995; Mckee & Westheimer, 1978). Third, we suggest that mental imagery mediates learning with identical stimuli (Experiment 3). Interestingly, none of the observers reported after the experiment to have perceived the center line not to be offset during the training phase. Likewise, only one out of twenty listeners in the study by Amitay et al. (2013) reported to hear no differences in tones. In the same line, Gosselin and Schyns (2003) showed that observers perceive (nonexisting) letters or faces in white noise when told they exist, and Sterzer, Frith, and Petrovic (2010) showed that a belief about a forthcoming stimulus can strongly change percepts, even though the stimulus is always the same. 

Amitay et al. (2010) found that listeners with low nonverbal IQ (NVIQ) did not improve performance when there was no feedback. In addition, there is a positive relationship between intelligence and mental imagery (Shaw & DeMers, 1986; González, Campos, & Pérez, 2010), supporting the hypothesis that imagery plays a crucial role in perceptual learning with identical stimuli. Similarly, Dupuis-Roy and Gosselin (2007) showed that top-down processes can be sufficient to produce substantial perceptual learning without any signal. Finally, visual perceptual learning can be induced by task irrelevant mental imagery (Shibata, Watanabe, Sasaki, & Kawato, 2011). Altogether, these experiments support the idea that mental imagery relies on the same sensory representations as stimulus presentation (Berger & Ehrsson, 2013; Borst & Kosslyn, 2008; Ishai & Sagi, 1995; Kosslyn, Ganis, & Thompson, 2001; Kosslyn, Thompson, Kim, & Alpert, 1995; Pearson, Clifford, & Tong, 2008) and that imagery perceptual learning can be explained by changes of these representations by imagery. 

It is not easy to investigate imagery perceptual learning since imagery is not easy to control. We controlled for unspecific learning effects by asking observers to perform a length discrimination task, where observers did not imagine the central line to be offset. There was no transfer to the bisection task, a result which rules out that the mere presentation of the bisection stimulus is sufficient for learning (Experiment 4). One may argue that in the Experiment 4, the horizontal flanking lines interfered with the bisection stimulus (e.g., a crowding effect). We tested this hypothesis by comparing bisection thresholds for bisection stimuli with and without horizontal flanking lines. Thresholds in both conditions did not differ (Supplementary Material). We need to mention that another control experiment has led to significant transfer. Participants indicated which of the two outer lines was of higher luminance. Performance improved for this task and transferred to the bisection task (Supplementary Material). One explanation is that indeed perceptual learning is rather unspecific. The other explanation is that the luminance difference has led to attraction of the center line towards one of the outer lines, as we show in a control experiment (Supplementary Material 1.2). Similar effects of attraction and repulsion caused by manipulation of luminance were previously found with Vernier (Badcock & Westheimer, 1985; Westheimer & McKee, 1977) and other stimuli (Bulatov, Bulatova, & Surkys, 2012; Morgan, Ward, & Cleary, 1994; Whitaker, Mcgraw, Pacey, & Barrett, 1996), strongly supporting the theory of spatial pooling by light. Learning with imagined and identical stimuli cannot be explained by classic neural network models, in which learning is purely stimulus driven (for a review, see Tsodyks & Gilbert, 2004). Learning with imagined and identical stimuli cannot be explained by models of hyperacuity (Klein & Levi, 1985; Wilson, 1986) because these models extract variations of the stimuli (left vs. right offset), which is missing in imagery learning and learning with identical stimuli. How can imagery learning be explained? Imagery enhances presynaptic neurons similarly to the activation generated by stimuli (Kosslyn et al., 1995). Likewise, it was shown that attention can boost activity in primary visual cortex even before stimuli presentation (Gandhi, Heeger, & Boynton, 1999; Silver, Ress, & Heeger, 2007). Hence, these presynaptic activations may be used in classic learning models to learn. Another mechanism is internal noise reduction (Amitay et al., 2013; Amitay, Zhang, Jones, & Moore, 2014; Micheyl, McDermott, & Oxenham, 2009). For example, there might be noise sources in the brain, which may unspecifically add noise to neurons involved in the task. Inhibiting these neurons can thus lead to improvements in learning and to transfer, which we often observe in imagery learning but not in training with proper stimuli. When such factors are important in learning with identical and imagined stimuli, then they might be important as well in learning with proper stimuli. 

Acknowledgments

We would like to thank Marc Repnow for technical support; Takeo Watanabe, Aaron Clark, and Gregory Francis for their useful comments. Lukasz Grzeczkowski was funded by the projects “Mental Imagery and Perceptual Learning” (Project Number: 100014 135303/1) and “Learning from Delayed and Sparse Feedback” (Project Number: CRSII2_147636) of the Swiss National Science Foundation (SNFS). 

Commercial relationships: none 

Corresponding author: Lukasz Grzeczkowski. 

Email: [e-mail]. 

Address: Laboratory of Psychophysics, Brain Mind Institute, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland.