New is always better: Novelty modulates oculomotor learning

Abstract

Saccadic adaptation aims at keeping saccades accurate to enable precise foveation of objects. It has been believed to be a rather low-level adjustment, responding chiefly to direction and magnitude of postsaccadic position error. However, recent studies have shown that image content can modify saccadic adaptation. Adaptation is more complete for saccades toward socially relevant human figures in comparison to noise when time constraints exist. In the present experiment, we show that saccadic adaptation is also susceptible to the novelty of a stimulus. In a scanning adaptation paradigm, 20 subjects participated in two sessions of forward adaptation to one position at which the same human picture was always displayed versus a position at which a new human figure was presented in every trial. Saccadic adaptation was more complete to the novel-target position. This suggests that novelty can increase oculomotor learning and corroborates the claim that saccadic adaptation includes influences that reflect the target's visual properties.

Introduction

Novelty seeking and curiosity are innate human features. They are associated with the big-five personality traits and may be substantial for our evolutionary development. Novel items attract attention and trigger an orientation or approach response (Ranganath & Rainer, 2003; Wittmann, Bunzeck, Dolan, & Duzel, 2007). Several reasons might account for the orientation toward novel stimuli. First, novel stimuli may yield new information. In a visual scene, humans attend to and look at regions that are most informative (Antes, 1974; Mackworth & Morandi, 1967; Yarbus, 1967). Second, novel items present initially a behavioral option with an unknown outcome. This may bias humans to explore what type of outcome this new option might afford (Bunzeck & Duzel, 2006). Third, unexpected and surprising events often merit closer inspection, and provide an important stimulus for adaptation of behavior and learning (Rescorla, 1968). 

The overt reaction to a novel visual target consists of a saccadic eye movement toward the target. Although saccades are considered highly stereotyped movements, recent studies have shown that the content of a visual target can influence the execution of a saccadic eye movement toward it. For example, saccadic peak velocity is increased for saccades toward images of faces (Xu-Wilson, Zee, & Shadmehr, 2009), and latency is reduced (Collins, 2012). The content of a visual stimulus also influences oculomotor learning in saccadic adaptation (Meermeier, Gremmler, & Lappe, 2016). Saccadic adaptation is a process that aims at keeping saccades accurate in case of muscle weakness or other alterations of the oculomotor plant. Saccadic accuracy is key to high-resolution perception of the world, since the center of visual acuity—the fovea—measures only approximately 1° in diameter. In the lab, saccadic adaptation can be triggered using the double-step paradigm (McLaughlin, 1967). The saccade target is stepped during the saccade, inducing an error at the saccade's end. Since saccades are too short for online correction, this error usually triggers a secondary, corrective saccade to bring the target onto the fovea. On consistent repetition of the midflight step, however, the primary saccade will aim better for the new stepped target, thereby increasing or decreasing saccadic amplitude depending on step direction. 

In the present study, we investigate how novelty of a target affects saccadic adaptation. As saccades are usually made in order to look at things, a successful view of the target might be a rewarding situation to the sensorimotor system (Collins, 2012; Collins & Wallman, 2012; Madelain, Herman, & Harwood, 2013; Madelain, Paeye, & Wallman, 2011b; Meermeier et al., 2016). In that sense, it is possible to look at saccades as operant behavior (Madelain, Paeye, & Darcheville, 2011) and the foveal view of the image as a reinforcing event. Saccadic adaptation, in this view, is driven by surprise, or prediction error—that is, the difference between the expected view of the target and the actual view of the target after the saccade. Learning is then basically affected by two parameters: a learning rate describing how fast the behavior can be modified, and the strength of the reinforcer describing how much the behavior can be modified (Rescorla & Wagner, 1972). 

In positive-reinforcement learning, organisms are in pursuit of positive prediction errors (more reward than predicted) instead of negative ones (less reward than predicted; Schultz, 2016). In the brain, dopamine neurons encode very similarly whether there is more or less reward than expected (Pan, Schmidt, Wickens, & Hyland, 2005; Schultz, Apicella, Ljundberg, Romo, & Scarnati, 1993; Zaghloul et al., 2009). They become active upon positive prediction errors and decrease activation upon negative prediction errors (Schultz, Tremblay, & Hollerman, 1998). If an outcome is as expected, they do not show a distinct pattern of activation. During a phase of learning, the activation of the dopamine neurons shifts from shortly after the reward is delivered to the point in time when the earliest predictive cue is given (Schultz et al., 1993). In this way, the functioning of dopamine neurons is coherent with the rationale of reinforcement learning of the Rescorla–Wagner type. As novel stimuli are, by definition, unexpected, novelty of a stimulus is also associated with dopaminergic signaling. The dopaminergic network has been linked to more efficient encoding and increased learning of novel stimuli (Bunzeck et al., 2010; Ranganath & Rainer, 2003; Schultz & Dickinson, 2000). In the present experiment we show that novel visual stimuli produce stronger saccadic adaptation than repetitive stimuli, even when the intrasaccadic target step is the same in both cases. 

Material and methods

Participants

Twenty subjects (mean age [SD] = 21.80 [3.22] years; 17 women, three men) participated. All gave informed consent and had normal or corrected-to-normal vision. Sample size was determined via preliminary testing and adjusted to allow for a full counterbalancing of the experimental design. Experimental procedures adhered to the tenets of the Declaration of Helsinki and were approved by the ethics committee of the Department of Psychology and Sports Science of the University of Muenster. 

Apparatus

Participants sat 57 cm in front of an Eizo FlexScan 22-in. monitor with a visual display size of 40° × 30° of visual angle. Display resolution was 1,152 × 864 pixels at a refresh rate of 75 Hz. Eye position was recorded with an EyeLink 1000 at 1000 Hz. Viewing was binocular, but only the left eye was recorded. A chin rest minimized movements of the head. Experimental code was written in MATLAB (The MathWorks, Natick, MA), and stimuli were presented via the Psychophysics Toolbox extensions (Brainard, 1997). 

Stimuli

Stimuli and procedures followed the study of Meermeier et al. (2016). To investigate the influence of novelty on saccadic adaptation, we compared saccadic adaptation to targets that either always showed the same image or showed a different image in each trial. Images were color photos of women at a size of 50 × 75 pixels (1.76° × 2.63°; Figure 1a). A total of 241 different pictures were used. A further set of 241 random-noise images of the same size was used as masks. 

Figure 1

 

Depiction of the stimulus setup for one trial of counterclockwise scanning with the novel target position in the upper left position. (a) In one session at one position (up left) in every trial, a new stimulus of a human figure is displayed. In the other position, it is always the same picture that is displayed (down right). Colored arrows indicate the direction in which the whole pattern shifted during outward adaptation. Colors illustrate novel stimuli (red) and repeating stimulus (blue). (b) Illustration of the saccade sequence of one adaptation trial. Black arrows illustrate the scan path beginning and ending on the fixation cross (panels 1 and 5). During horizontal saccades in panels 1 and 3, the whole pattern shifts by 4° in the direction of the saccade (panels 2 and 4).

Behavioral task and adaptation procedure

The stimulus arrangement and adaptation procedure allowed for a measurement of adaptation toward both novel and repeating targets in a single session. This is important because differences in overall motivation might occur between sessions. Saccadic adaptation is specific to saccade direction (Deubel, 1987; Wallman & Fuchs, 1998; for reviews, see Hopp & Fuchs, 2004; Pelisson, Alahyane, Panouilleres, & Tilikete, 2010), so adaptation toward the novel target and adaptation to the repeating target occurred independently of each other (see Figure 1a). Sessions were separated by at least 48 hr but preferably more, resulting in on average (SD) 17.6 (10.3) interim days. 

Four stimuli at 12° distance each were positioned in quadratic arrangement around a black fixation cross (2.3° × 2.3°) on a gray background (Figure 1a). At one position a new image was displayed in every trial; in the other position it was always the same image. The remaining two targets were homogenous dark-gray rectangles. Participants were instructed to scan the four stimuli starting and ending on the central fixation cross (Figure 1b). Scanning direction was either clockwise or counterclockwise. The first saccade of the sequence went either to the upper left corner, proceeded by clockwise scanning, or to the upper right corner, with counterclockwise scanning. The target of this first saccade was always a simple gray rectangle. The second and fourth saccades in the sequence were horizontal (Figure 1b, panels 1 and 3). The novel images and the repeating image were targets of these two horizontal saccades. The position of the novel stimulus (right or left side, top or bottom) resulted in four distinct manifestations of the task: two for clockwise and two for counterclockwise scanning. These were counterbalanced across participants. In the second session, subjects made saccades in the same scanning direction but with reversed novel and repeating stimulus. 

The trials started with a central fixation of the fixation cross for at least 300 ms. Then a small red arrow (1.5°) was presented briefly at the fixation cross for 200 ms and indicated the direction of the first saccade. Subjects looked at the stimuli in a fixed sequence but at their own pace. Each stimulus had to be fixated for at least 100 ms, which was controlled via position and velocity criteria on the eye data. If one fixation was left out, the trial was repeated. Repetition of a trial occurred in 4.9% of all trials. These erroneous saccade amplitudes were not included in our analysis. The two horizontal saccades were adapted. For detection of saccade onset, eye position had to exceed a spatial distance of 3° from the current stimulus and eye velocity had to exceed 100°/s for at least four consecutive samples. When saccade onset was detected, the entire stimulus arrangement was shifted 4° in the saccade direction, thereby introducing a consistent postsaccadic error (Figure 1b, panels 2 and 4). The human image was displayed until 200 ms after saccade onset, allowing a brief glimpse of the target. Then it was masked by a noise pattern of equal size. After successful completion of the entire sequence, the fixation cross turned red upon fixation. The next trial started after 1.5 s. 

Procedure

Participants conducted 20 preadaptation trials followed by 200 adaptation trials and 20 postadaptation trials in two separate sessions. The pre- and postadaptation trials were identical to the adaptation trials, with the limitation that the stimuli on the screen remained stationary throughout the trial. Target images were displayed in a pseudorandom order. 

Data analysis

From the recorded eye movements, we analyzed the primary saccades during which the adaptation procedure took place. Primary saccades that were not in the expected direction, or whose amplitudes were either smaller than three standard deviations below the preadaptation trials or larger than three standard deviations above the last 40 adaptation trials were excluded from analysis (3.95% of all trials). 

Amplitude change in percent (AC) was calculated as  

\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\bf{\alpha}}\)\(\def\bupbeta{\bf{\beta}}\)\(\def\bupgamma{\bf{\gamma}}\)\(\def\bupdelta{\bf{\delta}}\)\(\def\bupvarepsilon{\bf{\varepsilon}}\)\(\def\bupzeta{\bf{\zeta}}\)\(\def\bupeta{\bf{\eta}}\)\(\def\buptheta{\bf{\theta}}\)\(\def\bupiota{\bf{\iota}}\)\(\def\bupkappa{\bf{\kappa}}\)\(\def\buplambda{\bf{\lambda}}\)\(\def\bupmu{\bf{\mu}}\)\(\def\bupnu{\bf{\nu}}\)\(\def\bupxi{\bf{\xi}}\)\(\def\bupomicron{\bf{\micron}}\)\(\def\buppi{\bf{\pi}}\)\(\def\buprho{\bf{\rho}}\)\(\def\bupsigma{\bf{\sigma}}\)\(\def\buptau{\bf{\tau}}\)\(\def\bupupsilon{\bf{\upsilon}}\)\(\def\bupphi{\bf{\phi}}\)\(\def\bupchi{\bf{\chi}}\)\(\def\buppsy{\bf{\psy}}\)\(\def\bupomega{\bf{\omega}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\begin{equation}\tag{1}{{{\rm{AC}} = \left( {{A_{{\rm{all}}}} - {A_{{\rm{pre}}}}} \right)} \mathord{\left/ {\vphantom {{{\rm{AC}} = \left( {{A_{{\rm{all}}}} - {A_{{\rm{pre}}}}} \right)} {{A_{{\rm{pre}}}} * {\rm{1}}00}}} \right. \kern-1.2pt} {{A_{{\rm{pre}}}} * {\rm{1}}00}}{\rm .}\end{equation}

 

To quantify the rate of adaptation, we fitted the series of primary-saccade amplitudes during adaptation of each single session with an exponential:  

\begin{equation}\tag{2}y = a{\rm{\ }} + {\rm{\ }}b * {\rm {exp}}\left( { - x/c} \right){\rm .}\end{equation}

 

As a measure of adaptation rate we took 1/c. Since the design was fully counterbalanced, we analyzed subjects' AC values averaged across both sessions. We separated the adaptation phase into five blocks of 40 adaptation trials each. Saccades toward novel and repeating stimuli were measured within each trial and within each subject; that is why we computed a repeated-measures ANOVA with novelty as a factor with two levels and adaptation phase as a factor with five levels. To compare preadaptation trials, we computed paired-samples t tests and Wilcoxon's signed-rank test in cases in which the underlying distribution was skewed. All computations were made with the MATLAB Statistics Toolbox (R2014a). 

Results

We compared saccadic adaptation in a condition in which a novel target image appeared in each trial to a condition in which the same target image was presented in every trial. In the preadaptation trials—that is, before the intrasaccadic target shift was introduced—average saccadic amplitude was 12.30° (SD = 0.33°) toward novel stimuli and 12.36° (SD = 0.42°) toward repeating stimuli, which was not significantly different, t(19) = −1.078, p = 0.29. Further analysis focuses on the computed change of saccadic amplitude (AC). 

A repeated-measures ANOVA of AC revealed a significant main effect of novelty, F(1, 19) = 6.177, p = 0.02, η_p² = 0.245. AC values for the novel targets were on average 9.04% (standard error = 0.72%) and thereby higher than those for the repeating targets (7.95%; standard error = 0.68%). Furthermore, there was a main effect of adaptation phase (Greenhouse–Geisser corrected), F(2.02, 38.28) = 101.722, p < 0.001, η_p² = 0.843. 

This shows that the modulation of saccadic amplitude through saccadic adaptation was effective. The interaction of novelty and adaptation phase was not significant (Greenhouse–Geisser corrected), F(2.80, 53.06) = 1.67, p = 0.19, η_p² = 0.081, indicating that novelty affects the whole adaptation phase. Figure 2 illustrates the data averaged across sessions. 

Figure 2

 

Grand averages of series of saccadic amplitudes in AC. Red triangles represent saccades toward novel targets; blue triangles depict saccades toward the repeating stimulus. Gray background depicts the pre- and postadaptation trials, in which the stimuli remain stationary.

To investigate the temporal progress of saccadic adaptation, we fitted the series of adaptation trials with an exponential function and compared the best-fitting parameters across conditions (Figure 3a). The rate of adaptation did not differ between novel and repeating stimuli (Z = −0.709, p = 0.48; Figure 3b). 

Figure 3

 

Primary saccades' rates of adaptation. (a) Single subjects' exponential fits. Red lines (left plot) are for data toward novel stimuli; blue lines (right plot) are for data to the repeating stimulus. (b) Average rate of adaptation toward novel (red) and repeating stimuli (blue). Lines indicate standard deviation.

Discussion

We were interested in whether novelty of a visual target can increase the amount of saccadic adaptation in comparison to a repeating target. We found that saccadic adaptation toward novel stimuli was stronger than toward a repeating stimulus. Rates of adaptation, however, did not differ between novel and repeating stimuli. 

These results are similar to those of Meermeier et al. (2016), who found an increased amount of adaptation but no difference in learning rate when comparing meaningful images of female human figures with luminance- and spatial-frequency-matched noise. Hence, both comparisons (image to noise and novel to repeating stimuli) resulted in a higher amount of saccadic adaptation. A common ground of both findings might be that in both cases, one target might be more rewarding to look at in comparison to the other one. In the comparison between image and noise, the most pronounced difference is the content of the targets, one being a meaningful stimulus and the other meaningless noise. In the comparison between novel and repeating stimuli, the content of both target categories is equivalent (a human figure), and so are low-level image properties; novelty is the only different factor. 

The nature of the influence of novelty and content in the mechanisms of adaptation is seen in their specific contribution to adaptation magnitude. We did not find a difference in rate of adaptation between the two conditions, nor did we observe an interaction between stimulus type and adaptation phase. Both results show that learning is not faster in the novel condition than in the repeating condition. This is consistent with the findings of our previous study comparing human figures to meaningless noise stimuli, which also showed no difference in rate of adaptation. In both studies, the difference between conditions occurred with respect to the amount of adaptation, not the speed. According to current theories of learning, the most basic of which are of the Rescorla–Wagner type, learning is affected by both a rate constant and the strength of the reinforcer. The latter basically sets the asymptote of the learning curve. Situations in which reinforcers of different strength are used under the same learning rate produce a larger amount of learning but a progression with the same speed. In this case, a stronger effect is observed throughout the learning process in the condition with the stronger reinforcer, and there should be no interaction with speed. This is very much what we observed. In addition, and consistent with this explanation, the difference in amplitude is specifically present during adaptation and not during the preadaptation trials. In the framework of reinforcement learning we should therefore consider the novel stimulus a stronger reinforcer than the repeating stimulus. 

These results lend further support to the recent discussion that saccadic adaptation involves more than a simple, automatic, error-based, low-level motor learning process (Collins & Wallman, 2012; Herman, Blangero, Madelain, Khan, & Harwood, 2013; Levy-Bencheton, Pisella, Salemme, Tilikete, & Pelisson, 2013; Madelain, Paeye, & Wallman, 2011; Meermeier et al., 2016; Panouilleres et al., 2014; Schütz & Souto, 2015; Zimmermann & Lappe, 2016). The top-down modulation on adaptation of scanning saccades observed in the present study may originate from frontal cortical areas or from the basal ganglia. Dopamine neurons become active when an outcome is better than expected, thereby unifying both the surprise and the reward components of novelty (Schultz, 1998). The basal ganglia, with their dopaminergic signaling, are involved in eye-movement control as well as reward-based learning, and modify eye-movement vigor to rewarding stimuli (Hikosaka, Kim, Yasuda, & Yamamoto, 2014; Wittmann et al., 2007). An involvement of the basal ganglia in saccadic adaptation has not been researched systematically, but evidence from an individual with Parkinson's disease suggests a possible contribution (MacAskill, Anderson, & Jones, 2002). Still, insight into the role of basal ganglia in saccadic adaptation is incomplete. The current findings might also be explained by the framework of corticothalamic-cerebellar loops (Ide & Li, 2011) or with the idea of the cerebellum as a monitoring instance in the brain (Peterburs & Desmond, 2016). 

We conclude that novelty affects saccadic adaptation and that a clear view of a novel target might be more rewarding to the subject than a clear view of an old target. However, the notion of reward is a very complex and idiosyncratic issue. There are many possible variables that have to be considered: dispositions and personal preferences of the subject, which contribute to the value of the reinforcer, as well as the subject's recent history and experiences, which in turn form reward expectations (Killeen & Jacobs, 2016). Furthermore, features of the stimulus including its content, social importance, and salience have relevant effects on behavior. Our study suggests that novelty might be one important aspect in this equation, and that the rewarding character of high-resolution vision might be important in saccadic behavior. 

Acknowledgments

This study was supported by DFG La 952/6-1. 

The authors declare no competing financial interests. 

Commercial relationships: none. 

Corresponding author: Annegret Meermeier. 

Email: . 

Address: University of Muenster, Muenster, Germany.