Discriminative control of saccade latencies

Abstract

Recent studies have demonstrated that saccadic reaction times (SRTs) are influenced by the temporal regularities of dynamic environments (Vullings & Madelain, 2018). Here, we ask whether discriminative control (i.e., the possibility to use external stimuli signaling the future state of the environment) of latencies in a search task might be established using reinforcement contingencies. Eight participants made saccades within 80–750 ms toward a target displayed among distractors. We constructed two latency classes, “short” and “long,” using the first and last quartiles of the individual baseline distributions. We then used a latency-contingent display paradigm in which finding the visual target among other items was made contingent upon specific SRTs. For a first group, the postsaccadic target was displayed only following short latencies with leftward saccades, and following long latencies with rightward saccades. The opposite was true for a second group. When short- and long-latency saccades were reinforced (i.e., the target was displayed) depending on the saccade direction, median latencies differed by 74 ms on average (all outside the 98% null hypothesis confidence intervals). Posttraining, in the absence of reinforcement, we still observed strong differences in latency distributions, averaging 64 ms for leftward versus rightward saccades. Our results demonstrate the discriminative control of SRTs, further supporting the effects of reinforcement learning for saccade. This study reveals that saccade triggering is finely controlled by learned temporal and spatial properties of the environment using predictive mechanisms.

Introduction

Because of the physical constraints of causation, environmental events occur in an orderly temporal sequence. Organisms are equipped to take advantage of these environmental regularities to learn about relations between events and behave in anticipation of what is about to happen. Both respondant—also termed classical or Pavlovian—and operant—also termed instrumental or Skinnerian—conditioning rely on this ability to learn cues signaling future events. For instance, eyeblink conditioning provides an illustration of this type of prediction. The repeated presentation of a 1000 Hz tone 750 ms before a gentle puff of air is delivered toward the eye induces a rapid increase in the proportion of trials in which an eye blink is triggered by the tone from about 10% to 80%. In contrast, the proportion of trials with a conditioned response, i.e., blinking at the tone onset, remains unchanged if the tone and the air puff is not paired (Ivkovich, Collins, Eckerman, Krasnegor, & Stanton, 1999). The 5-month-old infants from the experimental group learned to predict the occurrence of the air puff using the tone onset based on the repeated association of the two events. It is noteworthy that our use of the term “prediction” describes the fact that a cue signaling the future state of the environment comes to control the behavior. It differs from another use of prediction describing how information about the future state of the environment might be extracted from the current state of the world, possibly relying on computation. Importantly, respondant conditioning has been formalized in an influential model based on prediction errors proposed by Rescorla and Wagner (Rescorla & Wagner, 1972; Wagner & Rescorla, 1972; see Jozefowiez, 2018 for a related discussion). It is noteworthy that this theoretical view has proved to be fruitful at the neurophysiological level (Schultz, 2015) or in the field of reinforcement learning in artificial intelligence (Sutton & Barto, 1998). Operant conditioning may similarly be viewed as predictive learning, although the organism's response is, in this case, necessary to produce the outcome: Should reinforcement be available whenever a specific visual stimulus is displayed, pigeons will quickly learn to peck in the presence of this stimulus and not in its absence (e.g., Reynolds, 1961). Here, the organism learns to predict that food will be delivered if the key is pecked in the presence of the stimulus. This general ability to use the presence of cues signaling the future state of the environment, termed “discriminative control,” is particularly remarkable in the case of sensorimotor control as it provides a means to adapt movements to future consequences. 

Saccadic eye movements are used in everyday life to acquire information from our visual surroundings by placing the retinal image of an object of interest on the high-acuity region of the retina, the fovea. Maintaining saccade accuracy despite possible weakening of the extraocular muscles or changes in the saccadic circuitry is therefore essential for efficient visually guided behaviors. Learning in the saccadic system has been repeatedly demonstrated by the use of a double-step paradigm inducing saccade adaptation. In these experiments, a postsaccadic position error is introduced by surreptitiously shifting the visual target backward during the saccade (McLaughlin, 1967): Saccade amplitudes progressively decrease over the course of the experiment. Importantly, this phenomenon is also observed in natural settings, as masking the good eye in patients with paretic eye induces an increase in saccade amplitude in the weak eye, which previously undershot visual targets (Abel, Schmidt, Dell'Osso, & Daroff, 1978; Kommerell, Olivier, & Theopold, 1976). Saccade adaptation has been extensively studied and it is established that the changes in saccade amplitude result from a true learning rather than a simple recalibration of the saccadic system (Herman, Blangero, Madelain, Khan, & Harwood, 2013; Hopp & Fuchs, 2006; Pélisson, Alahyane, Panouillères, & Tilikete, 2010). Moreover, saccade adaptation might be placed under discriminative control of contextual features such as target eccentricity and depth (Chaturvedi & Van Gisbergen, 1997), horizontal and vertical orbital eye position (Alahyane & Pélisson, 2004; Havermann, Zimmermann, & Lappe, 2011; Shelhamer & Clendaniel, 2002), head orientation (Shelhamer & Clendaniel, 2002; Shelhamer, Peng, Ramat, & Patel, 2002), or target motion (Azadi & Harwood, 2014). In these experiments, two different contexts are associated with two different intrasaccadic steps during the same session, and a discriminative control is revealed when saccades produced in the two contexts can be adapted differently. For instance, one might induce a gain increase by introducing a forward step if the saccade target is moving clockwise while simultaneously inducing a gain decrease by introducing a backward step if the saccade target is moving counterclockwise (Azadi & Harwood, 2014). Participants learn that the direction of the target movement signals the direction of the intrasaccadic step and adjust the saccade gain accordingly to reduce the postsaccadic position error. 

Because of their primary visual function, saccades are usually viewed as mainly concerned with spatial position and the temporal dimension is typically regarded as holding a limited role: Saccadic latencies are conventionally thought of as reflecting the accumulation of information during decision-making process (see Gold & Shadlen, 2007 for a review). Indeed, most models of decision making, such as the LATER (linear approach to threshold with ergodic rate) model (Carpenter & Williams, 1995; Genest, Hammond, & Carpenter, 2016; Noorani & Carpenter, 2016) or diffusion models (Ratcliff & Rouder, 1998; Ratcliff, Smith, Brown, & McKoon, 2016) are based on the hypothesis of some noisy accumulation of information to decision criterion. An important feature of these accumulation models is that saccadic latency is supposed to reveal the time needed to reach a decision regarding the saccade target location; one would have a precise control over where to move the eyes but not when. Departing from this long-lasting information accumulation hypothesis, it has been proposed that saccades are driven by the visual information available within the first 100 ms after target onset (Ludwig, 2009; Ludwig, Gilchrist, McSorley, & Baddeley, 2005). Because saccadic reaction times (SRTs) are typically longer than 100 ms, the ability to control latencies would be functional, as it might be a way of prioritizing eye movements: Saccade latencies would result from a cost-benefit tradeoff in which both the benefit of a saccade—typically the enhanced visual information at a new location—and the cost associated with it—such as the temporary impairment of vision or commitment cost—are evaluated when triggering a movement (Harwood, Madelain, Krauzlis, & Wallman, 2008). This cost-benefit weighting should therefore result in changes in saccadic latency distributions depending on the actual benefit of making a saccade. 

The possibility that saccade latencies might be actively regulated is supported by results demonstrating the temporal adaptation of saccade allocation in response to the environmental contingencies in force. Indeed, saccadic latency distributions might be extensively manipulated in the laboratory using either a biofeedback (Bibi & Edelman, 2009) or dedicated reinforcement schedules (Madelain, Champrenaut, & Chauvin, 2007; Vullings & Madelain, 2018). Moreover, shorter latencies demanded by urgency (e.g., Montagnini & Chelazzi, 2005) or longer latencies due to reduced visual benefits of saccades (e.g., Harwood et al., 2008; Madelain, Krauzlis, & Wallman, 2005) have been reported, revealing the ability to learn when to saccade. This temporal learning is incidentally thought to be at play outside the laboratory with sportsmen (Ceyte, Lion, Caudron, Perrin, & Gauchard, 2017) or video-gamers (Chisholm & Kingstone, 2015). Recently, Vullings and Madelain (2018) have investigated how the temporal organization of the environment constrains the temporal allocation of saccades. They found that participants were able to adjust their SRT distributions depending on the programmed reinforcement contingencies in force: More short-latency saccades were observed when they were more likely to yield a reinforcer while more long latencies were recorded when contingencies were such that reinforcers were more frequent for longer SRTs. 

An important outcome of this research demonstrating the adaptability of SRTs is that specific latencies may be reinforced. This type of conditioning implies that it should be possible to place saccade latencies under discriminative control. To investigate the discriminative control of saccadic latencies, we trained participants to have short latencies when they made saccades toward one side of the visual field and long latencies for the other side. We developed a novel latency-contingent paradigm in which finding the target among distractors in a visual search task depended on the latency of the current saccade. Our results demonstrate that it is possible to place saccadic latencies under discriminative control using visual consequences as reinforcement. 

Methods

Participants

Eight adults (18–26 years old, four females) participated in this research. They were naive as to the purpose of the study and had normal or corrected-to-normal vision. Participants were instructed to find with one saccade a target stimulus among items displayed on an imaginary circle. To encourage participants to carry on the task, they were told they would earn a point every time they found the target stimulus; no further explanation was given as to how to earn points. When the experimental conditions changed, the same instruction was given again. Participants received 10 euros for participating, plus an additional sum depending on the points collected (one point equaled 2 cts); they received 74 euros on average at the end of the experiment. All experimental procedures received approval from the Ethical Committee in behavioral sciences of the University of Lille (Agreement n°2017-2-S50) and conformed to the standards set by the Declaration of Helsinki. All participants gave informed written consent. 

Apparatus

Stimuli were generated using the Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997) for MATLAB (MathWorks, Natick, MA) and displayed on a video monitor (Iiyama HM204DT, 100 Hz, 22 in.). Participants were seated on an adjustable stool in a darkened, quiet room, facing the center of the computer screen at a viewing distance of 60 cm. To minimize measurement errors, the subject's head movements were restrained using a chin and forehead rest, so that the eyes in primary gaze position were directed toward the center of the screen. Viewing was binocular, but only the right eye position was digitized in both the vertical and horizontal axes. Eye movements were measured continuously with an infrared video-based eye tracking system (Eyelink, SR Research Ltd.), sampled at 2000 Hz. Data were transferred, stored, and analyzed via programs written in MATLAB running on an Ubuntu Linux computer. 

Stimuli were light gray 2.2° lines (luminance = 15.99 cd/m²) displayed on a dark gray background (luminance = 1.78 cd/m²). The 12 stimuli were arranged on an imaginary circle (with a radius of 9° of visual angle), forming an 18° rotated clock face, such that half of the items were on the left side of the screen and the other half on the right side (Figure 1). Out of the 12 items, the target stimulus was a horizontal line (Figure 1B) flanked by four distractors (lines tilted by −70°, −20°, 20°, and 70° from the horizontal) and the seven other irrelevant items were vertical lines. 

Figure 1

 

Illustration of the experimental design. (A) Experimental design of an ongoing trial during baseline. i) The fixation cross appears at the center of the screen for a bout varying from 750 to 1250 ms. ii) The target and distractors are displayed for 30 ms. iii) A neutral arrangement is displayed until a saccade is emitted, to which latency has to be comprised between 80 and 750 ms. iv) Squared patches of noise are displayed at the stimulus locations for 10 ms while the eyes are moving. v) The same stimulus organization as ii) is displayed for 500 ms. vi) A neutral arrangement is displayed for 200 ms. (B) The target was always a horizontal line and the four flankers were tilted lines. (C) Illustration of the latency-contingent display in a leftward experimental trial. The top right-hand corner panel represents the latency-contingent display (Figure 1A v.) in the instance of the saccadic latency being within the leftward criterion. The bottom right-hand corner panel represents the latency-contingent display (figure 1A v.) in the instance of the saccadic latency being outside the leftward criterion. The blue dashed lines represent the eye location and the orange circle highlights the target position. (D) Probability of the eyes to land on the target position during baseline for the best and worst performances.

Before each experimental session, we calibrated the eye tracker by having the subject fixate a set of 13 fixed locations distributed across the screen. Every 50 trials, subjects looked at a target displayed on the center of the screen for a 1-point calibration drift check. 

Procedure

The experiment lasted thirty 384-trial sessions divided between baseline and reinforcement sessions. Two daily sessions were typically conducted, separated by 5-min breaks during which participants were free to move. The experiment lasted 15 consecutive days (5 days a week, from Monday to Friday). Regardless of the actual condition, participants were required to make a saccade toward the location at which they thought the target stimulus was. Our general goal was to create a situation in which finding the target stimulus was made contingent upon specific saccadic latencies. 

Baseline

Four 384-trial baseline sessions were completed, in which participants were instructed to find the target stimulus, i.e., the horizontal line, with their first saccade. At the beginning of the trial, the participant looked at the fixation cross displayed at the center of the screen for a period varying randomly from a uniform distribution between 750 and 1250 ms (Figure 1A, i). The target arrangement, consisting in the horizontal target stimulus pseudorandomly assigned to one of the 12 possible locations (Figure 1A, ii, and Figure 1C) flanked by the four distractors and seven vertical lines, was then presented for 30 ms. Only five items were not vertical lines in order to cue the area where the target stimulus was displayed. Immediately after this 30-ms bout, we displayed a neutral arrangement that consisted of 12 vertical lines, and the participant made a saccade (Figure 1A, iii). The saccade latency had to be between 80 and 750 ms (this range was defined based on latencies measured in pilot studies). Following the saccade offset, we displayed a mask consisting in 12 rectangular noise patches at the stimulus locations for one frame (i.e., 10 ms; Figure 1A, iv), then the target arrangement (i.e., the target stimulus, four flankers and seven irrelevant vertical lines at the same location as it was at the beginning of the trial) for 500 ms (Figure 1A, v). Finally, the neutral arrangement (i.e., the 12 vertical lines) was displayed again for 200 ms (Figure 1A, vi). On canceled trials, the target arrangement (Figure 1A, v) was not shown and the 12-vertical lines neutral arrangement was shown for 700 ms instead (Figure 1A, vi). A trial was cancelled in four possible cases: (a) the latency was shorter than 80 ms or longer than 750 ms, (b) the saccade direction was such that the saccade endpoint was located between two items, (c) the participant made a saccade towards the opposite side of the screen with respect to the target stimulus location, or (d) no saccade was detected. Saccadic latency was defined as the interval of time elapsed between the target onset and the saccade onset. Figure 1D plots the probability of the postsaccadic eye position to be at one of the 12 item locations as a function of the target stimulus location during baseline for the best and worst individual baseline performances. If the participants were able to systematically find the target, the diagonal would be red (p = 1). As it is shown, the probability of landing on target was quite low (on average p = 0.3) but the participants approximately aimed at either the target stimulus or its two closest flankers. 

After completion of the baseline, we constructed two individual latency classes—“short” and “long” saccadic latencies—using, respectively, the first and last quartiles of the baseline latency distributions for each participant. These individual class boundaries were fixed for the remainder of the experiment. Latencies comprised between 80 ms and the first quartile were categorized as short latencies (i.e., the shortest 25% of all baseline SRTs; e.g., 80–293 ms for S1) and latencies comprised between the last quartile and 750 ms were categorized as long latencies (i.e., the longest 25% of all baseline SRTs; e.g., 430–750 ms for S1). The first and last quartiles were 293 and 430 ms, 236 and 321 ms, 222 and 304 ms, 255 and 432 ms, 243 and 352 ms, 241 and 287 ms, 222 and 286 ms, and 230 and 298 ms for participants S1 to S8, respectively. One could point out that these latencies appear to be quite long, a fact that might be attributed to our paradigm in which the fixation point was always displayed (Figure 1A) leading to an overlap configuration, which has been shown to significantly delay saccades (e.g. Saslow, 1967) 

Reinforcement sessions

In all reinforcement sessions, short and long latencies were independently reinforced on a multiple schedule of continuous reinforcement using a latency-contingent display, in which finding the target was made contingent upon specific ranges of latencies. In continuous schedules, reinforcement is systematically delivered contingent on a correct response—in our case, the short or long latencies. In a multiple schedule, different schedules of reinforcement are in effect during different trials signaled by particular stimuli or contexts—in our case, the target stimulus being located on either the left or right side of the screen. We aimed at training participants to have one class of latencies for the leftward target stimulus locations and the other one for the rightward target locations: For Group 1, the leftward-trial criterion required short latencies and the rightward-trial criterion required long latencies; the opposite was true for Group 2. 

The latency-contingent display used for reinforcement consisted in displaying the target at the postsaccadic eye location upon specific latencies. Figure 1C shows an instance of the latency-contingent display for Group 1, in which the target stimulus is initially located on the left, thus requiring a short latency. If the SRT was short (as in the top right-hand corner), the 12 items were arranged such that the target was displayed at the postsaccadic eye location flanked by the four distractors. If the SRT was too long (as illustrated in the bottom right-hand panel), the 12 items were arranged such that the target was located one position away from the postsaccadic eye location, either clockwise or counterclockwise. 

Using this latency-contingent display, the participant could see the target at any of the six left item locations he would be saccading to, providing the saccade latency was short. To illustrate the procedure, we will use the case of a leftward-trial for participant S2 as an example. S2 was assigned to Group 1, requiring short latencies (i.e., in his case, any latency between 80 and 236 ms) for the leftward-trial criterion and long latencies (i.e., in his case, any latency between 321 and 750 ms) for the rightward-trial criterion. On a given trial, the initial target arrangement was such that the target stimulus was placed at position 12. If participant S2 made a saccade to position 11, with a latency of 230 ms, we would then arrange the items with the target stimulus at the postsaccadic location, in this case, location 11. This would be true for any of the postsaccadic eye locations between position 7 and 12. The participant would therefore find the target stimulus for any saccade directed toward the correct side of the screen (i.e., the left side in our example) with a criterial latency. Finding the target stimulus would then be signaled by a brief auditory feedback tone (100 ms, 500 Hz), and the participant would earn a point, which would be exchanged for 2 cts at the end of the experiment. However, if the participant S2 made a saccade toward location 11 with a latency of 237 ms or longer, we would rotate the items' arrangement by 30° either clockwise or counterclockwise with respect to position 11 such that the target stimulus would instead be randomly placed at either the location 10 (as in Figure 1C, bottom right panel) or 12. In other words, the participant would not see the target stimulus at the postsaccadic eye position for saccades with noncriterial latencies. 

We used three different types of reinforcement sessions, manipulating the initial location of the target between the left and right sides of the screen. 

Massed sessions

For discrimination training, we first massed trials on one side of the screen (leftward-trial sessions followed rightward-trial sessions). 

Alternating block sessions

Then, we alternated blocks of consecutive leftward/rightward trials and progressively decreased their length (24-, 12-, six-, and three-trial blocks). For instance, in a 12-trial block session the participant experienced 12 leftward trials followed by 12 rightward trials and so on until 384 trials were completed. 

Interleaved session

The last sessions of reinforcement consisted in pseudorandomly interleaved leftward and rightward trials. 

Retention probe session

Finally, we conducted a retention probe session, in which the latency-contingent display was withdrawn. Therefore, in these sessions the trials were identical to the baseline ones (see Figure 1A). 

The total number of 384-trial reinforcement sessions completed in each step depended on the time needed to reach stable reaction time distributions. We simultaneously used two criteria to decide when to advance the experiment from one type of reinforcement session to the next. The first learning criterion used the median, first, and last quartiles of the saccadic latency distribution for the last session with the requirement that these statistics for the leftward saccades should be different (outside the 98% CIs of the null hypothesis) from those of the rightward saccades. Our second criterion used the same statistics for the last two sessions with the requirement that these statistics for the leftward and rightward saccades should both be different (outside the 98% CIs of the null hypothesis) from those of the baseline sessions. 

Acquisition and data analysis

Eye movements were recorded and measured throughout each trial. For online saccade detection, we used the Eyelink® online saccade detector to identify saccade onset and offset, using a 30°/s velocity and 8000°/s² acceleration thresholds. Saccade parameters were retrieved on average with a 12-ms delay after saccade offset. For offline analyses, a human observer first validated each saccade manually; we discarded the saccades with amplitude gain shorter than 0.5 or for which the distance angle between the eye and the stimulus position was greater than 10°. On average, we kept 74% (SD = 14) of saccades per participant. 

We used bootstrapping methods (resampling with replacement 100,000 times) to estimate all the individual statistical parameters and 98% CIs (Efron, 1979). Medians were compared using Fisher's exact text with 100 000 permutations. 

Results

Figure 2 represents the individual difference in latencies for leftward versus rightward saccades for the two groups of participants during the baseline session, last session of reinforcement, and retention probe session. During baseline (Figure 2A), although two participants showed a latency bias, there was no significant consistent difference between the two sides across participants (absolute mean difference = 14 ms). During reinforcement (Figure 2B), we were expecting a negative difference for the Group 1 (as we reinforced short latencies for leftward saccades and long latencies for rightward saccades) and a positive difference for the Group 2 (as we reinforced long latencies for leftward saccades and short latencies for rightward saccades). We observed a large difference in latencies between leftward and rightward saccades (absolute mean difference of 74 ms; all values greater than the null hypothesis 98% CIs). This difference was systematic across participants and in the right direction for both groups, except for participant S6 for whom the difference was in the opposite direction: This participant exhibited a positive latency difference even though he was assigned to Group 1, in which the leftward saccade latencies should have been shorter than the rightward ones. Finally, during retention—that is when the target location was no longer contingent upon the latency (Figure 2C)—five participants had significant differences that were in the right direction (S1–S5, on average 95 ms; all values greater than the null hypothesis 98% CIs). Three participants had no significant difference (S6, S7, and S8, on average 12 ms; all values within the null hypothesis 98% CIs). Overall, the participants maintained a 64-ms difference in the probe retention session. 

Figure 2

 

Differences in median latencies for leftward versus rightward saccades in baseline (A), reinforcement (B) and retention probe session (C). Groups 1 and 2 are represented in blue and green, respectively. The corresponding 98% confidence intervals of the null hypothesis (i.e., the two saccade directions have identical probability distribution) are shown.

Figure 3A shows the boxplots of the saccadic latency distribution for the leftward and rightward saccades (in green and blue, respectively) during the same interleaved-location sessions for one representative participant (S3, in Group 2). During baseline, there was no difference in latencies between leftward and rightward saccades (i.e., 249 ms for the left and 257 ms for the right; 8-ms difference, within the null hypothesis 98% CI). During reinforcement, when the target location was randomized across trials, we observed a large difference in latencies between the two directions with longer latencies (median = 486 ms) for leftward saccades and shorter ones (median = 386 ms) for rightward saccades (100-ms difference, greater than the null hypothesis 98% CI). Finally, during the retention probe session, we still observed a large latency difference between the two directions (i.e., 324 ms for leftward saccades and 278 ms for rightward saccades; 46-ms difference, greater than the null hypothesis 98% CI). Interestingly, we observed large differences in saccadic latency distributions after learning, but also an increase of the rightward saccade latencies, which were reinforced to be shorter. Figure 3B breaks down the effect on saccadic latencies for the same participant across the phases of the experiment. We plot, on the left-hand panel, the percentage of trials in which the target was found, and on the right-hand panel the quartiles of the respective saccadic latency distributions for the baseline, the reinforcement 24-, 12-, six- and three-trial alternating blocks, the reinforcement interleaved-location session, and the retention probe session. First, it appears that the percentage of trials in which the target was found was always higher for the leftward saccades (on average 57.60%) than for the rightward saccades (on average 29.53%). Since participant S3 was in Group 2, this implies that long latencies were reinforced almost twice as often as short latencies. Second, we can observe that there were two kinds of effect on saccadic latency distributions throughout the time course of the experiment. On the one hand, the difference in latency was due to a shift toward shorter values for rightward saccades combined with a shift toward longer values for leftward saccades (as in the 24-, 12-, six- and three-trial alternating block sessions): The percentage of rightward trials in which the target was found was on average 37.94%. On the other hand, the difference in latency was due to a shift toward longer values for rightward saccades and a shift toward the longest values for leftward saccades (as in the reinforcement interleaved-location session): The proportion of finding the target in rigthward trials fell to 1.34%. Interestingly, in spite of the fact that the short latencies were almost never reinforced, the difference in the saccadic latency distributions maintained. This trend was also found with the other participants, for whom long latencies were much more frequently reinforced than short latencies (see Table 1). 

Figure 3

 

(A) Boxplots of the saccadic latency distribution for the leftward and rightward saccades (in green and blue, respectively) during baseline, reinforcement and retention probe session for a representative participant (S3 in Group 2). (B) Left-hand panel: the percentage of trials in which the target was found for leftward and rightward saccades (in green an blue, respectively). Right-hand panel: quartiles of the lefward and rightward saccadic latency distributions for the baseline, the reinforcement 24-, 12-, 6- and 3-trial alternating blocks, the reinforcement interleaved-location session and the retention probe session.

Table 1

 

Proportion of trials in which the target was found in leftward and rightward trials for all participants during baseline, reinforcement and retention. Note: For the interleaved reinforcement session, the letter indicates whether short (S) or long (L) latencies were reinforced.

 

Discussion

The present study introduced a novel latency-contingent paradigm to investigate discriminative control over latencies in a search task using visual reinforcement. Saccadic latency distributions were considerably affected by visual consequences and we observed significant latency differences between leftward and rightward saccades. These results provide evidence that saccadic latencies might be placed under discriminative control and further reveal the extent of reinforcement learning for saccades and sensorimotor prediction. 

Inducing and maintening discriminative control of latencies

The procedure we used induced differences in latencies between leftward and rightward saccades such that saccades were on average 74 ms longer for one direction than for the other. In seven out of our eight participants, this difference was in the right direction, revealing that SRTs changed according to the reinforcement contingencies. However, these differences required some training to install: Participants first experienced massed training and then alternating block sessions in which they had to perform several saccades toward the same visual hemifield in a row. In these sessions, the direction of the saccade was perfectly predictable and participants learned to perform both short and long latency saccades according to the saccade direction. When leftward and rightward trials were interleaved we found that all but one participant (i.e., S6) did produce long latencies for the direction associating reinforcement with longer latencies but increased their latencies for the other direction to the point that these saccades were almost never reinforced (see Table 1 and also Figure 3, which illustrates this effect for one participant). This indicates that, although our procedure induced strong bias in latencies depending on the saccade direction, participants had a limited control over their shorter latencies in this last reinforcement session (see Nevin, Cate, & Alsop, 1993 for similar effects). As we have previously found, the effects of reinforcement on latencies develop over time and require quite a large number of trials to stabilize (e.g., Madelain et al., 2007; Vullings & Madelain, 2018, see their figure 2A), and one might argue that with a longer training period participants might have increased their reinforcement rate. Our paradigm may be regarded as a dual-task in which identifying the appropriate context interferes with the saccade task itself, a possibility that could explain the performance level. More generally, the necessity for a long training might be due to three possible sources of uncertainty in SRT control: the perception of the actual reinforcement contingency (Davison & Nevin, 1999; Gallistel, Mark, King, & Latham, 2001; Killeen & Smith, 1984), the perception of time—here the perception of one's own reaction time (Wearden, 2016), and the SRT production itself (Sumner, 2011). Although further research is necessary to disentangle these three sources of noise, the fact that the control of short latencies was stronger in blocked than in interleaved sessions seems to point toward a noisy discrimination of the contingencies. In the blocked sessions, participants might have taken advantage of the increased predictability due to serial dependencies across trials, whereas in the interleaved session they could only rely on the appearance of the target to adjust their SRTs. In both cases, this implies discriminative control over saccade latencies, but it could be that serial dependency is a more powerful contextual cue than saccade direction, particularly for short-latency saccades. 

Five out of eight participants (S1–S5) exhibited a significant difference in latencies during the retention probe session while the remaining two participants who also exhibited some discriminative control in the interleaved session did not (S7 and S8; see Figure 2C). The fact that not all participants showed retention is not surprising since it is commonplace to extinguish the stimulus control when reinforcement is withdrawn (Staddon, 2016)—in our case the removal of the monetary reinforcer associated with finding the visual target. In spite of not getting any monetary reinforcer, five participants still responded differentially as a function of the saccade direction, indicating that the discriminative control was firmly established in their case. Another factor that might have contributed to washing out the discriminative control in participants S7 and S8 could be that, in the retention session, any latencies were potentially reinforced by finding the visual target. Indeed, during the retention probe session, we implemented a return-to-baseline, in which the postsaccadic target was displayed at the same location as the presaccadic one regardless of the saccade latency (see Figure 1A). This means that participants could find the target even if their saccadic latency was not one previously reinforced (the percentage of trials in which the target was found was 21.52% and 65.06% for participants S7 and S8, respectively). Therefore, one might argue that the absence of contingency between the latency and the ability to find the target contributed to reducing the SRT differences across leftward and rightward saccades. Finally, in some trials of the retention probe session, the saccadic latencies were much longer than the median latency and may not be representative of what was happening throughout the whole session. Indeed, because we used a quite large latency range (80–750 ms), some participants presented a progressive increase in latencies (see Figure 3B). One might wonder whether a 750-ms saccade latency is still a regular reaction time (Ratcliff, 1993), whether it depends on the same underlying process as a 150-ms latency, or whether there was a change in the participant's strategy. Interestingly, if we only consider the trials with latencies under 500 ms, the differences in latencies for these two participants become significant (all values outside the 98% null hypothesis CIs): 32.38 ms (S7 in Group 2) and −74.56 ms (S8 in Group 1). Thus, the data from these participants is consistent with a trend in favor of stimulus control. 

Previous research concluded that predicting a reward tends to reduce saccadic latencies, with mean SRT being systematically shorter in rewarded than in nonrewarded condition (e.g., Dunne, Ellison, & Smith, 2015; Glaser et al., 2016; Takikawa, Kawagoe, Itoh, Nakahara, & Hikosaka, 2002; Watanabe, Lauwereyns, & Hikosaka, 2003). This was not the case in our experiment and the fact that participants collected more reinforcers for saccades requiring longer latencies indicates that reward expectancy does not necessarily imply a decrease in saccade latencies. One key specificity of our procedure is that we established a latency-contingent reinforcement program so that reward expectancy was associated with both longer and shorter SRTs depending on the saccade direction. Previous research using a latency-contingent paradigm also found that there is no systematic relation between reward expectancy and reduced latencies (Madelain et al., 2007; Vullings & Madelain, 2018). One should therefore conclude that the SRTs truly adapt to the actual contingency in force rather than being reduced by the outcome expectation. 

Reinforcers

Previous studies have investigated the impact of reinforcement learning on gaze control both in experimental and natural settings (for reviews, see Glimcher, 2003; Hayhoe & Matthis, 2018; Madelain, Paeye, & Darcheville, 2011; Trommershäuser, Glimcher, & Gegenfurtner, 2009). Most learning experiments use extraneous consequences as reinforcers for saccades (e.g., food or liquid with nonhuman animals or money with humans), while the real-life benefit of saccades is associated with changes in the visual perception of the targeted object. It has been well documented that extraneous reinforcers such as monetary profits affect saccade target selection (e.g., Chen, Mihalas, Niebur, & Stuphorn, 2013; Liston & Stone, 2008; Lou, Hsu, & Sajda, 2015; Schütz, Trommershauser, & Gegenfurtner, 2012; Stritzke, Trommershäuser, & Gegenfurtner, 2009) and influence saccade latencies (e.g., Madelain et al., 2007; Milstein & Dorris, 2007; Rothkirch, Ostendorf, Sax, & Sterzer, 2013; Vullings & Madelain, 2018). However, limited effects have often been reported when using arbitrary reinforcement for saccade latency (Madelain et al., 2007; Vullings & Madelain, 2018) or saccade amplitude (Madelain, Paeye, & Wallman, 2011; Meermeier, Gremmler, Richert, Eckermann, & Lappe, 2017). Because biological constraints are known to have a substantial impact on operant conditioning (e.g., Domjan & Galef, 1983), one explanation for these effects is that the extraneous reinforcing consequences commonly used so far might be less biologically relevant than the ones obtained in natural settings (i.e., foveating the targeted object). In line with the fact that acquisition of information is reinforcing (Wyckoff, 1952), some studies demonstrated that visual consequences could as well act as a reinforcer for saccadic amplitude (Madelain, Paeye, & Wallman, 2011; Meermeier et al., 2017; Paeye & Madelain, 2014). It has also been shown that humans exhibit shorter latencies when the target remains visible after the saccade than when it is systematically extinguished (Collins, 2012). Furthermore, Montagnini and Chelazzi (2005) elegantly demonstrated that making the visual perception of a target contingent on short latencies has a dramatic effect on saccadic latency distributions in a visual-discrimination task. Taken together these results imply that saccades are reinforced by the ability to carry on a visually guided task. That a reinforcer might be a behavior rather than an object (such as food or money) has been established before (e.g., Timberlake, 1995; Timberlake & Allison, 1974). In the case of saccades, the outcome is information: Saccades enhance the perceived high spatial frequency of a visual object by placing its retinal image on the center of the retina. One could argue that whether this information is meaningful depends on the flow of activities in which the organism is engaged. In other words, changing what one sees constitutes a favorable outcome if the ongoing task benefits from this additional information. Paradigms such as ours (see also Montagnini & Chelazzi, 2005) attempt to mimic this natural state of affair by establishing relations between a specific dimension of saccadic eye movements, in our case their latency, and the ability to perform a visually guided task. That these relations come to change the saccades reveal the reinforcing effect of these outcomes. 

Saccadic latencies and discriminative control

Participants had either a positive or negative latency differences between leftward and rightward saccades depending on the reinforcement contingencies (Figure 2). Although it has already been shown that saccade latencies could become shorter (e.g., Ikeda & Hikosaka, 2007; Lauwereyns, Watanabe, Coe, & Hikosaka, 2002; Montagnini & Chelazzi, 2005; Watanabe et al., 2003) or longer (e.g., De Vries, Azadi, & Harwood, 2016; Harwood et al., 2008; Madelain et al., 2005) depending on the contingencies in force, our study is to our best knowledge the first demonstrating that one can induce a discriminative control of saccadic latencies using reinforcement. The discriminative control of behavior, demonstrated by differential responding associated with changes in stimuli or context, is a fundamental aspect of how organisms are sensitive to specific features of the environment and adjust to it (Cowie & Davison, 2016) as their survival depends on the ability to behave in a way that is appropriate to future circumstances (Domjan, 2010). Unraveling the conditions under which a behavior comes under discriminative control is critical for a better comprehension of how an organism interacts with its environment (Bouton, Todd, & León, 2014). Indeed, discriminative control of saccades results in responses that are controlled by the present state of the environment in such a way that they are adapted to the future state of the environment. Understanding how the outcome of saccades might be predicted provides critical insights about how these motor responses are attuned to environmental exigencies. 

The ability to use the current state of the environment to adjust saccades according to future outcomes has been explored before. For instance, Fleuriet and Goffart (2012) proposed that saccade endpoint in the presence of a moving target might be accounted for by the spatiotemporal characteristics of the target at the time of saccade onset. As we previously discussed, saccade adaptation may be placed under discriminative control such that an amplitude increase might be associated with one context while an amplitude decrease is associated with another context (e.g., Azadi & Harwood 2016). More generally, when exploring a visual scene, eye movements are perturbed by the presence of an inconsistent object or by having objects violating the laws of physics (Võ & Henderson, 2011). In other words, eye movements are perturbed by unfamiliar context revealing that familiar context usually do control exploration saccades. Research also demonstrated that saccade may become predictive (i.e., with a 0-ms latency) when a visual target steps between two locations with a fixed interstimulus interval (Lee et al., 2016): The context formed by the spatiotemporal regularities of the target appearance comes to control saccade triggering. 

Importantly, discriminative control may also be observed in situations in which the reinforcer delivery itself forms the context. For instance, it has been shown that monkeys might learn to adequately choose between two physical targets depending on the probability of reinforcement associated with each target (Sugrue, Corrado, & Newsome, 2004). When the reinforcement probabilities changed, the probability of saccading toward one or the other target changed very rapidly such that the local relative frequency of choice matched the local relative frequency of reinforcement. The dynamic adjustment of saccade allocation prompted the authors to propose a model based on a local formulation of the matching law. This indicates that monkeys were able to discriminate among the various contingencies that in return came to control the saccades. Research also demonstrated that human observers may learn the temporal properties of a dynamical environment to allocate their gaze toward a specific region based on the associated frequency of reinforcement (Hoppe & Rothkopf, 2016). A similar result was obtained in a latency-contingent paradigm in which changes in reinforcement contingencies induced changes in saccade latency distributions (Vullings & Madelain, 2018). That organisms may adjust their behavior to reinforcement contingencies despite the absence of explicit cues in the environment has been taken as evidence that reinforcers themselves have discriminative properties as they signal a change in differential reinforcement (Cowie & Davison, 2016). 

Previous research established that manipulating reinforcement contingencies induces changes in SRT distributions (e.g., Madelain et al., 2007, Vullings & Madelain, 2018), and we previously proposed that the allocation of saccade in time depends on the specific temporal organization of the environment. Indeed, one could argue that saccades are information-foraging responses that must take into account the ways information is distributed in space but also in time (see Hoppe & Rothkopf, 2016 for a similar argument): The ability to use the temporal properties of our environments to shift gaze toward potentially informative locations at the right time allows efficient exploration of the visual environment. The present results reveal that human observers were able to learn to use environmental cues to adjust their saccade latencies. Taken together, these results indicate that saccade latencies are constrained by the environmental temporal properties, allowing information-foraging to be attuned to the specific dynamics of our environment. 

Conclusion

Expanding prior findings showing that saccades are affected by reinforcement, we demonstrated that saccadic latencies can be placed under discriminative control by operant conditioning. Our results indicate that the outstanding plasticity of the saccadic system depends on the state of the environment at the time of the saccade. The ability to use predictive signals to adapt the temporal allocation of saccades in response to the specific organization of information sheds a new light on the mechanisms by which reinforcing events affect motor control. 

Acknowledgments

This research was supported by Agence Nationale de Recherche grant ANR-13-APPR-008 (LM) and a scholarship from the French Ministry of Research (CV). 

Commercial relationships: none. 

Corresponding author: Laurent Madelain. 

Email: . 

Address: Université de Lille, CNRS, CHU Lille, UMR 9193 - SCALab - Sciences Cognitives et Sciences Affectives, Lille, France.