Neural correlates of intra-saccadic motion perception

Abstract

Retinal motion of the visual scene is not consciously perceived during ocular saccades in normal everyday conditions. It has been suggested that extra-retinal signals actively suppress intra-saccadic motion perception to preserve stable perception of the visual world. However, using stimuli optimized to preferentially activate the M-pathway, Castet and Masson (2000) demonstrated that motion can be perceived during a saccade. Based on this psychophysical paradigm, we used electroencephalography and eye-tracking recordings to investigate the neural correlates related to the conscious perception of intra-saccadic motion. We demonstrated the effective involvement during saccades of the cortical areas V1-V2 and MT-V5, which convey motion information along the M-pathway. We also showed that individual motion perception was related to retinal temporal frequency.

Introduction

Saccades play an essential role in the active process of vision (Findlay & Gilchrist, 2003), which is defined as a succession of decisions (saccade programming) on where and when to direct the gaze. Although ocular fixation is a well-studied phenomenon (for reviews, see: Henderson, 2003; Rayner, 2009), the ocular saccade is often considered as a simple transition between two fixations rather than as an intriguing and complex phenomenon in its own right. However, two fundamental issues invite discussion (Ibbotson & Krekelberg, 2011): (i) what happens just before and just after the saccade to enable perception of a stable representation of the world when the location of objects on the retinal image changes due to a saccade shift, and (ii) why intra-saccadic retinal motion flow does not elicit motion perception. The first issue is mainly related to trans-saccadic fusion and space constancy (Deubel, Schneider, & Bridgeman, 2002), whereas the second issue is mainly related to intra-saccadic motion perception (see the review in Castet, 2010). The main objective of this study was to provide a deeper insight into the latter by revealing the underlying neural correlates of intra-saccadic motion perception using electroencephalography (EEG) and eye-tracking recordings. 

Intra-saccadic retinal motion occurs within a short period of time (saccade duration: between 20 and 40 ms) during which the visual scene may change. In normal everyday conditions, the motion of a visual scene on the retina is not perceived during this short period of time. Two broad types of process have been proposed to explain this lack of awareness of the retinal motion of the visual scene during saccades. The first type of process proposed relies on the visual factors responsible for a reduction in visual sensitivity during saccades in the absence of any influence from extra-retinal signals. It has been suggested that, in view of the low sensitivity of retinal cells at high temporal frequencies, this reduction in sensitivity to luminance and chrominance contrasts might be the consequence of fast retinal motion during saccades. A more convincing argument is that fast motion of the visual scene on the retina generates motion smear, and that the clear images seen before and after saccades momentarily mask the smeared intra-saccadic signals (Campbell & Wurtz, 1978; Castet, Jeanjean, & Masson, 2002; Ilg & Hoffmann, 1993), although the involvement of temporal masking does not seem absolute (Duyck, Wexler, Castet, & Collins, 2018). In the terminology of Campbell and Wurtz (1978), this latter “saccadic omission” process acts as “a visual masking phenomenon” entailing the omission of the perception of the smeared intra-saccadic motion, due to the presence of the pre- and post-saccadic images. The second type of process proposed is called “saccadic suppression.” This relies on a reduction in visual luminance contrast sensitivity for peri-saccadically flashed stimuli, and is actively triggered by extra-retinal signals linked to the motor control of saccades. Several psychophysical studies have shown that, during saccades, sensitivity to flashed gratings modulated by luminance is greatly attenuated, in particular, at low spatial frequencies (Volkmann, Riggs, White, & Moore, 1978; Burr, Holt, Johnstone, & Ross, 1982; Shioiri & Cavanagh, 1989; Burr, Morrone, & Ross, 1994; Diamond, Ross, & Morrone, 2000). In a follow-up study featuring an isoluminant stimulus, Burr and collaborators (1994) also demonstrated the absence of this reduction in sensitivity, with no loss at high spatial frequencies. Based on this observation, and on the fact that the magnocellular (M-) pathway conveys luminance information (Kaplan, 2004), the “suppression” theory postulates that the reduction in luminance contrast sensitivity is selective and limited to the achromatic magnocellular pathway, and that the reduction also occurs early in the lateral geniculate nucleus (LGN) supplying all visual areas (for reviews see Burr, et al., 1994; Ross, Morrone, Goldberg, & Burr, 2001). However, Braun, Schütz, and Gegenfurtner (2017) obtained less clear-cut results regarding the preservation of chromatic sensitivity during saccades for low spatial frequency stimuli. This was also shown by Rolfs and Castet (2014). 

Moreover, the theory concerning saccadic suppression with a reduced perisaccadic response along the M-pathway, from the LGN to the MT-V5 area, through the primary visual areas (V1-V2), has been partially confirmed by electrophysiology studies (for a review, see Ibbotson, 2009), by single cell recordings in non-human primates (Bremmer, Kubischik, Hoffman, & Krekelberg, 2009; Cloherty, Mustari, Rosa, & Ibbotson, 2010; Hass & Horwitz, 2011; Ibbotson & Krekelberg, 2011; Krock & Moore, 2014) by noninvasive functional magnetic resonance imaging (MRI) recordings in humans (Kleiser, Seitz, & Krekelberg, 2004; Sylvester & Rees, 2006; Vallines & Greenlee, 2006), and by transcranial magnetic stimulation (Thilo, Santoro, Walsh, & Blakemore, 2004). However, as in psychophysiological experiments, discrepancies exist, notably regarding the extent of the modulation of perisaccadic activity along the dorsal and ventral visual pathways (Krock & Moore, 2014). Braun and collaborators (2017) studied all results on the selectivity of “saccadic suppression” and reported on the extensive range of experimental conditions and stimuli used (contrast, frequency, and size), which may partially explain the different findings. In spite of their differences, all of these studies nevertheless provide evidence that saccadic suppression predominantly affects the M-pathway and the dorsal stream, which are mainly involved in motion processing. Finally, Braun and collaborators (2017) concluded that perisaccadic modulations of contrast sensitivity “were clearly stimulus specific.” 

Recently, Idrees, Baumann, Franke, Münch, and Hafed (2020) showed that saccadic suppression occurred as early as in the first stage of visual processing, in the retina. They ran behavioral experiments on humans, using saccades or shifts with simulated saccades on textural backgrounds. They also performed multi-electrode array recordings on mouse and pig retinas to record ganglion cells in the retina. Finally, they observed that saccadic suppression in retinal ganglion cells lasted longer for simulated saccades than for real saccades. The results for simulated saccades illustrated saccadic suppression per se due to the spatio-temporal features of visual processing in the retina. For the results on real saccades, Idrees and collaborators suggested that the suppression shown by Burr et al. (1994) is essentially visual suppression not extra-retinal suppression, and moreover that extra-retinal signals could shorten saccadic suppression to minimize the duration of visual disruptions due to saccades. 

A number of studies based on the “visual factor” theory have demonstrated that it is relatively easy to perceive motion during saccades when special psychophysical setups are used. To achieve optimal intra-saccadic motion perception, the stimulus should be optimized to preferentially activate the M-pathway (Castet & Masson, 2000; Castet et al., 2002). The idea is to use eye velocity during saccades to reduce the retinal temporal frequency of a grating (rapidly moving in the world) in order to make it compliant with the bandwidth of motion detectors in the M-pathway. In their initial experiments, Castet and Masson (2000) used a stimulus grating specially designed in accordance with the spatio-temporal properties of the M-pathway to maximize its response. This type of fast-moving grating with a low spatial frequency was not perceived during fixations because the velocity of the stimulus projected on the retina was too high and exceeded the bandpass of the temporal frequencies of the visual system. However, Castet and Masson (2000) demonstrated that stimulus motion could be perceived during saccades and that motion perception was mediated by the temporal frequency of the stimulus projected on the retina. This suggests that visual perception, in particular, motion perception of luminance contrast stimuli, was not completely inhibited during saccades. More precisely, their results showed that intra-saccadic “compelling” motion perception occurred when the grating moved in the direction of the saccade and induced an average retinal speed around the peak velocity time that optimally stimulated the selective cells of the motion-sensitive MT-V5 area (Movshon & Newsome, 1996). Furthermore, there is clear electrophysiological evidence that the motion-sensitive area MT-V5 responds to intra-saccadic motion. Bair and O'Keefe (1998) showed that the retinal motion signals generated by saccades reach the cortical area MT by transiently modifying the firing rate of MT neurons, and that this depends on saccade direction. The firing rate of MT neurons increases when the preferred direction of MT neurons and saccade direction are the same, and decreases when this is not the case. Other studies confirm that the directionally selective cells in MT are stimulated by intra-saccadic motion (Thiele, Henning, Kubischik, & Hoffmann, 2002; Price, Ono, Mustari, & Ibbotson, 2005; Ibbotson, Price, Crowder, Ono, & Mustari, 2007). Moreover, motion direction-selective adaptation of contrast sensitivity has shown that intra-saccadic motion perception appears to be related to the low-level motion detectors in the M-pathway (Castet & Masson, 2000). It has also been shown that the first stages of the dorsal visual pathway can respond to motion during saccades. Previous studies on monkeys showed, for instance, that the cortical striate cells in the primary visual areas (V1) responded in the same way to motion produced by saccades and to stimulus motion during eye fixations (Wurtz, 1969; Fischer, Boch, & Bach, 1981). 

For a better understanding of intra-saccadic motion processing and perception, we investigated the neural correlates of intra-saccadic motion perception using joint EEG and eye-tracking recordings. In order to do so, we adapted the psychophysical paradigm initially proposed by Castet and Masson (2000) to EEG experiments. First, to reduce the duration of the experiment, the paradigm focused mainly on situations where the perception of the motion stimulus was strong. Second, we added a control condition in which perception of the motion stimulus was impossible, irrespective of the type of saccadic movement required. The spatio-temporal stimulus was optimized to activate low-level motion receptors in the M-pathway. When viewing this type of stimulus, participants performed horizontal saccadic movements of a controlled saccade size, toward a designated target. The evoked potential at saccade onset was estimated both in space (in the sensor space and in the source space after source localization), and time to demonstrate the involvement (i) of the first steps of the dorsal visual pathway (primary visual areas: V1-V2) in processing visual stimulation during saccades, and (ii) of the middle temporal cortex (MT-V5) in the perception of motion information per se during saccades (Ibbotson et al., 2007). We expected to observe differences in the EEG signal in the dorsal visual processing stream when we compared saccades, which were the same in all other respects, in the condition in which strong perception of the stimulus was reported, to those in the control condition where no motion perception was reported. 

Materials and methods

Participants

Twenty-one subjects (9 women; 21.6 ± 2.4 years) were recruited. All participants had normal or corrected-to-normal vision. None of the participants presented any neurological or psychiatric disorders, past or present, and they were free of any medical treatment. They were naive to the purpose of the experiment. Data from two participants were discarded, one due to a misunderstanding of the task and the other because of excessive noise in EEG signals. The whole study was approved by the CERNI (French ethics committee) of “Pôle Grenoble Cognition” (IRB no.: IRB00010290-2017-12-12-29) and conducted according to the principles expressed in the Declaration of Helsinki. Informed consent was obtained from each participant included in the study. All participants received vouchers in compensation for their participation. 

Apparatus

The visual stimulus was displayed at a visual angle of 27 degrees × 21 degrees on a computer screen (21-inch Sony CRT monitor) positioned 68 cm from the participant. The resolution of the screen was 640 × 480 pixels and the refresh rate was 160 Hz. Luminance calibration of the computer screen was performed with a spectrophotometer (SpyderX Elite). Mono-ocular activity (left eye) was recorded using EyeLink 1000 (SR Research) at a sampling rate of 1000 Hz. The EEG signals were acquired with 32 active electrodes including one electrode for reference (FCz). The right lobe was used as the ground electrode. All electrodes were placed on the left side of the scalp in accordance with the international extended 10-20 system, and made up of Ag/Ag-Cl. Electrode impedances were kept below 5 KΩ. The signals were amplified using a GAMMAsys (g.tec, Inc.) with a sampling rate of 1200 Hz. In order to correct ocular artifacts, the left eye electro-oculographic activity (EOG) was recorded using four electrodes, positioned to the left, right, top, and bottom of the left eye. The reference electrode was placed in the middle of the forehead, and the ground electrode on the left shoulder. 

Visual stimulation

The visual stimulus was a horizontal or vertical moving greyscale grating of low spatial frequency (F_S = 0.17 cy/degrees) and of high velocity (V_Stim = 360 degrees/s), which resulted in a high temporal frequency (F_T = 61.2 Hz):  

\begin{equation}{F_T} = {F_S}.{V_{Stim}}\end{equation}

(1)

 

Based on the luminance measurements, we used a lookup table to linearize the screen phosphor responses to an 8-bit luminance resolution. The grating had a mean luminance of 22 cd/m² with a contrast varying between 0 and 17%. 

Each trial lasted between 3300 to 3500 ms and was divided into five periods based on the time evolution of the grating contrast of the visual stimulus (Figure 1). During the first period from T₀ to T₁, which was called “null contrast” and lasted between 400 and 600 ms, a grey background (22 cd/m²) was displayed. Two isoluminant (22 cd/m²) markers, a red cross and a green plain circle, were added and equidistantly positioned, respectively to the left and right of the center of the screen, at a random distance between 1.5 degrees and 6.5 degrees. During the second period from T₁ to T₂, which was called “raising contrast” and lasted between 400 and 600 ms,¹ the stimulus contrast increased from 0 to 17% using the raising profile of a cosine function. When the contrast reached its maximum at T₂, the red cross then changed into a plain red circle, and was used by the participant as the visual cue to execute a saccade toward the plain green circle. From T₂, the grating contrast was maximal and remained constant. Thus, the luminance varied between (1 + 0.17) × 22 and (1 - 0.17) × 22 (i.e. between 25.74 and 18.26 cd/m²). This third period was called “constant contrast,” and lasted 1500 ms. As in the raising contrast period, during the fourth period from T₃ to T₄, called “decreasing contrast” and lasting 500 ms, the stimulus contrast decreased from 17% to 0% using the decreasing profile of a cosine function. Finally, during the last period from T₄ to T₅, called “null contrast” and lasting 300 ms, the contrast of the stimulus remained null, and luminance was constant at 22 cd/m². 

Figure 1.

 

Illustration of visual stimulation during one trial. (a) Representation of the evolution over time of the contrast of the stimulus, from 0% to 17%, (b) layouts over time in the stimulus condition. During the gaze stabilization periods (T₀ ≤ t ≤ T₂) and (T₄ ≤ t ≤ T₅), the layout was a red cross and a plain green circle on a grey background. At time T₂, the red cross changed into a plain red circle, which was the visual cue to execute the saccade. Between times T₂ and T₄, the layout consisted of two plain circles with the stimulus moving from left to right (black arrow), (c) illustration of gaze during the trial. First, the eyes were fixed on the red cross (T₀ ≤ t ≤ T₂), then the visual cue appeared and a saccade was executed from the plain red circle towards the plain green circle (T₂ ≤ t ≤ T₄. Perception of motion occurred during this period (T₂ ≤ t ≤ T₄). Finally, the eyes were fixed on the plain green circle (T₄ ≤ t ≤ T₅).

At the beginning of each trial, the participant had to stabilize his/her gaze on the red cross (left side of the screen). Then, he/she had to make a saccade as naturally as possible toward the plain green circle (right part of the screen) when the red marker changed from a cross into a plain circle. Once the saccade had been executed, he/she had to fixate the plain green circle. 

Two visual conditions, in which the spatial orientation and temporal direction of the stimulus differed, were implemented. The evolution of contrast was the same in both conditions. In the stimulus condition, the stimulus was a vertical spatial grating moving from left to right. Because the direction of both the stimulus motion and the requested saccade were identical, the possibility of intra-saccadic motion perception was dependent on eye velocity. In the control condition, the stimulus was a horizontal spatial grating moving from the bottom upward (i.e. in a direction orthogonal to the requested saccade). Intra-saccadic motion perception of the stimulus was not possible in this condition. These two conditions were randomly shuffled for each participant. 

The stimulus was generated using a program written in Python. The toolbox Psychopy was used to control the time course of the visual stimulus, and the toolbox Pylink to control the eye tracker. All parameters defining the stimulus (mean luminance, contrast, spatial frequency, and velocity) and its time courses were based on the original psychophysical experiment proposed by Castet and Masson (2000). 

Categorization task

In their original paper, Castet and Masson (2000) elegantly manipulated the temporal frequency of the stimulus projected onto the retina around the time of the saccadic peak velocity, the latter depending on saccade size (Hyde, 1959). In each trial, average retinal temporal frequency around peak velocity time was therefore mediated by the size of the saccade executed by participants. In our experiment, participants had to perform a horizontal saccade at the onset of the visual cue (plain red circle) toward the target (plain green circle) located on the right side of his/her visual field, and to fixate this target until the end of the trial. The participant then had to categorize her/his motion perception in one of two ways. The participant had to press the “strong” key when he/she saw the stimulus motion with a strong apparent spatio-temporal contrast (i.e. the contrast between the black and white vertical bars of the visual grating stimulus). When this was not the case, he/she had to press the “weak/null” key. 

Because the perception of motion during saccades is unusual, two training phases were performed by each participant before the main experiment. During the first training phase, in order to become familiar with this intra-saccadic perception and to learn the “perceptual pattern” to be classified in the “strong motion perception” category, the participant had to make a saccade between the two markers separated horizontally by 4 degrees in both directions to the left and to the right, and report to the experimenter what he/she perceived for each saccade direction. The amplitude of 4 degrees was chosen because the peak velocity of the saccade of this size is around 250 degrees/s and is the point at which motion perception becomes conspicuous (Castet & Masson, 2000). The stimulus displayed was the same as in the stimulus condition (i.e. a vertical grating moving from left to right [360 degrees/s]), but at a constant stimulus contrast corresponding to its maximum level (17 %). The participant therefore became familiar with his/her individual experience of a strong versus a weak/null motion perception. This first training phase lasted until the participant was confident with this perception of the stimulus motion during a saccade, and with motion perception categorization. 

The second training phase was then introduced, with the same time course as the main experiment, as illustrated in Figure 1, but with a reduced number of trials. This training phase was composed of 80 trials and lasted for about 5 minutes. At the end of each trial, the participant indicated his/her motion perception categorization by pressing the corresponding key on a keyboard. A new trial was initialized once the key had been pressed. During this training phase, the participant adjusted his/her own internal categorization threshold in order to have confidence in his/her categorization in the time course of the main experiment. 

Finally, the main experiment was composed of five blocks of 96 trials. Each block (lasting about 6 minutes) was separated by a pause lasting a few minutes. In all, 480 trials were recorded: 300 trials for the stimulus condition and 180 trials for the control condition. The total duration of the main experiment was approximately 1 hour. 

Data acquisition and preprocessing

Eye movements

A calibration using 9 points evenly distributed across the entire screen was performed at the beginning of each block. This calibration was repeated when the participant failed to stabilize his/her gaze at the beginning of each trial. Saccades and fixations were automatically detected by EyeLink software, based on three different thresholds: a distance of more than 0.1 degrees from the previous gaze position, a velocity greater than 30 degrees/s, and an acceleration greater than 8000 degrees/s². Four features were extracted for each saccade: the latency of the saccade at the time T₂, the horizontal amplitude, the profile of the horizontal velocity and the horizontal peak velocity. Saccade velocity was estimated using a nonlinear extension of the Savitsky-Golay filter (Nyström & Holmqvist, 2010; Dai, Selesnick, Rizzo, Rucker, & Hudson, 2017). This filter allowed the temporal derivatives from time series with abrupt variations to be estimated more accurately. Peak velocity was estimated as the maximum value of horizontal velocity during saccades. The temporal frequency of the stimulus projected onto the retina during the horizontal saccade of interest in the stimulus condition, was called the retinal temporal frequency (Hz) and was estimated from this peak velocity using the following equation:  

\begin{equation}{F_R} = \left( {{V_{Stim}} - {V_{Peak}}} \right).{F_S},\end{equation}

(2)

with V_Stim the visual stimulus velocity in the direction of the saccade (360 degrees/s), V_Peak the peak velocity of the saccade and F_S the spatial frequency of the stimulus (0.17 cy/degrees). This temporal frequency F_R was calculated for all trials irrespective of the condition, however, its interpretation as retinal temporal frequency was relevant for the stimulus condition only. As this temporal frequency depended only on the peak velocity of the saccade (Equation 2), with all the other parameters remaining constant, it became equivalent for the selection of the valid trials to define a threshold based on either the peak velocity or on this temporal frequency. We preferred the latter (thresholds on temporal frequency ξ_FL, ξ_FH, see below), because, in the stimulus condition, this temporal frequency was the retinal temporal frequency which mediated perception of stimulus motion (Castet & Masson, 2000). Because there is no psychophysical interpretation of this temporal frequency in the control condition, the frequency thresholding corresponded only to the transcription in Hertz of an equivalent thresholding based on the velocity peak. 

Five criteria were used to determine the validity of a trial and to select, during the trial, the horizontal saccade of interest, later called the “main saccade.” These five criteria associated with three thresholds (ξ_FL, ξ_FH,and ξ_VH) allowed us to ensure that the selected main saccade was the only saccade between T₂ to T₃ that could induce intra-saccadic motion perception when its peak velocity was sufficient. These criteria were defined as follows: 

• (1) No eye blink from T₁ (raising contrast) to T₂ (constant contrast);
• (2) No saccade from T₁ to T₂ with a peak velocity providing a retinal temporal frequency between ξ_FL and ξ_FH;
• (3) Only one saccade after T₂ with a peak velocity providing a retinal temporal frequency between ξ_FL and ξ_FH;
• (4) From T₂, the rank of the saccade which is the closest to the target eccentricity, must be equal to one or two;
• (5) Main saccades with a peak velocity higher than a given threshold (ξ_VH) were considered as outliers.

The threshold ξ_FH was set at 40 Hz, in line with the band-pass of the temporal frequency of the motion detectors (Hawken, Shapley, & Grosof, 1996). The threshold ξ_FL was introduced to ensure that the peak velocity of the main saccade was inferior to the velocity of the stimulus,² and was set at 2 Hz. Identification of the main saccade could be ambiguous when more than one saccade induced a retinal temporal frequency lower than ξ_FH. The last threshold ξ_VH allowed us to detect the outliers in the distribution of the peak velocity of the main saccade:  

\begin{equation}{\xi _{VH}} = {Q_{75{\rm{\% }}}} + 1.5*\left( {{Q_{75{\rm{\% }}}} - {Q_{25{\rm{\% }}}}} \right),\end{equation}

(3)

with Q_25% the first quantile and Q_75% the third quantile of the distribution for each participant. 

Motion perception

In the stimulus condition, retinal temporal frequency was computed according to Equation 2, using the peak velocity estimated for each saccade by the nonlinear extension of the Savitsky-Golay (Dai et al., 2017). Castet and Masson (2000) found that the probability of perceiving motion as a function of retinal temporal frequency had an inverted U-shape with an optimal retinal temporal frequency maximizing the probability of perceiving motion. The relationships between saccade size and the probability of perceiving motion are quantitatively illustrated in Figure 2. Considering the probability function with an inverted U-shape, and according to the results reported by Castet and Masson (2000), the average peak velocity optimizing this probability corresponded to approximately 250 degrees/s, and 18.7 Hz (Equation 2) for the retinal temporal frequency. With this value of peak velocity and a standard main sequence, such as the one represented in Figure 2 (bottom left quadrant), the corresponding saccade size was 4 degrees. For this reason, we chose this saccade size for the training phase of our protocol so that participants could learn the prototypical perceptual pattern corresponding to the stimulus-strong category with this saccade size. The relationship between the optimal retinal temporal frequency (18.7 Hz) the peak velocity (250 degrees/s) and the saccade size (4 degrees) is illustrated in Figure 2 (plain line with blue arrow from the top right quadrant to the bottom left quadrant). Consequently, we expected a decision threshold, named ζ_S0.5 and expressed in saccade size, lower than 4 degrees in our protocol. In other words, the probability of assigning the motion perception to the stimulus-strong category would be at chance level, for a saccade size equal to ζ_S0.5, inducing a peak velocity equal to ζ_V0.5 and therefore a retinal temporal frequency equal to ζ_F0.5. The retinal temporal frequency ζ_F0.5 denotes the decision threshold expressed in Hz. This chain relationship from the saccade size ζ_S0.5 to the probability at chance level (1/2) is illustrated in Figure 2 (plain line with red arrow from the bottom left quadrant to the top right quadrant). The expected probability function would therefore pass through the point (ζ_F0.5, 1/2), as illustrated in Figure 2 (red curve in the top right quadrant). The expected curve of the probability of assigning the motion perception to the stimulus-strong category would be shifted farther to the left than the probability function found by Castet and Masson (2000), as illustrated in Figure 2 (top right quadrant). Moreover, due to the restricted range of the target saccade size ([1.5 degrees° and;  6.5 degrees°], red dotted line in Figure 2) and the threshold ξ_FL (2 Hz), the descending profile (plain black line in Figure 2, top right quadrant) would not be observed. Finally, the probability of assigning the motion perception to the stimulus-strong category, related to the retinal temporal frequency was estimated for each participant to fit a psychometric function by using the toolbox psignifit (Schütt, Harmeling, Macke, & Wichmann, 2016). 

Figure 2.

 

Illustration of the relationships between the saccade size and the probability function through the peak velocity and the retinal temporal frequency: probability related to the retinal temporal frequency (top right quadrant), itself related to the peak velocity (bottom right quadrant), itself related to the saccade size (bottom left quadrant). The probability of motion perception related to the peak velocity is sketched (top right quadrant, in blue) from the results obtained by Castet and Masson (2000; Figure 2). The saccade size, peak velocity, and retinal temporal frequency eliciting a probability of the stimulus-strong category at chance level are denoted by ζ_S0.5, ζ_V0.5, and ζ_F0.5, respectively. See the text for further explanation of the graphical construction.

EEG signals

EEG signals and eye-gaze positions were synchronized using a trigger sent simultaneously on both systems during the experiment. The EEG data were resampled at the sampling rate of the eye-tracker (i.e. 1000 Hz). The preprocessing pipeline was implemented on EEGlab Software (Delorme & Makeig, 2004). First, the EEG raw signal was filtered with a band-pass between 1 Hz and 70 Hz, and a 50 Hz notch filter was used to avoid power-line contamination. EEG data were then segmented from 500 ms before T₁ to 3000 ms (including T₅) to create epochs of interest. For quality control, these epochs were inspected visually offline. Epochs with muscular activity or non-physiological artifacts were removed. The Independent Component Analysis - InfoMax algorithm, (Bell & Sejnowski, 1995) and the signal recorded from the EOG electrodes were used to correct ocular artifacts. The sources with the highest correlation with the vertical and horizontal deviations of the EOG were selected to correct artifacts due to blinks or saccades respectively. Sources for blinks,³ were easily identified and then removed (Viola, Thorne, Edmonds, Schneider, Eichele, & Debener, 2009). For saccades, identification of the selected sources based on their temporal evolution and their spatial topography was ambiguous for some datasets. In order to have the same preprocessing pipeline for all datasets, the correction for the saccade artefact was not implemented. Epochs whose standard variance exceeded the main standard variance across all segments by three standard deviations were then removed. The rejected channels were then spatially interpolated (spherical interpolation). For T₂ timestamps, a label was added offline, based on the categorization of the stimulus made by the participant (strong versus weak/null), and on the type of condition (stimulus versus control) in order to estimate evoked potentials separately for each configuration. Main saccades were also marked offline according to their amplitude and their induced retinal temporal frequency. Finally, in order to focus our analysis on the stimulus perception period, data were segmented into epochs starting at 200 ms before T₂ and ending 1500 ms after T₂. 

Estimation of evoked potentials

Estimation was only performed to compare potentials evoked by saccades which either did or did not induce intra-saccadic motion perception. Our two categories of interest were, therefore, stimulus-strong and control. Consequently, no EEG analysis was carried out for the stimulus-weak/null category. 

For this protocol, it was essential to consider two saccade features: the peak velocity, which mediated motion perception through retinal temporal frequency (Equation 2), and saccade size. Saccade size is already known to modulate brain activity for ESRP estimation (Yagi, 1979; Dandekar, Privitera, Carney, & Klein, 2012; Nikolaev, Meghanathan, & Van Leeuwen, 2016; Ries, Slayback, & Touryan, 2018). Taken together, the epochs for EEG analysis were selected in order to match the saccade features in the two categories of interest, stimulus-strong versus control. All epochs whose main saccade induced a retinal temporal frequency lower than a given threshold were retained, thus maximizing the probability of assigning motion perception to the stimulus-strong category. When the value of this threshold was low, the number of epochs was small, and the distribution of the saccadic features would match across the two categories of interest. When the value of this threshold was high, the number of epochs was larger, thus increasing the variability between ocular features. This would result in mismatched distributions between the two categories. We therefore implemented a procedure to maximize this threshold in respect to the matching of the distributions of the saccade size and the peak velocity across the two categories. This threshold for retinal temporal frequency is called ξ_f and corresponds to the threshold at which the distributions of the saccade metrics from the two categories matched. The corresponding threshold of ξ_f for peak velocity is called ξ_v. In this way, we obtained the maximum number of epochs while ensuring that each distribution of saccade features was matched between the two categories of interest. Due to the linear relationship between saccade size and saccade duration (Baloh, Sills, Kumley, & Honrubia, 1975), we expected that the distributions of saccade duration would match in both categories. Once this was defined, the only difference between the two categories of interest was the categorization of intra-saccadic motion perception made by participants for each epoch, all ocular features being otherwise equal. 

Another concern for the estimation of evoked potentials relating to saccades (ESRP) was that the overlap of evoked brain activities could, potentially, bias estimations. Estimations based on the classical synchronous average could have been biased by the overlap between the potential elicited at stimulus onset and the potential elicited at the onset of the main saccade in our present protocol. The time needed for the evoked potential at stimulus onset to fade is about 700 ms (Dimigen, Sommer, Hochlfed, Jacobs, & Kiegl, 2011) with an expected latency of around 250 ms for the main saccade (Clark, 1999). Evoked potentials were therefore estimated using a general linear model (GLM), as proposed for the deconvolution of evoked potentials with overlap (Smith & Kutas, 2015a; Kristensen, Guerin-Dugué, & Rivet, 2017). We considered the GLM as follows:  

\begin{eqnarray} {x_i}\left( t \right) &=& stim\left( t \right) + sac{c_{main}}\left( {t - {\tau _i}} \right) \nonumber\\ &&+ \mathop \sum \nolimits_{l = 1}^{L\left( i \right)} sac{c_{other}}\left( {t - \tau _i^{^{\prime}\left( l \right)}} \right) + {n_i}\left( t \right)\qquad\end{eqnarray}

(4)

where for a given epoch i, the time course of the response x_i(t) is a linear combination of stim(t), the evoked potential at stimulus onset, sacc_main(t), the evoked potential at main saccade onset, sacc_other(t), the potential elicited by saccades other than the main one and n_i(t) the noise of ongoing activity, with τ_i the latency of the main saccade and \(\tau _i^{^{\prime}( l )}\) the latency of the l^th saccade other than the main one. By concatenating all trials, stim(t), sacc_main(t) and sacc_other(t) were estimated in the sense of least squares by minimizing the power of the noise n(t) (Kristensen, Guerin-Dugué, & Rivet, 2017). This model was applied to the two categories of interest, and to each participant. The regressed evoked potentials \(\widehat {sac{c_{other}}}( t )\;\)and \(\widehat {stim}( t )\) were obtained per category for each participant and considered as covariates of noninterest in our analysis. The regressed evoked potentials \(\widehat {sac{c_{main}}}( t )\) (ESRP) were obtained per category for each participant, first in the sensor space (see Supplementary Material) before transformation in the source space. 

Source localization

From the \(\widehat {sac{c_{main}}}( t )\) estimation obtained in the sensor space, we used Brainstorm software (Tadel, Baillet, Mosher, Pantazis, & Leahy, 2011) to compute the potential elicited by the onset of the main saccade in the source space. For anatomic localization, we used the ICBM152 template (Fonov, Evans, McKinstry, Almli, & Collins, 2009), in which skull and scalp were extracted to compute a surface mesh of the brain. The conductivity values were automatically set at 1 S/m and 0.0125 S/m for the brain, scalp, and skull, respectively. We used the Boundary Element Method (BEM) in OpenMEEG to interpolate the triangular meshes (Gramfort, Papadopoulo, Olivi, & Clerc, 2010). Each dipole orientation was set perpendicular to the cortical surface. The weighted Minimum Norm Estimation (wMNE; Hämäläinen & Ilmoniemi, 1994; Hauk, 2004; Gramfort, Luessi, Larson, Engemann, Strohmeier, Brodbeck, Parkkonen, & Hämäläinen, 2014) was used to estimate the sources for each region of interest (ROI). For each participant, the noise covariance matrix used to estimate the sources was calculated on the concatenated baseline signals (i.e. the signal between −500 ms and 0 ms before T₁). The order of the depth weighting was equal to 0.5 and the maximal amount of depth weighting was equal to 10. The regularization parameter of noise covariance was equal to 10. The current densities were normalized by z-score with respect to a pre-saccade period equal to (−200; −50) ms (i.e. between −200 ms and -50 ms before main saccade onset; Mangalathu-Arumana, Beardsley, & Liebenthal, 2012; Harquel, Bacle, Beynel, Marendaz, Chauvin, & David, 2016; Tadel, Bock, Niso, Mosher, Cousineau, Pantazis, Leahy, & Baillet, 2019). This unitless measure corresponds to the number of standard deviations from a baseline period. The normalized estimated source maps were then spatially smoothed by a gaussian filter with a full width at half maximum of 4 mm. This spatial smoothing was performed on absolute values of these maps to prevent the positive and negative peaks that are close to each other in the minimum norm maps from cancelling each other out (Tadel et al., 2019). 

We used the Destrieux Atlas (Destrieux, Fischl, Dale, & Halgren, 2010) to localize our functional ROIs. The latency windows were visually defined for each ROI by delimiting the components of each evoked potential. 

Statistical analyses

For group analysis, we performed repeated-measures ANOVAs as a within-subject factor. For eye movements, repeated-measure ANOVAs were first carried out on the distributions of saccade features for all trials across all conditions and categories. We then performed repeated-measure ANOVAs on the distributions of saccade features of the epochs selected for evoked potential estimation after thresholding on peak velocity (ξ_v), or equivalently on retinal temporal frequency (ξ_f) in the stimulus-strong category. Finally, for evoked potentials, ANOVA analysis was performed for each ROI on the average activity inside each selected latency window. Statistical analyses were performed using Statistica version 10.0 Sofware (Statsoft, Tulsa, OK, USA) and the significance level of tests was set at α = 0.05. For each significant effect of a given factor, we used the Tukey post hoc tests for pairwise comparisons. 

Results

Summary of statistics on analyzed data

After the selection of trials based on the five ocular criteria, 19.5% (± 3.0) of trials per participant were removed. These criteria were applied to all trials across all conditions. The remaining trials were preprocessed, and trials with excessive noise were removed. Finally, Table 1 shows, after preprocessing, the total number of trials per visual condition and perceptual category, and the average number of trials per participant, based on individual means. 

Table 1.

 

Total number of trials after EEG and oculometry pre-processing steps and average number of trials per participant, based on individual means.

 

As expected, Table 1 indicates a balanced number of trials for the strong category and the weak/null category, for the stimulus and the control conditions respectively. This is a relevant indicator of the significance of the statistical results. For the EEG analysis, one epoch was extracted at the onset of the visual cue (i.e. when the red cross changed into a plain red circle) in each trial. In other words, the number of epochs corresponded to the number of trials. The stimulus-strong and control-weak/null categories defined the contrast of interest for the EEG analyses and then had to be studied using a similar signal-to-noise ratio related to the number of EEG epochs. 

Moreover, we expected to obtain a similar number of trials in the stimulus-weak/null category and in the stimulus-strong category in order to be able to correctly estimate the probability of assigning motion perception to the stimulus-strong category. As expected, the number of trials for motion perception (strong category) in the control condition was very low (21 trials). These trials were removed from all subsequent analyses. In the following sections, for reasons of simplicity, the control-weak/null category will be referred to only as control. 

Eye movements

To ensure that participants performed the task correctly, we analyzed three parameters: saccade size, peak velocity, and participants’ responses. Figure 3 illustrates the corresponding results for one participant (S05). When he/she performed the task correctly, the saccade size followed the target eccentricity (Figure 3a). This was the case for all participants with a mean relative error of saccade size of 11 ± 0.7%. We observed the relationship between saccade size and peak velocity (Figure 3b), namely the main sequence (Bahill, Clark, & Stark, 1975), which was fitted with an exponential function (Baloh et al., 1975). In the stimulus condition when the peak velocity increased, more trials were categorized as strong (Figure 3c). Finally, the probability of assigning motion perception to the stimulus-strong category increased when peak velocity increased, as shown in Figure 3d. 

Figure 3.

 

Representation of the three parameters for one subject (S05): (a) for all trials, saccade size as a function of target eccentricity, (b) for all trials, the peak velocity of the saccade as a function of saccade size, and its exponential fitting, (c) same as in b but only for trials in the stimulus condition, with in red, the “strong” response and in green the weak/null response (d) only for trials in the stimulus condition, the probability of the stimulus-strong category as a function of peak velocity. The open circles denote data pooled by bin.

Figure 4a illustrates, for all participants, the distributions of the three saccade features by condition, and Figure 4b illustrates these distributions by condition and by motion categorization. Table 2 summarizes the mean ± standard error of the three saccade features by condition and by motion categorization. 

Figure 4.

 

Distributions of the three saccade features, from left to right, size [°], peak velocity [°/s] and duration [ms], (a) by condition (stimulus versus control), (b) by condition and by motion perception (stimulus-strong in red, stimulus-weak/null in green, and control in blue). The mean of each distribution is marked by a cross.

Table 2.

 

Mean ± standard error of saccade size, peak velocity and duration by condition and by motion categorization, based on individual means.

 

Repeated-measure ANOVAs were carried out with Condition (stimulus and control) as the within-subject factor for each saccade feature. Differences were not significant for the three saccade features (saccade size: F (1,18) = 0.00, p = 0.98, ηp² =0.00; peak velocity: F (1,18) = 0.02, p = 0.89, ηp² = 0.00; saccade duration: F (1,18) = 0.55, p = 0.47, ηp² = 0.03). Repeated-measure ANOVAs were carried out with category (stimulus-strong, stimulus-weak/null, and control) as the within-subject factor for each saccade feature. Differences were significant for all three saccade features. More specifically, saccade size was significantly modulated by category (F (2,36) = 206.54, p < 0.001, ηp² = 0.92). Saccade size in the stimulus-strong category (4.48 ± 0.10 degrees) was significantly higher (p < 0.001) than in the control category (3.87 ± 0.10 degrees), which was in turn significantly higher (p < 0.001) than in the stimulus-weak/null category (3.02 ± 0.10 degrees). Peak velocity was significantly modulated by category (F (2,36) = 127.82, p < 0.001, ηp² = 0.88). Peak velocity in the stimulus-strong category (249.40 ± 8.63 degrees/s) was significantly higher (p < 0.001) than in the control category (227.92 ± 7.46 degrees/s), which was in turn significantly higher (p < 0.001) than in the stimulus-weak/null category (197.67 ± 7.46 degrees/s). Finally, saccade duration was significantly modulated by category (F (2,36) = 73.60, p < 0.001, ηp² = 0.80). Saccade duration in the stimulus-strong category (40.42 ± 1.06 ms) was significantly higher (p < 0.001) than in the control category (37.09 ± 0.82 ms), which was in turn significantly higher (p < 0.001) than in the stimulus-weak/null category (32.43 ± 0.96 ms). 

Motion perception

Figure 5 shows the average (± standard error) peak velocity as a function of saccade size for all trials after exponential fitting, and the probability of strong motion perception categorization as a function of retinal temporal frequency after psychometric fitting. Under our experimental conditions, based on individual means, we found that retinal temporal frequency had to be inferior to 22.24 ± 2.02 Hz in order to have a probability above chance level of inducing strong motion perception, and that peak velocity had to be superior to 229.2 ± 11.9 degrees/s. corresponding to an average saccade size of 3.22 degrees ± 0.23 degrees. These three values correspond to ζ_F0.5, ζ_V0.5, and ζ_S0.5, respectively, and are mentioned in Figure 2 and appear in Figure 5. 

Figure 5.

 

(a) Exponential fitting of the average (grey envelop: ± standard error) peak velocity of saccades as a function of saccade size, (b) the probability of the stimulus-strong category. Based on individual means, the retinal temporal frequency ζ_F0.5 eliciting a probability of the stimulus-strong category at chance level, corresponds to peak velocity ζ_V0.5 and to saccade size ζ_S0.5. These values are shown in red (± standard error).

Functional regions of interest for EEG source analysis

Based on the Destrieux Atlas (Destrieux et al., 2010), the left occipital pole (no. 42) and the left anterior occipital sulcus and pre-occipital notch (no. 59) occupied 12.71 cm² and 15.60 cm², respectively (Figure 6). Using the Jülich human brain atlas (https://jubrain.fz-juelich.de/) based on cytoarchitectonic information, we verified that these two ROIs overlapped with the V1-V2 area (59%) and the MT-V5 area (85%), respectively. 

Figure 6.

 

Representation of the two ROIs from the Destrieux Atlas on the left hemisphere of the brain. In purple, the MT-V5 ROI (no. 59) and in red the V1-V2 ROI (no. 42) The surface of the brain was reconstructed and smoothed at 50%.

For V1-V2, we visually defined two latency windows, one of [60; 230] ms and the other of [290; 390] ms. For MT-V5, we defined two latency windows, one of [100; 215] ms and the other of [270; 450] ms. 

Evoked potentials at main saccade onset

In order to estimate the evoked potential at the onset of the main saccade, it was essential to carefully consider both the retinal temporal frequency mediating motion perception and the saccade size modulating EEG activity. As explained previously in the “Material and Methods” section (see “Estimation of evoked potentials” subsection), we looked for the retinal temporal frequency threshold with the highest possible value in order to ensure that for all epochs with a retinal temporal frequency lower than this threshold, the saccade features (saccade size and peak velocity) were distributed similarly between the two categories (stimulus-strong versus control). Thus, the threshold was set at ξ_f = 20.4 Hz corresponding to a peak velocity of ξ_v = 240 degrees/s (Figure 7). Based on these criteria, the datasets of three participants (S03, S12, and S19) were excluded from the EEG analysis. For participants “S03” and “S12,” all saccades had a peak velocity lower than ξ_v (see Figure 7a for the exponential fitting of their main sequences). Consequently, for these datasets, retinal temporal frequencies were higher than ξ_v (see Figure 7b). The dataset from participant “S19” was also removed from the EEG analysis because after thresholding, the remaining number of epochs was only five per category.⁴ Finally, Table 3 summarizes the total number of selected epochs and the average (± standard error) number of epochs per participant with a peak velocity higher than ξ_v (or equivalently with a retinal temporal frequency lower than ξ_f = 20.4 Hz in the stimulus-strong category). Figure 8 illustrates the distributions of the three saccade features in the two categories of interest for the epochs selected for evoked potential estimation. 

Figure 7.

 

For participants S03 (yellow) and S12 (purple), comparison with average behavior, (a) average value (grey envelop: ± standard error) of the exponential fitting of the peak velocity of the saccades as a function of saccade size, (b) the probability of the stimulus-strong category. The threshold ξ_f = 20.4 Hz and its corresponding peak velocity ξ_v = 240 degrees/s are depicted by the dotted line. The open circles denote data pooled by bin for these two specific datasets.

Table 3.

 

Total number of epochs and average ± standard error number of trials per participant for the EEG analysis, based on individual means.

 

Figure 8.

 

For the epochs selected for evoked potential estimation, distributions of the three saccade features with a peak velocity higher than ξ_v = 240 degrees/s (or equivalently with a retinal temporal frequency lower than ξ_f = 20.4 Hz in the stimulus-strong category). From left to right, size [°], peak velocity [°/s] and duration [ms]. The mean of each distribution is marked by a cross.

The similarity between the distributions in the two categories (stimulus-strong versus control) was assessed with repeated-measure ANOVAs (F (1,15) = 2.76, p = 0.12, ηp² = 0.15; F (1,15) = 0.71, p = 0.41, ηp² = 0.04) for saccade size and peak velocity, respectively. As also expected, the distributions of saccade duration were similar in both categories (F (1,15) = 0.03, p = 0.87, ηp² = 0.002). Compared to Table 1, this thresholding reduced the number of epochs used for EEG analysis while keeping a balanced number in the two conditions. However, the averaged numbers of epochs remained sufficient in view of the statistical power (Boudewyn, Luck, Farrens, & Kappenman, 2018). 

The 2D ERP images were computed for the evoked potential at the onset of the visual cue (T₂), with and without deconvolution, to ensure that the model (Equation 4) correctly performed deconvolution of the evoked potentials (Ehinger & Dimigen, 2019). Results are presented in the Supplementary Material for a given dataset (one participant). We report our EEG findings based on these epochs in the source space. The intermediate results in the sensor space are presented in the Supplementary Material. 

Figure 9 illustrates the grand-average of the estimated ESRP on the two ROI for each category (stimulus-strong and control). 

Figure 9.

 

Illustration of the grand-average of the evoked potential at main saccade onset in each ROI, (a) V1-V2, (b) MT-V5. The solid line represents the average of the evoked potential and the transparent area represents the intersubject standard error at each time sample. The predefined latency windows in each ROI are represented in grey.

Repeated-measure ANOVAs were carried out with Category (stimulus-strong and control) as the within-subject factor, for each latency window and on each region of interest. For the region of interest V1-V2, statistical results in the latency window of [60; 230] ms showed that the amplitude of the evoked potential at main saccade onset was significantly modulated by the stimulus-strong versus control contrast (F (1,15) = 11.92, p = 0.004, ηp² = 0.44). Regardless of the category, the peak of this early component was in the grand-average observed at 165 ms. This component started, on average, at 60 ms in the stimulus-strong category and at 83 ms in the control category. The average amplitude was higher in the stimulus-strong category (5.09 ± 0.76) than in the control category (3.18 ± 0.28). In the latency window [290; 390] ms, statistical results showed that the average amplitude of the evoked potential at main saccade onset was significantly modulated by stimulus-strong versus control contrast (F (1,15) = 8.25, p = 0.02, ηp² = 0.35). The average amplitude was higher in the stimulus-strong category (3.10 ± 0.46) than in the control category (2.21 ± 0.35). 

Statistical results in the latency window of [100; 215] ms for the region of interest MT-V5 showed that the amplitude of the evoked potential at main saccade onset was significantly modulated by the stimulus-strong versus control contrast (F (1,15) = 17.12, p < 0.001, ηp² = 0.53). The average amplitude was higher in the stimulus-strong category (3.68 ± 0.32) than in the control category (2.68 ± 0.23). Finally, in the latency window of [270; 450] ms, the amplitude of the evoked potential at main saccade onset was also significantly modulated by the stimulus-strong versus control contrast (F (1,15) = 10.34, p = 0.006, ηp² = 0.41). The average amplitude was higher in the stimulus-strong category (2.47 ± 0.27) than in the control category (1.91 ± 0.19). This component had a positive deviation which started at 270 ms and peaked at 370 ms, on average, in the stimulus-strong category. 

Discussion

To our knowledge, this study explores, for the first time, the neural correlates associated with motion perception during human ocular saccades. Our objective was to reveal the involvement of the magnocellular pathway in order to demonstrate that motion perception was not totally suppressed during saccades. For this purpose, we used EEG and eye tracker recordings, and selected the primary visual areas, the occipital and parieto-occipital regions in the sensor space, and V1-V2 and MT-V5 regions in the source space. 

In their seminal paper, Castet and Masson (2000) used a psychophysical experiment to demonstrate that motion perception occurred in specific conditions during saccades. In their protocol, motion perception of a stimulus was mediated by the retinal temporal frequency induced at the time of the saccadic peak velocity, which in its turn depended on saccade size. We adapted this protocol to neuro-imaging recordings. The main saccade in each experiment trial was selected based on eye recordings, and the corresponding evoked EEG activity at its onset was estimated. We demonstrated that there were changes in the amplitude of evoked potentials at the onset of the main saccade during strong perception induced by the intra-saccadic stimulus motion, as opposed to the control condition, which involved no perception of stimulus motion. Importantly, the only feature that differed in this comparison was motion perception, because all ocular features were matched in both categories. 

Experimental paradigm

In the original experiment by Castet and Masson (2000), participants reported whether or not stimulus motion was perceived during a saccade (yes/no task). In our study, the task was adapted in order to focus on strong motion perception (i.e. perception of stimulus motion with a strong apparent spatio-temporal contrast). Consequently, participants were required to make a binary categorization based on whether perception of stimulus motion was strong or not. The apparent spatio-temporal contrast of the stimulus was the discriminative measure, rather than the yes/no response to motion perception used in the original experiment (Castet & Masson, 2000). By using this categorization task, we expected no ambiguity on motion perception with a strong apparent spatio-temporal contrast in the category of interest. The opposite category (“non-strong motion perception”) was not defined in relation to what it was, but in relation to what it was not, and was presented to participants as the weak/null category. Using terminology from machine learning, we asked participants to perform a “one-class” categorization task to focus on the class of interest over all other configurations (Khan, & Madden, 2014; Deng, Li, Liu, Guo, & Newsam, 2018). The rationale was to obtain the most homogeneous class of interest with respect to a strong perception of stimulus motion. The price of such a choice was the heterogeneity of the opposite class (weak-null). This paradigm is highly suitable here, because there can be several reasons why stimulus motion is not perceived during a saccade: (i) the eyes are too static and consequently the stimulus motion projected on the retina is too fast, (ii) the eye movement is too slow and the apparent spatio-temporal contrast of the stimulus is too weak to perceive the stimulus motion, (iii) the saccade and the stimulus do not move in the same direction, and (iv) the eyes move too quickly and the stimulus seems to “flash”) on the retina (Castet & Masson, 2000). Therefore, no EEG analysis was carried out for the “opposite class” (i.e. the stimulus-weak/null category). 

We also added a control condition, which did not feature in the original experiment. In this condition, the stimulus was a horizontal spatial grating with vertical upward movement. By design, the motion of the stimulus could not be perceived in the control condition. Consequently, we directly compared stimulus-strong versus control categories in the EEG analysis. 

Methodological considerations

As Castet and Masson (2000) showed, the perception of stimulus motion is dependent on retinal temporal frequency. This relevant factor for motion perception corresponds to the difference between the grating velocity and the saccadic peak velocity in the same direction, expressed in Hertz by multiplication with the spatial frequency of the stimulus. This retinal temporal frequency is induced by the saccadic peak velocity, which in turn depends on saccade size (Hyde, 1959). Previous studies have demonstrated that the motion detectors of the magnocellular pathway only respond to a restricted range of spatiotemporal frequencies (Hawken et al., 1996: Movshon & Newsome, 1996). The stimulus proposed by Castet and Masson (2000) was specifically designed to activate the M-pathway neurons involved in motion perception with a low spatial frequency (0.17 cy/degrees) and a high temporal velocity (360 degrees/s). The temporal frequency of this stimulus was then equal to 0.17  ×  360 = 61.2 Hz. When the eyes are static, this temporal frequency is the retinal temporal frequency whose value is above the critical fusion frequency.⁵ During saccades, when saccade velocity increases, retinal temporal frequency decreases. 

To ensure that this retinal temporal frequency was inside the bandwidth of the magnocellular pathway, we introduced two thresholds, 40 Hz and 2 Hz, for the high and low thresholds respectively. These thresholds were used to select the trials for analysis. The high threshold on retinal temporal frequency (40 Hz) was set in line with the band-pass of the temporal frequency of the low-level motion detectors (Hawken et al., 1996). The low threshold in particular was introduced to ensure that the peak velocity of the main saccade was always smaller than the velocity of the stimulus in order to avoid a flash perception of the grating. Indeed, when the peak velocity reached the velocity of the stimulus, retinal temporal frequency was very low, and the participants in the original experiment reported a flash perception of the grating without motion perception. Moreover, as this flash perception was not a part of our study, and as it appeared especially for high peak velocities and large saccade sizes, we also decided to reduce the range of the target saccade sizes (1.5 degrees to 6.5 degrees) in comparison to the original experiment (2 degrees to 10 degrees). In view of these two elements and of the categorization task chosen, the ascending part of the probability curve, corresponding to low retinal frequencies (see Figure 2), was excluded from our analysis. Consequently, the data collected were consistent with the underlying phenomenology of the descending part of the probability curve. 

Because our focus was on the stimulus-strong category, for EEG analysis, we were interested in trials which induced low retinal temporal frequency or equivalently, a high peak velocity. It is, however, well known that saccades modulate EEG activity and can introduce confounding effects (Nikolaev et al., 2016). This is especially true for the Lambda response in the occipital region, which is modulated by saccade size (Yagi, 1979), and by saccade direction (Dandekar, et al., 2012). The Lambda response was of particular importance for this study because it has long been established that this component is associated with visual processing and is evoked at each saccade onset (Evans, 1953). In order to provide comparison between experimental situations, it was therefore necessary to have similar saccade size distributions in the same direction in order to remove this potential confounding factor. In this experiment, all saccades were in the same direction, namely from left to right, so no additional precautions were necessary. However, as saccade size is naturally related to saccade duration and peak velocity, and as peak velocity is itself linearly related to retinal temporal frequency, in order to select a maximum number of trials with low retinal temporal frequency, the largest possible threshold on the retinal frequency was set in order to ensure that saccade features (saccade size, peak velocity, and duration) were similarly distributed in the stimulus-strong and control categories, for trials with a retinal frequency lower than this threshold. Using this process, the threshold was set at 20.4 Hz, and only trials with a retinal frequency below this threshold were retained, or equivalently only trials with a peak velocity higher than \(360 - \frac{{20.4}}{{0.17}} = 240\) degrees/s were retained. 

Eye-tracker recordings

In our experimental design, the two conditions (stimulus versus control) differed in the orientation of the visual stimulus according to the spatial grating (vertical versus horizontal), and according to the direction of movement (left to right versus bottom to top). In both conditions, participants had to make a saccade in the same direction, from left to right, at a size indicated by the position of a target point. In other words, in the stimulus condition, the stimulus moved in the same direction as the requested saccade, whereas in the control condition, the stimulus moved in a direction orthogonal to the requested saccade. Three saccade features were analyzed, namely saccade size, peak velocity, and saccade duration. These three features are known to be linked - saccade size is linked linearly with saccade duration, and nonlinearly with peak velocity (Baloh et al., 1975). Our results on these saccade features showed that while the orientation of the stimulus was different in both conditions, there were no differences between the two conditions, irrespective of the perception of stimulus motion. 

However, when the perceptual category of stimulus motion (strong versus weak/null) was taken into consideration, our results demonstrated significant differences between these two categories in relation to saccade features. As a reminder, in the control-strong category, there were very few trials. The answers in these trials could be interpreted as response errors or as possible perceptions of a vertical motion induced by saccades with very pronounced upward curvatures, and they were therefore removed from our analysis. The control-weak/null category then became the control category. Our analyses on saccade features focused on the three remaining categories, stimulus-strong, stimulus-weak/null, and control. Castet and Masson (2000), showed that the perception of stimulus motion was mediated by retinal temporal frequency. This latter depends on the peak velocity, which in turn depends on saccade size. When saccade size increased, peak velocity also increased, retinal temporal frequency decreased, and, as a result, the probability of perceiving the stimulus motion increased. Our results on saccade features were thus consistent with those obtained by Castet and Masson (2000): larger saccade sizes and peak velocities when stimulus motion was perceived as strong, and smaller saccade sizes and peak velocities when stimulus motion was not perceived as strong (i.e. the weak/null category). 

Motion perception

The inverted U-shape probability curve shown in the study by Castet and Masson (2000) was based on a yes/no detection paradigm, in which the participant had to indicate if he/she saw stimulus motion or not. The maximum probability was reached for a retinal temporal frequency of about 20 Hz, irrespective of the velocity of the stimulus. In our study, the experimental paradigm included only one stimulus velocity and the task was a categorization task. This categorization task was relative to the perception of stimulus motion with a strong apparent spatio-temporal contrast. In other words, the discriminant variable was not only motion detection, but motion perception with a strong apparent spatio-temporal contrast. The consequence of this change was that the probability curve obtained in our study shifted toward lower retinal temporal frequencies (see Figure 2). In the original experiment, a saccade which was small in size, but which was sufficient to reduce the retinal temporal frequency and place it within the magnocellular bandwidth, had a non-zero probability of inducing motion perception. In our experiment, for this same saccade size and therefore this same retinal temporal frequency, the apparent spatio-temporal contrast was not sufficient for categorization as stimulus-strong. To obtain categorization as stimulus-strong, retinal temporal frequency had to decrease even more (i.e. saccade size had to increase even more). Moreover, the training phase preceding our main experiment was done with a saccade size of 4 degrees, corresponding to about 20 Hz - the retinal temporal frequency at which the probability of motion perception was maximized in the original experiment. Each participant had practiced the prototypical intra-saccadic perception for this apparent spatio-temporal contrast and had used this training phase to have enough confidence in his/her internal decision threshold to categorize a strong pattern for motion perception. Finally, we found that the decision threshold (ζ_F0.5) was equal to 22.24 Hz, on average. This did not mean that above this threshold, participants did not perceive motion of the stimulus, but that they perceived it with a strong apparent contrast with a probability lower than chance level. This threshold, expressed in Hz, corresponded to 229.2 degrees/s for the peak velocity and 3.22 degrees for saccade size. As expected, the internal threshold ζ_S0.5 = 3.22 degrees averaged across all participants, expressed in saccade size, was lower than the learning saccade size (4 degrees°). It follows that the average probability curve shifted more towards the low retinal temporal frequencies than the curve in the original experiment. Due to this shift, we did not observe the ascending part of the inverted U-shape, and our probability function represented only the descending part of the inverted U-shape found by Castet and Masson (2000). This was the consequence of the choice of the categorization task and of the reduction in the range of saccade sizes. Therefore, even though the question we asked participants was different, our experiment replicated the results of Castet and Masson (2000) well. 

EEG recordings

The estimation of evoked potentials at main saccade onset was completed with a deconvolution of temporally overlapped neural responses (Smith & Kutas 2015a; Kristensen et al. 2017). Modulation by saccade size was also taken into consideration (Yagi, 1979; Dandekar et al., 2012; Nikolaev et al., 2016). Before analyzing the evoked potential at main saccade onset, we verified in the sensor space that the evoked potential at the onset of the visual cue was the same, irrespective of condition (stimulus or control), and categorization (strong or weak/null). Participants did indeed gaze at the red fixation point at the onset of the visual cue. This result was in line with the work of Castet and Masson (2000). These authors demonstrated that when the eyes were static, motion could not be perceived because the temporal frequency of the stimulus was higher than the critical fusion frequency (Robson, 1966) and above the bandwidth of the magnocellular pathway (Hawken et al., 1996; Movshon & Newsome, 1996). 

As our present study focused on saccades, it was more natural to time-lock EEG signals to saccade onset and not to saccade offset. The consequence of this choice was twofold. Compared to classical results, the latency of the components observed on the ESRP was delayed by saccade duration. Moreover, as these components, namely the Lambda response, N1, and P300 are synchronized on saccade offset, their estimation from signals time-locked to saccade onset is a low-pass version of the estimation from signals time-locked to saccade offset. This low-pass filtering effect remained low here since the jitter that causes it corresponded to the variability of saccade duration (from around 20 to 60 ms). As the distributions of saccade duration were similar in the two categories of interest (stimulus-strong versus control), the effect of the low-pass filtering was also similar between the two categories and was not relevant to the comparison between the two categories. 

The results obtained in the sensor space (see Supplementary Material), showed that the intra-saccadic perception of the motion of this stimulus, which had been optimized for the magno-cellular pathway, sequentially activated visual processes, which were indicative of visual processing, a discrimination task, and a decision-making task. Markers of these processes in the sensor space have been extensively studied. They include the Lambda response (Barlow & Cigánek, 1969), the N1 component (Luck, Woodman, & Vogel, 2000), and the P300 component, respectively (for a review see Polich, 2012). The cortical sources of the Lambda response are located in the occipital region (Kazai & Yagi, 2003). The N1 component is made up of two subcomponents, one anterior in the fronto-central region, and another posterior in the parieto-occipital region (Harter, Aine, & Schroeder, 1982). The P300 component is a large deviation, which is visible on the whole scalp, and whose maximum amplitude is located in the centro-parietal region (Polich, 2012). We now discuss our results on the first two components (Lambda, N1), followed by our results on the third component (P300). 

First, for the three parieto-occipital electrode clusters (left, medium, and right), the Lambda response with a positive peak on average at 165 ms in the grand-average, was observed in the stimulus-strong and control categories. With similar distributions of saccade size, the comparison between the Lambda response evoked at saccade onset for trials in the stimulus-strong category and those in the control category was relevant. The average amplitude of the Lambda response between 120 and 190 ms was lower in the control category than in the stimulus-strong category. For these same electrode clusters, the posterior N1 component between 200 and 300 ms, with a negative peak at 240 ms in the grand-average was observed in the stimulus-strong category but not in the control category. It is worth summarizing what the two categories of trial, during which participants performed saccades of the same size, had in common, and where they differed, in relation to the visual stimulus. The saccade performed in both categories was an ocular movement from a red dot to a green dot. The Lambda response reflects the afferent input of visual information to the visual cortex (Evans, 1953). However, Ossandon and collaborators showed that saccades executed on a uniform grey image did not evoke a Lambda response (Ossandón, Helo, Montefusco-Siegmund, & Maldonado, 2010). In our case, there was a chromatic contrast (from red to green), and an evoked Lambda response revealing this visual scene change was therefore expected in both categories. In both categories, a sinusoidal grating with low spatial and high temporal frequency was superimposed on this visual scene. However, in the stimulus-strong category, the grating was vertically oriented and moved from left to right in the direction of the saccade, whereas in the control category, the grating, orthogonal to the previous one, was horizontally oriented, and moved from bottom to top. Because the saccade was made from left to right, the temporal frequency of this control stimulus projected onto the retina was not reduced, except for saccades with very curved upward trajectories. Saccades of this type were observed. They slightly reduced the retinal temporal frequency and induced some retinal motion energy from a bottom-up direction. It is thus not surprising to observe an evoked response in the control condition. For perception, since the categorization task involved the perception of a left-to-right motion, participants were not supposed to respond positively in this situation, as was the case for 99.1% of them (see Table 1, 2391/2412). We therefore cannot exclude the hypothesis that there was very marginal stimulus motion-induced EEG activity in the control category. By definition, all trials in the stimulus-strong category induced a strong perception of stimulus motion. Furthermore, the N1 component, which reveals discrimination processes between an expected and an unexpected stimulus (Luck & Hillyard, 1995; Vogel & Luck, 2000) was observed with a more negative amplitude in the stimulus-strong category than in the control category. In addition, we know that the parieto-occipital subcomponent of the N1 component shares similar cortical sources with the visual evoked P1 potential (Di Russo, Martinez, Sereno, Pitzalis, & Hillyard, 2002). We also know that the Lambda response and the P1 potential share similar cortical sources - the former is triggered by fixation onset and the latter by stimulus onset (Kazai & Yagi, 2003). The differences observed for the parieto-occipital clusters were interpreted as a whole, in terms of a “Lambda - posterior N1 complex” rather than as two distinct components, the Lambda response, on the one hand, and the posterior NI component on the other. The rationale was that these two early components were temporally sequential and had common cortical sources in the parieto-occipital region. 

Second, the P300 component between 350 and 500 ms with a positive peak at 385 ms in the grand-average, was observed over the entire scalp and in particular over the occipital electrode clusters at a higher amplitude in the stimulus-strong category than in the control category. The P300 component has been observed in numerous situations (Hruby, & Marsalek, 2003) where the common factor was participants’ involvement in a decision-making task (Rohrbaugh, Donchin, & Eriksen, 1974; Picton, 1992). In our experiment, participants had to make a decision about the status (here, motion) of the perceived stimulus. We found that visual perception was similar during fixations and during saccades when the eyes were moving, from the perspective of these three evoked components in the sensor space. 

Additionally, the analysis in the source space provides localization in cortical regions for these activities observed in the sensor space. Two ROI were selected, namely V1-V2 and MT-V5, in the dorsal pathway, driven predominantly by magnocellular inputs. For this purpose, a standard source reconstruction algorithm (Minimum Norm Estimation; Hämäläinen, & Ilmoniemi, 1994) was applied to the estimates of ESRP obtained by deconvolution over the whole scalp for both categories and for each participant. The reconstructed brain regions were defined according to the Destrieux Atlas (Destrieux, et al., 2010). The two selected regions each had a significant overlap with the V1-V2 and MT-V5 areas: 59% for the first region and theV1-V2 area, and 85% for the second region and the MT-V5 area. This overlap was determined using the Jülich human brain atlas based on cytoarchitectonic information. Our previous findings in the sensor space showed that the brain activity observed in the occipital region could be interpreted in terms of two periods, an early one which included the Lambda response and the posterior N1 component, and a later one with the P300 component. In the source space, this temporal distinction was also observed on the activity of V1-V2 and MT-V5 reconstructed sources. In these early and late activities, we found distinct patterns which depended on whether the stimulus motion was perceived as strong or not. A positive deviation was observed for early activity in both categories and both ROIs. For each ROI, a common latency window was visually defined for both categories by delimiting the early component with positive deviations, namely between [60; 230] ms in the V1-V2 ROI and between [100; 215] ms in the MT-V5 ROI. Regardless of the ROI, the average amplitude in the respective latency window was higher for the stimulus-strong category than for the control category (i.e. when stimulus motion was perceived as strong versus not perceived), the saccades being otherwise similar. For each ROI, the respective latency window of the early component was included in the union of the latency windows of the lambda response and of the posterior N1 component. For the V1-V2 ROI, the peak of this early component was in the grand average at 165 ms. Moreover, for visual evoked potentials, Di Russo and collaborators (2002) found that the early P1 component and subcomponent N150 of the posterior N1 component share similar cortical sources, located in Brodmann's area 18, in the dorsal pathway driving magnocellular inputs. This early component in the V1-V2 ROI was interpreted as partially contributing to the “Lambda-N1” complex observed in the sensor space. The average amplitude between 60 and 230 ms in the V1-V2 ROI was higher in the stimulus-strong category than in the control category. The difference in the Lambda response observed in the sensor space on the parieto-occipital clusters was therefore also observed in the source space in a larger and earlier latency window. First, the component observed on the V1-V2 ROI was larger and higher in the stimulus-strong category than in the control category. In the control category, the lambda response was evoked by a change in the visual scene before and after the saccade, constituted by a change in the color of the fixation point, and we observed that this evoked response in the V1-V2 ROI was similar in both latency and peak to the Lambda response in the sensor space. In contrast, in the stimulus-strong category, in addition to this visual scene, participants also perceived stimulus motion. They pressed the “strong” key if they perceived (1) a vertical grating, (2) with a leftward motion (3) with a high spatiotemporal contrast,⁶ and (4) during the saccade. Perception of the stimulus during the main saccade was the discriminant element in both categories, and this accounted for a greater expression of the observed potential in the V1-V2 ROI in the stimulus-strong category. Second, the latency of this component occurred earlier (60 ms) in the stimulus-strong category than in the control category (83 ms). However, both components peaked at about 165 ms irrespective of category. This shorter latency (60 ms) could be explained by taking into consideration the typical time course for saccadic suppression (ranging from 75 ms prior to, the saccade to 50 ms following the saccade) described by Diamond et al. (2000). At the end of this period, contrast sensitivity progressively returns to normal. The latency of this progressive return to normality was congruent with the latency of the increasing activity observed in the V1-V2 ROI in the stimulus-strong category. 

Proceeding along the dorsal pathway, we observed a larger and higher positive component in the stimulus-strong category than in the control category in the MT-V5 ROI. As explained previously, we cannot ignore the possibility that in view of the saccade curvature, perception of a residual upward movement occurred, or that this residual movement could partially explain the reconstructed activity by source localization in this area for the control category. According to the top-down theory of attentional control for the detection of features (see Corbetta & Shulman, 2002, for a review), the response has to be facilitated because it is an expected feature (here motion from left to right). Taken together, the reconstructed activity for the stimulus-strong category is unambiguously a neural correlate of motion perception during saccades. This provides a physiological reality for the psychophysical findings of Castet and Masson (2000). 

Finally, late activity with a latency between 270 and 400 ms in the MT-V5 ROI, is interpreted as contributing to the P300 component observed in the sensor space, with a positive deviation starting at 270 ms and peaking at 370 ms, on average. Recent studies using EEG and functional MRI have found some differences regarding the cortical sources of the P300 component, depending on the experimental paradigm used. A search task requires more top-down attention, and elicits a larger network in the frontal lobe. A pop out task requires more bottom-up attention, and elicits a larger network in the parietal lobe (Zhang, Luo, Zhang, Jin, & Li, 2020). In regions more involved in stimulus-driven tasks than in goal-driven tasks, the left angular gyrus is of particular importance as it is required in a great variety of high-level tasks and is connected to the occipito-temporal region where the MT-V5 area is located (Seghier, 2013). In view of this dichotomy, the task of motion perception we used required the allocation of more bottom-up attention than of top-down attention. The late activity observed could therefore be explained by a contribution to the P300 component. This would constitute a main contribution for reconstructed activity in the MT-V5 ROI, and a marginal contribution for reconstructed activity in the V1-V2 ROI. 

Our aim was to provide strong evidence of a correlation between intra-saccadic motion perception and activity in the V1-V2 and MT-V5 areas, when the stimulus was optimized to elicit intra-saccadic motion perception and to activate the magnocellular pathway. Our results in the sensor and source spaces clearly indicate that stimulus motion is perceived during saccades. This study does not challenge the reduction in magnocellular contrast sensitivity during saccades. This magno-specific reduction has been suggested by a range of psychophysical and neurophysiological studies (Ibbotson & Krekelberg, 2011; Binda & Morrone, 2018) but remains a matter of controversy (Hass & Horwitz, 2011; Braun, Schütz, & Gegenfurtner, 2017). However, low contrast sensitivity is not equivalent to complete blindness during saccades. This study shows that for stimuli which have been optimized for the magnocellular pathway, intra-saccadic motion perception activates the same early or late visual processes as the perception of the salient elements in a visual scene during each ocular fixation. This is consistent with recent evidence suggesting that intra-saccadic visual signals have a functional role. It seems that intra-saccadic motion streaks might provide cues to tracking objects which rapidly change location across saccades (Schweitzer & Rolfs, 2020; Schweitzer & Rolfs, 2021). Even if the functional role is still unclear and studies are few, these findings could provide an additional argument in favor of the perception of motion during saccades. 

Limitations

Participants had to categorize their motion experience during saccades as strong versus weak/null. The class strong was well defined (homogeneous in terms of motion and apparent spatio-temporal contrast), and provided a high-level of confidence regarding the modulation of neural activity versus the control condition. The weak/null class was, on the contrary, more heterogeneous. The perception of motion could not be ruled out, and it was not possible to implement a binary categorization of motion/no motion. Due to the use of 32 electrodes, the spatial resolution for EEG source estimation was limited. In this particular context and because our ROIs were defined at cortical level, we used the depth-weighting “Minimum Norm Estimation” method (MNE) to estimate the sources. This method estimates sources close to the sensors, and the depth-weighting allows the impact of deeper sources to be enhanced (see Hauk, Stenroos, & Treder, 2019). The use of an MRI anatomic template for the delineation of V1/V2 and MT-V5 areas had limited precision. Additional functional MRI experiments could be introduced to delineate these areas precisely in each individual. The accuracy of the EEG source localization could be greatly improved using functional MRI-informed algorithms (Cottereau, Ales, & Norcia, 2012). 

Conclusion

Our results demonstrate intra-saccadic activity in the magnocellular pathway and confirm the behavioral results of the study by Castet and Masson (2000). They provide a first step for the observation of the neural correlates of intra-saccadic motion perception in humans. In order to progress in this area in the future, this experiment will be replicated using a more sophisticated source localization algorithm in order to be able to precisely delineate the brain areas involved. By combining high spatial and temporal resolution source location and connectivity graphs, it will then be possible to have a better understanding of the network of the brain areas involved in intra-saccadic motion perception and of their time course. 

Acknowledgments

The authors thank all the participants in this experiment. 

G.N. is recipient of a grant from the University Grenoble Alpes. This study was partially funded by a grant from the LabEx PERSYVAL-Lab (ANR-11-LABX-0025-01). 

Author Contributions: E.C. and A.G.D. contributed to the conception and the design of the study. A.R. and A.G.D. performed data acquisition. G.N., A.R., E.K., and A.G.D. contributed to the analysis of the data. All authors contributed to drafting the text and preparing the figures. 

Commercial relationships: none. 

Corresponding author: Anne Guérin-Dugué. 

Email: [e-mail]. 

Address: GIPSA-Lab, 11 rue des Mathématiques, Grenoble Campus BP46, F-38402 SAINT MARTIN D'HERES CEDEX, France. 

Footnotes

1  The duration of this period was randomly determined according to a uniform distribution between 400 and 600 ms.

Footnotes

2  When the peak velocity reached the stimulus velocity, participants in Castet and Masson's experiment reported a flash perception of the spatial grating without motion perception. The study of this phenomenon is outside the scope of this article.

Footnotes

3  Most of the time, blink artifacts were concentrated on a unique blink source.

Footnotes

4  For this dataset, almost half of the trials had already been removed because they did not satisfy the fifth criterion for the selection of the main saccade.

Footnotes

5  The critical fusion frequency can be defined, for a given intermittent light source, as the lowest frequency of light which gives a continuous feeling of light. It is a threshold between two states, and if the stimulus frequency is subthreshold, there is a flicker. The flicker is the perception of visual fluctuations in intensity and unsteadiness in the presence of a light stimulus when the eyes are static. On the contrary, if the stimulus frequency is suprathreshold, there is a continuous feeling of light (Simonson & Brozek, 1952; Landis, 1954; Rey & Rey, 1964).

Footnotes

6  The pattern for a “strong” motion perception was learnt with saccades of a size of 4 degrees.