Modulation of microsaccade rate by task difficulty revealed through between- and within-trial comparisons

Abstract

Microsaccades (MSs) are small eye movements that occur during attempted visual fixation. While most studies concerning MSs focus on their roles in visual processing, some also suggest that the MS rate can be modulated by the amount of mental exertion involved in nonvisual processing. The current study focused on the effects of task difficulty on MS rate in a nonvisual mental arithmetic task. Experiment 1 revealed a general inverse relationship between MS rate and subjective task difficulty. During Experiment 2, three task phases with different requirements were identified: during calculation (between stimulus presentation and response), postcalculation (after reporting an answer), and a control condition (undergoing a matching sequence of events without the need to make a calculation). MS rate was observed to approximately double from the during-calculation phase to the postcalculation phase, and was significantly higher in the control condition compared to postcalculation. Only during calculation was the MS rate generally decreased with greater task difficulty. Our results suggest that the nonvisual cognitive processing can suppress MS rate, and that the extent of such suppression is related to the task difficulty.

Introduction

Microsaccades (MSs) are the largest and fastest component of fixational eye movements (for reviews, see Martinez-Conde, Macknik, & Hubel, 2004; Martinez-Conde, Otero-Millan, & Macknik, 2013; Rolfs, 2009). Most studies concerning MSs focus on MS roles with regard to visual perception. While MSs enhance vision by counteracting and preventing perceptual fading (Martinez-Conde et al., 2006; McCamy et al., 2012; McCamy, Macknik, & Martinez-Conde, 2014), they are not directly triggered by it (Collewijn & Kowler, 2008; Poletti & Rucci, 2010). They contribute to the maintenance of visual fixation (Cornsweet, 1956; Costela et al., 2014; Ditchburn & Ginsborg, 1953; Engbert & Kliegl, 2004; Otero-Millan, Schneider et al., 2013; Otero-Millan, Serra et al., 2011), are employed in scanning small regions in scenes (Haddad & Steinman, 1973; Ko, Poletti, & Rucci, 2010; Otero-Millan, Macknik et al., 2013; Otero-Millan, Troncoso, Macknik, Serrano-Pedraza, & Martinez-Conde, 2008; Steinman, Cunitz, Timberlake, & Herman, 1967; Steinman, Haddad, Skavenski, & Wyman, 1973), and improve performance in high-acuity tasks (Ko et al., 2010; Poletti, Listorti, & Rucci, 2013; Rucci, Iovin, Poletti, & Santini, 2007). However, MSs are also subject to microsaccadic suppression, a process by which visual sensitivity decreases for stimuli coincided with MSs (Hafed & Krauzlis, 2010; Hafed, Lovejoy, & Krauzlis, 2011; Herrington et al., 2009; Zuber & Stark, 1966). 

A unique line of research using various experimental paradigms has shown that the level of difficulty with regard to visual processing has a significant effect on MS frequency. The majority of studies in this area have indicated that fewer MSs occur during harder tasks. Laubrock, Engbert, and Kliegl (2005) used a Posner cuing paradigm in which participants had to make proper saccades according to the location and shape of a target. Compared to results obtained using a similar Posner cuing paradigm in which participants made saccades according to the target location only (Engbert & Kliegl, 2003), lower MS rates were observed in the former case. Pastukhov and Braun (2010) used visual recognition tasks that required different attention loads (e.g., reporting the identity vs. the color of a target letter) and found lower MS rates associated with tasks involving greater loads. 

In contrast, other studies have shown that MS rate increases in response to more demanding tasks. In a simulated driving experiment conducted by Benedetto, Pedrotti, and Bridgeman (2011), participants performed a lane-change task either independently (control task) or simultaneously with a visual search task, where drivers were asked to locate a target among distractors (dual task). Significantly greater MS rates were observed during the dual task. These inconsistencies in the correlation between MS rate and task difficulty could be attributed to people learning the costs or benefits of MSs to visual processing in a given situation, then voluntarily exerting control over MS rate to obtain the best performance (Pastukhov & Braun, 2010). 

Given the correlations identified between MS rate and difficulty in visual tasks, further investigation could explore whether these relationships extend to represent dynamics between MS rate and difficulty in nonvisual tasks. Correlations established relative to visual tasks could potentially be confounded by extraneous variables present within visual processing. It would therefore be useful to examine relations using an experimental phase in which the primary mental process does not rely on vision. Indeed, humans spend a considerable proportion of daily life on cognitive processing, such as problem solving, which does not always rely on vision. It is not clear whether nonvisual cognitive processing has any effect on the MS rate. And if so, how could task difficulty, which may indicate the degree of mental effort engaged in the problem solving, modulate MS rate? 

Kowler and Steinman (1977) found MS frequency to be extremely low (∼0.06/s) when participants were engaged in mental imagery tasks (e.g., reporting the number of windows on the second floor of their parents' home). However, the rate observed within the control condition, where participants were only required to maintain fixation on a fixation target, was also extremely low (∼0.06/s, averaged across subjects). This indicated that mental imagery did not have a significant influence on the occurrence of MSs. Betta and Turatto (2006) explored the effect of response preparation to a visual stimulus on MS rate. The frequency was observed to decrease when participants were engaged in preparation compared to the no-response control condition. A series of studies examined MS rate in experimental paradigms involving visual (Valsecchi, Betta, & Turatto, 2007) and auditory (Valsecchi & Turatto, 2009) oddballs. In the active condition, participants were required to mentally count the number of oddball (rare) targets. In the passive condition, the oddball targets were ignored. MS rates were observed to be significantly lower in the active condition after the presentation of oddballs compared to standard targets, but this pattern did not hold in the passive condition, indicating that the MSs were suppressed when mental counting was required. 

Siegenthaler et al. (2014) examined the relationship between task difficulty and MS rate in a nonvisual mental arithmetic task. Each trial in their experiment lasted for 3 min. In the easy task, participants were required to mentally count forward in steps of 2. In the hard task, participants were instructed to count backwards in steps of 17. It was revealed that MS rates were comparatively higher within the easy task. 

Most studies mentioned above suggested that there could be an inverse relationship between task difficulty and MS rate, when participants had to perform some types of task with different difficulty levels. However, when the comparison was made between conditions with verse without a task, the results were mixed. Whether the MS rate was generally suppressed during cognitive processing seems to depend on the baseline selected to which MS rate during cognitive processing was compared. For studies showing the suppressed MSs during cognitive processing, the baseline was selected from a brief period of time within a demanding trial, for example, ∼2–4 s before stimuli onset (Betta and Turatto, 2006; or within ∼1 s after stimuli presentation, Valsecchi et al., 2007; Valsecchi & Turatto, 2009). 

In contrast, in studies demonstrating that the MS rate was not suppressed by cognitive processing (Kowler & Steinman, 1977), the baseline was the averaged MS rate in a separate control condition during which only fixation was required for a relative long time (10 s). A similar approach was also taken by Siegenthaler et al. (2014) in which all trials (including the control trials in which no mental arithmetic was required) lasted 3 min and MS frequency in the control condition was lower compared to rates in the easy condition, but higher compared to rates in the hard condition. Siegenthaler et al. (2014) noted that data from their control condition should be interpreted with caution. Due to a lack of mental demand in the control condition, it is possible that participants engaged in mind wandering (during which eye movements seldom occurred (Antrobus, Antrobus, & Singer, 1964) or other similar processes. Consequently, comparing MS rates with this type of long-term and monotonous control condition might not be optimal in exploring the general effect of cognitive processing on MS rate. 

The present study aimed to re-examine the influence of cognitive processing and its difficulty on MS rates using a mental arithmetic task. In contrast to tasks used by Siegenthaler et al. (2014), which involved continuous (3 min in duration) calculation using the same operation, the present study employed a task in which participants performed a single randomly assigned addition or subtraction operation during each trial. This task offered a number of advantages. First, because the required operation was unpredictable, intense calculation was executed within a short period of time (∼1 s), followed by a period to maintain the response in memory. Thus, potential noise in the data due to mind wandering and other similar activities was minimized. This addressed a potential issue that could have arisen if a task involving the same calculation over an extended period of time was used without constantly monitoring participant performance. Secondly, the difficulty of the arithmetic task required mental processes that produced a suitable duration for reliable recording of MSs, considering the rates are usually quite low (∼1–2/s). However, they did not persist long enough to induce inattention. Furthermore, the task offered adequate time to sample MS rates at different stages of the task within a single trial. Finally, this design allowed identification of task difficulty through analysis of subjective complexity (Hess & Polt, 1964; Siegenthaler et al., 2014) and response time (RT), a more objective measure (Nakayama, Takahashi, & Shimizu, 2002). 

Two experiments were conducted. Experiment 1 aimed to explore the correlation between MS rate and task difficulty as defined by subjective complexity. Participants were required to report the results of their calculation at a predetermined time, presumably well after the calculation had been completed. MS data was averaged in reference to the onset of the stimulus, which allowed for observation of changes in frequency during the initial phase of cognitive processing. In Experiment 2, participants were required to give an immediate oral response upon finishing the calculation. Focus was placed on changes in MS rate between, before, and after this report was given. Data across trials was time-locked to the start of verbal response, which allowed for examination of MS rates with (during calculation) and without (postcalculation) intense mental processing to reveal the general effect of cognitive processing on MS. Separate analysis of MS rate during and after a given operation also allowed for more precise quantification of the effect of task difficulty. The preliminary results of these experiments have been presented in abstract form (Gao, Li, Cai, & Sun, 2013). 

Experiment 1

Materials and methods

Participants

Ten participants (three female, seven male) aged 23–27 years (average 24.6 years) were recruited and paid for their contributions. All individuals were majoring in science or engineering, achieved standard level scores in mathematics, and had normal, uncorrected visual acuity. While they were not initially made aware of the true purpose of the experiment, informed consent was obtained from all participants. This study was approved by the Committee for Ethics and Human Participants in Research at the University of Electronic Sciences and Technology of China in Chengdu, China. 

Apparatus

The experiment was performed in a sound-attenuated room specifically designed for psychophysics research, and illumination of the area was held constant across all participants. Individuals observed a screen (1024 × 768 pixels, 100-Hz refresh rate) from a distance of 51 cm, and their head movements were restricted by forehead and chin rests. The stimulus presentation program was compiled in MATLAB (MathWorks, Natick, MA) using Psychtoolbox (Brainard, 1997; Pelli, 1997). Eye movements were recorded binocularly with an infrared eye tracker (EyeLink1000, SR Research Ltd., Mississauga, ON, Canada) that sampled at 1000 Hz, allowing head movements of up to 25 × 25 × 10 mm (horizontal × vertical × depth). 

Stimulus

Stimuli, including operational signs of 0.8° and numbers of approximate 1.4° × 0.8° (height × width per digit) were displayed at the center of a screen in white against a gray background. All numbers were surrounded by a black circle 4° in diameter. In the response phase, to allow participants to select their answer, the screen showed multiple numbers evenly spaced in a circular arrangement, with one being the correct answer. This method of response selection was mainly adapted from the method outlined by Knops, Thirion, Hubbard, Michel, and Dehaene (2009). 

Procedure

Prior to the first and the 17th trials, a 9-point eye position calibration procedure was performed and validated. After every eighth trial, a drift correction was implemented. Prior to each trial, participants were required to fixate on a solid white circle of radius 0.2° (fixation spot), displayed at the center of the screen. The trial began when the individual's gaze position was detected within a 1.5° central area surrounding the fixation point for more than 100 ms. If the gaze did not fall within this critical zone, the calibration procedure was repeated. 

The sequence of events in a typical trial is depicted in Figure 1. After the disappearance of the fixation spot, an operational sign (+ for addition, − for subtraction) indicating the required subsequent calculation appeared at the center of the screen for 1200 ms. It was then replaced by the first number (N1), which was displayed for 300 ms before the second number (N2) appeared in its place for the same duration. The operational sign then reappeared in the central location for another 2000 ms, followed by eight numbers that served as multiple answer choices, with one being correct. These options appeared simultaneously, evenly spaced in a circular formation with angular separations of 45°, and remained on the screen until the participant responded, or for a maximum duration of 6000 ms. A blank screen was then presented for 3000 ms before the start of the next trial. Participants were asked to select their answers using a mouse and instructed to click on a blank space if their response was not among the eight options provided. During analysis, blanks were counted as incorrect responses since the right answer was always displayed. Throughout a given trial, participants were required to fixate on the center of the screen and minimize blinking activity until the response panel appeared. Each individual underwent six to eight blocks, with 30 trials per block. 

Figure 1

 

A schematic depiction of the sequence of events within a trial. After the initial presentation of the operational sign, two numbers were displayed successively, followed by a repeated presentation of the same operational sign. Then, following a 2000-ms (Experiment 1) or 5000-ms (Experiment 2) interval, eight options appeared on the screen. Participants were able to select their answer using a mouse. In Experiment 2 only, participants were required to make an oral response as soon as they completed the calculation before also manually selecting an answer.

Manipulation of task difficulty

The value of N1 was randomly selected from a range of 42–58 (50 ± 8) and the value of N2 was randomly selected from one of two ranges with equal probability: 3–9 (6 ± 3) and 23–29 (26 ± 3). We assumed that tasks involving the larger N2 (23–29, with two digits) should be more difficult than that involving relative smaller N2 (3–9, with single digit). Then, in consideration of the compound effect of the complexity of the operation, we also isolated a second factor from the magnitude of N2 (i.e., whether the arithmetic involved a carrying or borrowing step). The arithmetic operation could be labeled “easy” if it did not require addition with carrying or subtraction with borrowing, and “hard” if either were necessary. Overall, combinations of these attributes would yield four conditions: small-easy (e.g., 53 + 6 = 59, 56 − 4 = 52), small-hard (e.g., 56 + 7 = 63, 52 − 6 = 46), large-easy (e.g., 52 + 26 = 78, 57 − 23 = 34), and large-hard (e.g., 56 + 27 = 83, 44 − 28 = 16). These task difficulty levels were selected to yield a suitable duration of a single trial (long enough for good recording and short enough to avoid fatigue) and great enough variability. 

Data analysis

Only data acquired prior to participant response was included in the following analysis. Trials in which blinks occurred after the initial display of the operational sign (13.54%) and those with incorrect responses (4.70%) were also removed. In total, 18.24% of all trials were discarded. In this paper, all error bars and the ranges of error specified after the values of the mean were generated from SEM (i.e., mean ± SEM). 

MS detection

Binocular MSs were detected using the algorithm described by Engbert and Mergenthaler (2006), using λ = 6 for velocity threshold detection and a minimum MS duration of 6 ms. MSs with amplitudes greater than 1.5° were excluded in the analysis. Additionally, if the interval between two MSs was shorter than 20 ms, they were merged to exclude potential overshoot corrections (Møller, Laursen, Tygesen, & Sjølie, 2002). Figure 2 shows the significant linear relationship between the peak velocities of detected MSs and their respective amplitudes, known as the “main sequence.” This is typical of large saccades and MSs (Zuber, Stark, & Cook, 1965), and thus confirmed that MSs were accurately detected within the present study. Furthermore, a method of causal filter, which was initially developed to analyze neural firing rates (Dayan & Abbott, 2001), was adapted to compute continual MS rate patterns (Rolfs, Kliegl, & Engbert, 2008). The decay parameter was set to α = 1/40. 

Figure 2

 

Main sequences. The peak velocities of MSs were plotted against their respective amplitudes for Experiments 1 and 2. A significant linear relationship was observed in both experiments. N, r, and p in each Figure represent the total number of MSs, the correlation coefficient, and significant level, respectively.

Results

The overall accuracy of the mental arithmetic task was 95.3%. The mean performances of the four difficulty conditions (small-easy, small-hard, large-easy, and large-hard) were 0.97 ± 0.01, 0.95 ± 0.02, 0.93 ± 0.02, and 0.91 ± 0.03, respectively. Repeated-measure analysis of variance (ANOVA) showed that the differences in accuracy among these conditions did not reach the significant level, F(3) = 2.761, p = 0.061. Note that trials with incorrect responses were excluded from further data analysis. 

To exclude the influence of changes in visual stimuli on MSs, only data during the second presentation of the operational sign was included in this analysis, although the calculation could theoretically begin during the presentation of N2. Then, to assess the effects of task difficulty on MS rate, a repeated-measures ANOVA was performed using task difficulty as the within-subject factor (four levels: small-easy, small-hard, large-easy, large-hard). The analysis revealed a significant modulation of MS rate by task difficulty, F(3) = 15.82, p < 0.0001. Post hoc comparison revealed MS rate within the small-easy condition to be significantly higher than the respective rates identified within the small-hard (p < 0.05), large-easy (p < 0.01) and large-hard (p < 0.0001) conditions. 

Figure 3 depicts the comparison of MS rates between conditions with the presumed largest difference in task difficulty (small-easy and large-hard, Panel A) and a moderate or minimal difference (small-hard and large-easy, Panel B). The first 10 columns illustrate results for each participant, and the last column represents data averaged across all individuals. The difference in MS rates between the small-easy and large-hard conditions was significant (p < 0.0001), which was consistent throughout all participants. In the two intermediate difficulty conditions (small-hard and large-easy), the MS rate difference between them was not large and more variable in direction across participants (Figure 3B). Most subjects (seven out of 10) showed higher MS rate in the small-hard condition than in the large-easy condition, while others exhibited the reverse trend. This might because that when the difference in difficulties was small, the perceived difficulty was varied across participants due to different strategies of calculation. Overall, no significant difference in MS rate was observed between small-hard and large-easy conditions in the group analysis (p > 0.05). 

Figure 3

 

The overall MS rate within different conditions during the second presentation of the operational sign. The first 10 columns indicate results from individual participants, and the last column represents the result averaged across all individuals. The error bars represent the standard error of the mean (SEM) and the asterisks (*) indicate significant differences between the two conditions (***p < 0.0001). (A) Comparison of MS rate between small-easy and large-hard conditions. (B) Comparison of MS rate between small-hard and large-easy conditions.

In addition to the overall MS rate, frequency changes over time were also of interest. Figure 4A shows MS rates within small-easy and large-hard conditions as a function of time averaged across all participants. The zero on the abscissa represents the onset of the second displayed operational sign. Two curves largely overlapped prior to and soon after this point. However, at around 300 ms, they separated from each other, and MS rates within the small-easy condition were greater than those within the large-hard condition. Paired t tests were performed at each time point to compare differences in frequency. As nearly no p value was significant after the strict false discovery rate (FDR) correction procedure (Benjamini & Yekutieli, 2001), a moderate criterion of p < 0.005 was used (Lieberman & Cunningham, 2009). For most time points from 750–1600 ms during the calculation phase, MS rate was significantly higher during the small-easy tasks compared to large-hard tasks. Because visual stimuli were identical between the two conditions, with only the operational sign being displayed, it was inferred that the difference in MS rate between the two tasks most likely resulted from variations in task difficulty. 

Figure 4

 

The temporal dynamics of MS rate. (A) Comparison between small-easy and large-hard conditions. (B) Comparison between large-easy and small-hard conditions. The black lines at the bottom of each plot denote periods in which MS rate within the two conditions differed significantly (t test, p < 0.005). The thin lines above and below the thick line indicate the SEM envelopes.

Likewise, the two conditions encompassing moderate task difficulty were compared (see Figure 4B). Consistent with averaged results depicted in Figure 3, the two curves representing temporal dynamics of MS rate within small-hard and large-easy conditions nearly overlapped, even during the phase in which participants were engaged in mental arithmetic. 

Discussion

In Experiment 1, a significant relationship was found between MS rate and subjective task difficulty. This is consistent with results published by Siegenthaler et al. (2014), which reported an overall lower MS rate when individuals were engaged in more difficult tasks. In addition to the overall MS rate within different conditions (Figure 3), temporal dynamics were also examined. Comparison of the two conditions with the largest contrast in task difficulty revealed a significant difference in MS rate soon after the stimulus was presented. Since visual information was consistent between the various conditions during the calculation phase, this difference in MS rate was most likely induced by changes in task difficulty. 

In the current experiment, participants responded at a predetermined time long after the calculation was presumably completed. Consequently, the MS recording was free from artifacts that could possibly be induced by the manual response. However, this procedure did not allow identification of the moment at which the participant finished their calculation. Thus, the MSs within different phases of the task, such as during and postcalculation, could not be differentiated in Experiment 1. It is important to note that the transition between these phases could occur at different times for different trials. Thus, it is not clear whether the difference in MS rate resulted from the effects of task difficulty (small-easy vs. large-hard calculation) or different processing phases in the task (during calculation vs. postcalculation). To separate these effects on MS rates, it is necessary to separate the interval of intense mental processing from the interval after it is complete. Furthermore, in the present experiment, subjective complexity was used to determine task difficulty. Although the differences in accuracy across four difficulty conditions did not reach the significant level (p = 0.061), it may not necessarily indicate the absence of differences between four task difficulty levels. In Experiment 1, the participants were required to report their answers by mouse after the 2-s delay, and the response interface lasted for a maximum duration of 6 s. Thus, they have sufficient time to complete the arithmetic task. In this context, the accuracy might not be an optimal index to reflect the levels of task difficulty. A more objective form of measurement, such as RT, could be useful as an alternative determinant of this factor as shown in the next experiment. 

Experiment 2

Materials and methods

Participants

Participant selection criteria and recruitment methods were identical to those employed in Experiment 1. Fourteen participants (eight male, six female) aged 23–28 years (average 24.5 years) were involved in this experiment, with four of them participating in both Experiment 1 and Experiment 2, and the remaining 10 subjects only completing Experiment 2. 

Apparatus

In addition to the apparatus used in Experiment 1, a microphone (Takstar, Huizhou, China) and a sound card with a sample rate of 22050 Hz (Andigy-4 0610, Creative, Singapore) were used to record the participants' verbal responses. 

Stimuli and procedure

Three differences in stimuli and procedures were implemented between Experiments 1 and 2. First, the duration of the second-displayed operational sign was lengthened from 2000 ms to 5000 ms to ensure that all calculations could be finished within this duration. Second, RT was measured by having participants report their answers verbally as soon as they finished the arithmetic task. Following the second presentation of the operational sign (5 s), individuals were then asked to choose the same answer at the response panel, as they did in Experiment 1. Participants were requested to remain silent except when reporting their answers, and their respective verbal responses indicated the completion of calculation. Next, six control trials were randomly inserted within a block in which operational signs were replaced by a white dot of the same size. Participants were instructed beforehand to fixate on the dot without performing any arithmetic tasks for these trials. Collection of eye movement data during the presentation of the second dot lasted for 2000 ms and was used as a baseline reference. In total, 12 participants completed nine blocks of 30 trials and two stopped after eight blocks, reporting fatigue and requesting to terminate the experiment. Thus, only eight blocks of data were included in the analysis for these two individuals, with no substitute procedures implemented. 

Data analysis

Trials containing eye blinks during the second presentation of the operational sign were removed from the analysis. Furthermore, trials with incorrect answers and improper verbal responses were also not included. In total, 15.36% of noncontrol trials were discarded and 12.70% of control trials were rejected due to blinks. 

Measurement of RT

As in Experiment 1, the focus was placed on MS data during the second presentation of the operational sign due to identical visual displays being presented across trials. For simplicity and consistency, we also defined RT with reference to the onset of this time interval. 

For each trial, RT was defined as the time interval between the beginning of the second presentation of the operational sign and the start of verbal report, calculated according to analysis of audio data (Figure 5B, inset). First, baseline level was defined as the mean absolute amplitude of the audio recording within the first and last 200 ms of the 5-s operational sign presentation. Trials in which the peak sound amplitude fell within either of these respective intervals were rejected to ensure that no verbal responses occurred at baseline. Then, mean absolute sound amplitude was calculated over a sliding window (100-sample width) by five-sample step. The first and last bins in which mean absolute values were 10 times larger than those at baseline were defined as the respective start and end points of verbal report. Trials in which this interval exceeded 800 ms were not included in further analysis because most were accompanied by improper verbal response, which may have resulted in incorrect indication of RT. 

Figure 5

 

The respective RT distributions and means during each task difficulty condition. (A) Distribution of RTs during the four conditions based on the subjective complexity of the calculation; the bin size is 400 ms and the ticks on the abscissa represent the respective midpoints of each bin. (B) Average RTs during each of the four conditions. Mean RTs were averaged across all 14 participants and error bars represent SEM. The plot in the insert shows the sound pattern of a typical verbal response, with RTs being calculated from the onset of the second presentation of the operational sign (time zero) until the start of verbal report (left dashed vertical line).

Results

The respective RT distributions and means across four conditions

Figure 5A shows the respective distributions of RTs within each of the four difficulty conditions previously outlined in Experiment 1. RTs observed during trials in the small-easy condition were generally short. In contrast, most RTs identified during trials in the large-hard condition were comparatively long. The mean RT calculated during each of the four task difficulties is depicted in Figure 5B. RTs generally correlated with classifications made by the experimenters, with the shortest recorded during small-easy tasks (0.82 ± 0.04 s) and the longest recorded during large-hard tasks (1.95 ± 0.10 s). RTs during the two intermediate conditions were quite similar (1.34 ± 0.06 s and 1.43 ± 0.03 s during small-hard and large-easy, respectively). 

Additionally, although the overall accuracy was very high (∼96%), it was still influenced by the task difficulty, F(3) = 8.307, p < 0.001, the accuracies of the four task difficulty levels (small-easy, small-hard, large-easy, large-hard) were 0.98 ± 0.01, 0.98 ± 0.01, 0.95 ± 0.01, 0.92 ± 0.03, respectively. Note that incorrect trials were not included in further data analyses. 

MS rates for during and postcalculation

Figure 6A shows MS rates over time, averaged across all participants. The start of verbal response was treated as time zero for all trials, and the shaded area represents its average duration. Notably, this graph only presents MS rates recorded from 800 ms preresponse until 2000 ms postresponse. To ensure that this interval was captured in all trials, only data revealing RTs between 800 ms and 3000 ms was graphed. In contrast, the rest of the analysis includes all trials regardless of RT value. 

Figure 6

 

The MS rates observed within during-calculation phase, postcalculation phase, and control condition. (A) MS rates over time were plotted in reference to RTs. The gray area represents the mean durations of verbal report. The thick dashed line indicates MS rate within the control condition. The thin dashed lines indicate SEM envelopes. (B) Averaged MS rate within different analysis intervals. The error bars represent SEM.

Considering the impact of head or mouth movements on MSs, those that occurred during verbal report were excluded in the quantitative analysis (Figures 6B and 7B). However, to maintain the integrity of the description, they were retained in the descriptive analysis (Figures 6A and 7A). During the calculation phase, MS rate was observed to gradually increase before response, but the frequency was relatively low compared to postresponse levels. Immediately after the verbal response, MS rate increased and remained at a level approximately double that of during-calculation phase. However, MS rates within any phase during the experimental condition were generally lower compared to those within the control condition. To quantify average frequencies among different phases within the task, the mean MS rate across all participants was calculated. As shown in Figure 6B, overall MS rates during calculation were significantly lower than those for postcalculation (paired t test, p = 0.015). Furthermore, postcalculation rates within the experimental condition were significantly lower compared to those within the control condition (paired t test, p = 0.006). 

Figure 7

 

Correlations between MS and RT. (A) Onset time of all MSs against RTs from corresponding trials. The MSs that occurred after verbal response (above the diagonal line) were more frequent than those occurred before (below the diagonal line). (B) General relationship between MS rate and RT. Each point in the graph represents the average MS rate across trials with their RTs fell within a bin of 400 ms. Ticks on the abscissa represent the centers of each bin.

In addition to MS rates within different processing phases of a task, the relationship between MS rate and task difficulty was also examined by comparing trials with different RTs, which represented the amount of time necessary for processing and completing the mental calculation. A longer RT was associated with more difficult calculation. 

Figure 7A illustrates the time of onset for all MSs against the corresponding RT from that trial. Trials without MSs were not displayed. MSs occurring before verbal responses (below the diagonal line) were fewer and more scattered compared to those observed after responses (above the diagonal line). 

The entire range of RTs was subdivided into nine bins of 400 ms in size (except the first and last bins, which were more than 400 ms). Mean MS rate was then calculated within each bin. Since trials with RTs below 400 ms and above 3600 ms were rare, data from these trials were merged into two single bins, respectively. In Figure 7B, each data point represents the averaged MS rate from its corresponding bin, which could be generated by partitioning the graph in Figure 7A into nine vertical columns, separately above and below the diagonal line, and calculating the mean within each column portion. While trials without MSs were not displayed in Figure 7A, MSs from all trials within a given time window were averaged in Figure 7B. However, MSs that occurred during verbal report were rejected, as was the case for Figure 6B. 

Again, MS rate was lower during calculation than after calculation, regardless of RT. A significant linear relationship was observed between the MS rate and the RT during calculation (Pearson correlation: r = −0.79, p = 0.01), but this did not hold in the postcalculation phase (r = −0.45, p = 0.23). This result suggested a correlation between MS rate and task difficulty only during periods of intense mental processing. 

Discussion

In Experiment 2, RT was employed as a more precise and objective method for determining task difficulty compared to subjective complexity. Differences in RT distributions across the four difficulty conditions were observed. These patterns correlated with the subjective complexity of tasks identified in Experiments 1 and 2. By isolating the period of mental calculation, MS rate was also found to have an inverse relationship with task difficulty. Additionally, MS rate within the during-calculation phase was much lower than the postcalculation phase, which was comparatively lower relative to the control condition. 

It might be argued that the higher MS rate observed within the postcalculation phase was due to lingering effects produced by head and mouth movements from verbal report, despite the exclusion of MS data during actual verbal responses (Figures 6B and 7B). However, there are several reasons why any residual speech-producing movement could not have had a profound effect on the overall pattern of results. First, the postcalculation MS rate remained persistently higher than the during-calculation frequency, even after a considerable length of time had elapsed since verbal report (Figure 6A). Secondly, comparable result patterns existed regardless of whether MS data collected during verbal report was included (Figure 6A, B). Finally, in Experiment 1, no verbal answer was required, yet general response patterns such as the inverse relationship between MS rate and task difficulty were similar to those found within Experiment 2. Thus, although movements involved in verbal report could create some noise within the data for that specific time interval, it remained an effective, if not necessary, means by which to measure the RT of a cognitive task and distinguish between different processing phases. Verbal report allowed participants to respond in a way that was more informative than pressing a button to indicate completion of a task. In the latter case, participants could have chosen to report an answer before the actual processing had been completed. 

The results of Experiment 2 also help explain some results obtained during Experiment 1. Initially, participants were not required to respond until the end of a 2000-ms calculation phase. However, for most trials, individuals completed the calculation well before this time interval had expired. The overall effect of task difficulty on MS rates could represent the influence of cognitive load required for task completion, as well as different processing phases within the task (with vs. without intense cognitive processing). At a given time following stimulus presentation, some trials could still be in a during-calculation phase while others have already moved into postcalculation phase. As observed in Experiment 2, trials with comparatively lower RTs (e.g., those in the small-easy condition, Figure 5) corresponded with higher MS rates in the during-calculation phase. The reverse is also true: Relatively higher RTs (e.g., those in the large-hard condition, Figure 5) were accompanied with the lower MS rates in the during-calculation phase. Furthermore, since arithmetic problems within small-easy trials were generally solved faster, participants entered the postcalculation phase earlier, which could have led to higher MS rates. Overall, the combined effect would lead to the comparatively more frequent MSs within small-easy trials observed in Experiment 1. 

General discussion

While the majority of research concerning MSs explores their role in visual processing, the present study focused on their behaviors during nonvisual cognitive processing. Two main findings were identified. First, MS rate was observed to decrease with increased task difficulty during calculation. Secondly, MSs were suppressed during intense cognitive processing (mental arithmetic) regardless of task difficulty. 

To investigate the relationship between task difficulty and MS rate, in Experiment 1, task difficulty was varied across trials by subjective complexity. The focus was placed on early ocular response, thus the temporal sequence of the data was aligned with stimulus onset time. In Experiment 2, we used a more objective measurement of task difficulty. Moreover, temporal sequence of the data was aligned with the verbal response onset time. This allowed us to isolate the data during calculation from those after calculation. In both experiments, we observed an inverse relationship between task difficulty and MS rate (to be more specific, MS rate during calculation in Experiment 2). 

In Experiment 2, to assess the general effect of cognitive processing, MS rate was also compared between phases with different cognitive requirements. A within-trial comparison between during-calculation and postcalculation phases was made that likely offered a more sensitive paradigm compared to a between-trials comparison utilized in many other studies (e.g., Siegenthaler et al., 2014). In addition, considering that mental processing was required even after the calculation but before response (i.e., the answer must be kept in memory while preparing for subsequent manual response), a control condition incorporating a matching sequence of events but without the need for calculation or memory was also implemented. The sampling time during this control condition was short (2000 ms during the presentation of the second dot) in order to minimize the possibility of mind-wandering activities. Thus, by comparing MS frequencies across phases with different cognitive demands, the general effect of mental processing on MS rate could reliably be observed. Overall, lower MS rates were observed within the during-calculation phase compared to postcalculation, and the highest frequencies were found in the control condition. This confirmed that MSs were suppressed by mental processing. 

Comparing the effects of task difficulty on MS rate revealed by between-trial and within-trial analysis

In the present study, by comparing MS rates between trials with different RTs (between trials) and between different phases in a task (within trial) for the same set of participants, we were able to offers an effective test of effect-of-task difficulty in two different scenarios. First, the effect-of-task difficulty can be revealed through comparison between trials with different levels of mental efforts required in performing the same type of task (but with different difficulties). The effect-of-task difficulty can also be revealed by comparing different phases (e.g., phases with vs. without intense cognitive processing) within the same trial. In Experiment 2 of the current study, because of different cognitive demands, one might consider the phase for during-calculation and the phase for postcalculation with different “cognitive states.” Our results showed that although an inverse relationship was observed between task difficulty and MS rate using between-trials comparisons of varying RTs, the amplitude of such modulation on MS rate by task difficulty was much smaller compared to the effect of cognitive states (task phase differences). For example, while an RT of 3200 ms is quadruple that of 800 ms, MS rates within trials exhibiting RTs of ∼3200 ms only decreased by one third (∼0.6 MSs/sec to 0.4 MSs/sec) compared to those exhibiting RTs of ∼800 ms. In contrast, during-calculation MS rates decreased by one half compared to postcalculation rates. The dramatic change in MS rate between phases in contrast to the small difference in MS rate between trials of different difficulties suggests that cognitive state has a much greater impact on MS rate than task difficulty does. 

Relations between MS rate and cognitive processing in other related studies

Findings in the present study are consistent with other research illustrating the relationship between MS rate and cognitive processing. These studies suggested that MS rate could be modulated by cognitive factors, such as the working memory load or general arousal level. Siegenthaler et al. (2014) explored the correlation between the task difficulty and MS rate in a mental arithmetic task and showed that MS rate decreased with increased task difficulty. They attributed the influence of task difficulty on MS to working memory load. Greater demands on working memory led to lower MS rates. 

Siegenthaler et al. (2014) also found that the MS rate in control condition, in which only prolonged fixation was required, fell between the easy and hard tasks. However, the present study observed that MS rates within the during-calculation phase were consistently lower than those within the postcalculation phase and control condition, regardless of variations in task difficulty. A possible explanation might be that during a long fixation period of 180 s (3 min), participants in Siegenthaler et al. (2014) might have sometimes engaged in mind wandering, which would have decreased MS rate, given that eye movements seldom occur during daydreaming (Antrobus et al., 1964). In the present study, the possibility of engaging in these activities was minimized, allowing more attention to be paid to maintaining fixation, and leading to relatively higher MS rates. 

Valsecchi et al. (2007) examined MS rate in an experimental paradigm of visual oddballs (rare targets) in which participants were required to mentally count the number of oddballs (active condition) or ignore them (passive condition). It was observed that in the active condition, MS rate was significantly lower after the presentation of oddballs compared to standard stimuli, but such a difference was not observed in the passive condition. A similar experimental paradigm in the auditory modality was also conducted (Valsecchi & Turatto, 2009) and researchers again observed a reduced MS rate after the presentation of auditory oddballs compared to standard stimuli, further indicating that the working memory load of mental counting could suppress the occurrence of MSs. 

Betta and Turatto (2006) explored the influence of response preparation on MS activity using a task in which participants were asked to respond to a visual stimulus. Before stimulus presentation, MS rate was observed to decrease in conditions requiring response to a future stimulus compared to those requiring no response. Since the response preparation phase preceded the presentation of visual stimuli, differences in MS rate most likely resulted from manipulation of cognitive factors, such as the general arousal level suggested by the authors. 

Correlations between MS rate and cognitive processing were also examined in large saccades that have been shown to share many common characteristics with MSs (Haddad & Steinman, 1973; Hafed, 2011; Hafed, Goffart, & Krauzlis, 2009; Otero-Millan, Macknik et al., 2013; Otero-Millan et al., 2008; Otero-Millan, Macknik et al., 2011; Steinman et al., 1973; Zuber et al., 1965). Ehrlichman, Micic, Sousa, and Zhu (2007) examined saccade rates during cognitive processing using a series of nonvisual tasks and generally found low frequencies when participants were engaged in working memory processing tasks such as mental counting, delayed repetition, and auditory N-back (Micic, Ehrlichman, & Chen, 2010). Other studies also observed reduced saccade rates when participants fell into stalemate situations while solving insight problems (e.g., match stick arithmetic task, Knoblich, Ohlsson, & Raney, 2001) and engaged in daydreaming (Antrobus et al., 1964). Thus, the present findings suggest a new commonality between saccades and MSs that both could be suppressed when people were engaged in cognitive processing (mainly involving demand of working memory). 

Mechanisms for the effect of cognitive processing on MS rate

Although the present study focused on effect of cognitive processing, participants were required to perform mental arithmetic and maintain eye fixation simultaneously during the task. To interpret changes in MS rate during different phases of the task, both mental and ocular control demand dynamics should be considered at any given time when participants performed a trial. Mental calculation could be considered the priority in the during-calculation phase of the task, as the participants were required to report the solutions verbally or manually afterwards, whereas the visual requirement, such as maintaining precise fixation, could be treated as a secondary task. Priority during the postcalculation phase could have been switched to accommodate visual demand, and MSs were recovered to maintain precise fixation (Cornsweet, 1956; Costela et al., 2014; Ditchburn & Ginsborg, 1953; Engbert & Kliegl, 2004; Otero-Millan et al., 2013; Otero-Millan et al., 2011). 

Therefore, when cognitive demand (attention, memory, and executive control, etc.) is high, MS rate would be expected to be low. Although there is no convincing evidence reflecting reciprocal interference between cognitive processing and ocular control, the (micro)saccade does consume a notable quantity of resources and seems to exhibit no functional role in cognitive processing. Apart from the motor execution of a saccade per se, extensive computation processes in the neural system also accompany its occurrence in order to maintain stable and continuous vision (for a review see Wurtz, 2008). Saccade suppression describes the phenomenon in which visual sensitivity and neural response along the visual path are suppressed shortly before and during saccades (Bridgeman & Macknik, 1995; Diamond, Ross, & Morrone, 2000; Ross, Morrone, Goldberg, & Burr, 2001; Wurtz, 1968, 1969). The perception of spatial position (Ross, Morrone, & Burr, 1997), time (Morrone, Ross, & Burr, 2005), and quantity (Binda, Morrone, Ross, & Burr, 2011; Burr, Ross, Binda, & Morrone, 2010) adjacent to saccades can be altered and compressed in a process known as saccade compression. Most of these mechanisms are also found to exist in MSs (Hafed, 2013; Hafed & Krauzlis, 2010; Hafed et al., 2011; Herrington et al., 2009; Zuber & Stark, 1966). Thus, when participants were engaged in a situation that did not require visual processing, MSs and their accompanying neural computations seem neither necessary nor helpful to solving the problem. Furthermore, with an increase in the mental effort or cognitive load required to complete the task, the costs associated with these eye movements seem increasingly unnecessary, and are thus inhibited to a greater extent. 

Comparing the effects of task difficulty of visual processing and nonvisual cognitive processing

Most studies exploring the relationship between MS rates and difficulty of (primarily) visual processing have also observed an inverse relationship between MS rate and task difficulty (Laubrock et al., 2005; Pastukhov & Braun, 2010, but see Benedetto et al., 2011). Pastukhov and Braun (2010) proposed that the observers might voluntarily control MS rate based on the perceptual cost or benefit of MSs in a given situation. Under conditions in which the visual target was presented briefly (Laubrock et al., 2005; Pastukhov & Braun, 2010), reduced MS rates within a comparatively difficult task (e.g., recognizing the shape of a letter as opposed to its color) might contribute to preventing MS suppression, in which the important visual target might be missed or suppressed when visual stimuli coincided with MSs (Hafed & Krauzlis, 2010; Hafed et al., 2011; Herrington et al., 2009; Zuber & Stark, 1966). Meanwhile, for conditions in which viewing time was not limited (Benedetto et al., 2011), increased MS rates observed within comparatively difficult tasks might benefit visual processing. For example, the MS could serve to scan small regions (Haddad & Steinman, 1973; Otero-Millan, Macknik et al., 2013; Otero-Millan et al., 2008; Steinman et al., 1967; Steinman et al., 1973), improve high-acuity visual processing (Ko et al., 2010; Poletti et al., 2013; Rucci et al., 2007), or maintain an optimal sampling strategy (Martinez-Conde & Macknik, 2008; Martinez-Conde, Macknik, Troncoso, & Hubel, 2009; McCamy, Otero-Millan et al., 2014). 

However, the nature of task difficulty in primarily visual studies might be notably different from its nature in primarily nonvisual investigations, as in the present study. In the former context, it is mainly reflected in the degree of mental excursion required to perform visual processing. In contrast, task difficulty in nonvisual tasks generally reflects the amount of mental effort involved in processing nonvisual information. Thus, the explanation proposed by Pastukhov and Braun (2010) might be reasonable for visual processing, but unsuitable for explaining the results obtained from the present study and other investigations concerning the influence of nonvisual cognitive processing on MSs. 

Conclusion

Using a novel mental arithmetic experimental paradigm and detailed data analysis throughout various phases of the cognitive task, it was observed that MS rate correlated negatively with task difficulty, regardless of whether the latter was measured via subjective task difficulty or the more objective RT measurement. Furthermore, MS rates differed greatly for different task phases within a trial with different cognitive states. MS in the postcalculation phase remained at double the rate obtained within the during-calculation phase, and MS rate in the control condition was much higher compared to postcalculation. Our results suggest that the nonvisual cognitive processing can suppress the MS rate, and that the extent of such suppression is related to the task difficulty. 

Acknowledgments

The authors wish to thank Chaoyi Li for his suggestions and thank Yongchun Cai for his contribution to the earlier part of project. This work was supported by the 973 Project (2013CB329401), Natural Science Foundations of China (91120013, 61375115, 31300912), fundamental research funds for Central Universities of China (ZYGX2013J098), and the Natural Sciences and Engineering Research Council of Canada. Funders contributed no input concerning the study design, data collection and analysis, decision to publish, or preparation of the study. 

Commercial relationships: none. 

Corresponding authors: Hongmei Yan and Hong-jin Sun. 

Email: [email protected]; [email protected]. 

Address: Key Laboratory for NeuroInformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu, China.