Open Access
Article  |   January 2023
Emerging order of anomalous eye movements with progressive drowsiness
Author Affiliations
  • Shunya Umemoto
    Department of Computer Science, Chubu University Graduate School of Engineering, Aichi, Japan
    [email protected]
  • Yutaka Hirata
    Department of Robotic Science and Technology, Chubu University College of Engineering, Aichi, Japan
    Academy of Emerging Sciences, Chubu University, Aichi, Japan
    Center for Mathematical Science and Artificial Intelligence, Chubu University, Aichi, Japan
    [email protected]
Journal of Vision January 2023, Vol.23, 17. doi:https://doi.org/10.1167/jov.23.1.17
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Shunya Umemoto, Yutaka Hirata; Emerging order of anomalous eye movements with progressive drowsiness. Journal of Vision 2023;23(1):17. https://doi.org/10.1167/jov.23.1.17.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

It has been widely recognized that human alertness is reflected in the eyes (e.g., when drowsiness, miosis, slow saccades, divergence, less compensatory vestibulo-ocular reflex, and less-accurate optokinetic response and smooth pursuit emerge). Previous studies that discovered these pupil/oculomotor anomalous behaviors along with lowering alertness evaluated only one or a few of them simultaneously, thus their emergence order is yet unknown. Presently, we focused on the following five pupil/oculomotor behaviors that can be evaluated under a natural stationary environment without giving external sensory stimulations: saccades, slow-phase eye movements, vergence, pupil diameter, and blinks. We demonstrate that their anomalous behaviors emerge in the following order: first: frequent saccades; second: slow saccades; third: divergence & miosis, then slow eye movement, while elongated eyelid closure duration emerges randomly in this sequence. These results provide a basis for the oculo-pupillometry–enabling objective monitoring of progressive drowsiness.

Introduction
Objective alertness monitoring has been considered inevitable to prevent human errors leading to fatal accidents (Wierwille, Wreggit, Kirn, Ellsworth, & Fairbanks, 1994), as well as to provide the preferable environment for swift sleep (Muzet, 2007) and effective work/study performance (Lovato & Lack, 2010). For these purposes, many studies have focused on the eyes, which have long been anecdotally referred to as the window to the mind. In support, neurophysiological and brain imaging studies have demonstrated that various brain regions including the cerebral cortex, brainstem, and cerebellum are involved in pupil/oculomotor control (Miall, Imamizu, & Miyauchi, 2000; Sweeney Luna, Keedy, McDowell, & Clementz, 2007; Tse, Baumgartner, & Greenlee, 2010). Consequently, various brain states have been found to be reflected in pupil/oculomotor behaviors in various forms. For example, increased mental effort dilates the pupil in parallel with the activation of the locus coeruleus (Alnæs, Sneve, Espeseth, Endestad, van de Pavert, & Laeng, 2014) whose activity is known to be related to alertness (Aston-Jones & Cohen, 2005; Carter, Yizhar, Chikahisa, Nguyen, Adamantidis, Nishino, Deisseroth, & de Lecea, 2010). Miosis (Nishiyama, Tanida, Kusumi, & Hirata, 2008; McGinley, Vinck, Reimer, Batista‐Brito, Zagha, Cadwell, Tolias, Cardin, & McCormick, 2015), divergence (Wakui & Hirata, 2011), and less compensatory vestibulo-ocular reflex (Nishiyama, Kinoshita, & Hirata, 2010) have been demonstrated to emerge before we perceive our drowsiness. After becoming aware of our own drowsiness, blink (eyelid closure) duration is elongated (Tanaka, 1999; Caffier, Erdmann, & Ullsperger, 2003), and frequent saccades (Schleicher, Galley, Briest, & Galley, 2008; Wakui & Hirata, 2013), slow saccades (Henn, Baloh, & Hepp, 1984; De Gennaro, Ferrara, Urbani, & Bertini, 2000; McClelland, Pilcher, & Moore, 2010), slow eye movements (Henn et al., 1984; Cajochen, Zeitzer, Czeisler, & Dijk, 2000; Wakui & Hirata, 2011; Wakui & Hirata, 2014), and less accurate optokinetic response (Reisine & Raphan, 1992; Yakushin, Raphan, & Cohen, 2017) and smooth pursuit emerge (Porcu, Ferrara, Urbani, Bellatreccia, & Casagrande, 1998; De Gennaro et al., 2000; Fransson, Patel, Magnusson, Berg, Almbladh, & Gomez, 2008). Collectively, these lines of evidence ensure that anomalous pupil/oculomotor behaviors, which rarely occur in one's awake state, can be reliable indexes for objective alertness monitoring. However, because only one or a few of these behaviors were evaluated simultaneously in the previous studies, it is yet unknown in what order they emerge while human alertness is decreasing. 
The current study aimed to identify the emergence order of anomalous pupil/oculomotor behaviors in human participants while their alertness is decreasing. We measured binocular eye, pupil, and eyelid movements under a stationary environment together with subjective alertness reports. We evaluated the following five behaviors that are spontaneously generated without external sensory stimulations: (1) saccades, (2) slow-phase eye movements, (3) vergence, (4) pupil diameter, and (5) blinks. 
Method
We defined anomalous pupil/oculomotor behaviors as those that never emerge in the fully awake state. We call these anomalous behaviors ocular alertness indexes (OAIs). 
Participants
Participants were 14 healthy male university students aged 22 to 26 years old who provided informed participatory consent after the aims of the study and procedures of the experiments were explained. The experiments and data management were conducted following the Declaration of Helsinki. We provided minimal constrictions to the participants: No caffeinated beverages and foods within one hour before the experiment to prevent them from maintaining high alertness or falling asleep abruptly throughout the experiment. 
Experiment
They participated in two experimental sessions: Control and Test. Data from Control sessions were regarded as normal and used to detect anomalous pupil/oculomotor behaviors in data from Test sessions. In both Control and Test sessions, participants wore a head-mount eye tracker, EyeSeeCam (EyeSeeTech, Munich, Germany) (Schneider, Villgrattner, Vockeroth, Bartl, Kohlbecher, Bardins, Ulbrich, & Brandt, 2009). The EyeSeeCam was calibrated with the target presented at the distance of “Penlight” (Figure 1 described later). Binocular horizontal and vertical eye positions, as well as pupil diameters, were recorded in a MacBook Pro (Apple, Cupertino, CA, USA) at a sampling rate of 256 Hz. No event that may have potentially risen their alertness (visual stimulus, audio noise, vibration, etc.) happened during any experimental sessions. The Control and Test sessions were conducted on the same day between 11 a.m. and 5 p.m. Six participants went through the Test session first, whereas others participated in the Control session first. 
Figure 1.
 
Schematic diagram of the experimental environment.
Figure 1.
 
Schematic diagram of the experimental environment.
Control session
In Control, participants kept their fully awake state by a mental arithmetic task (Nishiyama et al., 2010) in their supine position on a reclining chair in a dark room (Figure 1). The task was to subtract 7 from either 999, 1000, or 1001, which was randomly assigned to each participant. They were allowed to blink and breathe naturally. They were instructed to be stationary and asked to see straight-ahead around a predetermined point (“Penlight” in Figure 1) on the ceiling. The point on the ceiling was presented before the session started and was turned off during the session. The session lasted for about 200 seconds to evaluate alertness before the eyelids closed in the early stages of sleep onset. We regarded the pupil/oculomotor data for 30 seconds with correct answers during Control session in each participant as those in their fully awake state. 
Test session
In Test, the participants stayed in the same condition and went through the same procedures as in Control except for the mental arithmetic task. They were told not to resist drowsiness. From the data recorded in Test sessions, their OAIs were detected, and the emergence order was determined. 
Subjective drowsiness evaluation
Before and after each Test and Control session, the participants reported their subjective drowsiness by the visual analog scale (VAS) (Heller, Manuguerra, & Chow, 2016). In VAS, a 100 mm horizontal straight line with the phrases “Very drowsy” and “Not drowsy at all” at its left and right end, respectively, was presented. The participants were asked to draw a vertical line on the horizontal line somewhere between the left and right ends to express their current subjective drowsiness. The distance from “Not drowsy at all” to the vertical line was quantified as a subjective drowsiness level. 
Data analysis
The recorded binocular pupil and eye position data were imported to MATLAB (MathWorks, Inc., Natick, MA, USA) for offline analyses. The eye position when looking at the “Penlight” position (position where EyeSeeCam was calibrated) was recorded as 0°, but the horizontal eye position data when looking straight ahead was corrected offline to 0°. Subjective report data were also analyzed in MATLAB. 
OAIs
OAIs previously known to emerge when humans are drowsy are (1) frequent saccade (Schleicher et al., 2008; Wakui & Hirata, 2013), (2) slow saccade (Henn et al., 1984; De Gennaro et al., 2000; McClelland et al., 2010), (3) divergence (Wakui & Hirata, 2011), (4) miosis (Nishiyama et al., 2008; McClelland et al., 2010; Wakui & Hirata, 2011; McGinley et al., 2015), (5) slow eye movement (Henn et al., 1984; Cajochen et al., 2000; Wakui & Hirata, 2011; Wakui & Hirata, 2014), (6) elongated eyelid closure (Wierwille et al., 1994; Knipling, 1998), (7) elongated blink (Tanaka, 1999; Caffier et al., 2003), (8) less compensatory vestibulo-ocular reflex (Nishiyama et al., 2010), (9) less accurate optokinetic response (Reisine & Raphan, 1992; Yakushin et al., 2017), and (10) less accurate smooth pursuit (Porcu et al., 1998; De Gennaro et al., 2000; Fransson et al., 2008). In the current study, we evaluated (1), (2), and (5) (i.e., frequent saccade, slow saccade, and slow eye movement). As for (3) and (4), we did not evaluate them separately, because divergence and miosis, respectively, can emerge when one's gaze position shifts further and closer, irrespective of alertness. Instead, we evaluated the co-occurrence of divergence and miosis in violation of the near reflex, referring to it as divergence & miosis (Div-Mio) in the current study. As for (6) and (7), we evaluated them together, referring to them as elongated eyelid closure duration (EECD). We did not evaluate (8), (9), and (10) because we did not give any external sensory stimuli (vestibular and visual motion) to induce these types of eye movements in the current study. 
Detection algorithms of OAIs
There have been several algorithms developed for the automatic detection of OAIs and related pupil/oculomotor behaviors (saccade: Engbert, Sinn, Mergenthaler, & Trukenbrod, 2015a; slow eye movement: Shin, Sakai, & Uchiyama, 2011; eyelid closure: Wierwille et al., 1994; Wilkinson, Jackson, Westlake, Stevens, Barnes, Swann, Rajaratnam, & Howard, 2013; slower blink: Tanaka, 1999). In the current study, we used those conventional algorithms and modified/extended them to better detect OAIs that we currently evaluated. In all our algorithms, the OAIs that emerged within 10 seconds from the start were ignored because the data in this period tended to be unstable probably because of the psychological instability of participants. Furthermore, if an OAI was detected once but not re-detected for the next 30 seconds, we regarded it as a misdetection. Below, we summarize the conventional OAI detection algorithms and describe in detail the modifications/extensions we made in the current study. 
Slow saccade and slow eye movement
Slow saccades are saccadic eye movements whose peak velocities are slower than normal ones. Slow eye movements are unstable (i.e., fluctuating or drifting) fixational eye movements. Because very slowed slow saccades are no longer distinguishable from slow eye movements, we first detect them together by modifying one of the most popular conventional saccade detection algorithms (Engbert et al., 2015a). All conjugate eye movements (normal saccades, slow saccades, and slow eye movements) that emerge without external stimuli can be separated from fixation by adjusting the threshold of the algorithm for detecting normal saccades. The conventional algorithm uses two thresholds to detect normal saccades: one for saccadic velocity and the other for saccadic duration. One major caveat of this algorithm is that high-frequency noise in the eye data may be misdetected as saccades (see Test Session in the Evaluation of detection algorithms of OAIs section). It would not have been an issue in the study of Engbert, Trukenbrod, Barthelmé, and Wichmann (2015b), because they evaluated only normal saccades with an amplitude ≥ 0.5°. By contrast, the current study deals with smaller saccades and other eye movements (amplitude ≥ 0.07°). We modified this algorithm to detect normal saccades together with slow saccades and slow eye movements without misdetecting noise by using a double-step velocity thresholding procedure, as well as a duration threshold and an amplitude threshold (Figure 2A). In the first step of the double-step velocity thresholding, a velocity threshold was set to three times the SD of eye velocity data instead of five times in the conventional algorithm to detect normal saccades, slow saccades, and slow eye movements with relatively high peak velocities (Figure 2A①). In the second step, another velocity threshold was set to 1 SD of the eye velocity data after excluding those detected by the first step (Figure 2A②). Furthermore, we reset these thresholds every 10 seconds to set them adaptively in accordance with changes in eye velocity characteristics corresponding to participants’ alertness (Wakui & Hirata, 2013). In this way, small normal saccades, slow saccades, and slow eye movements that all have comparable peak velocities may be reliably detected altogether. Start times of these eye movements were assigned just before their eye velocities exceeded 2°/s before exceeding the threshold (3 or 1 SD of eye velocity for each participant) (Figure 2A③). Their end times were assigned to the timing when eye velocities fell below 2°/s for the first time after exceeding the threshold if they did not overshoot, or after gaze corrections that followed over-shoots (Bahill, Clark, & Stark, 1975a; Kapoula, Robinson, & Hain, 1986) (Figure 2A③). The end time of one of these eye movements and a start time of another sometimes overlapped. If these two were in the same direction, they were regarded as one continuum eye movement (Figure 2A④). To prevent misdetections strictly, in our modified algorithm, we adopted a longer duration threshold of ≥ 0.04 second instead of 0.006 second used in the conventional algorithm (Figure 2A⑤), as well as an amplitude threshold of ≥ 0.07° (Figure 2A⑥). 
Figure 2.
 
Flowcharts of our detection algorithms. (A) Our algorithm to detect eye movements. (B) Our algorithm to segregate normal saccade, slow saccade, and slow eye movement, and detect frequent saccade. Note: Sacc = Saccade; Slow EM = Slow eye movement. (C) Our algorithm to detect EECD.
Figure 2.
 
Flowcharts of our detection algorithms. (A) Our algorithm to detect eye movements. (B) Our algorithm to segregate normal saccade, slow saccade, and slow eye movement, and detect frequent saccade. Note: Sacc = Saccade; Slow EM = Slow eye movement. (C) Our algorithm to detect EECD.
Next, normal saccades, slow saccades, and slow eye movements were segregated as follows. Shin et al. (2011) used an algorithm based on velocity and amplitude thresholds on a diagram called the main sequence to detect slow eye movements. Their algorithm was not intended to detect slow saccades. In their main sequence diagram, they took the mean velocity of individual saccade for the ordinate instead of peak velocity which is more commonly used and known to be proportional to saccadic amplitudes up to 15° to 20° (Bahill, Clark, & Stark, 1975b). In our modified algorithm (Figure 2B), we aimed to segregate normal saccades, slow saccades, and slow eye movements by using the commonly used main sequence diagram whose ordinate is peak velocity. We evaluated the angle formed by the abscissa and the line connecting the origin and each data point representing each eye movement detected so far in the main sequence diagram. A data point with faster peak eye velocity (ordinate) and smaller amplitude (abscissa) has a greater angle. Then we incorporated parameters of fully awake state data recorded in Control session for each participant. The minimum angle in the fully awake data was set as the angle threshold for each participant. This angle threshold was applied to Test session data, and those with angles equal to or larger than the angle threshold were categorized as normal saccades (Figure 2B①). Among the rest, those larger than the amplitude threshold (5°, same as the conventional algorithm) were categorized into slow eye movements, and the remainings were slow saccades (Figure 2B②). 
Frequent saccade (Wakui & Hirata, 2013) and slow eye movement (Porte, 2004; Shin et al., 2011) are known to be often horizontal eye movements, whereas complete horizontal or vertical ones are probably rare. Therefore, in the above two algorithms (Figures 2A, 2B), we set thresholds on the oblique eye velocity or amplitude (c) calculated by the following equation:  
\begin{eqnarray} \qquad\qquad c = \sqrt {{a^2} + {b^2}}\qquad \end{eqnarray}
(1)
where a is the horizontal eye velocity or amplitude and b is the vertical one. 
Frequent saccade
Frequent saccades are saccadic eye movements made more frequently in shorter inter-saccadic intervals (ISIs) than those of normal saccades. We first calculated ISIs of all those detected normal saccades and slow saccades. We then defined the detected frequent saccades as pairs of saccades (including both normal and slow saccades) whose ISIs are shorter than 0.15 second (interval threshold), which is the shortest in normal adults (Carpenter, 1988; Schleicher et al., 2008) (Figure 2B③). We avoided using the minimum ISI in the fully awake state as the interval threshold to detect frequent saccade because it was shorter than 0.15 second in 10 out of 14 participants even in the fully awake state (we will discuss this finding in greater detail in the Discussion section). Note that the emergence timing of frequent saccade may coincide with that of slow saccade if the latter is produced with a shorter ISI than the interval threshold (0.15 second). 
Div-Mio
Wakui & Hirata (2011) showed that in drowsy participants miosis and divergence emerged simultaneously (diverged even to negative values), namely violating the near reflex. They visually detected this phenomenon without documenting any coded algorithms. Since no conventional algorithm had been proposed, we developed an algorithm to detect Div-Mio. 
First, we obtained vergence angle data by subtracting the right horizontal eye position data from the left one so that the positive vergence angle represents convergence. Because the horizontal eye position data when looking straight ahead was corrected to be 0° (previously mentioned), the vergence angle was 0° when the gazes of the left and right eyes were parallel. The near reflex states that under a constant light environment, simultaneous convergence and miosis, or divergence and mydriasis are induced when we shift our gaze from far to near or near to far, respectively. However, in our experiment, we occasionally found brief periods of small divergence and miosis emerging simultaneously even during 30 seconds of fully awake data recorded in the Control session. Thus, in our algorithm to detect Div-Mio, we first set the maximum velocity of divergence and that of miosis during these “brief periods” as thresholds (Div and Mio thresholds) for each participant. These two thresholds were applied to the Test session and detected Div-Mio as those exceeding these thresholds simultaneously. Note that this Div-Mio detection algorithm was not applied to the portions of the data where saccades (both normal and slow saccades) were generated because, in the previous study (Wakui & Hirata, 2011) that first reported Div-Mio, their participants with lowered alertness did not show Div-Mio during saccades. 
EECD
EECD is defined in the current study as eyelids closing duration lasting longer than the longest duration found during a fully awake state in each participant. PERCLOS is a widely used conventional OAI characterizing eyelid closure (Wierwille et al., 1994; Knipling, 1998). It is usually defined as the ratio of eyelid closure duration per minute, and it is judged as lowered alertness when the ratio exceeded 80% (Knipling, 1998). PERCLOS takes into account both the frequency and duration of eyelid closure. The former changes not only with alertness but also with mental factors such as tension and anxiety (Harrigan & O'Connell, 1996), as well as the content of the experimental task (Tanaka, 1999). On the other hand, Wilkinson et al. (2013) and Tanaka (1999) proposed another conventional algorithm that takes only the latter into account. They showed that the eyelid closure or blink duration elongated with decreased alertness. We employed their algorithm to detect EECD. First, we detected (Figure 2C①) durations of individual eyelid closure during the fully awake state in the Control session for each participant by using pupil data in which abnormal values were recorded while the eyes were closing. The maximum duration was used as the threshold (EECD threshold) to detect EECD in the data from the Test session (Figure 2C②). Note that EECD and other OAIs were not detected simultaneously because other OAIs were immeasurable when the eyelids were closed. 
Determination of the emergence order of OAIs
We evaluated each OAI's relative emergence timing in the data from Test session in each participant. We counted participants in which one OAI emerged before another, and also evaluated the possibility of the reverse order. In addition, one-tailed paired t-tests were used to evaluate whether the differences in the first emergence timings of pairs of OAIs were significant. When type I error was a concern, p values were corrected using the false discovery rate correction (Benjamini & Hochberg, 1995). 
Subjective reports
In VAS, subjective drowsiness was evaluated based on the distance from “Not drowsy at all” to the vertical line drawn on the 100 mm horizontal line by each participant. We defined a report of 0 to 33 mm as “awake” and a report of 34 to 100 mm as “drowsy.” 
We analyzed the data from participants who were awake and became drowsy in pre- and post-Test-session VAS tests, respectively. We also analyzed the data from those who were awake in both pre- and post-Test-session VAS tests because they may have decreased their alertness without perceiving it (Nishiyama et al., 2008; Nishiyama et al., 2010). By contrast, we did not analyze the data from those who were already drowsy in their pre-Test-session VAS test. Furthermore, participants who were drowsy in both pre- and post-Control-session VAS tests were eliminated from further analyses because we could not obtain data in the fully awake state of these participants. 
Evaluation of detection algorithms of OAIs
Below, we show examples of applying our algorithms for OAI detection, in which thresholds were set based on the Control session data in 30 seconds when participants were fully awake as estimated from answers to mental arithmetic and their pre- and post-Control session VAS tests, to the Test session data in the participants excluding those who reported drowsy in their pre-Test session VAS tests. 
Control session
Among the 14 participants, two reported being drowsy in the VAS tests conducted in both pre- and post-Control-session (Table 1). They were eliminated from further data analyses because their Control session data may not have reflected their fully awake state. For the remaining 12 participants, we determined proper thresholds from their Control session data and applied them to detect OAIs in their Test session data. The 30 seconds when these 12 participants were fully awake (i.e., answered the mental arithmetic correctly in the Control session) is listed in Table 2. All threshold values to detect OAIs are listed in Table 3 for each of these 12 participants. 
Table 1.
 
VAS on subjective drowsiness of all participants. Note: Before: Reports before sessions (the higher the number, the greater the drowsiness), After: Reports after sessions, Change: Changes before and after the sessions (positive changes indicate increased subjective drowsiness).
Table 1.
 
VAS on subjective drowsiness of all participants. Note: Before: Reports before sessions (the higher the number, the greater the drowsiness), After: Reports after sessions, Change: Changes before and after the sessions (positive changes indicate increased subjective drowsiness).
Table 2.
 
Time when the participants were fully awake (i.e., answered the mental arithmetic correctly) in the Control session.
Table 2.
 
Time when the participants were fully awake (i.e., answered the mental arithmetic correctly) in the Control session.
Table 3.
 
Thresholds to detect OAIs. Note: Angle threshold: Threshold to segregate slow saccade and slow eye movement from normal saccade (Figures 2B and 4N). Amplitude threshold: Threshold to segregate slow saccade and slow eye movement (Figures 2B and 4N). Interval threshold: Threshold to detect frequent saccade from normal saccade and slow saccade (Figure 2B). Div threshold: Threshold to detect Div parts of Div-Mio (Figure 4P). Mio threshold: Threshold to detect Mio parts of Div-Mio (Figure 4P). EECD threshold: Threshold to detect EECD (Figures 2C and 4R).
Table 3.
 
Thresholds to detect OAIs. Note: Angle threshold: Threshold to segregate slow saccade and slow eye movement from normal saccade (Figures 2B and 4N). Amplitude threshold: Threshold to segregate slow saccade and slow eye movement (Figures 2B and 4N). Interval threshold: Threshold to detect frequent saccade from normal saccade and slow saccade (Figure 2B). Div threshold: Threshold to detect Div parts of Div-Mio (Figure 4P). Mio threshold: Threshold to detect Mio parts of Div-Mio (Figure 4P). EECD threshold: Threshold to detect EECD (Figures 2C and 4R).
Table 4.
 
First emergence timings of the OAIs detected by our detection algorithm in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Table 4.
 
First emergence timings of the OAIs detected by our detection algorithm in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 3 shows horizontal eye position (A), vertical eye position (B), vergence angle, and pupil diameter (C) traces during an entire Control session in a representative participant (participant g). The dashed rectangle portions of a to f in A to C are the data for 30 seconds (“participant g” in Table 2) when he was fully awake (i.e., answered the mental arithmetic correctly in the Control session) and are enlarged in D to F. 
Figure 3.
 
Pupil/oculomotor behaviors during an entire Control session (participant g) and enlarged data for 30 seconds with correct answers to mental arithmetic. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position during the fully awake state (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position during the fully awake state (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter during the fully awake state (purple line = vergence angle; orange line = pupil diameter).
Figure 3.
 
Pupil/oculomotor behaviors during an entire Control session (participant g) and enlarged data for 30 seconds with correct answers to mental arithmetic. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position during the fully awake state (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position during the fully awake state (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter during the fully awake state (purple line = vergence angle; orange line = pupil diameter).
Test session
In Test sessions, nine of the 12 participants reported that they were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests (seven participants), or they were awake in both pre- and post-Test session VAS tests (two participants). We evaluated their Test session data to determine the emergence order of OAIs. The results of VAS tests for all participants are summarized in Table 1
Figure 4 shows horizontal eye position (A), vertical eye position (B), vergence angle, and pupil diameter (C) traces during an entire Test session in a representative participant (participant g). The dashed rectangle portions in A to C are enlarged in D to F, and those in D to F are further enlarged in G to L, O, and Q. 
Figure 4.
 
Pupil/oculomotor behaviors during an entire test session (participant g), enlarged data and OAIs, and evaluation of our detection algorithms for OAIs. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position and OAIs (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter), and OAIs. (G) Frequent saccade (gray line = right eye; dark red line = normal saccade; pink line= frequent saccade). (H) Slow saccade (gray line = right eye; yellow line = slow saccade). (I) Saccade detection by the conventional algorithm (Engbert et al., 2015a) (gray line = right eye; light purple line = saccade or misdetection). Note that the ordinate is not the actual value. (J) Eye movement detection by our modified algorithm (gray line = right eye; light purple line = eye movement). Note that the ordinate is not the actual value. (K) Segregating normal saccades, non-slow eye movement, and slow eye movements by the conventional algorithm (gray line = right eye; dark green line = non-slow eye movement; light blue line = slow eye movement). Note that the ordinate is not the actual value. (L) Segregating normal saccades, slow saccade, and slow eye movements by our modified algorithm (gray line = right eye; dark red line = normal saccade; yellow line = slow saccade; light blue line = slow eye movement). Note that the ordinate is not the actual value. (M) Conventional algorithm for slow eye movement detection (Shin et al., 2011) (dark red circle = normal saccade; dark green triangle = non-slow eye movement; light blue rhombus = slow eye movement). (N) Modified algorithm to segregate normal saccades, slow saccade, and slow eye movements (dark red circle = normal saccade; yellow rectangle = slow saccade; light blue rhombus = slow eye movement; black line = angle threshold; black dashed line = amplitude threshold). (O) Enlarged vergence angle and pupil diameter, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; filled green rectangle = Div-Mio). (P) Enlarged vergence angular velocity and pupil velocity, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; black line = Div and Mio thresholds; filled green rectangle = Div-Mio). (Q) Enlarged pupil diameter (orange line = pupil diameter; black line = eyelid closure). (R) Eyelid closure duration and EECD (navy blue line = eyelid closure duration; black line = EECD threshold). Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 4.
 
Pupil/oculomotor behaviors during an entire test session (participant g), enlarged data and OAIs, and evaluation of our detection algorithms for OAIs. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position and OAIs (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter), and OAIs. (G) Frequent saccade (gray line = right eye; dark red line = normal saccade; pink line= frequent saccade). (H) Slow saccade (gray line = right eye; yellow line = slow saccade). (I) Saccade detection by the conventional algorithm (Engbert et al., 2015a) (gray line = right eye; light purple line = saccade or misdetection). Note that the ordinate is not the actual value. (J) Eye movement detection by our modified algorithm (gray line = right eye; light purple line = eye movement). Note that the ordinate is not the actual value. (K) Segregating normal saccades, non-slow eye movement, and slow eye movements by the conventional algorithm (gray line = right eye; dark green line = non-slow eye movement; light blue line = slow eye movement). Note that the ordinate is not the actual value. (L) Segregating normal saccades, slow saccade, and slow eye movements by our modified algorithm (gray line = right eye; dark red line = normal saccade; yellow line = slow saccade; light blue line = slow eye movement). Note that the ordinate is not the actual value. (M) Conventional algorithm for slow eye movement detection (Shin et al., 2011) (dark red circle = normal saccade; dark green triangle = non-slow eye movement; light blue rhombus = slow eye movement). (N) Modified algorithm to segregate normal saccades, slow saccade, and slow eye movements (dark red circle = normal saccade; yellow rectangle = slow saccade; light blue rhombus = slow eye movement; black line = angle threshold; black dashed line = amplitude threshold). (O) Enlarged vergence angle and pupil diameter, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; filled green rectangle = Div-Mio). (P) Enlarged vergence angular velocity and pupil velocity, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; black line = Div and Mio thresholds; filled green rectangle = Div-Mio). (Q) Enlarged pupil diameter (orange line = pupil diameter; black line = eyelid closure). (R) Eyelid closure duration and EECD (navy blue line = eyelid closure duration; black line = EECD threshold). Note: Sacc = Saccade; Slow EM = Slow eye movement.
We applied our algorithm for detecting OAIs, in which thresholds were set based on the data when he was fully awake in the Control session (Figures 3D–F), to the data of the Test session. Below, we show the results of OAIs detection by our algorithms. 
Slow saccade, slow eye movement, and frequent saccade
Figure 4J shows exemplified results of slow saccade and slow eye movement detections together with detected normal saccades by our modified algorithms, whereas Figure 4I compares those by the conventional algorithm (Engbert et al., 2015a). Notably, our modified algorithms shown in J did not misdetect those noise components that were misdetected by the conventional algorithm shown in I. In all 12 participants, our modified algorithm reduced misdetections by 96.6% compared to the conventional algorithm. In addition, the conventional algorithm only detected individual saccades (I), whereas our modified algorithm was able to detect one continuum eye movement formed by connecting a series of multiple eye movements (“One continuum eye movement” in J). Such an eye movement is often an anomalously slow eye movement (slow saccade or slow eye movement), and as expected it was categorized as slow eye movement as shown in the next paragraph. 
Figures 4L and 4N illustrate exemplified results of slow saccades and slow eye movement detection by our modified algorithm while Figures 4K and 4M compare slow eye movement detection by the conventional algorithm (Shin et al., 2011). The conventional algorithm shown in K and M was not intended to detect slow saccade. On the other hand, our modified algorithm shown in L and N was able to detect slow saccade, as well as slow eye movement. Figure 4G shows exemplified successive saccades detected as frequent saccades whose ISIs are shorter than 0.15 second. 
Div-Mio
Figure 4O illustrates exemplified results of Div-Mio detection by our algorithm (negative vergence angle and pupil diameter represent divergence and miosis, respectively). Figure 4P illustrates the first derivatives of changes in vergence angle and pupil diameter. There is no conventional algorithm to detect Div-Mio. Our algorithm shown in O and P was able to detect Div-Mio (filled green rectangle portion). 
EECD
Figures 4Q and 4R show exemplified results of EECD detection by our algorithm. The black portions of the pupil diameter data in Q, automatically detected by our algorithm (Figure 2C), are eyelid closure periods in which the pupil was not visible (occluded by the eyelid) to accurately record its diameter. We measured the duration of each black portion to obtain eyelid closure duration data. Our algorithm shown in R was able to detect EECD. 
Results
We evaluate the emergence order of OAIs. Table 4 illustrates the timing of the first emergence of each OAI of individual participants during Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests (A), and the two participants who were awake in both pre- and post-Test-session VAS tests (B). A dash indicates that each OAI was not detected. Frequent saccade emerged in all nine participants. Slow saccade and slow eye movement emerged in eight and seven participants, respectively, whereas Div-Mio and EECD emerged in five participants. Figure 5 illustrates the emergence timing of each OAI (except for EECD) shown in Table 4, sorted in the emergence order. Of the eight participants who presented both frequent saccade and slow saccade, frequent saccade emerged earlier than slow saccade in six participants whereas they emerged in the opposite order in the remaining two. After frequent saccade or slow saccade, slow eye movement emerged in seven participants, while Div-Mio emerged in five. Notably, slow eye movement and Div-Mio emerged always after the emergence of frequent saccade or slow saccade. Of the five participants who presented both Div-Mio and slow eye movement, Div-Mio emerged earlier than slow eye movement in four participants. Collectively, the emergence order of these OAIs is typically as frequent saccade → slow saccade → Div-Mio → slow eye movement, where in some participants (two out of eight) frequent saccade and slow saccade are reversed and in only one out of five participants who presented both Div-Mio and slow eye movement, their emergence order is reversed. Figure 6 shows a raster plot indicating all emergence timings and black circles indicating the first emergence timing (note that the OAIs that emerged within 10 seconds from the start were ignored. See Method section) of each OAI in a representative participant (participant l) in whom OAIs emerged in the typical order mentioned above (frequent saccade → slow saccade → Div-Mio → slow eye movement). Interestingly, EECD emerged sporadically in the typical emergence order, but it always emerged later than frequent saccade (Table 4). In summary, OAIs typically emerged in the order frequent saccade → slow saccade → Div-Mio → slow eye movement, whereas EECD emerged at inconsistent timing. 
Figure 5.
 
First emergence timings of OAIs (other than EECD) sorted in the emergence order in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 5.
 
First emergence timings of OAIs (other than EECD) sorted in the emergence order in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 6.
 
Raster plots of the emergence timings of each OAI detected by our detection algorithm in the Test session (participant l). Frequent saccade (pink), slow saccade (yellow), Div-Mio (green), slow eye movement (light blue), and EECD (navy blue) in order from the top (typical emergence order). Black circles indicate the first emergence of each OAI. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 6.
 
Raster plots of the emergence timings of each OAI detected by our detection algorithm in the Test session (participant l). Frequent saccade (pink), slow saccade (yellow), Div-Mio (green), slow eye movement (light blue), and EECD (navy blue) in order from the top (typical emergence order). Black circles indicate the first emergence of each OAI. Note: Sacc = Saccade; Slow EM = Slow eye movement.
We quantify the relative emergence timings of OAIs. The difference between the first emergence timings of frequent saccades and slow saccades was 40.28 seconds on average (maximum 187.0 seconds; minimum −15.33 seconds) (t(7) = 1.63, p > 0.05, Cohen's d = 0.92). The difference between the first emergence timings of slow saccades and Div-Mio was 70.54 seconds on average (maximum 131.6 seconds; minimum 25.96 seconds) (t(4) = 3.86, false discovery rate-corrected p < 0.05, Cohen's d = 0.66). The difference between the first emergence timings of Div-Mio and slow eye movements was 55.33 seconds on average (maximum 105.5 seconds; minimum −71.04 seconds) (t(4) = 1.72, false discovery rate-corrected p > 0.05, Cohen's d = 1.63). Among the OAIs, only the EECD, emerged at completely different timings for each participant. However, the first emergence timings of EECD were later than that of frequent saccades (the interval was 95.25 seconds on average, maximum 150.3 seconds; minimum 35.73 seconds) (t(4) = 4.07, false discovery rate-corrected p < 0.05, Cohen's d = 3.19). In summary, there were 40.28- to 95.25-second differences in the emergence timings of these OAIs. 
Discussion
The eyes have been regarded as the window to the mind in many cultures, and various features of the eye, eyelid, and pupil movements have been referred to as objective alertness indices (Henn et al., 1984; Knipling, 1998; Schleicher et al., 2008; Nishiyama et al., 2008; Wakui & Hirata, 2011; Wakui & Hirata, 2013). Nonetheless, each study evaluated only one or a few of them simultaneously, leaving their emergence order in decreasing alertness unidentified. Currently, we evaluated those that commonly emerge in a natural stationary state and identified their emergence order. Below we discuss the identified emergence order of OAIs in conjunction with possible neural mechanisms underlying each OAI. 
Neural mechanisms underlying OAIs
Most of the OAIs (frequent saccade, slow saccade, slow eye movement, and Div parts of Div-Mio) evaluated in the current study can be considered as features related to saccades and fixation between saccades, because other types of eye movements (see Method section) were not generated during the experiment in which no external sensory (visual, vestibular, auditory) stimulations were given. EECD and Mio parts of Div-Mio are features extracted, respectively, from the eyelids and the pupil movements, which are not directly related to saccades and fixation. Below we discuss possible neural mechanisms underlying these OAIs based on known neuroanatomy and physiology to evaluate the validity of the current results (emergence order of OAIs) from the neural mechanism and, conversely, to consider the neural mechanism that is yet unknown from the current results. 
Slow saccade and slow eye movement
The generation of saccades during fixation begins in the superior colliculus, which receives signals from many cortical and subcortical regions and sends outputs to the premotor areas involved in the eye and head movement control (Sparks, 2002). One of the premotor neuron populations receiving the superior colliculus (especially rostral superior colliculus) output is the omni-pause neurons in the raphe nucleus (Büttner-Ennever, Horn, A. K., Henn, V., & Cohen, 1999). During fixation, omni-pause neurons discharge at a relatively constant firing frequency, but stop firing just (approximately 0.02 second) before and during saccades in any direction. These neurons inhibit excitatory burst neurons in the paramedian pontine reticular formation (Scudder, Kaneko, & Fuchs, 2002). As the result, excitatory burst neurons do not generate spikes during fixation and show high-frequency burst firing activities just (approximately 0.01 second) before and during saccades (Keller, 1974). The excitatory burst neurons make excitatory monosynaptic connections with the ipsilateral abducens motoneurons that innervate the ipsilateral lateral rectus muscle (agonist). Therefore disinhibition from omni-pause neurons of excitatory burst neurons causes a rapid ipsilateral eye position change as a saccade. Another kind of input to abducens motoneurons is a tonic signal from the oculomotor neural integrator which conducts temporal integration of excitatory burst neuron activities to convert their eye velocity information into eye position (Moschovakis, 1997). These two major inputs to abducens motoneurons make their firing activities burst-tonic during a saccade and the following fixation. 
Henn et al. (1984) recorded extracellular activities of single omni-pause neuron, excitatory burst neuron (medium lead excitatory burst neuron), and abducens motoneuron in drowsy monkeys. Their monkeys exhibited slow saccade in their light sleep condition followed by slow eye movement in the later period. The order coincides with the current result in human participants. Just before this period in which the animal exhibited slow saccades, a representative omni-pause neuron decreased its tonic firing rate and paused slowly, while a representative excitatory burst neuron fired at a lower frequency with a longer duration than it did for normal rapid saccades. As the result, the burst phasic activities of abducens motoneurons which are mainly caused by the activities of excitatory burst neurons decreased in frequency and increased in duration, corresponding with the increased duration of slow saccades around the onset of drowsiness. As for the tonic component of abducens motoneurons representing the contribution from oculomotor neural integrator, Henn et al. (1984) reported that during light sleep the amount of their tonic firing rate for a given eye position decreased and became more variable. These results suggest that changes in omni-pause neuron activities in the process of increasing drowsiness cause excitatory burst neurons to fire at a lower frequency with a longer duration for a given saccade, resulting in a slower saccade (i.e., slow saccade). When excitatory burst neuron activities become no longer burst, the temporal integration of these signals by oculomotor neural integrator would fluctuate slowly and so would the tonic component of abducens motoneurons, resulting in a slow drift of gaze as slow eye movement. In human functional magnetic resonance imaging studies (Beliveau, Svarer, Frokjaer, Knudsen, Greve, & Fisher, 2015; Beliveau, Ganz, Feng, Ozenne, Højgaard, Fisher, Svarer, Greve, & Knudsen, 2017), the raphe nucleus where omni-pause neurons locate has been demonstrated to present highly correlated activities with wake-sleep cycles (Portas, Bjorvatn, & Ursin, 2000). 
Div-Mio
In normal awake humans, when shifting gaze from far to near, not only accommodation but also convergence and miosis emerge. The co-occurrence of these three behaviors is called the near reflex. Therefore, as an OAI, we focused on Div-Mio, which was defined as the co-occurrence of miosis and divergence, violating the rule of the near reflex. Neurons presenting sensitivities to vergence angle or velocity were found in the supraoculomotor area located dorsal and lateral to the oculomotor nucleus (Mays, 1984; Mays, Porter, Gamlin, & Tello, 1986; Judge & Cumming, 1986). A recent study found neurons showing burst activities during disjunctive saccades (namely rapid saccadic eye movements resulting in convergence or divergence) in the central mesencephalic reticular formation lateral to the oculomotor nucleus (Quinet, Schultz, May, & Gamlin, 2020) where premotor lens accommodation neurons are located (May, Billig, Gamlin, & Quinet, 2019). The central mesencephalic reticular formation neurons have been demonstrated to supply input to medial-rectus motor neurons in the oculomotor nucleus as well as to the preganglionic Edinger-Westphal nucleus (May, Warren, Bohlen, Barnerssoi, & Horn, 2016; May et al., 2019; Bohlen, Warren, & May, 2017), which contains motoneurons controlling miosis and accommodation. Taken together, these midbrain areas are suggested to be responsible for the violation of the near reflex manifested as Div-Mio, although the direct causality is yet to be examined. 
Another possible neural mechanism generating Div-Mio is unbalanced outputs from the oculomotor neural integrator producing eye position signal for each of the eyes (Okamura, Baker, & Hirata, 2011). Wakui & Hirata (2011) thought that when the alertness decreased, it became difficult to hold the binocular gaze at one point (maybe because of unbalanced output from the oculomotor neural integrator), and the gaze of each eye was directed toward the anatomical resting eye position (the anatomical resting position is more outward than the front in many cases) (Kawamura, 1986), resulting in divergence. 
Frequent saccade
Frequent saccades observed in the current study are small in amplitude (56.4% of them were less than 1°) and emerged involuntarily, suggesting that most of them were equivalent to microsaccades. Many studies demonstrated that microsaccades are tightly related to covert spatial visual attention (Hafed & Clark, 2002). In addition, neurons in various visual areas including superior colliculus, lateral geniculate nucleus, visual cortical areas such as V1, V2, V4, and the inferior temporal cortex present attentional modulations which are influenced by microsaccade generation timings (Leopold & Logothetis, 1998; Kagan, Gur, & Snodderly, 2008; Krauzlis, Lovejoy, & Zénon, 2013; Lowet, Gomes, Srinivasan, Zhou, Schafer, & Desimone, 2018), although a recent study demonstrated superior collicular neurons were not causally modulated by microsaccade generation (Yu, Herman, Katz, & Krauzlis, 2022). Schleicher et al. (2008) thought that frequent saccades were generated as a reflection of the decreasing interest in the environment because of a decrease in alertness. Microsaccades are considered to share mostly the same underlying neural mechanism as larger saccades that were summarized above. 
EECD
When the eyelids are consciously opened, the fast-twitch muscles of the levator palpebrae superioris muscle contract, stimulating the mechanoreceptors in Muller's muscle. Then, the slow-twitch muscles of the levator palpebrae superioris muscle to which this stimulation is transmitted contract, and eyelid opening is maintained without constant awareness (Ban, Matsuo, Osada, Ban, & Yuzuriha, 2010). Furthermore, the stimulation of the Muller's muscle by widening the eyes may activate the locus coeruleus (Aston-Jones & Cohen, 2005; Carter et al., 2010), which is closely related to awake, via the mesencephalic trigeminal nucleus (Matsuo, Ban, Hama, & Yuzuriha, 2015). The locus coeruleus and the mesencephalic trigeminal nucleus are not involved in OAIs other than EECD, except that the locus coeruleus is known to be associated with Mio parts of Div-Mio. 
Emergence order of OAIs
In the current study, we evaluated the following eye, eyelid, and pupil movements: saccades, slow-phase eye movements, vergence, pupil diameter, and blinks. We defined the OAIs as those that never emerge while the participants were fully awake. We found that the OAIs emerged typically in the following order: frequent saccade → slow saccade → Div-Mio → slow eye movement, although the emergence order of frequent saccade and slow saccade, or Div-Mio and slow eye movement, could in rare cases be reversed. This result suggests that the OAIs we currently defined can be reliable objective measures to monitor progressive drowsiness. Notably, EECD, which is basically equivalent to PERCLOS and has been extensively used to detect human drowsiness (Wilkinson et al., 2013; Tanaka, 1999), emerged at inconsistent timing among the OAIs we evaluated. These results provide a basis for the oculo-pupillometry enabling to monitor progressively decreasing alertness. This oculo-pupillometry may also evaluate a potential decrease in alertness, as OAIs emerged in an order similar to the typical order even in participants who were awake in both pre- and post-Test session VAS test (“awake → awake” in Table 4 and Figure 5). 
The current study quantified relative alertness as a difference in the emergence timings of OAIs. If the difference in the emergence timings of OAIs was slight (e.g., only several seconds difference), it would be difficult to monitor the process of decreasing alertness by detecting these OAIs. However, we found that the difference in the emergence timings of OAIs was as large as 40.28 to 95.25 seconds, indicating that OAIs are indices of a wide range of alertness. Therefore it is suggested that our proposed oculo-pupillometry can monitor the process of decreasing alertness by detecting OAIs. 
Although it was unknown which emerges first between increased frequency and decreased velocity of saccade with decreased alertness, frequent saccade typically emerged earlier than slow saccade in the current experiment. Frequent saccade is likely to reflect the decreasing interest in the environment (as mentioned in “Frequent Saccade” in the Neural Mechanisms Underlying OAIs section) caused by, or as a precursor to, decreased alertness. The current result, in which 10 out of 14 participants had a minimum ISI that was shorter than the normal minimum ISI of 0.15 second even in their fully awake state (see Method section), suggests that frequent saccade reflects the decreasing interest in the environment, possibly caused by the dark room and not necessarily associated with decreased alertness. However, frequent saccade emerging in environments that are not entirely dark, such as while driving, may often be a reflection or a precursor of decreased alertness. Note that pupil/oculomotor behaviors may also differ between dark and light conditions. In the current study, we performed experiments in the dark to eliminate eye movements caused by visual stimuli and evaluate OAI caused purely by decreased alertness. When alertness is evaluated by detecting OAIs (including OAIs other than frequent saccade) in light conditions in future studies, it is desirable to evaluate the likelihood of the emergence of OAIs for each environment such as visual stimuli while referring to the emergence order of OAIs found in the current study. 
As mentioned in “Slow Saccade and Slow Eye Movement” in the Neural Mechanisms Underlying OAIs section, it is already known that slow saccade emerges earlier than slow eye movement during the activity of the raphe nucleus decreases in parallel with drowsiness. On the other hand, partly because the neural mechanism underlying Div-Mio is not yet clarified (as mentioned in “Div-Mio” in the Neural Mechanisms Underlying OAIs section), the emergence order of Div-Mio and other OAIs has been unknown. We currently showed that the emergence order of Div-Mio and slow eye movement, which is likely produced by malfunctioning oculomotor neural integrator (as discussed in “Slow Saccade and Slow Eye Movement” in the Neural Mechanisms Underlying OAIs section), were infrequently reversed from the typical order. Although the link between the oculomotor neural integrator and the miosis that has been associated with the activity of locus coeruleus is unknown, the current results raise the possibility that the oculomotor neural integrator is involved in the emergence of Div-Mio. However, because Div-Mio typically emerged earlier than slow eye movement in the current experiment, likely, neural mechanisms and brain areas other than oculomotor neural integrator, such as the midbrain areas described in “Div-Mio” in the Neural Mechanisms Underlying OAIs section, are also involved in the emergence of Div-Mio. The absence of a strong connection between the neural mechanisms of EECD and other OAIs, as mentioned in “EECD” in the Neural Mechanisms Underlying OAIs section, supports the current result that EECD emerged at inconsistent timing among the OAIs. 
Conclusions
We evaluated the emergence order of the OAIs that emerges sequentially while human participants are decreasing their alertness. Frequent saccades emerged as alertness began to decrease or as a precursor to decreased alertness. Then, slow saccades emerged, and further decreased alertness made it difficult for participants to keep their gaze, resulting in Div-Mio and slow eye movements. Although it should be noted that the emergence order of frequent saccade and slow saccade, or Div-Mio and slow eye movement, could in rare cases be reversed, the current results suggest that these OAIs can be a reliable objective measure to monitor human drowsiness. Furthermore, EECD could emerge any time after frequent saccade. These results provide a basis for the oculo-pupillometry enabling to monitor progressively decreasing alertness. 
Acknowledgments
Supported by JSPS KAKENHI 20H04286, 19K06756, 18KK0286JST, and CREST Grant Number JPMJCR22P5. 
Commercial relationships: none. 
Corresponding author: Yutaka Hirata. 
Address: Department of Robotic Science and Technology, Chubu University Graduate School of Engineering, 1200 Matsumoto-cho, Kasugai-shi, Aichi 487-8501, Japan. 
References
Alnæs, D., Sneve, M. H., Espeseth, T., Endestad, T., van de Pavert, S. H., & Laeng, B. (2014). Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. Journal of Vision, 14(4), 1, https://doi.org/10.1167/14.4.1. [PubMed]
Aston-Jones, G., & Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403–450, https://doi.org/10.1146/annurev.neuro.28.061604.135709. [PubMed]
Bahill, A. T., Clark, M. R., & Stark, L. (1975a). Dynamic overshoot in saccadic eye movements is caused by neurological control signed reversals. Experimental Neurology, 48(1), 107–122, https://doi.org/10.1016/0014-4886(75)90226-5. [PubMed]
Bahill, A. T., Clark, M. R., & Stark, L. (1975b). The main sequence, a tool for studying human eye movements. Mathematical Biosciences, 24, 191–204, https://doi.org/10.1016/0025-5564(75)90075-9.
Ban, R., Matsuo, K., Osada, Y., Ban, M., & Yuzuriha, S. (2010). Reflexive contraction of the levator palpebrae superioris muscle to involuntarily sustain the effective eyelid retraction through the transverse trigeminal proprioceptive nerve on the proximal Mueller's muscle: verification with evoked electromyography. Journal of Plastic, Reconstructive & Aesthetic Surgery, 63(1), 59–64, https://doi.org/10.1016/j.bjps.2008.07.021.
Beliveau, V., Ganz, M., Feng, L., Ozenne, B., Højgaard, L., Fisher, P. M., &hellip; Knudsen, G. M. (2017). A high-resolution in vivo atlas of the human brain's serotonin system. The Journal of Neuroscience, 37(1), 120–128, https://doi.org/10.1523/JNEUROSCI.2830-16.2016.
Beliveau, V., Svarer, C., Frokjaer, V. G., Knudsen, G. M., Greve, D. N., & Fisher, P. M. (2015). Functional connectivity of the dorsal and median raphe nuclei at rest. NeuroImage, 116, 187–195, https://doi.org/10.1016/j.neuroimage.2015.04.065. [PubMed]
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300, http://www.jstor.org/stable/2346101.
Bohlen, M. O., Warren, S., & May, P. J. (2017). A central mesencephalic reticular formation projection to medial rectus motoneurons supplying singly and multiply innervated extraocular muscle fibers. The Journal of Comparative Neurology, 525(8), 2000–2018, https://doi.org/10.1002/cne.24187. [PubMed]
Büttner-Ennever, J. A., Horn, A. K., Henn, V., & Cohen, B. (1999). Projections from the superior colliculus motor map to omnipause neurons in monkey. The Journal of Comparative Neurology, 413(1), 55–67, https://doi.org/10.1002/(sici)1096-9861(19991011)413:1≤55::aid-cne3≥3.0.co;2-k. [PubMed]
Caffier, P. P., Erdmann, U., & Ullsperger, P. (2003). Experimental evaluation of eye-blink parameters as a drowsiness measure. European Journal of Applied Physiology, 89(3-4), 319–325, https://doi.org/10.1007/s00421-003-0807-5. [PubMed]
Cajochen, C., Zeitzer, J. M., Czeisler, C. A., & Dijk, D. J. (2000). Dose-response relationship for light intensity and ocular and electroencephalographic correlates of human alertness. Behavioural Brain Research, 115(1), 75–83, https://doi.org/10.1016/s0166-4328(00)00236-9. [PubMed]
Carpenter, R. H. S. (1988). Movements of the Eyes (2nd ed.). London: Pion Ltd.
Carter, M. E., Yizhar, O., Chikahisa, S., Nguyen, H., Adamantidis, A., Nishino, S., &hellip; de Lecea, L. (2010). Tuning arousal with optogenetic modulation of locus coeruleus neurons. Nature Neuroscience, 13(12), 1526–1533, https://doi.org/10.1038/nn.2682. [PubMed]
De Gennaro, L., Ferrara, M., Urbani, L., & Bertini, M. (2000). Oculomotor impairment after 1 night of total sleep deprivation: a dissociation between measures of speed and accuracy. Clinical Neurophysiology, 111(10), 1771–1778, https://doi.org/10.1016/s1388-2457(00)00393-x.
Engbert, R., Sinn, P., Mergenthaler, K., & Trukenbrod, H. (2015a). Microsaccade toolbox. Retrieved March 3, 2021 from http://read.psych.uni-potsdam.de/index.php?option=com_content&view=article&id=140:engbert-et-al-2015-microsaccade-toolbox-for-r&catid=26:publications&Itemid=34.
Engbert, R., Trukenbrod, H. A., Barthelmé, S., & Wichmann, F. A. (2015b). Spatial statistics and attentional dynamics in scene viewing. Journal of Vision, 15(1), 15.1.14, https://doi.org/10.1167/15.1.14. [PubMed]
Fransson, P. A., Patel, M., Magnusson, M., Berg, S., Almbladh, P., & Gomez, S. (2008). Effects of 24-hour and 36-hour sleep deprivation on smooth pursuit and saccadic eye movements. Journal of Vestibular Research: Equilibrium & Orientation, 18(4), 209–222. [PubMed]
Hafed, Z. M., & Clark, J. J. (2002). Microsaccades as an overt measure of covert attention shifts. Vision Research, 42(22), 2533–2545, https://doi.org/10.1016/s0042-6989(02)00263-8. [PubMed]
Harrigan, J. N., & O'Connell, D. M. (1996). How do you look when feeling anxious? Facial display of anxiety. Personality and Individual Differences, 21(2), 205–212, https://doi.org/10.1016/0191-8869(96)00050-5.
Heller, G. Z., Manuguerra, M., & Chow, R. (2016). How to analyze the visual analogue scale: myths, truths and clinical relevance. Scandinavian Journal of Pain, 13, 67–75, https://doi.org/10.1016/j.sjpain.2016.06.012. [PubMed]
Henn, V., Baloh, R. W., & Hepp, K. (1984). The sleep-wake transition in the oculomoter system. Experimental Brain Research, 54(1), 166–176, https://doi.org/10.1007/BF00235828. [PubMed]
Judge, S. J., & Cumming, B. G. (1986). Neurons in the monkey midbrain with activity related to vergence eye movement and accommodation. Journal of Neurophysiology, 55(5), 915–930, https://doi.org/10.1152/jn.1986.55.5.915. [PubMed]
Kagan, I., Gur, M., & Snodderly, D. M. (2008). Saccades and drifts differentially modulate neuronal activity in V1: effects of retinal image motion, position, and extraretinal influences. Journal of Vision, 8(14), 1–25, https://doi.org/10.1167/8.14.19. [PubMed]
Kapoula, Z. A., Robinson, D. A., & Hain, T. C. (1986). Motion of the eye immediately after a saccade. Experimental Brain Research, 61(2), 386–394, https://doi.org/10.1007/BF00239527. [PubMed]
Kawamura, M. (1986). The examination for ocular position and binocular vision. Japanese Orthoptic Journal, 14, 1–11, https://doi.org/10.4263/jorthoptic.14.1.
Keller, E. L. (1974). Participation of medial pontine reticular formation in eye movement generation in monkey. Journal of Neurophysiology, 37(2), 316–332, https://doi.org/10.1152/jn.1974.37.2.316. [PubMed]
Knipling, R. (1998). PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance. Tech Brief FHWA-MCRT-98-006. U.S. Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006.
Krauzlis, R. J., Lovejoy, L. P., & Zénon, A. (2013). Superior colliculus and visual spatial attention. Annual Review of Neuroscience, 36, 165–182, https://doi.org/10.1146/annurev-neuro-062012-170249. [PubMed]
Leopold, D. A., & Logothetis, N. K. (1998). Microsaccades differentially modulate neural activity in the striate and extrastriate visual cortex. Experimental Brain Research, 123(3), 341–345, https://doi.org/10.1007/s002210050577. [PubMed]
Lovato, N., & Lack, L. (2010). The effects of napping on cognitive functioning. Progress in Brain Research, 185, 155–166, https://doi.org/10.1016/B978-0-444-53702-7.00009-9. [PubMed]
Lowet, E., Gomes, B., Srinivasan, K., Zhou, H., Schafer, R. J., & Desimone, R. (2018). Enhanced neural processing by covert attention only during microsaccades directed toward the attended stimulus. Neuron, 99(1), 207–214.e3, https://doi.org/10.1016/j.neuron.2018.05.041. [PubMed]
Matsuo, K., Ban, R., Hama, Y., & Yuzuriha, S. (2015). Eyelid opening with trigeminal proprioceptive activation regulates a brainstem arousal mechanism. PLoS ONE, 10(8), 1–19, https://doi.org/10.1371/journal.pone.0134659.
May, P. J., Billig, I., Gamlin, P. D., & Quinet, J. (2019). Central mesencephalic reticular formation control of the near response: lens accommodation circuits. Journal of Neurophysiology, 121(5), 1692–1703, https://doi.org/10.1152/jn.00846.2018. [PubMed]
May, P. J., Warren, S., Bohlen, M. O., Barnerssoi, M., & Horn, A. K. (2016). A central mesencephalic reticular formation projection to the Edinger-Westphal nuclei. Brain Structure & Function, 221(8), 4073–4089, https://doi.org/10.1007/s00429-015-1147-z. [PubMed]
Mays, L. E. (1984). Neural control of vergence eye movements: convergence and divergence neurons in midbrain. Journal of Neurophysiology, 51(5), 1091–1108, https://doi.org/10.1152/jn.1984.51.5.1091. [PubMed]
Mays, L. E., Porter, J. D., Gamlin, P. D., & Tello, C. A. (1986). Neural control of vergence eye movements: neurons encoding vergence velocity. Journal of Neurophysiology, 56(4), 1007–1021, https://doi.org/10.1152/jn.1986.56.4.1007. [PubMed]
McClelland, L. E., Pilcher, J. J., & Moore, D. D. (2010). Oculomotor measures as predictors of performance during sleep deprivation. Aviation, Space, and Environmental Medicine, 81(9), 833–842, https://doi.org/10.3357/asem.2653.2010. [PubMed]
McGinley, M. J., Vinck, M., Reimer, J., Batista-Brito, R., Zagha, E., Cadwell, C. R., & McCormick, D. A. (2015). Waking state: rapid variations modulate neural and behavioral responses. Neuron, 87(6), 1143–1161, https://doi.org/10.1016/j.neuron.2015.09.012. [PubMed]
Miall, R. C., Imamizu, H., & Miyauchi, S. (2000). Activation of the cerebellum in co-ordinated eye and hand tracking movements: an fMRI study. Experimental Brain Research, 135(1), 22–33, https://doi.org/10.1007/s002210000491. [PubMed]
Moschovakis, A. K. (1997). The neural integrators of the mammalian saccadic system. Frontiers in Bioscience, 2, d552–d577, https://doi.org/10.2741/a212.
Muzet, A. (2007). Environmental noise, sleep and health. Sleep Medicine Reviews, 11(2), 135–142, https://doi.org/10.1016/j.smrv.2006.09.001. [PubMed]
Nishiyama, J., Kinoshita, S., & Hirata, Y. (2010). Prediction of drowsiness by the vestibulo-ocular reflex. Transactions of the Japanese Society for Medical and Biological Engineering, 48(1), 1–10, https://doi.org/10.11239/jsmbe.48.1.
Nishiyama, J., Tanida, K., Kusumi, M., & Hirata, Y. (2008). Evaluation of the drowsiness by pupil fluctuation. Transactions of the Japanese Society for Medical and Biological Engineering, 46(2), 212–217, https://doi.org/10.11239/jsmbe.46.212.
Okamura, N., Baker, R., & Hirata, Y. (2011). Monocular eye position specificity in the oculomotor neural integrator. Twentieth Annual Computational Neuroscience Meeting, Stockholm, Sweden, July, 2011, https://doi.org/10.1186/1471-2202-12-S1-P151.
Porcu, S., Ferrara, M., Urbani, L., Bellatreccia, A., & Casagrande, M. (1998). Smooth pursuit and saccadic eye movement as possible indicators of nighttime sleepiness. Physiology & Behavior, 65(3), 437–443, https://doi.org/10.1016/s0031-9384(98)00181-4. [PubMed]
Portas, C. M., Bjorvatn, B., & Ursin, R. (2000). Serotonin and the sleep/wake cycle: special emphasis on microdialysis studies. Progress in Neurobiology, 60(1), 13–35, https://doi.org/10.1016/s0301-0082(98)00097-5. [PubMed]
Porte, H. S. (2004). Slow horizontal eye movement at human sleep onset. Journal of Sleep Research, 13(3), 239–249, https://doi.org/10.1111/j.1365-2869.2004.00413.x. [PubMed]
Quinet, J., Schultz, K., May, P. J., & Gamlin, P. D. (2020). Neural control of rapid binocular eye movements: saccade-vergence burst neurons. Proceedings of the National Academy of Sciences, 117(46), 29123–29132, https://doi.org/10.1073/pnas.2015318117.
Reisine, H., & Raphan, T. (1992). Neural basis for eye velocity generation in the vestibular nuclei of alert monkeys during off-vertical axis rotation. Experimental Brain Research, 92(2), 209–226, https://doi.org/10.1007/BF00227966. [PubMed]
Schleicher, R., Galley, N., Briest, S., & Galley, L. (2008). Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired?. Ergonomics, 51(7), 982–1010, https://doi.org/10.1080/00140130701817062. [PubMed]
Schneider, E., Villgrattner, T., Vockeroth, J., Bartl, K., Kohlbecher, S., Bardins, S., & Brandt, T. (2009). EyeSeeCam: an eye movement-driven head camera for the examination of natural visual exploration. Annals of the New York Academy of Sciences, 1164, 461–467, https://doi.org/10.1111/j.1749-6632.2009.03858.x. [PubMed]
Scudder, C. A., Kaneko, C. S., & Fuchs, A. F. (2002). The brainstem burst generator for saccadic eye movements: a modern synthesis. Experimental Brain Research, 142(4), 439–462, https://doi.org/10.1007/s00221-001-0912-9. [PubMed]
Shin, D., Sakai, H., & Uchiyama, Y. (2011). Slow eye movement detection can prevent sleep-related accidents effectively in a simulated driving task. Journal of Sleep Research, 20(3), 416–424, https://doi.org/10.1111/j.1365-2869.2010.00891.x. [PubMed]
Sparks, D. L. (2002). The brainstem control of saccadic eye movements. Nature Reviews. Neuroscience, 3(12), 952–964, https://doi.org/10.1038/nrn986. [PubMed]
Sweeney, J. A., Luna, B., Keedy, S. K., McDowell, J. E., & Clementz, B. A. (2007). fMRI studies of eye movement control: investigating the interaction of cognitive and sensorimotor brain systems. NeuroImage, 36(Suppl 2), T54–T60, https://doi.org/10.1016/j.neuroimage.2007.03.018. [PubMed]
Tanaka, H. (1999). Arousal level and blink activity. The Japanese Journal of Psychology, 70(1), 1–8, https://doi.org/10.4992/jjpsy.70.1.
Tse, P. U., Baumgartner, F. J., & Greenlee, M. W. (2010). Event-related functional MRI of cortical activity evoked by microsaccades, small visually-guided saccades, and eyeblinks in human visual cortex. NeuroImage, 49(1), 805–816, https://doi.org/10.1016/j.neuroimage.2009.07.052. [PubMed]
Wakui, H., & Hirata, Y. (2011). Detection of aimless stage by the vergence angle. Transactions of the Japanese Society for Medical and Biological Engineering, 49(5), 693–702, https://doi.org/10.11239/jsmbe.49.693.
Wakui, H., & Hirata, Y. (2013). Detection of reduced arousal by saccadic eye movements. Transactions of the Japanese Society for Medical and Biological Engineering, 51(6), 328–341, https://doi.org/10.11239/jsmbe.51.328.
Wakui, H., & Hirata, Y. (2014). Eye movements and pupil fluctuation reflecting alertness and their neuronal mechanisms. The Brain & Neural Networks, 21(1), 20–31, https://doi.org/10.3902/jnns.21.20.
Wierwille, W. W., Wreggit, S. S., Kirn, C. L., Ellsworth L. A., & Fairbanks, R. J. (1994). Research on vehicle-based driver status/performance monitoring: development, validation, and refinement of algorithms for detection of driver drowsiness. No. HS-808 247.
Wilkinson, V. E., Jackson, M. L., Westlake, J., Stevens, B., Barnes, M., Swann, P., &hellip; Howard, M. E. (2013). The accuracy of eyelid movement parameters for drowsiness detection. Journal of Clinical Sleep Medicine, 9(12), 1315–1324, https://doi.org/10.5664/jcsm.3278.
Yakushin, S. B., Raphan, T., & Cohen, B. (2017). Coding of velocity storage in the vestibular nuclei. Frontiers in Neurology, 8, 1–19, https://doi.org/10.3389/fneur.2017.00386. [PubMed]
Yu, G., Herman, J. P., Katz, L. N., & Krauzlis, R. J. (2022). Microsaccades as a marker not a cause for attention-related modulation. eLife, 11, e74168, https://doi.org/10.7554/eLife.74168. [PubMed]
Figure 1.
 
Schematic diagram of the experimental environment.
Figure 1.
 
Schematic diagram of the experimental environment.
Figure 2.
 
Flowcharts of our detection algorithms. (A) Our algorithm to detect eye movements. (B) Our algorithm to segregate normal saccade, slow saccade, and slow eye movement, and detect frequent saccade. Note: Sacc = Saccade; Slow EM = Slow eye movement. (C) Our algorithm to detect EECD.
Figure 2.
 
Flowcharts of our detection algorithms. (A) Our algorithm to detect eye movements. (B) Our algorithm to segregate normal saccade, slow saccade, and slow eye movement, and detect frequent saccade. Note: Sacc = Saccade; Slow EM = Slow eye movement. (C) Our algorithm to detect EECD.
Figure 3.
 
Pupil/oculomotor behaviors during an entire Control session (participant g) and enlarged data for 30 seconds with correct answers to mental arithmetic. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position during the fully awake state (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position during the fully awake state (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter during the fully awake state (purple line = vergence angle; orange line = pupil diameter).
Figure 3.
 
Pupil/oculomotor behaviors during an entire Control session (participant g) and enlarged data for 30 seconds with correct answers to mental arithmetic. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position during the fully awake state (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position during the fully awake state (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter during the fully awake state (purple line = vergence angle; orange line = pupil diameter).
Figure 4.
 
Pupil/oculomotor behaviors during an entire test session (participant g), enlarged data and OAIs, and evaluation of our detection algorithms for OAIs. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position and OAIs (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter), and OAIs. (G) Frequent saccade (gray line = right eye; dark red line = normal saccade; pink line= frequent saccade). (H) Slow saccade (gray line = right eye; yellow line = slow saccade). (I) Saccade detection by the conventional algorithm (Engbert et al., 2015a) (gray line = right eye; light purple line = saccade or misdetection). Note that the ordinate is not the actual value. (J) Eye movement detection by our modified algorithm (gray line = right eye; light purple line = eye movement). Note that the ordinate is not the actual value. (K) Segregating normal saccades, non-slow eye movement, and slow eye movements by the conventional algorithm (gray line = right eye; dark green line = non-slow eye movement; light blue line = slow eye movement). Note that the ordinate is not the actual value. (L) Segregating normal saccades, slow saccade, and slow eye movements by our modified algorithm (gray line = right eye; dark red line = normal saccade; yellow line = slow saccade; light blue line = slow eye movement). Note that the ordinate is not the actual value. (M) Conventional algorithm for slow eye movement detection (Shin et al., 2011) (dark red circle = normal saccade; dark green triangle = non-slow eye movement; light blue rhombus = slow eye movement). (N) Modified algorithm to segregate normal saccades, slow saccade, and slow eye movements (dark red circle = normal saccade; yellow rectangle = slow saccade; light blue rhombus = slow eye movement; black line = angle threshold; black dashed line = amplitude threshold). (O) Enlarged vergence angle and pupil diameter, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; filled green rectangle = Div-Mio). (P) Enlarged vergence angular velocity and pupil velocity, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; black line = Div and Mio thresholds; filled green rectangle = Div-Mio). (Q) Enlarged pupil diameter (orange line = pupil diameter; black line = eyelid closure). (R) Eyelid closure duration and EECD (navy blue line = eyelid closure duration; black line = EECD threshold). Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 4.
 
Pupil/oculomotor behaviors during an entire test session (participant g), enlarged data and OAIs, and evaluation of our detection algorithms for OAIs. (A) Horizontal eye position (blue line = left eye; red line = right eye). (B) Vertical eye position (cyan line = left eye; magenta line = right eye). (C) Vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter). (D) Enlarged horizontal eye position and OAIs (blue line = left eye; red line = right eye). (E) Enlarged vertical eye position (cyan line = left eye; magenta line = right eye). (F) Enlarged vergence angle and pupil diameter (purple line = vergence angle; orange line = pupil diameter), and OAIs. (G) Frequent saccade (gray line = right eye; dark red line = normal saccade; pink line= frequent saccade). (H) Slow saccade (gray line = right eye; yellow line = slow saccade). (I) Saccade detection by the conventional algorithm (Engbert et al., 2015a) (gray line = right eye; light purple line = saccade or misdetection). Note that the ordinate is not the actual value. (J) Eye movement detection by our modified algorithm (gray line = right eye; light purple line = eye movement). Note that the ordinate is not the actual value. (K) Segregating normal saccades, non-slow eye movement, and slow eye movements by the conventional algorithm (gray line = right eye; dark green line = non-slow eye movement; light blue line = slow eye movement). Note that the ordinate is not the actual value. (L) Segregating normal saccades, slow saccade, and slow eye movements by our modified algorithm (gray line = right eye; dark red line = normal saccade; yellow line = slow saccade; light blue line = slow eye movement). Note that the ordinate is not the actual value. (M) Conventional algorithm for slow eye movement detection (Shin et al., 2011) (dark red circle = normal saccade; dark green triangle = non-slow eye movement; light blue rhombus = slow eye movement). (N) Modified algorithm to segregate normal saccades, slow saccade, and slow eye movements (dark red circle = normal saccade; yellow rectangle = slow saccade; light blue rhombus = slow eye movement; black line = angle threshold; black dashed line = amplitude threshold). (O) Enlarged vergence angle and pupil diameter, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; filled green rectangle = Div-Mio). (P) Enlarged vergence angular velocity and pupil velocity, and Div-Mio detection (purple line = vergence angle; orange line = pupil diameter; black line = Div and Mio thresholds; filled green rectangle = Div-Mio). (Q) Enlarged pupil diameter (orange line = pupil diameter; black line = eyelid closure). (R) Eyelid closure duration and EECD (navy blue line = eyelid closure duration; black line = EECD threshold). Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 5.
 
First emergence timings of OAIs (other than EECD) sorted in the emergence order in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 5.
 
First emergence timings of OAIs (other than EECD) sorted in the emergence order in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 6.
 
Raster plots of the emergence timings of each OAI detected by our detection algorithm in the Test session (participant l). Frequent saccade (pink), slow saccade (yellow), Div-Mio (green), slow eye movement (light blue), and EECD (navy blue) in order from the top (typical emergence order). Black circles indicate the first emergence of each OAI. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Figure 6.
 
Raster plots of the emergence timings of each OAI detected by our detection algorithm in the Test session (participant l). Frequent saccade (pink), slow saccade (yellow), Div-Mio (green), slow eye movement (light blue), and EECD (navy blue) in order from the top (typical emergence order). Black circles indicate the first emergence of each OAI. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Table 1.
 
VAS on subjective drowsiness of all participants. Note: Before: Reports before sessions (the higher the number, the greater the drowsiness), After: Reports after sessions, Change: Changes before and after the sessions (positive changes indicate increased subjective drowsiness).
Table 1.
 
VAS on subjective drowsiness of all participants. Note: Before: Reports before sessions (the higher the number, the greater the drowsiness), After: Reports after sessions, Change: Changes before and after the sessions (positive changes indicate increased subjective drowsiness).
Table 2.
 
Time when the participants were fully awake (i.e., answered the mental arithmetic correctly) in the Control session.
Table 2.
 
Time when the participants were fully awake (i.e., answered the mental arithmetic correctly) in the Control session.
Table 3.
 
Thresholds to detect OAIs. Note: Angle threshold: Threshold to segregate slow saccade and slow eye movement from normal saccade (Figures 2B and 4N). Amplitude threshold: Threshold to segregate slow saccade and slow eye movement (Figures 2B and 4N). Interval threshold: Threshold to detect frequent saccade from normal saccade and slow saccade (Figure 2B). Div threshold: Threshold to detect Div parts of Div-Mio (Figure 4P). Mio threshold: Threshold to detect Mio parts of Div-Mio (Figure 4P). EECD threshold: Threshold to detect EECD (Figures 2C and 4R).
Table 3.
 
Thresholds to detect OAIs. Note: Angle threshold: Threshold to segregate slow saccade and slow eye movement from normal saccade (Figures 2B and 4N). Amplitude threshold: Threshold to segregate slow saccade and slow eye movement (Figures 2B and 4N). Interval threshold: Threshold to detect frequent saccade from normal saccade and slow saccade (Figure 2B). Div threshold: Threshold to detect Div parts of Div-Mio (Figure 4P). Mio threshold: Threshold to detect Mio parts of Div-Mio (Figure 4P). EECD threshold: Threshold to detect EECD (Figures 2C and 4R).
Table 4.
 
First emergence timings of the OAIs detected by our detection algorithm in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
Table 4.
 
First emergence timings of the OAIs detected by our detection algorithm in the Test session in the seven participants who were awake in their pre-Test session VAS tests and became drowsy in their post-Test session VAS tests and the two participants who were awake in both pre- and post-Test-session VAS tests. Note: Sacc = Saccade; Slow EM = Slow eye movement.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×