The relationship between pupillary baseline manipulated by mental effort or luminance and subsequent pupillary responses

Abstract

Measuring pupillary response is a prevalent technique to evaluate mental states. It is indispensable to conduct a correction procedure for the pupillary baseline to get a meaningful conclusion from the pupillary response. However, the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response varies among studies. In this study, we used the subtractive and proportional baseline corrections to analyze the results. Furthermore, we manipulated the pupillary baseline through mental effort or luminance in the baseline period and investigated whether the subsequent stimulus-evoked pupillary responses were affected. We found that the mental effort–evoked pupillary response was attenuated with a larger pupillary baseline manipulated by a higher mental effort, whereas it was unaffected with the baseline manipulated by luminance. Also, the luminance-evoked pupillary response was attenuated with a smaller pupillary baseline manipulated by a brighter disk, whereas it was unaffected with the baseline manipulated by mental effort. The results could be obtained from subtractive and proportional baseline corrections. Our results suggest that mental effort manipulated pupillary baseline interacts with the subsequent mental effort elicited pupillary response, but not with the luminance elicited pupillary response; the luminance manipulated pupillary baseline interacts with the subsequent luminance elicited pupillary response, but not with the mental effort elicited pupillary response. It is important to consider the ways of controlling the pupillary baseline and subsequent pupillary response simultaneously.

Introduction

The pupillary response is a prevalent technique in psychological studies. Pupil diameters can be affected by many factors, such as luminance, spatial frequency, and color (Cocker & Moseley, 1996; Ellis, 1981; Hu, Hisakata, & Kaneko, 2019; Lobato-Rincón, Cabanillas-Campos, Bonnin-Arias, Chamorro-Gutiérrez, Murciano-Cespedosa, & Sánchez-Ramos Roda, 2014; Loewenfeld, 1999; Yahia, Hamburg, Sher, Ner, Yassin, Chibel, Mimouni, Derazne, Belkin, & Rotenstreich, 2018). For example, the pupil constricts when people look at a bright than a dark stimulus (Ellis, 1981; Loewenfeld, 1999). Such a phenomenon is known as the pupillary light reflex (PLR). Cognitive processing is another main factor affecting pupil diameters (Eckstein, Guerra-Carrillo, Miller Singley, & Bunge, 2017; Peinkhofer, Knudsen, Moretti, & Kondziella, 2019; van der Wel & van Steenbergen, 2018). For example, it has been shown that pupillary responses are modulated by attention (Binda, Pereverzeva, & Murray, 2013; Binda, Pereverzeva, & Murray, 2014; Hu et al., 2019). The pupil constricts when people covertly attend to a bright stimulus compared to a dark stimulus, and it is referred to as the attentional modulation of PLR (Binda et al., 2014). Furthermore, the pupil dilates with a high mental effort, such as when people perform a difficult mathematical problem or N-back task (van der Wel & van Steenbergen, 2018). 

Despite the prevalence of using pupillary response for psychological studies, the relationship between pupillary baseline and the subsequent stimulus-evoked pupillary response reported by many researchers remains contradictive (Cherng, Baird, Chen, & Wang, 2020; Gilzenrat, Nieuwenhuis, Jepma, & Cohen, 2010; Palinko & Kun, 2011; Peysakhovich, Vachon, & Dehais, 2017; Reilly, Kelly, Kim, Jett, & Zuckerman, 2019). Reilly et al. (2019) manipulated the pupillary baseline through luminance and found that different baselines did not affect the pupillary response evoked by the auditory target detection task. However, by manipulating the pupillary baseline through luminance, Cherng et al. (2020) found that the pupillary response evoked by emotional auditory stimuli was enhanced with a larger baseline. In addition, they also reported that the pupillary response evoked by saccade preparation was attenuated with a larger baseline, suggesting the ways of evoking the pupillary response should be considered. Peysakhovich et al. (2017) simultaneously manipulated the pupillary baseline through mental effort and luminance in another study. Their results showed that the pupillary response evoked by a mathematical task was enhanced with a smaller baseline when the baseline was manipulated by mental effort. In contrast, the pupillary response evoked by the same mathematical task was not affected when the baseline was manipulated by luminance. This study suggests that the ways of manipulating the pupillary baseline should be considered when predicting the subsequent pupillary response. From the results reported above, we presume that there is an interaction between the ways of controlling the pupillary baseline and the pupillary response. 

Another concern about the relationship between pupillary baseline and the subsequent stimulus-evoked pupillary response is how to conduct the baseline correction. Since the absolute pupil diameters are different among different people, even among different mental states for the same people, it is essential to conduct a baseline correction when analyzing the pupillary response (Loewenfeld, 1999; Mathôt, Fabius, Van Heusden, & Van der Stigchel, 2018). The correction can be either subtractive (subtracting the pupillary baseline from the observed pupillary response) or proportional (subtracting the pupillary baseline from the observed pupillary response and then divided by the pupillary baseline). Although a wonderful comparison between the two baseline corrections was made by Mathôt et al. (2018), we analyzed our data using subtractive and proportional methods separately to avoid confusing conclusions drawn from mathematical calculations. 

In this study, we aimed to examine the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response. More importantly, we manipulated the pupillary baseline through mental effort or luminance and investigated how the subsequent pupillary response evoked by mental effort or luminance was affected. Therefore, there were two periods in one trial. In the first period, participants were asked to perform an easy/difficult task to have different baselines manipulated by mental effort or look at a bright/dark disk to have different baselines manipulated by luminance. In the second period, participants were asked to perform an easy or difficult task to have different responses evoked by mental effort or to attend to a bright or dark disk to have different responses evoked by luminance. There are two reasons why we asked participants to attend to the bright or dark disk in the second period. First, the magnitude of pupil changes evoked by the mental effort is comparable with that evoked by luminance under such operation (Mathôt, 2018). Usually, the pupil changes about 10% of the pupillary baseline because of the stimulus luminance, whereas the pupil changes 1% to 5% because of cognitive factors. To avoid the possible contamination caused by how much the pupil changed relative to the pupillary baseline, we used the attentional modulation in PLR to achieve the luminance-evoked pupillary response. Second, it has been reported that PLR is attenuated when performing a demanding task (Steinhauer, Condray, & Kasparek, 2000). The interaction between mental effort–evoked pupillary response and luminance-evoked pupillary response can interfere with the conclusions when examining the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response. Therefore the first step in this study was to examine whether the attentional modulation in PLR interacted with the mental effort evoked–pupillary response (Experiment 1). After that, the pupillary baseline was manipulated by mental effort or luminance at first, followed by the stimulus-evoked pupillary response (Experiment 2 and Experiment 3). 

Experiment 1

Objective

In this experiment, we aimed to examine whether the attentional modulation in PLR interacted with the mental effort–evoked pupillary response. Participants were asked to attend to either bright or dark disk while performing either easy or difficult task. 

Participants and apparatus

Twenty-one graduate students (age range 23–27 years, eight females) from Tokyo Institute of Technology (Japan) and Shaanxi Normal University (China) were recruited in Experiment 1. All of them had normal or corrected-to-normal vision. We obtained informed consent from each of them. The Ethics Committee of the Tokyo Institute of Technology and Shaanxi Normal University approved the protocol. 

We recorded participants' eye movements and pupillary diameters using an infrared video-based eye tracker (EyeLink 1000 Plus, SR Research, 500 Hz sampling rate for participants from Tokyo Institute of Technology; Eyelink Portable Duo, SR Research, 500 Hz sampling rate for participants from Shaanxi Normal University). The pupil size resolutions of both apparatuses were 0.1% of the diameter. We presented the stimuli on a cathode-ray tube monitor (Sony GDM-F500R, 1280 × 1024 pixels, 60 Hz) for participants from Tokyo Institute of Technology and a liquid crystal display (Eizo FlexScan S1923, 1280 × 1024 pixels, 75 Hz) for participants from Shaanxi Normal University. Both monitors, which were gamma-corrected to present the stimuli with identical luminance, were placed 57 cm in front of the participants. During the experiment, participants were asked to refrain from head movements by putting their chins on a chin rest, and they were asked to respond to the corresponding tasks using a ten-key pad. The experiments were conducted in a completely dark room. 

Stimulus and procedure

The stimulus comprised three components: a cue line, a bright disk, and a dark disk (Figure 1). The luminance of each component was 0.06 cd/m², 67.2 cd/m², and 0.06 cd/m², respectively. The length of the cue line was 0.3°. The visual angle from the fixation dot to the center of the two disks was 4°, and the radius of the disks was 2°. A black fixation dot (0.06 cd/m² in luminance, 0.02° in radius) was presented at the center of the monitor throughout the experiment. A digit stream was presented at the center of the cued disk. Each digit was restricted to an area of 0.4° × 0.7°. The luminance of the digit was 33.6 cd/m². The luminance of the background was also 33.6 cd/m². 

Figure 1.

 

The time course of each trial in Experiment 1. Each trial consisted of a 1.5-second fixation period, a 3-second stimulus presentation period, and a response period. A cue indicating the attended brightness was presented from the beginning to just before the response period. During the stimulus presentation period, a digit stream containing six repetitions of 0.25-second digit and 0.25-second blank was presented at the center of the cued disk. Participants were asked to respond by pressing a key on a 10-key pad to an easy task (count the number of the non-1 digit in the stream) or a difficult task (multiply the displayed digits to answer the one-digit value of the result) based on the experimenter's instruction. Participants could rest during the response period. After responding, the next trial began. Note that the size of the cue, digit, and disk were rescaled for clarity in this figure.

The time course of each trial is depicted in Figure 1. Each trial started with a 1.5-second fixation period, followed by a three-second stimulus presentation period. Participants were asked to look at the fixation dot presented throughout the experiments. A cue line indicating attended brightness (left or right) was presented during the fixation period. Subsequently, the bright and dark disks were presented on the left and right hemifields. The combination of disk luminance and position was randomly chosen. Participants were asked to attend to the cued disk, of which a digit stream was presented at the center. The stream was composed of six digits chosen from 1 to 9. Each digit lasted 0.25 second, followed by a 0.25-second blank. The same digits were not presented in succession. As a result, there were three seconds for the stimulus presentation period. For an easy task (normal mental effort), participants were asked to count how many times the digit in the digit stream was not “1.” The digits other than “1” might appear once, twice, or three times. For a difficult task (high mental effort), participants were asked to multiply the displayed digits and answer the one-digit value of the result. The number of digits other than “1” correlated positively with the difficulties of the difficult task. The high mental effort might not be accomplished if the number was too small. Based on our preliminary experiments, four digits other than “1” in the difficult task could induce high enough mental efforts for all participants. Furthermore, we excluded the digit “5” in the difficult task to prevent a multiplying result of zero, which was relatively easy. After that, the cue line and two disks disappeared, during which participants were asked to respond to the task by pressing a key from a ten-key pad. Participants were asked to refrain from head movements, eye movements, and blinking during the fixation and stimulus presentation periods. They could blink during the response period. Once they were prepared, they could start the next trial by answering the task. 

We assigned 20 trials for each attended brightness, leading to 40 trials (20 repetitions × two attended brightness) in a block. There were two blocks, with easy and difficult tasks conducted separately. Each block lasted approximately eight minutes with at least five minutes’ rest inserted between blocks. The sequence of two blocks was counterbalanced among the participants. 

Data analysis

The detailed data analysis procedures have been described previously (Hu et al., 2019). Briefly, we used a cubic Hermite spline interpolation and a Savitzky-Golay filter (order 2, window length 0.1 second) to interpolate the missing samples and smooth the raw data separately. After preprocessing, the pupillary baseline was calculated as the average of 500 ms before the start of the stimulus presentation period. Then, we conducted the subtraction and proportional baseline corrections separately to normalize the pupillary trace. After that, to conduct statistical analysis, we calculated the mean pupil change by averaging across the samples of 1.5 seconds before the stimulus offset. This chosen period should not be affected by the manipulation before the stimulus task and the initial change elicited by the occurrence of the stimulus itself. At last, we excluded the trials with deviated gaze positions (1° away from the fixation position) to avoid measurement error due to the foreshortening pupil image, those with deviated baselines (2.5 standard deviations away from the average baselines) to avoid possible baseline artifacts, and those with incorrect answers to avoid inattentive states during the experiments (Gagl, Hawelka, S., & Hutzler, 2011; Mathôt et al., 2018). The same analysis was used for all three experiments. 

Results

The proportion of correct answers was 98.5% (standard error [SE] ± 0.5%) for the easy task and 83.0% (SE ± 2.3%) for the difficult task. We performed a two-sided paired t-test to evaluate the effect of the type of task (easy or difficult) on the answer rate. The correct rate for the easy task was significantly higher than that for the difficult task (t(20) = 6.608, p < 0.001, Cohen's d = 1.44). The excluding rate was 3.5% (SE ± 0.7%) for the easy task and 18.7% (SE ± 2.3%) for the difficult task. 

The average pupillary baseline was 4.46 mm (SE ± 0.02 mm) for the easy task and 4.82 mm (SE ± 0.03 mm) for the difficult task (Figure 2A). Similarly, we performed a two-sided paired t-test to evaluate the effect of the type of task on the pupillary baseline. The pupillary baseline for the easy task was significantly smaller than that for the difficult task (t(20) = 5.598, p < 0.001, Cohen's d = 1.22). This significant difference might be caused by the preparation effect of the experimental instructions (Ayasse & Wingfield, 2020; DiCriscio, Hu, & Troiani, 2018; Steinhauer, Siegle, Condray, & Pless, 2004). To sum, these results indicated that the two types of tasks successfully induced different mental efforts. 

Figure 2.

 

The results of Experiment 1. (A) Pupillary baselines. The abscissa represents the stimulus task difficulty, and the ordinate represents the absolute pupillary diameters (in mm). (B) Average pupillary traces after subtractive baseline corrections. The normalized pupil change (in mm) is plotted as a function of time (in seconds). Time 0 represents the start of the stimulus presentation period. The combination of line shape and color indicates the experimental conditions. The solid and dotted lines represent the results when performing the easy and difficult tasks, respectively. The black and gray lines represent the results when attending to the dark and bright disks, respectively. (C) Mean pupil change across participants after subtractive baseline corrections. The abscissa represents the stimulus task difficulty, and the ordinate represents the mean pupil change (in mm). The color of bars represents participants' attended brightness, with gray and black bars indicating the mean pupil changes when attending to the bright and dark disks, respectively. The error bars and the thin lines surrounding the traces represent the standard error of the means across 21 participants.

The results of subtractive corrected pupillary traces are depicted in Figure 2B. The abscissa represents the time, with time 0 indicating the start of the stimulus presentation period. The ordinate represents the normalized pupil change, with a positive value indicating pupillary dilation and a negative value indicating pupillary constriction. The combination of color and line type represents different attended brightness (bright or dark disk) and types of stimulus tasks (easy or difficult task). As shown in Figure 2B, the pupil was smaller when attending to a bright than a darker disk and smaller when performing an easy than a difficult task. 

The results of subtractive corrected mean pupil change are depicted in Figure 2C. The abscissa represents the stimulus task difficulty, and the ordinate represents the mean pupil change across participants. The gray and black bars represent the results when attending to the bright and dark disks, respectively. We performed a two-way repeated analysis of variance (ANOVA) on the mean pupil change to examine whether there was an interaction effect. The within-subject factors included the attended brightness (bright or dark) and the stimulus task difficulty (easy or difficult). We found significantly larger pupil sizes when performing difficult compared to easy stimulus task (F(1,20) = 16.331, p < 0.001, η_p² = 0.45) and when attending to dark compared to bright stimulus disk (F(1,20) = 29.610, p < 0.001, η_p² = 0.60). However, there was no significant interaction effect between the stimulus task difficulty and attended brightness (F(1,8) = 0.464, p = 0.503, η_p² = 0.02). We got the same conclusions by using the proportional baseline correction. Please see Supplementary Figure S1 in the supplementary material. 

The results showed that the pupil was significantly smaller when attending to a bright than a dark disk. Also, the pupil was significantly smaller when performing an easy than a difficult task. Our results confirmed the attentional modulation in PLR and the effect of mental effort on the pupillary response (Binda et al., 2013; Binda et al., 2014; van der Wel & van Steenbergen, 2018). Furthermore, there was no interaction between the attended brightness (attentional modulation in PLR) and stimulus task difficulty (effect of mental effort on the pupillary response). 

Experiment 2

Objective

In this experiment, we manipulated the pupillary baseline through mental effort and investigated whether the subsequent mental effort–evoked pupillary response and luminance-evoked pupillary response were affected. To simplify the description for the remainder of this article, we referred to the “baseline task” as the task presented before the stimulus presentation period, the “stimulus task” as the task during the stimulus presentation period, and the “stimulus disk” as the disk presented during the stimulus presentation period. Participants were asked first to perform an easy or difficult baseline task and then attend to a bright or dark stimulus disk while performing an easy or difficult stimulus task as they did in previous experiments. 

Participants and apparatus

Nineteen graduate students (age range 23–27 years, eight females) from Tokyo Institute of Technology (Japan) and Shaanxi Normal University (China) who had participated in Experiment 1 were recruited. 

The apparatuses used in Experiment 2 were the same as those used in Experiment 1. 

Stimulus and procedure

The stimulus was the same as that used in Experiment 1. To manipulate the pupillary baseline through the mental effort, we presented a baseline task before stimulus presentation. The baseline task was the same as the task used during the stimulus presentation period. Participants were asked to count how many times the digit in the digit stream was not “1” in the easy baseline task or multiply the displayed digits and answer the one-digit value of the results in the difficult baseline task. During the baseline task period, each digit was restricted to an area of 0.2° × 0.35° and presented on the top of the fixation dot. The luminance of the digit was 0.06 cd/m². 

The time course of each trial is depicted in Figure 3. The procedure was similar to that in Experiment 1, except for the following. Instead of the 1.5-second fixation period in Experiment 1, a one-second fixation period was followed by a four-second baseline task period. Note that a one-second blank period after the baseline task was inserted to allow participants to keep the answer of the baseline task in mind. The baseline task and the stimulus task were independent of each other. As a result, participants were asked to answer two digits by pressing two keys during the response period. 

Figure 3.

 

The time course of each trial in Experiment 2. Each trial consisted of a one-second fixation period, a four-second baseline task period, a three-second stimulus presentation period, and a response period. A cue indicating the attended brightness was presented from the beginning to just before the response period. A digit stream containing six repetitions of 0.25-second digit and 0.25-second blank was presented at the top of the fixation dot during the baseline task period. An easy task (count the number of the non-1 digit in the stream) or a difficult task (multiply the displayed digits to answer the one-digit value of the result) followed by a one-second blank was presented before the stimulus presentation period.

There were four separate blocks in this experiment. Within each block, the attended brightness (left or right) in each trial was randomly chosen, whereas the baseline task difficulty (easy or difficult) and the stimulus task difficulty (easy or difficult) were fixed. Consequently, there were four blocks: easy baseline task and easy stimulus task, easy baseline task and difficult stimulus task, difficult baseline task and easy stimulus task, and difficult baseline task and difficult stimulus task. We assigned 20 trials for each attended brightness, leading to 40 trials (20 repetitions × two attended brightness) in a block. Each block lasted approximately eight minutes with at least five minutes’ rest inserted between blocks. The four blocks were conducted on the same day but with counterbalanced sequences among the participants. Although there might be extra mental efforts evoked by switching between two tasks, we presumed that it would not affect our results because the difficult task we deployed could induce a high enough mental effort to overwhelm the load induced by switching between tasks. Before the experiments, participants were asked to do two complete practice blocks (easy baseline task and difficult stimulus task, difficult baseline task and difficult stimulus task, 40 trials each). 

Results

The proportion of correct answers was 95.1% (SE ± 1.0%) for the easy baseline task and easy stimulus task, 87.3% (SE ± 2.1%) for the easy baseline task and difficult stimulus task, 81.2% (SE ± 3.0%) for the difficult baseline task and easy stimulus task, and 73.0% (SE ± 3.3%) for the difficult baseline task and difficult stimulus task. We performed a two-way repeated ANOVA to evaluate the effects of the baseline task difficulty (easy or difficult) and the stimulus task difficulty (easy or difficult) on the answer rate. We found significantly higher correct rates when performing easy compared to difficult baseline task (F(1,18) = 39.427, p < 0.001, η_p² = 0.69) and when performing easy compared to difficult stimulus task (F(1,18) = 20.990, p < 0.001, η_p² = 0.54). However, there was no significant interaction effect between the baseline task difficulty and stimulus task (F(1,18) = 0.018, p = 0.894, η_p² < 0.01). The excluding rate was 6.4% (SE ± 1.0%) for the block of the easy baseline task and easy stimulus task, 14.5% (SE ± 2.1%) for that of the easy baseline task and difficult stimulus task, 21.1% (SE ± 2.8%) for that of the difficult baseline task and easy stimulus task, and 28.3% (SE ± 3.2%) for that of the difficult baseline task and difficult stimulus task. 

The average pupillary baseline was 4.67 mm (SE ± 0.02 mm) for the block of the easy baseline task and easy stimulus task, 4.76 mm (SE ± 0.03 mm) for that of the easy baseline task and difficult stimulus task, 5.13 mm (SE ± 0.03 mm) for that of the difficult baseline task and easy stimulus task, and 5.19 mm (SE ± 0.03 mm) for that of the difficult baseline task and difficult stimulus task (Figure 4A). We performed a two-way repeated ANOVA to evaluate the effects of the baseline task difficulty and stimulus task difficulty on the pupillary baseline. We found significantly larger pupillary baselines when performing difficult compared to easy baseline task (F(1,18) = 42.588, p < 0.001, η_p² = 0.70). However, there were no significant difference for performing difficult compared to easy stimulus task (F(1,18) = 2.871, p = 0.107, η_p² = 0.14) and no significant interaction effect between the baseline task difficulty and stimulus task difficulty (F(1,18) = 0.134, p = 0.719, η_p² = 0.01). To sum, the results indicated that the different baseline task difficulties could successfully induce different mental efforts and manipulate the pupillary baseline. Furthermore, the range of pupillary baselines (4.67–5.19 mm) was appropriate to allow further pupillary constriction or dilation (Loewenfeld, 1999). 

Figure 4.

 

The results of Experiment 2. (A) Pupillary baselines. The two different baseline conditions are separately illustrated. In each panel, the abscissa represents the stimulus task difficulty, and the ordinate represents the absolute pupillary diameters (in mm). (B) Individual results after subtractive baseline corrections. The abscissa represents the baseline task difficulty, and the ordinate represents the mental effort difference. See the text for details. Each paired dot represents a different individual. (C) Average pupillary traces after subtractive baseline corrections. The two different baseline conditions are separately illustrated. The thin lines surrounding the traces represent the standard error of the means across 19 participants. (D) Mean pupil change across participants after subtractive baseline corrections. The two different baseline conditions are separately illustrated. The error bars represent the standard error of the means across 19 participants.

The results of subtractive corrected pupillary traces are depicted in Figure 4C. The left and right panels represent the results of the blocks of the easy and difficult baseline tasks, respectively. In each panel, the abscissa represents the time, with time 0 indicating the start of the stimulus presentation period. The ordinate represents the normalized pupil change, with a positive value indicating pupillary dilation and a negative value indicating pupillary constriction. The combination of color and line type represents different attended brightness and types of stimulus tasks. As shown in Figure 4C, the difference between the pupillary responses when performing an easy stimulus task and a difficult stimulus task was larger in the left panel (easy baseline task) than in the right panel (difficult baseline task). 

The results of subtractive corrected mean pupil change are depicted in Figure 4D. The left and right panels represent the results of the blocks of the easy and difficult baseline tasks, respectively. In each panel, the abscissa represents the stimulus task difficulty, and the ordinate represents the mean pupil change across participants. The gray and black bars represent the results when attending to the bright and dark stimulus disks, respectively. We performed a three-way repeated ANOVA on the mean pupil change to examine whether there was an interaction effect. The within-subject factors included the baseline task difficulty (easy or difficult), attended brightness (bright or dark), and stimulus task difficulty (easy or difficult). We found significantly larger pupil sizes when performing difficult compared to easy baseline task (F(1,18) = 57.539, p < 0.001, η_p² = 0.76), when performing difficult compared to easy stimulus task (F(1,18) = 51.658, p < 0.001, η_p² = 0.74), and when attending to dark compared to bright stimulus disk (F(1,18) = 88.149, p < 0.001, η_p² = 0.83). We also found that the interaction effect between the baseline task difficulty and stimulus task difficulty was significant (F(1,18) = 19.105, p < 0.001, η_p² = 0.52). In contrast, there were no interaction effects between the baseline task difficulty and attended brightness, between the stimulus task difficulty and attended brightness, and among three factors (all p > 0.050). We got the similar conclusions by using the proportional baseline correction. Please see Supplementary Figure S2 in the supplementary material. 

Since the interaction effect between the baseline task difficulty and stimulus task difficulty was significant, we analyzed the effect of stimulus task difficulty on different types of baseline tasks. P-values were adjusted using the Bonferroni multiple testing correction method. We found significantly larger pupil sizes when performing difficult than easy stimulus tasks, irrespective of the baseline task difficulty (Figure 4D, left panel, p < 0.001; right panel, p < 0.001). Note that the mean difference of the mean pupil change between easy and difficult stimulus tasks was much larger (0.24 mm or 5.1%) when following an easy baseline task than that (0.15 mm or 3.0%) when following a difficult baseline task. 

To further verify the key interactions, we conducted a Bayesian Repeated Measured ANOVA using JASP (JASP Team, 2022). We found that the Bayesian factor (BF) across matched models was 7.832 for the interaction between baseline task difficulty and stimulus task difficulty, indicating strong evidence for the interaction. However, BF was 0.261 for the interaction between baseline task difficulty and attended brightness, indicating the lack of interaction as the classical ANOVA pointed out. 

The results of subtractive corrected individual results are depicted in Figure 4B. We calculated the mental effort difference as the difference of mean pupil change between the conditions when performing the difficult stimulus task and easy stimulus task. Then, we drew a scatter plot with the baseline task difficulty as the abscissa and the mental effort difference as the ordinate. Each paired dot represents a different individual. Note that, due to the significant interaction between the baseline task difficulty and stimulus task difficulty, the values in the left part were higher than those in the right part (six exceptions, dotted line), indicating that the effects of mental effort were attenuated when performing the difficult baseline task. 

There were two points worthy of notice. First, the effect of the mental effort was largely dismissed when performing a difficult baseline task before stimulus presentation, which was consistent with the previous studies (Aston-Jones & Cohen, 2005; de Gee et al., 2014; Gilzenrat et al., 2010; Murphy et al., 2011; Peysakhovich et al., 2017). Second, although the pupillary baseline differed significantly by manipulating the baseline task, the luminance-evoked pupillary response was not affected. 

Experiment 3

Objective

In this experiment, we manipulated the pupillary baseline through luminance and investigated whether the subsequent mental effort–evoked pupillary response and luminance-evoked pupillary response were affected. To simplify the description for the remainder of this article, we referred to the “baseline disk” as the disk presented before the stimulus presentation period, the “stimulus task” as the task during the stimulus presentation period, and the “stimulus disk” as the disk presented during the stimulus presentation period. Participants were asked to first look at a bright or dark baseline disk at the center and then attend to one of the bright and dark stimulus disks in the periphery while performing an easy or difficult stimulus task as they did in previous experiments. 

Participants and apparatus

Nineteen graduate students (age range: 23–27 years, eight females) from Tokyo Institute of Technology (Japan) and Shaanxi Normal University (China) who had participated in Experiment 1 and Experiment 2 were recruited. 

The apparatuses used in Experiment 2 were the same as those used in Experiment 1 and Experiment 2. 

Stimulus and procedure

The stimulus was the same as that used in Experiment 1. To manipulate the pupillary baseline through luminance, we presented a bright or dark baseline disk at the center of the monitor before stimulus presentation. The luminance of bright and dark baseline disks was 67.2 cd/m² and 0.06 cd/m², respectively. The radius of the baseline disks was 6°. 

The time course of each trial is depicted in Figure 5. The procedure was similar to that in Experiment 1, except for the following. Instead of the 1.5 s fixation period in Experiment 1, a 1 s fixation period was followed by a 3 s baseline disk period. During the baseline disk period, a bright or dark baseline disk was presented at the center of the monitor, and participants were instructed to look at the fixation dot. 

Figure 5.

 

The time course of each trial in Experiment 3. Each trial consisted of a 1-second fixation period, a 3-second baseline disk period, a 3-second stimulus presentation period, and a response period. A cue indicating the attended brightness was presented from the beginning to just before the response period. A bright or dark baseline disk was presented before stimulus presentation and lasted three seconds.

There were four separate blocks in this experiment. Within each block, the attended brightness (left or right) was randomly chosen in each trial, whereas the luminance of the baseline disk (bright or dark) presented during the baseline disk period and the stimulus task difficulty (easy or difficult) during the stimulus presentation period were fixed. Consequently, there were four blocks: dark baseline disk and easy stimulus task, dark baseline disk and difficult stimulus task, bright baseline disk and easy stimulus task, and bright baseline disk and difficult stimulus task. We assigned 20 trials for each attended brightness, leading to 40 trials (20 repetitions × two attended brightness) in a block. Each block lasted approximately 6 mins with at least 5 mins rest inserted between blocks. The four blocks were conducted on the same day but with counterbalanced sequences among the participants. Before the experiments, participants were asked to do one complete practice block (bright baseline disk and difficult stimulus task, 40 trials). 

Results

The proportion of correct answers was 98.7% (SE ±0.5%) for the dark baseline disk and easy stimulus task, 91.3% (SE ± 2.2%) for the dark baseline disk and difficult stimulus task, 98.8% (SE ± 0.4%) for the bright baseline disk and easy stimulus task, and 90.9% (SE ± 2.5%) for the bright baseline disk and difficult stimulus task. We performed a two-way repeated ANOVA to evaluate the effects of luminance of baseline disk (bright or dark) and stimulus task difficulty (easy or difficult) on the answer rate. We found significantly higher correct rates when performing easy compared to difficult stimulus task (F(1,18) = 12.277, p = 0.003, η_p² = 0.41). However, there were no main effect of baseline disk brightness (F(1,18) = 0.033, p = 0.857, η_p² < 0.01) and the interaction (F(1,18) = 0.071, p = 0.793, η_p² < 0.01). The excluding rate was 3.6% (SE ± 0.9%) for the block of the dark baseline disk and easy stimulus task, 10.7% (SE ± 2.2%) for that of the dark baseline disk and difficult stimulus task, 3.0% (SE ± 0.7%) for that of the bright baseline disk and easy stimulus task, and 10.4% (SE ± 2.5%) for that of the bright baseline disk and difficult stimulus task. 

The average pupillary baseline was 5.30 mm (SE ± 0.02 mm) for the block of the dark baseline disk and easy stimulus task, 5.35 mm (SE ± 0.02 mm) for that of the dark baseline disk and difficult stimulus task, 4.08 mm (SE ± 0.02 mm) for that of the bright baseline disk and easy stimulus task, and 4.15 mm (SE ± 0.02 mm) for that of the bright baseline disk and difficult stimulus task (Figure 6A). We performed a two-way repeated ANOVA to evaluate the effects of baseline disk brightness and stimulus task difficulty on the pupillary baseline. We found significantly larger pupillary baselines when looking at dark compared to bright disk (F(1,18) = 430.149, p < 0.001, η_p² = 0.96). However, there were no significant differences for performing difficult compared to easy stimulus tasks (F(1,18) = 0.869, p = 0.364, η_p² = 0.05) and no significant interaction effect between the baseline disk brightness and stimulus task difficulty (F(1,18) = 0.343, p = 0.565, η_p² = 0.02). To sum, the results indicated that the baseline disk brightness could successfully manipulate the pupillary baseline. Furthermore, the range of pupillary baselines (4.08–5.35 mm) was appropriate to allow further pupillary constriction or dilation (Loewenfeld, 1999). 

Figure 6.

 

The results of Experiment 3. (A) Pupillary baselines. The two different baseline conditions are separately illustrated. In each panel, the abscissa represents the stimulus task difficulty, and the ordinate represents the absolute pupillary diameters (in mm). (B) Individual results after subtractive baseline corrections. The abscissa represents the baseline disk brightness, and the ordinate represents the luminance difference. See the text for details. Each paired dot represents a different individual. (C) Average pupillary traces after subtractive baseline corrections. The two different baseline conditions are separately illustrated. The thin lines surrounding the traces represent the standard error of the means across 19 participants. (D) Mean pupil change across participants after subtractive baseline corrections. The two different baseline conditions are separately illustrated. The error bars represent the standard error of the means across 19 participants.

The results of subtractive corrected pupillary traces are depicted in Figure 6C. The left and right panels represent the results of the blocks of the bright and dark baseline disks, respectively. In each panel, the abscissa represents the time, with time 0 indicating the start of the stimulus presentation period. The ordinate represents the normalized pupil change, with a positive value indicating pupillary dilation and a negative value indicating pupillary constriction. The combination of color and line type represents different attended brightness and types of stimulus tasks. As shown in Figure 6C, the difference between the pupil when attending to a bright stimulus disk and a dark stimulus disk was smaller in the left panel (bright baseline disk) than in the right panel (dark baseline disk). 

The results of subtractive corrected mean pupil change are depicted in Figure 6D. The left and right panels represent the results of the blocks of the bright and dark baseline disks, respectively. In each panel, the abscissa represents the stimulus task difficulty, and the ordinate represents the mean pupil change across participants. The gray and black bars represent the results when attending to the bright and dark stimulus disks, respectively. We performed a three-way repeated ANOVA on the mean pupil change to examine whether there was an interaction effect. The within-subject factors included the baseline disk brightness (bright or dark), attended brightness (bright or dark), and stimulus task difficulty (easy and difficult). We found significantly larger pupil sizes when looking at dark compared to bright baseline disk (F(1,18) = 331.263, p < 0.001, η_p² = 0.95), when performing difficult compared to easy stimulus task (F(1,18) = 63.020, p < 0.001, η_p² = 0.78), and when attending to dark compared to bright stimulus disk (F(1,18) = 58.977, p < 0.001, η_p² = 0.77). We also found that the significant interaction effect between baseline disk brightness and attended brightness (F(1,18) = 19.866, p < 0.001, η_p² = 0.53) and between stimulus task difficulty and attended brightness (F(1,18) = 4.448, p = 0.049, η_p² = 0.20). In contrast, there were no interaction effects between the baseline disk brightness and stimulus task difficulty and among three factors (all p > 0.050). We got the similar conclusions by using the proportional baseline correction except for the lower statistical power compared to the subtractive baseline correction, which was consistent with the finding of Mathôt et al. (2018). Please see Supplementary Figure S3 in the supplementary material. 

Because the interaction effect between the baseline disk brightness and attended brightness was significant, we analyzed the effect of attended brightness at different luminance of the baseline disk. The p values were adjusted using the Bonferroni multiple testing correction method. We found significantly larger pupil sizes when attending to dark compared to bright stimulus disk, irrespective of the baseline disk brightness (Figure 6D, left panel, p < 0.001; right panel, p < 0.001). Note that the mean difference of the mean pupil change between attending to bright and dark disks was much larger (0.11 mm or 2.1%) when following a dark baseline disk than that (0.04 mm or 1.0%) when following a bright baseline disk. 

To further verify the key interactions, we conducted a Bayesian Repeated Measured ANOVA using JASP. We found that the BF across matched models was 0.257 for the interaction between baseline disk brightness and stimulus task difficulty, indicating the lack of interaction as the classical ANOVA pointed out. However, BF was 0.598 for the interaction between baseline disk brightness and attended brightness, showing weak evidence for the interaction effect. Although the result obtained from Bayesian ANOVA was not as strong as that from the classical ANOVA, it would not affect our conclusions on the lack of interaction between baseline disk brightness and stimulus task difficulty. 

The results of subtractive corrected individual results are depicted in Figure 6B. We calculated the luminance difference as the difference of mean pupil change between the conditions when attending to the dark stimulus disk and bright stimulus disk. Then, we drew a scatter plot with the baseline disk brightness as the abscissa and the luminance difference as the ordinate. Each paired dot represents a different individual. Note that, because of the significant interaction between the baseline disk brightness and attended brightness, the values in the left part were lower than those in the right part (four exceptions, dotted line), indicating that the effects of luminance were attenuated when looking at the bright baseline disk. 

There were three points worthy of notice. First, since the baseline disk was bright in the left panels of Figure 6, the pupils dilated overall (a positive value); since the baseline disk was dark in the right panels, the pupils constricted overall (a negative value). Second, the attentional modulation in PLR was largely dismissed when looking at a bright baseline disk before stimulus presentation, which was consistent with the previous study (Hu et al., 2021). Third, although the pupillary baseline differed significantly by manipulating the baseline disk brightness, the mental effort–evoked pupillary response was not affected. 

Discussion

We conducted three experiments to examine the relationship between pupillary baseline and stimulus-evoked pupillary response. The pupillary baseline was manipulated through the mental effort or luminance, and the pupillary response was evoked by mental effort or luminance. We used the subtractive and proportional baseline corrections to analyze our results separately. In Experiment 1, we reported that there was no interaction between luminance-evoked pupillary response (achieved by attentional modulation in PLR) and mental effort–evoked pupillary response (achieved by mathematical multiplying task), which provided the foundation for further experiments. In Experiment 2, we found that the mental effort–evoked pupillary response was attenuated with a larger pupillary baseline manipulated by a difficult task. In contrast, the luminance-evoked pupillary response was not affected by the different pupillary baselines manipulated by the different mental efforts. In Experiment 3, we found that luminance-evoked pupillary response was attenuated with a smaller pupillary baseline manipulated by a bright disk. In contrast, the mental effort–evoked pupillary response was not affected by the different pupillary baselines manipulated by the different luminance. Moreover, the results mentioned above were obtained for subtractive and proportional baseline corrections. 

One of the possible explanations for the results could be the adaption effect caused by the baseline task or disk because the manipulation of easy/difficult baseline task and bright/dark baseline disk was the same as the cognitive task and disk during the stimulus presentation period in Experiment 2 and Experiment 3, respectively. It was possible that participants were adapted to the baseline task and had impaired the ability for the subsequent stimulus task, whereas the stimulus disk remained no adaption in Experiment 2. However, when an easy baseline task was performed, the pupillary response evoked by the same task was not impaired in Experiment 2 (Figure 4). These results suggested that the baseline task per se did not affect the pupillary response evoked by the stimulus task. It was the difficulty of the baseline task that took effect. The same logic holds for the results of Experiment 3. Someone might be concerned that participants were adapted to the baseline disk and had impaired ability for the stimulus disk, whereas the stimulus task remained no adaption in Experiment 3. However, when a dark baseline disk was presented, the pupillary response evoked by the same disk was not impaired in Experiment 3 (Figure 6). These results suggested that the baseline disk per se did not affect the pupillary response evoked by the stimulus disk. It was the luminance of the baseline disk that took effect. Therefore, we supposed that the present results were not due to the adaption effect. 

Another concern about the present results is the physiological limit of the pupillary response per se, which is in the range of 2 to 8 mm (Loewenfeld, 1999). In Experiment 2, the pupillary baseline was manipulated by participants' mental effort (easy or difficult baseline task), leading to a difference of 0.52 mm of pupillary diameter at most. On the other hand, in Experiment 3, the pupillary baseline was manipulated by luminance (bright or dark baseline disk), leading to a difference of 1.27 mm of pupillary diameter at most. Although pupillary baselines were different in Experiment 2 and Experiment 3, the absolute value of pupillary baseline was 4.08 to 5.35 mm, a range far from the physiological limits of pupillary diameter (2 or 8 mm) (Loewenfeld, 1999). Furthermore, the critical pupil size, defined as the value when the pupillary response becomes nonlinear due to the iris muscles, is 3.36 mm for pupillary constriction and 5.72 mm for pupillary dilation (average across six normal participants) (Loewenfeld, 1999). Although the pupillary baselines were precisely manipulated in Experiment 2 (4.67-5.19 mm) and Experiment 3 (4.08-5.35 mm), both ranges were in the physiological ranges of the critical pupil size. Therefore the pupillary response in this study should be linear from the physiological perspective. The physiological limits of pupillary responses could not explain our results. 

We reported a significant main effect of the stimulus task difficulty for the pupillary baseline in Experiment 1, which was reported in previous studies (Ayasse & Wingfield, 2020; DiCriscio et al., 2018; Steinhauer et al., 2004). Steinhauer et al. (2004) assumed that participants unconsciously prepared to perform a challenging cognitive task once they were informed of the type of task before each block. Hence, the pupillary baseline was larger when participants were informed to perform a difficult stimulus task than an easy stimulus task in the present study. However, such preparation effect was not reported in Experiment 2 and Experiment 3, wherein participants were asked to perform an easy or difficult baseline task and look at the fixation dot embedded within a bright or dark baseline disk, respectively. This finding might be suggestive because it has always been difficult to establish a completely task-free condition once the task instruction is informed in advance. Further study is needed to eliminate the confounding preparation activity evoked by experimenters' instruction. 

The lack of interaction effect in Experiment 1 and the significant interaction effects in Experiment 2 and Experiment 3 suggested the independence between mental effort–evoked pupillary response and luminance-evoked pupillary response. However, previous research showed that PLR was attenuated when performing a demanding task (Steinhauer et al., 2000). The different conclusions might be caused by the way how the pupillary response was evoked by luminance. In this study, participants were asked to attend to the bright or dark stimulus while the stimulus was kept unchanged during the experiments. In contrast, participants looked at the stimulus with different luminance directly in the previous research (Steinhauer et al., 2000). It has been shown that the subcortical pathway, passing through the olivary pretectal nucleus, Edinger-Westphal nucleus, and ciliary ganglion, innervates the PLR, and a cortical pathway, passing through superior colliculus (SC), innervates the attentional modulation in PLR (Mathôt, 2018; Wang & Munoz, 2015; Wang & Munoz, 2018). Therefore the discrepancy between the current and previous research might be caused by the different pathways underlying the luminance-evoked pupillary response. Nevertheless, our results were inconsistent with previous research wherein researchers found that the SC was involved in the cognition-related pupillary response (Wang & Munoz, 2021). Although the direct evidence linking LC to other nuclei has not been found yet, the LC-NE (Locus Coeruleus-Norepinephrine) system is likely involved in the arousal-related pupil circuit. LC can innervate pupil size by projecting NE and functionally connect with SC (Joshi & Gold, 2020; Peinkhofer et al., 2019; van der Wel & van Steenbergen, 2018; Wang & Munoz, 2018). One of the possible explanations for reconciling the inconsistency is the role of SC. As Wang & Munoz (2021) reported, the luminance signals did not affect SC activity. It is possible that the role of SC in the attentional modulation in PLR might be simply the initiation of attention per se and not related to the luminance-evoked pupil changes. In other conditions where attention and cognition are related, the role of SC might be related to pupil change, such as the pupil could track the lapse of attention (van den Brink, Murphy, & Nieuwenhuis, 2016; Unsworth & Robison, 2016). This is supported by the layered structure of the SC (Wang, Boehnke, White, & Munoz, 2012). They found that the microstimulation of the intermedia SC layers could cause pupillary dilation, whereas that of the superficial SC layers did not. 

We analyzed our results using the subtractive and proportional baseline corrections, respectively, and we found no difference between the two methods. This was not a surprise. Although Mathôt et al. (2018) recommended the subtractive method over the proportional method due to the increased statistical power and less pronounced distortion, they also pointed out that both methods were reliable if the trials with baseline artifacts were not involved in the analysis. In our experiments, we excluded trials with possible baseline artifacts before further analyses (See Section 2.4). Therefore we could draw the same conclusions from the subtractive and proportional methods. 

Pupillometry has been a prevalent physiological method for human-computer interaction (HCI) due to its non-invasive and non-obtrusive properties compared to other methods, such as heart rate, electromyography, functional near-infrared spectroscopy (Ahmad, Keller, Robb, & Lohan, 2020; Köles, 2017; Lohani, Payne, & Strayer, 2019; Mathôt et al., 2016; Naber et al., 2013; Pfleging et al., 2016). In addition, the apparatus is relatively small and easy to operate. It is enough to measure the pupillary response by locating an eye-tracking apparatus before users, so there is no need for them to wear extra devices. Both users' cognitive states and attended brightness can be assessed using the same pupillary diameter data under laboratory environments. For example, Pfleging et al. (2016) built a model to assess participants' working loads independently of the lighting condition in 75% of the tested conditions. Mathôt, Melmi, Van der Linden, & Van der Stigchel (2016) assessed participants' attended brightness with an average of 87.6% accuracy for an eight-input visual interface. Because the attentional modulation in PLR and the effect of mental effort on pupillary response were independent of each other, as shown in our results, it is reasonable to use the attentional modulation in PLR to measure users' attended brightness without considering their cognitive states. On the other hand, it is reasonable to use the effect of mental effort on pupillary response to measure users' mental efforts without considering their attentional states. Hence, we suggested that HCI that assesses the mental effort and attended brightness can be used in a more complicated natural environment than a precisely manipulated laboratory environment. 

Conclusions

In summary, we examined the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response. The mental effort–evoked pupillary response was attenuated with a larger pupillary baseline manipulated by a difficult task, whereas the luminance-evoked pupillary response was not affected by the different pupillary baselines manipulated by the mental effort. The luminance-evoked pupillary response was attenuated with a smaller pupillary baseline manipulated by a bright disk, whereas the mental effort–evoked pupillary response was not affected by the different pupillary baselines manipulated by the luminance. Our results suggest that mental effort manipulated pupillary baseline interacts with the subsequent mental effort elicited pupillary response, but not with the luminance elicited pupillary response; the luminance-manipulated pupillary baseline interacts with the subsequent luminance-elicited pupillary response, but not with the mental effort–elicited pupillary response. 

Acknowledgments

Supported by the AI for Accessibility Program of Microsoft and JSPS KAKENHI Grant Number 21K11977. 

Commercial relationships: none. 

Corresponding author: Xiaofei Hu. 

Email: [e-mail]. 

Address: School of Psychology, Shaanxi Normal University, No. 199, South Chang'an Road, Yanta District, Xi'an 710062, China.