Assessing the relationship between pupil diameter and visuocortical activity

Abstract

Visuocortical activity and pupil diameter both increase in tasks involving memory, attention, and physiological arousal. Thus, the question arises whether pupil dilation prompts a subsequent increase in visuocortical activity. In this study, we investigated the extent to which changes in visuocortical activity relate to changes in pupil diameter. The amplitude of the sustained visuocortical response to a flickering stimulus (i.e., steady-state visually evoked potential [ssVEP] power) was examined in 39 participants while pupil diameter was measured. To generalize across stimulus conditions, Gabor stimuli varied in brightness and ssVEP driving frequency. As expected, brighter stimuli prompted pupil constriction and larger ssVEP power. To determine whether momentary fluctuations in pupil size contribute to the ssVEP amplitude under conditions of constant luminance and frequency, the single-trial means from each measure were correlated and the shape of the pupil-diameter waveform related to the ssVEP amplitude time course, both within and between participants. Under constant conditions, changes in pupil diameter were not related to changes in ssVEP amplitude, at any luminance level or driving frequency. Findings suggest that pupil dilation does not systematically prompt subsequent changes in visuocortical activity, and thus is not a sufficient cause of visuocortical modulation in cognitive or affective tasks.

Introduction

Over the course of a day, the pupil dilates and constricts to adapt to light intensity (lumens) over a range of 10 billion to one (Dowling, 1967). It is well known that as global luminance changes, the pupil constricts in a highly predictable fashion (Watson & Yellott, 2012). While pupil diameter is reliably modulated as a function of light intensity, cognitive tasks also robustly modulate pupil size. Specifically, increased working-memory load (Kahneman & Beatty, 1966), increased visual attention (Blumenfeld, Tyson, & Geng, 2013; Mathôt, Melmi, Van der Linden, & Van der Stigchel, 2015), and emotionally arousing content (Hess & Polt, 1960; Libby, Lacey, & Lacey, 1973; Aboyoun & Dabbs, 1998; Bradley, Miccoli, Escrig, & Lang, 2008; Bradley, Keil, & Lang, 2012; Bradley, Sapigao, & Lang, 2017) all reliably prompt pupil dilation. Thus, even when luminance is held constant, pupil diameter fluctuates in a meaningful fashion. 

When cognitive factors are held constant, brighter stimuli are associated with a smaller pupil diameter (Watson & Yellott, 2012; Binda & Gamlin, 2017) and increased visuocortical activity, as indicated by larger visual event-related potential components such as the P1 and N1 (Johannes, Münte, Heinze, & Mangun, 1995) and larger steady-state visually evoked potential (ssVEP) power (Regan, 1973). Thus, pupil diameter and ssVEP power have a negative relationship across luminance levels. However, when luminance is held constant, larger pupil diameter is often associated with larger visuocortical activity. That is, the amplitude of the ssVEP—a brain-electric signal primarily reflecting striate-cortex activity (Müller, Teder, & Hillyard, 1997; Di Russo et al., 2007)—also increases in response to increased working-memory load (Silberstein, Nunez, Pipingas, Harris, & Danieli, 2001), for attended relative to unattended stimuli (Morgan, Hansen, & Hillyard, 1996; Hillyard et al., 1997; Müller et al., 1998; Müller, Malinowski, Gruber, & Hillyard, 2003), and for emotional compared to neutral stimuli (Keil et al., 2003; Keil, Moratti, Sabatinelli, Bradley, & Lang, 2005; McTeague, Shumen, Wieser, Lang, & Keil, 2011; Wieser, Miskovic, & Keil, 2016). Considering the large range of conditions under which pupil diameter and visuocortical activity covary when luminance is held constant, the question arises whether these two variables are causally linked, and to what degree they cofluctuate. 

Changes in local luminance that cause transient pupil constriction are referred to as the light reflex; this is followed by a slow rebound (Barbur, 1995). Thus, increases in visuocortical activity 0.5 s after stimulus onset could reflect this rebound from the initial constriction. If additional visual input stimulates the retina due to this rebound in pupil diameter (Laughlin, 1992), increased retinal activation could propagate back to striate cortex (Doty & Grimm, 1962), amplifying the recorded ssVEP power. Supporting this notion, brighter stimuli evoke larger early visual responses such as the P1 and N1 (Johannes et al., 1995). Information from striate cortex is also thought to be involved in a feedback loop to the pupil, as there is a reduced, but not absent, pupillary light reflex in response to visual stimuli in both monkeys and humans following striatal ablations (Weiskrantz, Cowey, & Le Mare, 1998; Weiskrantz, Cowey, & Barbur, 1999). More recent findings also indicate that cortical visual processing can mediate the basic pupillary light-reflex pathway (Ebitz & Moore, 2017). Thus, especially under conditions of sustained stimulation, the constriction due to this feedback could affect the amount of visual input that subsequently reaches striate cortex. 

Given a specific, constant luminance level, momentary fluctuations in pupil size driven by cognitive tasks may affect the size of subsequently recorded visuocortical responses. The hypothesis that pupil size and visuocortical activity are directly related is supported by the notion that when the pupil dilates, more visual information stimulates the retina (Laughlin, 1992; Watson, 2013), and the retinal ganglion cells have an anatomical connection via the geniculostriate pathway to early visuocortical regions (Zihl & Von, 1982). Thus, greater retinal stimulation, due to pupil dilation, could lead to increased visuocortical activity. Because dilation of the pupil increases retinal stimulus, the question arises whether this stimulation leads to downstream activation in visuocortical activity. The magnitude of luminance modulation due to pupil fluctuations is typically smaller and less temporally defined than stimulus-evoked luminance changes (from a black screen to a bright stimulus). Thus, if pupil-induced changes in visuocortical processing exist, they should be most readily detectable in conditions where luminance is held constant over sustained periods of time. 

On the other hand, in terms of luminance representation, the retina, lateral geniculate nucleus, and V1 undergo divisive normalization (for a review, see Carandini & Heeger, 2011), which effectively normalizes the response to the absolute luminance intensity such that only activity reflecting local and global contrast remains. Thus, light information is completely and effectively normalized in the feed-forward pathway, giving rise to the hypothesis that pupil dilation may be unrelated to visuocortical activity. A null relationship between these two indices would suggest that the covariance between pupil size and ssVEP amplitude due to cognitive factors is not mediated by a causal relationship between pupil size and visuocortical response magnitude when luminance is held constant. 

Pupil size and electroencephalography (EEG) were therefore measured while participants viewed a series of flickering Gabor gratings. The stimulus flickered (for a duration of 3 s) at a specific frequency to elicit ssVEPs, which are defined as a stimulus-evoked oscillatory response measured at the frequency of the driving stimulus and its higher harmonics (Norcia, Appelbaum, Ales, Cottereau, & Rossion, 2015). The ssVEP originates primarily from the visual striate cortex (Di Russo et al., 2007). Critically, the power at the driving frequency can be taken as a measure of visuocortical population activity, and the envelope of the amplitude at the driving frequency can be extracted in the temporal domain (e.g., McTeague et al., 2018) to assess fluctuations in visuocortical activity across time (i.e., many seconds) with millisecond resolution. To generalize across a range of luminance levels used in studies of both pupil size and EEG, the relationship between pupil size and visuocortical activity was examined under five luminance conditions, ranging from 0.4 to 70.7 cd/m². Participants were randomly assigned to view stimuli with driving frequencies at either 6, 10, or 15 Hz (n = 13 for each frequency), to generalize across a range of driving frequencies typically used in ssVEP studies. As a method check, the relationship across brightness levels of both the pupil diameter and the ssVEP amplitude was assessed, for each frequency group, with the expectation that as brightness increases, pupil diameter should decrease (i.e., constrict) and ssVEP amplitude should increase, consistent with increased visuocortical activity. 

To determine whether momentary fluctuations in pupil diameter contribute to the ssVEP amplitude under conditions of constant luminance and stimulation frequency, three relationships were assessed: the correlation between the single-trial ssVEP power and pupil diameter, the sample cross-correlation function of each measure's time series, and between-participants tertile analysis assessing the change in overall ssVEP power with an individual's pupil-diameter change across the epoch. We expect that the pupil diameter shows a canonical time course, observed reliably in response to stimulus onset: Pupil diameter constricts within 1 s of stimulus onset (i.e., the light reflex), followed by a diameter increase later in the trial that eventually plateaus. If this diameter increase results in increased visual input reaching striate cortex, ssVEP power should increase as well during the late portion of the trial, prompting significant single-trial positive correlations between ssVEP power and pupil size. Moreover, the sample cross-correlation function of the individual time series would also show significant relationships at a particular lag, with the specific lag being indicative of the time it takes for the pupil size to affect the subsequent visuocortical activity level. Finally, if pupil diameter is related to subsequent visuocortical activity, participants with larger pupil dilation across the epoch should display greater ssVEP power increases late in the trial compared to those with smaller pupil dilation. 

Method

Participants

Thirty-nine participants (23 women, 16 men; age: range = 18–24, M = 20.0, SD = 1.6) were recruited from general-psychology courses at the University of Florida. To be included in the study a participant had to have more than 50% of trials free of artifact, as defined later. Based on this criterion, data from an additional seven participants (five women, two men) were not included in this sample. All participants reported normal or corrected-to-normal vision and no personal or family history of epilepsy or photic seizures. All participants provided informed consent in accordance with the Declaration of Helsinki and the institutional review board of the University of Florida. 

Stimuli and procedure

Stimuli were five square Gabor patches differing in luminance, presented on a 23-in. 3-D LED monitor (Samsung LS23A950) set to a vertical refresh rate of 120 Hz, which was connected to a PC (running Windows XP; stimulus presentation: Psychtoolbox in Matlab 2009). The five stimuli differed in luminance, such that the darkest part of the Gabors was held constant at the same level as the dark background (i.e., 0.01 cd/m²), while the brightest part varied across five conditions on a logarithmic RGB scale: 13, 26, 52, 104, and 208. With a luminance meter (Gossen MavoSpot), these settings resulted in empirical luminances of 0.4, 1.5, 4.9, 18.9, and 70.7 cd/m². All Gabor patches were vertically oriented, had a spatial frequency of 1.6 c/°, and occupied 8° of visual angle (parafoveal). 

Each trial consisted of a fixation cross occupying 1° of visual angle presented for 2 ± 1 s (square distribution). Fixation crosses had a mean luminance of 0.4, which matched the mean luminance of the darkest Gabor. The fixation cross was then replaced by a Gabor patch for 3 s. Both the fixation cross and all Gabor patches were shown on a black background (0.01 cd/m²), leading to consistent Michelson contrasts across all Gabors (respectively, 0.9753, 0.9931, 0.9980, 0.9995, and 0.9999). The local or Weber's contrasts were 39, 149, 489, 1,889, and 7,069, respectively. Participants then viewed a black screen for 7 s (0.01 cd/m²), to allow time for the pupil to readjust to the black screen, for an adequate baseline estimation of pupil diameter. Each participant viewed stimuli flickering at one of three driving frequencies: 6, 10, or 15 Hz (n = 13 for each frequency). Driving frequency was manipulated to control for varying ssVEP amplitude sizes at different frequencies, and to encompass a range of frequencies typically used in ssVEP studies. 

After signing the informed consent form, participants were seated in a comfortable chair in a dimly lit room. They had an EEG cap placed over their head, neck, and parts of their face, and were asked to place their head comfortably against a headrest, to ensure a constant distance from the monitor throughout the experiment. Participants were instructed to remain as still as possible and were discouraged from making head or jaw movements during the experiment. Once settled, they were told to look at any stimuli that appeared on the screen, that they would be given three short breaks (30 s) where they were free to relax and make small movements, and that the total duration of the experiment would be approximately 45 min. 

Eye-tracking acquisition and preprocessing

Participants' pupil and gaze were recorded continuously at 500 Hz throughout the experiment using an EyeLink 1000 Plus eye tracker. A 16-mm lens was used, and the eye tracker was situated such that the illuminator was parallel with, and as close as possible to, the lower edge of the visible part of the monitor. The height of the chair was adjusted such that participants could comfortably rest their head on a chin rest, adjusted such that their eyes were parallel with the upper 75% of the monitor. Pupil diameter was approximated using ellipse mode, which determines the center of the pupil by fitting an ellipse to the thresholded pupil mass. The illumination level of the infrared signal was set to 100%, and biases were adjusted based on visual inspection both for the pupil (M = 0.96, SD = 0.10) and corneal reflection (M = 1.2, SD = 0.11). Participants were comfortably seated 57 cm from the monitor throughout the experiment, which was calculated as the optimal distance for pupil estimation based on monitor size. 

For eye-tracking calibration and validation, participants were asked to fixate on a nine-point grid, which showed a white circle (5° visual angle) at nine locations around a black screen. Pupil diameter was monitored at all locations, and if lost, the lens and pupil thresholds were adjusted until the pupil could be measured at each location. For optimal pupil resolution, pupil segments were discarded if the diameter was not between 0.01 and 5 mm (400–16,000 units), resulting in a noise level of 0.2% of the diameter. Offline, pupil trials were rejected if they contained any eye movements (e.g., the participant looked offscreen) or if artifacts (e.g., blinks) obscured more than 200 sample points in the epoch. Across the 39 participants, this led to a mean of 166 trials out of 200 kept for analysis (SD = 22). The mean number of trials kept for each luminance condition was, in order from darkest to brightest stimulus, 23 (SD = 8), 31 (SD = 6), 36 (SD = 5), 38 (SD = 3), and 38 (SD = 3) out of 40. On kept trials, brief disturbances in the data (e.g., due to blinks) were interpolated using a shape-preserving piecewise cubic interpolation (Mathôt, 2013). 

EEG acquisition and preprocessing

Electrophysiological (EEG) data were recorded continuously with a 128-channel Geodesic Sensor Net (Figure 1), and impedance was kept below 40 kΩ at each sensor. All channels were digitized at a rate of 500 Hz using an online Butterworth low-pass filter with a 3-dB cutoff point at 170 Hz and a Butterworth high-pass filter with a 3-dB point at 0.05 Hz. After data collection, EEG data were digitally filtered offline using a second-order Butterworth high-pass filter with a 1-dB point at 0.01 Hz and a 23rd-order Butterworth low-pass filter with a 3-dB point at 50 Hz. 

Figure 1

 

(a) The steady-state visual evoked potential (ssVEP) averaged across all trials and an occipital electrode cluster (Oz and two nearest neighbors), for the groups of participants who viewed stimuli flickering at 6, 10, and 15 Hz (n = 13 for each frequency). Time 0 indicates the onset of a Gabor patch, which was on the screen for 3 s. (b) The power of the ssVEP signal taken from a discrete Fourier transform across the late window (1.5–3 s). (c) The topography of the ssVEP is shown on a scalp map, based on the average of all participants' principal-component-analysis weights, indicative of the topographical distribution of the ssVEP power at that driving frequency. (d) The pupil-diameter time course, averaged across all trials and participants by driving-frequency group.

To obtain epochs for each stimulus presentation, both EEG and pupil data were segmented 0.5 s before until 3.25 s after stimulus onset. EEG segments were submitted to a semiautomated artifact-detection procedure designed for multichannel electrophysiology; trials with artifacts were first detected based on the recording reference (i.e., Cz), and then global artifacts were detected using the average reference. Distinct sensors from individual trials were excluded based on the distribution of their amplitude, standard deviation, and gradient. Excluded channels in each trial were interpolated using spherical splines. Channels that fell two standard deviations above the mean across trials were globally interpolated across all time points (Junghöfer, Elbert, Tucker, & Rockstroh, 2000). Data at eliminated channels were replaced with a statistically weighted spherical spline interpolation from the full channel set (Junghöfer, Elbert, Leiderer, Berg, & Rockstroh, 1997). Using an interactive algorithm (Junghöfer et al., 2000), it was ensured via visual inspection that extrapolated channels were not clustered in a focal region of the scalp, which would render the interpolation invalid. 

Quantifying the pupil diameter on single trials

The pupil waveform was obtained in arbitrary units as standard in EyeLink 1000. To quantify the mean pupil diameter during the late window on single trials, the data were averaged for each epoch across time points from 1.5 to 3 s after stimulus onset, relative to a 0.25-s baseline. 

Quantifying the ssVEP power on single trials

To quantify visuocortical activity on the single-trial level, the spectral power of the ssVEP at the second harmonic was extracted from 1.5 to 3 s after stimulus onset. The analysis window was chosen both to allow sufficient time for entrainment and to match the window used to quantify the pupil diameter. The second harmonic was chosen because it often provides the largest signal-to-noise ratio with flickered stimuli, as observed in this data set and also discussed by Norcia et al. (2015). Given that single-trial estimates in EEG data are often noisy, two steps were taken to increase the signal-to-noise ratio before power extraction (Figure 2). 

Figure 2

 

Quantifying the ssVEP power on single trials. (a) A moving average was conducted in bins of four cycles of the frequency of interest across the late window, resulting in a mean ssVEP for each EEG sensor and participant. (b) To determine each participant's ssVEP-power topography, principal-component-analysis weights for each sensor were derived using an input matrix with 129 sensors and n time points, where n is the number of time points in the mean ssVEP window. A flat projection of the weights for an example participant is shown from a top view in the matrix. (c) Following weighting of each sensor based on the principal-component analysis, a single four-cycle ssVEP-amplitude time series defined the single-trial estimate, and then (d) the power at the frequency of interest (e.g., 30 Hz) was used as the single-trial estimate of visuocortical activity.

First, a moving-average window analysis was conducted to increase the signal-to-noise ratio at the exact frequency of interest (FOI)—that is, 12, 20, or 30 Hz—for the respective driving-frequency groups at 6, 10, and 15 Hz. This procedure increases the signal-to-noise ratio at the FOI by punishing any activity at frequencies that are not phase aligned with the exact FOI (or a multiple of the FOI—i.e., harmonics). To punish the phase of non-FOI frequencies, a window that contains exactly four cycles of the FOI is defined and then shifted and averaged many times across the duration of the late window, in steps of one cycle of the FOI. To define and shift this moving window, the four-cycle window must contain an integer number of sample points. To accomplish an integer number of sample points at all FOIs (i.e., 12, 20, and 30 Hz), the original time window was resampled to 900 Hz using a piecewise cubic spline interpolation. A moving-average window with a length of four FOI cycles was then shifted in one-cycle steps across the late window (1.5–3 s), resulting in 18, 30, and 45 shifts, respectively, for FOIs of 12, 20, and 30 Hz. This resulted in a time window containing four complete cycles of the FOI, for all sensors and participants. 

Second, to reduce the sensor dimension and further increase the signal-to-noise ratio, a spatial filter was applied to the ssVEP time series using a principal-component analysis (PCA), with sensors as the observations and time points as variables (Kambhatla & Leen, 1997; Zanotelli, Santos Filho, & Tierra-Criollo, 2010; Cohen, 2014). The weights for each sensor were obtained from the first resulting PCA coefficient for each trial, and then those weights were averaged across trials by participant, resulting in a topography of weights specific to each participant. The first component of the PCA was chosen because it explained a large majority of the variance for all participants (median = 68%; range = 48%–94%). These weights were multiplied with the moving-average time series (channels by time points), which reduced the spatial dimension, resulting in a single time series. 

After taking steps to increase the signal-to-noise ratio in both the temporal and spatial domains, we extracted the spectral power using a discrete Fourier transform (DFT). The four-cycle time series resulting from the moving-average window and the PCA entered this DFT for each frequency of interest (12, 20, and 30 Hz). Given that this time series was now sampled at 900 Hz, each segment entering the DFT had a length of 300, 180, or 120 sample points. These window lengths allowed for a frequency resolution of 3, 5, and 7.5 Hz, respectively, and thus contained exact frequency bins at the FOIs 12, 20, and 30 Hz. After the DFT, the power at only the FOI was exacted, and thus one value representing the second harmonic of the ssVEP power was obtained for each trial in the experiment, representing power between 1.5 and 3 s after stimulus onset from each participant's unique ssVEP power topography. As a method check, these single-trial power estimates were subjected to formal reliability and validity analyses, as described in Appendix A. 

Quantifying the ssVEP amplitude across time

To assess visuocortical activity across time, the time-varying envelope at the second harmonic of the respective driving frequency was calculated via the filter-Hilbert method (Wieser et al., 2016). This method allows us to quantify the ssVEP signal in a way that is specific to the viewed stimulus by isolating the amplitude of the frequency related to the flicker manipulation. To this end, the epoched EEG data were averaged by luminance condition in the time domain across trials for a posterior electrode cluster (Oz and its two nearest neighbors), representing the scalp topography where the ssVEP amplitude tends to be maximal (Figure 1). The averaged time series were then windowed using a cosine squared window with length 200 ms, and band-pass filtered using a Butterworth filter, with a bandwidth of ± 0.5 Hz (3-dB attenuation) centered around the FOI (Selesnick & Burrus, 1998). Filter orders were chosen based on the driving frequency, such that the second harmonics of the driving frequencies (12, 20, and 30 Hz) had filter orders of 12 (full width at half maximum = 442 ms), 14 (full width at half maximum = 310 ms), and 18 (full width at half maximum = 270 ms), respectively. All filters were intended to be zero phase, applied in a forward and backward direction using Matlab's filtfilt function. Butterworth filters were chosen for their favorable properties in terms of phase stability and flat filter response function in the pass band. In Matlab software, the Butterworth filters are first implemented as analog filters and then transformed to the digital domain using the bilinear transformation method. Filter quality and correctness were tested on empirical and simulated data before implementation in this study. A Hilbert transform was conducted on the band-pass-filtered data, generating an analytical phase-shifted version of the empirical time series. The envelope at each FOI was then taken by the modulus of the analytical and empirical time series for each time point. Thus, the time-varying ssVEP amplitude at the second harmonic of the driving frequency was obtained for each participant and luminance condition across the epoch. The Hilbert-based approach was chosen because of its ability to achieve favorable time resolution compared to many wavelet-based approaches or complex demodulation, which are alternative ways for finding the time-varying ssVEP envelope. 

Statistical analysis

Relating variability in pupil diameter to variability in ssVEP power

The relationship between pupil diameter and ssVEP power, within luminance condition, was assessed by three methods: correlation of the single-trial estimates of pupil diameter and ssVEP power, sample cross-correlation function of the pupil and ssVEP time series, and between-participants tertile analysis. These analyses are detailed in the Results. 

Pupil-diameter and ssVEP-power modulations due to luminance manipulations

As a method check, pupil diameter and ssVEP power were assessed as they varied across the luminance levels and frequency groups. The pupil diameter was assessed using a repeated-measures analysis of variance (ANOVA) with a five-level within-participant factor of luminance and a three-level between-participants factor of frequency. The mean ssVEP power across trials entered an ANOVA structured in an identical fashion (luminance × frequency). In all analyses, when Mauchly's test of sphericity was violated, degrees of freedom and F and p statistics were corrected by means of the Greenhouse–Geisser method (Greenhouse & Geisser, 1959). All statistical analyses were conducted in JASP (JASP Team, 2018). 

Results

Relating variability in pupil diameter to variability in ssVEP power

To assess the relationship between the amplitude of the pupil diameter and the ssVEP power across single trials when luminance was held constant, the two measures were Spearman (rank-order) correlated, separately for each luminance condition (Figure 3). To reduce effects due to between-participants variance, the single-trial values for each measure within a luminance condition were z-scored within participants before entering the correlation across the entire epoch. Results indicated that in none of the five luminance conditions were ssVEP amplitude and pupil diameter significantly correlated; from darkest to brightest luminance conditions: r = 0.01, p = 0.80; r = 0.03, p = 0.31; r = 0.02, p = 0.64; r = 0.04, p = 0.14; r = 0.01, p = 0.77. To assess the confidence of these null results, a Bayesian Kendall's tau-b correlation with each model (H₀ and H₁) containing an equal prior probability of 0.5 was conducted on each luminance level; from darkest to brightest: τ = 0.008, BF₀₁ = 17.8; τ = 0.023, BF₀₁ = 13.6; τ = 0.01, BF₀₁ = 21.4; τ = 0.030, BF₀₁ = 8.5; τ = 0.006, BF₀₁ = 24.4. 

Figure 3

 

Single-trial pupil-diameter and ssVEP-power means taken from the late window (1.5–3 s), overlaid for the five luminance conditions.

Next, a sample cross-correlation function analysis assessed the degree of similarity between the pupil-diameter time series and the time-varying ssVEP amplitude, while adjusting for potential temporal displacement. For each participant and luminance condition, the pupil-diameter time series and the time-varying ssVEP amplitude were correlated across the epoch using the rank-based nonparametric Spearman's rho coefficient. The correlation function was obtained across a sequence of lags between the two time series: The first lag started with both the pupil-diameter and ssVEP-amplitude time series beginning at 0.5—that is, the initiation of the light reflex. Each subsequent lag was obtained by shifting the pupil-diameter window forward in steps of 2 ms and obtaining a correlation coefficient at each lag. The final lag (2.5 s) was the last time point in the epoch window minus 0.5 s, to avoid correlating time courses with too few points (fewer than 250 points), which would lead to unreliable estimates of the Spearman's rho coefficients. Thus, each consecutive correlation was conducted on a time series of length 1,000 sample points (sampled at 500 Hz) minus i lags. The decreasing length of the time segments was corrected for by adjusting the significance threshold for each correlation, via the inverse Student's t (Zar, 1972). This sample cross-correlation analysis assesses any relationship between pupil-diameter changes and modulations in subsequent visuocortical activity at different time intervals between the two metrics. 

The mean sample cross-correlation of the time series across a late window for pupil-diameter changes and ssVEP power did not reach the significance at any lag (Figure 4). 

Figure 4

 

The mean (black) and bootstrapped standard deviation (gray error bars) taken from the sample cross-correlation function coefficients obtained by correlating the pupil-diameter time series with the ssVEP-amplitude time series, shifted across the epoch to adjust for any potential temporal displacement. The correlation does not reach the significance threshold (shown in green) at any lag. The sample cross-correlation equation is included, such that a represents the pupil-diameter time course and b represents the ssVEP-amplitude time course, which are cross-correlated from time index t from iteration i to n.

Next, a between-participants analysis assessed whether there were individual differences in the temporal relationship between pupil diameter and ssVEP (Figure 5a). Participants were divided into three groups based on amplitude of pupil change (Figure 5c). The first tertile consisted of participants with a large pupil-diameter change after the light reflex (i.e., largest rebound), and the third tertile consisted of participants with the smallest pupil-diameter change after the light reflex (i.e., smallest rebound). This difference was calculated as the mean change from the average light-reflex response (0.75–1.5 s) to the last second of the trial (2–3 s). The time-varying ssVEP amplitudes from the Hilbert analysis were then averaged according to these tertile groups to assess whether any late ssVEP-amplitude changes occurred in participants with larger pupil-diameter changes across a trial. The late window (1.5–3 s) of the ssVEP-amplitude time series was then scored by averaging across time. These ssVEP-amplitude means entered a one-way ANOVA that contained three levels (pupil-based tertiles). This ANOVA was not significant, F(2, 36) = 0.011, p = 0.99. A Bayesian ANOVA with the same factors was conducted with each model (H₀ and H₁) at an equal prior probability of 0.5, resulting in a BF₀₁ of 5.5, which suggests that the null hypothesis was 5.5 times more likely to be observed (Wagenmakers, Wetzels, Borsboom, & Van Der Maas, 2011). The means for these three tertiles, from smallest to largest pupil difference, were 0.44 (SD = 0.3), 0.45 (SD = 0.4), and 0.43 (SD = 0.4), as shown in Figure 5b. 

Figure 5

 

Tertile analysis. (a) The data from each averaged tertile group are shown across the epoch, with the pupil diameter shown in red and the ssVEP amplitude shown in blue. (b) The mean ssVEP amplitude is shown during the late window of the trial for each pupil-based tertile group. Each dot represents one participant. (c) The difference score of the pupil is shown for each of the three tertiles to illustrate that the division sufficiently resulted in significantly different groups of participants based on the difference from the light reflex to the last second of the trial.

Pupil-diameter and ssVEP-power modulations by luminance manipulations

As a method check, pupil diameter (Figure 6a) and ssVEP power (Figure 6b) were assessed across luminance conditions and between-participants frequency groups. A two-way repeated-measures ANOVA was conducted on the pupil-diameter means across the late-window. Mauchly's test of sphericity indicated a violation of sphericity for the luminance factor, χ²(4) = 0.23, p < 0.001, and thus the reported statistics are adjusted using the Greenhouse–Geisser estimates of sphericity (ε = 0.30). Results show a main effect of luminance, F(1.2, 43.4) = 91.06, p < 0.001, η_p² = 0.72; no main effect of frequency, F(2, 36) = 1.53, p = 0.23, η_p² = 0.08; and no interaction, F(2.4, 43.4) = 0.53, p = 0.53, η_p² = 0.04. Follow-up paired t tests suggest that the pupil constricted monotonically as a function of luminance (the following conditions are numbered from darkest [first] to brightest [fifth])—first versus second: t(38) = 6.51, p < 0.001, d = 0.57; second versus third: t(38) = 7.28, p < 0.001, d = 0.85; third versus fourth: t(38) = 9.86, p < 0.001, d = 0.75; fourth versus fifth: t(38) = 8.43, p < 0.001, d = 0.68. 

Figure 6

 

(a) The mean pupil diameter in arbitrary units averaged across all trials and participants (n = 39), for the five luminance conditions from darkest, 1, to brightest, 5. (b) The mean ssVEP power in microvolts taken from a Hilbert analysis, averaged across all trials and participants (n = 39).

An ANOVA was conducted on the late-window ssVEP-power means. Mauchly's test for sphericity was violated for the luminance factor, χ²(4) = 0.23, p < 0.001, and thus the reported statistics are adjusted using the Greenhouse–Geisser estimates of sphericity (ε = 0.64). The results show a main effect of luminance, F(2.6, 91.8) = 27.27, p < 0.001, η_p² = 0.43; a main effect of frequency, F(2, 36) = 14.29, p < 0.001, η_p² = 0.46; and no interaction, F(5.1, 91.8) = 1.57, p = 0.18, η_p² = 0.08. The main effect of frequency followed a 1/f distribution, such that higher frequencies were associated with proportionately less power, as expected. Follow-up paired t tests (one tailed) on the luminance conditions were all either significant or trending, which suggests that ssVEP power monotonically increased as a function of luminance (the following conditions are numbered from darkest [first] to brightest [fifth])—first versus second: t(38) = −1.44, p = 0.079, d = 0.23; second versus third: t(38) = −4.38, p < 0.001, d = 0.70; third versus fourth: t(38) = −1.97, p = 0.023, d = 0.32; fourth versus fifth: t(38) = −4.28, p < 0.001, d = 0.69. 

Discussion

Three analyses converged on the conclusion that when luminance is constant, fluctuations in pupil diameter were not related to changes in visuocortical activity, as measured by ssVEP power. First, no relationship was found between the single-trial pupil diameter and ssVEP power. Second, the shape of the temporal waveforms for pupil-diameter changes and ssVEP amplitude showed nonsignificant correlation coefficients at all lags. Third, there was no evidence that the shape of the pupil-diameter waveform was associated with differences in the time-varying ssVEP-amplitude waveform in a between-participants analysis. These findings were replicated across five within-participant luminance levels and across three between-participants driving-frequency groups. On the other hand, the method check confirmed that brighter Gabor patches prompted a reliable increase in ssVEP power and smaller pupil diameter. 

If pupil dilation results in enhanced visual input and associated downstream visuocortical activity in early visuocortical areas such as V1 or V2, this should be observed in a relationship in which larger pupil diameter is associated with larger ssVEP power. Our findings did not support this prediction. Importantly, the single-trial pupil-diameter and ssVEP-power variance observed here is in a range applicable to studies of cognition and emotion using these same measures. Considering the variety of studies that measure ssVEP power, the present work is critical in establishing that these early visuocortical increases are not solely an artifact of the pupil dilation in response to task-relevant stimulation. The data are consistent with animal models of divisive normalization (Wagenmakers et al., 2011), which demonstrate that luminance is normalized at both the retinal and lateral geniculate modules before reaching early visuocortical regions such as V1 and V2, and therefore representing contrast rather than overall intensity. 

In comparing pupil diameter and ssVEP amplitude across the five luminance levels, a negative relationship was observed, as expected. Thus, as luminance increased, the pupil constricted and a robust increase in ssVEP amplitude was observed across the epoch. The ssVEP-amplitude increase with luminance level is most likely due to increased local (i.e., Weber) contrast (Hadjipapas, Lowet, Roberts, Peter, & De Weerd, 2015) rather than overall light intensity (Carandini & Heeger, 2011), despite the global (i.e., Michelson) contrast remaining at near ceiling across all conditions. Previously, stimulus luminance differences modulating the local contrast of a stimulus have been observed to modulate early event-related potential components such as the P1 and N1 (Johannes et al., 1995), which originate in early visuocortical regions (striate and extrastriate cortex; Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2002). While the P1 and N1 represent early, transient visuocortical activity in response to a stimulus, the ssVEP amplitude provides a sustained representation of visuocortical activity for the duration of the stimulus (Norcia et al., 2015), and thus suggests that this increased synchronization in response to stimuli with higher local contrast is sustained throughout the viewing period. 

Although pupil size has been reported to modulate the striate response in a previous study (Bombeke, Duthoo, Mueller, Hopf, & Boehler, 2016), as measured by the C1 amplitude of the visual event-related potential, those authors note that local contrast changes could not be ruled out as a potential mechanism. Our results support this interpretation, because both pupil diameter and the amplitude of ssVEP power were highly related to local contrast changes across the five luminance conditions but were not related within luminance conditions when local contrast was held constant. This variable is important to consider when designing studies that present complex visual information, such as scenes or videos, given that the local contrast of each stimulus is more difficult to control than the overall luminance level of the stimulus, although a recent study has described one solution to this problem for static scenes (Bradley et al., 2017). 

In conclusion, our results are consistent with a hypothesis that divisive normalization sufficiently controls for variance in visual input to the retina, such that these changes are no longer represented at the level of the striate and extrastriate cortex. That is, with luminance held constant, increased ssVEP amplitude is not related to pupil diameter. These data suggest that visuocortical activity modulated by cognitive factors such as memory, attention, or emotion is not solely an artifact of pupil dilation prompted by these cognitive manipulations. 

Acknowledgments

This work was supported by the National Institutes of Health, grant 1F31 MH11434-01A1 to NT and Grants R01 MH112558 and R01 MH097320 to AK. This work was also supported by the Office of Naval Research, grant ONR N00014-18-1-2306 to AK. The authors would like to thank Austin Spitz and Clever Nguyen for their help with data collection. 

Commercial relationships: none. 

Corresponding author: Nina N. Thigpen. 

Email: . 

Address: Center for the Study of Emotion & Attention, Department of Psychology, University of Florida, Gainesville, FL, USA. 

Appendix A: Reliability and validity of the single-trial ssVEP quantification

For the single-trial correlation analysis between measures, we extracted a single-trial estimate of the ssVEP power on each trial. Given that single-trial estimates of EEG data are sometimes too noisy to be meaningful, we compared a standard method of extracting power—that is, obtained power from a Fourier analysis on the late window (1.5–3 s) on an Oz cluster (Oz and its two nearest neighbors)—with a method of extracting power designed for single-trial ssVEP data—that is, a cycle-based moving-average procedure to heighten the signal-to-noise ratio of the ssVEP and a principal-component analysis to reduce the space dimension, before conducting the fast Fourier transform to obtain the ssVEP power on single trials (as described in Keil et al., 2008). 

To examine the reliability of these estimates, we conducted an internal consistency analysis using Cronbach's alpha (Cronbach, 1951; Thigpen, Kappenman, & Keil, 2017). The single-trial estimates served as items and the participants as observations. To control for the varying number of trials kept for different participants, we chose the lowest number of trials kept in any participant or luminance condition (8) and conducted the analysis using that many randomly selected trials. This random selection was conducted 10,000 times, and the median Cronbach's alpha coefficient and its 95% confidence interval were obtained across the iterations separately for each luminance condition. 

In order to discern the degree to which the ssVEP amplitude in response to different luminance levels produces meaningful, robust modulations to the recorded signal, the validity of the signal was assessed. To assess the validity of the single-trial quantification method of the ssVEP amplitude used here, a linear discriminant analysis was conducted. The degree to which the luminance conditions could be discriminated across the participants was assessed, using the single-trial estimates as features. A chi-square test was used to test the discrimination between luminance conditions. 

Reliability and validity of the single-trial ssVEP quantification

The reliability analysis resulted in nonreliable values for the standard method and reliable values for the single-trial method (Figure A1a). Specifically, the standard-method reliability test resulted in median Cronbach's alpha levels below the acceptable threshold (0.7) for all luminance conditions: 0.67, 0.55, 0.43, 0.56, 0.49, from darkest to brightest. The single-trial-method reliability test resulted in median Cronbach's alpha levels above the acceptable threshold for all luminance conditions: 0.84, 0.91, 0.94, 0.97, 0.96, from darkest to brightest. 

Figure A1

 

Reliability and validity. (a) Comparing the reliability of the standard method versus the single-trial method for quantifying the ssVEP power. Cronbach's alpha demonstrated that the standard method was not sufficient to achieve acceptable reliability (0.7, as shown in green), while the single-trial method was reliable, in all conditions. (b) The single-trial values used as features in a linear discriminant analysis, which demonstrated sufficient discrimination across the five luminance levels, suggesting the method is valid.

The validity analysis resulted in better discrimination for the single-trial method compared to the standard method. While the chi-square test was significant for both the standard method, χ²(4, N = 39) = 119.97, p < 0.001, and the single-trial method, χ²(4, N = 39) = 136.55, p < 0.001, the single-trial method had overall better accuracy (52%) compared to the standard method (50%).