The pupillary light response reflects exogenous attention and inhibition of return

Abstract

Here we show that the pupillary light response reflects exogenous (involuntary) shifts of attention and inhibition of return. Participants fixated in the center of a display that was divided into a bright and a dark half. An exogenous cue attracted attention to the bright or dark side of the display. Initially, the pupil constricted when the bright, as compared to the dark, side of the display was cued, reflecting a shift of attention toward the exogenous cue. Crucially, this pattern reversed about 1 s after cue presentation. This later-occurring, relative dilation (when the bright side was cued) reflected disengagement from the previously attended location, analogous to the behavioral phenomenon of inhibition of return. Indeed, we observed a reliable correlation between “pupillary inhibition” and behavioral inhibition of return. Our results support the view that inhibition of return results from habituation to (or short-term depression of) visual input. We conclude that the pupillary light response is a complex eye movement that reflects how we selectively parse and interpret visual input.

Introduction

The pupillary light response (PLR) is a well-known physiological response that is described in most neuroanatomy textbooks. What is less well known is that the PLR is not purely reflexive, as is often assumed, but is modulated by cognitive factors. For example, we and others have recently shown that the pupil constricts when you covertly (i.e., without looking at) attend to a bright stimulus as compared to a dark stimulus (Binda, Pereverzeva, & Murray, 2013a, in press; Mathôt, van der Linden, Grainger, & Vitu, 2013, 2014; Naber, Alvarez, & Nakayama, 2013). Moreover, stimuli that subjectively appear very bright trigger a constriction, relative to equiluminant stimuli that appear less bright, e.g., pictures of the sun versus indoor scenes (Binda, Pereverzeva, & Murray, 2013b; Naber & Nakayama, 2013; see also Laeng & Endestad, 2012). Together, these and related findings show that the PLR is not merely a reflexive response to the amount of light that falls on the retina, but is modulated by the way in which we process and interpret visual input. 

To date, studies that have investigated the link between visual attention and the PLR have focused on enhancement of the PLR by endogenous (voluntary) shifts of attention (Binda et al., 2013a; Mathôt et al., 2013; Naber et al., 2013). It has not yet been investigated whether the PLR is similarly modulated by exogenous (reflexive) shifts of attention, such as elicited by suddenly appearing stimuli. Exogenous attention provides a particularly interesting case because, unlike endogenous attention (Chica, Klein, Rafal, & Hopfinger, 2010), it is characterized by a biphasic response: For a brief moment after attention is exogenously drawn to a cued location, detection and discrimination improve for stimuli presented at that location. This initial facilitation is followed by a period of inhibition of return (IOR), during which detection and discrimination are impaired for the previously attended location (Posner & Cohen, 1984). Plausibly, IOR improves the efficiency of visual search by temporarily marking locations that have received attention and do not need to be visited again (e.g., Klein, 1988). It is currently unknown whether exogenous attention is accompanied by an increased light response to attended locations and whether behavioral IOR is accompanied by a reduced light response to inhibited locations. If so, this would mean that the PLR fully tracks the dynamics of covert visual attention, be it spatial (Binda et al., 2013a; Mathôt et al., 2013; Naber et al., 2013) or feature-based (Binda et al., in press), endogenous (e.g., Mathôt et al., 2013), exogenous (the present study), facilitatory (e.g., Mathôt et al., 2013), or inhibitory (the present study). 

In addition, a link between behavioral IOR and the PLR would support the view that IOR results from habituation to (or short-term depression of) sensory input (Fecteau & Munoz, 2005; Satel, Wang, Trappenberg, & Klein, 2011). In this view, when attention is drawn to a bright stimulus, the visual system habituates to brightness. Consequently, when attention disengages, the pupil should dilate relative to when the bright stimulus would not have been attended, and no habituation to brightness would have occurred. 

In the present study, we investigated whether the PLR is modulated by exogenous shifts of attention, using a variation of the Posner-cuing task. Participants continuously fixated the center of a display that was divided into a bright and a dark half. A brief motion stimulus attracted attention to the bright or dark side. We predicted that the pupil would initially constrict when attention was drawn to the bright side, relative to the dark side, reflecting attentional facilitation. We further predicted that this pattern would reverse at longer intervals after the motion stimulus, reflecting IOR. 

Methods

Materials and availability

Data, materials, and analysis scripts are available from https://github.com/smathot/materials_for_P0009.1. 

Participants, software, and apparatus

Seventeen naive observers (nine women; age range 19–24 years) participated in the experiment. Participants provided written informed consent. The experiment was conducted with approval of the Aix-Marseille Université ethics committee. The right eye was recorded with an EyeLink 1000 (SR Research, Mississauga, ON, Canada), a video-based eye tracker sampling at 1000 Hz. Stimuli were presented on a 21-in. ViewSonic p227f CRT monitor (1024 × 768 px, 100 Hz) with OpenSesame (Mathôt, Schreij, & Theeuwes, 2012)/PsychoPy (Peirce, 2007). 

Stimuli and procedure

Before the experiment, a nine-point eye-tracker calibration was performed. At the start of the trial, the display was divided into a bright (88.5 cd/m²) and a dark (0.2 cd/m²) half, separated by a central luminance gradient (10.0° wide). Participants were instructed to fixate a blue central fixation dot throughout the trial. Two horizontally oriented Gabor patches (σ = 0.63°, sf = 0.85 c/°, 100% contrast, 30% opacity) were presented 10° to the left and right of the center. After an adaptation period of 1250 ms, an automatic one-point recalibration (“drift correction”) was performed. From this point onward, the luminance gradient was locked to horizontal gaze position by a gaze-contingent algorithm. The display was updated every 10 ms, using up-to-date gaze-position information. This retinal-stabilization procedure ascertained that gaze was always centered exactly in between the bright and dark sides and kept visual stimulation constant even when participants made fixational eye movements. After another adaptation period of 1250 ms, one Gabor patch (the cue) changed phase for 50 ms. This gave the appearance of a sudden upward motion. The cue served to capture attention but did not predict the location of the target. After a stimulus-onset asynchrony (SOA) of 100, 1000, or 2500 ms relative to cue onset, both Gabor patches changed orientation. One patch (the distractor) changed to a vertical orientation. The other patch (the target) was tilted 45° clockwise or counterclockwise from a vertical orientation. After 50 ms, both stimuli were masked with random-noise patches with the same size, average brightness, and opacity as the Gabor patches. Participants indicated the orientation of the target as quickly and accurately as possible by pressing the left (for counterclockwise) or right (for clockwise) button on a response box. The trial ended when a button was pressed or when a time-out occurred after 3000 ms. 

Cue validity (50% valid, 50% invalid), SOA (25% 100 ms, 25% 1000 ms, 50% 2500 ms), and brightness of the cued side (50% bright, 50% dark) were mixed within blocks. Because only the 2500-ms SOA provided a sufficiently long “uncontaminated” interval for the pupil-trace analysis, this SOA occurred on half of the trials and is the focus of our main analyses. The experiment consisted of one practice block, followed by 16 experimental blocks (512 trials), and lasted approximately 2 hr. 

Results

Significance and trial-exclusion criteria

For the individual differences and correlation analyses, we used a significance threshold of p < 0.05. For the linear mixed-effects (LME) analyses, we used t > 2. This is comparable to a p < 0.05 threshold (Baayen, Davidson, & Bates, 2008). However, in light of recent concerns about p value estimation for LME models, we have omitted explicit p values. For the pupil-trace analysis, we considered only sequences of at least 200 consecutive samples for which t > 2 to be significant (cf. Mathôt et al., 2013). For the correlation-trace analysis, we considered only sequences of at least 200 consecutive data points for which p < 0.05 to be significant. Trials were excluded when, at any point after cue onset and before feedback, participants fixated more than 2.9° from the left or right of the horizontal display center (4.9%). No other filtering criteria were applied. No participants were excluded. In total, 8,278 trials (95.1%) were entered into the analysis. 

Behavioral results

We conducted an LME analysis with SOA (continuous: 100, 1000, 2500 ms) and Cue Validity (valid, invalid) as fixed effects, participant as random effect on the intercept, and accuracy as dependent measure (see Figure 2). This revealed marginal main effects of Cue Validity (t = 1.71) and SOA (t = 1.51) and a reliable Cue Validity by SOA interaction (t = 2.47). The same analysis using inverse response time (1/RT) as a dependent measure (we used inversion as an objective alternative to outlier removal, cf. Ratcliff, 1993) revealed a similar pattern of results: A marginal main effect of Cue Validity (t = 1.27), a reliable main effect of SOA (t = 3.68), and a reliable Cue Validity by SOA interaction (t = 2.51). Separate analyses, as above but with only Cue Validity as fixed effect, showed facilitation at the 100 ms SOA (accuracy: t = 3.20; 1/RT: t = 8.67) and IOR at the combined 1000 and 2500 ms SOAs (accuracy: t = 2.36; 1/RT: t = 2.31). In summary, the behavioral results showed the classic biphasic pattern of facilitation at the short SOA, followed by IOR at longer SOAs. Our results also confirm previous reports that behavioral IOR is stable at long SOAs (up to 3000 ms; Samuel & Kat, 2003). 

Figure 1

 

Schematic experimental paradigm. This example demonstrates a Valid Bright-Side-Cued trial because the movement cue appeared on the bright side of the display and correctly predicted the location of the target. The exact duration of the adaptation period depended on the duration of the drift-correction procedure. The upward arrow indicates the motion direction of the cue.

Figure 2

 

Accuracy (a) and response time (b) as a function of Cue Validity and SOA. Error bars indicate standard errors. Where not shown, error bars are smaller than symbols. (b) Data points are based on the harmonic mean, consistent with the analysis described in the main text.

Pupil-trace analysis

We analyzed pupil surface during the cue-target epoch, relative to a baseline period of 100 ms prior to the cue onset (cf. Mathôt et al., 2013). Blinks were reconstructed using cubic-spline interpolation (Mathôt, 2013). No other filtering or smoothing procedures were applied. Our main analyses focus on the 2500 ms SOA, which provides a large temporal window during which pupil size can be analyzed. (See Appendix for additional analyses.) For each 1-ms sample, we separately conducted an LME with Cued Side Brightness (Bright, Dark) as fixed effect, participant as random effect on the intercept, and pupil size as dependent measure. 

The presentation of the cue triggered a fast, overall dilation, which reflects an orienting response (Wang, Boehnke, White, & Munoz, 2012). In addition, there was a slow dilation that persisted until the end of the trial and reflects steadily increasing arousal. For our purpose, pupil dilation relative to the start of the trial is not informative because it does not contain spatial information: It tells you that the participant paid attention to something but not where in space attention was directed. Therefore, we focus on the difference in pupil size between Cue-on-Bright and Cue-on-Dark trials, which does contain spatial information: This pupil-size difference reflects whether attention was (mostly) directed at the cued or the uncued side of the display and is shown as the “pupillary cuing effect” in Figure 3 (see Binda et al., 2013a; Mathôt et al., 2013, in press for a similar logic). 

Figure 3

 

Pupil size as a function of Cued Side Brightness and time since cue onset (gray and orange lines) and the difference in pupil size between Cue-on-Dark and Cue-on-Bright trials (pupillary cuing effect; blue line). Data from the 2500-ms SOA. Pupil size is shown as a proportion of baseline size during the 100 ms prior to cue onset. Error shadings indicate standard errors. Green shading indicates pupillary facilitation. Red shading indicates pupillary inhibition. Blue shadings indicate cue (0–50 ms) and target (2500–2550 ms) presentation. Data reflects the unsmoothed grand mean signal.

From 476 to 893 ms after cue onset, the pupil was smaller when the bright side, relative to the dark side, of the display was cued (from now on: pupillary facilitation). This effect peaked after 665 ms and, in absolute terms, corresponded to a relative 2.8% pupil-area decrease. This is comparable to that observed for endogenous cuing (Mathôt et al., 2013) but much smaller than the pupil-area decrease that results from direct fixation under similar conditions (Mathôt et al., in press). This pattern reversed significantly from 1054 to 1316 ms after cue onset (from now on: pupillary inhibition), reaching a relative peak pupil-area increase of 1.0% after 1126 ms. Pupillary inhibition qualitatively persisted until the end of the trial. 

Individual differences and correlation analyses

In general, there is considerable between-subject variability in IOR (e.g., Theeuwes, Mathôt, & Grainger, 2014). This was apparent in our data as well. However, rather than assume that these individual differences reflected measurement noise, we tested their reliability and used them to link pupil size to behavior. 

First, to test the reliability of the individual differences, we randomly split the data into two subsets and determined the correlation between the strength of behavioral IOR in both subsets (2500 ms SOA only; ±128 trials per participant in each subset). We repeated this procedure 10,000 times to create bootstrap estimates and 95% confidence intervals for the correlation (i.e., the split-half reliability). For behavioral IOR based on accuracy, this gave r = .71 (.49–.88). For behavioral IOR based on RTs, this gave r = .46 (.02–.79). We conducted the same analysis for pupillary inhibition, i.e., the difference in pupil size between Cued-Side-Bright and Cued-Side-Dark trials for the sample at which overall pupillary inhibition was largest (cf. Figure 4b). This gave r = .41 (.05–.72). 

Figure 4

 

(a) Correlation between pupillary inhibition and behavioral IOR (based on accuracy and RTs) for the 2500-ms SOA as a function of time since cue onset. The gray shading indicates a significant correlation based on accuracy. (b) Individual data points for the strongest correlation, observed for accuracy at 1852 ms postcue. Dots in the red area indicate participants that show both pupillary inhibition and behavioral IOR. Dots in the green area indicate participants that show (sustained) pupillary facilitation and behavioral facilitation.

Finally, we tested whether pupillary inhibition was related to behavioral IOR. We quantified behavioral IOR (for the 2500 ms SOA) for each participant and for both accuracy and RTs. We also quantified pupillary inhibition for each participant and for each 1-ms sample. Next, we determined correlations between pupillary inhibition and behavioral IOR separately for accuracy and RT and for each 1-ms sample. This analysis resulted in two “correlation traces,” shown in Figure 4. 

The correlation between pupillary inhibition and behavioral IOR based on accuracy was reliable from 817 ms postcue until target presentation, i.e., roughly during the interval at which overall pupillary inhibition was observed (Figure 3). A similar but less reliable correlation was observed between pupillary inhibition and behavioral IOR based on RTs. Correlation traces for the other SOAs are included in the Appendix. (As is visible in Figure 4b, one participant showed particularly strong behavioral IOR and pupillary inhibition. To check whether the behavior-pupil correlations were not driven by “outliers,” we conducted the same analysis but excluded, for every correlation, participants for which the behavioral and/or pupillary cuing effect deviated more than 2 SD from the mean. This analysis also revealed reliable correlations that peaked at approximately the same time.) 

To summarize, individual differences in the strength of behavioral IOR and pupillary inhibition were reliable. Moreover, participants who showed strong pupillary inhibition showed strong behavioral IOR and vice versa. 

Discussion

Here we report, for the first time, that the PLR reflects the focus of exogenous (reflexive) shifts of attention and subsequent IOR. When attention was drawn to a bright surface, the pupil initially constricted, relative to when attention was drawn to a dark surface (pupillary facilitation). This shows that the PLR is driven specifically by the brightness of attended stimuli and is not a simple reflex to retinal illumination (in line with Binda et al., 2013a, in press; Mathôt et al., 2013, in press; Naber et al., 2013). Crucially, this effect reversed after about 1 s, at which point the pupil dilated when attention had been drawn to a bright, relative to a dark, surface (pupillary inhibition). This is analogous to the behavioral phenomenon of IOR, which is the inhibitory phase that typically follows exogenous shifts of attention (Posner & Cohen, 1984). Individual differences revealed a reliable correlation between pupillary inhibition and behavioral IOR: Participants who showed pupillary inhibition also showed behavioral IOR and vice versa. 

The finding that behavioral IOR is accompanied by a reduced PLR supports the view that IOR results from habituation to visual input (or short-term depression, cf. Satel et al., 2011): When attention is exogenously drawn to a location, habituation sets in, and visual stimuli that are subsequently presented at this location (but not elsewhere) elicit a weaker neural response (Fecteau & Munoz, 2005). Behaviorally, this results in slowed responses to stimuli presented at previously attended locations. With respect to pupil size, as we show here, this results in a reduced light response to previously attended locations. 

Moreover, the present results provide a hint as to how the link between visuospatial attention and the PLR may arise. We surmise that the PLR is modulated by activity in the superior colliculus (SC), a midbrain area that is known for its role in attention and eye movements (Awh, Armstrong, & Moore, 2006). Although the SC is not traditionally considered part of the PLR pathway, Wang et al. (2012) recently showed that microstimulation to intermediate layers of the monkey SC (SCi) triggered pupillary dilation (see also Corneil & Munoz, 2014). This dilation largely reflected an orienting response (i.e., not luminance-related and thus distinct from the PLR), but one aspect of their results suggested involvement of the PLR as well: Dilation was more pronounced when monkeys viewed dark, as compared to bright, displays. One interpretation of this result is that microstimulation induced a covert shift of attention to some (unspecified) region on the display (Awh et al., 2006). Analogous to the present results, this may have triggered a pupillary constriction when the attended region was bright, relative to when it was dark. 

A link between the PLR and the SCi is also supported by the fact that neurons in this area exhibit the same biphasic pattern of facilitation and inhibition (or short-term depression, cf. Satel et al., 2011) that is observed in behavior and that we have demonstrated here using pupillometry (Fecteau & Munoz, 2005). Neurons in the nearby superficial layers of the SC (SCs) neither show this biphasic response (Fecteau & Munoz, 2005) nor trigger pupillary dilation when stimulated (Wang et al., 2012). In other words, there appears to be a highly specific link between the PLR and SCi activity. This link may play a crucial role in modulation of the PLR by factors such as spatial attention (Binda et al., 2013a; Mathôt et al., 2013; Naber et al., 2013), eye-movement preparation (Mathôt et al., in press), feature-based attention (Binda et al., in press), mental imagery (Laeng & Sulutvedt, 2014), binocular rivalry (Bárány & Halldén, 1948; Fahle, Stemmler, & Spang, 2011; Harms, 1937; Lowe & Ogle, 1966; Naber, Frassle, & Einhauser, 2011), and illusory brightness (Binda et al., 2013b; Laeng & Endestad, 2012; Naber & Nakayama, 2013). 

In summary, we have shown that, after an exogenous shift of attention, the PLR exhibits the same biphasic pattern of facilitation followed by inhibition that is observed in behavior and neural activity. This illustrates that the PLR is not merely a low-level reflex to light, but is a complex eye movement that reflects how we selectively parse and interpret visual input. 

Acknowledgments

The research leading to these results has received funding from the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/2007-2013) under REA grant agreement no. 622738. SM and JG were supported by ERC grant 230313 (http://erc.europa.eu/). ESD was supported through European Union FP7 Marie Curie ITN grant 606901. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors declare no competing financial interests. We thank Virgile Luong for his help in collecting the data. Author contributions: All authors contributed to the study design and contributed to writing of the manuscript. SM developed the experiment and analyzed the data. 

Commercial relationships: none. 

Corresponding author: Sebastiaan Mathôt. 

Email: [e-mail]. 

Address: Aix-Marseille University, CNRS, Laboratoire de Psychologie Cognitive, UMR 7290, Marseille, France. 

Appendix

Our main analyses focus on the 2500-ms SOA (Figures 3 and 4) because only this SOA provided a sufficiently long window for pupillary inhibition to emerge. For this reason, we also collected twice as many trials in the 2500-ms SOA as in the other SOAs. However, similar analyses can be performed for the 1000-ms SOA and between SOAs by cross-correlating behavior from one SOA with pupil size from another SOA. 

Pupillary cuing effect in the 1000-ms SOA

Figure 5 shows the pupillary cuing effect for the 1000-ms SOA. It is a near-perfect replication of the pupillary facilitation observed for the 2500-ms SOA (compare with Figure 3). However, the interval is too short to observe pupillary inhibition. 

Figure 5

 

Pupil size as a function of Cued Side Brightness and time since cue onset (gray and orange lines) and the difference in pupil size between Cue-on-Dark and Cue-on-Bright trials (pupillary cuing effect; blue line). Data from the 1000-ms SOA. For details, see main text and Figure 3.

Pupil size–behavior correlations

Figure 6a through d shows the cross-correlations between the behavioral cuing effect at the 1000- and 2500-ms SOAs and the pupillary cuing effect at these same SOAs. It is clear from the strikingly similar correlation traces that behavioral IOR correlates reliably with pupillary inhibition. Interestingly, behavior in the 1000-ms SOA is most strongly correlated with pupil size relatively early in the trial (at 973 ms; Figure 6c) whereas behavior in the 2500-ms SOA is most strongly correlated with pupil size near the end of the trial (at 1852 ms; Figure 6a). In other words, behavior probed at a certain moment correlates most strongly with pupil size around that same moment. 

Figure 6

 

The correlation between pupillary and behavioral cuing effects. Left column (a, c, e): pupillary data from the 2500-ms SOA; right column (b, d, f): pupillary data from the 1000-ms SOA; top row: behavioral data from 2500-ms SOA; middle row: behavioral data from 1000-ms SOA; bottom row: behavioral data from 100-ms SOA. Pupillary data from the 100-ms SOA is not shown because the cue-target interval was too short for any pupillary cuing effects to arise. For details, see main text and Figure 4.

Figure 6e and f shows the cross-correlations between the behavioral cuing effect at the 100-ms SOA and the pupillary cuing effect at the 1000- and 2500-ms SOAs. It is clear from the erratic correlation traces that attentional facilitation in behavior does not correlate reliably with pupillary facilitation. A possible explanation for this lack of a correlation is that the facilitatory effects of exogenous cuing are relative stable across individuals (unlike IOR and unlike endogenous cuing; Mathôt et al., 2013). When there are few individual differences, between-subjects correlations do not reveal anything. However, given that our sample is small, we caution against overinterpreting this null effect. 

Horizontal gaze bias

As shown in Figure 7, there was no systematic bias in horizontal gaze position in any condition. 

Figure 7

 

Mean horizontal gaze position over time as a function of Cued Side (left, right) and Cued-Side Brightness (dark, bright). Zero degrees corresponds to the display center. Positive values are toward the right. The eccentricity of the target and distractor is indicated by the dotted horizontal lines.