Probing the temporal dynamics of the exploration–exploitation dilemma of eye movements

Abstract

When scanning a visual scene, we are in a constant decision process regarding whether to further exploit the information content at the current fixation or to go on and explore the scene. The balance of these two processes determines the distribution of fixation durations. Using a gaze-contingent paradigm, we experimentally interrupt this process to probe its state. Here, we developed a guided-viewing task where only a single 3° aperture of an image (“bubble”) is displayed. Subjects had to fixate the bubble for an experimentally controlled time (forced fixation time). Then, the previously fixated bubble disappeared, and one to five bubbles emerged at different locations. The subjects freely selected one of these by performing a saccade toward it. By repeating this procedure, the subjects explored the image. We modeled the resulting saccadic reaction times (choice times) from bubble offset to saccade onset using a Bayesian linear mixed model. We observed an exponential decay between the forced fixation time and the choice time: Short fixation durations elicited longer choice times. In trials with multiple bubbles, the choice time increased monotonically with the number of possible future targets. Additionally, we found only weak influences of the saccade amplitude, low-level stimulus properties, and saccade angle on the choice times. The exponential decay of the choice times suggests that the sampling and processing of the current stimulus were exhausted for long fixation durations, biasing toward faster exploration. This observation also shows that the decision process took into account processing demands at the current fixation location.

Introduction

Decisions are a central aspect of cognition. Because the world is noisy, evidence of the states of the world needs to be integrated over time (Vickers, 1970). Indeed, physiological studies using the random dot motion paradigm (Gold & Shadlen, 2007; Shadlen, Britten, Newsome, & Movshon, 1996; Shadlen & Newsome, 2001; Smith & Ratcliff, 2004) suggest such an evidence accumulation process. Furthermore, several other decision processes comply with such models. Examples are categorization (Heekeren, Marrett, Bandettini, & Ungerleider, 2004), eye movements (Leach & Carpenter, 2001), or self-initiated button presses (Schurger, Sitt, & Dehaene, 2012). Most commonly, these decision processes are modeled by using a biologically plausible drift-diffusion process (Ratcliff, 2001). The properties and neurobiological mechanisms of decisions are clearly a vast and active research field (Heekeren, Marrett, & Ungerleider, 2008). 

While scanning a scene with eye movements, we need to decide when to move our eyes and what target to select for every single saccade. This is arguably the decision with which we are confronted the most often throughout our lives. Eye movements result in significant changes in signals to the brain, and they influence our conscious perception. The obvious way in which they influence our conscious perception is by changing the visual input, but also, more subtly, by the decision about what part of an environment to sample in the future (Kietzmann & König, 2015). A much-investigated example of this decision process can be found in the act of reading: To read this text, you continuously select the target of the next saccade. Extensive data and models have been published in this domain alone (Rayner, 1998, 2009). The decision of where to look next is, of course, not restricted to reading but occurs in all viewing behaviors, e.g., visual search (Najemnik & Geisler, 2005) or free viewing. Eye movements, i.e., the selection of the next fixation points, are evidently prime examples of decision processes. 

Two primary decision processes occur in parallel: One decides when to look, the other where to look. A lot of work has been invested in understanding the selection of the next fixation location. Clearly, many different factors contribute to the decision of where to look next (Kollmorgen, Nortmann, Schröder, & König, 2010; König et al., 2016). Most notably, task-dependent factors (Buswell, 1935; Hayhoe & Ballard, 2005; Rothkopf, Ballard, & Hayhoe, 2007), stimulus dependencies (Einhäuser, Spain, & Perona, 2008; Foulsham & Underwood, 2008, 2009; Koehler, Guo, Zhang, & Eckstein, 2014), and geometric dependencies of the trajectory (Henderson & Smith, 2009; Hooge, Over, Van Wezel, & Frens, 2005; Kaspar & König, 2011b; Motter & Belky, 1998; Tatler, Baddeley, & Vincent, 2006; Tatler & Vincent, 2009) exist. For the future fixation location, some of these factors have been summarized in the saliency model (Itti, Koch, & Niebur, 1998; Koch & Ullman, 1985). In recent years, the performance of saliency models has slowly converged to the interindividual noise ceiling. In other words, models based on features become as good as predictions based on other subjects, and therefore, only interindividual differences remain to be explained (Afsari, Ossandón, & König, 2016; Bylinskii, Judd, Oliva, Torralba, & Durand, 2016; Kümmerer, Wallis, & Bethge, 2015; Wilming, Betz, Kietzmann, & König, 2011). Furthermore, the concept of a saliency map is not just a computational convenience. Studies investigating neglect patients provide evidence for the existence of a saliency map in humans (Ossandón et al., 2012), presumably in the superior colliculus (White et al., 2017). Most models estimate saliency based on low-level stimulus properties, such as luminance, contrast, motion, or edges. In addition, high-level object-related features, such as the recall frequencies of objects in a scene, predict eye locations even better (Einhäuser et al., 2008). In recent years, geometric dependencies on the trajectory have increasingly been included in these models. Spatial bias (Tatler & Vincent, 2009), saccadic momentum (Posner & Cohen, 1980; Wilming, Harst, Schmidt, & König, 2013), and horizontal asymmetries (Ossandón, Onat, & König, 2014) are incorporated not only to predict average fixation locations, but to model whole gaze paths (Schütt et al., 2017). Taken together, quite extensive literature and profound insights exist on the decision of where to look next. 

The “when” question, i.e., the decision to initiate a saccade to a new target, determines the time available for the processing of the visual information available at the current fixation. Investigating this ongoing decision process solely based on the distribution of fixation durations is difficult, as selecting a new saccade target and extracting information from the current fixation are temporally overlapping processes (Findlay & Walker, 1999; Henderson & Smith, 2009). Several clever paradigms have been established to dissociate these two factors. In gaze-contingent paradigms, such as scene onset delay or scene quality change paradigms, subjects explore an image, and at a critical fixation, the image is temporarily exchanged with a mask (Henderson & Pierce, 2008; Henderson & Smith, 2009), or visually altered (Henderson, Olejarczyk, Luke, & Schmidt, 2014; Walshe & Nuthmann, 2014). These paradigms show that these changes influence the fixation duration immediately. They support the direct control theory (Gould, 1973; Rayner, 1978) of fixation durations. The direct control theory states that the processing difficulty should influence the fixation durations and thus the information content of the current fixation. A lower limit exists for how recent information can still influence the choice of what saccade to perform. The double-step paradigm (Becker & Jürgens, 1979) allows for the establishment of a minimal time of ∼80 ms when a saccade can still be reprogrammed (Findlay & Harris, 1984) depending on other factors, for example task conditions (Walshe & Nuthmann, 2015). However, not all fixations are under direct control. Henderson and Pierce (2008) found a different set of fixation durations where presumably the saccadic program could not be stopped. However, explaining this second set of fixation durations by a pure direct control theory might also be possible (Pannasch, Schulz, & Velichkovsky, 2011). These findings have been computationally modeled by using the CRISP model of fixation durations (Nuthmann, Smith, Engbert, & Henderson, 2010). CRISP consists of three major components: a random timer that initiates saccade programs, a two-stage saccade programming step, and a modulation based on the current visual processing. Nuthmann et al. show that with this model, they can model a wide range of paradigms, including the mentioned scene onset delay paradigms (Nuthmann & Henderson, 2012; Nuthmann et al., 2010). For unrestricted eye movements, the literature is a bit sparser. The most comprehensive study comes from Nuthmann (2017), where she analyzed unrestricted eye movement data associated with multiple tasks for relevant factors for fixation duration. In another recent study, Einhäuser and Nuthmann analyzed the interactions of the “when” and “where” question in free viewing (2016). In spite of this progress, the available literature for the “when” question is lacking compared with the literature for the “where” question concerning eye movements. 

The ongoing foveal processing during a fixation is an example of an exploitation process. In contrast, the initiation of saccades and thereby the inspection of the environment exemplifies an exploration process (Gameiro, Kaspar, König, Nordholt, & König, 2017). These together establish a dilemma, quite similar to the exploration–exploitation dilemma commonly observed in other disciplines (Berger-Tal, Nathan, Meron, & Saltz, 2014; Cohen, McClure, & Yu, 2007; Daw, O'Doherty, Dayan, Dolan, & Seymour, 2006): It is a dilemma because at a given point in time, we can either exploit or explore, but not both. We are in a continuous decision process between exploring the image and exploiting the current view. It follows that the balance of these two processes determines whether we maintain the current gaze location or initiate a new saccade and consequently the distribution of fixation durations. Compared with the traditional exploration–exploitation dilemma, knowledge gain continues throughout fixation. However, the formulation of eye movements as an exploration–exploitation dilemma places the focus on the ongoing decision process. The distribution of fixation durations is the main observable outcome of this dilemma. 

In this paper, we investigate the exploration–exploitation dilemma in a guided viewing paradigm. During free viewing (as in unrestricted viewing), experimenters usually do not have direct control over fixation durations. They can usually bias the distribution but not control it. Of course, causal interpretations require controlled experimental interventions. Therefore, the ideal experimental conditions would allow us to control these factors by discretizing the temporal and spatial aspects of eye viewing behavior. Such a paradigm should allow one to precisely modulate all parameters of interest, i.e., fixation duration, the number of the possible next fixation targets, the saliency at the current and next locations, and the geometric features of multiple saccades. In this study, we made the first step toward such a paradigm. We used guided viewing, where subjects saw only a small aperture of a scene (a bubble) at a time at each fixation (Gosselin & Schyns, 2001; Kollmorgen et al., 2010). Following this, we exchanged the current bubble by up to five different bubbles at other parts of the underlying stimulus. The subjects then selected one of these bubbles via a saccade, and the unselected bubbles were removed. The participants explored the image by repeating this process. At the end of the trial, we used a memory task as a distractor task, but also to compare the task performance to trials where the bubbles are shown concurrently and not sequentially. The main benefit of this new paradigm is that it allowed us to disentangle the processing phase from the time to select a new fixation location. 

We were interested in three predictions that the paradigm allowed us to test. First, it follows from the exploration–exploitation dilemma that after sufficient exploitation and analysis of the current view, the system will be ready to continue exploring the environment. Exploitation will finish, or saturate eventually, because information content of a single bubble is limited. If this process is stopped early, resources needed to continue exploration will still be used for exploitation, and the readiness to perform a saccade should be low. Our paradigm allowed us to probe the dilemma: The display time of the fixated bubble was experimentally controlled. With a short display time, the visual system is still in the exploitation phase, and subsequent eye movements should be generated more slowly. Vice versa, with a long display time, the visual system is ready, literally, to move on, and saccades should be elicited more quickly. 

Second, we tested whether saccade planning can be naively thought of as multiple evidence accumulators that race independently to a fixed threshold. The first to reach the threshold decided the time and place of the saccade. Multiple potential targets, and thus multiple accumulators, should result in a shorter delay to elicit a saccade. In our paradigm, we tested this by introducing an additional experimental manipulation: Instead of using the classical guided viewing paradigm, which operates with a single future target location, we allowed subjects to decide between multiple locations. 

Third, we investigated how far these processes depend on the actual information being analyzed. We, therefore, used noise and urban images with low and respectively high information content. A reasonable assumption is that urban images contain more information that needs to be exploited. Thus choice times should generally be longer. 

Summarizing, we hypothesized that longer forced fixation times on the current bubble lead to a shorter time to elicit a saccade to the next target. Furthermore, we expected a decrease in the time needed to elicit a saccade with the number of possible future targets. Finally, we hypothesized that the choice time is positively correlated with the processing demand at the current fixation location. 

Methods

Subjects

In the primary study, 35 subjects participated (18–42 years, mean age: 24 years, eight male, 27 female, three left handed, 14 left dominant eye). In a second experiment (an internal replication and a context experiment), we recorded 10 additional subjects (18–22 years, mean age: 19.5 years, one male, nine female, one left handed, one left dominant eye). 

In the first experiment, we excluded five subjects. Four sessions had to stop early on the subjects' request; one other subject did not reliably look at the bubbles. In the second experiment, three subjects were excluded from further analysis. One subject stopped early, and two other subjects had excessive errors in eye tracker calibration and drift corrections. Each subject provided written informed consent. The ethics committee of Osnabrück University approved the experiment. 

Apparatus and recording

We used a 24-in. LCD monitor (Benq XL2420T; BenQ Corp., Taipei, Taiwan, China) with a screen resolution of 1,920 × 1,080 pixels and a refresh rate of 120 Hz for presentation purposes. The participants viewed the screen from a distance of 80 cm. The participant's left eye movements were tracked at 500 Hz by using an EyeLink II system (SR Research Ltd., Mississauga, Ontario, Canada). We used a 13-point calibration with a mean validation error of <0.5° and a maximal validation error of <1°. Furthermore, we performed a drift correction before each trial. 

Procedure

In every trial, the subjects explored a given image using small apertures, inspired by the bubble technique (Gosselin & Schyns, 2001): Subjects performed a drift correction and an additional subsequent fixation on a cross-shaped fixation point for 300–700 ms. Then, a trial followed with the display of one random single bubble (a subtrial) visible for an experimentally specified time (forced fixation time; Figure 1A). In this first subtrial, we always used a single bubble. After the forced fixation time, the initial single bubble display was replaced with one to five new bubbles (one bubble in 51.6%, two in 25.8%, three in 12.9%, four in 6.5%, and five in 3.2% of fixations). This staggering was used to ensure a higher signal-to-noise ratio for the single bubbles, allowing a more certain estimation of the main effect of forced fixation time at cost of estimating the number of bubbles effect or possible interactions thereof. Out of a pool of precomputed bubbles, we randomly selected a set of new bubbles (choice bubbles), with a linear bias toward bubbles close to the current one. Thus, this procedure gave preference to bubbles that were closer to the currently displayed bubble. The subjects chose one of the new bubbles by performing a saccade toward it. The time needed to initiate the new saccade was termed the choice time and was our primary dependent variable. We used the phrase choice time instead of saccadic reaction time, as several bubbles offered a choice for the subject to make. Thus, the choice time included the time to select the new target as well as the time to initiate a saccade. During this choice time period, the current bubble was switched off; only the future bubbles were visible. After the saccade to the new target, this new target was displayed for the next forced fixation time. Concurrently with the saccade detection, the other, nonchosen bubbles were removed from display. As subjects could in principle saccade to a distant bubble passing over bubbles in between, we used an algorithm based on the last three samples to estimate the saccade velocity. After the detection of a saccade, defined by a velocity of greater than 30°/s, we checked for the closest bubble only after the velocity was lower than 50°/s again. These parameters resulted in the best time-delay/spatial accuracy tradeoff. In the analyzed data, we observed that the other choice bubbles were removed in the online algorithm after on average 11 ms (90% range of −4 ms to 23 ms) after the end of the saccades as the eye tracker detected offline. Please note that minimizing this time was no easy feat, as the eye tracker had a delay of ∼5 ms to deliver the sample. The monitor took an additional ∼5 ms to switch to 95% target luminance, and with 120 Hz, we had ∼4 ms (±4 ms) for the vertical retrace to start again. We think that an average delay of 11 ms to turn off the nonchosen peripheral stimuli is negligible. In any case, a slower reaction time had an influence only on when the other bubbles were turned off. The selected bubble was constantly displayed until its forced fixation time expired. For the analysis, we used the forced fixation time starting at the end of the saccade as the eye tracker detected offline. 

Figure 1

 

(A) Experimental Procedure. Subjects foveate a single bubble for the forced fixation time. It is sampled from an exponential distribution with a mean of 300 ms to ensure a flat hazard function. The next step removes the bubble and simultaneously displays one to five new bubbles (in this example three). The dependent variable is the time until a saccade is initiated. After the selection of one bubble, and most often before the saccade ends, the other bubbles are removed. The newly foveated stimulus is shown for a newly sampled forced fixation time. (B) Example stimuli urban and pink noise. (C) The sampling points for bubbles following the pseudorandom Poisson disc sampling algorithm. (D) A single bubble as seen by the subject.

The distribution of forced fixation times followed an exponential function with a mean of 295 ms, which resulted in a constant hazard function. Thus, at any point in time of the trial, the probability that a new bubble would appear was held constant. Consequently, subjects could not anticipate when the currently fixated bubble would disappear and a saccade was necessary. 

The subjects saw parts of the image at two times. One was the forced fixation time, when the time subjects foveated a single bubble. The second was the choice time after the forced fixation time when the subjects saw only stimuli in the periphery, and the foveated bubble was removed. The first, the summed-up display time over all fixated bubbles, was on average 6.1 s (95% quantile: [6.0 s, 6.2 s]) and was similar to the free viewing conditions used in other experiments. Thus, depending on the forced fixation times in one trial, subjects were presented on average 19.3 bubbles (95% quantile: [11, 26], SD of 4.5). The second part, i.e., when stimuli were visible only in the periphery (the summed-up choice times), took on average 3.2 s (95% quantile: [2.7 s, 3.9 s]). Taking the first part together with the second, we obtained an average of 9.3 s (95% quantile: [8.9 s, 10.0 s]) during which subjects saw the content of the stimuli. 

Stimuli

Each subject completed 128 trials in total. Ninety-six trials consisted of the guided viewing bubble paradigm; the other 32 trials belonged to a static image condition with a different research question in mind. In this static condition bubbles were not shown subsequently. They were shown simultaneously instead, and subjects were allowed to explore the bubbled-stimulus at their own pace (these trials are discussed further in the Results section under “2-AFC memory task”). If not explicitly stated otherwise, we only discuss data of the 96 trials with the guided viewing bubble paradigm. In half of all trials, we used grayscale urban images (1,280 × 960 pixels). These were photographs of Zürich city and surrounding cities and had been used in several previous studies (Wilming et al., 2011). In the other half, we used grayscale pink noise images with the same luminance distribution as the urban images (SHINE-Toolbox, Willenbockel et al., 2010). The bubbles (Figure 1B through D) were Gaussian patches with a diameter of 3°, acting as apertures on the underlying image (Kollmorgen et al., 2010). Within each bubble, a fovea filter (Loschky, McConkie, Yang, & Miller, 2005) and a Gaussian filter were applied to imitate the visual acuity of the human eye and smooth the transition to the background. The fovea filter ensured that all information of a bubble could be extracted with a single fixation at the center. That is, there was no information gained by exploring a bubble by repeated saccades to different locations within it. A Poisson disk sampling algorithm placed bubbles pseudorandomly on each image with a minimum distance constraint (Bridson, 2007). The minimum distance of two bubble centers was set to the bubble radius. Thus, the maximal area overlap of two bubbles was smaller than 15%. In contrast to a random placement from a uniform distribution, this algorithm avoided clusters or holes during sampling (Figure 1C). Successive bubbles were enforced to be without any overlap. We used on average 101 bubble locations (range 91–109) on each image. 

Subjects were instructed to look at the center of the bubbles. The distribution of the distance of fixations to the bubble center (Figure 2A) shows that subjects fixated closely to the center. The dashed green line (Figure 2B) shows the density of distances when sampling from the Gaussian visibility mask. The distance to zero reflects that if one draws two samples (x and y) from a Gaussian, it is unlikely that both of them will be close to zero. Refixations (red) lead to a distribution that supersedes this density (Figure 2B). From that, we conclude that subjects targeted the center of the bubbles and performed corrective saccades (see also Kollmorgen et al., 2010 for similar results). 

Figure 2

 

(A) 2D density plot of the number of fixations. The center is close to the origin. (B) Kernel density of distance to bubble center. Blue/Dotted: first fixation; red/solid: first refixation; green/dashed: theoretic distribution based on the Gaussian bubble mask.

2-AFC memory task

After each trial, the participants performed a 2-AFC memory task. We presented two bubbles simultaneously next to each other. One was chosen randomly from the pool of possible bubbles in the previously displayed image but was not necessarily shown during the trial. The other bubble was chosen randomly from any another image. The participants indicated via a button press which of the two bubbles could have been part of the image they saw. There was no time limit. The aim of the task was twofold: to ensure that subjects processed the image content in some depth and to use it as a motivation and distractor task. 

Data analysis

Data preprocessing

We based our analysis on the premise that subjects fixated on the displayed bubbles (n = 67,454). Thus, we removed all subtrials where they did not fixate on the bubble, where the saccade detection algorithm failed, or where the calibration was not good enough to directly detect which bubble was fixated on. We excluded subtrials, with a fixation outside of the currently displayed bubble during the forced fixation time (n = 11,390, 16% of total) and where there was no direct saccade to the next bubble but an intermediate one in-between bubbles (n = 22,613, 33% of total). Further, we excluded subtrials, where the planned forced fixation time was different from the observed forced fixation time by an arbitrary threshold of more than 40 ms (n = 6,147, 9% of total). Such a discrepancy could arise when the online saccade-detection algorithm detected the saccade earlier or later than the more sophisticated offline saccade detection. We used the observed forced fixation time in all analyses, which is the actual time in which the subjects saw the stimulus. In total, we removed 30185 subtrials. Thus, all in all, 55.3% subtrials remained (mean per subject 1,062.3, range 806–1,283). Subtrials with more than one fixation inside the currently displayed bubble (25.2% of the remaining bubbles) were kept in the analysis. A reanalysis of choice times of subtrials with only a single fixation did not change the results. 

In order to remove extensive outliers that would strongly influence our analysis, we used an outlier detection algorithm (Leys, Ley, Klein, Bernard, & Licata, 2013) based on three times the median deviation (MAD) distance from the median for each subject (with a constant factor of 1.4826 included; thus, it defaults back to σ in the normal case). This procedure removed an additional 5.6% of the subtrials and is below the 10%–15% recommended criterion (Ratcliff, 1993). 

Linear mixed model

Fixation durations and, in our case, choice times are dependent on a multitude of factors. These can be categorical (e.g., the type of scene, “urban against noise”), or in our study most often continuous (e.g., the amplitude of the previous saccade, the angle between saccades, or, in our case, the forced fixation time). Either a repeated measures ANOVA or a linear mixed model could account for the repeated measures of subjects. However, a repeated measures ANOVA does not allow for continuous factors. Thus, we choose a Bayesian linear mixed model with uniform priors on all parameters. 

The first set of predictors describes our main experimental manipulations. Two of the predictors are of primary interest in this study: the forced fixation time and the number of bubbles. The interaction between these two factors indicates a possible dependency between the underlying processes. In addition, we model the categorical stimulus type, urban images, against noise stimuli. 

The second set of effects is correlative in nature and relates to the spatial relation of the fixated bubble to the bubble fixated on in the previous and next subtrial. As we did not experimentally influence saccade trajectory, for example, by enforcing a fixed trajectory of bubbles, these spatial effects are correlative predictors and do not allow conclusions on causal relations. The angle in absolute monitor coordinates is a circular predictor. In order to model this dependency, we used a Fourier decomposition with one and two periods for the 360° of the predictors (thus, four predictors in total, including two sines and two cosines). We modeled this effect once from the previous to the current bubble and once from the current to the next bubble. Also, we used the difference of the angles between the previous and the current bubble and the angle between the current and the next bubble. We also included the distances of the bubbles. To account for a central spatial bias, we included the standardized z-transformed x- and y-position and the squared, z-transformed x- and y-position of the bubbles in order to account for a quadratic decay toward the edges of the image, which is usually observed with spatial biases. Also, we model a term to describe the distance of a bubble to the center of the screen. 

The third set of predictors incorporated temporal dependencies and sequential influences between the trials and bubble presentations. We used trial number, previous forced fixation time, and previous choice time as predictors. 

Bayesian model fit

We analyzed the data using hierarchical logistic mixed effects models fitted by the No-U-Turn-Sampler in STAN (B. Carpenter et al., 2017; Homan & Gelman, 2014). For the model specification, we followed the implementation by Sorensen, Hohenstein, and Vasishth (2016). In maximum likelihood linear modeling terms, all within-subject effects were modeled using random slopes clustered in subject and a random-intercept for subject. We estimate all covariances between the random effects with an LKJ-Prior (v = 2), which slightly emphasizes the diagonal over the off-diagonal. We used treatment coding for all categorical factors and interpret the coefficients accordingly. 

We used six MCMC-chains with 1,000 iterations each, with 50% of the iterations used for the warmup period. We visually confirmed convergence through autocorrelation functions and trace plots. Furthermore, we calculated the scale reduction factors (Gelman et al., 2013) and ensured the recommended criterion for convergence (Rhat < 1.1). To control for an adequate model fit, we calculated 1,000 posterior predictive draws and plotted the median and 95th percentile together with the raw data. The posterior predictions matched the data well, and the model seems to be adequate for our inferences. When displaying the data and posterior predictions in their quantile ranges, the posterior predictive checks showed that our model did not capture all features of the data but merely the central tendency. We find a mismatch in the 2.5% quantile and 97.5% quantile. There, the raw data have higher choice times than the posterior predictive. This results from the skewed distribution of choice times on the subject level (Figure 3). But, most importantly, the posterior predictive median and mean value fit the data appropriately. 

Figure 3

 

Kernel density estimates of choice times for each subject. The horizontal line depicts the mean of the subjects' median choice times.

Reported statistics

For Bayesian posterior predictive checks, we report 95% credible intervals (CI) using the Cousineau correction for grouped data (Cousineau, 2005). For posterior parameter estimates, we report 95% posterior CI. For other reported data, we use 95% bias-corrected and accelerated bootstrapped confidence intervals of the mean with Cousineau correction for grouped data where applicable (Cousineau, 2005). Cousineau correction adjusts confidence intervals of repeated or grouped estimates by first subtracting the total average of each subject before calculating the variance. 

Software

Experimental software was written using PyGame (http://pygame.org). We processed data in MATLAB (MathWorks, Natick, MA), Python (http://python.org), and R (R Core Team, 2013). We acknowledge the use of other packages used to analyze the data (Bates, 2010; Carr, Lewin-Koh, & Maechler, 2015; Pinheiro, Bates, DebRoy, Sarkar, & R-Core-Team, 2016; van der Walt, Colbert, & Varoquaux, 2011; Wickham, 2007, 2015). All experimental scripts, analysis scripts, and data are publically available under http://osf.io/ba2pn. 

Results

In this study, we use a new paradigm to break down the free viewing paradigm into controlled subprocesses. Whereas several experimental paradigms exist to experimentally manipulate fixation durations, here, we directly control them using forced fixation periods. Furthermore, we control the participants' saccade trajectories with the use of small bubble-like stimuli that act as an aperture on the underlying image. These bubble stimuli allow us to guide the participants in their exploration. 

Thirty-five subjects viewed 96 images each (plus 32 images intermixed with other trials; see Results, 2-AFC memory task, for a description), and we analyze 35,105 fixations based on the guided viewing paradigm. The mean of the median choice time, that is, the duration until subjects initiated the next saccade, was 156.7 ms (95th percentile range: [123.1 ms, 210.3 ms]; Figure 3). As expected for fixation durations and reaction times, our choice time distributions are skewed toward the right tail. We used a Bayesian linear mixed model to explain the variations of the choice time. The units of the linear model parameters were chosen to be intuitive and to facilitate comparison between the effect sizes. We either used the average values of the predictors (246 ms based on the average fixation duration in parts of a big free viewing data set (Wilming et al., 2017)) or maximal values (the total number of trials, 96 from this experiment plus 32 intermixed ones, resulting in 128 trials and 19 bubbles per trial average). We additionally grouped the predictors into three distinct classes: experimental factors, correlative factors, and sequential effects. Experimental factors are controlled, randomized, and balanced. Correlative factors were analyzed based on the subjects' behavior. The sequential effects capture possible influences due to previous trials. We present the posterior marginal density results in Figure 4. 

Figure 4

 

Linear mixed model parameter estimates and Bayesian 95% posterior credibility intervals. We state the parameters in intuitive units (third column) to aid interpretation. A “max” depicts the highest observed value of this factors, and “avg” the average of independent data (Wilming et al., 2017).

Forced fixation time

Our first prediction relates to the exploration–exploitation dilemma: Sampling the decision process early after fixation onset should result in longer reaction times than eliciting a new saccade after prolonged initial fixation. All subjects show an exponential decline in the choice time (Figure 5A and B). For short forced fixation times below 100 ms, we observe choice times of around 175 ms. In contrast, in the case of long forced fixation times, choice times saturate around 150 ms. This compressive nonlinearity has an instantaneous rate of growth of −34.0 [−38.3, −29.6] (95% credibility interval; see Methods) times the forced fixation time normalized by an average fixation duration in free viewing images of 246 ms. We find an additional linear effect of forced fixation time with a slope of 0.7 [−0.1, 1.5]. This effect is small (a change of 0.7 ms in 246 ms, or a ∼3 ms change in 1,000 ms) and the parameter estimate contains 0. Therefore, we do not discuss the linear part of the forced fixation time effect further. Descriptively, we observe an increase in the choice times for forced fixation times larger than 1s (not shown). Due to the exponentially distributed forced fixation times, on average, only 2.4% of the forced fixation times are larger than 1 s. Therefore, we do not feel confident in making estimates for values larger than 1s, which are unlikely to influence the model estimates. From the mixed model results, it is clear that the forced fixation time increases the reaction time: The longer the current fixation lasted, the faster subjects responded. 

Figure 5

 

Main effect of forced fixation time. Subjects viewed the bubbles for an experimentally controlled time, depicted on the abscissa. We record the time until they choose to perform a saccade to the next bubble on the ordinate. (A) Blue: data with bootstrapped 95% CI of the mean using Cousineau correction. Red: posterior predictive distribution with 95% credibility interval. (B) Single subject kernel density estimates of the data.

Number of bubbles

The second experimental manipulation is the number of bubbles. After fixating a stimulus for the forced fixation time, the stimulus disappeared and one to five bubbles emerged in the periphery. The subjects then decided on one and performed a saccade onto it. The other bubbles then disappeared before fixation onset. Here, we observed a monotonic logarithmical increase of the choice time from one to five bubbles (Figure 6), with an effect size of 25.1 ms [21.0 ms, 29.2 ms] over five bubbles. While the logarithmical effect captured the data well, a categorical or piecewise-linear component (e.g., one bubble is unique, 2 to 5 bubbles are linear) seemed to capture the data equally well. Theoretic considerations for a possibly higher number of bubbles in future experiments make the logarithm the most reasonable choice with the best generalization. It is evident from the data that there is a monotonic increase in the choice time: The more bubbles that are available for a decision, the longer it takes for one to choose a bubble. 

Figure 6

 

We varied the number of bubbles from which subjects could choose. (A) Main effect number of bubbles: blue: data with bootstrapped 95% CI of the mean using Cousineau correction; red: posterior predictive distribution with 95% credibility interval. (B) Distributions for individual subjects. Each subject showed an increase over the number of bubbles.

Interaction of forced fixation time and number of bubbles

Next, we analyze the interaction between the forced fixation time and the number of bubbles (Figure 7) on the choice time. Here we find an interaction between the number of bubbles and the forced fixation time of −14.7 ms [−19.0 ms, −10.5 ms]. In Figure 7, we smoothed the display to get a more intuitive understanding of this relationship. An example to understand the effect size for this log-log interaction is helpful. The predicted difference between a forced fixation time of 100 ms and 1,000 ms for a single bubble is 14.7 ms; for five bubbles, it is 20.0 ms. Thus, in this case, the interaction boosts the single bubble forced fixation effect by 40%. We acknowledge the small absolute effect size. Nevertheless, the credibility interval for this effect is far removed from 0. The interaction shows that the effects of the forced fixation and the number of bubbles on the choice time are dependent on each other: The more bubbles are available to choose from, the larger the effect of the current forced fixation time. 

Figure 7

 

(A) The main effects of the number of bubbles (columns) and the forced fixation time (abscissa). The upper row depicts raw data with 95% bootstrapped mean with Cousineau correction, and the lower row depicts the posterior predictive with 95% Cousineau credibility intervals (B) We show the interaction between the forced fixation time and the number of bubbles. The data from (A), lower row, were further smoothed for this plot (generalized additive model with thin plate regression splines), and the mean of each curve was removed. This eliminates the main effect of the number of bubbles. A sharper decline results in trials with many bubbles to choose from compared to trials with a low number of bubbles.

Image category

The last experimental effect is the image category. The bubbles of one trial were drawn from urban images in half of the trials and pink-noise images in the other half of the trials. We observed a main effect of 4.3 ms [3.1 ms, 5.5 ms], which means urban bubbles have a longer choice time than noise bubbles. This main effect is surprisingly small, given the vast differences in image statistics between the two categories. 

Geometric effects

We observed various correlative effects (Figure 8). These effects were not experimentally controlled or balanced and thus are correlative in nature. The first effect is the angle between the currently fixated bubble and the next bubble (Figure 8A, inset). We modeled this effect using four parameters describing the sine and cosine of the base and two times the base frequency. This procedure allowed us to capture the dynamics of the circular nature of this effect (Figure 8A). The shape of the marginal effect display follows a smooth sinusoidal curve: Reaction times of saccades toward the upper part of the screen seem to be faster than toward the lower part of the display. But if we look at the model estimate, the maximal average difference (the amplitude), and thus the effect sizes of this summed curve, is only 0.9 ms [0.4 ms, 1.5 ms]. The mismatch between the marginal display and the parameter effect size is due to other predictors explaining a large share of the variance of the effect. We observed a similarly small effect of the angle between the previously fixated bubble and the currently fixated bubble on the decision process at the currently fixated bubble. Here the maximal difference over angles is 0.7 ms [0.3 ms, 1.1 ms]. This effect follows a double-u shape, with increased choice times for vertical saccades and faster choice times for horizontal saccades. 

Figure 8

 

Geometric predictors. (A) Angle to the next bubble. (B) Difference in angle between two bubbles, over termed saccadic momentum. (C) Distance to the next bubble. (D–E) Horizontal and vertical position of the bubble that has been selected (not the current foveated one); the distance of the selected bubble to the center of the image. (C–F) Degrees depict visual angle.

The difference between the two angles is shown in Figure 8B and is commonly referred to as saccadic momentum. In line with other research (Wilming et al., 2013), return saccades appear slower than forward saccades, but in our study, this is visible only in the marginal plot. Taking into account other factors, the effect disappears. Whereas in free viewing Wilming et al. observed effect sizes of around 45 ms, here we did not see a reliable effect, with on average 0.4 ms [−0.9 ms, 1.7 ms] for the maximal difference of 0° to 180° (see Discussion). The logarithmic distance to the next bubble (Figure 8C) predicted the choice time with a maximal effect size of 2.6 ms [0.0 ms, 5.3 ms] between a theoretical saccade of 0.2° and the maximum of 30°. Similarly, it is unlikely that there exists an effect of the distance to the last bubble, with an estimate of −1.3 ms [−3.3 ms, 0.9 ms] for the maximal range. 

Next, we discuss the predictors for the absolute position of fixation. Without these predictors, the aforementioned nonsignificant effects all were significant (although small, model results not shown here). We modeled the absolute position of the chosen (thus future) bubble using a linear and a quadratic term to capture the symmetric and quadratic nature of this spatial bias. The horizontal position had a linear influence of −3 ms [−4.6 ms, −1.5 ms] for the maximal range. The vertical position had a bigger effect of −8 ms [−10.0 ms, −5.9 ms] for the maximal range (which in absolute measures is smaller than the horizontal position). We also observed quadratic effects for the horizontal position of −7.8 ms [−12.8 ms, −2.9 ms] and for the vertical position of −9.0 ms [−12.5 ms, −5.7 ms]. In addition to the absolute position, we also observed a linear distance-to-center effect of −5.1 ms [−8.0 ms, −1.8 ms]. These effects indicate a quicker reaction time the farther away from the center one's eye rest. 

Sequential effects

The last effects we modeled are sequential effects: trial number, bubble number, previous choice time, and previous forced fixation time. The trial number influenced the choice time by 2.9 ms [0.9 ms, 4.8 ms] throughout the 128 trials of the experiment (Figure 9A). The bubble number describes the nth bubble a subject saw in each trial. It had, on average, an influence of 2.5 ms [1.5 ms, 3.5 ms] (Figure 9B) throughout one image, i.e., on average over 19 bubbles. The previous choice time had an influence of −1.9 ms [−3.0 ms, −0.8 ms] for an average fixation time of 246 ms. The previous forced fixation time did not seem to influence (0.01 ms [−0.2 ms, 0.3 ms]) the current choice time. All these effects are considered small. 

Figure 9

 

(A) Trial number. In total, 128 trials were used, but 32 intermixed trials were only related to the memory task. (B) On average, 19 bubbles were shown in a single trial; this effect depicts a very small increase of reaction time during one trial.

This concludes the correlative effects. Compared to the experimental effects, the effect sizes are very small. 

2-AFC memory task

Subjects performed a 2-AFC task after completing the sequence of bubbles based on one image. They viewed one bubble taken from the previously shown image (but not necessarily a bubble shown during the previous trial) and another from a different image. The most informative difference happens when two urban bubbles need to be differentiated. Here we can see whether there is a difference in performance between this sequential task and the additionally recorded static condition. 

For two noise stimuli, performance was 52.7% (CI₉₅ 49.7%–56.0%, SD: 9.4%), as to be expected at chance level. For trials with one noise bubble and one urban bubble, performance was close to perfect, with 97.4% (CI₉₅ 95.8%–98.2%, SD: 3.3%). For trials wherein two urban bubbles were compared in the memory task, subjects showed an average performance of 72.8% (CI₉₅ 69.0%–76.0%, SD: 10.6%). 

This static image condition was recorded in addition to the regular trials interleaved in the same session. In this condition, a bubbled version of the image was visible, which consisted of 21 nonoverlapping bubbles. The number was slightly higher than the on average 19 bubbles in the current study. The image was visible for 6 s, a similar time to the total viewing time in the regular trials. 16 noise and 16 urban images were shown intermixed with the sequential trials. Here we see very similar results: For two noise stimuli, performance was 53.6% (CI₉₅ 47.9%–60.6%, SD: 18.4%). For trials with one noise bubble and one urban bubble, performance was at 96.5% (CI₉₅ 93.8%–98.1%, SD: 6.2%). For trials with two urban images, performance was at 75.6% (CI₉₅ 70.7%–80.6%, SD: 15.1%). The confidence intervals for the sequential and static trials are largely overlapping. It is reasonable to conclude that performances in the two tasks do not differ and changing a static information uptake to a dynamic one does not introduce strong artefacts in the processing of information. 

Second experiment: Internal replication and context

In light of the recent replication crisis in the (cognitive) psychological sciences subjects (Aarts et al., 2015; Pashler & Wagenmakers, 2012), we strived to internally replicate our findings on an independent set of subjects. In addition, we were interested in what way context shown at the beginning of a trial influenced the choice times and integration performance. We performed a second experiment (n = 10) where we briefly flashed the entire scene for 92 ms (11 displayed frames at 120 Hz) at the beginning of half of the trials. Similar to the first experiment, subjects performed a drift correction before fixating on a fixation cross for 300–700 ms. After, the whole scene was flashed for 92 ms and then the experiment started with the first bubble (always a single bubble) at a location around the fixation spot (we used the same algorithm as in the first experiment, see Methods: Procedure). This allowed subjects to extract the gist of the scene (gist condition) but did not allow making any saccades. It has been shown that at around 100 ms, subjects already extract the main features of a scene (Potter, 1976), even when the image was masked, which we did not do here. In the other half of the trials, the experiment remained identical to the original experiment to allow for within-subject comparisons. The two types of trials were randomly intermixed. 

As can be seen in Figure 10, we replicated all effects we found in the first experiment. In addition, we did not observe an interaction with the gist of a scene in any of the factors. In the 2-AFC memory task, subjects performed at very similar levels: The case of two noise stimuli was not different from chance level (50.3%, CI_boot95: 44.5%–59.3%). One noise and one urban bubble resulted in near perfect performance (98.3%, CI_boot95: 97.1%–99.2%). And we found the same results as in the previous experiment. When subjects had to choose between two urban bubbles, there was no improvement in either gist condition (no-gist: 78.8%, CI_boot95: 70.4%–84.2%, and with-gist 70.5%, CI_boot95: 55%–79.0%). Nevertheless, the second experiment replicates all of our choice time results in a new cohort of subjects. Furthermore, briefly presenting the gist of a scene did not influence either the choice times or performance in the memory task. 

Figure 10

 

The upmost interval of each posterior predictive triplet depicts parameter estimates when subjects did not see the gist of a scene (bright color). The middle interval (dark color) shows parameter estimates when the gist was shown for 92 ms before exploration started (n = 10, paired). The lowest interval (gray) depicts the parameter estimate shown in Figure 4 (different subjects, n = 35), wherein the gist was not seen either. Showing the gist does not induce any relevant difference in the within-subjects comparison or in comparison with the cohort of the previous experiment.

Discussion

Summary

The distribution of fixation durations can be described by an exploration–exploitation decision process. Here, we used a guided viewing paradigm to control the decision processes occurring during a fixation and dissociated it from the processing of the fixated location. We found an exponential decrease of the time needed to choose the next fixation target dependent on the time available for processing the stimulus at the current fixation location. This dependence provides evidence for the exploration–exploitation dilemma in the decision process. Secondly, we found a monotonic increase in choice time with the number of available saccade locations. These data indicate that potential future saccade targets are accumulating evidence in their favor in a dependent manner. 

Exploration–exploitation

Our first main result directly tested the exploration–exploitation idea: We described eye movements as an ongoing decision process between further exploitation of the current view and further exploration of new, unseen elements. In this study, we interrupted the subjects during the exploitation stage of the current bubble at unpredictable points in time and investigated the influence on the choice time. From the literature, we expected saccadic planning times to be between 100 and 175 ms (Rayner, 1998; Schall & Thompson, 1999), which is in line with the present observation of ∼150 ms. Please note that the choice time was measured from stimulus offset. It was not measured from the previous saccade offset, as at that time, no targets for further saccades were available. Thus, the intersaccade time amounts to an average of ∼450 ms. In principle, the choice time (starting with stimulus offset and target onset) could be constant and independent of the forced fixation time, ending with stimulus offset and target onset. Instead, we observed an exponential relationship of the choice time and the forced fixation time. This demonstrates that the choice time is dependent on the degree of processing at the current fixation location and gives support for the exploration–exploitation view. 

Are there alternative explanations for the exponential decay?

In simple reaction-time experiments, Drazin (1961) observed an exponential decay when the foreperiod, the period between warning signal and go signal, was increased. This mimics our forced fixation effect and could be an alternative explanation. As discussed in Niemi and Näätänen (1981) and more recently using saccades (Oswal, Ogden, & Carpenter, 2007), the important factor influencing this effect is the predictability of the stimulus. The hazard function, the instantaneous probability that a foreperiod/forced fixation is ending, is essential here. With a nonuniform hazard function, as in Drazin, subjects can predict with higher certainty when a stimulus is going to appear, whereas when a nonpredictable, flat hazard rate is used (Baumeister & Joubert, 1969; Mowbray, 1964; Oswal et al., 2007), no effect of foreperiod on the resulting reaction time can be found. We used a uniform, thus unpredictable, hazard function. Therefore we think it is unlikely that our effect can be explained by predictability effects of reaction time. There are two limitations when comparing our study to previous studies. First, the foreperiods in previous studies are usually much longer (in the range of seconds). Second, all previous studies had a fixed, constant foreperiod. Our constant foreperiod, the saccade duration, is effectively very small. We do not think that these limitations can explain the exponential decay. 

A different potential confound is based on refixation strategies. It is possible that the instruction to look at the center of the bubble initiated a refixation program at each fixation. This could result in prolonged choice times for short forced fixation times. This prolongation should be linear with a slope of −1 ms choice time per 1 ms forced fixation time. Because we observe an exponential decay in our data, we think that this confound cannot capture our results. 

Related paradigms

A paradigm that has a very similar procedure to the current study is the mask-onset delay paradigm (Glaholt, Rayner, & Reingold, 2012; Rayner, Smith, Malcolm, & Henderson, 2009). Their goal was to measure the minimal time viewers need to see a scene at each fixation in order to obtain normal viewing behavior. In order to test that, Glaholt et al. masked the stimulus after a given time (50 ms, 75 ms, and 100 ms). This stops any further incoming information. Either the whole stimulus (Glaholt et al., 2012; Rayner et al., 2009) or a circular part around the current fixation was masked (Glaholt et al., 2012). In addition, Glaholt et al. (2012) also controlled for the sudden onset of the mask which could elicit saccadic inhibition. They measured the time until subjects started the next fixation. They found that in the full-scramble condition, subjects elicited similar saccades and reached the same performance as in free viewing only at 100 ms viewing time in either of their tasks. Our results indicate that even more time is needed (at least ∼400 ms) until the effect of forced fixation time saturates for all fixations. We speculate that the information content of the momentary view and the task (Glaholt et al., 2012) are crucial factors here. For example, in a reading experiment, Rayner and colleagues conclude that only 50–60 ms of visible stimulus is needed for fixation behavior to be indistinguishable from unrestricted information access (Rayner, Inhoff, Morrison, Slowiaczek, & Bertera, 1981). Another important factor may be local versus global masking. In Glaholt et al. (2012), scrambling only the local mask reduced this effect. Here, 50 ms was enough to extract the information to solve either task. It seems possible, though, that this task could have been solved without foveal information and only using peripheral information alone as proposed by Nuthmann (2014). Glaholt et al. (2012) also found a bimodal distribution of fixation durations, which they explain by a saccadic inhibition mechanism due to the rapid onset of the stimulus mask. Saccadic inhibition (e.g., Reingold & Stampe, 2002) is a delay of saccade production that the onset of a (possibly irrelevant) stimulus has. Its most salient feature is a bimodal distribution in the fixation durations; shortly after the inhibitory stimulus fewer saccades are generated. In our case we did not observe this bimodality in the choice times (Figure 3) or when adding the forced fixation times to the choice times (not shown). Thus, saccadic inhibition cannot explain our results. 

Another phenomenon related to the one reported here could be seen in the stimulus onset delay paradigm (Vaughan, 1982; Vaughan & Graefe, 1977). In this paradigm, subjects searched for a target at two fixation points. At each fixation there was a variable stimulus onset delay, replacing the fixation point either with a target or with a distractor. The time to elicit the next saccade can be seen mirrored in our choice time. Similarly, the stimulus onset delay is mirrored in the forced fixation time. If the stimulus was shown with a delay of 300 ms after onset, Vaughan found decreasing response times by 100–150 ms compared to immediate display. Thus, the delay speeds up the response time to the appearing stimulus. Vaughan (1982) discusses a possible explanation based on predictability, similar to the foreperiod effect discussed above. It is likely that the crucial difference between the two tasks is that in our paradigm the stimulus is directly visible at fixation onset, then subsequently vanishes and the new target appears; whereas in the stimulus onset task, the stimulus is not visible upon fixation onset but becomes visible after the forced nonstimulation period. We conclude that due to the very different effect sizes (30 ms in our study vs. 150 ms), the difference in task, and the unpredictable foreperiod in our study, it seems unlikely that the effect observed due to the stimulus onset delay is related to the exploration–exploitation effect we investigate in the present study. 

Evidence accumulation with multiple targets

Our second prediction relates to the evidence accumulation between multiple targets. We assumed one independent evidence accumulator for each target that race to a fixed threshold. The first to reach the threshold is selected. Our observed monotonic increase with the number of future target locations is incompatible with this explanation. This result is compatible with data and evidence accumulator models from Leach and Carpenter (2001). To reconcile these data with such an evidence accumulator model, interactions between the integrators of different locations are necessary. This might occur through different thresholds for multiple future locations as can be found in monkey LIP neurons that decreased drift rates through lateral inhibition due to limitations in sampling capacities (R. H. S. Carpenter & Williams, 1995; Churchland, Kiani, & Shadlen, 2008; Leach & Carpenter, 2001) or more complex interactions with time-varying thresholds (Ludwig, 2009). A future analysis step with this paradigm is to fit drift-diffusion models taking into account the effects described here. 

Van den Berg introduced two processes that are decisive for the duration of a fixation, one starting a new saccade and one staying at the current fixation. This is made explicit in visual search (Beintema, van Loon, & van den Berg, 2005; van den Berg & van Loon, 2005). In their model, both processes are explicitly modeled as two dependent integrators racing to individual thresholds that decide whether to continue exploiting the current view or go on and explore the scene. In visual search experiments, it seems as though the processing of the peripheral stimulus is secondary to the processing difficulty at fixation (Hooge & Erkelens, 1999; Wu & Kowler, 2013). The conclusion from these studies is that in visual search subjects do not bias fixation durations for better target selection, but only for discrimination of the target at hand. This is not to say that the peripheral information is ignored; it is still used for target selection, it just shows a weak influence on the fixation durations. It seems that the subsequent integration of foveal and peripheral information can occur independently (Ludwig, Davies, & Eckstein, 2014), but not always so (van den Berg & van Loon, 2005; VanRullen, Reddy, & Koch, 2004). Contrary to our initial prediction, we found only a very small effect of foveal processing in our experiment: a 4 ms difference between urban and pink noise images. On the other hand, the peripheral decision task showed a difference of up to 25 ms depending on the number of bubbles. Thus, in our experiment, it seems we have a reverse relation to observations in visual search (Hooge & Erkelens, 1999). Of course the task and general structure of the experiment are quite different. In Hooge et al., the target is available for as long as the subjects prefer, whereas in our study we forcefully interrupt the information extraction process. These results show how strong task dependency can influence the interaction and integration of peripheral and foveal information. 

We observed a logarithmic increase between the number of possible targets and reaction times, which is similar to the “Hick's”-effect (Hick, 1952; Proctor & Schneider, 2017). But there is also evidence that saccades do not follow this rule: the “anti-Hicks” effect (Lawrence, 2010; Lawrence, St John, Abrams, & Snyder, 2008). This anti-Hicks effect is commonly observed when there are multiple possible target locations to choose from. These authors differentiate between exogenous (e.g., prosaccades) and endogenous (e.g., antisaccades), where an anti-Hicks effect can be found in the former but not the later. We cannot replicate this finding here. It is well possible that selecting between multiple possible targets is an intrinsic task and our results are in agreement with earlier findings. We think there are two additional important differences: One is that subjects in our study could not predict how many bubbles would appear compared to a blocked design in Lawrence. A second difference is that in our study there is no correct stimulus, whereas in Lawrence there was always one correct target. We could reconcile this by a new experiment with a modified version of our paradigm presenting multiple bubbles but highlighting one as a target. In this case, an anti-Hicks effect could be observed. 

A functional explanation can be seen by the “cost of a saccade” (see discussion of De Vries, Azadi, & Harwood, 2016). Based on their ideas in the discussion section, a saccade has two consequences in our paradigm. For the duration of the saccade, saccadic inhibition suppresses information processing. In addition, in the case of multiple bubbles, a saccade also removes peripheral information that could have been further exploited, would the pre-saccadic fixation duration (the choice time) have been prolonged. Thus, longer choice times for trials with many bubbles would allow for longer exploitation of the peripheral content. It cannot be disentangled by this study alone whether longer choice times for more targets are based on a more difficult decision process or a more detailed peripheral information extraction of the stimuli, or both. 

From our finding of a monotonic increase followed a new prediction for free-viewing paradigms. In free viewing, some parts of an image are preferably gazed upon. This can be quantified by empirical saliency, usually measured by the density maps of fixations over multiple observers. It has been recently shown that locations with higher saliency have higher fixation durations (Einhäuser & Nuthmann, 2016). Given our findings and what has been found about the interaction of saliency and fixation durations, we predicted prolonged fixation durations if an image has multiple distinct peaks of empirical salience against other locations without these distinct peaks. The peaks in saliency mimic multiple bubbles as shown in our study. 

Why are there no stimulus content dependencies?

Contrary to our prediction, we found only a small effect of bubble content on the choice time. We think that this effect is due to chance and does not reflect a real effect. We imagine two explanations of why the effect could not be observed. It is possible that there is truly no dependency between the information extraction process and the forced fixation effect. The effect then reflects a content-independent property of the system, in a sense a hardwired, probabilistic solution to the exploration–exploitation dilemma. Alternatively the time required to analyze pink noise and urban images may be comparable. This would be in line with high fixation durations during free viewing of pink noise stimuli (Kaspar & König, 2011a). Similar rates should lead to similar choice time distributions. Further comparisons with other stimuli, or modifications of bubble-saliency, could be used to understand this better. 

Computational models of fixation duration

Models of fixation durations were proposed in the reading literature (Engbert, Nuthmann, Richter, & Kliegl, 2005; Reichle, Pollatsek, Fisher, & Rayner, 1998). In recent years, fixation duration models for free viewing and visual search have emerged as well; for example, the CRISP (Nuthmann et al., 2010) and ICAT (Trukenbrod & Engbert, 2014) models. In these two models, fixation durations are modeled by two main components, a stochastic random walk for the timing of saccades and one for the saccade generation. For the saccade generation part, the authors used either a random draw from a gamma distribution (CRISP) or another stochastic random walk (ICAT). Our data could be used to test specific model assumptions made by CRISP and ICAT, in particular the ones relating to saccade generation. 

Instead of truly generative models of fixation durations, as in the models described above, a simplified model using drift-diffusion modeling could be used. The LATER model (R. H. S. Carpenter & Williams, 1995) is a popular race-to-threshold model for (saccadic) reaction times (Noorani & Carpenter, 2016) and has recently been applied to explain both fixation durations and locations (Tatler, Brockmole, & Carpenter, 2017) in scene viewing. Similar to the previous, more advanced models, LATER does not only model the average choice times, but the whole distribution. Differences between experimental manipulations could be caused, or be hidden in the shape of the response distributions (for a Bayesian hierarchical solution based on Weibull-functions, see Rouder, Lu, Speckman, Sun, & Jiang, 2005) and not captured by the central tendency measurement. The discrete notion of exploitation (fixation on a stimulus) and exploration (which stimulus to select) can help differentiate between drift rates of saccade initiation and saccade generation. 

Comparison of other factors to free viewing

Many dependencies of fixation duration are described in the literature and consequently modeled in our study. Comparing our results to the literature, it is obvious that our observed effects are drastically smaller, and sometimes not existent. One example is the absolute angle of a saccade: Saccades from the center to above the horizon are generally observed to be faster than saccades below it (for an overview see Greene, Brown, & Dauphin, 2014). For example Heywood and Churcher (1980) reported an effect of 31 ms, and Tzelepi, Yang and Kapoula, 2005 reported 27 ms. Our effect is estimated to be around 0.8 ms and thus does not exist in practical terms. Another example is the amplitude of the previous saccade, which has been described in a search paradigm by Salthouse and Ellis (1980) with ∼90 ms for 15°. In our study, we found only 2.6 ms. A third example is an angle to the previous fixation, termed saccadic momentum or inhibition of return (Anderson, Yadav, & Carpenter, 2008; Dorris, Taylor, Klein, & Munoz, 1999; Wilming et al., 2013), which has been found to influence fixation duration with a piecewise linear relationship by approximately 70 ms per 180° (Wilming et al., 2013). In our study we did not find evidence for such an effect. Note that some effects do increase in size when not taking into account shared variance with other predictors. This is evident from the mismatch between effect size and marginal data as plotted in the figures. Still, even interpreting the marginal effects, the same qualitative judgment (smaller effects than in free viewing) persists. 

Some of the relationships described in the literature seem to be task dependent. For example, in a recent study (Nuthmann, 2017), a saccadic momentum effect could only be found for memorization and aesthetic preference but only very weak during a search task. This could partially explain the mismatch of our effect sizes. Yet another factor is that in our paradigm, peripheral information is restricted and the controlled fixation duration might have a strong influence on the oculomotor system to show the same effects that can be seen in unrestricted viewing (Nuthmann, 2014). We propose a third, nonexclusive explanation: In our study, we separate the processing of the current input from the planning and execution of the next saccade. Also, our observed choice times are quite fast. In free viewing, these two processes occur in parallel. Thus, saccadic planning can start quite early in the current fixation. Consequently, in free viewing more time is available for other processes to intervene and modify either the duration of the exploitation phase or the planning of the next saccade. To test whether the exploitation phase is modified in free viewing by these geometric factors, we propose a modification to our experiment: Instead of hiding the currently exploited bubble and stopping the exploitation process, we ought to keep it together with the new target bubbles. Thus, exploitation can continue and is partially under the subjects' control, and we expected to see effect sizes of similar size found in free viewing. 

Comparison to Nuthmann (2017)

In a recent study by Nuthmann (2017), a complementary approach to this study was used. She analyzed fixation durations from free viewing with three different tasks using linear mixed models. Of course, the paradigm differs in many ways. Nuthmann used unrestricted free viewing with different tasks, but we used the combination of the bubble and guided-viewing paradigm, a restricted viewing task. Nuthmann focused on the influence of stimulus features on the fixation durations; we merely analyzed the difference between background images based on pink noise against urban images as our focus lay on the direct control of fixation durations. Both studies analyzed oculomotor effects but have a slightly different set of predictors. In this study, we additionally included the absolute angle and the absolute position of the fixation. 

In general, Nuthmann found very strong oculomotor effects over all tasks, with some exceptions. For example, Nuthmann observed strong saccadic momentum in two tasks, but much weaker momentum in the visual search task. After controlling for other oculomotor effects, the initially observed distance-to-center effect of Nuthmann vanished. This was not the case in our study. We found an effect of distance to center even though modeling very similar other oculomotor predictors. Building upon the work of Nuthmann on feature influences, our paradigm could be used to directly check her image feature-based findings on a causal level: One can modify the low-level features of certain bubbles. For example, in a new experiment one can experimentally modify the “clutter” of certain bubbles and then expect a modified choice time. 

Alternative statistical models and model critique

We analyzed the choice times using a Bayesian linear mixed model. Linear mixed models (Bates, Mächler, Bolker, & Walker, 2014) and other hierarchical models (Gelman & Hill, 2007) are steadily replacing the need to use traditional ANOVA/ANCOVA (Baayen & Milin, 2010; Bagiella, Sloan, & Heitjan, 2000; Kliegl, Masson, & Richter, 2010; Nuthmann, 2017; Quené & Van Den Bergh, 2004; Richter, 2006). On the other hand, statistical recommendations for applied mixed models are still being developed (Barr, Levy, Scheepers, & Tily, 2013; Bates, Kliegl, Vasishth, & Baayen, 2015) and sometimes puzzling results can be observed (Hodges, 2014). One common problem is that maximum likelihood estimates fail to converge, mostly due to the complexity of the covariance matrix between random effects (Barr et al., 2013; Bates et al., 2015). Due to the high number of predictors, our model is prone to this problem. Therefore, we used a Bayesian version of the mixed model (Sorensen et al., 2016) that allows us to put a small prior on the covariance matrix that de-emphasizes correlations between random slopes. This allowed us to fit the complex model and interpret the maximal instead of a reduced (possibly parsimonious) mixed model. In the case sufficient evidence from the data for a nonzero correlation exists, this will overrule this weak prior. This contrasts with forcing correlations to be zero as commonly done in more parsimonious models. It is currently up for debate whether it is possible to use those parsimonious models while still preserving correct type-1 error rates (Bates et al., 2015; Matuschek, Kliegl, Vasishth, Baayen, & Bates, 2015). Often only Bayesian mixed models allow estimating all random slopes in the mixed model to achieve a full statistical coverage when the model is sufficiently complex. Without all random slopes, posteriors, standard errors, or confidence intervals being too narrow, resulting p values are too small, type-1 errors are too high, and thus the effect estimates are too liberal (Barr et al., 2013; Schielzeth & Forstmeier, 2009; Sorensen et al., 2016). After the model fit, we used posterior predictive model checks to see whether the model adequately captures the data. As expected from a Gaussian fit of skewed data, the model checks revealed that we adequately fitted the mean of the distributions, but failed to model the tails appropriately. A possible enhancement for future analysis is to model the choice times using skewed normal, lognormal, or mixed normal distribution to reflect the most extreme quantiles. 

The bubbled guided-viewing paradigm as a research tool

We introduced a new paradigm to investigate fixation durations in free viewing. There are some benefits over current free viewing paradigms, but also shortcomings. One problem with this paradigm is that we do not have a satisfying explanation of why the geometrical correlative effects are so much smaller than in free viewing. This could be a potential problem in studying any geometric effect with this paradigm. It might be that using a less naïve forced fixation duration distribution could alleviate this problem. Second, due to strict criteria, we had to remove ∼45% of the data. The biggest factor was intermediate fixations between two bubbles. Further analysis is needed to determine whether better online fixation detection algorithms or higher accuracy can improve the amount of retained data. 

On the other hand, the benefits are quite clear: We have a direct window to the decision-making mechanisms during free viewing. It controls the time a stimulus is available, how many and where the next stimuli are displayed, and thus, gives valuable variance estimates to constrain computational models. The paradigm is easily expandable to other spatial sampling mechanisms. In this case, we used a Gaussian distance measure, but it is straightforward to relate the geometrics of exploration to real free-viewing scan paths of subjects recorded beforehand. An alternative could be the use of empirical saliency maps that mean sampling the points as bubbles that are most likely fixated by other subjects. In addition, it is not clear how the number-of-bubbles effect generalizes to more bubbles, but there are two likely candidates: an asymptotic behavior or an inverse cubic function. Summarizing the benefits, we can state that we have a new tool to experimentally probe the exploration–exploitation state of the system that allows for close control and is flexible in its sampling scheme. 

Conclusions

Here we developed a new paradigm that allows for experimental control over fixation durations and exploration behavior. We observed that we could selectively interrupt exploitation behavior and confirm predictions from the exploration–exploitation idea. In addition, we show a monotonic increase of choice time with the number of future targets. 

Acknowledgments

The authors thank Jiameng Wu for helping with the data collection and analysis. This work was supported by SPP 1665 and the European Union through the H2020 FET Proactive Project socSMCs (GA no 641321). We acknowledge support by Deutsche Forschungsgemeinschaft (DFG) and Open Access Publishing Fund of Osnabrück University. 

Commercial relationships: none. 

Corresponding author: Benedikt V. Ehinger. 

Email: . 

Address: Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany.