Temporal stimulus properties that attract gaze to the periphery and repel gaze from fixation

Abstract

Abstract

Abstract:

Abstract  Humans use saccadic eye movements to fixate different parts of their visual environment. While stimulus features that determine the location of the next fixation in static images have been extensively studied, temporal stimulus features that determine the timing of the gaze shifts received less attention. It is also unclear if stimulus features at the present gaze location can trigger gaze shifts to another location. To investigate these questions, we asked observers to switch their gaze between two blobs. In three different conditions, either the fixated blob, the peripheral blob, or both blobs were flickering. A time-frequency analysis of the flickering noise values, time locked to the gaze shifts, revealed significant phase locking in a time window 300 to 100 ms before the gaze shift at temporal frequencies below 20 Hz. The average phase angles at these time-frequency points indicated that observer's gaze was repelled by decreasing contrast of the fixated blob and attracted by increasing contrast of the peripheral blob. These results show that temporal properties of both, fixated, and peripheral stimuli are capable of triggering gaze shifts.

Introduction

Humans shift their gaze to a new location several times a second. How exactly they choose their gaze location is one of the most exciting questions in research about vision and eye movements (for recent reviews see Eckstein, 2011; Schütz, Braun, & Gegenfurtner, 2011; Tatler, Hayhoe, Land, & Ballard, 2011). Although it is clear that gaze selection is partially determined by top-down signals like task demands (Land, 2006) or value information (Schütz, Trommershäuser, & Gegenfurtner, 2012), we want to concentrate here on bottom-up signals. 

For a complete model of bottom-up gaze control, the “where” and the “when” of the next fixation have to be determined. The majority of studies focused on the spatial aspect of the next fixation. Gaze positions in free viewing can be explained to a certain degree by salience models that combine salient stimulus features like color, luminance, and orientation in a priority map (Itti & Koch, 2001). In these models, the gaze is shifted to the peak of activation in the priority map, which has been identified in different brain areas like visual areas V1 (Li, 2002), V4 (Mazer & Gallant, 2003), the lateral intraparietal area (LIP) (Goldberg, Bisley, Powell, & Gottlieb, 2006), the frontal eye fields (FEF) (Thompson & Bichot, 2005), and the the superior colliculus (SC) (McPeek & Keller, 2002). These low-level salience models have been challenged by the finding that objects and faces clearly attract gaze (Einhäuser, Spain, & Perona, 2008; Cerf, Frady, & Koch, 2009; Nuthmann & Henderson, 2010), showing the important influence of higher-level features. Visual salience in the retinal periphery also affects fixation locations in dynamic scenes. In a dynamic 2-D bar code pattern, gaze is attracted by a dark spot (Rasche & Gegenfurtner, 2010). When observers are watching movies of natural scenes, moving objects are especially effective in attracting gaze (Dorr, Martinetz, Gegenfurtner, & Barth, 2010; Mital, Smith, Hill, & Henderson, 2011), showing that spatiotemporal stimulus aspects are also an important determinant of gaze location. 

The aforementioned studies investigated how information in the retinal periphery is used to determine the next fixation location in static images or dynamic scenes. However, information at the current fixation location might also be used to determine the next fixation location. So far, only two studies investigated this aspect. For static images, it was observed that the next fixation location had a similar orientation structure to the fixated area if the distance to the current fixation was small and a dissimilar orientation structure if the distance was large (Dragoi & Sur, 2006). This has been interpreted as evidence that the information at the current fixation biases the selection of the next fixation. Feature correlations between successive fixations were also found in dynamic scenes but could be explained completely by the existing correlations in natural scenes and by human preferences to fixate objects (Dorr, Gegenfurtner, & Barth, 2009). Hence there is mixed evidence for an influence of the currently fixated stimulus on the selection of the next fixation location. 

Compared to the spatial determinants of the next fixation, considerably less is known about its temporal aspects, i.e., when one shifts gaze from one location to the next. There is some evidence that peripheral as well as foveal information affects the timing of gaze shifts. Of course the appearance of peripheral targets can break the current fixation and it has been shown that the latency of the following gaze shift depends on the contrast and spatial frequency of the stimulus (Ludwig, Gilchrist, & McSorley, 2004). When the observers have to choose between two peripheral targets, one can study the dynamics of saccadic decision making and information integration. When observers are instructed to saccade to the brighter one of two flickering blobs, they only use the first 100 ms after stimulus onset for their saccadic decision (Ludwig, Gilchrist, McSorley, & Baddeley, 2005). Furthermore gaze shifts are often executed too early, before the available information is completely processed (Beutter, Eckstein, & Stone, 2003; Schütz et al., 2012). Although the information presented directly before the execution of a saccade cannot be used to guide this saccade, it is still processed and used for subsequent saccades (Caspi, Beutter, & Eckstein, 2004). These studies reveal that gaze shifts tend to be executed very quickly, sometimes even before the peripheral information was fully analyzed. 

A less obvious question is whether the currently fixated stimulus can also trigger gaze shifts. In static images, the reduction of visibility leads to longer fixation durations in search tasks (Loftus, 1985; Hooge & Erkelens, 1996). This suggests that gaze duration is prolonged to gain more information about the currently fixated stimulus. Similarly word length and frequency of occurrence affects fixation durations in reading (Kliegl, Nuthmann, & Engbert, 2006). In dynamic conditions, saccade latencies are reduced in the gap paradigm in which the current fixation target disappears shortly before the saccade target appears (Saslow, 1967). It has been hypothesized that the disappearance of the fixation target deactivates a fixation system that has been related to activity of rostral neurons in the superior colliculus (SC) (Dorris & Munoz, 1995; Dorris, Pare, & Munoz, 1997). Saccade generation in the intermediate layers of the SC is typically modeled by an inhibitory mechanism between foveal (rostral) and peripheral (caudal) neurons (Trappenberg, Dorris, Munoz, & Klein, 2001; Munoz & Fecteau, 2002). Activity in peripheral neurons suppresses existing activity in foveal neurons until eventually a saccade is executed to the peripheral peak of activity. In this model, it has not yet been tested if the modulation of foveal activity alone can trigger saccades. 

The short literature overview shows that the influence of the fixated stimulus on gaze shifts is not covered in current models of bottom-up eye movement control. Furthermore it is unknown which temporal features attract gaze in dynamic environments. We investigated these questions by asking observers to alternate their gaze between two blobs. Either one or both blobs were flickering. The data were first analyzed using the classification images approach (for reviews see Abbey & Eckstein, 2002; Murray, 2011), which has been previously applied in eye movement studies (Caspi et al., 2004; Rajashekar, Bovik, & Cormack, 2006). To differentiate effects at different temporal frequencies, the classification images were submitted to a time-frequency analysis, which is commonly used in the analysis of electroencephalography (EEG) data (for reviews see Makeig, Debener, Onton, & Delorme, 2004; Roach & Mathalon, 2008). 

Methods

Observers

Twelve naïve observers participated in these experiments (five males and seven females). The naïve observers were students of the Justus-Liebig-University Giessen and were paid for participation. Experiments were in accordance with the principles of the Declaration of Helsinki and approved by the local ethics committee LEK FB06 at the University Giessen (Proposal Number 2009-0008). Each of the two experiments was performed by six observers. 

Visual stimuli

Two Gaussian blobs were presented on the horizontal meridian. The horizontal distance between the blobs was 16° of visual angle in Experiment 1 and 8° in Experiment 2 and the blobs were placed symmetrically around the vertical meridian. The standard deviation of the Gaussian envelope was 0.5°. Most of the spatial energy of the blobs was concentrated below 1 c/° and the mean spatial frequency amounted to 0.4 c/°. In two different conditions, either the currently fixated blob or the peripheral blob changed their contrast throughout the trials. If the eyes crossed the vertical meridian, the two blobs changed their position so that the same blob was always fixated. In a third condition, both blobs flickered at the same time. Here the blobs stayed at the same position throughout the trial. Since we did not find any significant effects for this condition, we do not show its data. This condition contains saccades that are triggered by the fixated blob, the peripheral blob, or some combination of both blobs. Since all these conditions are lumped together, their effects presumably cancel each other in the analysis. 

Presentation of the three conditions was interleaved. The contrasts of the flickering blobs were drawn from a noise distribution with a power spectrum of 1/f^0.7 and were updated every 10 ms. Thus, the blobs were flickering at 100 Hz which is well beyond the critical flicker frequency of about 60 to 70 Hz (for reviews see Hartmann, Lachenmayr, & Brettel, 1979; Strasburger, Rentschler, & Juttner, 2011). The exponent of 0.7 is close to the distribution of power in natural images (Dong & Atick, 1995; Geisler, 2008) and was chosen to obtain a good compromise between speed of the noise changes and low noise correlation. The constant blobs could be either black or white with a fixed contrast of 0.38. This was approximately the same as the average contrast value of the flickering blob of 0.34. Stimulus presentation was controlled by Matlab using the Psychophysics toolbox (Brainard, 1997; Pelli, 1997). 

Experimental procedure

The observers had to fixate a bull's eye at the beginning of each trial. The bull's eye was located 8° or 4° to the left or right of the vertical meridian in separate experiments. By pressing a button, the observers started the trial and immediately saw the blobs. After 8 s, the trial ended. The observers were instructed to look only at the blobs and to switch between them whenever they wished but without creating a rhythmic pattern with their eyes. In this task, eye movements could be triggered internally by some timing mechanism (for review see Nobre, Correa, & Coull, 2007) or externally by the flickering blobs. Of course the influence of one of these factors will be limited by the influence of the other factor. Here we focused on the effects of the flickering blobs. 

Equipment

The observer's head was stabilized by a chin and forehead rest. Stimuli were displayed on a 21-in. SONY GDM-F520 CRT monitor driven by an Nvidia Quadro NVS 290 graphics board with a refresh rate of 100 Hz noninterlaced. At a viewing distance of 47 cm, the active screen area subtended 45° horizontally and 36° vertically. With a spatial resolution of 1280 × 1024 pixels, this results in 28 pixels/°. The luminance output of the monitor was linearized using the measurements of a photometer (Optometer 370; UDT Instruments, San Diego, CA). The luminances of white, gray, and black pixels were 83, 41.4, and 0.18 cd/m², respectively. Eye position signals of the right eye were recorded with a video-based eye tracker (EyeLink 1000; SR Research, Kanata, Ontario, Canada) and were sampled at 1000 Hz. The eye tracker was driven by the EyeLink toolbox (Cornelissen, Peters, & Palmer, 2002). Saccade onset and offsets were determined with the EyeLink saccade algorithm. 

Eye-movement analysis

Saccade onsets that were followed by a crossing of the vertical meridian were identified and the noise values of the blobs were split into noise segments from 640 ms before to 380 ms after each of these saccade onset (Figure 1A–C). The absolute value of the noise contrast was used to remove the luminance polarity from the data. We excluded saccades for the 16° and 8° separation if the trial onset was closer than 640 ms (9 % | 6 %) or if the trial ended within 380 ms (4% | 4%). These restrictions were chosen so that the following time-frequency analysis yielded results for a time window from 350 ms before to 100 ms after saccade onset. The final data set contained 11,117 and 8,093 noise segments for separations of 16° and 8°, respectively. Since there were large interindividual differences in the fixation duration (Figure 3), the proportion of segments contributed by individual observers varied between 6% and 37% with a standard deviation of 8%. 

Figure 1

 

Statistical analysis. (A) Noise contrast (violet) and eye position (green) of an exemplary trial (8° separation). The noise contrast from another random trial (orange) was used to create the pseudo noise segments. Time windows from 640 ms before to 380 ms after the saccade onsets were extracted (saturated colors) from the real and the pseudo noise. (B) Three real (violet) and pseudo (orange) noise windows extracted from the trial in A. The absolute value of the contrast is shown. The time-frequency decomposition of one of the real noise segments (saturated violet) is shown in C and D. (C) Power spectrum of the saturated real noise segment in B. (D) Phase spectrum of the saturated real noise segment in B.

Time-frequency analysis

A continuous-wavelet transform was used on the noise segments to derive the time-frequency power and phase (Figure 1C, D, Figure 2A–C). Frequency was represented from 2 to 50 Hz in 25 logarithmic steps and time was represented from −350 to 100 ms in 47 linear steps. Morlet wavelets had one cycle at 2 Hz and four cycles at 50 Hz. The time-frequency decomposition was computed using EEGLAB (Delorme & Makeig, 2004). We calculated the noise power (P, Equation 1, Figure 2F), the inter-trial phase coherence (ITPC, Equation 2, Figure 2E), and the inter-trial linear coherence (ITLC, Equation 3, Figure 2D) (Tallon-Baudry, Bertrand, Delpuech, & Pernier, 1996; Makeig et al., 2004). In our case the coherence measures were calculated for the extracted segments and not for the 8 s trials. We nevertheless use the terms inter-trial phase coherence and inter-trial linear coherence because they are well established in the EEG literature. F_k(f, t) is the complex phase and amplitude spectrum at frequency f, time t for trial k:  

 

 

 

 

 

 

Figure 2

 

Time-frequency analysis. (A) Exemplary artificial signals. (B) Morlet wavelet. Real and imaginary parts are phase offset by a quarter of a cycle and plotted with solid and dashed lines, respectively. The resulting phase angle of the wavelet, when real and imaginary parts are represented in polar coordinates, is plotted in gray. The complete wavelet bank is composed of several wavelets with different temporal frequencies, centered at different points in time. (C) Exemplary signals from A, filtered by the Morlet wavelet from B and displayed in Cartesian coordinates. Real and imaginary parts are plotted with solid and dashed lines, respectively. (D) Calculation of inter-trial linear coherence (ITLC). Phase and magnitude information of the exemplary signals are used for the vector average (black). (E) Calculation of inter-trial phase coherence (ITPC). All signal vectors are normalized to a magnitude of unity, such that only phase information is used for the vector average (black). (F) Calculation of power. The phase of the vectors is omitted. (A–F) The colored lines indicate different exemplary signals. (D–F) The black line indicates the vector average for the three exemplary signals. ITLC, ITPC, and power are represented by the length of this average vector. (D, E) Peaks and troughs in the signal are represented by phase values of zero and π, respectively. Rising and falling signals are represented by phase values of π/2 and −π/2, respectively. Consider for instance the yellow signal in A. It is decreasing during the time window of the wavelet in B. If the yellow signal is multiplied by the phase of the wavelet (gray line in B), the average of the resulting vector is close to −π/2, as shown in D.

Figure 3

 

Fixation durations in seconds. Data for 16° separation are plotted in red; data for 8° separation in blue. Open symbols represent individual observers; filled symbols the average across observers. Error bars indicate 95% confidence intervals.

The power is represented in decibels to account for the different absolute levels of power at different temporal frequencies. The ITPC and the ITLC are zero if phases are completely unaligned and unity if phases are perfectly aligned. For the ITPC, the components are normalized so that the power is omitted (Figure 2E). For the ITLC, the components are weighted according to their root mean square (RMS) power (Figure 2D). 

Statistical analysis

In order to compare the empirical data to a pseudorandom baseline, we used the saccades from one trial to segment the noise from another, random trial taken from the same condition but not necessarily from the same observer (a trial being one continuous stimulus presentation of 8 s). Each trial was used only once for the generation of pseudo noise segments. This pseudo data had identical properties as the empirical data, but since saccades and noise values were from different trials and thus not related, it should be free of any saccade related effects. Indeed the actual saccades of the pseudo noise segments were evenly distributed across the whole time span. To sample a sufficiently large amount of pseudo data, we calculated 1000 of these pseudo sets, each having the same number of noise segments as the empirical data. The statistical significance was computed by the following four steps: (a) The studied parameter (average noise contrast, power, ITPC, or ITLC) was calculated for each observer separately for the empirical data set and each of the 1000 pseudo data sets. (b) An average baseline was calculated for each observer and condition across all pseudo data sets. This baseline was subtracted from the respective empirical and pseudo data sets in order to set their average to zero. (c) For all data sets, the grand mean across observers was calculated. Each observer was weighted according to the number of contributed noise segments. This weighting was used to reflect the large differences in the number of contributed noise segments between observers. (d) The maximum value across time and frequency was calculated for each of the pseudo data sets. Values in the empirical data set were considered as significant if they were amongst the 5% most extreme (corresponding to a two-sided test for noise contrast and noise power) or largest (corresponding to a one-sided test for ITPC and ITLC) values in the estimated distribution. Since the maximum value across all time-frequency bins was calculated for each pseudo data set, this procedure effectively controls the family-wise error rate so that no further correction for multiple comparisons is necessary (Nichols & Holmes, 2002). Since we estimated an empirical distribution of maximum parameter values in the pseudo data sets, no assumptions about the underlying statistical distribution have to be made. We also performed the statistical analysis for the classification images on the data of individual observers by leaving out the third step. There were no significant results for the individual observers, presumably because of a lack of statistical power. 

Results

We presented two blobs, of which either the fixated or the peripheral blob was flickering. We asked our observers to fixate one of the blobs and to switch the blobs whenever they wanted. 

First we analyzed the average fixation duration on one blob (Figure 3). The average fixation duration was 1.62 s (SD 0.62, min 0.78, max 2.74) when the fixated blob was flickering and 1.31 s (SD 0.64, min 0.63, max 2.92) when the peripheral blob was flickering and this difference was significant, F(1, 10) = 10.62, p = 0.009. The average fixation duration was 1.15 s (SD 0.43, min 0.74, max 1.95) for 16° separation and 1.79 s (SD 0.67, min 0.63, max 2.92) for 8° separation, the difference being marginally significant, F(1, 10) = 4.36, p = 0.063. The minimum and maximum fixation durations show that observers differed a lot in their gaze behavior. Despite these large interindividual differences, gaze moved consistently more often when the peripheral blob was flickering and when it was further away. 

Classification images

Following the standard classification images approach, we time locked the noise to the onset of saccades and averaged across all noise segments. To obtain a pseudo-random baseline, we randomly reassigned eye movements and noise values from different trials (Figure 1). Finally, we calculated the average difference between real and pseudo noise contrast, with positive values indicating higher contrast in the real noise. Across experiments and conditions, there were some consistent effects of noise contrast in the aggregate data. When the fixated blob was flickering (Figure 4A, C), the real noise contrast was significantly higher from 220 to 180 ms before the saccade for the 16° separation and from 270 to 200 ms for the 8° separation. These results suggest that the observers left the fixated blob when it had a particularly high contrast. This pattern of the aggregate data is also visible in the data of most individual observers. When the peripheral blob was flickering (Figure 4B, D), the results were less consistent for the 16° and 8° separation. In the 16° separation, the real noise contrast tended to be lower from 250 to 150 ms before saccade onset, but this effect did only reach significance for two short time spans at 230 and 180 ms. In the 8° separation, the real noise contrast was also significantly lower, here from 210 to 160 ms before saccade onset. This effect was followed by a significantly higher contrast in the real noise from 130 to 70 ms before saccade onset. These results suggest that observers looked towards the peripheral blob when it had a particularly low contrast (16° and 8° separation) or when it had a particularly high contrast (8° separation). Overall the classification images indicate that the observer's gaze was repelled by high contrast at the fixated blob and attracted by low or high contrast at the peripheral blob. 

Figure 4

 

Classification images. Average noise contrast difference between real and pseudo noise. The gray shaded area represents the 95% confidence interval of the noise differences based on the distribution of noise differences in different pseudo noise sets. The thin gray lines represent the noise contrast differences for six individual observers. The thick horizontal bars indicate time windows with significant inter-trial linear coherence (ITLC) from Figure 7 and Figure 8. (A, C) Fixated blob flickering. (B, D) Peripheral blob flickering. (A, B) 16° separation. (C, D) 8° separation.

Time-frequency analysis: Power

The actual effects of stimulus contrast on saccades might be smeared out in the classification images because of three reasons. First, the saccade latency will vary within and across observers. Since we do not know which noise feature triggered the saccades, the actual saccade latency in a given trial cannot be determined. As a result the potential effects are jittered along the time axis and might be averaged out. Second, the flickering noise of the blobs contains different temporal frequencies and observers might react to different temporal frequencies in different trials. Averaging trials with effects at different temporal frequencies might also extinguish these effects. Third, in the classification images, the direction (increase vs. decrease) and the magnitude (high vs. low) of the contrast changes are inseparable, so that the magnitude is automatically incorporated in the averaging process. The linear classification images will be inevitable compromised if observers are not sensitive to the magnitude or if they use a different weighting strategy, which is likely given the nonlinear nature of early visual processing (Heeger, Simoncelli, & Movshon, 1996). Nonlinearities can be captured by an extension of classification images (Neri & Heeger, 2002; Neri, 2004, 2009), but this still does not separate different temporal frequencies. In order to separate magnitude and phase of contrast changes and to distinguish potential effects at different temporal frequencies, we ran a time-frequency analysis on the noise segments. In the first step, we analyzed the power spectrum to test if observers were simply looking at the blob with stronger contrast changes. We analyzed the difference of the power spectrum (Equation 1, Figure 2F) between the real and the pseudo noise segments. Positive differences indicate higher power in the real than the pseudo segments. There was only one significant difference for the 8° separation when the fixated dot was flickering (Figure 5C). Here the power was significantly higher in the real noise at 6.7 Hz, 225 ms before saccade onset (p = 0.019). However this effect is negligible since it was only present in one of four conditions and only significant for one of overall 1,175 time-frequency points. These data show that the power spectrum of the real noise before a saccade was not different from the arbitrary power spectrum in the pseudo noise. Hence saccades were not consistently triggered by large contrast changes when the direction of contrast changes was ignored. 

Figure 5

 

Power analysis. Difference of power between real and pseudo noise in decibels. Warm colors represent more power in the real noise, cold colors represent less power in the real noise. The white lines enclose areas with significant differences (p < 0.05, two-sided). (A, C) Fixated blob flickering. (B, D) Peripheral blob flickering. (A, B) 16° separation. (C, D) 8° separation.

Time-frequency analysis: Inter-trial phase coherence (ITPC)

It is possible that the power spectrum did not show any effects because the observer's gaze was controlled by the phase of the noise changes rather than by their magnitude. To test this hypothesis we calculated the inter-trial phase coherence (ITPC, Equation 2, Figure 2E), which is a measure of the magnitude of phase alignment, ignoring the power (Tallon-Baudry et al., 1996). If observers reacted to certain changes of the noise, its phase should be aligned relative to the saccade onset across trials. We calculated the difference between the ITPC in real and pseudo noise so that positive values indicate higher ITPC in real than in pseudo noise. We first describe the data for the condition in which the fixated blob was flickering (Figure 6A, C). For the 16° separation, there was a significant peak at 5.8 Hz, 327 ms before the saccade onset (p = 0.022 for peak; p < 0.05 for five time-frequency bins). For the 8° separation there was a significant peak at 6.7 Hz, 125 ms before the saccade onset (p = 0.001 for peak; p < 0.05 for 50 time-frequency bins). 

Figure 6

 

Inter-trial phase coherence (ITPC). Difference of ITPC between real and pseudo noise. Warm colors represent higher ITPC in the real noise, cold colors represent lower ITPC in the real noise. The white lines enclose areas with significantly higher ITPC in the real noise (p < 0.05, one-sided). (A, C) Fixated blob flickering. (B, D) Peripheral blob flickering. (A, B) 16° separation. (C, D) 8° separation.

Similar effects were present for the condition in which the peripheral blob was flickering (Figure 6B, D). For the 16° separation, there was a significant peak at 3.9 Hz, 186 ms before the saccade (p = 0.001 for peak; p < 0.05 for 71 time-frequency bins). For the 8° separation, there was a significant peak at 5.8 Hz, 175 ms before the saccade (p = 0.001 for peak; p < 0.05 for 182 time-frequency bins). 

These data show clearly that the ITPC, time locked to the saccade onset, was significantly higher in the real noise than in the pseudo noise. Hence the saccades were partially triggered by coherent changes of about 2 to 10 Hz in the flicker. 

Figure 7

 

Inter-trial linear coherence (ITLC) when the fixated blob was flickering. (A–C) 16° separation. (D–F) 8° separation. (A, D) Difference of ITLC between real and pseudo noise. Warm colors represent higher ITLC in the real noise, cold colors represent lower ITLC in the real noise. (B, E) p values. Light colors represent small p values. (C, F) Average phase angles. Warm colors represent troughs (at phase π and −π); Cold colors represent peaks (at phase zero). (A–F) The white lines enclose areas with significantly higher ITLC in the real noise (p < 0.05, one-sided).

Figure 8

 

Inter-trial linear coherence (ITLC) when the peripheral blob was flickering. (A–C) 16° separation. (D–F) 8° separation. (A, D) Difference of ITLC between real and pseudo noise. (B, E) p values. (C, F) Average phase angles. Conventions are the same as in Figure 7.

Time-frequency analysis: Inter-trial linear coherence (ITLC)

Finally we tested whether saccades could be predicted better by a combination of power and phase information than by phase information alone. Instead of normalizing each time-frequency component to a magnitude of unity, like in the ITPC, we weighted each component according to its RMS power in the inter-trial linear coherence (ITLC, Equation 3, Figure 2D). As a result, segments with high power will have a stronger influence on the phase coherence than segments with low power. 

There were significant effects when the fixated blob was flickering. For the 16° separation (Figure 7A–C) the ITLC was significantly higher for real noise from 246 to 155 ms before saccade onset at frequencies from 2.3 to 8.7 Hz (p = 0.004 for peak; p < 0.05 for 81 time-frequency bins). A similar but broader pattern emerged for the 8° separation (Figure 7D–F). Here the ITLC was significantly higher for real noise from 266 to 105 ms before saccade onset at frequencies from 2.0 to 11.4 Hz (p = 0.001 for peak; p < 0.05 for 174 time-frequency bins). 

Significant ITLC was also present when the peripheral blob was flickering. For the 16° separation (Figure 8A–C) the ITLC was significantly higher for real noise from 145 to 135 ms before saccade onset at frequencies from 5.8 to 10.0 Hz (p = 0.03 for peak; p < 0.05 for eight time-frequency bins). Again, a similar but broader pattern emerged for the 8° separation (Figure 8D–F). Here the ITLC was significantly higher for real noise from 266 to 44 ms before saccade onset at frequencies from 2.0 to 17.1 Hz (p = 0.001 for peak; p < 0.05 for 287 time-frequency bins). 

Except in the condition with 16° separation and a peripheral flickering blob, there were more significant time-frequency points for the ITLC than for the ITPC. This suggests that the phase coherence improved when the time-frequency components were weighted according to their magnitude. 

By looking at the average phase values in the time-frequency bins with a significant ITLC, we can identify the type of change that triggered the eye movements. Phase values of zero and π indicate contrast peaks and troughs, respectively. Phase values of π/2 and –π/2 indicate increasing and decreasing contrast, respectively (Figure 2). When the fixated blob was flickering, the average phases were −0.05 π and −0.39 π for 16° and 8° separation, respectively. This indicates that the fixated blob triggered an eye movement at a contrast peak or at decreasing contrast. This result was consistent with the classification images, which showed a contrast peak and a steep decline of contrast in the same time span (Figure 4A, C). When the peripheral blob was flickering, the average phases were 0.52 π and 0.59 π for 16° and 8°, respectively. This indicates that the peripheral blob triggered eye movements at increasing contrast. This was also consistent with the classification images, which showed a steep contrast increase in the same time span (Figure 4B, D). 

The above mentioned results showed mainly significant ITLC at low temporal frequencies. Three reasons could account for this. First, it might be that existing effects at high temporal frequencies were not detected because the analysis of these temporal frequency components suffers especially much from the saccade latency jitter across trials and observers. Second, the power distribution of the noise might be responsible for these results. With a 1/f^0.7 power spectrum, the noise contained more energy at low than at high temporal frequencies. Third, the result might also reflect differences in human temporal contrast sensitivity. To analyze the influence of temporal frequency in more detail, we calculated temporal phase-locking functions by averaging the ITLC across all time bins that contained a significant effect and by normalizing them to a maximum of unity (Figure 9A). This normalization renders the absolute values of the functions uninformative but allows a convenient comparison of their shape across temporal frequencies. All the temporal phase-locking functions showed a clear band-pass characteristic with peaks between 3.9 and 7.7 Hz. The power spectrum of the noise however showed a low-pass characteristic, so that there was less power at the temporal frequencies where the phase-locking functions actually peaked. This difference was especially obvious at low temporal frequencies, where the noise power spectrum was declining but the temporal phase-locking functions rising. Hence, the observed effects were not determined by the power spectrum of the flickering noise. 

Figure 9

 

Temporal frequency and latency effects. (A) Temporal frequency effects. The red and blue lines show the ITLC from Figure 7 and Figure 8, averaged across all time points with a significant effect. The black line shows the temporal contrast sensitivity function (TCSF) for a grating with a spatial frequency of 0.4 c/°, taken from Kelly (1979). The green line shows the power spectrum of the noise flicker. (B) Latency effects. The red and blue lines show the ITLC from Figure 7 and Figure 8, averaged across all temporal frequency points with a significant effect. (A, B) All functions are normalized to a maximum value of unity to facilitate comparison. Hence the absolute values of the functions are not meaningful, only their shapes and their positions on the x-axis can be compared. Red and blue indicate 16° and 8° separation, respectively. Solid and dashed lines indicate fixated and peripheral blob flickering, respectively.

We also compared the temporal phase-locking functions with human contrast sensitivity. Since the Gaussian blobs had an average spatial frequency of 0.4 c/°, we used the temporal contrast sensitivity for a 0.4 c/° grating as a comparison (Kelly, 1979). The temporal contrast sensitivity function shows a band-pass shape, very similar to the temporal phase-locking functions. Contrast sensitivity peaks at 5.6 Hz, which is in the same range as the peaks of the temporal phase coherence functions. The temporal contrast sensitivity functions were obtained for foveal vision with retinal stabilization (Kelly, 1979), hence they do not perfectly match our experimental conditions. This is especially the case when the peripheral blob was flickering at a separation of 16°. In this case the temporal phase-locking function was shifted to higher temporal frequencies and suppressed at lower temporal frequencies compared to the other phase-locking functions and the foveal temporal contrast sensitivity function. However these differential effects are consistent with psychophysical data on temporal contrast sensitivity at different retinal eccentricities (Snowden & Hess, 1992), showing that the peak contrast sensitivity is shifted to higher temporal frequencies at larger eccentricities. Overall, these results suggest that the temporal shape of the phase locking primarily reflects the human temporal contrast sensitivity and is not determined by the power spectrum of the noise or the temporal jitter induced by the variability of saccade latencies. Hence the observed effects are most likely determined by properties of the human visual system and not by the properties of our experimental paradigm or data analysis. 

Finally we wanted to investigate the latency of the phase locking more closely. To this end, we followed a similar logic as for the temporal phase-locking functions above: We averaged the ITLC across all temporal-frequency bins that contained a significant effect and normalized them to a maximum of unity. Most of the functions showed a bell-shaped curve between 300 and 50 ms before saccade onset (Figure 9B). When the fixated blob was flickering, the phase coherence increased early, at about 300 ms before saccade onset phase. It peaked at 186 and 165 ms before saccade onset for 16° and 8° separation, respectively. The full width at half height (FWHH) of the latency functions were 172 and 161 ms for 16° and 8° separation, respectively. When the peripheral blob was flickering, the phase coherence increased later, at about 200 ms before saccade onset. Consistently it peaked later, at 145 and 135 ms and also had smaller FWHHs of 71 and 131 ms for 16° and 8°, respectively. Hence the saccade latencies tended to be longer and more variable when the fixated blob was flickering than when the peripheral blob was flickering. 

When the peripheral blob was flickering at a separation of 16°, the FWHH was much smaller than in the other conditions. However this particular difference might be caused by properties of the time-frequency analysis, which has different temporal resolutions at different temporal frequencies. According to the Heisenberg inequality (Folland & Sitaram, 1997) a signal cannot be represented with high resolution in time and frequency at the same time. The chosen wavelet transformation uses shorter time windows at high temporal frequencies, which results in higher temporal resolution at high temporal frequencies. When the peripheral blob was flickering at 16°, the phase locking occurred at higher temporal frequencies than in all other conditions (Figure 9A). Together with the higher temporal resolution of the time-frequency analysis at high temporal frequencies, this might result in the smaller FWHH of the latency functions. 

Discussion

We asked observers to alternate gaze between two blobs that could be flickering. Observers switched gaze between the two blobs more often if the peripheral blob was flickering than if the foveated blob was flickering (Figure 3). Two well-known oculomotor phenomena may contribute to this effect under the assumption that the flickering stimulus resembles an onset stimulus when its contrast increased from a low initial value. First, there is oculomotor capture by sudden onsets in the periphery (Theeuwes, Kramer, Hahn, & Irwin, 1998) which may have reduced fixation durations when the peripheral blob was flickering. Second, there is a delay in saccadic response times when an irrelevant stimulus is presented at target onset in the fovea (Walker, Deubel, Schneider, & Findlay, 1997; Born & Kerzel, 2008) that may contribute to longer fixation durations when the fixated blob was flickering. Furthermore fixation durations decreased with increases in retinal eccentricity. On a speculative note, one may propose that observers were even less sure about what was happening in the peripheral blob when the separation was large and therefore made saccades more often in order to decrease the higher uncertainty. However further research is needed to fully understand the effect of eccentricity. 

The noise values of the flickering blobs were time locked to the saccade onsets and average classification images were calculated. The classification images showed significant effects from 270 to 70 ms before saccade onset (Figure 4). Noise contrast was at an increased level or was falling when the fixated blob was flickering. Noise contrast was at a decreased level or was rising when the peripheral blob was flickering. These data show that gaze switches can be triggered by foveal and peripheral information. Importantly, our research shows that gaze is not only attracted by changes in peripheral information, but also repelled by changes in foveal information. 

To disentangle effects at different temporal frequencies, we submitted the noise segments to time-frequency decomposition. We did not find strong predictive effects of noise power (Figure 5) but predictive ITPC (Figure 6) and ITLC (Figures 7, 8) of the noise phase values before the onset of saccades. This means that saccades were more triggered by consistent changes in the noise contrast and less so by the magnitude of these changes. However there were more significant time-frequency points for the ITLC than for the ITPC. Although the power of the noise could not explain gaze behavior itself, it increased the phase coherence if taken into account. Hence, observers were sensitive to the phase of noise changes and to the magnitude of these changes. 

The phase angles in the significant time-frequency bins were close to −π/2 when the fixated blob was flickering. This suggests that the observers left the fixated blob when its contrast was falling. Pointing in the same direction, the classification images showed falling contrast values (Figure 4A, C) at the time of significant ITLC. When the peripheral blob was flickering, the phase angles in the significant time-frequency bins were close to π/2. This means that the observers moved their gaze to the peripheral blob when its contrast was rising. Similarly, the classification images showed rising contrast (Figure 4B, D) at the time of significant ITLC. Taken together these results suggest that the fixated blob repelled gaze at falling contrast and the peripheral blob attracted gaze at rising contrast. The falling contrast of the fixated blob is similar to the disappearance of the fixation target, which can lead to oculomotor disengagement as in the gap effect (Saslow, 1967). The rising contrast of the peripheral blob resembles onset stimuli that have been shown to attract attention as in oculomotor capture (Theeuwes et al., 1998). These well-known oculomotor phenomena have been studied mostly in conditions where eye movements are largely constrained and where observers move their eyes only a few times in a given trial. Our results extend these findings to a continuous paradigm in which the observers are free to move their eyes whenever they want. 

Across experiments and conditions, observers reacted primarily to low temporal frequencies since the phase coherences peaked between 3.9 to 7.7 Hz (Figure 9A). The temporal phase coherence functions showed very similar low-pass shapes as the temporal contrast sensitivity function (Kelly, 1979). This suggests that these results reflect properties of the human visual system rather than properties of the experimental paradigm. In that view, gaze shifts were mainly triggered by changes at temporal frequencies to which humans are especially sensitive. 

In all conditions and experiments, a significant ITLC appeared about 100 to 250 ms before the gaze switch (Figure 9B). These measured latencies are mainly at the lower end of the usual saccade latencies of about 100 to 400 ms (for review see Findlay & Gilchrist, 2003). However this is not surprising, given that the saccades in our experiment could have been preprogrammed, since the target of the saccades (the peripheral blob) was predetermined. This general agreement between our estimated latencies and the typical latencies in the literature also suggests that the obtained results reflect properties of the human visual- or eye movement system rather than properties of the data analysis. The estimated latencies tended to be longer and also more variable when the fixated blob was flickering than when the peripheral blob was flickering. Taken together with the longer fixation durations in this condition, one could argue that the fixated blob had a weaker or less direct influence on gaze behavior than the peripheral blob. 

In traditional EEG analysis, the EEG data are aligned to an event and averaged across a large number of trials, in order to obtain an event-related potential (ERP) (Luck, 2005). This is comparable to the traditional approach in the classification image analysis. In recent years, time-frequency analysis has become increasingly popular in the analysis of EEG data, because it overcomes certain limitations of the traditional ERP approach (Makeig et al., 2004; Mouraux & Iannetti, 2008; Roach & Mathalon, 2008). Time-frequency analysis, for instance, allows the separation of oscillatory power and phase, which lead to the discovery of non-phase locked oscillations in humans (Tallon-Baudry et al., 1996). Separation of power and phase also revealed that human visual perception (Busch, Dubois, & VanRullen, 2009) and eye movements (Drewes & VanRullen, 2011) depend on the phase of brain oscillations. We suggest that time-frequency and inter-trial coherence analyses might supplement the traditional classification images approach, just as they supplemented the traditional ERP analysis. The present study shows that time-frequency analysis is especially useful whenever the observers could be influenced by different temporal frequencies of the stimulus (Caspi et al., 2004; Ludwig et al., 2005). 

Acknowledgments

We thank Jan Drewes for helpful discussion on the time-frequency decomposition, two anonymous reviewer for helpful comments on a previous version of the manuscript, Sabine Born for technical assistance, and Jan Christopher Werner for help with data collection. This work was supported by the DFG grant SCHU 2628/2-1 to ACS. 

Commercial relationships: none. 

Corresponding author: Alexander C. Schütz. 

Email: [e-mail]. 

Address: Abteilung Allgemeine Psychologie, Justus-Liebig-Universität Gießen, Gießen, Germany.