Perceptual compression of visual space during eye–head gaze shifts

Abstract

In primates, inspection of a visual scene is typically interrupted by frequent gaze shifts, occurring at an average rate of three to five times per second. Perceptually, these gaze shifts are accompanied by a compression of visual space toward the saccade target, which may be attributed to an oculomotor signal that transiently influences visual processing. While previous studies of compression have focused exclusively on saccadic eye movements made with the head artificially immobilized, many brain structures involved in saccade generation also encode combined eye–head gaze shifts. Thus, in order to understand the interaction between gaze motor and visual signals, we studied perception during eye–head gaze shifts and found a powerful compression of visual space that was spatially directed toward the intended gaze (and not the eye movement) target location. This perceptual compression was nearly constant in duration across gaze shift amplitudes, suggesting that the signal that triggers compression is largely independent of the size and kinematics of the gaze shift. The spatial pattern of results could be captured by a model that involves interactions, on a logarithmic map of visual space, between two loci of neural activity that encode the gaze shift vector and visual stimulus position relative to the fovea.

Introduction

Stimuli that are briefly flashed around the time of saccades are misperceived spatially. This phenomenon is referred to as perisaccadic mislocalization, and localization errors have been demonstrated both in completely dark conditions (Bischof & Kramer, 1968; Dassonville, Schlag, & Schlag-Rey, 1995; Matin & Pearce, 1965) and when visual references are available (Lappe, Awater, & Krekelberg, 2000; Ross, Morrone, & Burr, 1997). For subjects tested in the dark, the errors consist of a perceived translational shift of an object's location in the direction of the eye saccade (Honda, 1989; Mateeff, 1978; Matin & Pearce, 1965; Schlag & Schlag-Rey, 1995) or eye–head gaze shift (van Wetter & van Opstal, 2008). In contrast, when visual references are available, localization errors converge toward the saccade endpoint, resulting in a phenomenon called perisaccadic compression (Lappe et al., 2000; Ross et al., 1997). This compression of visual space is due in part to an extraretinal signal related to the eye movement command, since little or no such effect is observed in the absence of an eye movement (Morrone, Ross, & Burr, 1997). Recent modeling work (Hamker, Zirnsak, Calow, & Lappe, 2008; Richard, Churan, Guitton, & Pack, 2009; Zirnsak, Lappe, & Hamker, 2010) has suggested that compression may be explained by the interaction of sensory (i.e., visual) and extraretinal (i.e., motor) signals on a retinotopically encoded logarithmic map of visual space. 

The majority of previous experiments studied compression during saccadic eye movements made with the head artificially immobilized throughout each trial. However, many brain structures in the primate visuomotor pathway encode the target of gaze movements that involve both the head and the eye (in superior colliculus: Munoz, Guitton, & Pélisson, 1991; frontal eye field: Martinez-Trujillo, Klier, Wang, & Crawford, 2003; supplementary eye field: Martinez-Trujillo, Medendorp, Wang, & Crawford, 2004). In seeking to understand how motor information is involved in compression, it is therefore crucial to distinguish between the role of eye- and gaze-related motor signals. Here, we studied human subjects who made saccadic gaze shifts during a standard paradigm for measuring perceptual compression. 

As two independent effectors move the visual axis in head-unrestrained gaze shifts, it is possible to experimentally manipulate the relative contributions of each to the overall gaze trajectory and kinematics. This has allowed us to uncouple the gaze kinematics from the dynamics of the compression illusion, revealing that the latter has a fixed timing relative to the gaze saccade. Spatially, we find that compression is unambiguously linked to the target of the gaze shift rather than to the endpoint of the eye saccade that initiates each gaze shift. This suggests that the spatial aspects of perceptual compression are largely independent of the effectors used to execute the gaze shift. Moreover, we were able to simulate our results using a model inspired originally by data on head-fixed saccades (Richard et al., 2009). A key feature of this model is an interaction between motor and visual signals on a logarithmic map similar to those found in brain regions that encode both types of signals. Our results therefore link the perception of visual space around the time of eye–head gaze shifts to the properties of oculomotor structures in the primate brain. 

Methods

Subjects

Data were collected from three male subjects (1 author and 2 naive), each of whom had normal vision. Informed consent was obtained from all subjects before their participation, and all experimental protocols were approved by the Montreal Neurological Institute and Hospital Research Ethics Committee. Subjects participated in 20–30 sessions, each lasting approximately 1 h. 

Stimuli and experimental setup

Subjects were seated in front of a semi-transparent screen subtending 90° of visual angle horizontally and 40° vertically. Viewing was binocular at a distance of 56 cm. The head was stabilized with an adjustable head strap and bite bar. Stimuli were generated in Matlab using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) and back-projected onto the screen at 85 Hz with an Electrohome Marquee 8000 projector (resolution: 1024 × 768 pixels) against a homogenous black background (luminance < 0.01 cd·m⁻²). To facilitate the perception of compression, we followed Lappe et al. (2000) and Ross et al. (1997) and presented a horizontal reference ruler, with vertical ticks and ordinally arranged numbers at 10° intervals (luminance of 118.4 cd·m⁻²) for the duration of each trial. Eye position was monitored continuously at 120 Hz using infrared oculography (ASL Laboratories Eye Tracker). Event timing, online displays, and the data acquisition were controlled using REX, a QNX-based real-time data acquisition system (Hays, Richmond, & Optican, 1982). 

Experimental procedures

Figure 1A illustrates the monitor frames for a typical trial. The beginning of each trial was marked by the presentation of a fixation cross (1° × 1°; luminance of 118.4 cd·m⁻²) at 20° from the left of screen center for all 40° gaze shifts. For 60° gaze shifts, the initial fixation position was at 40° left to allow more screen space for the presentation of stimuli beyond the gaze saccade endpoint (see next paragraph). After a brief period of fixation, the fixation cross disappeared, and at the same time, a saccade target was flashed for 24 ms at 20° to the right of screen center, to trigger gaze shift amplitudes of 40° or 60°, depending on the position of the initial fixation point. With the exception of one experiment in which the head was immobilized by a bite bar and adjustable head strap (the head-restrained, or HR, condition), subjects were required to make a combined eye–head gaze shift (hereafter referred to as head-unrestrained or HU conditions) to within ±2.5° of the target for a trial to be completed successfully. The angular position of the subject's head was monitored with both a low-torque precision potentiometer and a magnetic search coil (sampling at 1 kHz). 

Figure 1

 

Task design. (A) Experiment frames depicting the sequence of events on the screen for a given trial. Subjects fixated a fixation cross (FC) for a brief period, after which the gaze saccade target (ST) was flashed for two frames (24 ms), and a vertical narrow bar was flashed for one frame (12 ms) after a variable delay. After successful completion of a gaze saccade, the observer was cued to manually report the apparent position of the bar, by using a mouse to move a cursor on the horizontal ruler present on the screen. (B) Depiction of the start of a typical trial for a 40° gaze shift in the eye-misaligned condition. Observers began each trial with the head aligned to a long vertical bar at 40° in space or 20° in the opposite direction of the impending gaze saccade from the FC. This manipulation resulted in the eye being offset in the orbit by 20° in the direction of the gaze saccade. FC and the head alignment bar were turned off, and simultaneously, ST was then turned on, cuing the subject to generate the saccade.

At an unpredictable time (10–500 ms) after the saccade target was flashed, a localization target (LT) in the form of a vertical bar 20° in length and 1° in width was presented for 24 ms at a random location relative to the gaze target. The luminance of the LT was always 118.4 cd·m⁻². LT presentation time was within an interval ±200 ms relative to mean gaze onset, which was determined in pilot experiments to be 256 ± 38 ms (mean ± SD). The successful completion of a gaze shift was followed by the appearance of a cursor, which subjects could move horizontally by means of a mouse. No auditory cues were provided at any point, as such cues affect perception during head and eye movements differently (Leung, Alais, & Carlile, 2008). Subjects indicated the perceived location of the LT with a mouse click after moving the cursor to the appropriate location. During this part of each trial, subjects were free to move their gaze. For most experimental conditions, subjects performed two blocks of 300 successful trials, with the magnitude of the gaze shift remaining constant (i.e., either 40° or 60°) for the duration of each block. 

We tested the 3 subjects in a total of 6 experimental conditions, each run in a block of trials. We focused first on 40° gaze shifts, which were small enough that we could compare the compression effect in the HR and HU conditions. We also studied compression during 60° gaze shifts, but for these we could not run 60° head-fixed saccades due to limits on the eye deviation in the orbit (≈±45°; Guitton & Volle, 1987). To study the possibility that compression might depend on the velocity of a gaze shift, as previously reported for head-fixed saccades (Ostendorf, Fischer, Finke, & Ploner, 2007), we carried out an additional experiment in which we instructed subjects to generate voluntarily, either fast or slow 40° gaze shifts, in different blocks of trials. They did this by making either fast or slow head movements. Such voluntary control on gaze velocity is impossible for HR saccades. 

For comparison with saccades made with the eye alone (HR), we studied compression during 40° gaze shifts executed mainly with the head. To do this, we implemented an eye-misaligned condition in which gaze shifts were initiated with the eye in the orbit initially deviated by 20° in the direction of the intended gaze shift. This was accomplished, before the gaze shift was triggered, by aligning the subject's head using a head-mounted laser (Figure 1B) to a long vertical bar that was offset 20° from the fixation point in the direction opposite to the impending gaze movement. Such an initial condition on the eye and head resulted in the eye being close to the limits of ocular motility, thereby limiting saccade amplitude to ≈25°. The eye-misaligned condition differed from the standard eye-aligned condition, in which subjects began each trial with both their eyes and heads aligned to the fixation point. 

To summarize, we studied subjects making 5 types of 40° gaze shifts: (1) head-restrained (HR); (2) a natural head-unrestrained (HU) condition in which the eyes and head were initially aligned (HU aligned) and subjects were given no instructions about the desired gaze velocity; (3) HU aligned, slow; (4) HU aligned, fast; (5) HU misaligned. We also studied 60° gaze shifts made in the natural eye–head aligned condition, for a total of 6 experimental conditions. 

Data analysis

Calibration of the eyes and head was performed at the beginning of each experimental session. Analog signals of the eye and head movements, in addition to the actual and perceived positions of the LT, were analyzed offline using Matlab (Mathworks). Gaze onset time was calculated using a variation of the method described by Carl and Gellman (1987), as the intersection between the linear regression of a data sample obtained after a velocity criterion (60°/s) was reached and the average eye position during the previous fixation. A velocity criterion (30°/s) was also used to determine gaze offset time for the HR condition. However, we did not use a velocity criterion to determine when an HU gaze shift ended because many of the trials in the fast, slow, and misaligned conditions showed gaze shifts that ended with slowly declining velocity profiles. Hence, to avoid underestimating gaze duration in these trials by choosing a velocity criterion that was too high, we chose for all trials a threshold defining gaze end, equal to a position error of ±5% from intended gaze shift amplitude. In subsequent analysis, we found that the velocity criterion of 30°/s provided an estimate of gaze end statistically similar to the position threshold (Supplementary Figure 6; 2-sample t-test, α = 0.05). 

Trials were automatically discarded for any of the following reasons: if the reaction time exceeded 500 ms, if the saccade/gaze endpoint fell outside a ±2.5° window centered on the saccade target, if a blink occurred around the time of bar presentation, or if a saccade occurred before target onset. On average, 7% of all trials were discarded per experiment. Analog data from HU experiments were further subjected to a trial-by-trial screening by a human operator (author AR) to ensure gaze accuracy and signal integrity. Data from separate blocks of the same experiment were pooled for each subject. Mean eye, head, and gaze trajectories were calculated by pooling 100 trials from each subject. 

To quantify the magnitude of perceptual compression in some of our analyses, we used the compression index (CI_L) initially proposed by Lappe et al. (2000), calculated as the SD of the LT perceived positions, normalized to the average SD during the periods 150–200 ms before and after the gaze saccade. According to this metric, a CI_L value of 1 denotes no mislocalization, while a value of 0 refers to maximal compression, which would occur if all LTs were perceived at the same position in space. 

Model

Previous work from our laboratory has shown that the main features of compression can be captured by a simple model that invokes the interaction of populations of neurons that encode, respectively, the positions of the saccade motor signal and the visual stimulus on a retinotopically encoded logarithmic map of visual space (Richard et al., 2009). The model was premised on the manner in which both spatial and movement vectors are retinotopically mapped throughout the visuomotor pathway. This logarithmic encoding allocates more neural tissue to objects near compared to far away from the fovea (e.g., in primary visual cortex: Schwartz, 1977; and in the brainstem's superior colliculus: Ottes, Van Gisbergen, & Eggermont, 1986). By extension, the visual and oculomotor activities attributable to the LT and saccade goal (see Figure 8A for an explanation and Richard et al., 2009) may, under certain experimental conditions, overlap and be difficult to discern from one another. Allowing for the possibility that the output of such a map may be linked to perception around the time of saccades, this interaction could conceivably bias the observer's perception of the LT toward the saccade target location, where the extent of mislocalization would depend on the proximity of these two populations on the map. 

In order to quantify perceptual compression on the basis of individual LT positions, we defined the compression index (CI) as the distance between the perceived position P of the LT and the saccade goal S, normalized by the distance between the veridical LT position (B) and S. 

Our previous model (Richard et al., 2009) was formulated to reflect the hypothesis that the magnitude of perceptual compression, given by CI, is proportional to the distance, dx, on a logarithmically encoded map between the LT and saccade goal: 

$\mathrm{C}\mathrm{I}=\frac{P-S}{B-S}\propto \mathrm{d}\mathrm{x},$

(1)

where 

$\mathrm{d}\mathrm{x}=|k_1·\log (S+1)-k_1·\log (B+1)|.$

(2)

In this formulation, k ₁ is a constant whose value we found to be approximately the same across conditions within individuals but slightly different across individuals (see Table 1). The final formulation of the model included an additional constant, k ₂, that accounted for a small residual translational shift in P similar to that found in experiments that studied perisaccadic mislocalization in complete darkness (e.g., Michels & Lappe, 2004; van Wetter & van Opstal, 2008): 

$\frac{P-S}{B-S}=|k_1·\log \left(\frac{B+1}{S+k_2+1}\right)|.$

(3)

 

Table 1

 

Summary of modelling parameters (k ₁, k ₂, and mean values used for Figure 8) and statistics for all observers and conditions.

Conditions	C.P.				M.G.				S.N.
	k 1	k 2	χ R 2	P value	k 1	k 2	χ R 2	P value	k 1	k 2	χ R 2	P value
40° HR	0.78	10.61	0.68	0.76	0.77	9.69	0.49	0.91	1.15	13.60	0.65	0.79
40° HU aligned	0.83	10.98	0.39	0.97	0.85	9.76	0.27	0.99	0.78	9.93	1.33	0.20
40° HU misaligned	0.91	12.55	0.18	0.99	0.99	13.09	0.16	0.99	0.76	10.79	1.39	0.17
40° Slow	0.90	9.21	0.82	0.62	0.84	10.82	0.35	0.98	1.23	10.83	0.79	0.65
40° Fast	1.04	12.86	0.57	0.86	0.78	9.49	0.32	0.99	1.04	11.84	0.71	0.73
60° HU aligned	1.23	13.78	0.09	0.99	1.12	8.69	0.79	0.60	0.96	14.46	1.12	0.34
Mean ± SD	0.95 ± 0.16	11.67 ± 1.69	–	–	0.89 ± 0.14	10.26 ± 1.55	–	–	0.99 ± 0.19	11.91 ± 1.77	–	–
All values mean ± SD	k 1: 0.94 ± 0.05
	k 2: 11.28 ± 0.98

In this arrangement, the term on the left describes the visual mislocalization, and the term on the right represents the distance between B and S in log coordinates (Schwartz, 1977). The model in Equation 3, with its two free parameters kept fixed for any given subject, was sufficient to account for a large HR data set that spanned several saccade amplitudes (14, 20, and 30°; see Richard et al., 2009). Specifically, we optimized both the k ₁ and k ₂ parameters with a least squares fitting procedure (lsqcurvefit, Matlab) for each subject in each condition (see values in Table 1). We then used the mean parameter values for each observer across all conditions to generate all model predictions and statistics for each observer. Thus, the results in Figure 8 are based on a model that involves only two parameters per observer. 

Results

Overview of eye–head gaze shifts

Previous work has shown a dramatic compression of visual space around the time of an eye saccade (Lappe et al., 2000; Ross et al., 1997), and theoretical models have indicated that such a process may be related to mechanisms of perceptual stability (Hamker et al., 2008). However, in naturalistic settings, gaze shifts are often accomplished using various combinations of eye and head movements. We therefore sought to determine how perceptual compression depends on the relative contributions of the different effectors to a given gaze movement. To this end, we tested 3 observers in a standard perceptual compression task in which they executed gaze shifts of given amplitudes that had different trajectories and different contributions from the eye and head. Specifically, we tested 40° saccades made with the head restrained (HR), in addition to 40° head-unrestrained (HU) gaze shifts in which the eye and head began each trial aligned or misaligned (Figure 1B). We also tested two conditions in which subjects were specifically instructed to make slow or fast eye–head gaze shifts. Finally, for comparison, we also tested subjects in a condition that required a 60° eye–head gaze shift. These manipulations resulted in a rich data set in which, for a given gaze amplitude, the eye and head amplitude and velocity contributions were variable across conditions (Figure 2 and Supplementary Figure 1). 

Figure 2

 

Average (±SD) eye, head, and gaze traces for each observer in 5 experimental conditions (observer C.P.; for other observers, see Supplementary Figure 1). Gaze = eye in space = eye in head + head in space. Upper row of each panel shows the analog position traces (in the horizontal plane), with gaze (black), head (blue), and eye (red) traces overlaid; the lower row shows the corresponding velocity profiles. Gaze onset and offset times for all panels are indicated with short vertical lines.

We begin by briefly describing the main characteristics of gaze shifts in the different conditions. A full treatment of this topic is available from many previous studies on this subject (reviewed in Freedman, 2008; Guitton, 1992; Guitton, Bergeron, & Choi, 2004). However, to orient the reader, we will briefly summarize the main points. Figures 2A–2E (upper) show (for example, subject C.P.) typical gaze shifts (mean ± SD) and their respective eye and head components, for the 5 HU conditions we tested. The corresponding velocity traces are shown below the position traces. Of these five HU tasks, Figures 2A, 2B, and 2E illustrate the biggest variation in the relative contributions of eye (red traces) and head (blue traces) amplitudes and velocity to a gaze shift (black traces). The trajectories for the other 2 observers (M.G. and S.N.) are similar and shown in Supplementary Figure 1. 

Several important differences in the gaze trajectories are observable by inspection of Figure 2. When the eye and head were aligned at the start of the trial in 40° and 60° gaze shifts (Figures 2A and 2E), the eye saccade (red) always started before the head (blue) and contributed mainly to the early portion of the gaze (black) displacement, while the head movement contributed mainly to the late gaze displacement. In conformity with many previous studies of gaze shifts (e.g., Guitton & Volle, 1987), the peak eye movement amplitude was 30–40°, with the eye reversing its direction of motion before gaze attained its goal, particularly in the 40° misaligned (Figure 2B) and 60° conditions (Figure 2E). In the former condition, the initial eye position was close to the ocular saturation position for this subject—approximately 40° in the orbit—and, due to this saturation, the saccade amplitude was limited to ≤20°. Consequently, the head contribution to carrying gaze was particularly important, such that the gaze velocity profile was more asymmetric than in the aligned condition, often creating a gaze trajectory that ended with a slow ramp. Head motion also contributed proportionally more to gaze amplitude in the 60° condition. 

Across subjects, there was no significant difference in gaze amplitude between the conditions in the 40° amplitude gaze shifts (two-way ANOVA, p > 0.5). For example, the mean gaze amplitude ± SD is 41.2° ± 3.3 for the aligned condition (Figure 2A) and 39.8° ± 4.1 for the misaligned condition (Figure 2B; paired t-test, p < 0.05). In a subsequent analysis below, we will require an estimate of gaze shift duration, for which there is no standard definition in the head-free condition. Here, we used two criteria: (1) a ±5% error on intended (40° or 60°) gaze shift amplitude indicated by a vertical tic mark on each gaze position trace in Figure 2 and (2) a 30°/s threshold indicated by a vertical tic on each gaze velocity trace in Figure 2. We have found these definitions of gaze end to be similar and highly accurate in determining when the visual axis was on target (i.e., within 2–3°) across both experiments and observers (Supplementary Figure 6). 

Spatial characteristics of perceptual compression

Figure 3 shows (for example, observer C.P.) the perceived position of the LT as a function of its presentation time relative to the onset of a saccadic gaze shift made in the 6 different types of gaze shifts studied (explained in the Methods section and in the previous section). Data for the other 2 subjects are shown in Supplementary Figure 2. Each point in a panel is the result of one trial. Figures 3A–3E show data for 40° gaze shifts in the 5 different conditions we studied, including data obtained in the head-restrained (HR) condition (Figure 3A). The 40° amplitude condition was chosen because we could make the important comparison of HU versus HR data, which is not possible for larger gaze shifts because of the ≈45° limit to the ocular deviation in the orbit (Guitton & Volle, 1987). Figure 3F shows results for 60° gaze shifts. The temporal extent of the shaded column in each panel shows the mean gaze shift duration based on a 30°/s velocity threshold (Figure 2). 

Figure 3

 

Raw data for all experiments for one observer (C.P.; for other observers, see Supplementary Figures 2 and 3). (A–F) Perceived positions of a briefly flashed bar are aligned relative to saccade onset (time = 0). Data were collected for a total of six different experimental conditions ((A) 40° head-restrained saccades (HR); (B–E) 40° head-unrestrained gaze shifts (HU) in different conditions; (F) 60° head-unrestrained gaze shifts). The thick horizontal black dashed line indicates the gaze saccade target position, and the gray area highlights the average gaze saccade duration for a given condition. Horizontal gray dotted lines in (B)–(E) indicate the average eye endpoint in space had the head not moved. The average eye endpoint in (F) was 27° (not shown due to axis limits). Vertical dotted lines within the gray areas denote the duration of the eye saccade portion of the gaze shift. Data points (each color indicating a different bar position) represent responses for 4 representative bar positions surrounding the ST. For the entire data set in all experiments, see Figure 4 and Supplementary Figures 4–6. Lines through the data points, for each bar position, were calculated as running averages obtained with a Gaussian filter (σ = 15 ms). (G) In order to show that mislocalization was always toward the gaze (and not the eye) endpoint, we calculated the average difference (±SD) between the perceived bar position and eye endpoint (P − E) versus the average difference between the perceived bar position and the gaze endpoint (P − G), for all conditions (color code shown to right). (H) Compression index (after Lappe et al., 2000) as a function of time relative to gaze saccade onset for the four bar positions shown across all conditions, with color coding as in (G).

The solid lines running through the data points in each panel of Figure 3 show the mean perceived LT positions, calculated as running averages through the data points after smoothing with a Gaussian filter (σ = 15 ms). In agreement with previous results (Honda, 1993; Lappe et al., 2000; Ross et al., 1997), there is a systematic mislocalization of the LT that begins well before gaze shift onset and continues after onset of the movement. 

The data shown in Figure 3A indicate a compression of visual space toward the target of the 40° HR saccadic eye movement, but this interpretation cannot be distinguished from the possibility that the compression effect is determined more generally with respect to the endpoint of a more naturalistic gaze shift that might include head movements. Indeed, for HR movements, the trajectory of the eye saccade and the gaze shift are identical, but for HU gaze shifts, the eye's contribution is less because the head also carries gaze. In this case, the compression could be directed either toward the spatial locus of the eye saccade had the head not moved or toward that of the HU gaze shift. 

Figures 3B–3E show the perceived positions of the LTs when 40° gaze shifts were executed with the head unrestrained in the various conditions. For these experiments, the perceived LT position was, unambiguously, toward the gaze target location (thick black dashed horizontal line) and not the endpoint of the initial eye saccade had the head not moved (indicated by the horizontal gray thick dotted lines). This is particularly evident for the 60° HU condition (Figure 3F) where eye-in-head motion is severely limited to ≈35° (below the abscissa) relative to gaze amplitude. Put another way, 60° is considerably beyond the possible deviation of an eye saccade alone. These results suggest that the spatial target of the gaze, rather than the target of the initial eye movement, determines perceptual compression. Figure 3C emphasizes the generality of this result by showing data in the eye-misaligned condition, in which gaze shift amplitude was still 40° but began with the eye rotated by 20° toward the target of the gaze shift. This manipulation substantially reduced the magnitude of the initial eye saccade relative to the eye-aligned condition, leading to significant differences in eye and gaze kinematics (e.g., in Figures 2A and 2B compare eye traces, red lines). The vertical gray hatched lines in Figures 3B–3F denote the eye saccade duration for each gaze shift. As with the other experimental conditions, in the eye-misaligned condition, compression was directed toward the gaze target. 

In summary, visual observation of the results for subject C.P. suggests that compression was directed toward the gaze target. Similar results were seen for other observers (Supplementary Figure 2), indicating that compression is toward the gaze goal independent of the individual contributions of the eyes and head to the gaze shift command. To assess this more formally, we calculated the mean difference (in degrees) for the period ±50 ms relative to gaze shift onset between the perceived LT position (P) and the gaze and eye (had the head not moved) endpoints (G and E, respectively), for all subjects and experiments (Figure 3G). As expected, these two groups of points (P − G and P − E) had statistically different means (paired t-test, p < 0.001 for all observers), confirming that compressive errors were always closer to the gaze target location than the eye's endpoint in space. 

The compression effect has often been presented quantitatively in the literature using a global compression index, first proposed by Lappe et al. (2000; CI_L, Methods section), that shows how the overall compression measured across all bar positions varies as a function of time. The time course of CI_L is summarized for each experimental condition in Figure 3H for observer C.P. Note in particular that the time at which CI_L returns to unity (at compression end) is approximately constant for all conditions, independent of gaze shift duration. We will examine this property more closely in the next section. 

Temporal properties of perceptual compression

The time course of CI_L in Figure 3H, as well as of the lines through the data points in Figures 3A–3F, suggests that the compression effect began and ended at fixed times relative to gaze onset, despite substantial variation in gaze shift duration. This can be further appreciated from Figure 4, which plots, for subject C.P., the raw data for all bar positions tested in 40-ms intervals, for the 40° eye-aligned (Figure 4A), eye-misaligned (Figure 4B), and 60° (Figure 4C) HU conditions (for the remaining conditions, see Supplementary Figure 3; for remaining observers, see Supplementary Figures 4 and 5). Throughout these plots, the oblique dashed unity line depicts veridical perception. Any deviation away from this line denotes mislocalization, such that perfect compression would be represented as a straight horizontal line through the location of the gaze target (star symbol). In agreement with our previous work (Richard et al., 2009), when the raw data are plotted in this manner they indicate a nonlinear relationship between perceived and real LT positions. As can be seen for the three experiments in Figure 4, the peak compression occurred in the period about 0–40 ms after gaze shift onset. Thereafter, compression declined such that the perceived bar position was essentially veridical by 120 ms after gaze onset, which, in most cases, was before the end of the gaze shift. Indeed, comparing the time course of compression relative to the temporal extent of the shaded columns in Figures 3A–3F suggests that compression ended before the end of long-duration gaze shifts. This observation raises the possibility that while the spatial pattern of compression is toward the gaze target, the temporal profile of the effect may be related to the time course of the eye saccade (vertical dotted lines in each panel of Figures 3B–3F indicate eye saccade end, which can be compared to gaze shift end in all experiments for C.P.). 

Figure 4

 

Perceptual data for all bar positions at different times relative to gaze saccade onset (data shown for C.P.; see Supplementary Figures 4 and 5 for other observers). Data for the 40° eye-aligned (11 bar positions), 40° eye-misaligned (11 bar positions), and 60° eye-aligned (8 bar positions) conditions are shown in (A)–(C), respectively (for the remaining head-unrestrained conditions, see Supplementary Figure 3). Each data point shows the perceived versus real bar positions that occurs at the specified perisaccadic time points (−80, −40, 0, 40, 80, and 120 ms) relative to gaze saccade onset. For each time point, the retinal position of the LT (abscissa) relative to the fovea is adjusted for the average amount the eye has moved. Gray star symbols refer to gaze saccade target position, and the dashed oblique line represents the unity relationship between real and perceived bar positions. Perfect compression is a horizontal line through star symbol. No compression is the unity line. Color coding for different conditions as in Figures 3G–3H.

We therefore examined quantitatively the temporal characteristics of perceptual compression by measuring, for each LT position, the onset and offset times of the compression effect (Figure 5). These were defined as the times, relative to gaze shift onset, at which the absolute magnitude of the CI_L reached 10% of its maximum. Figures 5A–5C and Figures 5D–5F plot, for all observers, the means of these times (±SD) as a function of both mean gaze duration and mean saccade duration, respectively, across all experimental conditions including the data for HR saccades reported previously (Richard et al., 2009). Thus, the 9 points in each panel represent: (1) the 4 HR conditions (open circles), including eye saccades of 10°, 20°, and 30° amplitudes from Richard et al. (2009) and the 40° condition (open squares) from the present study, and (2) the 5 HU experiments, including 4 different 40° conditions (circles) and a 60° eye-aligned condition (triangles; all experiments color coded as in Figures 3G–3H). The results show that the dependence of compression onset times and offset times (Figures 5A–5C) on gaze and saccade duration are similar across observers. Note that all durations were calculated using the velocity criterion described in the Methods section, which gives similar results to the calculation of duration using a spatial error criterion (see Supplementary Figure 6.) Taking observer C.P., for example (Figure 5A), for small 14° saccade durations, compression begins about 50 ms before and ends 70 ms after saccade onset, whereas for the large, long-duration gaze shifts compression begins at about 90 ms before and ends about 90 ms after gaze shift onset. Across subjects, the mean onset of compression for small (14–30°) saccades was 69 ms before the saccade (SD = 15 ms), with compression in these conditions ending on average 91 ms after saccade onset (SD = 9 ms). For larger (40–60°) saccades, these values were 82 ms (SD = 8 ms) and 97 ms (SD = 11 ms). 

Figure 5

 

Compression timing as a function of mean gaze and saccade durations for all observers and experimental conditions. (A–C) Times of compression onset (lower) and offset (upper) ± SD for each observer (C.P., M.G., and S.N., respectively) versus mean gaze duration. Onsets and offsets were defined as the time relative to gaze onset at which the compression index (Figure 3F, see also Lappe et al., 2000) crossed the value of 0.9. (E–F) Time of compression offset (relative to gaze onset) versus eye saccade duration for all experimental conditions (see key in bottom of (B), color coding as in Figure 3). Eye saccade duration was calculated with a 30°/s velocity threshold. Head-restrained data for 14°, 20°, and 30° saccades (open circles) are taken from a previously published data set (Richard et al., 2009).

The data of Figure 5A (upper) also show that there is no compression during any time segment of a gaze shift beyond about 100 ms. Thus, the overall duration of compression (i.e., the difference between the two sets of data points in Figure 5A) for subject C.P. is less than 150 ms for small saccades and saturates at ≈180 ms for large gaze shifts (correspondingly, compression duration saturates at approximately 210 ms for M.G. and 200 ms for S.N.). Put another way, the duration of the compression effect is largely independent of gaze shift duration for amplitudes >30°, despite the large variation in gaze durations due to the different motor strategies implemented across conditions. 

One possible explanation for this result is that the duration of compression is linked to the duration of the eye saccade. In order to test this possibility, we plotted the time of compression offset against the average saccade duration (using, as for gaze, a velocity offset threshold of 30°/s). Figures 5D–5F show that the points for large gaze shifts tend to cluster near the unity line, indicating that perceptual compression appears to end, on average, at roughly at the same time as the eye saccade. However, because there is little correlation between saccade duration and compression offset in 2 out of 3 observers, we cannot say with any certainty that these two factors are linked. Indeed, the duration of compression may be shorter than the duration of the full eye–head gaze shift for other reasons that we return to in the Discussion section. We summarize the temporal findings across all subjects in Figure 6, which plots the mean gaze, saccade, and compression durations for all experimental HU conditions. On average, the gaze duration was always significantly different from both the eye saccade and compression durations (paired t-test, min p < 0.05), while the latter two could not be statistically distinguished from one another. In order to further investigate the temporal properties of the compression effect, we now consider how the marked differences in gaze kinematics between our experimental conditions may influence the strength of compression. 

Figure 6

 

Bar graph comparing the mean durations (±SD) of compression (blue), eye saccade (green), and gaze saccade (red) across observers for all HU experiments. Asterisks indicate a significant difference between the mean gaze duration in relation to either the eye saccade or the compression durations (paired t-test, min p < 0.05).

Effects of gaze velocity on perceptual compression

Previous work has shown that the strength of perceptual compression is correlated with the velocity of head-restrained gaze shifts (Ostendorf et al., 2007). These results showed that subjects who made faster saccades exhibited stronger perceptual compression. Whether this result is due to saccadic eye velocity per se is unclear however, as other intersubject differences could have contributed to the results. Because eye saccades are highly stereotyped (Bahill, Bahill, Clark, & Stark, 1975), the effect of eye velocity cannot be isolated in the head-restrained condition in any one subject, although head velocity can be controlled voluntarily. We therefore examined the influence of gaze shift velocity by instructing subjects to execute either slow or fast 40° head-unrestrained gaze shifts (the slow and fast conditions shown in Figures 2C and 2D), which allowed us to examine the effect of velocity on perceptual compression. 

Figures 7A and 7B show the average gaze and head position and velocity traces for one observer (see Supplementary Figure 1 for others) for the slow (red) and fast (blue) conditions. As is evident from Figure 7B, peak head velocity declined by roughly 50% from the fast to the slow experiments, which resulted in an ≈35% reduction in peak gaze velocity. To quantify the difference in compression between these two conditions, we subtracted the magnitude of peak compression (defined globally by the CI_L; see Methods section) in the slow condition from that obtained in the fast condition. These values were 0.08 for observer M.G. (CI_L curves are shown for M.G. in Figure 7C), 0.02 for observer C.P., and 0.06 for observer S.N., indicating that these CI_Ls are nearly identical. The relatively small differences in peak compression are also evident in the time course shown in Figure 3H, suggesting that velocity per se is not an important factor in determining the strength of the compression effect. The lack of a strong effect of gaze velocity on perceptual compression is consistent with the conceptual basis of our previously published model (Richard et al., 2009), which is based strictly on the spatial relationships between the LTs and the gaze targets. 

Figure 7

 

Kinematic and perceptual differences between the 40° fast and slow conditions for observer M.G. who produced the biggest difference between the two conditions. Mean (±SD in shading) gaze position (A) and gaze velocity (B) traces for the fast (blue) and slow (red) conditions, with the corresponding head movement and velocity trajectories shown in the inset. (C) Compression index (after Lappe et al., 2000) as a function of time relative to gaze onset for the fast (blue) and slow (red) conditions.

Spatial model of compression

We have previously shown (Richard et al., 2009) that the experimental observations on perceptual compression can be predicted by a model that assumes that the magnitude of compression is proportional to the distance between two loci of neural activity—visually evoked activity specifying a bar's position relative to the fovea in visual space and a motor command specifying the saccade vector—on a retinotopically encoded logarithmic map. The data in Figure 8 compare the perceived and actual positions of various LTs, presented to observer C.P., at gaze shift onset, with the points in each panel representing the raw data and the solid lines indicating the model fits. For the three conditions in which we tested LT positions in the visual hemifield contralateral to the gaze shift (open gray circles, Figures 8A, 8C, and 8F), the compression effect appears to be conserved, as suggested by previous work (Morrone et al., 1997). 

Figure 8

 

Conceptual basis for our model and model fits for compression data at gaze onset for all experimental conditions (for summary of remaining observers, see Table 1). (A) Schematic of how the neural activity attributable to the bar (visual hill) and a 40° gaze command (motor hill) might look if both were represented on a single, logarithmic retinotopic map, such as the one found in the superior colliculus. Model (from Richard et al., 2009) invokes interactions between the two hills of activity. (B–G) As in Figure 4, each data point (observer C.P.) represents the perceived position, corresponding to the mislocalization that occurs for each bar presented at gaze saccade onset. Model fits are shown as the black lines, with p values given for appropriate degrees of freedom (df; defined as number of bars − 1). The gray star symbols refer to gaze saccade target position, and the dashed oblique line represents the unity relationship for real versus perceived bar positions. Color code for different conditions is consistent with our other figures (originally defined in Figure 3). Perfect compression would be a horizontal line through star symbol. Panels (B), (D), and (G) identifies data points in the contralateral visual field as gray circles. These were not included in the model fits.

The goodness of fit of our model to the data for LTs in the ipsilateral hemifield to the gaze shift was assessed using a reduced chi-square statistic (χ _R ²), which is the Pearson's χ ² value normalized by the number of degrees of freedom (df; defined as the number of data points minus the number of free parameters) in the test model (Taylor, 1997). This statistic allowed us to compare models with different numbers of free parameters, since fewer bar positions were tested in the 60° condition due to screen size limitations (resulting in 8 df compared to 11 df in some 40° conditions; see below). Our spatial model has only two parameters per observer, but nonetheless the fits were good, as revealed by visual inspection of Figure 8 (parameters and statistics for all observers are summarized in Table 1). Similarly, close fits were obtained using the same model for smaller saccades (Richard et al., 2009), indicating that this formulation captures the magnitude and spatial distribution of perceptual compression across a wide range of head-fixed and head-unrestrained conditions. As the mathematical formulation of the model was not able to accommodate contralateral LT positions, we excluded those data points from model fitting and from the reduced chi-square statistic. 

Discussion

Prior studies in head-restrained (HR) human subjects have shown that visual objects that are briefly presented at different positions in visual space, just before and during an eye saccade, are perceived to be located near the saccade endpoint (Lappe et al., 2000; Ross et al., 1997). This phenomenon has been called the perisaccadic perceptual compression of visual space. Here, we have shown that perceptual compression occurs for head-free gaze shifts and that it is consistently directed toward the spatial target of the gaze shift rather than to the endpoint of the eye saccade (Figure 3). In contrast, the timing of compression is only loosely related to the timing of the gaze shift, with compression ending in most cases well before the end of the movement (Figure 5). Voluntary modulation of the speed of a head-free gaze shift had little effect on perceptual compression (Figure 7), suggesting again that the spatial aspects of the movements, rather than the details of the kinematics, are paramount in determining the percept. These results were well predicted by our previously published model (Richard et al., 2009), which ascribes perceptual compression to the interaction of visual and motor signals on a logarithmic map of visual space (Figure 8). 

On the problem of relating compression to head-unrestrained gaze shifts

To show that compression is related to the endpoint of a gaze shift, and not to that of the eye component alone, we tested subjects using 40° gaze shifts in which the eye started eccentrically, off-center at 20° in the orbit (our misaligned condition). In these trials, it is impossible for the eye alone to reach the target due to neural limits on the “oculomotor range” (Guitton & Volle, 1987). Therefore, a contribution from the head is absolutely necessary. This is also true for 60° gaze shifts, because a head-fixed 60° eye saccade beginning at orbital center is impossible to generate. Thus, our results for the 40° misaligned and 60° gaze shifts show that peak compression occurs on the gaze saccade endpoint and not on the goal of the eye saccade component. 

We also wanted to compare same-amplitude HR eye saccades and HU gaze saccades to obtain a direct estimate of the effects of gaze kinematics. This experiment could only be done for 40° gaze shifts because for smaller gaze movements the head contributes little and for larger movements the eye alone cannot reach the goal. The resulting compression effects were highly similar across conditions, despite large variations in the eye saccade trajectories (Figure 2). These differences in 40° gaze trajectories across conditions argue against early gaze control models, which suggested that the head's contribution during HU movements is subtracted out by the vestibulo-ocular reflex (VOR; reviewed in Freedman, 2008; Guitton et al., 2004). In fact, in the majority of subjects, the gain of the VOR is thought to actually reduce during gaze shifts (e.g., Guitton & Volle, 1987; Roy & Cullen, 2004). 

Neurophysiological implications

Compression of visual space has been linked to the dynamic remapping of receptive fields in parietal area LIP, wherein visual receptive fields shift before the onset of saccades from a coordinate frame linked to the initial fixation point to a new frame tied to the post-saccadic fixation point (Husain & Jackson, 2001; Ross, Morrone, Goldberg, & Burr, 2001). However, to explain compression, the recent models of Hamker et al. (2008), Richard et al. (2009), and Zirnsak et al. (2010) do not require, for all positions in space, a shift of the reference frame in the direction of the saccade. Rather, they require a different neural substrate that contains three key elements: (1) a copy, or corollary discharge, of the saccade motor signal; (2) a visual signal encoding the object of interest; and (3) the encoding and interaction of both these signals on a logarithmically encoded retinotopic map. Here, we add another constraint: The corollary discharge must encode the goal of gaze shifts made by combinations of eye and head movements. Thus, the neural substrate for compression probably involves structures that carry eye and head motor signals. Of course, this does not rule out the idea that compression and remapping are related, as suggested in Zirnsak et al. Furthermore, remapping models have been shown to explain the perceived remapping of time, as well as space (Binda, Cicchini, Burr, & Morrone, 2009; Hamker et al., 2011; Maij, Brenner, & Smeets, 2011). 

Much current evidence (reviewed in Hamker et al., 2008; Richard et al., 2009) suggests that the superior colliculus (SC) is the source of the ascending corollary discharge signal that modifies activity in visual areas, such that perception is biased toward the saccade target. First, the SC sends ascending signals that reach extrastriate visual areas either via the pulvinar or via thalamus and frontal cortical structures (Berman & Wurtz, 2010; Sommer & Wurtz, 2004). Stimulation of the SC, at intensities below threshold for triggering saccades, induces a shift of attention and an increase in visibility in the movement field associated with the stimulated site (Cavanaugh & Wurtz, 2004; Ignashchenkova et al., 2004; Muller et al., 2004). In the context of our results, which show that compression is unambiguously toward the gaze target location, a role for this kind of feedback signal is consistent with recent behavioral studies that demonstrated a functional role for attentional modulation before and during eye–head gaze shifts (Khan, Blohm, McPeek, & Lefevre, 2009). Furthermore, reversible inactivation of the SC leads to impairments in subjects' ability to selectively control attention to select a visual stimulus that will inform a perceptual judgment, even in the absence of eye movements (Lovejoy & Krauzlis, 2010). Finally, in a human patient with a lesion of the SC, there is an impairment of covert spatial attention (Sereno, Briand, Amador, & Szapiel, 2006). 

In the HU monkey, the discharge of collicular neurons encodes eye–head gaze shifts (Choi & Guitton, 2006, 2009; Freedman & Sparks, 1997). This property, taken together with the arguments presented above, provides strong evidence that compression is related to a corollary discharge signal of the gaze motor command that ascends to extrastriate areas and whose source is the SC. 

On the temporal saturation of the compression effect

We showed that the compression effect begins at ∼50 ms before 14° gaze shifts and this lead time increases to a saturation value of ∼80 ms before the onset of gaze shifts ≥40° (Figure 5). It is of interest to link this property to the suggestion by Hamker et al. (2008) that the ascending corollary discharge signal originates in SC cells. Sommer and Wurtz (2004) described the neural signals in a pathway that arises in the SC and, via relay neurons in the mediodorsal nucleus of the thalamus (MD), ends in the frontal eye field (FEF). Their data were obtained for HR saccades mostly in the 10–20° range. They found that SC neurons that project to MD have a peak in their burst discharge that occurs on average 9 ms before saccades, with the onset of the burst occurring 85 ms before saccades. By comparison, the peak discharge and burst onset of FEF recipient neurons occur 3 ms and 54 ms before saccades, respectively. These numbers can be compared with our observations that compression, for saccades 10–20°, starts 50–80 ms before saccades and peaks near saccade onset. Thus, the timing of discharges along the SC–MD–FEF pathway is compatible with a role in initiating compression. 

It is more difficult to speculate on whether discharges in this pathway are related to the end of compression, which we showed occurs 80–100 ms after the onset of 10–20° HR saccades and which saturates, for large HU gaze saccades, at 80–130 ms, depending on the subject (Figures 5D–5E). Sommer and Wurtz (2004) do not report on the timing of FEF bursts relative to saccade end. However, Hamker et al. (2008) suggested that the end of compression is determined by SC cells whose discharge ends at gaze shift end (“clipped” cells). Neural recordings in HU monkeys have indeed shown that burst duration increases with gaze saccade duration and that there is a tight correlation between burst end and gaze shift end (Choi & Guitton, 2009; Freedman & Sparks, 1997). However, the burst of SC cells that encode large ∼60° HU gaze shifts outlasts the end of gaze shifts, and at gaze end, their firing frequency is still about 1/3 of the peak value (Choi & Guitton, 2009). Thus, the assumption that compression is related to ascending SC–FEF signals (Hamker et al., 2008; Hamker et al., 2011) would require a threshold, somewhere in that pathway, on SC gaze saccade-related discharges below which compression is not affected. In this scenario, the duration of compression would not be determined by the duration of the eye saccade, as is suggested by our data in Figure 5, but rather by the characteristics of SC ascending activity and the postulated threshold. 

Limitations of the model

The model presented here and in Richard et al. (2009) provides a heuristic for relating perceptual compression to oculomotor structures in the brain. Importantly, it can be expressed in a simple-closed form equation that can be fit to psychophysical data. However, as a consequence of the simplicity of its formulation, the model has a number of limitations. First, the model output is undefined for LTs flashed far into the visual hemifield opposite the saccade direction, and so we cannot use it to make predictions about this experimental manipulation (Figure 8). Second, the model predicts similar compression for LTs flashed along axes parallel and orthogonal to the saccade direction, while the data (Kaiser & Lappe, 2004) are better captured by a model that introduces a small anisotropy in these different axes (Hamker et al., 2008). Finally, as mentioned above, the model does not provide an obvious explanation for the perceived compression of time, which more detailed models can reproduce in conjunction with spatial compression (Binda et al., 2009; Maij et al., 2011). 

Comparison with prior studies

Our data showing that compression was directed toward the gaze target in conditions where a head movement necessarily contributed to the gaze shift is not in accord with the observations of Jackson et al. (2005), who reported that the compression effect is absent in a patient with ophthalmoplegia (no eye movements since birth), who orients with only head movements. These authors argued that the lack of compression might have been due to the patient's slow head saccades, which were about ten times slower than a typical eye saccade of the same amplitude. However, we have shown that gaze shift velocity, when it is measured within subjects, modulates perceptual compression very little (Figure 7). An alternative possibility is that the patient generated head movements via the motor cortex, in which case the ascending corollary discharge from the SC might have been unable to influence visual perception. 

Supplementary Materials

Supplementary PDF - Supplementary PDF 

Acknowledgments

This work was funded by grants from NSERC (341534-07) and the EJLB Foundation to C.C.P. and CIHR (MOP-9222) to D.G. The authors also wish to thank subjects M.G. and S.N. for their long-standing devotion to the study. 

Commercial relationships: none. 

Corresponding author: Alby Richard. 

Email: [e-mail]. 

Address: Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC H3A 2B4, Canada.