Evolution has endowed primates with a highly specialized fovea, which allows them to perform detailed and sophisticated visual processing, albeit restricted to a small fraction of the visual field. They have also developed many specialized visual cortical areas, that greatly emphasize the representation of the central visual field and that support an impressive array of perceptual capabilities. In parallel, primates display a remarkable repertoire of oculomotor behaviors that take advantage of their powerful foveal processing and allow them to use their eyes to locate, acquire, and pursue objects embedded in complex, dynamic visual scenes. Perhaps these perceptual and eye-movement mechanisms evolved independently. However, it would seem more likely that the new visual and oculomotor capabilities, unique to primates, evolved together as integrally linked systems.
The papers in this special issue of the Journal of Vision examine links between eye movements and visual perception. They focus on three related questions.
First, how do eye movements affect visual performance? Eye movements can enhance performance under some circumstances (e.g. performance in a steering task is improved when drivers’ gaze movements are unconstrained,
Wilkie & Wann, 2003; fixational eye movements improve visual discrimination performance,
Rucci & Desbordes, 2003), yet provide little or no advantage under other conditions (e.g. direction discrimination judgments are unaffected by pursuit eye movements,
Krukowski, Pirog, Beutter, Brooks, & Stone, 2003; bisection judgments are virtually unaffected by fixation shifts,
Trommershauser, Maloney, & Landy, 2003). Perceptual performance appears to depend on the most reliable signal available, whether the signal is sensory or motor in origin. For example, eye movements can improve perceptual performance in both slant and depth estimation tasks when objects are widely separated in space (
Backus & Matza-Brown, 2003;
Zhang, Berends, & Schor, 2003), or when disparity signals are noisy (
Berends, Zhang, & Schor, 2003). Furthermore, the dependence of oculomotor behavior on the visual task suggests that eye-movement strategies are tailored to acquire specific visual information (
Welchman & Harris, 2003).
Second, to what extent are eye movements limited by raw sensory signals or by higher-order perceptual signals? Two studies that examine the pursuit responses to perceptually ambiguous stimuli reveal that pursuit eye movements are directly related to perceptual choices on a moment-by-moment basis, even when the stimulus remains completely unchanged (
Madelain & Krauzlis, 2003;
Stone & Krauzlis, 2003). Furthermore, pursuit is subject to the same motion processing limitations as perception (
Watamaniuk & Heinen, 2003). Using illusions, studies found that both the conjugate (
McCarley, Kramer, DiGirolamo, 2003) and vergence (
Both, Ee, & Erkelens, 2003;
Sheliga & Miles, 2003) components of voluntary saccades respond to perceived target location and depth even when the percept is in conflict with the raw retinal images.
Third, to what extent is visual perception driven by retinal signals versus oculomotor signals? Several studies found that reliable and precise visual percepts of motion can be driven by an oculomotor signal with little or no retinal motion (
Freeman, Sumnall, & Snowden, 2003;
Krukowski et al, 2003) or that depth judgments can be influenced by oculomotor signals, even when in conflict with retinal disparity (
Backus & Matza-Brown, 2003;
Nawrot, 2003). Efference-copy signals can also foster perceptual mis-localizations, even when the responsible eye movement is not perceived (
Blohm, Missal, & Lefevre, 2003). Hamker (
2003) provides a model mechanism by which motor signals might be fed back from higher-order cortical areas to influence processing in earlier visual areas.
Although this special issue is not meant to provide a complete overview of the current status of this field, it does highlight some interesting and important recent examples of links between oculomotor and visual behavior. There are, we believe, three take-home messages. First, oculomotor strategies can affect visual performance either directly by recruitment of efference-copy information or indirectly by foveation and stabilization of the retinal image, but performance effects will depend on the salience of the visual and efference-copy cues, and on the demands of the task (see also e.g.,
Zelinsky & Sheinberg 1997;
Findlay, 1998;
Hooge & Erkelens, 1999;
Eckstein, Beutter, & Stone, 2001). Second, saccadic, pursuit, and vergence eye movements are strongly influenced by higher-order visual processing related to perception and cognition, and cannot be explained by retinal inputs alone (see also e.g.,
Steinbach, 1976;
Khurana & Kowler, 1987;
Ringach, Hawken, & Shapley, 1996;
Krauzlis & Stone, 1999). Third, feedback of motor commands, long known to play a critical role in controlling oculomotor behavior (
Robinson, 1981), plays a critical role in visual perception as well (see also e.g.,
Pola & Wyatt, 1989;
Freeman & Banks, 1998;
Turano & Heidenreich, 1999).
A major conclusion is that computational models of primate visual perception need to be extended to incorporate an explicit role for eye movements, together with their associated attentional shifts and motor commands. Traditional linear-system models of oculomotor behavior (e.g.
Robinson, 1981;
Lisberger, Morris, & Tyschen, 1987), which limit visual processing to subtraction and differentiation of retinal signals and a few static non-linearities, must incorporate the higher-order, fundamentally non-linear visual processing associated with perception. In sum, the two fields of visual psychophysics and oculomotor behavior, and their associated neurophysiological counterparts, need to coalesce. The picture emerging is that perception is a sensorimotor process, the final step in an interactive dance between sensation and action.