The extended horopter: Quantifying retinal correspondence across changes of 3D eye position

Abstract

The theoretical horopter is an interesting qualitative tool for conceptualizing binocular correspondence, but its quantitative applications have been limited because they have ignored ocular kinematics and vertical binocular sensory fusion. Here we extend the mathematical definition of the horopter to a full surface over visual space, and we use this extended horopter to quantify binocular alignment and visualize its dependence on eye position. We reproduce the deformation of the theoretical horopter into a spiral shape in tertiary gaze as first described by Helmholtz (1867). We also describe a new effect of ocular torsion, where the Vieth–Müller circle rotates out of the visual plane for symmetric vergence conditions in elevated or depressed gaze. We demonstrate how these deformations are reduced or abolished when the eyes follow the modification of Listing's law during convergence called L2, which enlarges the extended horopter and keeps its location and shape constant across gaze directions.

Introduction

Based mainly on ideas developed independently by Vieth (1818) and Müller (1826), the theoretical point horopter has been devised as a theoretical tool for simplifying binocular vision geometry. Assuming there exists some correspondence between retinal loci in the two eyes, one can ask what object locations in space would project onto corresponding retinal points. The set of all of these spatial locations is called the point horopter. Other definitions have occasionally been used including the haplopia or singleness horopter, which is the set of points in space that, empirically, are seen single or fused. Here, we will focus on an extension of the point and haplopia horopters; for an account of other types, see the review chapter by Tyler (1991). With this extended horopter, we can clarify the influence of eye movements on stereopsis by charting the changing shape and location of the horopter surface. 

To model these changes, we need an optical model of the eye and a correspondence mapping between the two retinas, as well as a model of 3D eye movement patterns (see below). A common, simplified model of the eye consists of a single lens with the nodal point located in the center of the retinal sphere (LeGrand & ElHage, 1980). The nodel point is assumed to coincide with the center of rotation for eye movements. Retinal locations can then be specified by their angular distance from the line of sight, the projection line from the fovea, independent of the actual shape of the eye. 

The simplest assumption for the pattern of retinal correspondence is that the pairing is between points that align when the two retinae are superimposed. These corresponding points then have equal angular distance from the line of sight in both eyes. Retinal loci that correspond in this way are sometimes called identical points. We can then define the horopter as all the points in space that project onto corresponding retinal loci in this sense. 

What does this surface look like? When the eyes are oriented in parallel, and are looking at a target infinitely far away, the projections from all pairs of corresponding points are parallel because they are at equal angles from the parallel lines of sight. Any two parallel lines will intersect at infinity, so the point horopter for far viewing is at infinite distance in all directions. 

Eye movements

For fixation on a near target, projection lines in the visual plane (i.e., the plane containing the two eyes' nodal points and the fixation target) will intersect, regardless of the vergence angle of the two eyes. And as Vieth (1818) and Müller (1826) have shown independently, all the intersections of identical points' projections lie on a circle through the fixation target and the two nodal points called the Vieth–Müller circle. 

Above and below the visual plane, intersections exist only in the midsagittal plane, which is the plane orthogonal to the baseline (a line connecting the two eyes' nodal points) and halfway between them. Any object located in this plane will be at an equal distance from both eyes, and objects located outside the plane will be at different distances from the left and right eyes. The visual elevation angle of an object at a given height above or below the visual plane scales with its distance. This means that only objects in the midsagittal plane can have equal elevation angles and project onto identical points. In fact, there is only a single line in the midsagittal plane that projects onto corresponding retinal points for a converged eye position (Helmholtz, 1867). This line is orthogonal to the visual plane and intersects the Vieth–Müller circle and has been called the theoretical vertical horopter (Howard & Rogers, 2002). 

This geometric description of the theoretical horopter is independent of horizontal eye position within the visual plane. A change in version will shift the fixation point along the Vieth–Müller circle, leaving the horopter entirely unchanged, whereas a change in vergence will change the radius of the circle, but not the basic geometry. If for elevated or depressed fixation targets the eyes were to simply rotate vertically, this would leave all projections unchanged. The theoretical horopter would still be a circle in the visual plane—now tilted with elevation—and a vertical line perpendicular to it. 

But ocular rotations have three degrees of freedom. When gaze direction is expressed in horizontal and vertical angles, whether it is in the Helmholtz or Fick coordinate system (Haslwanter, 1995), there is an additional rotational component around the line of sight called torsion. In human eye movements, the amount of torsion is a function of the gaze angle, a fact known as Donders' (1848) law. The actual torsion angle for monocular viewing conditions is specified by Listing's law. In its geometrically simplest form, Listing's law specifies that at each gaze direction, the eye is positioned as though that gaze direction had been reached from a specific primary gaze position by a single rotation around an axis in a head-fixed plane orthogonal to this primary position. This plane of axes is called Listing's plane. With Listing's law, primary gaze determines 3D eye position for all gaze directions. Commonly, eye positions are expressed in rotation vector form, where the direction of the vector specifies the axis of rotation from primary position and its length specifies the rotation angle. All these eye position vectors will lie in Listing's plane as well. 

For binocular viewing, a deviation from Listing's law has been described (van Rijn & van den Berg, 1993; Mok, Ro, Cadera, Crawford, & Vilis, 1992). This has been called the binocular extension of Listing's law, or L2. In L2, the torsional angle of each eye depends on both gaze direction and convergence of the eyes. In terms of the description of Listing's law above, the Listing's plane of eye position vectors changes with vergence, rotating temporally in each eye by half the vergence angle. Because primary position is orthogonal to Listing's planes, this also means that the eyes' primary positions diverge by half the convergence angle. The resulting pattern of torsion affects the horopter. 

Studies of the shape of the horopter for fixation targets outside the horizontal plane and at finite viewing distances have been exceedingly rare. The only theoretical analysis we are aware of is provided by Helmholtz (1867) in his Treatise on Physiological Optics, where he describes the horopter for tertiary binocular gaze as a line in the midsagittal plane spiraling around to approach the Vieth–Müller circle from above, meeting it at the eye's nodal point, reemerging in symmetrical fashion from the other eye's nodal point, and bending away downward and back to vertical (see Figure 4). But Helmholtz's description of the mathematical reasoning leading him to this description is impenetrably dense. 

The extended horopter

While the theoretical horopter helps clarify binocular correspondence, its usefulness is limited by the fact that for almost all eye positions it consists of only two lines. This restriction clashes with reality, in which binocular fusion is not limited to such narrow parts of the visual field. And it restricts the use of the horopter to the mapping of two retinal major circles, one in the visual plane and the other roughly vertical on the retina. 

We propose to extend the definition of the theoretical horopter, so that it becomes a full surface in 3D space for any fixation target. With this surface we will demonstrate how different oculomotor strategies like Listing's law and L2 affect binocular alignment of retinal images. 

Methods

All simulations were carried out using Matlab programming environment, Version 6.5 (Release 13). 

To construct a point on the classical theoretical horopter, retinal points in the two eyes are projected out through the eyes' nodal points to find the two rays' point of intersection, if one exists. This is mathematically equivalent to solving an equation system (see 1). The fact that the horopter does not exist for most pairs of retinal points in most eye positions corresponds to the mathematical fact that most such equation systems have no solution. 

They do, however, have approximate solutions, which can be found using extension of the matrix inversion operator, called the Moore–Penrose pseudoinverse (Moore, 1920; Penrose, 1955), which finds approximate solutions to any system of linear equations. Using this pseudoinverse and the following procedure, we can find unique points of the horopter for any pair of retinal locations, and in any eye position (see Figure 1). The mathematical fact that this is an approximate rather than an exact solution reflects the physiological fact that not only objects projecting onto corresponding points are seen as single, but so are all projections in a retinal area around correspondence (i.e., Panum's fusional area). 

Figure 1

 

Retinal correspondence. When the eyes fixate F, the classical theoretical horopter is the Vieth–Müller circle (the black curve). Given any pair of identical retinal loci in the visual plane (e.g., v_l and v_r), their projection rays (green) will intersect on the circle (e.g., at point V). But when the pair of corresponding identical loci (c_l and c_r) lies outside the visual plane, or for nonidentical corresponding point pairs in general (c_l and c_r), the projection rays (cyan) do not intersect anywhere. We define the horopter point H for such correspondence pairs to be on the shortest connecting line between their two projection rays. H is located such that its retinal projections (h_l and h_r) are equidistant from c_l and c_r; that is, the distance d(c_l − h_l) = d(c_r − h_r), although the difference vectors c_l − h_l and c_r − h_r usually differ in direction.

Using the pseudoinverse to solve the intersection equation, we obtain the pair of points on the projection rays with the shortest connecting line. The point of the extended horopter is located on this connecting line (drawn in red in Figure 1) so that its projections onto the two retinas have the same angular distance from the respective corresponding points. 

When the two rays intersect, the connecting line has zero length and H coincides with the actual intersection of the rays, and thus the classical horopter point. Thus, the extended theoretical horopter is truly an extension of the classical theoretical horopter and contains it for any eye position. 

Points that are part of the extended, but not the classical, theoretical horopter do not project exactly onto the corresponding retinal points used to determine them. They are slightly disparate. When we try to quantify this disparity, we discover a problematic ambiguity in the definition of retinal disparity. If we start with a mapping of corresponding points on the two retinas that we define as a having zero disparity, and then try to define the disparity of a visual target relative to that zero mapping, there is no unique way of doing so. This problem in the definition of disparity arises even within the visual plane whenever the empirical horopter is not the isovergence line (i.e., Vieth–Müller circle). We will expand on this problem in the discussion section. 

For practical purposes, we need to decide on some definition of binocular disparity, even if it cannot be fully justified theoretically. The most obvious such definition has disparity measured relative to identical retinal points and ignores the empirical correspondence mapping for measuring disparities. This, however, would mean that even on the Vieth–Müller circle disparity would not be zero because in general corresponding points are not identical points. This is contrary to the common notion of disparity as coding depth relative to the horopter. 

Instead, we here define the disparity of a point of the extended horopter as the average of the angular differences in the left and right eye between the projection of this point onto the retina and the location of the corresponding point associated with that horopter point (i.e., the average of h_r − c_r and h_l − c_l shown in Figure 1). While the spatial location of the point on the extended horopter and the disparity vector thus defined are independent of the coordinate system used, the same is not true for the horizontal and vertical components of the vector. In our simulations, we use a retinal Helmholtz (1867) coordinate system, corresponding to the epipolar line set for infinite distance viewing. 

This disparity described in Helmholtz (1867) coordinates can be used to determine the part of the extended horopter that exceeds some set limit for disparity from the corresponding points. This can either be achieved by setting an upper limit to the length of the disparity vector, or by applying a direction-dependent disparity limit. If the horopter is thought of as containing all spatial locations at which objects can be fused, these limits define Panum's fusional area. If these fusional areas are based on empirical data, the extended horopter becomes the haplopia or singleness horopter (Ogle, 1964). 

Finally, to obtain a single, numerical measure of the quality of binocular alignment, the area of the extended horopter, as defined above, and as restricted by the disparity limit, is computed, as projected onto a unit sphere to compensate for distance (see 1 for details). For any given disparity limit, this area then represents the portion of visual directional space in which objects can be placed to appear single. For other directions, the minimal retinal disparity possible will exceed the retinal disparity limit. 

This definition allows for the possibility that the pattern of retinal correspondence might be gaze dependent. But because Hillis and Banks (2001) have demonstrated convincingly that earlier reports of changes in retinal correspondence with changes in vergence can be explained by various confounding factors, and that when they controlled for these factors they found no evidence for such changes, we assume correspondence to be fixed for all simulations presented here. 

Results

This extended horopter is quite versatile, in allowing irregularly shaped Panum's areas as well as arbitrary patterns of retinal correspondence. We will start this discussion with one of the simplest cases, correspondence between identical points, and any disparity limits applied will be constant across the retina and uniform in direction, that is, the restriction will be placed on the length of the disparity vector, independent of its direction and its location on the retina. 

Effects of gaze and ocular torsion

Using the above definition, we obtain the extended horopter surface for a gaze position straight ahead, at a vergence angle of 20 deg (see Figure 2). The surface color represents the length of the disparity vector for each point (see Figure 3). Clearly visible are the Vieth–Müller circle and the classical vertical horopter in black, representing zero disparity. The small inset in all the horopter figures shows the same extended horopter restricted to disparities of 0.5 deg or less, emphasizing the classical theoretical horopter contained within the extended one. 

 

Figure 2

 

The horopter when the eyes fixate a target in the midsagittal plane at eye level and the vergence angle is 20 deg. Drag with your mouse to rotate and see the 3D shape of this surface. Units for all three axes in this and all other figures are in centimeters with the origin located in the center of the right eye. The dashed green line represents the straight-ahead direction.

Figure 3

 

Color scale for the horopter plots. The scale ranges from 0 deg (black) to 10 deg (white) of retinal disparity.

Changing gaze direction to a tertiary position and leaving vergence unchanged at 20 deg, the horopter's shape changes dramatically, as shown in Figure 4. The zero disparity lines, denoting the classical theoretical horopter, now show the spiral shape as originally described by Helmholtz (1867). 

 

Figure 4

 

The horopter for Listing's law, with gaze at 15 deg eccentricity both horizontally and vertically, and at a vergence angle of 20 deg. Note the bent shape of the black region of zero disparity, corresponding to Helmholtz's (1867) description.

Even if gaze is kept central horizontally, only elevating or depressing the eyes in symmetrical vergence, the horopter changes. Figure 5 shows the horopter at a gaze elevation of 40 deg to better visualize the changes. The Vieth–Müller circle is tilted out of the visual plane, and back toward the horizontal plane. 

 

Figure 5

 

The horopter for Listing's law, with gaze at 40 deg elevation, and at a vergence angle of 20 deg. The horopter's geometry is similar to that of Figure 2, but the Vieth–Müller circle is tilted out of the visual plane toward the horizontal plane.

Both the spiral shape in Figure 4 and the vertical tilt in Figure 5 are not due simply to the change in 2D gaze direction. If the eyes were simply to rotate around the interocular axis to an elevated position at constant vergence, the projection geometry would not change at all, and the Vieth–Müller circle would stay in the visual plane. Isovergent horizontal gaze movements then would have the same effect at any gaze elevation and not change the horopter shape or position. 

The deviations depicted in Figures 4 and 5 are due to ocular torsion. Specifically, the eyes have been rotated according to Listing's law in producing both figures. The torsion produced by Listing's law is necessary to obtain the spiral horopter described by Helmholtz (1867), and it also produces the horopter tilt in Figure 5. This tilt is due to the cyclovergence component of Listing's law in symmetrical gaze, as described by Howard and Rogers (2002). 

When the eyes move according to L2 instead of Listing's law, the resulting pattern (shown in Figure 6) is that of Figure 2. In other words, the torsion prescribed by L2 keeps the classical theoretical horopter a circle in the visual plane and a vertical line in the midsagittal plane, making its basic shape invariant across gaze movements. 

 

Figure 6

 

The horopter for L2, with gaze at 15 deg eccentricity both horizontally and vertically at a vergence angle of 20 deg. The region of zero disparity is the same shape it was in Figure 2.

Figure 7 shows the retinal disparity pattern for a tertiary gaze direction, both for Listing's law and L2, and it illustrates how L2 improves alignment. 

Figure 7

 

Retinal disparity patterns in tertiary gaze. Plots are in retinal Helmholtz (1867) coordinates. Shown is a grid of identical retinal corresponding points (red x's), and for each location in this grid, projections of the horopter point onto the left (green circles) and right retinas (red circles). Fixation is at 20 deg vergence and 15 deg up and to the left. The fovea is marked with a filled red circle. In the grey zones, disparity falls below a 0.5 deg uniform limit. (A) Pattern for Listing's law, corresponding to the horopter shown in Figure 2. Note the zero disparity lines forming the spiral shape and the gradient in vertical disparity. (B) Pattern for L2, corresponding to Figure 6. Disparities are smaller overall than in panel A. The zero disparity lines form a cross straight ahead. Because L2 produces a small amount of Helmholtz (1867) cycloversion, the Vieth–Müller circle does not project onto the horizontal retinal meridian. It does, however, lie in the visual plane.

To quantify the difference between Listing's law and L2, we computed the size of the horopter with a uniform disparity limit of 0.5 deg, in units of retinal area, for both motor programs and across a range of eye positions. This disparity limit is not meant to reflect the actual extent of Panum's area across the retina. Rather, we chose its size such that significant parts of the visual field would not have an associated horopter point. This allowed for the difference between the motor programs to have a clear effect on the extent of the horopter. This does not mean that for larger and more realistic sizes of Panum's area the type of oculomotor control does not make a difference, but rather that the advantage of smaller disparities for vision enabled by torsional changes would not be captured by the horopter size measure if all disparities were to fall below Panum's limit. 

Figure 8 shows a comparison between Listing's law and L2 for different gaze directions at a vergence angle of 16 deg. For pure horizontal gaze changes, torsion is the same for Listing's law and L2; for pure vertical gaze changes, whereas Listing's law produces cyclovergence and L2 does not, as we have seen, the effect of this cyclovergence is a tilt of the Horopter out of the visual plane back toward the horizontal plane. This does not change the size of the fusable area. Thus, the horopters for L2 and Listing's law have the same size for pure horizontal and vertical gaze directions, but everywhere else the horopter is larger for L2 than it is for Listing's law. This difference becomes bigger as eccentricity increases. 

Figure 8

 

Size ratio between the horopter for Listing's law and L2, depending on gaze direction, at 16 deg of vergence. The disparity limit used to restrict the horopter size was 0.5 deg of uniform retinal disparity.

Figure 9 shows that the advantage of L2 over Listing's law for horopter size also depends on vergence and is biggest in the periphery of the visual field. We plot the horopter size ratio, averaged across the central ±30 deg of horizontal and vertical gaze. We also included an oculomotor strategy in the simulations where ocular Helmholtz (1867) torsion is kept at zero, and one where Listing's planes are tilted between those for L2 and Listing's law, at 60% of L2. 

Figure 9

 

Absolute (solid lines) and normalized to the size for no Helmholtz (1867) torsion (dashed lines) horopter size as a function of vergence angle, for a disparity limit of 0.5 deg averaged over gaze directions of ±30 horizontally and vertically. Shown are horopter sizes for no Helmholtz (1867) torsion (“No HH”, blue), L2 (red), a realistic 60% mix of L2 and Listing's law (green), and pure Listing's law (turquoise). The blue (No HH) plot for absolute horopter size is obscured by the red (L2) plot, with which it almost coincides.

For far viewing, L2 and Listing's law prescribe similar torsion for all gaze angles, but on closer viewing the torsion angles differ increasingly. This is reflected in a horopter size ratio of 1 for far viewing, with L2 producing a larger horopter and better binocular alignment for closer binocular gaze. Note that keeping Helmholtz (1867) torsion constant at zero, always aligning the retinal horizontal meridians with the visual plane has no advantage over L2 in terms of horopter size. With such alignment, the horopter size is limited by the mismatch of epipolar line geometry in tertiary parts of the visual field that is brought about by vergence. This misalignment cannot be corrected by any torsional eye movement, and it outweighs the additional disparities created by cycloversion. 

It has been reported repeatedly that human eye movements do not actually follow L2 precisely (Mok et al., 1992; van Rijn & van den Berg, 1993; Tweed, 1997). While eye movement vectors fall on Listing's planes, their rotation differs from both that prescribed by Listing's law and L2. It has been argued (Schreiber, Crawford, Fetter, & Tweed, 2001) that the actual angle of rotation of Listing's planes with vergence strikes a compromise between the motor advantages associated with Listing's law, and the improvement of binocular image alignment L2 brings about. Figure 9 shows a horopter size plot for an intermediate oculomotor strategy, where Listing's planes rotate approximately 60% (for exact description, see Methods section) of the angle required for L2, indicating that any rotation of Listing's planes will result in an increase in horopter size, with maximum effect for pure L2. 

This increase is mainly due to a decrease in Helmholtz (1867) cyclovergence—binocular eye positions conforming to L2 have no Helmholtz cyclovergence at all. Because Helmholtz torsion represents a rotation relative to the visual plane, minimizing cyclovergence brings the two eyes into torsional alignment relative to the visual plane, and thus relative to the visual world. 

Nonidentical correspondence

The classical horopter is usually based on the assumption of identical retinal points, leading to the geometry of the Vieth–Müller circle described above. It is well known, however, that the empirical horopter does not have the shape of the Vieth–Müller circle, and empirical retinal correspondence does not pair up identical retinal points (Ames, Ogle, & Gliddon, 1932; Ogle, 1964). For locations within the visual plane, the deviation of the empirical horopter's shape from the Vieth–Müller circle indicates a retinal compression of corresponding points on the temporal side relative to the nasal side. This is called Hering–Hillebrandt deviation. It is characterized by a single parameter, the abathic distance, which is the fixation distance at which the empirical horopter becomes a straight line within the frontoparallel plane (Howard & Rogers, 2002; Ogle, 1964). 

Outside the visual plane, the empirical vertical horopter reveals a tilt such that its top part slants away from the observer, consistent with a horizontal shear of correspondence relative to identical points (Helmholtz, 1867). 

Not much is known about the empirical pattern of retinal correspondence outside the primary meridians of the eye. Ledgeway and Rogers (1999) used the apparent motion of extended lines to map retinal shear and argued that there is evidence for uniform horizontal shear of the vertical meridians up to at least 16 deg of eccentricity. It is unclear, however, how these results are related to point correspondence and whether their measurements can be extended into tertiary sectors of the visual field. 

To provide a general impression of the effect of these deviations from exact correspondence on the empirical horopter, we have generalized both the Hering–Hillebrandt deviation and the Helmholtz (1867) shear pattern to the full retina by applying them uniformly in retinal Helmholtz coordinates. The Hering–Hillebrandt deviation thus was implemented by compressing correspondence along retina fixed great circles that contain the interocular axis when gaze was straight. A shear of correspondence along the same coordinate lines was applied to produce Helmholtz shear. The horopters resulting from these manipulations are shown in Figures 10 and 11. 

 

Figure 10

 

The horopter with a Hering–Hillebrandt deviation. Abathic vergence is at 3.7 deg, corresponding to an abathic distance of 1 meter. Gaze is straight ahead at a vergence of 10 deg. Note the flattening of the horopter surface relative to Figure 2.

 

Figure 11

 

The horopter with a vertical shear deviation from a pattern of identical corresponding points. Gaze is straight ahead at a vergence of 10 deg.

Discussion

The expansion of the definition of the classical horopter has extended it from a tool for visualizing the geometry of correspondence into a quantitative measure of binocular alignment, useful for a range of questions concerning the interaction of eye movements and vision. 

We have used this new tool to show why the human oculomotor system breaks Listing's law during vergence. It has been argued that the use of L2 instead of Listing's law during near vision improves binocular alignment (Tweed, 1997; Schreiber et al., 2001). This improved alignment maximizes the benefit of using the epipolar constraint for the matching problem, without the accompanying cost of having to compute the gaze-dependent location of the epipolar lines on the retina (Schreiber et al., 2001). 

Using the extended horopter, we have shown how the change from Listing's law to L2 expands the utility of binocular correspondence, by keeping the shape of the horopter similar across eye position changes, and by increasing the fusable part of visual space for a given disparity limit. 

We also demonstrate a rotation of the classical theoretical horopter out of the visual plane with targets in the midsagittal plane when the eyes follow Listing's law (Figure 5). We have shown that L2 eliminates this rotation. 

This realignment of the retinal images by changes in ocular torsion has significance beyond simple matching alignment. Current models of stereoscopic slant perception (Backus, Banks, van Ee, & Crowell, 1999; Banks, Hooge, & Backus, 2001) use vertical disparity signals, in the form of the vertical size ratio (VSR) or the gradient of vertical disparities, as a cue to slant. In eccentric gaze, Listing's law changes cyclovergence for both vertical and horizontal eye position changes. A change in cyclovergence creates a gradient of vertical disparities along the horizontal retinal meridian, and a gradient of horizontal disparity along the vertical meridian. These gradients complicate the interpretation of the overall disparity field, but they could also be used to determine the viewing situation and compute an azimuth signal. Backus et al. (1999) explicitly ignored the effects of ocular torsion in their study of slant about a vertical axis. Banks et al. (2001), on the other hand, looked at signals influencing the perception of slant about a horizontal axis and found that ocular torsion is not taken into account at all. 

If the eyes follow L2 instead of Listing's law, Helmholtz (1867) cyclovergence is kept at zero for all gaze directions and vergence angles. This explains why cyclovergence does not have to be taken into account for perceiving slant about a horizontal axis, and it also justifies ignoring torsion in models of azimuth estimation and slant perception. In effect, the motor system driving the eyes assures that the visual system can rely on the simple assumption of zero cyclovergence. 

This motor strategy then is what makes it possible for the visual system to ignore ocular torsion in its perceptual computations. For this strategy to work, the motor program needs a mechanism to keep itself calibrated. It has been demonstrated recently that the control of ocular torsion can indeed be changed by a cyclodisparity stimulus (Maxwell, Graf, & Schor, 2001; Maxwell & Schor, 1999; Schor, Maxwell, & Graf, 2001). This suggests a view where ocular torsion programs are dynamically controlled to optimize binocular image alignment and simplify the calculations necessary for veridical slant perception. 

There are at least two further theoretical considerations arising from the work presented here. 

As we have stated in the methods section, the attempt to define a disparity vector for each horopter point relative to its pair of corresponding points reveals a weakness in the concept of retinal disparity. This weakness disappears for a situation of identical corresponding points, but for any other empirical pattern of retinal correspondence there is no unambiguous definition of absolute disparity relative to that pattern. Specifically, suppose C is the transformation producing the coordinates of retinal point R in the right eye corresponding to point L in the left eye, that is, R = C(L). For an object O in space that projects onto o_l and o_r in the left and right eyes, respectively, there then exist two equally appropriate retinal disparities relative to the correspondence mapping in the two eyes, namely, d_l = o_l − C⁻¹(o_r) for the left eye and d_r = o_r − C(o_l) for the right eye. If C is an identity mapping, that is, if corresponding points are identical points, these two definitions coincide except for sign and represent the common definition of retinal disparity as the difference in visual angle between the two projections. But for general correspondence functions C, the two eyes' disparity vectors differ in size and direction. In other words, there is no unambiguous retinal disparity relative to nonidentical retinal mapping. 

We have avoided this theoretical problem here by defining disparity as the average of the two eyes' disparity vectors. Furthermore, as long as disparity direction is ignored (i.e., as long as Panum's areas are isotropic around corresponding points), our definition of horopter points ensures that the disparity vectors have the same length in either eye. More work is needed to determine the implications of this ambiguity for the concept of retinal disparity as a physiological signal used in the visual system. 

Secondly, the use of a disparity metric in the definition of our extended horopter implies a retinal coordinate system for measuring the length of this disparity vector and for constraining it to a fusional zone. While the shape of our extended horopter, that is, the location of its points in space, is independent of the coordinate systems used in describing either eye movements or retinal locations, the same is not true for the disparity vectors. The value of their length and the meaning of their components, that is, which directions from a corresponding point we will call horizontal and vertical, depends on the retinal coordinate system used. What is the correct coordinate system then? 

Horizontal and vertical disparities are thought to code different aspects of the geometry of the visual scene. This is based on the geometrical fact that target motion in depth results in the shift of its retinal projections along epipolar lines. For static eyes, this means that the depth of objects is coded as a retinal disparity in a certain retinal direction. Disparities in the orthogonal direction cannot change for real targets at all (ignoring added lenses and equivalent distortions). Eye movements change the arrangement of epipolar lines, sliding and rotating them on the retina. In general, an object's projections that fall on the epipolar lines for the current eye position will not fall on the epipolar lines for a different eye position. While there is ample evidence for differential processing of horizontal and vertical disparities, the retinal coordinate system in which these signals are coded and whether that coordinate system is static or changes with eye position is presently unknown. 

A related further empirical question raised by our empirical horopter is the shape and extent of Panum's fusional areas across the retina. Foveally, the limits for stereomatching have been reported to be nonuniform on the retina (Stevenson & Schor, 1997), with horizontal disparity limits being about 1 deg and vertical limits about 0.5 deg. 

These limits, when measured for a given spatial frequency, are independent of retinal eccentricity (Schor, Wood, & Ogawa, 1984; Wilson, Blake, & Pokorny, 1988), but they scale with the spatial period of the stimulus for spatial frequencies lower than 2.5 cpd (Schor et al., 1984; Schor, Wesson, & Robertson, 1986). The upper cutoff spatial frequency of the visual system decreases with retinal eccentricity, falling below 2.5 cpd for eccentricities above 10 deg. By this reasoning, Panum's area is constant out to 10 deg and then increases with larger retinal eccentricities, in proportion to the period of the spatial frequency cutoff. 

For our simulations, however, we chose uniform and constant disparity limits instead, partly because the direction for vertical disparities would be unclear in the periphery because of the coordinate system problem described above. More importantly, had we increased the size of Panum's area with eccentricity, this increase in the disparity limits would have more than matched the increase in retinal disparity at eccentric retinal locations. This means that there would have been a horopter point projecting within Panum's area everywhere, and the extended horopter would have covered all of the visual field. This would have made it impossible to compare the size of the fusable surface across eye movement patterns. 

But even when every part of the visual horopter surface is fusable, stereo acuity is reduced as the stimulus moves away from perfect correspondence (Badcock & Schor, 1985), meaning that larger disparities are detrimental to perception even if they fall within Panum's area. So regardless of the size and shape of Panum's area, there is value in reducing retinal disparity by using L2 rather than Listing's law. 

Appendix

The pseudoinverse. To obtain the classical point horopter from a map of corresponding retinal locations, we intersect the projection rays from corresponding point pairs. The intersection, if it exists, is part of the horopter. Mathematically, this means solving the vector equation,  

$a_1{\overrightarrow{p}}_1=a_2{\overrightarrow{p}}_2+\overrightarrow{i}$

(1)

for the a_i, where p_i are the two eyes' projection vectors and i is the interocular vector. For our simulations, we assumed an interocular distance of 6 cm. This equation has no solution for many pairs of projection rays, reflecting the fact that most of them do not intersect at all. If we rewrite this equation in matrix form as  

$\mathrm{M}\left(\begin{array}{c}{a}_1\\ -a_2\end{array}\right)=\overrightarrow{i}$

(2)

with  

$\mathrm{M}=\left({\overrightarrow{p}}_1{\overrightarrow{p}}_2\right)$

(3)

we can solve it by multiplying both sides in Equation 2 from the left with the inverse of M,  

$\left(\begin{array}{c}{a}_1\\ -a_2\end{array}\right)={\mathrm{M}}^{-1}*\overrightarrow{i}$

(4)

 

The nonsquare matrix M is not invertible, but Moore (1920) and Penrose (1955) independently introduced a generalized matrix inverse for solving this type of problem. This inverse exists for any matrix, including nonsquare and singular matrices. It provides the closest solution in the least squares sense to Equation 2. Geometrically, this produces a pair of points, one on each projection ray, that is the smallest possible distance apart in space. 

To obtain a unique horopter point, the distance between these two points was then bisected such that the bisection ratio matched the ratio of the distance of the bisector from each eye's projection center. This assured that the angular distance of the horopter projection from the corresponding point was the same in each eye. 

Eye movements. The effect of eye movements on correspondence and the horopter was modeled by obtaining the projection vectors for the corresponding rays for the eyes pointed straight ahead, and then rotating them according to the motor program used. These rotations were carried out in Helmholtz (1867) angles; thus, the projection vector with the eyes rotated on target was obtained from the one for straight gaze by rotating torsionally, horizontally, and vertically, in that order:  

$p_s=M_v{M}_h{M}_t{p}_o$

 

These spatial projection vectors in space were then solved using Equation 2. 

Listing's law was implemented by having each eye's Helmholtz torsion depend on its horizontal and vertical position:  

$t_{l,r}=\frac{h_{l,r}v}{2}$

 

For L2, a similar formula was used, but instead of the individual eye's horizontal angle, horizontal version was used by adding or subtracting half the vergence angle g from the horizontal position, producing identical torsion in both eyes.  

$\begin{array}{c}g=\frac{h_r-h_l}{2}\\ \\ t_{l,r}=\frac{(h_{l,r}\pm g{)}^{}v}{2}=\frac{(h_l+h_r{)}^{}v}{4}\end{array}$

 

The 60% mix of Listing's law and L2 was defined as  

$t_{l,r}=-\frac{(h_{l,r}\pm {0.6}^{}g{)}^{}v}{2}$

 

Retinal area. We first determined for which of the points in a regular Helmholtz (1867) grid of corresponding points the horopter point would fall within a fusable distance from the correspondence. The total retinal area covered by the horopter was then calculated from this subset of corresponding points by summing the areas of grid patches centered on these dots and subtending the grid spacing angle in the horizontal and vertical direction:  

$A=\sum _i{\left(\sin \right(h_i+\frac{s}{2})-\sin (h_i-\frac{s}{2}\left)\right)}^{}s$

where h is the horizontal position of each corresponding point and s is the grid spacing. 

Acknowledgments

This research was supported by National Institutes of Health Grant EY 08882 and the Emmy Noether Fellowship of the German Academic Research Council (DFG). 

We thank two anonymous reviewers for helpful comments and suggestions. 

Commercial relationships: none. 

Corresponding author: Kai M. Schreiber. 

Email: [e-mail]. 

Address: 360 Minor Hall, Berkeley, CA 94720-2020, USA.