A tilted frame deceives the eye and the hand

Abstract

A roll-tilted visual frame induced a vertical line to appear roll-tilted in the opposite direction (rod-and-frame illusion). This visual illusion was measured by finding the physical roll-tilt of the line that appeared vertical—visually perceived vertical (VPV). In separate measurements, the roll-tilted visual frame was also found to induce an illusion in the felt orientation of the hand. This manual illusion was measured by setting the orientation of the hand to feel vertical in the presence of the inducing frame (manually perceived vertical—MPV), with distance of the hand from the body as a parameter. The manual illusion was large when the hand/arm was fully extended (60 cm from the body). When the hand was close to the body (in the midfrontal plane), there was no manual illusion. In a third set of measurements, subjects matched the roll-tilt of the hand to a line set by the experimenter to the subject's VPV or to VPV ± 5°. These manual/visual matches (MVM) were accurate at 60 cm—the distance at which the manual illusion was greatest. In effect, the large manual illusion at this distance tended to compensate for the visual illusion when the subject matched the orientation of the hand to the VPV. However, when the hand was close to the body, where there was no manual illusion, the manual/visual match was poor (no manual illusion to compensate for the visual illusion). The results are discussed in terms of a proximal/distal model that employs a linear weighted sum of influences from body-referenced and visual mechanisms. The model accounts for 61% of the MVM in terms of the inducer's effect on MPV. Considered in isolation, the gross VPV errors in conjunction with the accurate MVM settings with the extended arm might appear to support a theory based on perception/action dissociation. However, that theory cannot explain the dramatic effects of hand-to-body distance. The present results are very similar to those previously reported for the dimension of elevation where hand-to-body distance dependence was also found to modulate visually guided manual behavior.

Introduction

Two visual systems

Accurate manual reaching to a visual object requires the integration of information about the object's position in space together with information about the position of the arm. Under normal conditions, information about arm and object is consistent and reaching is accurate. However, a number of behavioral studies have reported that visual stimulation leading to misperception of an object's size, orientation, or location is accompanied by visually guided manual behavior to the misperceived object that may be accurate (Aglioti, DeSouza, & Goodale, 1995; Bridgeman, Lewis, Heit, & Nagle, 1979; Daprati & Gentilucci, 1997; Goodale & Haffenden, 1998; Goodale, Milner, Jakobson, & Carey, 1991; Goodale & Murphy, 1997; Haffenden & Goodale, 2000; Li & Matin, 2005a; Welch & Post, 1996). Two of these studies (Aglioti et al., 1995; Goodale et al., 1991) have provided the central basis of support for a widely discussed (Bruno, 2001; Carey, 2001; Franz, 2001; Li & Matin, 2005a; Prinz & Hommel, 2002) theory (Milner & Goodale, 1995), which proposes that the two streams emerging from primary visual cortex serve two dissociated functions: conscious perception of an object's properties (ventral stream) and the visual guidance of sensorimotor actions toward the object (dorsal stream). In one of these studies, an observer with visual form agnosia (DF) was able to orient a handheld card correctly while posting it in a variably oriented slot located in an erect frontoparallel surface at some distance in front of her. However, she was unable to report verbally about the orientation of the slot or to match the slot's orientation manually (Goodale et al., 1991). In the second study (Aglioti et al., 1995; Goodale & Haffenden, 1998; Haffenden & Goodale, 2000), observers with normal perceptual abilities viewed a circle surrounded by a ring of eleven smaller circles. They reported that this circle appeared larger than the identical circle surrounded by a ring of five larger circles (classic Ebbinghaus/Titchener illusion of size contrast). However, when the central circle in the illusion was a disk that an observer grasped with forefinger and thumb, the interfinger aperture shortly before contact did not vary with the size of the surround circles. 

The dissociated perception/action model described above is a version of the two-visual-systems theory (Held, Ingle, Schneider, & Trevarthen, 1967). It has been presented as an alternative to an earlier interpretation of the behavioral effects of the anatomically and neurophysiologically defined bifurcation between ventral and dorsal streams flowing from visual cortex (Milner & Goodale, 1995; for relevant neuroanatomy, see Baizer, Ungerleider, & Desimone, 1991; Morel & Bullier, 1990; Ungerleider & Mishkin, 1982). In that earlier interpretation, derived from behavioral experiments on monkeys (Ungerleider & Mishkin, 1982), the bifurcation separated a “what” stream mediating object recognition and identification (ventral stream) from a “where” stream mediating localization and orientation (dorsal stream). In fact, two-visual-system theories have a long history. Both of the above versions (“what/where” and the “dissociated perception/action” theories) place the centers of the two streams in cerebral cortex. However, earlier work supported a separation between “where” in the superior colliculus and “what” in cerebral cortex (Held, 1968; Ingle, 1967; Schneider, 1967, 1969; Trevarthen, 1968). Developments in two-visual-systems theory have depended heavily on the continuous interplay of work in perception/behavior and neurophysiology/neuroanatomy. 

Hand-to-body distance-dependent manual behavior

In a recent report, we described three experiments in which subjects made manual matches to a visual target that was viewed in the presence of a visual inducer. Accuracy of the matches was measured as a function of the distance of the hand from the body (Li & Matin, 2005a). The results led us to question the dissociated perception/action version of two-visual-systems theory. In those experiments, the physical elevation that was perceived to correspond to eye level (visually perceived eye level—VPEL) was measured in the presence of a single long (50° visual angle) inducing line viewed in total darkness. Variation of the visual pitch of the inducer at 10° intervals over the range from 30° topbackward (−30°) to 30° topforward induced a linear change in the average VPEL setting from 13° below true eye level (−13°) to 5° above true eye level. Subsequently, when attempting to point to the target set at VPEL with the unseen finger/hand close to the body, subjects made large pointing errors. However, with the arm fully extended, subjects pointed accurately at the visually mislocalized target. Intermediate arm extensions generated errors that increased linearly with proximity of the hand to the midfrontal plane (also see Matin & Li, 1995; Robison, Li, & Matin, 1995; Stoper, 1997; Welch & Post, 1996). Essentially identical hand-to-body distance dependence was also measured with a height-matching task that involved a very different constellation of motor behaviors. 

These results suggested that accurate matching behavior is possible because the inducer's illusory effect on visual perception is compensated by its effect on manually felt elevation. This explanation was examined in a fourth experiment (Li & Matin, 2005a). With the unseen hand, the subject pointed to the elevation that was felt to be horizontal in the presence of a variably pitched inducer. As in the matching experiments, distance from the body was varied parametrically. With the fully extended arm, the effect of the inducer on the felt position of the arm was large and only slightly less than its effect on VPEL. This result confirmed the prediction from the manual/visual match results: the manual illusion compensated for the visual illusion as expected. With the hand in the midfrontal plane, however, the setting of the hand to feel horizontal was independent of the pitch of the visual inducer—no manual illusion was found to compensate for the visual illusion. We also proposed a quantitative model—the proximal/distal model—in which the visual illusion, the manual illusion, and the manual/visual matches were treated as the result of weighted averages of inputs from the orientation of the visual field (distal process) and from the body-referenced mechanism¹ (proximal process). The excellent quantitative fits to the data provided strong support for the model. 

The present experiments

In the present experiments, we sought to generalize the findings of Li and Matin (2005a) by examining the influence of visual roll-tilt employing a roll-tilted inducer based on the classic rod-and-frame paradigm introduced by Witkin and Asch (1948). A great deal of work during the past 50 years has followed this original work and employed a large square roll-tilted outline frame in darkness that induced large changes in the frontoparallel orientation perceived to correspond to the orientation of visually perceived vertical (VPV). Recently Li and Matin (2005b, 2005c) showed that a single long roll-tilted line in otherwise total darkness generates an influence on VPV that is only about 17% less than the influence generated by the full four-sided frame, and a 2-line inducer generates an influence only 7% less than the full frame. The present experiments made use of those findings and employed an inducer consisting of two lines that met at a right angle because it permitted manual behavior that did not interfere with either the subject's view of the inducer or the subject's manual behavior (see Methods section for details). 

The present experiment involved measurements on each of the six subjects in three different tasks:

1.  
   VPV was measured with a short test line while viewing the inducer in otherwise total darkness. The inducer's roll-tilt was varied systematically (−10° ccw, vertical, and +15° cw).
2.  
   The manually perceived vertical (MPV) was measured while viewing the inducer in otherwise total darkness (test line absent). For these measurements, subjects set the orientation of the hand to feel vertical. Measurements of the MPV were made at three values of body-to-hand distance for each of the three inducer roll-tilts used in the visual measurements.
3.  
   While viewing the inducer together with the test line set at either VPV or at VPV ± 5°, the subject set the orientation of the palm to match the orientation of the test line (MVM). These measurements were made with the palm at each of three distances from the midfrontal plane.

In addition to the strong visual illusion produced by the 2-line inducer (Li & Matin, 2005c), a strong manual illusion was discovered that varied with distance of the hand from the body. Moreover, the experiments showed that manual accuracy in matching the orientation of the hand to a visual standard was also strongly dependent on hand-to-body distance. The distance-dependent manual illusion accounted for a large portion of the influence of the inducing frame on the manual match to the visual target. This pattern of results is essentially identical to the pattern that was discovered earlier in the elevation dimension and suggests the operation of a universal influence of hand-to-body distance as described by the proximal/distal model. 

Methods

Subjects

Six subjects with normal or corrected-to-normal vision served in all tasks. With the exception of two of the authors, all subjects were Columbia undergraduates who were paid an hourly wage for participating. Recruitment and the experimental protocol met the requirements of the Institutional Review Board (IRB) of Columbia University. Three other subjects whose visual illusions were very small were excluded from any further measurements. ² 

Visual stimuli

Visual inducer

The inducer consisted of two 52.8° × 8.5′ (80 cm × 0.2 cm) dim, luminous lines ( Figure 1). Each line was constructed from a strip of phosphorescent tape mounted on a black plastic bar that was attached by Velcro to a large free-standing, physically erect board set at 0° pitch and 0° yaw relative to the viewing eye of the subject; thus, the lines fell within a frontoparallel plane and met at a right angle. They formed two legs of an implicit geometric square whose center lay on the midline of the observer's viewing eye at true eye level, with the intersection of the two lines 56.4° (visual angle) from this center. The 40.3-cm distance from the center to the normal of each of the two lines equaled 29.75° (visual angle). The inducer was presented at each of three orientations within the frontoparallel plane, −10° (counterclockwise, ccw), 0° (vertical), or +15° (clockwise, cw) in otherwise complete darkness at a viewing distance of 70 cm. The axis of inducer rotation was horizontal within the midsagittal (median) plane of the subject's viewing eye at true eye level; i.e., the center of the implicit square coincided with the intersections between the median, frontoparallel, and horizontal planes. 

Figure 1

 

The three orientations of the 2-line inducer (the frame) are each shown along with an erect test line. These visual stimuli were presented in otherwise complete darkness.

Test line

A 6.5°-long × 10′-wide luminous test line that could be freely rotated around its center was mounted on a separate vertical panel centered within the inducer and positioned at the same distance (70 cm) from the observer's viewing eye. The test line was centered on a large protractor (resolution: 0.25°) at the rear of the apparatus; rotation of the test line was controllable by the experimenter around an axis in the subject's frontoparallel plane and was linked to a pointer that provided angular readings of the test line's physical roll-tilt. The protractor and pointer were visible to the experimenter when a small light was intermittently shone on it to obtain readings, but invisible to the observer at all times. The test line was centered in the median plane of the viewing eye at true eye level and was viewed against the 2-line inducer that was centered at the same point in the frontoparallel plane as the test line ( Figure 1). The center of rotation for the test line coincided with the center of rotation for the inducer. 

Both the inducing and the test lines received a brief exposure (approximately 2 min) to normal room illumination at the beginning of each experimental session. In addition, the lines were refreshed with approximately 30–60 s of normal room illumination following each set of 4 VPV measurements. The luminance of the lines was approximately 0.032 cd/m ² (EG&G photometer/radiometer model 550). At the beginning of each session, the observer's frontoparallel plane and eye level were brought into registration with the apparatus by adjusting a chinrest and the stool on which the observer sat. 

All viewing was with the left eye; all manual settings were with the right hand. All subjects were right-hand dominant (self-report). The 2-line inducer ( Figure 1) was employed instead of the classical square 4-line frame in order to restrict its position within the visual field to a region that would not be utilized by the right hand/arm when making manual settings. Hand orientation was monitored in three spatial dimensions continuously throughout the experiments and no problems were ever encountered in which the hand intruded into the space occupied by the 2-line inducer or the test line. As noted in the Introduction section, there was little loss in inducer potency from the 2-line stimulus relative to what would have been obtained with the classical 4-sided frame. 

The three tasks and four sessions

Measurements on each of the six subjects were made in a series of four experimental sessions. The first session was devoted entirely to measurements of VPV and lasted less than 1/2 h; no manual measurements were involved. Sessions 2, 3, and 4 were devoted to measurements on the other two tasks involving manual behavior: (a) setting the orientation of the palm to match the orientation of a visual line, and (b) setting the orientation of the palm to feel vertical; each of sessions 2, 3, and 4 lasted about 45 min. 

Measurement of visually perceived vertical (VPV): Session 1

The psychophysical method of adjustment was employed to measure VPV in the presence of each of the three inducer orientations in otherwise total darkness. Four trials were run at one inducer orientation before going on to another inducer orientation. In each set of four trials, the experimenter's setting of the test line's initial orientation was far clockwise relative to true vertical on two trials and far counterclockwise on the other two trials in abba order. On each trial, the experimenter adjusted the orientation of the test line until the subject reported that it was vertical. The subject's eyes were closed while the experimenter changed the test line's orientation. Although subjects were permitted as much time as needed to make a response, they almost always responded within 1 or 2 s after the eyes were opened and fixation of the test line's center was attained. The mean of the four settings for a given inducer orientation was the subject's VPV for that orientation. The three inducer orientations were sequenced in an independent random order within a session that was different for each subject. In addition, a series of four trials was run in which VPV was measured in the absence of the inducer (in total darkness) before and following the three inducer conditions in a session. 

Measurements of hand orientation

In Sessions 2, 3, and 4, measurements of hand orientation were made in two kinds of tasks: MPV (manually perceived vertical) trials in which the subject set the orientation of the hand to feel vertical; MVM (manual/visual match) trials in which the subject set the orientation of the hand to match the orientation of a visual standard. A different one of the three inducer orientations was employed in each of the three sessions. 

In order to measure the orientation of the hand in the MPV trials and in the MVM trials, the orientation of the subject's right hand was mounted on a wooden plate that was freely rotatable within an invisible wooden frame around a horizontal axis in a sagittal plane ( Figure 2). The plate was located at shoulder level either within the subject's midfrontal plane, 30 cm in front of the midfrontal plane, or 60 cm in front of the midfrontal plane. A search coil on the plate, part of a Polhemus 3-SPACE FASTRAK electromagnetic motion tracking system, provided signals resulting in simultaneous and continuous measurement of three dimensions of manual rotation and three dimensions of manual translation. Although the hand was monitored in six dimensions throughout the experimental sessions, the only values that received further processing were those obtained at the time the subject indicated a final response on a trial. 

Figure 2

 

Measuring the orientation of the hand. A search-coil-based Polhemus 3-SPACE FASTRAK electromagnetic device was employed to measure the orientation of the hand. The device was mounted on a track that allowed it to be set at various hand-to-body distances (0 cm, 30 cm, or 60 cm was employed). With the exception of the 2-line frame on MPV trials, and the 2-line frame and the test line on MVM trials, the visual field was completely dark; thus the apparatus and the subject's hand were not visible during the experiments. Pretest and posttest MPV trials were also made with the 2-line frame absent.

Setting the hand to manually perceived vertical (MPV): Sessions 2, 3, and 4

For these measurements, the subject set the orientation of the right hand to feel vertical (MPV setting) in the presence of the inducer in otherwise total darkness. The test line was not visible during these trials. 

Manually matching the roll-tilt of the visual target (MVM): Sessions 2, 3, and 4

For these measurements, the test line was set either at the VPV previously measured in Session 1 for the given subject, or at +5° (cw) relative to VPV, or at −5° (ccw) relative to VPV. These orientations provided the visual standards for the manual/visual matches. Because the visual field was dark except for the inducer and the test line, the subject's hand was invisible while matching at all hand-to-body distances. 

Experimental design for manual settings: Sessions 2, 3, and 4

As noted above, a different one of the three inducer orientations was employed throughout each of the three sessions during which measurements of manual behavior were made. The random order of inducer orientation across the three sessions was independently chosen for each of the six subjects. Within a session, all MVM trials and MPV trials were run with the hand at one of the three hand-to-body distances before another one of the distances was employed. The three hand-to-body distances were separately randomized across the six subjects. Nested within the hand-to-body distance variable was the variation of the three different orientations of the test line (VPV, VPV + 5°, VPV − 5°) in four randomized blocks of three trials for a total of 12 trials. Thus, nine different MVM conditions (3 visual standard orientations × 3 hand-to-body distances) were run in each session, with 4 trials in each of the nine combinations. 

At the beginning of each of the three segments of the session involving a different one of the hand-to-body distances, two MPV trials were run; two more MPV trials were run following the 12 MVM trials. 

At the beginning of each session, prior to the introduction of the inducer, four measurements of MPV were taken in total darkness with the hand in the midfrontal plane (pretest); another set of four MPV measurements was taken in total darkness at the end of the session (posttest). These baseline measurements were repeated in each of sessions 2, 3, and 4. 

Results

Visually perceived vertical (VPV) changes linearly with roll-tilt of the inducer

In the presence of the inducer, the vertical test line appeared to be tilted in the opposite direction (clockwise when the inducer was counterclockwise and vice versa). Accordingly, to be perceived as vertical, the line had to be tilted in the direction of the inducer. Figure 3a shows the VPV settings as a function of the roll-tilt of the inducer. Each data point is the average setting across six subjects. The straight line fitted to the three points has a slope of 0.50. The average value of the standard deviation in a series of four trials for a single subject across the six subjects and three frame orientations was 0.92° with no differences of any note among them. 

Figure 3

 

The inducer's effect on the visually and manually perceived vertical (VPV and MPV, respectively). Panel a shows VPV as a function of the inducer's roll-tilt. Panel b shows MPV as a function of the inducer's roll-tilt at each of three hand-to-body distances. Each data point is the average setting for the six subjects.

Setting the hand to feel vertical (MPV) under visual induction and in darkness

Figure 3b shows the effect of the inducer on MPV with distance of the hand from the body as the parameter of the set of functions. With the hand 60 cm from the midfrontal plane of the body (arm fully extended), there is a strong manual illusion: the MPV increases linearly with the roll-tilt of the inducer with a slope of 0.27 (filled green circles). However, with the hand in the midfrontal plane, the settings were essentially independent of the inducer's roll-tilt and close to the physical vertical at all inducer orientations (filled blue triangles). The slope of the latter function is 0.01—no manual illusion. At the intermediate distance, the slope of the MPV- vs-inducer roll-tilt was 0.17 (filled red squares). The slopes for the individual subjects with hand/arm extension at 60 cm ranged from 0.10 to 0.40; the individual slopes at 30-cm extension ranged from 0.15 to 0.20; the individual slopes at 0-cm extension ranged from −0.03 to 0.03. There was no aspect of the response variability that is of particular note. The standard deviation for the series of four trials in a given condition averaged across all subjects, hand-to-body distances, and inducer roll-tilts was 0.81° with only one of the 54 individual values as large as 2.8° and four others as large as 1.4°. 

The pretest and posttest MPV settings in complete darkness are not shown graphically. Averaged across all subjects, these settings were not significantly different from physical vertical with no significant differences between pretest and posttest values. 

Matching manual roll-tilt to a visual standard under visual induction (MVM)

Figure 4 displays the average manual matches to a line that was set at the VPV or at ±5 deg from VPV. The inducer was visible during these matches. The physical roll-tilts of the test lines (the three visual standards) are shown as stipled guidelines without data points; the manual matches are the data points. The three panels ( Figures 4a– 4c) display the results for hand-to-body distance equal to 0 cm, 30 cm, and 60 cm, respectively. 

Figure 4

 

In each panel, the average MVM setting for the six subjects is plotted against inducer roll-tilt for manual matches to the standard visual target set at VPV, at 5° ccw (−5°) from VPV, and at 5° cw (+5°) from VPV. Hand-to-body distances were 0, 30, and 60 cm from the midfrontal plane in panels (a), (b), and (c), respectively. The straight diagonal line in each panel composed of short black dashes is the line of best fit to the VPV- vs-roll-tilt data in Figure 3; the straight diagonal lines composed of long red dashes or of blue dots were drawn parallel to this line to represent the locus of the standard for the MVM measurements with the standard at +5° and −5°, respectively. The small two-headed arrows in panels a and b represent the discrepancies between the visual standard and the subject's manual match (the action errors). Note that there are no significant action errors in panel c. Taken by itself, panel c would therefore appear to support the dissociated perception/action theory. However, the errors in panels a and b would not be predicted by that theory.

With the hand in the midfrontal plane (hand-to-body distance = 0 cm; Figure 4a) the average manual matches to the visual standard at VPV (filled circles) did not vary with inducer orientation. Instead, the matches were close to the true physical vertical—and differed grossly from the physical orientations perceived as vertical. Although the physical orientation of the standard that was perceived as vertical (VPV) varied over a 12.50° range as the roll-tilt of the inducer changed from 10° ccw to 15° cw, the orientation of the hand matched to the visual standard at VPV was not significantly different from true physical vertical for each of the three orientations of the inducer ( t = −0.82, p = 0.49; t = −0.68, p = 0.53; and t = 2.01, p = 0.10) or from each other [slope of the manual setting- vs-roll-tilt function = 0.02; F(2, 10) = 2.60, p = 0.12]. With the standard at VPV + 5° and at VPV − 5°, the settings were also independent of inducer orientation as shown by the red squares and blue triangles in Figure 4a, respectively [ F(2, 10) = 2.26, p = 0.15, F(2, 10) = 4.0, p = 0.05], and also independent of the standard. The matches to these standards deviate from the matches to the standards at VPV by ±5°, respectively, a result that indicates the linearity of the MVM settings with physical orientation of the roll-tilt of the visual standard. 

However, as displayed in Figures 4b and 4c, constancy in the manual matches with change in the roll-tilt of the inducer did not occur when the match was performed with the arm extended either 30 cm or 60 cm from the body. Instead, at both of these hand-to-body distances the manual matches to VPV, VPV + 5°, and VPV − 5° were increasing linear functions of the inducer's roll-tilt. The results at 60 cm are particularly striking: At this distance, the slope of the manual setting- vs-roll-tilt function equaled 0.48—essentially indistinguishable from the VPV- vs-roll-tilt slope of 0.50. The orientation of the hand was not significantly different from the standard's physical orientation for each of the three roll-tilts of the inducer ( t = 0.73, p = 0.50; t = −1.24, p = 0.27; and t = −1.13, p = 0.31). In effect, the subject saw the stimulus incorrectly—but put the hand where the stimulus appeared to be. At the intermediate hand-to-body distance (30 cm), the slope of the manual setting- vs-roll-tilt function averaged 0.24 ( Figure 4b)—approximately halfway between the slope in the midfrontal plane and the slope with the hand at 60 cm. For all orientations of the inducer in Figure 4b, the matching hand's orientation was intermediate between the physical orientation of the visual standard and its visually perceived orientation. 

The results of the manual matches to the visual standard ±5° from the VPV settings made it clear that the subjects could not have made their settings by simply attempting to set the hand to visually perceived vertical directly instead of performing independently under the different conditions when manually matching to the orientation of the standard. The trials with the standard set to VPV ± 5° were mixed with trials in which the standard was set to VPV. Performances when the standard was set at VPV + 5° (cw) and at VPV − 5° (ccw) were also dependent on the distance of the hand from the body. As noted above, for both the standard lines at VPV + 5° and at VPV − 5°, the orientation of the midfrontal plane manual match ( Figure 4a) did not change significantly with the roll-tilt of the standard over the 12.5° range. Thus, there is a substantial match error for the standard line at both VPV + 5° and at VPV − 5° for the midfrontal plane manual match. However, the full-extension manual match was accurate throughout the range of roll-tilts of the standard ( Figure 4c). Therefore, the patterns of errors in the midfrontal plane matches and accuracy of the full-extension matches held over a roll-tilt range of at least ±5° centered on VPV. The results for the intermediate hand-to-body distance at VPV + 5° and VPV − 5° ( Figure 4b) were also consistent with the results at VPV: the manual settings were halfway between the settings at the two extremes of hand-to-body distance in Figures 4a and 4c. 

The slopes for the individual subjects with hand/arm extension at 60 cm across the three roll-tilts of the visual standard ranged from 0.41 to 0.53; all individual slopes for 30-cm extension fell between 0.19 and 0.30; all individual slopes at 0-cm extension fell between −0.01 and 0.07. No aspect of the response variability was of particular note. The standard deviation for the series of four trials in a given condition averaged across all subjects, hand-to-body distances, and inducer tilts was 0.95° with only two of the 162 individual values as large as 3° and five others as large as 2°. 

The relation between MPV and MVM

Figure 5 shows the relation between the slope of the MVM- vs-inducer roll-tilt function (manual/visual matches from Figure 4) and the slope of the MPV- vs-inducer roll-tilt function (manual illusion from Figure 3b) at the three hand-to-body distances. If the manual illusion provided the sole basis for the manual match, the relation in Figure 5 would be linear with a slope of 1, as shown by the dashed line in the figure. If the manual illusion did not contribute to the basis for the manual match, the slope would be 0. The actual slope is 0.61. This point will be revisited in our theoretical discussion of the data in relation to the proximal/distal model. 

Figure 5

 

Comparison of the two manual behaviors (manually perceived vertical, MPV, and manual/visual match, MVM). Average slopes of the MPV- vs-roll-tilt of the inducer from Figure 3b are shown on the ordinate and the corresponding average slopes of the MVM- vs-roll-tilt of the inducer from Figure 4 are shown on the abscissa. The three data points display the correspondences at the three hand-to-body distances, respectively (0 cm, 30 cm, and 60 cm). The dashed line shows the expected result if the effects of the inducer on MPV and on MVM were identical (slope = 1). However, the slope of the solid best-fitting line to the data points is 0.61, showing that the inducer's effect on the MPV accounts for 61% of its effect on the MVM.

Another perspective on the relation between the manual matches and the manual illusion is shown in Figure 6. In this plot, the manual errors in the felt vertical (MPV) and in the manual matches (MVM) are shown as a function of hand-to-body distance for each of the three inducer orientations (−10°, 0°, and +15° in panels a, b, and c, respectively). When the inducer is tilted, the magnitude of the manual illusion increases linearly with hand-to-body distance (black data points). The magnitude of the errors in the manual matches, however, decreases (open circles). For both the VPV + 5° and the VPV − 5° tilts of the inducer, there is virtually no error in MVM at 60 cm. Both the MVM errors and the MPV errors are small (panel b) when the inducer is vertical. 

Figure 6

 

The MVM errors and the MPV errors are plotted as a function of hand-to-body distance. Each panel plots the average error for the six subjects for one inducer orientation. Although the MVM and MPV errors change in the same direction with increase in hand-to-body distance, the magnitude of the MVM error decreases and that of the MPV error increases with hand-to-body distance in the presence of the two non-vertical (−10°, +15°) inducers. The dashed line in each panel is the locus for accuracy for both the MVM and the MPV.

Discussion

The present results and the proximal/distal model

As was the case in our previous experiments with the dimension of perceived elevation (Li & Matin, 2004, 2005a), the present results show that properties of the visual field that generate visual illusions also generate manual illusions and affect manual matches to a visual stimulus. A theory that addresses this range of effects—the proximal/distal model—is the subject of this section. 

The proximal/distal model employs a weighted average of two processes—a proximal process and a distal process—whose conjunction controls both egocentric space perception and visually guided sensorimotor behavior. The core of the model was originally proposed to account for egocentric space perception alone (Matin & Fox, 1989). It was recently extended to account for the effect of hand-to-body distance on manual behavior in the perception of elevation (Li & Matin, 2005a). In the present article, we show that the 2005 model also provides a good fit to the present data. 

In the initial work on the influence of visual induction on egocentric space perception, the model accounted for the large near-linear influence of visual pitch on visually perceived eye level—VPEL (Matin & Fox, 1989), predicting the value of VPEL as a weighted average of influences from the visual field and a body-referenced mechanism. With the same parameter values, the model also predicted the linear relation across subjects between the elevation of VPEL in complete darkness and in an erect illuminated visual field. It has since been employed to predict VPEL under a large variety of conditions with 1-line and 2-line pitched and roll-tilted inducers in darkness and under illumination (Li, Dallal, & Matin, 2001; Li & Matin, 1996, 1998; Matin & Fox, 1989; Matin & Li, 1992, 1994a, 1994b, 1994c, 1995, 1999, 2001). The same weighted average model also predicts the multiplicative tradeoff between the influence of the visual field and the influence of gravitointertial force (g) along the subject's z axis from 0 to 4 g (Chelette, Li, Esken, & Matin, 1995; Li et al., 2001). It is also one of the components of a more complex multichannel neuromathematical treatment of VPEL (Matin & Li, 2001). The recent conversion of the original model for visual perception to the proximal/distal model for perception and sensorimotor behavior was accomplished by the introduction of a hand-to-body distance parameter, inclusion of the influence of visual induction within the distal process, and by the identification of the body-referenced mechanism with the proximal process (Li & Matin, 2005a). In addition to accounting for the variation of VPEL with pitch of an inducer, the weighted average treatment accounted for two new aspects of the experiments: distance-dependence of manual accuracy and the strong concordance between manual errors of elevation in matching the hand to a visual target and the felt elevation of the finger/hand relative to horizontal (an internal standard or “norm”) within the dimension of elevation. 

In the proximal/distal model's treatment of the present results ( Figure 7), VPV, MVM, and MPV are each a consequence of a linear weighted sum of the inputs from two neural systems: a “distal” system, D, and a “proximal” system, P. 

Figure 7

 

The proximal/distal model. The proximal system (P) and the distal system (D) affect all of the measured behaviors—VPV, MPV, and, MVM. The values of α and β that feed into the three measured behaviors refer to the relative contributions (normalized weights) of the proximal and distal systems, respectively. See Table 1 and the text for details.

In the distal system, location and orientation are expressed in coordinates referenced to the space outside the observer. In the present context, visual information (V, roll-tilt of the inducer) is the experimentally manipulated aspect of the distal system's stimulus. In contrast, in the proximal system, location and orientation are expressed in coordinates referenced to the body of the observer; visual input is of minimal influence, whereas responses to gravity, somesthesis, proprioception, and efference are highly significant. As has been well known since at least the work of Mach (1894/1943), gravity exerts a great deal of control on VPV (also see Chelette et al., 1995; Cohen, 1973; Li et al., 2001; Witkin, 1949; Witkin & Asch, 1948). Accordingly, we lump its influence with the other factors that affect the body-referenced mechanism and employ the term “proximal system” for the mechanism that processes all nonvisual contributions. 

In the model's treatment of VPV in the present experiments ( Equation 1), ³ we first obtain the weights, β _VPV and α _VPV, which express the relative influences of the inducer's roll-tilt (v) and the body-referenced influences (b) on VPV.  

$\begin{array}{c}\mathrm{V}\mathrm{P}\mathrm{V}(\mathrm{b}\mathrm{,}\mathrm{v})={\alpha }_{\mathrm{V}\mathrm{P}\mathrm{V}}\mathrm{P}\left(\mathrm{b}\right)+{\beta }_{\mathrm{V}\mathrm{P}\mathrm{V}}\mathrm{D}\left(\mathrm{v}\right)\\ \\ {\alpha }_{\mathrm{V}\mathrm{P}\mathrm{V}}+{\beta }_{\mathrm{V}\mathrm{P}\mathrm{V}}=1.\end{array}$

(1)

 

The slope of the linear VPV- vs-roll-tilt function, dVPV(b,v)/dv is a measure of visual sensitivity to the roll-tilted inducer (see Figure 3a). Its value provides a quantitative measure of the relative influence of the inducer's roll-tilt on the VPV: If the roll-tilt of the visual inducer was the only source of control of VPV, the slope of the VPV- vs-roll-tilt function in Figure 3a would be 1.0; if the inducer's roll-tilt exerted no control on the VPV setting, the slope would be zero. The slope is thus a measure of the observer's sensitivity to the inducer's roll-tilt. We employ the slope as a weight whose magnitude measures the influence of roll-tilt in Equation 1, and set it equal to the value of β _VPV.  

$\mathrm{d}\mathrm{V}\mathrm{P}\mathrm{V}(\mathrm{b}\mathrm{,}\mathrm{v})/\mathrm{d}\mathrm{v}={\beta }_{\mathrm{V}\mathrm{P}\mathrm{V}}.$

(2)

 

Thus, β _VPV is normalized on the scale from 0 to 1. ⁴ The experimentally obtained value of the slope of the VPV- vs-roll-tilt function is 0.50. Because α _VPV + β _VPV = 1, α _VPV is also 0.5. Accordingly, the roll-tilt of the visual inducer is clearly not the only source of control for VPV. 

Proceeding in a similar fashion for MPV, but parameterizing for distance, we obtain β(d) _MPV for each value of hand-to-body distance (d) from the slopes of the three MPV- vs-roll-tilt functions in Figure 3b. These three values of β(d) _MPV [and the corresponding values of α(d) _MPV] are shown in Figure 7 and in Table 1. 

Table 1

 

Fitting the proximal/distal model to two sets of experiments. The table shows the weights for the proximal process ( α) and for the distal process ( β).

The β weights for MVM can be obtained from the slopes of the MVM- vs-roll-tilt functions in Figure 4. However, this case is somewhat more complicated than the case for VPV because there were three visual standards (VPV and VPV ± 5°). Inspection of the figure suggests, however, that the slopes at each distance were essentially the same for all three visual standards. Moreover, the slopes increase as a linear function of distance. Accordingly, we used all nine slopes as the data in a regression of slope on distance (see Figure 8a). The values on the line of best fit at the three distances (0 cm, 30 cm, and 60 cm) are the β(d) _MVM weights in Figure 7 and Table 1. With the fully extended arm, MVM was essentially accurate at all inducer orientations ( Figures 4c and 6). This is manifested in the model by the fact that the α and β weights for MVM are very nearly the same as the VPV weights at this distance [ β(60) _MVM = 0.48 and α(60) _MVM = 0.52; β _VPV = 0.50 and α _VPV = 0.50; see Figure 7 and Table 1]. However, as hand-to-body distance decreases these weights shift progressively. The weights of the distal system decrease to near zero values while weights of the proximal system increase. 

Figure 8

 

(a) Fitting the proximal/distal model to the MVM measurements. The β(d) _MVM weights are equal to the slopes of the MVM- vs-roll-tilt functions (from Figure 4) and are plotted against hand-to-body distance. At each distance, there are three data points (corresponding to the three roll-tilts of the inducer at that distance), for a total of nine data points. The values on the line of best fit for each of the three distances are the measures of β(d) _MVM of the model that are shown in Figure 7 and Table 1. (b) The values of β( d) _MVM for VPEL are equal to the slopes of the MVM- vs-pitch functions and are plotted against hand-to-body distance for three elevations from the earlier experiments (Li & Matin, 2005a; also see Table 1).

In the proximal/distal model's treatment of the present results, the concordance between VPV and MVM when the arm was fully extended (see Figure 4c) is a manifestation of the fact that they share virtually identical weights for the inputs from the distal and proximal systems. It may seem peculiar for the model to predict that MVM is in agreement with VPV even though visual perception manifests severe distortions. In fact, the near-identity of the weights for VPV with the weights for MVM with the fully extended arm shows that VPV and MVM are both influenced equally by the inducing line, so the accurate matching is not surprising. The concordance between the relative weights of MVM and VPV diminishes as the distance of the hand to the body diminishes (visual target distance is unchanged) and is minimal with the hand in the midfrontal plane. With the hand close to the body (d = 0), manual matching is primarily controlled by the proximal system and minimally by the distal system. Accordingly, the sensorimotor guidance of the hand in the midfrontal plane is not modified by the distant visual field. Because VPV is modified by the inducer and MVM is not, MVM is in error by the magnitude of VPV's modification. 

Although the theoretical work in this section concerns variations in sensorimotor behavior (i.e., the manual matches to the visual standards—MVM), our interpretation does not rely in any way on effects in the motor system; we treat all inputs to the distal and proximal systems as sensory. Such motoric effects might be obtained by modifying the signals at the myoneural junction of the muscles controlling the hand. This was done in another study by curarizing the neuromuscular junction of the extraocular muscles (Matin et al., 1982; Matin, Stevens, & Picoult, 1983). The curarization procedure reduced the extent of the eye's movement in response to a motor command and also generated severe systematic perceptual mislocalizations of egocentric space. We have no evidence that the accuracy of the hand was affected by any such influence in the present experiments. But other motor effects—in cortical or sub-cortical structures, for example—could be involved. In any event, because the inducer's effect on MPV was only 61% as large as its effect on MVM (see Figure 5) further work will be needed to establish the source of the remaining 39%. 

The present results and the dissociated perception/action theory

Considered in isolation, the gross visual errors ( Figure 3a) in conjunction with the accurate manual matching with the extended arm ( Figure 4c) might appear to support the dissociated perception/action theory (Aglioti et al., 1995; Goodale & Haffenden, 1998; Goodale & Murphy, 1997; Goodale et al., 1991; Milner & Goodale, 1995; Servos, Matin, & Goodale, 1995). However, the totality of the results is not predicted by that theory. In particular, the theory cannot explain the dramatic effects of hand-to-body distance. 

As noted in the Introduction section, major sources of evidence for perception/action dissociation are the findings about visual and manual errors with DF, the patient with visual agnosia, and the findings with normal subjects viewing the Ebbinghaus illusion. In both sets of studies, the authors do not discuss hand-to-body distance explicitly. Nonetheless, their descriptions of the subjects' behaviors suggest that hand-to-body distance played a significant role. 

The effect of hand-to-body distance in DF's case can be seen in Figure 1 of Goodale et al. (1991): Figure 1a shows DF's gross errors in matching the orientation of a handheld card to the orientation of a slot located at a distance of 45 cm from the body with the hand close to the body. However, in attempts at posting the card in the slot, just before making contact with the slot (Figure 1b), her errors in matching the card's orientation were small (very similar to the errors of two normal control subjects). 

In a similar vein, errors in estimating the perceptual size of the central circular disc in the Ebbinghaus illusion, coupled with relative accuracy in the interfinger aperture (the manual/visual match) just before grasping the illusion-influenced stimulus with the fingers at the end of an extended arm were also interpreted as evidence for perception/action dissociation (Aglioti et al., 1995; Milner & Goodale, 1995). However, the manual estimates of disk size given by the subjects in darkness with the hand/arm close to the body “… were strongly biased in the direction of the illusion” (Goodale & Haffenden, 1998, p. 168).⁵ 

In the proximal/distal model, these distance effects are readily interpreted as the consequence of the large weight of the proximal system when the hand is in the midfrontal plane. As the hand approached the target, however, the weights for the inputs from the distal and proximal systems on the sensorimotor behavior would become increasingly similar to those controlling visual perception. At the termination of the reach, the weights controlling the sensorimotor matching behavior would become identical to the weights for VPV. 

Physiological evidence

A number of suggestive parallels exist between the present results and the results of studies of the neural structures controlling sensorimotor behavior in the macaque monkey (Berti & Rizzolatti, 2002; Fogassi et al., 1992, 1996; Gentilucci, Scandolara, Pigarev, & Rizzolatti, 1983; Graziano & Gross, 1994, 1995; Graziano, Hu, & Gross, 1997; Jeannerod, 1988, 1997; Rizzolatti, Matelli, & Pavesi, 1983; Rizzolatti, Riggio, & Sheliga, 1994; Rizzolatti, Scandolara, Matelli, & Gentilucci, 1981). Unilateral lesions in macaque cortical area 6 (premotor cortex) produce visual neglect in contralateral near space (reachable space) but no deficit in far space (Rizzolatti et al., 1983). Thus, there is a distance-dependent effect in the neural processing of reaching. Furthermore, visual/tactile bimodal units in areas 6 and 7b (posterior parietal cortex) are sensitive to visual stimuli located up to a given distance from the body and are unresponsive to more distant visual stimuli, with different neural units showing maximal responding at different distances (Graziano & Gross, 1994, 1995; Graziano et al., 1997). Far space is not represented in the 7b–6 (F4) loop (Graziano & Gross, 1994, 1995; Rizzolatti et al., 1994). These bimodal units constitute a significant percentage of the individual neural units in areas 6 and 7b, and their receptive fields for vision and for touch correspond in physical space (Fogassi et al., 1992, 1996; Gentilucci et al., 1983; Graziano & Gross, 1994, 1995; Graziano et al., 1997; Rizzolatti et al., 1994, 1981). Furthermore, the visual and tactile receptive fields remain in correspondence when the arm moves, but almost all macaque bimodal units centered on the arm do not move with either the eye or the head (Fogassi et al., 1996; Graziano & Gross, 1994, 1995; Graziano et al., 1997). This lack of correspondence with eye and head movements fits with the finding that the perception of elevation (including VPEL) in humans is virtually independent of eye and head position (Li & Matin, 1993; Matin & Li, 1995). These bimodal units become active during reaching into near space and, as has been previously suggested, are likely to be significant in the control of visually guided reaching. We suggest, therefore, that the bimodal units might serve as the neural underpinning of a population model (Georgopoulos, Schwartz, & Kettner, 1986) that would be consistent with our psychophysical results which are distance-based and direction-based for controlling manual pointing, reaching, height matching, and manual orienting. 

Ecological considerations

The main aspects of the distance dependence of manual behavior in the roll-tilt dimension described in the present report are essentially identical to the VPEL measurements previously described for the dimension of elevation (Li & Matin, 2005a). The similarity suggests a general basis in ecological utility (see Figure 8 and Table 1 for a comparison of the α and β weights in the two experiments). We make some suggestions about these issues in the following paragraphs. 

When reaching for a distant object, the most important outcome is accurate contact with the object. This priority for manual behavior is present regardless of where the object is localized visually and whether the visual perception is accurate or not. It can only be met if any change in visual localization of the object is accompanied by a compensatory shift in manual behavior. On the other hand, many essential manual behaviors with the hand close to the body and without sight of the hand relate to the body itself. Examples of such behaviors are putting food into the mouth, touching and/or scratching a part of the body, putting clothes on the body, and bringing a second hand into contact with an object in the first hand. In these cases, manual localization needs to maintain a stable relation to the body in the presence of visual induction. Scratching one's own ear, for example, could not be done accurately if manual localization relative to the head had been shifted by visual induction from a distant stimulus. Because many such behaviors relative to one's own body are based on proprioceptive signals, shifts of manual localization in the midfrontal plane based on visual input would be counterproductive. This picture suggests that our findings of distance-dependent accuracy in both elevation and roll-tilt serve significant functions at both extremes of distance. 

Considerations about ecological utility and distance-dependent behavior also raise matters related to perceptual adaptation (Harris, 1965; Held, 1965; Helmholtz, 1866; Kornheiser, 1976; Welch, 1978, 1986). These studies show considerable flexibility between perception and visually guided motor behavior by employing feedback to bring them into alignment when chronic misalignment exists (e.g., when the arms grow during normal development). Whether the distance-dependence we measured in the present experiments is a consequence of such long-term adaptation or is innate is an interesting question that will need to be addressed in future research. To date, we have measured short-term influences of adaptation and memory on the effects of hand-to-body distance (Li & Matin, 2004). However, we have not examined such influences beyond the period immediately following introduction of the inducing stimulus. Whether the effects of hand-to-body distance are task-dependent is also of considerable interest. In our experiments in which the inducer's visual pitch was systematically varied and in which hand-to-body distance dependence was first discovered in the dimension of elevation (Li & Matin, 2005a), the different constellations of motor behavior involved in pointing/reaching and in height matching to visual targets at VPEL as well as at several elevations above or below VPEL manifested indistinguishable influences of hand-to-body distance. These issues need to be further addressed in future research. However, our findings of very similar influences in the pitch and roll-tilt dimensions suggest a great deal of generality in the hand-to-body distance dependence that we have described: manual accuracy to mislocalized visual targets is graded with distance. For the extended arm, matching is near-perfect. However, gross errors occur when the hand is close to the body.⁶ 

Acknowledgments

This research was supported by the National Science Foundation Grant BCS-06-16654 and National Institutes of Health Grant EY-10534 from NEI. 

Commercial relationships: none. 

Corresponding author: Wenxun Li. 

Email: [email protected]. 

Address: 406 Schermerhorn Hall, MC. 5501, Department of Psychology, Columbia University, 1190 Amsterdam Ave., New York, NY 10027, USA. 

Footnotes

Footnotes

¹  The term “body-referenced mechanism” was introduced (Matin & Fox, 1989) to refer to the combination of all extraretinal influences on the perception of interest. It processes extraretinal information about eye position, head orientation (including information regarding the head relative to the body and the head relative to gravity), other effects of gravity on the body, pressure cues from the surfaces of the body, cues from the joint receptors, and the vestibular organ. In addition, it processes the basic local sign information from the visual target employed to measure the discrimination itself. The term was introduced to separate the processing of visual from nonvisual influences on visual perception.

Footnotes

²  Since we did not know what the manual behavior would be prior to the experiment, we wanted to maximize our chances of finding manual effects if they were there by selecting subjects who showed substantial effects of the inducer on the visual perception. Thus, we required a VPV- vs-roll-tilt slope greater than 0.4 for inclusion of a subject in the full experiment. Three individuals with slopes equal to 0.21, 0.23, and 0.37 were thus eliminated from the experiment following VPV measurements; no manual measurements were made on these three. Thus the six that constituted our subject group manifested VPV- vs-roll-tilt slopes that were probably close to the upper half of the population of individual slopes (our rough estimate from prior work: e.g., Li & Matin, 2005b, 2005c). The slopes of these six subjects ranged from 0.48 to 0.55. As it turned out, the manual effects we measured and report here are very substantial, and we had no need for subject selection.

Footnotes

³  It is assumed that V = V _S − V _O and B = B _S − B _O in which V _S and B _S are components (angular directions) due to inputs to the visual system and the body-referenced mechanism, respectively, and V _O and B _O are idiosyncratic biasing constants related to visual stimulation and stimulation by the body-referenced mechanism, respectively, specified separately for perception and for the manual behavior. This analysis was carried out in the original presentation of the weighted average model for VPEL (Matin & Fox, 1986, 1989) and in several of our subsequent reports (Matin & Li, 1994a, 1999). It was developed further in the first report on hand-to-body distance dependence related to pointing/reaching and height matching (Li & Matin, 2005a). However, although some further use was made of the biasing constants, V_O and B_O, and values for them were calculated in some of the earlier work as well as some later work (Bertz, Li, & Matin, 2005), we have not included them in the present treatment since their inclusion would complicate the clear lines of the treatment unduly without adding significant useful information.

Footnotes

⁴  Although empirical slopes of the VPV- vs-roll-tilt function larger than 1.00 are not prohibited, we have never measured one nor are we aware of any that have been published.

Footnotes

⁵  These manual estimations were described by Goodale and Haffenden (1998) and then reported in detail by Haffenden and Goodale (2000) where they state “While making the estimation subjects were required to rest the heel of their hand on the start button, and move only their index finger and thumb.” (p. 1602). The latter statement provides the support for our assumption that the hand/arm was “close to the body.”

Footnotes

⁶  Post, Welch, and Olson (2004) also reported measurements of manual behavior induced by the tilted frame of Witkin and Asch at the meeting of the Visual Sciences Society at which the experiments in the present report were originally presented. Since their measurements were made at only one hand-to-body distance, their results could not be used to draw inferences regarding hand-to-body distance-dependence. They used a different procedure for measuring the effect than we employed here and in the preliminary description of the manual-match-to-a-standard experiment (Matin, Li, & Bertz, 2004): Instead of directly measuring the roll-tilt orientation of the hand, they required the subject to point successively at each end of the perceptually mislocalized rod. From these measurements, calculations were made that indicated a manual roll-tilt halfway between the true rod orientation and the orientation that corresponded to the magnitude of the perceived illusion; they suggest that their results indicate a “partial dissociation of vision and action.”