Visually directed action

Abstract

When people throw or walk to targets in front of them without visual feedback, they often respond short. With feedback, responses rapidly become approximately accurate. To understand this, an experiment is performed with four stages. 1) The errors in blind walking and blind throwing are measured in a virtual environment in light and dark cue conditions. 2) Error feedback is introduced and the resulting learning measured. 3) Transfer to the other response is then measured. 4) Finally, responses to the perceived distances of the targets are measured. There is large initial under-responding. Feedback rapidly makes responses almost accurate. Throw training transfers completely to walking. Walk training produces a small effect on throwing. Under instructions to respond to perceived distances, under-responding recurs. The phenomena are well described by a model in which the relation between target distance and response distance is determined by a sequence of a perceptual, a cognitive, and a motor transform. Walk learning is primarily motor; throw learning is cognitive.

Introduction

Early in the history of psychology, an issue arose as to whether continuous vision is necessary to execute visuomotor responses accurately. In reaching for a target with vision, there is often an initial response to the approximate position of the target, then a second, corrective response (Woodworth, 1899). If vision is occluded during the initial response, the corrective response does not occur. Woodworth hypothesized that it is possible to preplan motor acts to some extent and execute these plans without vision. In acts such as throwing or hitting, there is no opportunity for correction, so they must be preplanned. 

I have observed that, when people throw an object at a near target in a novel situation, they often throw short. After very few throws, they become much more accurate. Similar phenomena occur in other visuomotor tasks when vision is occluded during the response. For example, if one views a near target and then reaches for it with an unseen hand, the target is usually overreached (Foley, 1975; Foley & Held, 1972). I refer to such an action as a visually directed action, by which I mean an action in which visual stimulation is available to plan the action, but not to guide the action after it has been initiated. Some actions are by their nature visually directed because once they are initiated, their effects cannot be controlled. Other actions could be guided by feedback, but they become visually directed when the feedback is eliminated. Throwing is an example of the first; blind walking is an example of the second. Visually directed responses are sometimes referred to as open-loop responses. Domini and associates have done an extensive series of experiments on reaching and grasping with an unseen hand (visually directed responses) with only binocular cues available (Campagnoli, Croom, & Domini, 2017). There are systematic errors in reaching and grasping, which they measure in the same situation as the motor responses and which they have shown to be very closely related to the systematic errors in distance and depth perception (Foley, 1980). 

This study is an attempt to understand visually directed responses. What produces the initial errors? What is required for the responses to become accurate? Is endpoint error feedback sufficient? If so, what processes underlie this learning? Can a model be created that describes and predicts them? 

The biological processes that bring about perception, learning, and motor performance are extensive and complex, and much has been learned about them in recent years. This article does not address them. Instead, it examines some simple behavioral phenomena associated with learning to make visually directed responses accurately. There is not yet agreement about these phenomena. This article describes an experiment that measures them and presents a simple mathematical model that describes these phenomena well and may predict them. Thus, it puts some constraints on more detailed models. 

As to what produces the initial errors, an obvious possibility supported by a substantial body of research, is that the distance from an observer to a target is often mis-perceived. There are a large number of experiments that show that, even when there are effective cues to distance, verbal reports of perceived distance increase more slowly than physical distance (Da Silva, 1985; Teghtsoonian & Teghtsoonian, 1970). The data are often fitted with straight lines with slopes of about 0.8 or slightly concave downward functions. Other methods that require judgements of the relative lengths of perceived extents (e.g., Foley et al., 2004) also indicate that, even with multiple cues, perceived distances are less than physical distances and a slightly concave downward function of them. When cues are reduced perceived distances decrease further. 

However, there is not agreement on the accuracy of distance perception or how to measure it. Based on an extensive set of experiments, Loomis and associates concluded that visually directed actions provide a relatively pure measure of perceived distance, although cognitive and motor processes sometimes have an influence (Loomis and Philbeck, 2008). They and others found that blind walking to a target in the presence of full cues is close to accurate and they concluded that the target distance is accurately perceived in that condition. To the extent that visually directed actions measure perceived distance, errors in these actions are due to the misperception of distance. 

Loomis’ group and others have done a large number of experiments mostly using blind walking, a visually directed response not common in everyday life (Loomis, Da Silva, Fujita, & Fukusima, 1992; Loomis, Da Silva, Philbeck, & Fukusima, 1996). Subjects either walked directly to the target or by an indirect path. When subjects blind-walk straight to a target after viewing it with effective distance cues, responses are quite accurate. When subjects walk indirectly to a target or indicate its position while walking, the indicated target position is consistent with the position indicated by direct walking, and when effective cues are present, near the target. This literature is reviewed by Loomis and Philbeck (2008). Thus, there is inconsistency between perceived distance measured explicitly and blind-walked distance. This raises the question of whether factors other than perceived distance may affect visually directed responses. 

This study is concerned with throwing and blind walking. There is research on throwing without error feedback that shows that throws to targets up to 10 m away are approximately accurate (Eby & Loomis, 1987; Sahm, Creem-Regehr, Thompson, & Willemsen, 2005). However, in both of these studies of open-loop throwing, subjects practiced throwing with feedback just prior to the test. As will be seen, this has a large effect on performance. 

Blind walking in a natural lighted environment, even in the absence of deliberate practice, is often approximately accurate. This was first shown by Thomson (1983) and there have been several replications since. These are reviewed by Loomis and Philbeck (2008). However, prior to the blind walking, there may have been an opportunity for learning by walking around in cue conditions like those of the experiment (Li, Phillips, & Durgin, 2011). 

Thus, in natural lighted environments there are sometimes initial errors in open loop visuomotor responses and sometimes not. However, when distance cues are reduced, substantial errors occur in both blind walking and blind throwing (Philbeck & Loomis, 1997) and in blind reaching (Foley, 1975, 1977). These errors are highly correlated with verbal reports of the target distances, suggesting that these motor errors are closely related to perceptual errors. 

When virtual environment technology was introduced, it was hoped that virtual environments could be substituted for natural environments in research studies (Loomis, Blascovich, & Beall, 1999). They offer more flexibility and opportunities for control than do natural environments. However, it has been shown that in both head-mounted and large screen virtual environments, both motor and verbal indications of distance are shorter than in similar natural environments (see reviews by Piryankova, de la Rosa, Kloos, Bulthoff, and Mohler (2013) and Thompson et al. (2004)). Blind walking, blind throwing, and verbal reports are all shorter in virtual environments than in natural environments with similar cues. Attempts to improve virtual displays have not greatly reduced these differences, although verbal reports are more accurate in high quality virtual environments (Kunz, Wouters, Smith, Thompson, & Creem-Regehr, 2009). The under-responding in virtual environments is an advantage for the present study because the study's focus is on learning to correct initial motor errors. If we start with relatively large errors, we can more easily detect them and measure any correction. 

It has been shown that error feedback can reduce under-responding to distance in virtual environments (Richardson & Waller, 2005, 2007; Waller & Richardson, 2008; Kunz, Creem-Regehr, & Thompson, 2015; Mohler, Creem-Regehr, & Thompson, 2006). This study further examines the effects of error feedback. 

Although light carries information sufficient to compute the distances of most objects that produce or reflect it, and the human visual system has the capacity to extract much of this information, there are large, systematic errors in space perception (e, g., Foley, Ribeiro, and Da Silva, 2004; Durgin 2014). The most precise and reliable cues provide only relative distance information by themselves, but in combination they can determine absolute distance. For example, relative size and binocular disparity together do this. Yet humans often systematically make errors in responding to the location of objects and the extents among them. Why? It appears that the visual system does not use the information available to it to compute positions and extents accurately. This is manifested in conscious perception when we experience as equal, extents that are physically different. This distance misperception contributes to errors in visually directed responses. The fact that errors in visually directed walking and throwing are highly correlated with verbal reports of the misperceived target distances (Philbeck & Loomis, 1997) is evidence of this. 

In this study perceived distance will not be measured directly. It will enter as one factor required to account for responses to targets at different distances. I will refer to the relation between physical distance and perceived distance as the perceptual transform. As will be seen, it contributes to the initial error, but it does not always completely account for the magnitude of the initial error. 

When we consider the improvement in accuracy that occurs when feedback is provided, cognitive learning and motor learning are two obvious possibilities. By cognitive learning, I mean substituting another distance for the perceived distance in determining what response to make. I will refer to this as a cognitive transform. Another possibility is that a change is made in the relation between the target distance (perceived or cognitive) and the motor response. I will refer to this as a motor transform. Its effect is to change the motor response, that is, to adjust the force of the throws or the duration, speed, or step size of the walk required to reach the perceived or cognitive distance. The essential difference between cognitive and motor transforms, as those terms will be used here, is that a cognitive transform is independent of the task; it affects responses to all distance tasks. A motor transform is specific to the task, although there is some generalization to similar tasks (Kunz, Creem-Regehr, & Thompson, 2013). These transforms may or may not be associated with conscious awareness. 

Experiment

Tasks

To study these phenomena I chose two common visually directed tasks, blind throwing and blind walking. I also chose to do the study in a virtual environment. This has two distinct advantages for this purpose. First, the experimenter has complete control of the environment: it can be whatever the experimenter wants; objects can be made to appear, move around, and disappear at will; and the subject's position and orientation can be tracked continuously. Second, it has another property that is usually seen as a disadvantage, but is an advantage here: initial distance responses in a virtual environment often show substantial errors. 

Overview of experiment

An experiment was performed to determine what goes on when a human subject learns to perform visually directed responses accurately. Subjects first walk and throw to a target without any error feedback. This determines initial errors. They are then trained with visual error feedback on one of the two tasks. Next a transfer test is performed without feedback to determine the transfer of this training, if any, to the other task. Finally, they are instructed to ignore what they have learned and respond to the perceived positions of the targets. 

The experiment has four stages. In the first stage, human subjects walk and throw a beanbag to visual targets at different distances without visual feedback. They view the target and prepare to respond. The scene then goes black and they make their response. There are consistent errors, almost always in the direction of under-responding, and errors are greater for throwing than for walking. In the second stage, the subjects perform one of the two tasks again, this time with visual error feedback. After they respond in the dark, the scene reappears. In walk training, they see where they are and they see the target. They then walk to it. In throw training trials, the scene reappears with the beanbag visible where it landed in the virtual environment. In the second stage, half of the subjects are trained to walk to the target; the other half are trained to throw to the target. Improvement for both is substantial and very rapid. The third stage is a transfer of training stage. Each subject performs the task that he or she was not trained on. There is no feedback, just as in the first stage. In the fourth stage, subjects are instructed to ignore what they have learned and respond to the perceived positions of the targets. The purpose of the fourth stage is to test whether a conscious cognitive correction was learned in the training stage. If it was, and it can be ignored, the stage 4 responses should be different from the stage 2 or 3 responses and more like stage 1 responses. 

Method

Apparatus

An immersive virtual environment system was used to present stimuli and to measure responses. It is a version of the system developed by Beall and Loomis. It is described in Loomis, Beall, Macuga, Kelly, and Smith (2006). The version used in this experiment employed a G Force 4 graphics card and a Virtual Research V8 head-mounted display with a field of view of approximately 48 degrees horizontal x 36 degrees vertical. It used a four-camera Precision Position Tracker ver. 2.17 and Vizard software ver.2.5, both produced by WorldViz. The experimental control program was written in Python. 

On throw trials subjects threw a real half-pound beanbag underhand so as to land on the target, which lay on the floor of a virtual room. 

Stimuli

The virtual environment resembled a laboratory room in the building in which the experiment took place. A gray-level image of the room is shown in Figure 1. The walls resembled cinder block construction with distinct mortar lines. The floor resembled a red carpet with gold blobs of various shapes and sizes and smaller red blobs inside the gold. The ceiling was covered with square tiles containing a pattern of black dots. The room was 7.75 m long, 2 m wide, and 3.5 m high. There was a start line 1 m from the end of the room. The target was a luminous green disk 20 cm in diameter. It was placed on the floor of the virtual room, straight ahead of the start line and on the center line of the room at one of seven distances. The virtual room was situated inside a larger laboratory room. The floor of this room was carpeted and there were no obstacles in or close to the virtual room. The laboratory was dimly lit so that the experimenter could see the subject and mark where throws landed. If the subject walked outside the virtual room, a siren warned him or her to stop walking. This happened rarely. 

Figure 1.

 

Grayscale images of the virtual room. Top: Lighted room. The room appeared in color with the wall blocks in various shades of gray and tan. The carpet was red and yellow, and the ceiling was gray with black circles. Bottom: Dark Room. Everything was black except the target disk lying on the floor. The disk was luminous green.

There were two cue conditions, lighted room and dark room. When the room was dark, nothing was visible to the subject except the target, the start line, and five dim disks on the wall behind the start line. There were many more distance cues in the lighted room. Figure 1 is a gray-level image of the lighted and dark rooms as viewed from the start line. 

Subjects

There were eight subjects. They were recruited from the university community and were paid for their participation. They were told what they would be doing in the experiment. All had acuity of 20/20 or better with or without correction, normal stereovision and no visual or motor disability. There were five women and three men. Four were randomly assigned to the walk-trained group and four to the throw-trained group. They were told what tasks they would perform, but not about the rationale for the experiment. They got the instructions for each stage only at the start of that stage. 

Procedure

At the start of the experiment, the subject put on the helmet and was aided in getting it straight on his or her head and properly adjusted and fastened. The subject made three short walks and three throws after viewing the virtual environment with no target present. They saw the room, then it went dark, then they responded just as they would on experimental trials. The virtual room was completely dark between trials except for the dim start line and five dim disks on the wall behind the start line. 

The subjects were instructed: “On each trial a green disk will be on the floor of the room. It will be easy to see. Look at the disk and the room. You are free to look around and to move your head. Your task is to see where the disk is in the room and prepare to respond to it. The room will then go dark and you will indicate where the disk was located by walking to the location and standing on the disk or by throwing a beanbag so that it will land on top of the disk.” 

For walking trials, subjects were instructed: “Do not start walking until the room goes dark. Walk to where the disk is. Walk to the point where your feet would be centered on the disk. When you get there, stop and say ‘OK.’ If you hear a loud siren, stop walking. This means that you are outside the experimental area. The experimenter will tell you what to do.” 

For throwing trials, subjects were instructed: “Prior to the trial the experimenter will hand you a beanbag. When the room goes dark, throw the bag underhand so that it will land in the center of the disk. You may bend your knees to make the throw, but do not step forward.” 

At the beginning of each trial, the subject said, “Ready.” The subject then saw the word “walk” or “throw” indicating the task for that trial. This was followed by a warning sound. The room with the target present was then presented for 8 seconds. In the light condition, the entire room was visible; in the dark condition only the target disk was visible. The experimenter recorded the end position of a walk by pressing a key on the control computer. The subject then turned and walked back to the start line. The room remained dark except for the start line and five dim disks on the wall behind the start line, which were faintly visible in the dark room. On throw trials the experimenter spotted where the beanbag initially landed. Although the beanbag did not slide on the carpet, it sometimes moved slightly after landing. The experimenter then placed the base of a vertical rod with a light at the top over the landing position and recorded this position using the position tracker. This took a few seconds. 

The trials were blocked by task and cue condition. Target distance varied randomly within blocks and the order of the blocks varied in a counterbalanced order over sessions. 

Design

Stage 1: Walking and throwing without feedback

Stage 1 was designed to measure the initial responses in each task and cue condition. Subjects were instructed to walk and throw to where the target was. No distinction was made at this stage between apparent and physical position. Each session consisted of 28 trials, two tasks x two cue conditions x seven distances. There were five sessions. 

Stage 2: Walking or throwing with visual feedback and correction

Stage 2 was designed to measure learning in one of the tasks when error feedback was provided. Each subject was trained on one of the two tasks. The first part of each trial was the same as in stage 1. On walk trials, after the walk end position was recorded, the target was presented again on the floor of the lighted or dark room. If the target was in front of the subject he/she walked forward until standing on the target. If the target was behind the subject, he/she walked backward until standing on the target. During the feedback phase, the target had a vertical line extending straight up from it so that the subjects would know when they were directly above it. On throw trials, a few seconds after each throw, the target appeared again, together with an image of the beanbag located where it landed in the virtual room. During stage 2 only, the target appeared in the same position in the same cue condition for three successive trials. Subjects were instructed to use the error feedback to make their responses as close as possible to the target positions, so that their feet or the beanbag would be on the target. 

There were four sessions of training. In each session, each of the seven target distances occurred once in each of the two cue conditions. Each target distance was presented three times in succession, so there were 42 trials in each training session, 168 training trials total, 12 for each distance and cue condition. 

Stage 3: Transfer of training to the untrained task

Subjects were instructed to make their responses as close as possible to the target positions. There was no error feedback. Each cue condition, dark and light, was presented twice in each session, so there were 28 trials/session. There were three sessions. 

Stage 4: Open-loop walking and throwing to perceived positions

The subjects were instructed to ignore what they had learned in stage 2 and to walk and throw to “where the target appears to be.” There was no feedback. Each of the four conditions came up once in each session, so there were 28 trials/session. There were four sessions. 

Over the four stages, there were two tasks in stage 1 and in stage 4 and one task in stage 2 and stage 3 for a total of six tasks. Each was performed in the light room and the dark room, so there were 12 conditions. The experiment took place on weekdays over a period of four weeks. Each stage occurred on successive days in a single week. There were two to four days between stages. Table 1 summarizes the experimental design. At the end of the entire experiment, there was an open-ended debriefing in which subjects were asked to describe their experience. The proposed project was reviewed by the UCSB Human Subjects Committee and found to satisfy all federal, state, and university requirements with respect to the use of human subjects in research. 

Table 1.

 

Experimental design. Codes for conditions: D: dark, L: light, W: walk, T: throw. The design for the walk-trained and throw-trained groups was the same, except for which task was trained in stage 2. The other task was tested in stage 3. There were seven target distances in each condition.

 

Results

The position system recorded the x (lateral) and y (distance) coordinates of each response. These coordinates were used to compute the radial distance from the center of the start line to the response position; this is the actual response distance. All distances shown in the graphs are radial distances determined in this way. The average lateral error was small so the use of radial distance rather than the distance coordinate had a very small effect. 

Figure 2 shows the mean response distance as a function of the target distance over the four subjects in the walk-trained group in all 12 conditions. The smooth lines in the graphs correspond to a model that will be described in the model section. 

Figure 2.

 

Mean responses of the four subjects in the walk-trained group in the 12 experimental conditions. Top: Stage 1 (initial responses)—throwing and walking in light and dark prior to training. Middle: Stage 2 (walk training)—walking responses averaged over sessions 2, 3, and 4 of walk training and Stage 3 (transfer)— throwing after walk training with no feedback. Bottom: Stage 4 (respond to perceived position)—walk and throw.

In stage 1, prior to training, both the walk response and the throw response were short of the target. The error was greater for the throw response. Both responses were shorter in dark than in light in both tasks and all stages. The relation between target distance and response distance is close to linear, but often slightly concave downward. 

In stage 2, targets were under-walked on the first trial, but responses very quickly became much more accurate. In the first training session, not only did responses become more accurate over the three successive walks to the same target, but also the improved accuracy generalized to targets at different distances. As a consequence, walking became essentially accurate in the first training session. Responses in sessions 2, 3, and 4 of stage 2 were averaged to get the walk-trained response function shown in Figure 2. Walk training produced essentially accurate responses the light condition with slight under-walking in the dark condition. 

The stage 3 responses show that there is a small magnitude transfer of training from the walk task to the throw task. In stage 4, with instructions to respond to the perceived positions of the target, the responses are between those of stage 1 and stage 2 or 3, although for the throw response there is very little difference between the stages. Walk training has only a small effect on the throw response. 

Figure 3 shows the responses of the throw-trained subjects in all 12 conditions. The response functions tend to be slightly concave downward. As for the walk-trained group, in stage 1 these subjects initially walked and threw short of the targets. In both tasks, the errors were smaller than those for the walk-trained group. Since stage 1 was the same for both groups, this difference appears to be due to subject differences. 

Figure 3.

 

Mean responses of the four subjects in the throw-trained group in the 12 experimental conditions. Top: Stage 1 (initial responses)—throwing and walking in light and dark prior to training. Middle: Stage 2 (throw training)—throwing responses averaged over sessions 2, 3, and 4 of throw training and Stage 3 (transfer)—walking after throw training with no feedback. Bottom: Stage 4 (respond to perceived position)—walk and throw.

In stage 2, the first throw was an under-throw, but with the error feedback provided by seeing where the beanbag landed relative to the target, accuracy improved very quickly. By the end of the first session, after three throws at each target in each cue condition, throwing was almost accurate in the light condition and slightly less accurate in the dark condition. In stage 3, there was a substantial increase in the walked distance. The mean walk exceeded the target distance by as much as 0.5 m at the intermediate distances. Thus, throw training had a substantial effect on the walk response. In stage 4 both responses were shorter than in stages 2 or 3, but not back to stage 1 distances. Data for individual subjects are found in the Supplementary Data File. 

What is going on here? How can the response functions, which differ initially and are affected differently by training and instruction, be accounted for? 

Model

This section presents a model of these phenomena. The model is an elaboration of earlier models (Foley, 1977, 1991). The fundamental idea of the model is that there is not a single fixed relation between the stimulus and the motor response. Instead, the response depends on a sequence of up to three transforms that intervene between the stimulus and the response, and that vary independently from one condition to another. The first transform relates the stimulus to the perceived position (perceptual transform). The second transform relates perceived position to what I will call the cognitive position (cognitive transform). A cognitive transform will occur when the perceiver judges that the target is somewhere different from its perceived position. The third transform relates perceived or cognitive position to the motor response. The sequence of three transforms relates the target distance to the response distance. 

Since the three types of transforms depend on different factors, we would expect them to be affected differently in the 12 conditions of the experiment. The perceptual transform depends on the stimulus. We would expect it to be different in the light and dark cue conditions. Both the cognitive transform and the motor transform depend on error information, so we would expect them to be affected by feedback training. 

I sought to find a plausible set of assumptions, which when expressed mathematically would describe the main features of the data. I started with a very general model, which could fit almost any set of smooth monotonically increasing response functions. I then reduced the number of free parameters to find the best version of the model in the sense that the root mean squared error (RMSE) between the model prediction and the data was increased to a statistically significant extent by any further reduction in parameters and could not be reduced to a statistically significant extent by adding parameters. This led to different versions of the general model for the throw-trained group and the walk-trained group. 

Figure 4 illustrates the general model. Each target distance is perceptually transformed to a perceived distance. Often the perceived distance is cognitively transformed to a cognitive distance. The distance then undergoes a motor transform to produce the response distance. Some of the transforms can be null transforms that have no effect. To compute the response distance, start with the target distance, D, then apply the three transforms in sequence to compute the response distance, D_pcw or D_pct. The transforms all have the same mathematical form, differing only in their parameters and the constraints that the model places on the parameters. The figure simplifies the model somewhat. The perceptual transform depends on the cues available and could possibly depend on the task. 

Figure 4.

 

A schematic illustration of the model. D: target distance, D_p: perceived distance; D_pc, cognitive distance; D_pcw, walked distance; D_pct, thrown distance. The three distances are related by mathematical transforms that are referred to as perceptual (p), cognitive (c), and walk (w) or throw (t). Some of the transforms can be null transforms that have no effect.

All the transforms have the same mathematical form. It is a function used by Gilinsky (1951) to describe performance in a perceived distance bisection task. It has also been shown to describe how egocentric distance depends on physical distance in a study that modeled all the perceived extents among vertical stakes standing in an open field (Foley et al., 2004). The general form of the function is:  

\begin{equation}{D_{pcm}} = \;D/\left( {\left( {{F_m}\! + \!{G_m}{F_c} + {G_m}{G_c}{F_p}} \right)D + {G_m}{G_c}{G_p}} \right)\end{equation}

(1)

where F and G are constants that differ from one transform to another, and D and D_pcm are the input and output distances. In the general model m is the subscript for any motor transform. 

For each of the 12 conditions, the response distance is a transform of the target distance. This transform is composed of a sequence of three transforms. Some of these transforms are nulled by fixing G = 0 and F = 1, so that they have no effect, and the signal passes untransformed. If D is the target distance and D_pcm is the response distance, the most general statement of the model is:  

\begin{equation}{D_{pcm}} = m\left( {c\;\left( {p\;\left( D \right)} \right)} \right),\end{equation}

(2)

where p is the perceptual transform, c is the cognitive transform, and m is the motor transform, which is found to be different for walk and throw. Each transform has the form of Equation 1 and is applied to the output of the preceding transform to determine the response distance. 

Using the subscripts, p, c, and m to designate the parameters in the three transforms, the successive outputs can be expressed as:  

\begin{equation}{D_p} = \;D/\left( {{F_p}D + {G_p}} \right),\end{equation}

(3)

 

\begin{equation}{D_{pc}} = \;{D_p}/\left( {{F_c}{D_p} + {G_c}} \right),\end{equation}

(4)

 

\begin{equation}{D_{pcm}} = \;{D_{pc}}/\left( {{F_m}{D_{pc}} + {G_m}} \right),\end{equation}

(5)

where D_pcm is the output of the overall transform. 

We can make this overall transform more explicit by substituting the expression for D_p in D_pc:  

\begin{equation}{D_{pc}} = \;\frac{D}{{\left( {{F_p}D + {G_p}} \right)}}/\left( {\frac{{{F_c}D}}{{\left( {{F_p}D + {G_p}} \right)}}\;\; + {G_c}} \right),\end{equation}

(6)

 

\begin{equation}{D_{pc}} = \;D/\left( {\left( {{F_c} + {G_c}{F_p}} \right)D + {G_c}{G_p}} \right),\end{equation}

(7)

 

We can then substitute the expression for D_pc in Equation 5 to get the expression for the overall transform:  

\begin{equation}{D_{pcm}} = \;D/\left({\left({{F_m} + {G_m}{F_c} + {G_m}{G_c}{F_p}} \right)D + {G_m}{G_c}{G_p}} \right),\end{equation}

(8)

 

D_pcm is the output of the overall transform from target distance to motor response distance. It is the distance walked or thrown in one condition. It has the same mathematical form as the individual transforms. The walk and throw responses to the same target are generally different, so in practice, the motor parameters will be different for the two tasks as illustrated in Figure 4. 

For every one of the response functions in the 12 conditions, there are six parameters, two for each transform, making 72 parameters and 36 transforms in all. We could fit this model with all parameters free. However, as will be seen, some of the same transforms are applied in several conditions; so many fewer than 36 transforms are required to fit the data. Since the overall transform for each condition has the same form as the individual transforms, one can fit the transform function directly to the response functions in the 12 conditions. This model has 24 free parameters and fits as well as the best possible fit of the general model to the data. For the walk-trained group the RMSE of the best fit to this model is 8.75 cm; for the throw-trained group it is 10 cm. Consequently, it does not make sense to consider versions of the model with more than 24 free parameters. Models that have fewer than 24 free parameters must have fewer than two free parameters for each condition, so there must be constraints on parameter values across conditions. It turns out that such a model with many fewer free parameters fits each of the two data sets almost as well as the 24 free parameter model that employs two free parameters to fit the response function in each of the 12 conditions. There are two ways to reduce the number of free parameters. One is to null their effects by fixing F values to 0 and G values to 1. The other is to link the values of free parameters, so that transforms have the same parameter values in two or more conditions. Both ways of reducing the number of parameters were employed in fitting the data of this experiment. 

Even with limiting the number of free parameters to 24, there are a very large number of possible models. In practice, by fitting about 30 of these models to each data set, it was possible to reduce the number of possibly best models to very few. The models were fitted to the data using the Matlab fminsearch function, which finds the parameter values that minimize the RMSE. Since there are many local minima in the fit space, to find the lowest minimum it was necessary to make many fits of each model using different sets of starting values. The models differed in which parameters were free and which were linked by constraining them to be equal. The initial fits showed which parameters are potentially important. I systematically examined nested model sets with these free parameters to determine which model was best in the sense described above. 

The best models for the two training groups are shown in Tables 2 and 3. Each row corresponds to one of the 12 experimental conditions and contains the F and G parameters for the transforms used to fit the response function for that condition. For most conditions three transforms are required. Transforms that are linked to earlier transforms have the same parameter values. 

Table 2.

 

Walk-trained group. Parameters of the best model. RMSE = 13.4 cm. The model has six different transforms, each of which has two parameters, which are shown in bold. All the other transforms are equal to one of these six.

 

Table 3.

 

Throw-trained group. Parameters of the best model. RMSE = 10.5 cm. The model has five transforms, the parameters of which are in bold.

 

For the walk-trained group the best model has six different transforms. See Table 2. There are different perceptual transforms for the dark and light cue conditions. These two perceptual transforms are constant throughout the experiment, implying that perceived distance is not changed by either the stage 2 training or the stage 4 instruction. There is a throw transform in stage 1, which accounts for throws being shorter than walks. This transform also remains constant throughout the experiment. Walk training in stage 2 produces both a cognitive transform and a walk transform, each having the effect of increasing response distances, making them more accurate. The cognitive and walk transforms are the same in both cue conditions. The walk transform persists through the rest of the experiment. The cognitive transform is reduced in stage 4, indicating that, when subjects were instructed to respond to perceived positions, there was at least some reduction of the cognitive transform learned in stage 2. Thus, the model suggests that, in response to the walk training, subjects learn two things: the targets are farther away than they look and the subjects are not walking as far as they sensed themselves to be walking. The RMSE of the best fit is 13.4 cm. 

For the throw-trained group the best model has only five transforms. See Table 3. Again, there are different perceptual transforms for the dark and light cue conditions and these are constant throughout the experiment. There is a throw transform in stage 1, which accounts for throws being shorter than walks and remains constant throughout the experiment. Therefore, the stage 1 model is the same for both training groups, except for the different values of the parameters, which are attributed to subject differences. Throw training, unlike walk training, does not produce a motor transform. This is the one difference between the two models. The cognitive transform, learned in stage 2, is of greater magnitude than the cognitive transform learned by the walk-trained group, and it accounts for all of the learning. In stage 4 the cognitive transform is reduced, but is not completely nulled, by the instruction to respond to perceived positions. The RMSE of the best fit is 10.5 cm, just slightly worse than the best possible 24-parameter model. 

For both data sets, some values of the F parameter are negative. This implies that the transform is concave upward. These values are near zero and can be replaced by 0 with a very small increase in RMSE. This would make those functions linear. Most values of F are positive indicating that the functions are concave downward. 

Although many models were fitted to each data set, according to the criteria that I used, there was a best model for each set. Table 4 compares goodness of fit of the best models with the closest models in the nested set of best models for each data set. For both data sets, adding an additional free parameter does not significantly improve the fit and eliminating a parameter does make the fit significantly worse. Of the 30 models fitted to each data set, no other model fits these data sets better with the same number of parameters. Below I will consider some models that fit equally well. 

Table 4.

 

Comparison of the three best model fits to the data of each training group. p: perceptual, d: dark, l: light, c: cognitive, t: throw, w: walk. Numbers in the free-transform list refer to the stage at which a transform is introduced. Number refers to the number of free transforms. Parameters refers to the number of free parameters. RMSE is in cm. The F values correspond to the comparison of each model to the model with one more free transform. The best model for each data set is shown in bold.

 

In addition to these models, I fitted models in which all values of F were fixed equal to zero for both data sets, making the response functions linear. For the walk-trained group the best of these models produced an RMSE of 14.6 cm. This increase in error relative to the 12 free parameter model is not statistically significant at the 0.05 significance level. Therefore, this six free parameter model is the best model for the walk trained data set. However, for the throw-trained group, fixing the F values to 0 produced a RMSE of 17.4 cm, which is highly significantly worse than the corresponding model with F parameters free. This difference occurs because the throw functions are more concave downward than the walk functions. The F parameters will almost certainly be needed for experiments with either task that cover a larger distance range. 

For each of the best fitting models that I have described, there are other models that fit the data exactly equally well. For both data sets, the assumption that the task affects perceived distance can be substituted for the throw transform. Since the throw transform is applied on every throw trial, this transform can be combined with the perceptual transform for the walk trials to produce a different perceptual transform for the throw trials. This transform makes perceived distance on throw trials shorter than perceived distance on walk trials. Thus, the results are consistent with the action-dependent perception hypothesis that says, in this context, that when one views a target while intending an action toward it, its perceived distance depends on the intended action, and more specifically, that the more energy required for the response, the farther away the object will appear. Walking to a target requires more energy than throwing a beanbag to it. Evidence concerning this hypothesis has been mixed. See Philbeck and Witt (2015) for a comprehensive review. Thus, the results of the present study are equally consistent with the motor transform and the action dependent perceptual transform explanation of the difference between walking and throwing responses in stage 1. 

A simpler change is to substitute a walk transform for the throw transform in stage 1 to account for the difference between the two responses. This is equivalent to assuming that subjects throw to the perceived distance and walk substantially farther than the perceived distance. This model fits the throw-trained data as well as the best model and fits the walk-trained data almost as well, but is inconsistent with blind walking experiments. 

A third model that fits the data as well as the best model is one in which it is assumed that both training and the stage 4 instruction change the perceived distances. Thus, the effect of learning in stage 2 is to increase perceived distances and the effect of the stage 4 instruction is to decrease perceived distances. This second implication, that an instruction to respond to perceived distance changes the perceived distance, is not plausible. 

The specific models that were found to be best here are probably not general models of learning to walk and throw to targets. As described in the introduction, there is evidence that blind walking sometimes does not correspond to perceived distance measured explicitly. Thus, there may be both walk and throw transforms in stage 1. A more extensive experiment would be needed to determine that. Motor transforms appear to be quite flexible and may depend on the last situation in which they were used. Learning can affect cognitive or motor transforms. It may be that differences in situations, instructions, or expectations will determine what happens in a particular case. 

Stage 4 results

The stage 4 results merit more attention. In stage 4, when subjects were instructed to respond to the perceived positions of the targets, mean response distances decreased, but not enough to match the stage 1 responses. Why? I examined the response functions for individual subjects to see if they would help to explain this. For the walk-trained subjects, partial reversion of the walk responses is consistent with a nulling of the cognitive transform while retaining the walk transform. However, as Table 2 shows, the cognitive transform is not completely nulled. Partial nulling is found for three of the four subjects. The fourth shows partial nulling only at the longer distances. For the throwing response, individual subjects are inconsistent. One shows improved accuracy in stage 4, another shows over-nulling and the other two do not show a consistent effect. Overall, the throw responses are consistent with partial nulling of the cognitive transform. 

For the throw-trained subjects stage 4 effects are larger and what happened is clearer. Figure 5 illustrates this. It shows responses in the throw task in the light cue condition for the four subjects in this group in stages 1, 2, and 4. If there was only a cognitive transform, and if the stage 4 instruction was followed, we would expect complete reversion to stage 1 responses. Subjects 6 and 7 show essentially complete reversion. Subjects 5 and 8 show no reversion. Their stage 4 responses are like their stage 2 responses. The same subjects’ data for the walk task are similar, showing complete reversion for the same two subjects and very little and no reversion for the other two. Thus, the stage 4 instruction can have no effect or completely null the cognitive transform. For the stage 4 instruction to have an effect, the subject would have to distinguish between the perceived positions of the targets and the positions that they had learned to respond to. It appears that some subjects did not learn this. 

Figure 5.

 

Responses of all four throw-trained subjects in the light throw task in stages 1, 2, and 4. For subjects 6 and 7, stage 4 responses are close to stage 1 responses, consistent with the cognitive transform being completely nulled in stage 4. For subjects 5 and 8, there was essentially no nulling. Stage 4 responses are like stage 2 responses. The stage 4 instruction had no effect on these subjects’ responses.

At the end of the entire experiment, subjects underwent an open-ended debriefing. Since they had not been told to expect this, they were not prepared for it. They were asked to describe their experience in each stage. The typical response was, “I tried to follow the instructions.” In stage 2, with feedback presented on every trial, all subjects recognized that they were making errors at first and then became more accurate. Three of the four walk-trained subjects said that, in stage 2, they realized that they were not walking far enough, so they walked farther. None of the throw-trained subjects gave any explanation for their underthrowing. In stage 3, no subject said that he or she had done anything different from what they had done on the same task in stage 1, even though their responses were quite different. Since there was no feedback in this stage, they may not have known that they were throwing or walking farther. In stage 4, the two throw-trained subjects that showed complete reversion to stage 1 responses said that they responded differently in stage 4. One of the two that showed no reversion said that she responded to perceived positions throughout the experiment. Only one walk-trained subject had a clear sense of responding differently in stage 4. This subject showed partial reversion to stage 1 responses in both walking and throwing, consistent with nulling the cognitive transform, but not the walk transform. Overall, what the debriefing showed is that the subjects did not have much insight into what they had done in the different stages. It appears that some of them were not aware that they had learned anything, so when instructed to ignore what they had learned, they had nothing to ignore. 

Persistence of responses

In each stage of the experiment, there were several replications of each condition spread over a few days. The data presented are the mean responses over the stage. However, it is possible that the responses change from day to day, for example, due to forgetting. An important question about open-loop responses and especially learned corrections to perceptual errors is, how long do these responses persist in the absence of error feedback? To determine if there was any trend in the responses from day to day, I averaged the data across the four subjects in each condition for each daily session. In stage 2, of course, there was a change and this change persisted through the stage because feedback was given after every trial. Figure 6 shows mean responses over subjects as a function of session in stages 3 and 4, in which there was no feedback. Responses are shown for the dark cue conditions for both the throwing and walking responses. Although there is some variation from session to session, there are no consistent trends in responses. Thus, over the six replications of transfer measurement (two per session) and again over the four sessions with the stage 4 instruction, there is essentially no change in responses. Likewise, for the light cue condition there is no trend in the responses. It seems unlikely that the learned transforms will persist indefinitely. If they did, we would not get the under-responding found in stage 1. 

Figure 6.

 

Mean response over subjects as a function of session or replication in all the dark cue conditions of stages 3 and 4. Each line corresponds to one target distance. In stage 3 there were three sessions with two replications in each.

Discussion

In this study I started with responses that were inaccurate and examined how error feedback corrected them. There is a large literature on studies in which the visual stimulus produced by a scene is transformed in such a way as to produce inaccurate responses, and feedback is then employed to produce learning to correct them. These studies go back at least to Helmholtz's experiment in the 1860s (Helmholtz, 1962) in which he used wedge prisms to displace the images of targets left or right. This initially produced errors in the direction of pointing, but by pointing while seeing the errors, subjects quickly learned to point correctly. Stratton used prisms that inverted the optical images and Kohler used prisms that reversed the images left and right. Learning was much slower with these more radical transformations and never complete. Harris (1965) reviewed this literature and concluded that this learning can be ascribed to a change in the felt position of body parts relative to the head and eyes, a change in proprioception. In the present study the walk transform learned by the walk-trained group seems to be another kind of proprioceptive change, a change in the perception of how far one is walking. As the research on learning to compensate for optical transformations shows, such proprioceptive transforms cannot be nulled just by becoming aware of them; one has to learn another transform. 

There are more complex ways to change responses to targets. For example, Rieser, Pick, Ashmead, and Garing (1995) had subjects in pretests look at a target and then blind walk and blind throw to it. They do not say how accurate the responses were, but they used interventions to make them worse. In one experiment (experiment 6), they had subjects walk on a treadmill at a constant speed while the treadmill was towed at a different speed. Half walked faster than they were towed, half slower. Then the blind walking and throwing tests were repeated. The findings were that walking, but not throwing, was changed by walking on the moving treadmill. They also did the converse experiment to change throwing (experiment 7). Subjects stood on a trailer pulled by a minivan and threw beanbags to try to hit targets on the ground. In the throw harder condition, they faced backward; in the throw-easier condition they faced forward and threw to targets slightly off to the side. In the post-test subjects in the throw harder condition overthrew targets. There was no effect on blind walking. They interpret these effects as changes in the effort required to produce a desired response, again a proprioceptive transform. They use the term “calibrate” to refer to this adjustment of an action to be accurate under prevailing conditions. 

There are now many studies on calibration. Aside from being more complicated, they share many properties in common with the phenomena described in this article. In the Rieser et al. study, the throw-training condition has the goal of hitting a target; in walk training there is no goal in the training stage. The critical element seems to be that in both experiments subjects get visual feedback that is inconsistent with their proprioceptive sense of what they are doing. That false feedback changes their sense of what they are doing, so when they go to the post-adaptation test, they respond differently than they normally would to the perceived positions of the targets. In the language of this article, they have learned a motor transform. In the present experiment, the learned motor transform is adaptive to the test situation; in the Reiser et al. experiment, it is maladaptive. In calibration studies, the same issues are examined as are examined in the present study: what is required to change responses, is there transfer to other responses, what is required to change them back, what processes underlie these phenomena? The findings are also similar. In most calibration studies, some kind of error feedback is used to change responses. However, one's sense of movement while running can be changed just by eliminating all visual feedback (Durgin & Pelah, 1999; Durgin et al., 2005). There is sometimes, but not always, transfer to other responses. Loomis and Philbeck (2008) discuss recalibration in the context of measuring perceived distance. These studies and some others seek to understand the internal variables and processes that underlie recalibration. In this respect, they go beyond the present study in which transformations are represented by single-valued functions that simply take one distance into another. Motor and, sometimes, cognitive processes underlie these phenomena. What happens depends on the specific features of the experiment including pre-practice, expectations, specific instructions, and the nature of the feedback. 

I do not know how robust the results of the experiment described in this article are. The results show that under the particular conditions of the experiment, simple versions of the general model in which either a cognitive transform or motor transform or a combination of the two can account well for what happens at each stage. Changes in stimuli, instructions, or training method may change the outcome and require a different version of the general model. 

Conclusion

This study confirms that in a dark or lighted virtual environment, subjects walk and throw short of the target. It shows that, if they are given error feedback, subjects quickly correct the trained response to be near the target distance on average. Transfer tests show that throw training acts to produce near accurate throwing and walking. Walk training produces near accurate walking and improves throwing slightly. When, after the training and transfer test, subjects are instructed to respond to the perceived positions, some throw-trained subjects revert to their initial stage 1 responses and some show no reversion. Overall, walk-trained subjects show partial reversion, consistent with partial nulling the cognitive transform, but not the walk transform. Within a stage, responses are stable, except for stage 2, in which responses change rapidly in the first session. The results are accounted for by models in which different sets of a perceptual, a cognitive, and a motor transform are applied to determine the response functions in each condition. The effect of error feedback is to change one or both of these transforms. 

Acknowledgments

The author thank Jack Loomis for helpful discussion and the use of equipment. I thank Jerome Tietz for assistance with the experiment. 

Supported by NIH Grant EY 12743 and a grant from the University of California. 

Commercial relationships: none. 

Corresponding author: John M. Foley. 

Email: [e-mail]. 

Address: Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA, USA.