Integration of monocular motion signals and the analysis of interocular velocity differences for the perception of motion-in-depth

Abstract

We investigated how the mechanism for perceiving motion-in-depth based on interocular velocity differences (IOVDs) integrates signals from the motion spatial frequency (SF) channels. We focused on the question whether this integration is implemented before or after the comparison of the velocity signals from the two eyes. We measured spatial frequency selectivity of the MAE of motion in depth (3D MAE). The 3D MAE showed little spatial frequency selectivity, whereas the 2D lateral MAE showed clear spatial frequency selectivity in the same condition. This indicates that the outputs of the monocular motion SF channels are combined before analyzing the IOVD. The presumption was confirmed by the disappearance of the 3D MAE after exposure to superimposed gratings with different spatial frequencies moving in opposite directions. The direction of the 2D MAE depended on the test spatial frequency in the same condition. These results suggest that the IOVD is calculated at a relatively later stage of the motion analysis, and that some monocular information is preserved even after the integration of the motion SF channel outputs.

Introduction

Perception of three-dimensional (3D) motion is critical for surviving both in the wild and in modern human life. Information obtained by analyzing the motion signals of predators and prey and those of cars and pedestrians provides useful information for moving around safely and efficiently. Several cues are available for analyzing motion-in-depth signals through two-dimensional retinal images (see reviews; Howard & Rogers, 1995; Shioiri, Morinaga, & Yaguchi, 2002). If we focus on binocular perception, there are two cues to perceive motion-in-depth. One is the changing disparity over time (CDOT, or disparity change in time, DCT) and the other is the interocular velocity difference (IOVD). Disparity is the difference between the left and right retinal images according to the distant differences of objects. The interocular velocity difference is the direction and/or speed difference between the motion of the left and right retinal images caused by motion-in-depth of objects. 

Psychophysical studies have suggested that the visual system uses both of these cues. The use of the CDOT cues is supported by the finding that dynamic random-dot stereograms, in which disparity cues are isolated from velocity cues, provide perception of motion-in-depth (Cumming & Parker, 1994; Julesz, 1971; Read, 2002). The use of the IOVD cues is supported by several recent reports (Brooks, 2001, 2002a; Brooks & Mather, 2000; Fernandez & Farell, 2005, 2006; Maeda et al., 1999; Rokers, Cormack, & Huk, 2008; Shioiri, Kakehi, & Yaguchi, 2002; Shioiri, Nakajima, Kakehi, & Yaguchi, 2008; Shioiri, Saisho, & Yaguchi, 2000; Watanabe et al., 2008). Shioiri, Saisho and Yaguchi, for example, have shown that moving random dots in opposite directions between the two eyes provides motion-in-depth perception even when uncorrelated random-dots are presented to the left and right retinas to remove disparity cues. 

Although some studies suggest that the CDOT is more important than the IOVD as a cue for motion-in-depth (Harris & Rushton, 2003; Read, 2002), perhaps the relative importance depends on the conditions and tasks performed by the observer. The mechanism based on IOVD possibly plays a more important role than the mechanism based on CDOT in determining the speed and direction of 3D motion (Brooks, 2002a; Brooks & Mather, 2000; Brooks & Stone, 2006; Harris & Watamaniuk, 1995; Rokers et al., 2008; Shioiri et al., 2008). The velocity of 3D motion can be calculated from the velocities on the two retinas directly in the case of the IOVD when the eyes are fixating. In contrast, the 2D velocity components are required to calculate the 3D velocity if the motion-in-depth is detected using the CDOT, which has velocity information only along the depth axis. 

The physiological mechanism responsible for motion-in-depth is currently in dispute. Although cells sensitive to motion-in-depth have been reported in cat and monkey brains (Akase, Inokawa, & Toyama, 1998; Cynader & Regan, 1982; Poggio & Talbot, 1981; see also Takemura, Murata, Kawano, & Miles, 2007 for vergence controls), this sensitivity may be caused by the responses of cells sensitive to both disparity and frontoparallel motion (Chen, Wang, & Qian, 2001; Maunsell & van Essen, 1983; Qian & Andersen, 1997). The cells that are sensitive to both frontoparallel motion and disparity are not likely to contribute directly to the detection of either IOVD or CDOT cues. Although the difference in the directional selectivity of the left and right retinas is a key feature of the IOVD mechanism, the directional selectivity of binocular cells in the two eyes has been reported to be similar in physiological studies (Maunsell & van Essen, 1983; Ohzawa, DeAngelis, & Freeman, 1997). Sensitivity to disparity changes is a key feature of the CDOT mechanism. However, cells that are sensitive to both motion and disparity are sensitive to 2D front parallel motion at a certain depth plane (Born & Bradley, 2005; Born, Pack, Ponce, & Yim, 2006; DeAngelis & Newsome, 1999; Maunsell & van Essen, 1983; Ponce, Lomber, & Born, 2008). Although we have little knowledge of the sensitivity to motion-in-depth of the cortical cells, there should be a mechanism to cause the perception of motion-in-depth as we do perceive motion-in-depth. Hopefully, information obtained from psychophysical studies such as ours will help to identify the physiological mechanism responsible for perceiving motion-in-depth. 

The first purpose of the present study is to examine whether a monocular motion signal caused by the motion aftereffect (MAE) causes the perception of motion-in-depth. MAE is a phenomenon wherein motion is perceived in a static stimulus after the visual system is exposed to a moving stimulus. If the monocular MAE causes the perception of motion-in-depth, it is possible to use the phenomenon to investigate the IOVD-sensitive mechanism. The second and main purpose of this study is to estimate the spatial frequency tuning of the IOVD mechanism. Recent reports suggest that monocular MAE influences the perception of motion-in-depth (Brooks, 2002b; Fernandez & Farell, 2005, 2006). Brooks, for example, reported that prolonged exposure to monocular motion influenced the direction of 3D motion. He measured the directions from which the stimulus appeared to be approaching the center of the face and found a shift in direction toward the eye of monocular adaptation. This suggests that the monocular MAE contributes to the perception of motion-in-depth. There is, however, an issue that should be considered. The IOVD cue of motion-in-depth might not be perfectly isolated in the conditions in previous experiments. Since disparity cues were available in the test stimuli in previous experiments, it is possible that the MAE effect on position information (Nishida & Johnston, 1999; Snowden, 1998) influenced the perception of motion-in-depth under the influence of the disparity detection process. In Experiment 1, we examined whether the monocular MAE causes the perception of motion-in-depth in the test conditions, in which little contribution from CDOT cues was expected. 

We adopted direct method in order to examine whether the 3D MAE occurred after monocular motion adaptation. First, one eye was exposed to lateral motion to produce the monocular MAE. After exposure to the motion stimulus, the observer responded by indicating the direction of the motion-in-depth perceived in a static stimulus presented binocularly. We used gratings tilted in opposite directions between the two eyes to minimize the contribution of the disparity detection process. This manipulation is justified by the fact that physiological studies have shown no evidence of the presence of cells sensitive to different orientations between the left and right retinas (Bridge & Cumming, 2001), and also by the result of the third supplemental experiment of Experiment 1 (see later). 

If motion signals caused by the monocular MAE feed to the motion-in-depth process, the difference in perceived speed between the left and right retinal images will provide motion-in-depth perception ( Figure 1). Perceiving a rightward MAE with the right eye and no MAE in the left eye is expected to cause the receding motion-in-depth accompanied with rightward lateral motion (the direction of motion from front left to back right). If the observers perceive motion-in-depth in the predicted direction, we can conclude that the monocular MAE signals feed to the motion-in-depth process. A pilot experiment revealed that the MAE of the motion-in-depth (3D MAE) was perceived with a lateral motion aftereffect ( Figure 1b).    

 

Figure 1b

Figure 1

 

(a) Monocular MAE causes the perception of motion-in-depth. The observer looks at the center of the motion stimulus with one eye and observes the MAE with both eyes when it stops. Assumption of the IOVD process predicts the apparent perception of motion-in-depth. If the right eye is used to observe the leftward moving stimulus, the grating is predicted to be moving away with a rightward lateral motion. (b) After exposure of the right eye to the moving stimulus (leftward in the figure), the monocular MAE is perceived with the right eye. If the motion-in-depth mechanism that uses interocular velocity differences is sensitive to the monocular MAE, we expect to perceive motion-in-depth in the direction indicated by the thick line shown in the right of Figure 1a in the top grating. This can be examined by observing the MAE binocularly after looking at the movie with the right eye (left eye closed).

The main purpose of this study is to investigate the spatial frequency tuning of the IOVD process. For this purpose, the second experiment measured the 3D MAE for various conditions of the stimulus spatial frequency. The monocular motion detectors, which are assumed to exist at the first stage of the motion analysis, are likely to have a narrow-band spatial frequency tuning (Anderson, Burr, & Morrone, 1991; Bex, Verstraten, & Mareschal, 1996; Cameron, Baker, & Boulton, 1992; Nishida, Ledgeway, & Edwards, 1997). Here we refer to such channels as motion spatial frequency (motion SF) channels. The detectors at later stages, on the other hand, might have a broader spatial-frequency tuning, integrating the outputs of the motion SF channels. The fact that cells in the middle temporal (MT) area are more selective to speed than to temporal frequency (Maunsell & van Essen, 1983) suggests that the motion analysis in the MT area integrates the signals of motion SF channels. It is necessary to combine the signals from multiple SF channels in order for a motion detector to be selective to speed. 

For the process of motion-in-depth, there are two possibilities for this integration ( Figure 2). The IOVD process might combine motion signals of the left and right motion detectors with the same spatial frequency tuning. In this case, we expect that the 3D MAE will show narrow-band spatial frequency selectivity, which is similar to that of lateral monocular motion (2D MAE). Instead, the motion-in-depth detector might combine motion signals of the left and right motion detectors after integrating the signals of the motion SF channels in each eye. In this case, we expect that the 3D MAE will show selectivity to a broad range of spatial frequencies. In Experiment 2, we measured the spatial frequency selectivity of the 3D MAE, in order to reveal whether the interocular integration or the SF channel integration occurs first. In Experiment 3, we examined this issue by using superimposed gratings with different spatial frequencies moving in opposite directions as adaptation stimuli (Shioiri & Matsumiya, 2009). Independent analyses of multiple processes, such as the motion SF channels, cause the spatial frequency selectivity of the MAE. In this case, the MAE direction changes depending on the spatial frequency of the test stimulus. In contrast, a single motion process causes a MAE in the same direction across different spatial frequency test stimuli. 

Figure 2

 

The two possible motion-in-depth pathways based on IOVD. (a) The monocular motion signal integrated over all spatial frequency channels feeds into the motion-in-depth process, where motion signals from the left and the right eye are compared. (b) Interocular velocity differences are calculated for each spatial frequency channels before integrating spatial frequency channel outputs.

Experiment 1

The first experiment examined whether the monocular motion aftereffect causes the perception of motion-in-depth under conditions where we ensured that the contributions of disparity cues were minimized. In addition to the main experiment, we also conducted three supplemental experiments in order to confirm the effect of the monocular MAE on the perception of motion-in-depth. 

Method

We used a monocular motion stimulus to induce the MAE in either eye and measured the direction and the duration of the 3D MAE in a static test viewed binocularly. If the IOVD process is stimulated by the interocular velocity difference caused by the monocular MAE, we expect to observe 3D MAE after the monocular motion adaptation. 

Stimulus

The experiment consisted of an adaptation phase and a test phase. The adaptation stimuli were a pair of vertical gratings drifting horizontally in opposite directions (leftward on top and rightward on bottom in this experiment) with a drifting rate of 5 Hz (Figure 3a). We used relative motion stimuli since motion-in-depth is barely perceptible without relative motion components (Erkelens & Collewijn, 1985; Regan, Erkelens, & Collewijn, 1986; Shioiri et al., 2000). The adaptation stimulus was presented to either the left eye or the right eye (the right eye adaptation is shown in Figure 3a). The stimulus to the other eye was a uniform gray field with the same luminance as the average luminance of the gratings. The eye of the adaptation stimulus was chosen randomly from trial to trial. The observer viewed the stimuli through a mirror stereoscope to fuse the left and right images. A fixation cross and vertical and horizontal nonius lines were used to aid and confirm the fusion. The test stimulus was a pair of static gratings oriented in opposite directions (±45°). Opposite orientations of the images were shown to each eye (Figure 3b). This minimized or eliminated the influence of the disparity sensitive mechanism. Fusing the images provides an unstable surface with spatially non-uniform binocular rivalry. Even with unstable depth perception, the motion aftereffect was observed clearly. For both the adaptation and test stimuli, the spatial frequency was 1.25 c/deg and the luminance contrast was 0.5 with an average luminance of 30 cd/m². The areas outside the stimulus regions on the screen were dark (<0.01 cd/m²) throughout the experiment. 

Figure 3

 

Experiment 1. (a) Adaptation stimulus and (b) test stimulus. Adaptation gratings were presented either to the left or to the right eye (the right eye adaptation is shown). The direction and the drifting temporal frequency were unchanged in Experiment 1. The adaptation and test gratings had the same spatial frequency of 1.25 c/deg. All the test stimuli were oriented in opposite directions between the two eyes to eliminate any effect of disparity signals.

Apparatus

Images for the two eyes were presented on a CRT display (Sony CPD-G500J) controlled by a computer (Apple Power Macintosh G4). The refresh rate was 120 Hz. The observers viewed the display through mirrors arranged to fuse the images at an optical distance of 82 cm. Each stimulus field consisted of 300 × 300 pixels, which corresponds to a visual angle of 10° × 10°. 

Procedure

The sequence of the trial was as follows. After checking the alignment of the nonius lines, the observer pressed a key to initiate the presentation of the adaptation stimulus, which lasted for 30 s. The test stimulus was presented 0.5 s after the termination of the adaptation stimulus and lasted until the observer pressed a key to indicate the time of the MAE disappearance. A blank interval was interposed between the trials for 5 s to remove the carry over of the aftereffect. A uniform gray field of 30 cd/m ² was presented during the interval between the adaptation and the test as well as during the interval between trials. There were two keys that the observer used to indicate the direction in which the upper half of the stimulus appeared to move. One key indicated approaching motion and the other indicated receding motion. The observers were instructed to indicate the direction of only the depth motion components and to ignore the lateral motion components. In addition to the 3D MAE criterion, we used a criterion for the disappearance of 2D lateral motion, where the observers were instructed to press a key when the lateral motion disappeared (it usually disappeared after the 3D MAE). The two criteria were used in separate sessions. 

The direction of the adaptation stimulus was always leftward for the top stimulus and rightward for the bottom stimulus in Experiment 1. The direction of the 3D MAE was expected to be opposite for the left eye adaptation and the right eye adaptation. The three observers (one author and two naïve observers) with normal or corrected-to-normal visual acuity participated in the experiment after at least one training session. No feedback to responses was given in either the training or the experimental sessions. Each observer undertook two sessions of ten trials (five for adapting each eye) for each MAE criterion. 

We conducted three supplemental experiments to confirm the conclusions of the main experiment. In the first supplemental experiment, we measured the interocular transfer of the lateral MAE of the stimulus that was used in the main experiment. If, for example, the interocular transfer of the MAE is close to 100%, there should be little difference in velocity between the left and right eyes and no 3D MAE is expected to appear. The experimental procedure was the same as that in the main experiment except that the test stimuli were monocular. Either the adapted or unadapted eye was stimulated in the test phase. We used either vertical or oblique gratings as the test stimulus in order to examine the effect of the orientation difference between the adaptation and test stimuli. The same three observers participated and each observer undertook ten trials for each condition. 

In the second supplemental experiment, we used flickering test gratings, which were also tilted at 45°. It has been reported that the interocular transfer in a flicker test can be as high as 100% (Nishida, Ashida, & Sato, 1994). Weaker signals of motion-in-depth were expected in a flicker test than in a stationary test under the assumption that the IOVD caused by the difference in MAE between the eyes is expected to act as the source of motion-in-depth MAE. We used a square-wave flicker with a temporal frequency of 5 Hz. The observers judged the direction of motion-in-depth as in the main experiment. The same three observers participated in and each observer undertook three sessions of ten trials (five for adapting each eye). We used a larger number of trials in this experiment since the response was less stable than that in the other experiments. 

The third supplemental experiment investigated the effect of different orientations of the test stimuli between the left and right eyes. In the main experiment, we used gratings oriented in opposite directions in the test in order to minimize the influence of the disparity-sensitive mechanisms. Although we could assume that the use of the MAE isolates motion-sensitive mechanisms from those sensitive to position or disparity, this might not be perfect isolation. The MAE might also influence the position perception (Nishida & Johnston, 1999; Snowden, 1998) and the position difference might activate the CDOT process to cause motion-in-depth perception. Since gratings with opposite orientations cause no stable binocular depth perception, we consider that disparity cues do not play a role in the 3D MAE of the stimulus. However, the lack of perception of stable depth might not be sufficient to rule out the possible contribution of a disparity-sensitive process in a stimulus. There might be disparity-sensitive processes that do not directly influence depth perception although they influence it indirectly, for example, by controlling vergence eye movements. 

The third supplemental experiment afforded empirical evidence of the small influence of disparity cues for the 3D MAE. The experiment compared the MAE duration between the two conditions in order to examine whether the orientation difference between the two eyes changes the strength of the 3D MAE. If the disparity cue plays a significant role, the MAE should be stronger under the condition in which gratings with the same orientation were presented to the two eyes. The stimuli and the experimental procedure were the same as those in the main experiment with the exception that there were two test conditions and the adaptation period was 20 s. In the different orientation condition, the test grating differed in orientation between the left and right eyes as in the main experiment. In the same orientation condition, the test grating had the same orientation for both eyes. Three new observers with normal or corrected-to-normal vision participated in the experiment. We measured only the 3D MAE in this experiment, and each observer participated in two sessions of ten trials for each orientation condition. 

Results

Motion-in-depth MAE

In the main experiment, we assumed that the IOVD mechanism signals receding motion on top and approaching motion on bottom when the adaptation gratings were presented to the right eye ( Figure 1). The opposite direction in depth (with the lateral motion component in the same direction) was assumed when the adaptation gratings were presented to the left eye. We express our results as the percentage of responses that were consistent with these predictions. 

The percentage of responses in the predicted direction was 90% or higher for the three observers: 100%, 95% and 90% for DK, HU, and MNK, respectively, for the average of the left and right eye adaptation conditions. These results confirmed the pilot observation regarding the perception of the 3D MAE through the monocular MAE. It should be noted that the motion direction was less clear than that of the 2D MAE as can be speculated from the fact that (unlike the 2D MAE) the percentage of observed motion in the predicted direction was not 100% for two of the three observers. 

Figure 4 compares the durations of the 2D and 3D MAEs. They are averaged over the trials with responses in the predicted direction (including other responses in the analysis does not change the general trends of the results). Figure 4 shows that the 3D MAE lasted about half the duration of the 2D MAE. This corresponds with the observer's report that motion-in-depth was seen at the beginning and only 2D frontoparallel motion was seen later. The finding that the monocular motion signal causes motion-in-depth perception confirms that motion-in-depth can be perceived solely with IOVD cues and also that the MAE technique can be used to investigate the IOVD mechanism. 

Figure 4

 

Experiment 1. The MAE durations of 2D and 3D motion. White bars represent the 3D MAE and black bars represent the 2D MAE. The percentage of responses in the predicted direction for the 3D MAE was 100%, 95% and 90% for DK, HU, and MNK, respectively, for the average of the left and right eye adaptation conditions.

Interocular transfer

In the main experiment, we simplified the situation, assuming that motion aftereffect is a monocular phenomenon and the adaptation stimulus in one eye does not influence the motion perception in the other eye ( Figure 1). However, the MAE is transferred to the unadapted eye and the amount is often more than 50% (Nishida et al., 1994; Wade, Swanston, & de Weert, 1993). Since the interocular transfer of the MAE reduces interocular velocity differences, the amount of interocular transfer should be small enough to allow perception of motion-in-depth. The first and the second supplemental experiments were conducted to investigate the effects of interocular transfer. 

Figure 5 shows the MAE duration of the adapted and unadapted eyes for the vertical and oblique test gratings measured in the first supplemental experiment. The percentage shown near the bar is the ratio of MAE durations between the adapted and unadapted eyes. The longest MAE was found when the vertical test gratings were presented to the adapted eye. The second longest MAE was found when the oblique gratings were presented to the adapted eye and the third longest was when the vertical gratings to the unadapted eye. The shortest MAE was found for the oblique test of the unadapted eye. Differences in the orientation or in which eye is stimulated between the adaptation and the test reduced the MAE duration. 

Figure 5

 

Experiment 1. The duration of the 2D MAE for each test stimulus condition. The number attached to the result of each interocular condition indicates the magnitude of interocular transfer (the ratio of the MAE duration of the unadapted eye to that of the adapted eye).

The interocular velocity difference considered in the main experiment was between the oblique gratings in the adapted and the unadapted eyes. The interocular difference in MAE duration is larger with the oblique test than with the vertical test (see the ratio of the MAE duration between the adapted and the unadapted eyes in Figure 5). This indicates that the oblique test was more appropriate in the present study than the vertical test because of the larger IOVD. 

In the second supplemental experiment, we used flickering gratings in the test. The average percentage of responses in the predicted direction was 70%, 67% and 70% for DK, HU, and MNK, respectively. Less reliable 3D motion was seen in the flickering test than in the static test (where the percentage was 100%, 95%, and 90%). The MAE duration was also very short and it was not possible to measure because the MAE disappeared soon after the test presentation. This agrees with the prediction of a weak 3D MAE in the flicker test due to the large interocular transfer of the monocular MAE. Although no 3D MAE is expected in the flicker test if the interocular transfer of the flicker MAE is 100%, this is usually not the case (Hess, Demanins, & Bex, 1997; Nishida & Ashida, 2000; Steiner, Blake, & Rose, 1994). 

Orientation difference between the eyes in the test

In the third supplemental experiment, we compared the 3D MAE durations between test conditions with the same and opposite orientations. The result showed that the MAE duration was slightly longer in the different orientation condition than in the same orientation condition. The strong disparity cues did not increase, but rather slightly decreased the 3D MAE duration. The MAE durations of the three observers were 2.6 s, 5.0 s and 9.7 s in the different orientation condition (with 100%, 100% and 85% of responses in the predicted direction), and 2.1 s, 4.6 s, and 9.2 s in the same orientation condition (with 100%, 95% and 100% of responses in the predicted direction). A t-test showed the difference in MAE duration is statistically significant for one of the observers ( p < 0.01) while not significant for the other two ( p > 0.1). The effect of the test orientation is small and the effect is in the opposite direction than predicted by the assumption that the disparity cues contribute to the 3D MAE. These results rule out the possibility that disparity cues play a role in seeing 3D MAE. We therefore conclude that the interocular velocity differences caused by the monocular MAE cause the perception of motion-in-depth (3D MAE). 

Experiment 2

Experiment 2 compared the spatial frequency dependence between the 2D and 3D MAEs. It has been shown that multiple SF motion channels contribute to the 2D MAE. Several studies have shown that the MAE is largest when the test spatial frequency is the same as the adaptation spatial frequency (Bex et al., 1996; Cameron et al., 1992; Nishida et al., 1997). For the 3D MAE, there are two different predictions for the spatial frequency selectivity. First, it may be similar to the spatial frequency selectivity of the 2D MAE. If we assume that the motion-in-depth signal is calculated before combining the outputs from different spatial frequency channels (Figure 2b), the spatial frequency tuning for motion-in-depth should be similar to that for lateral 2D motion. Second, the spatial frequency selectivity of motion-in-depth may be broader than that of 2D motion. If the motion-in-depth signal is calculated after combining the outputs from the different SF motion channels of each eye (Figure 2a), the spatial frequency tuning for motion-in-depth should be broadened. 

The duration of the 2D MAE is longer than that of the 3D MAE (see Figure 4), and it is important to consider the effect of absolute MAE duration. A nonlinear effect, such as a flooring effect, may reduce the difference among different spatial frequency conditions, for example with short MAE durations. This could cause apparent difference in spatial frequency dependence between the 2D and 3D MAE. To examine the effect of absolute MAE duration on the spatial frequency dependence, we used two different temporal frequencies for adaptation: 1 Hz and 5 Hz. Since the MAE is known to be stronger with 5 Hz than with 1 Hz for 2D lateral MAE (Pantle, 1974), we can investigate the spatial frequency dependence with long and short average MAE durations. 

Method

The method was similar to that of Experiment 1. The differences from Experiment 1 were the spatial and temporal frequency conditions. There were two sets of experimental conditions. In the first set, the spatial frequency of the adaptation stimulus varied with a fixed temporal frequency, and the temporal frequency varied with a fixed spatial frequency in the second set. The test spatial frequency was varied in a similar manner in both sets (the test gratings were always tilted 45°). 

In the first set, three spatial frequencies (0.45, 1.25 and 3.75 c/deg) were used for both the adaptation and test stimuli. The adaptation grating moved at a temporal frequency of 5 Hz for 30 s. In the second set, either 1 Hz or 5 Hz was used, and the presentation duration was 20 s. The spatial frequency of the adaptation was fixed at 1.25 c/deg while five test spatial frequencies were used (0.45, 0.75, 1.25, 2.15 and 3.75 c/deg). Five new observers with normal or corrected-to-normal vision participated in the experiment. Two observers participated only in the first set and the two others participated only in the second set. One observer, an author participated in the both sets of conditions. Each observer ran two sessions (two adaptation directions) of ten trials (five adaptations for each eye) for each condition. 

Results

Figure 6 shows the average MAE duration for the three observers in the first set of conditions, where adaptation spatial frequency was varied. Different panels show different spatial frequency conditions of the adaptation stimulus. The arrow indicates the adaptation spatial frequency in each panel. The MAE is longest when the test spatial frequency is the same as the adaptation spatial frequency (a narrowband spatial frequency tuning). This is consistent with the existence of motion SF channels with different spatial tunings. 

Figure 6

 

Experiment 2. The MAE duration averaged for the three observers as a function of the spatial frequency of the test gratings for the first set of conditions. Different panels show the results from different adaptation spatial frequencies. Open symbols represent the 2D MAE and solid symbols represent the 3D MAE. Error bars represent the standard error across observers.

In contrast to the 2D MAE, the 3D MAE did not much depend on test spatial frequency. The MAE duration for motion-in-depth was approximately constant across test spatial frequencies (a broadband spatial frequency tuning). These results are consistent with the hypothesis that the motion-in-depth is calculated after combining the outputs from the motion SF channels of each eye ( Figure 2a). 

Figure 7 shows the results in the second set of conditions. The MAE duration was longer for the 5 Hz adaptation than for the 1 Hz adaptation. This is consistent with the prediction based on the fact that the motion sensitive pathway has a sensitivity peak at around 8 Hz with gradual reductions for higher and lower the temporal frequencies (de Lange, 1958; Kelly, 1979; Merigan & Maunsell, 1993; Pantle, 1974). However, the effect of the test spatial frequency on the MAE duration was similar for both adaptation temporal frequencies. The effect is also similar to that in the first set of conditions. The longest MAE duration is shown when the test spatial frequency is the same as that of the adaptation gratings for the 2D MAE. In contrast, there is little difference across test spatial frequencies for the 3D MAE. These results suggest that the difference in the spatial frequency selectivity between the 2D and the 3D MAEs cannot be attributed to the difference in average MAE duration or a flooring effect of the measurements. 

Figure 7

 

Experiment 2. The MAE duration averaged as a function of the spatial frequency of the test gratings for each observer of the second set of conditions. Open symbols represent the results of the 5 Hz adaptation and solid symbols represent the results of the 1 Hz adaptation. Error bars represent standard error over all trials.

The results of Experiment 2 suggest that the IOVD process has spatial frequency selectivity broader than that of the lateral motion mechanism. 

Experiment 3

In Experiment 3, we examined whether the 3D MAEs caused by two different spatial frequencies interact with each other. Experiment 2 suggests that the IOVD process integrates the signals from the motion SF channels before calculating the interocular velocity differences. In that case, the 3D MAEs caused by different spatial frequency adaptations should interact with each other. No 3D MAE might be seen after exposure to two superimposed adaptation gratings with different spatial frequencies moving in opposite directions even when the same adaptation causes a 2D MAE in the test of either spatial frequency. Each spatial frequency test is expected to cause a 2D MAE in the direction opposite to that of the motion of the same spatial frequency component in the adaptation stimulus. 

Experiment 3 used a stimulus with mixed spatial frequencies. The adaptation stimulus was a pair of superimposed drifting gratings with different spatial frequencies moving in opposite directions. If each of the two possible motion detectors with different spatial frequency selectivity is adapted by either of the two superimposed gratings, the MAE direction is opposite for the two test conditions with the spatial frequencies used in the adaptation. This pattern of results is expected for the 2D MAE because the 2D MAE is shown to be spatial frequency selective in Experiment 2. In contrast, the low spatial frequency selectivity of the 3D MAE suggests that the 3D MAE does not depend on the spatial frequency of the test stimulus. If, for example, no MAE is obtained with a given spatial frequency test, no MAE is expected with another spatial frequency test. 

Method

Stimulus

The adaptation stimulus in Experiment 3 consisted of superimposed vertical gratings with spatial frequencies of 0.45 and 1.25 c/deg ( Figure 8). They moved in opposite directions, and the directions of the upper and lower halves were also opposite (the 1.25 c/deg grating moved to the right and the 0.45 c/deg grating moved to the left on top, they moved to the left and right, respectively, on bottom). The test stimulus was a 0.45 c/deg grating, a 1.25 c/deg grating or the same combined gratings as the adaptation stimulus. The drifting rate of both spatial frequency gratings was fixed at 5 Hz. 

Figure 8

 

Experiment 3. The adaptation stimulus and the test stimulus used. (a) The adaptation stimulus consisted of superimposed gratings with different spatial frequencies moving in opposite directions (0.45 and 1.25 c/deg). The adaptation gratings were presented either to the left or to the right eye, while a gray uniform field was presented to the other eye. The direction and the speed were unchanged. We used several combinations of the relative contrasts between the two gratings to manipulate the relative strength of the two adaptation gratings. (b) The test stimulus was either a pair of 0.45 c/deg gratings, a pair of 1.25 c/deg gratings or a pair of overlapped gratings. All test stimuli were oriented in opposite directions between the two eyes.

Contrast adjustment

The relative contrast of the two gratings is an important factor in this experiment. We conducted a preliminary experiment in order to find the equivalent contrasts for the two superimposed gratings in terms of MAE strength. In this preliminary experiment, the contrast of the 1.25 c/deg grating was adjusted against a fixed contrast of 0.5 for the 0.45 c/deg grating in such a way that the superimposed grating test produced no 2D motion aftereffect. The preliminary experiment used a method similar to a staircase procedure. In this method, the contrast of the 1.25 c/deg grating in the adaptation stimulus changed according to the response of the MAE direction in the test. The observer responded the direction of the MAE (left or right) using two keys after every motion adaptation of 20 s until the observer judged there was no MAE. The observer then indicated that no motion was observed by using another key to terminate the trial. The contrasts obtained from two sessions of the preliminary experiment were 0.063, 0.102 and 0.050 for the three observers. Using this contrast as the standard, the contrast of the 1.25 c/deg grating in the adaptation stimulus was varied in multiples of 0.5, 1, 2, or 4. The contrast of the test gratings was fixed at 0.2. 

Procedure

The procedure of Experiment 3 was similar to that of the previous experiments. The observer responded the direction and the duration of the MAE. In this experiment, the observer was allowed to respond “no motion” since there were conditions in which no MAE was expected. We analyzed the results as follows. For each condition, a positive or negative sign was assigned to the MAE duration depending on the direction of the response. When the response was the consistent with the MAE direction of the 1.25 c/deg grating (i.e., opposite to the motion of the 1.25 c/deg grating of the adaptation), the duration was regarded as positive. When the response indicated the direction consistent with the MAE of the 0.45 c/deg grating, the duration was regarded as negative. The response of “no motion” was counted as an MAE duration of zero. Three observers from Experiment 2 participated in this experiment. Each observer undertook a session of ten trials (five trials for adaptation of each eye) in each condition. The adaptation motion was kept in the same direction. The 0.45 c/deg grating moved leftward and the 1.25 c/deg grating moved rightward on top and vice versa on bottom. 

Results

Figure 9 shows the MAE durations for the different test stimulus conditions. We analyzed only the responses to the dominant direction in each condition. The percentages of the other responses for the 3D MAE were 5%, 8%, and 2% for TT, MY and HN, respectively, while no such responses were obtained for the 2D MAE. The parameter is the relative contrast of the 1.25 c/deg grating of the adaptation stimulus. Positive durations indicate the MAE dominated by the 1.25 c/deg, and negative durations indicate the MAE dominated by the 0.45 c/deg. The duration of the 2D MAE shows a clear dependence on the test stimulus. After the equivalent contrast adaptation (contrast ratio of 1), no MAE was observed in the superimposed test (indicated by “0.45 & 1.25” in the figure). This is not surprising since the contrast combination used for the superimposed test was chosen in the preliminary experiment to be equivalent adapters for the two possible motion detectors sensitive to the 0.45 c/deg and 1.25 c/deg gratings. The MAE was found in the direction opposite to the 0.45 c/deg grating motion when the 0.45 c/deg test was used, and in the direction opposite to the 1.25 c/deg grating motion when the 1.25 c/deg test was used. This pattern indicates multiple motion detectors with different spatial frequency tunings (motion SF channels). 

Figure 9

 

Experiment 3. The MAE durations of three different test patterns. Different symbols represent different contrast ratios of the two combined gratings in the adaptation stimulus. Open symbols connected by dashed lines represent the 2D MAE and solid symbols connected by solid lines represent the 3D MAE. Error bars represent the standard error over all trials. Note that the MAE values for contrast ratios of 1 and 2 for the 3D MAE of TT are zero for all test stimuli.

Contrary to the 2D MAE, the 3D MAE showed little dependence on the test stimulus. No motion-in-depth was reported in any of the test conditions having equivalent contrast adaptations, with the exception of one (the 0.45 c/deg test of MY). With the contrast ratios of the two adaptation gratings larger than 1, the 3D MAE direction is opposite to the 1.25 c/deg grating motion (positive values in Figure 9) for all test conditions except the results of the contrast ratio of 2 for TT (no motion responses for all test conditions). With a contrast ratio of 0.5, the 3D MAE direction is opposite to the 0.45 c/deg grating motion (negative values in Figure 9) for all test conditions. These results are consistent with a single channel model, in which only one type of motion detector that is sensitive to both 0.45 c/deg and 1.25 c/deg spatial frequencies contributes to the MAE. The differences in direction between the 2D MAE and the 3D MAE strongly suggest that different motion mechanisms with different spatial frequency selectivities cause the 2D MAE and the 3D MAE. 

In terms of the integration of the motion signals, the present results support the model of the integration of monocular signals prior to interocular velocity comparison ( Figure 2a). This suggests that the motion-in-depth process based on IOVDs is at a stage later than the integration process of the signals from the motion SF channels. 

Discussion

Experiment 1 revealed that the monocular MAE caused the perception of motion-in-depth. Since the contribution of any disparity cue is highly unlikely in the experiment, this fact indicates that motion-in-depth can be perceived with solely IOVD cues. We further investigated the spatial properties of the IOVD process in the following experiments and found evidence suggesting that the interocular velocity differences are calculated after combining the outputs of the motion SF channels in each eye. 

In Experiment 2, we compared the spatial frequency dependence of the 2D and the 3D MAEs. Little or no spatial frequency dependence was found for the 3D MAE whereas the largest MAE was found at the spatial frequency that was the same as the adaptation stimulus for the 2D MAE. In Experiment 3, we investigated the MAE of two superimposed gratings with different spatial frequencies, which moved in opposite directions. The 2D MAEs of the different spatial frequency tests were in different directions after the adaptation condition that provides no MAE when the superimposed gratings were used in the test (the equivalent contrast adaptation). In contrast, no 3D MAE was observed with either spatial frequency test after the equivalent contrast adaptation. This suggests that the two MAEs in opposite directions canceled each other out in the integration of the motion SF channels. These results from Experiments 2 and 3 strongly support an IOVD model that integrates the signals of motion SF channels before the comparison of velocities between the eyes ( Figure 2a). 

The duration of the MAE is often used for estimating its strength. However, the strength and the duration do not necessarily reflect the same aspect of motion analysis. It is important to consider how the duration of the MAE is related to its strength. Similarity has been reported in terms of the temporal frequency dependence between the initial velocity and the MAE duration (Pantle, 1974): both showed a peak at about 5 Hz. In terms of spatial frequency selectivity, both the velocity estimation and the MAE duration measurements provided similar selectivity (Bex et al., 1996; Cameron et al., 1992). Therefore, we believe that the spatial frequency dependence of the MAE duration reflects the spatial frequency tuning of the concerned motion-sensitive mechanisms. Based on this argument, we conclude that the IOVD process has a broad spatial frequency tuning. 

2D and 3D motion pathways

Although the monocular MAE provides perception of motion-in-depth, this does not imply that all monocular motion signals are integrated in perceiving motion-in-depth. On the other hand, our results suggest that there is a pathway for 2D motion perception independent of that for motion-in-depth. In order to interpret the spatial frequency selectivity of the 2D MAE, the site of the MAE must be at each motion channel prior to the integration of the outputs of the motion SF channels. Under this assumption, a pathway different from the one mediating motion-in-depth is necessary for the perception of 2D motion. Without such a pathway, the 2D and 3D MAEs should show the same spatial frequency dependence, and possibly the same MAE duration as well, under any conditions. The differences between the results for 2D and 3D MAEs, therefore, suggest that the pathway for the 3D motion is at least partially independent of the pathway for the 2D motion. It also suggests the 3D MAE is determined primarily at the integration process of motion signals of each eye ( Figure 2a). However, no spatial frequency dependence of the 3D MAE is puzzling because the process concerns to the 3D MAE is assumed to have input from the spatial frequency selective mechanisms. Although small effects of test spatial frequency in 3D MAE shown in Figure 7 (MAE duration is slightly longer at 1.25 and 2.15 c/deg than other spatial frequencies on average) may be attributed to the MAE of the spatial frequency selective mechanisms, this is no more than speculation. Further investigation is required to answer this question. We discuss below the possibility of a 2D motion pathway where the left and right monocular motion signals are summed (or averaged), perhaps with keeping the spatial frequency selectivity. 

Differences also exist between the temporal characteristics of 2D and 3D stimulations. The temporal frequency tuning for the perception of depth has been measured with depth modulation in time (Lages, Mamassian, & Graf, 2003; Nagata, 1982; Regan, 1991; Regan & Beverley, 1973; Richards, 1972; Sumnall & Harris, 2002; Tao, Lankheet, van de Grind, & van Wezel, 2003; Tyler, 1971; Tyler & Cavanagh, 1991). The results can be summarized as a temporal frequency tuning with a peak at about 1.5 c/s (Shioiri, Morinaga, et al., 2002). Although these study used stimuli where both velocity and disparity cues were available, recent studies suggest that stimulus only with IOVD cues shows characteristics similar to those of the stimulus with both cues in many cases. Shioiri et al. (2008) reported that the peak temporal frequency was at around 1 c/s for the IOVD, which is similar to 1.5 c/s, and it was at 0.3 c/s or lower for CDOT. Rokers et al. (2008) found that the perception of motion in depth is largely unaffected by binocular anticorrelation in stimulus, which suggest that disparity cues are not critical for perceiving motion in depth in the conditions they used. 

The peak frequencies for motion in depth that have been shown are much lower than the sensitivity peak of 8 Hz of the mechanism sensitive to 2D motion (de Lange, 1958; Kelly, 1979; Merigan & Maunsell, 1993; Pantle, 1974). This indicates that the perception of motion-in-depth is mediated by a mechanism sensitive to relatively slow speeds. This is consistent with the binocular suppression of motion signals (Tyler, 1971). Tyler reported that the threshold for detecting oscillatory stereomotion along the line of sight was higher than the threshold for detecting monocular lateral motion of the same stimulus when the oscillation was higher than 0.5 Hz. Under these conditions, no motion was perceived when the stimulus was viewed binocularly, while the motion became visible upon closing one eye. This suggests that the motion detector for motion-in-depth is less sensitive to high temporal frequencies than the motion detector for lateral motion. 

The temporal properties of the interocular transfer of the MAE also support the difference in temporal properties between the 2D and the 3D motion processes. Tao et al. (2003) reported that the interocular transfer of the MAE is larger for faster motion adaptations. This suggests that IOVDs are not available in fast motion stimulations since a larger amount of the interocular transfer indicates smaller differentiation of the velocity differences between the two eyes. Fast-motion signals might be used to see lateral motion in general. 

The presumption that there are different motion pathways for the 2D and 3D motion processes is also supported by the strong effect of relative motion on motion-in-depth. Although adjacent stimuli usually strengthen the impression of motion including that of MAEs (Ashida & Susami, 1997; Day & Strelow, 1971; Shioiri, Ito, et al., 2002; Shioiri, Ono, & Sato, 2002; Wade, Spillmann, & Swanston, 1996; Walker & Powell, 1974; Watson & Eckert, 1994), the effect disappears when the stimulus is presented dichoptically in the other eye (Symons, Pearson, & Timney, 1996). Interestingly, motion-in-depth is barely perceptible without a relative motion cue as suggested first by Erkelens and Collewijn and investigated in detail by Regan et al. (Erkelens & Collewijn, 1985; Regan et al., 1986; see also Shioiri et al., 2000). Only relative motion signals may contribute to motion-in-depth. If only monocular motion detectors are sensitive to relative motion, the IOVD process perhaps also becomes sensitive to relative motion since the monocular motion signals from the two eyes are required in order to calculate the IOVDs. Relative motion signals are likely to be crucial for the IOVD process. 

The above consideration suggests that the IOVD process compares the slow relative motion signals from the two eyes. This might be related to the assumption that there are different motion processes one of which is sensitive to slow relative motion signals and the other is sensitive to fast uniform motion. Indeed, the motion detector sensitive to slow motion is suggested to be sensitive to relative motion while that sensitive to fast motion is not (Shioiri, Ito, et al., 2002; Shioiri & Matsumiya, 2009). It is likely that only parts of the motion signals on the retinas are used to perceive motion-in-depth on the basis of interocular velocity differences. Note that we do not claim that our results indicate that motion signals from spatial frequency selective mechanisms play no role in the IOVD computation. We claim that only a portion of the monocular motion detectors with sensitivity to low temporal frequencies contribute to the perception of motion in depth on the basis of the IOVD cues. 

Physiological correspondence

As the model depicts in Figure 2a, we presume that interocular velocity differences are calculated at a stage later than the integration of the outputs of the motion SF channels. This is consistent with the physiological fact that the directional selectivity of most of the V1 binocular cells is similar between the two eyes (Maunsell & van Essen, 1983; Ohzawa et al., 1997). No calculation of motion-in-depth with IOVD cues is expected at this level of the motion analysis. A possible physiological site that corresponds to the stage of the IOVD process might be the MT area. In this regard, spatial frequency selectivity of monkey MT cells is broader than that of V1 cells. Another possible physiological site that corresponds to the stage of the IOVD process might be the MST, in which cells have been suggested to analyze global motion signals (Saito et al., 1986). Fernandez and Farell have suggested that the IOVD process is based on a global motion signal, showing the different effects of translation and rotation on the 3D MAE (Fernandez & Farell, 2006). 

A recent fMRI study also suggested that human MT+ encodes the IOVD cues for motion in depth. Rokers, Cormack, and Huk (2009) found that the activation of human MT+ to motion-in-depth was similar in the two conditions: one with binocularly correlated stimuli and the other with binocular anti-correlation stimuli. Since this indicates that human MT+ is sensitive to motion-in-depth in the stimulus without disparity cues, MT+ is suggested to be the site for analyzing IOVD signals for perceiving motion-in-depth. 

There are still other possibilities. An fMRI study suggests that a part of the occipito-temporal regions is related to motion-in-depth perception (Likova & Tyler, 2007). Although their study used CDOT stimuli, this brain area might be responsible for motion-in-depth based on IOVD as well. There might also be areas responsible for IOVD calculation among other areas that have not been fully investigated, including V3A and LO (lateral occipital) which were reported to have some sensitivity to IOVD (Rokers et al., 2009). 

Wherever the site of the IOVD analysis, monocular information is required to be preserved before the IOVD calculation. Although cells in monkey MT or MST are often binocular, there is a report that some cells might preserve monocular motion information (Tailby, Majaj, & Movshon, 2007). This suggests that there is a pathway that conveys monocular motion signals at a later stage of motion analysis, where speed (not temporal frequency) is analyzed. Binocular integration of motion signals should be examined in detail in order to determine the areas in the brain that are responsible for the perception of motion-in-depth on the basis of IOVD cues. Future research with sufficient cue isolation is required for the purpose. The MAE method used in the present study is one of the most promising ways for investigating the IOVD process in isolation. 

Conclusions

We developed an MAE technique that isolates IOVD cues from CDOT cues in order to investigate the spatial property of the IOVD analysis for the perception of motion-in-depth. The observers perceived motion-in-depth on the basis of IOVD signals caused by monocular MAE. The MAE results indicate that the outputs of the monocular motion detectors with different spatial frequency tunings are combined before analyzing the velocity difference between the two retinas in the IOVD process. This suggests that the IOVD is calculated at a relatively later stage of the motion analysis, and that some monocular information is preserved to perceive motion-in-depth even after the integration of the motion SF channel outputs. 

Acknowledgments

This study was supported in part by grants of Hoso Bunka Kikin and of KAKENHI (18330153) to S. Shioiri as well as by the Cooperative Research Project of Research Institute of Electrical Communication (RIEC) at Tohoku University. The authors thank Julian Fernandez for his helpful comments on an earlier version of the manuscript. 

Commercial relationships: none. 

Corresponding author: Satoshi Shioiri. 

Email: [e-mail]. 

Address: Research Institute of Electrical Communication Tohoku University, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan. 

Authors Note

Footnotes

 Part of this study has been reported in abstract form elsewhere (Shioiri et al., 2002; Shioiri, Kakehi, Tashiro, & Yaguchi, 2003).