Aftereffect of motion-in-depth based on binocular cues: Effects of adaptation duration, interocular correlation, and temporal correlation

Abstract

Abstract

Abstract:

Abstract  There are at least two possible binocular cues to motion-in-depth, namely disparity change over time and interocular velocity differences. There has been significant controversy about their relative contributions to the perception of motion-in-depth. In the present study, we used the technique of selective adaptation to address this question. In Experiment 1, we found that adaptation to motion-in-depth depicted by temporally correlated random-dot stereograms, which contained coherent interocular velocity difference, produced motion aftereffect in the depth direction irrespective of the adaptors' interocular correlation for any adaptation duration tested. This suggests that coherent changing disparity does not contribute to motion-in-depth adaptation. Because the aftereffect duration did not saturate in the tested range of adaptation duration, it is unlikely that the lack of the effect of changing disparity was due to a ceiling effect. In Experiment 2, we used a novel adaptor that contained a unidirectional coherent interocular velocity difference signal and a bidirectional changing disparity signal that should not induce a motion aftereffect in depth. Following the adaptation, motion aftereffect in depth occurred in the opposite direction to the adaptor's motion-in-depth based on interocular velocity difference. Experiment 3 demonstrated that these results generalized in 12 untrained subjects. These experiments suggest that the contribution of interocular velocity difference to the perception of motion-in-depth is substantial, while that of changing disparity is very limited (if any), at least at the stages of direction-selective mechanisms subject to an aftereffect phenomenon.

Introduction

Detection of motion-in-depth is arguably one of our most important perceptual tasks, supporting the identification of approaching threats and obstacles. Important monocular cues such as looming exist and are perceptually salient, support avoidance behavior in humans and animals (Gabbiani, Krapp, Koch, & Laurent, 2002), and have specialized neural substrates. Motion-in-depth is also signaled by binocular information (Cumming & Parker, 1994; Regan, 1993). Compared to monocular cues to motion-in-depth, the utility, nature, and physiological substrate of the binocular information is less well understood. At least two possible sources of binocular information to motion-in-depth have been identified. First, since stereoscopic depth is signaled by binocular disparity, change in disparity (CD) over time can indicate motion-in-depth. Second, an object moving in depth moves in the opposite directions on the two retinae if the eyes are still. If the eyes track the moving object, then the eyes move in opposite directions (converging for approaching objects and diverging for receding objects). This dichoptic motion cue—particularly the case of differential retinal motion—is known as the interocular velocity difference (IOVD) cue. 

Investigators have attempted to isolate these mechanisms. A dynamic random-dot stereogram (DRDS) contains no coherent motion signal in each half image, and hence removes coherent IOVD, isolating the CD cue. Julesz (1971) showed that motion-in-depth could be perceived in such a stimulus. Thresholds for motion-in-depth in a DRDS are similar to those in a random-dot stereogram (RDS), which contains both CD and IOVD cues (Cumming & Parker, 1994; Regan, 1993); this suggests that the CD cues are sufficient for motion-in-depth perception (but see Brooks & Stone, 2006). The DRDS relies on temporally uncorrelated stimuli in the left and right eyes to eliminate coherent motion in each eye's image. To isolate the IOVD cue, interocularly uncorrelated signals have been used. Uncorrelated random-dot stereograms (URDS), composed of independent persistent samples of dots in each eye, have been used to provide stimuli that contain no coherent instantaneous disparity. Observers report compelling percepts of motion-in-depth when these stimuli move differentially in the two eyes (Allison, Howard, & Howard, 1998; Shioiri, Saisho, & Yaguchi, 2000), suggesting that IOVD is an effective cue for motion-in-depth. 

Although it has been reported that each cue contributes to the perception of motion-in-depth, there has been significant controversy about relative contributions of these cues. Some studies have claimed that the contribution of CD is dominant and that of IOVD is very weak or nonexistent (e.g., Cumming & Parker, 1994; Nefs & Harris, 2010; Nefs, O'Hare, & Harris, 2010; Regan, 1993), while others have reported the opposite results (e.g., Brooks, 2002; Czuba, Rokers, Guillet, Huk, & Cormack, 2011; Sakano, Allison, & Howard, 2012; Sakano, Allison, Howard, & Sadr, 2006; Wardle & Alais, 2013). One reason why past studies seem to be at odds may be that the mechanisms processing the two cues have different temporal frequency tuning (Shioiri, Nakajima, Kakehi, & Yaguchi, 2008) and the studies used stimuli with different temporal characteristics. Another reason, not necessarily excluding the first one, might be that multiple stages of processing or multiple mechanisms with different characteristics could be involved in the perception of motion-in-depth, even for one cue, as is the case with the mechanisms for the perception of lateral motion (Culham, Verstraten, Ashida, & Cavanagh, 2000; Nishida & Ashida, 2000). In the latter case, if the researchers used different methods and/or stimuli, the results could differ depending on the mechanism stimulated. In the present study, we focused on possible mechanisms specialized to detect motion-in-depth based on binocular cues. Specialized mechanisms imply that motion-in-depth is processed by dedicated low-level detector networks rather than by a succession of static depth percepts derived from binocular disparity (i.e., tracking of position in depth; Patterson, 1999). In other words, they are mechanisms selective for the direction of motion-in-depth (i.e., tuned to motion-in-depth; Czuba et al., 2011). 

To evaluate the contribution of CD mechanisms specialized to see motion-in-depth, it is necessary to discount possible effects of tracking of position in depth signaled by binocular disparity. To avoid such a possibility, the use of the technique of motion aftereffect (MAE) is effective (Sakano et al., 2012). If adaptation to motion-in-depth based solely on a particular binocular cue induces a directional (negative) MAE in depth, the contribution of a mechanism specialized to detect motion-in-depth based on that binocular cue is indicated. When stimulating an IOVD mechanism, contamination with positional tracking is unlikely because IOVD does not specify a specific depth. Nevertheless, the MAE technique is still effective (Sakano et al., 2012). 

In terms of the contribution of possible CD mechanisms specialized to detect motion-in-depth, previous studies have consistently found that the contribution, if any, is very weak. However, in regard to whether the contribution or the specialized mechanism itself exists, there is no consensus, and the interpretations of the results of previous studies are debatable. 

Brooks (2002) found that velocity aftereffect to approaching motion was significantly larger following RDS adaptation than URDS adaptation when the RDS adaptor had uncrossed disparity through the entire adaptation period but not when the RDS adaptor passed through a surrounding, similarly textured plane. The author interpreted this as evidence of CD adaptation, but the RDS stimulus had a well-defined disparity pedestal, while the mean disparity of the URDS stimulus was undefined and effectively zero. As a result, the difference could be due to more effective adaptation of stimuli that were separated from the fixation plane rather than to the presence of a CD signal. This could be because stimuli on a pedestal had increased visibility (Portfors & Regan, 1997), because there were persistent monocular occlusion regions, or simply because depth of adaptation varies with apparent depth or apparent speed (stimuli were presented with an uncrossed disparity pedestal, which geometrically predicts a higher speed than for stimuli at the fixation plane). 

Czuba et al. (2011) and Czuba, Rokers, Huk, and Cormack (2012) found a small shift in motion direction bias after CD adaptation. Based on these results, they claimed the contribution of CD mechanisms. However, there is a possibility that the aftereffect was caused by vergence eye movements, not by CD adaptation, because in both studies (the “unidirectional” condition for Czuba et al., 2012), the whole adaptation stimulus moved in the same direction in depth. We have found that observers cannot maintain fixed vergence with such stimuli. Opposite MAE in the two parts of a bipartite test stimulus following adaptation to opposite directions of motion-in-depth of the adaptor's top and bottom hemifields cannot be explained by vergence eye movements. In such stimuli, we have not found MAE in depth to CD stimuli (Sakano et al., 2012)—and neither did Czuba et al. (2012) in their equivalent “bidirectional” condition. In addition, Czuba et al. (2011) found that the bias in motion coherence judgments following adaptation to IOVD alone and IOVD plus CD stimuli were similar (95% confidence intervals substantially overlapped). 

Sakano et al. (2012) found that an MAE in depth did not occur after adaptation to pure CD stimuli when the effects of vergence eye movements and depth adaptation were controlled for. Moreover, they found no difference in the duration and frequency of MAE in depth following URDS (IOVD) compared to RDS (IOVD + CD) stimuli. Therefore they did not find any evidence for the existence of CD mechanisms specialized to detect motion-in-depth. However, there is a possibility that in their CD adaptation conditions, IOVD signals in random directions disturbed the potential weak coherent CD signal, resulting in no detectable aftereffect. Similarly, the equivalence of the aftereffects found in the URDS and RDS conditions could be due to saturation of the aftereffects. That is, there was no difference in the aftereffect because both stimuli produced the strongest aftereffect that could be elicited and hence there was no room for an effect of CD to be shown in terms of an aftereffect potentiation for the RDS compared to URDS stimuli. Such saturation could also explain why there was no difference between aftereffects elicited by IOVD alone and IOVD plus CD stimuli by Czuba et al. (2011). Both Czuba et al. (2011) and Sakano et al. (2012) used long-duration adaptation intended to produce robust aftereffects, making saturation more likely. In fact, it has been reported that aftereffects of other visual attributes (contour orientation: Suzuki, 2001; velocity of lateral motion: Hammett, Thompson, & Bedingham, 2000) saturate in shorter duration of adaptation (about 27 ms and 16–32 s, respectively) than the 120 s used by Sakano et al. (2012) or the 100 s used by Czuba et al. (2011). One possible mechanism of MAE in depth that could show such saturation would be the disruption of balanced responses of multiple stereomotion velocity-tuned channels analogous to those proposed for lateral MAE (Mather & Harris, 1998). 

In the first experiment of the present study, we therefore examined whether the similar aftereffects following RDS and URDS adaptation were due to saturation of the aftereffect. For that purpose, we varied adaptation duration and studied its effects on aftereffect duration. For the range of adaptation exposures where aftereffect duration increases with increasing adaptation exposure, the response is obviously not saturated, and we would expect to observe an increase in aftereffect strength with increase in adaptation stimulation. Hence, if CD cues contribute to adaptation in the RDS stimuli, the aftereffect should last longer than in the URDS case. Importantly, this comparison of the RDS and URDS adaptation conditions enabled us to avoid the latent disturbance of the potential weak coherent CD signal by random IOVD signals, which exist in DRDS. 

With respect to the contribution of IOVD mechanisms specialized to detect motion-in-depth, previous studies using adaptation methods have consistently suggested its existence (Brooks, 2002; Czuba et al., 2011, 2012; Sakano et al., 2012). For instance, Sakano et al. (2012) found that a motion aftereffect in depth occurred after adaptation to the URDS and RDS stimuli, while no aftereffect occurred following adaptation to the DRDS stimuli. Interestingly, a monocular test pattern also showed a comparable motion aftereffect in a diagonal direction in depth after adaptation to the temporally correlated stereograms (i.e., URDS and RDS). This pattern of results was consistent with the existence of a mechanism sensitive to interocular velocity differences, which is adaptable at binocular stages of motion-in-depth processing (for evidence that at least in part, monocularly adaptable mechanisms may also feed into motion-in-depth perception, see also Fernandez & Farell, 2005, 2006; Shioiri, Kakehi, Tashiro, & Yaguchi, 2009). 

On the other hand, Nefs and Harris (2010) evaluated the relative contributions of CD and IOVD by measuring sensitivity to the direction of motion-in-depth in a two-frame sequence (rather than using an adaptation method) and claimed that CD contributes substantially, while the IOVD cue is very weak or ineffective. 

In a second experiment of the present study, we used a novel, minimal two-frame adaptation sequence to test for the existence of an adaptable IOVD mechanism in the presence of balanced, rather than degraded and incoherent, changing disparity. During the adaptation phase, the two cues were coherent and consistent and the changing disparity was always well defined. Thus, the role of IOVD mechanisms could be evaluated in the context of CD without the need to degrade the disparity information in the stimuli. Previous studies have typically used URDS to test the effect of IOVD. However, there is a possibility that URDS contains latent CD signals caused by unintended random matching of the interocularly uncorrelated dots. By using the present novel stimulus, we were able to avoid such a possibility. If the adaptation to repeated two-frame motion-in-depth induces MAE in depth, it would be suggested that the IOVD mechanisms contribute significantly to the perception of motion-in-depth even in a two-frame motion sequence. 

Experiments 1 and 2 consisted of many measurements on a small number of subjects. In a third experiment, we tested key conditions from Experiments 1 and 2 with a substantial number of untrained subjects to confirm the generality of the results in light of a previous report of intersubject variability in the use of CD and IOVD cues (Nefs et al., 2010). 

Experiment 1: Effects of adaptation duration

Previous demonstrations of stereoscopic motion-in-depth perception have found no difference between adaptation to IOVD and adaptation to IOVD combined with CD cues (Sakano et al., 2012). In this experiment we ask whether the equivalent duration of MAE in depth under the interocularly correlated and uncorrelated adaptation conditions of Sakano et al. (2012) reflects saturation of adaptation to motion-in-depth. 

Methods

Subjects

Two authors (including subject 1) and four naïve volunteers (including subject 2) took part in Experiment 1. All subjects were previously screened for normal or corrected-to-normal acuity and normal binocular vision. All had experienced other psychophysical experiments on stereopsis and/or stereoscopic motion-in-depth. 

Apparatus

All the experiments were conducted in a darkened room. The stimuli were generated using Vision Egg software (Straw, 2008). They were presented on a 20-in. monochrome cathode-ray tube (CRT) monitor (Monoray Model M20ECD5RE, Clinton Electronics) refreshed at 120 Hz and viewed dichoptically from a distance of 120 cm through ferroelectric shutter goggles in order to produce separate left and right eye images (FE-1 Goggles, Cambridge Research Systems). The cross talk (i.e., the luminance of “ghost” images of bright objects seen by the fellow eye expressed as a percentage of the luminance of the object as seen by the intended eye) was less than 0.2%. The cross talk was determined by measuring luminance of a white square presented only for one eye through the lens of the goggles for the intended and fellow eyes. A chin rest was used to stabilize head position. 

Stimuli and procedure

Subjects adapted to a motion-in-depth stimulus for 7.5, 15, 30, 60, 120, or 240 s. The motion-in-depth was simulated by CD and/or IOVD. 

The adaptation stimulus consisted of two frontoparallel planes (each subtending 17.4° × 2.91°) depicted by dichoptic random dots, one above and one below a fixation point (Figure 1). Bright features in the stimulus were presented at 10.0 cd/m² on a dark background (<0.01 cd/m²) when viewed through shutter goggles. There were 400 dots in each plane, and each dot subtended 13.1 × 13.1 arcmin (12 × 12 pixels). The planes were placed symmetrically above and below the fixation point with a gap of 1.74° between the planes (inner edges of the planes were 0.87° above and below fixation). Subjects fixated a bright square subtending 4.36 arcmin placed in the center of the screen. An arrangement of vertical and horizontal lines beside the fixation point subtended 6.54 × 52.3 and 17.4 × 4.36 arcmin, respectively, and helped to stabilize fixation. Fixation was monitored by vertical nonius lines which subtended 4.36 × 17.4 arcmin placed above and below the fixation point, separated with a gap of 4.36 arcmin. Figure 1 shows a representation of one half image of the display. 

Figure 1

 

Stimulus for both inducing and probing motion-in-depth artifacts. The figure shows a half image of the stimulus. Subjects fixated the center of the stimulus with a fixation point and dichoptic zero-disparity lines to help stabilize fixation. Subjects monitored the stability of binocular fixation with a nonius line pair, one line in each half image (the upper line in the case of the illustration). The upper and lower dot fields moved in opposite directions to each other and in opposing directions in the two eyes. The dot patterns were either interocularly correlated (RDS) or uncorrelated (URDS).

In each monocular half image, the dots were randomly distributed throughout the planes. However, this distribution was either (a) binocularly correlated or identical in the two eyes (except for the lateral shift due to disparity) or (b) binocularly uncorrelated or independently distributed in each image. Thus, the motion-in-depth was specified by (a) both IOVD and CD, by using random dots that were binocularly and temporally correlated (RDS), or (b) IOVD alone, by using binocularly uncorrelated but temporally correlated dots (URDS). 

During adaptation, the bottom plane moved toward the observer while the top plane moved away, providing unidirectional motion-in-depth adaptation signals in the two parts of the display (Figure 2). Following each disparity ramp, the stimulus was reset to its initial position and the ramp repeated (sawtooth disparity profile). We have previously found that adaptation effects were similar for the complementary condition with the bottom plane receding and the top plane approaching (Sakano et al., 2012). The rate of CD or IOVD was 2.18°/s, simulating a mean motion-in-depth of 84.1 cm/s. The top plane traversed a disparity interval of −17.4 arcmin of crossed disparity to +17.4 arcmin of uncrossed disparity with respect to fixation on the screen, while the bottom moved over the same range in the opposite direction. This simulated a motion-in-depth of 22.4 cm for each plane. Each constant CD/IOVD interval (disparity ramp) lasted 283 ms from the beginning to end of the disparity interval. At the ends of the range, the stimulus disappeared for 16.7 ms. When the stimulus reappeared, the disparity was reset to the starting point of the disparity ramp and the ramp was repeated. This sequence was repeated until the adaptation interval was finished. 

Figure 2

 

In Experiment 1, observers viewed an adaptation stimulus moving in depth depicted by either interocularly correlated (RDS) or uncorrelated (URDS) dots. Adaptation time was varied trial to trial from 7.5 to 240 s. Following adaptation, observers viewed a stationary test stimulus consisting of a zero-disparity RDS and indicated with a key press when the MAE ceased.

Following the given adaptation interval, the moving stimulus disappeared and the test stimulus was presented. The test stimulus was a static RDS similar to the RDS adaptor but presented with constant zero disparity. However, when a negative motion aftereffect was present, the subject perceived the test stimulus planes moving in the opposite direction to the adaptation stimulus. The task of the subject was to press a key as soon as the sensation of motion (the motion aftereffect) ceased. The subject was also asked to confirm the direction and occurrence of a motion aftereffect after each trial. The subjects had experience in experiments on stereoscopic depth and/or stereoscopic motion-in-depth and thus knew the difference between motion-in-depth and lateral motion. Therefore, it is likely that there was no confusion between 3-D and 2-D MAE and the criterion was constant across trials. To avoid such contaminations, we also explicitly instructed the subjects to respond purely based on sensation of motion-in-depth, not lateral motion (no subjects reported experiencing lateral MAE). Following the test stimulus, a rest interval (intertrial interval) with a duration at least 1.5 times the preceding adaptation interval was imposed before the next trial started. During the rest interval, the dot patterns were not displayed but the fixation point, zero-disparity lines, and nonius lines were presented. 

All 12 combinations of the two adaptation stimuli (URDS versus RDS) and the six adaptation durations were presented once in each session in randomized order. Subjects participated in eight sessions, resulting in eight replications of each condition per subject. 

Results

All subjects reported a negative motion aftereffect in depth following both RDS and URDS adaptation on every trial, except for several trials where no aftereffect was reported. Figure 3 shows the duration of the motion-in-depth aftereffect as a function of adaptation time for different adaptation stimuli. Each plot shows individual mean adaptation duration (± standard error of the mean) for two typical observers along with the average across all six observers. Data from Sakano et al. (2012) on aftereffect durations following equivalent DRDS adaptation for two observers is also shown in the average graph. In that earlier experiment, an aftereffect could not be elicited from DRDS adaptation stimuli even after 10 min of adaptation. 

Figure 3

 

Motion-in-depth aftereffect duration as a function of adaptation time for URDS and RDS adaptation stimuli. Each plot shows individual adaptation duration (± standard error of the mean) averaged across all the trials for two typical observers (the top graphs). In trials in which the aftereffect did not occur, the duration was dealt with as 0 s. The aftereffect durations averaged across all six subjects are also shown (the bottom graphs). In these two graphs, in each subject, the aftereffect duration was averaged across all trials dealing with no-aftereffect trials as 0-s duration (the lower left graph) or was averaged across only trials in which the aftereffect occurred (the lower right graph). In the lower left graph, data from Sakano et al. (2012) for aftereffect durations following equivalent DRDS adaptation are also shown.

For both RDS and URDS adaptation, the aftereffect increased approximately proportionally as the adaptation time increased. There was no indication of saturation of the adaptation for subject 1 or subject 2 or in the average response. There was little difference in the aftereffect duration following URDS and RDS adaptation. 

Linear statistical modeling of the effects of adaptation type and adaptation duration on aftereffect duration was performed after log transformations of both the adaptation and aftereffect duration, to improve homogeneity of variance (Bowd, Rose, Phinney, & Patterson, 1996; Greenlee, Georgeson, Magnussen, & Harris, 1991). Two-way repeated-measures ANOVA demonstrated a significant effect of adaptation duration (p < 0.0001), but not of adaptation type (p = 0.810) or the interaction of adaptation type and duration (p = 0.994), on the aftereffect duration. The regression slopes were used to compare the effects of the adaptation type. For data pooled across the subjects, the regression coefficients of the RDS and URDS adaptation were significantly larger than zero (p = 0.0001 and 0.0002, respectively), confirming a significant increase in aftereffect duration with increase in adaptation period. There was no significant effect of adaptation type (RDS versus URDS) on the slope of the regression between adaptation duration and aftereffect duration (p = 0.567). These patterns were confirmed by similar analyses on individual data. The regression coefficients of the RDS and URDS adaptation were significantly larger than zero for every subject (ps between 0.0001 and 0.0156) except for the URDS adaptation in one subject, which was positive but not significant (p = 0.093). The effect of adaptation time (adaptation duration coefficient) was not significantly different for RDS compared to URDS adaptation for any subject (ps between 0.086 and 0.944). 

Post hoc testing with control for multiple comparisons using Tukey's HSD test confirmed that the aftereffect duration following URDS adaptation was not significantly smaller than following RDS adaptation for any adaptation interval. However, this does not appear to be due to aftereffect saturation, since aftereffects after 120 or 240 s were significantly larger than aftereffects after 30, 15, or 7.5 s. 

Experiment 2: Adaptation to repeated two-frame motion

Previous studies using adaptation methods have consistently suggested the significant contribution of IOVD to the perception of motion-in-depth (Brooks, 2002; Czuba et al., 2011, 2012; Sakano et al., 2012). However, Nefs and Harris (2010) reported very weak or no contribution of IOVD in experiments measuring sensitivity to the direction of motion-in-depth in a two-frame sequence (rather than an adaptation method). In Experiment 2, we used a novel, two-frame adaptation sequence to test for the contribution of an adaptable IOVD mechanism in the presence of balanced, rather than degraded and incoherent, changing disparity. During the adaptation phase, the two cues were coherent and consistent and the changing disparity was always well defined. Thus, the role of IOVD mechanisms could be evaluated in the context of CD without the need to degrade the disparity information in the stimuli. If the adaptation to repeated two-frame motion-in-depth induces MAE in depth, it would suggest that the adaptable IOVD mechanisms contribute significantly to the perception of motion-in-depth even in a two-frame motion sequence. 

Methods

The apparatus, stimuli, and procedure were similar to those in Experiment 1 with the following exceptions: The stimulus refresh rate was 120 Hz (60 Hz per eye), but the display was only updated every four refresh cycles, for a frame rate of 15 Hz. The adaptation stimulus was an RDS in which the absolute disparity alternated every frame between two values while the dot distribution changed randomly every second frame (Figure 4). Thus, instead of continuous approaching or receding motion, according to disparity the stimulus alternated back and forth every 67 ms (i.e., on each frame at 15 Hz), and approaching and receding disparity changes were balanced. Thus if changing disparity mechanisms adapted, they would be adapted to alternating motion, and no motion aftereffect in depth would be expected. The top field of dots alternated between zero and 4.36 arcmin crossed disparity, while the bottom field of dots alternated between zero and 4.36 arcmin uncrossed disparity. The two halves of the display changed disparity in opposite directions so that the top receded when the bottom approached and vice versa. As a consequence, both halves of the display were at zero disparity at the same frame and also disparate with respect to fixation at the same time. Note that unlike in Experiment 1, there was no blank period when the stimulus was reset in depth. 

Figure 4

 

Experiment 2 studied adaptation to an interocular velocity difference signal and a balanced changing disparity signal. Observers viewed an adaptation stimulus moving in depth as depicted by an interocularly correlated random-dot pattern that was refreshed every second frame. The stimulus alternated between having depth (the top panel on the left-hand side of the figure) to having zero depth/disparity (bottom panel on the left-hand side of the figure). The dots persisted across the transition from depth to no depth, but the sample of dots was changed (i.e., a new sample of dot positions was randomly chosen) when the stimulus transitioned from flat to having depth. Following adaptation, the observers viewed a stationary test stimulus consisting of a zero-disparity RDS (the right hand side of the figure) and indicated with a key press when the MAE ceased.

As well as alternating back and forth in disparity-defined depth, a new sample of locations for the random dots comprising the stimulus was chosen on every second frame (every 133 ms). In other words, the same dot pattern was shown on two consecutive frames, then a new sample of dots was generated for the stimulus for the next frame, which in turn persisted for two frames before being changed; and this pattern was repeated. When the dot pattern was the same across two frames, the displacement of the dot patterns in each eye produced a monocular apparent motion signal. Thus the change in disparity between two frames with persistent texture contained both changing disparity and interocular motion signals. In contrast, when the texture changed across frames, there was still a coherent change in disparity but no coherent motion signal in each half image. Across any two of the frames with texture change, the direction of motion-in-depth specified by the IOVD cue could be slightly biased to either approaching or receding motion depending on the dot distributions, but would be zero (i.e., no motion-in-depth) when averaged through the adaptation period. The direction of the coherent motion signal was determined by the relative phase between the disparity alterations and the texture changes. 

Thus, the stimulus provided a coherent approaching or receding interocular velocity difference signal for adaptation and a balanced changing disparity signal that averaged to zero over time. In this experiment, we changed the texture on the frames where the stimulus had zero disparity. Thus the top field of dots contained a receding motion signal and the bottom field of dots contained an approaching motion signal. 

The test stimulus was a stationary RDS with zero disparity equivalent to a frame of the adaptation stimulus (with a different random assignment of dot locations) depicting a frontoparallel plane. As in Experiment 1, subjects used a key press to indicate that the impression of motion-in-depth had ceased and confirmed the presence and direction of the MAE in depth. 

Two of the subjects from Experiment 1 (one author and one naïve volunteer, but they were not subject 1 or 2 of Experiment 1) participated in Experiment 2. Each condition was repeated eight times per subject. A rest period of at least 30 s was required between trials in order to minimize carryover of adaptation effects. 

Results

Both subjects reported a negative motion aftereffect in depth following adaptation on every trial. Thus, following receding motion signals but balanced changing disparity signals in the top half of the display, they perceived approaching motion in a stationary stimulus placed in the adaptation location (and conversely, after approaching motion in the bottom half of the display they reported receding motion in a stationary test stimulus). Figure 5 shows the frequency of occurrence and duration of the motion-in-depth aftereffect. The aftereffect was present on all trials for each subject and lasted several seconds. 

Figure 5

 

Motion-in-depth aftereffect frequency and duration following adaptation to an interocular velocity difference signal and a balanced changing disparity signal.

Experiment 3: Generality of the results

Recently, Nefs et al. (2010) reported individual differences in relative prevalence of the CD and IOVD mechanisms within a normal population. Moreover, they reported that in the majority of the population, the CD mechanism is more sensitive to motion-in-depth than the IOVD mechanism is. The results of Experiments 1 and 2 suggest that the MAE in depth was caused (at least mainly) by adaptation to IOVD but not to CD in our sample of subjects with previous experience in stereoscopic psychophysical experiments. Given the reported variation in sensitivity to IOVD, it is important to determine whether the MAEs in depth reported in Experiments 1 and 2 are representative of a normal population of untrained observers. To address this question, we repeated a subset of Experiments 1 and 2 with 12 naïve and untrained subjects in a short single-session experiment. 

Methods

Twelve naïve volunteers (two women and 10 men, 29–48 years of age, mean 39.4) participated in this experiment. All had normal or corrected-to-normal visual acuity and normal stereo acuity of at least 40 s of arc (Stereo Fly Test, Stereo Optical Company, Inc., Chicago, IL). None of them had previous experience in psychophysical experiments that presented stereoscopic motion-in-depth (including Experiments 1 and 2 of the present study). 

Methodological details were similar to those in Experiments 1 and 2, with the following exceptions: The stimuli were presented on a 19-in. CRT monitor (39.5 × 29.6 cm, Mitsubishi Diamondtron M² RDF223G) refreshed at 120 Hz and viewed through stereo shutter goggles (Crystal Eyes 3, StereoGraphics Co.). To minimize cross talk between the images presented to the two eyes, we used only red phosphor on the monitor, which was comparatively faster (Sakano & Ando, 2010). The measured cross talk was less than 0.6%. Bright features in the stimulus were presented at 1.99 cd/m² on a dark background (<0.001 cd/m²) when viewed through shutter goggles. The viewing distance was 50 cm. We ran the same experimental program as that used in Experiments 1 and 2 so that the time courses of each trial remained the same. The stimulus configuration also remained the same, but the angular size of the whole stimulus was 2.33 times larger than that of Experiments 1 and 2. The resulting dimensions of each of two frontal planes (see Figure 1) and each dot were 40.6° × 7.06° and 31.8 × 31.8 arcmin, respectively. Preliminary observations suggested that increasing the angular size of the stimulus produced a slightly clearer impression of MAE in depth in this apparatus. 

Each subject participated in a single session that consisted of: 

•  
  one trial of RDS and one trial of URDS 2-min adaptation stimuli from Experiment 1. The rate of CD or IOVD was 5.30°/s, simulating a mean motion-in-depth of 35.6 cm/s. The top plane traversed a disparity interval of −42.4 arcmin of crossed disparity to +42.4 arcmin of uncrossed disparity with respect to fixation on the screen, while the bottom moved over the same range in the opposite direction. This simulated a motion-in-depth of 9.5 cm for each plane. The adaptation lasted 2 min.
•  
  one trial of the two-frame stimulus from Experiment 2. The top field of dots alternated between zero and 10.6 arcmin crossed disparity, while the bottom field of dots alternated between zero and 10.6 arcmin uncrossed disparity.

The order of these three conditions (i.e., RDS, URDS, and two-frame) was randomized for each subject. A rest period of at least 30 s was imposed between trials in order to minimize carryover of adaptation effects. 

Results

In both the RDS and URDS adaptation conditions, 10 of 12 subjects reported a negative motion aftereffect in depth following adaptation (Figure 6). The other two subjects did not report the aftereffect in either the RDS or the URDS condition. After the two-frame adaptation, seven of 12 subjects reported the aftereffect. The two subjects who did not report the aftereffect in the RDS and URDS conditions did not report the aftereffect in the two-frame condition either. In terms of the aftereffect duration, we performed one-way, within-subjects ANOVA followed by Tukey's HSD post hoc test. ANOVA showed no significant effect of adaptation condition (p = 0.0583) when the data for two subjects who did not experience the aftereffect were excluded, while that effect was significant (p = 0.0032) when those data were included with a 0-s aftereffect duration. Importantly, post hoc testing with control for multiple comparisons using Tukey's HSD test showed that the aftereffect duration was comparable in the RDS and URDS conditions in both cases (p > 0.05). The average aftereffect duration in the two-frame adaptation condition was significantly shorter than those in the RDS and URDS conditions if reports of no aftereffect were treated as 0-s duration (p < 0.05). When trials in which an aftereffect was not reported were excluded rather than counted as zero duration, the average aftereffect duration in the two-frame adaptation condition was still shorter than those in the RDS and URDS conditions, but the difference was not significant. 

Figure 6

 

Results of Experiment 3. The number of subjects in whom motion-in-depth aftereffect occurred (the top graph), the duration of the aftereffect averaged across the subjects in whom the aftereffect occurred (the middle graph), and the average duration across all subjects treating no-aftereffect as 0-s duration (the bottom graph). The dashed line in the top graph indicates the number of subjects (12 subjects). The error bars indicate ±1 standard error of the mean.

Discussion

These experiments provide evidence for a significant contribution of an adaptable mechanism based on dichoptic motion for the perception of motion-in-depth even in a two-frame motion sequence. In addition, these results provide evidence against the existence of a CD mechanism specialized to see motion-in-depth. We will discuss these mechanisms in detail. 

CD mechanisms specialized to detect motion-in-depth

In the first experiment, we tested whether the equivalence of motion aftereffects after adaptation to RDS (CD + IOVD) and URDS (IOVD) stimuli found in our previous study (Sakano et al., 2012) could be explained by saturation of the motion aftereffect. We found that aftereffect duration increased with exposure duration, with little evidence of saturation over the range tested. Furthermore, RDS and URDS stimuli produced equivalent aftereffect durations at all adaptation durations. Thus, we conclude that URDS stimuli with only coherent IOVD produce as strong an aftereffect as RDS stimuli with coherent CD and IOVD, and that this is not a result of ceiling effects. 

Also, in the third experiment we confirmed the findings of the first experiment with 12 naïve and untrained subjects for the two-minute adaptation conditions only. We found that MAE in depth occurred in the majority of the subjects (10 of 12 subjects) both in the RDS and URDS adaptation conditions. Importantly, all 10 subjects who reported the aftereffect in the RDS (CD + IOVD) condition also reported the aftereffect in the URDS (IOVD) condition. Moreover, as in Experiment 1, the aftereffect duration was comparable in the RDS and URDS conditions. These results suggest that adaptation to IOVD, but not to CD, is responsible for MAE in depth even in untrained observers. 

As described in the Introduction, interpretations of previous adaptation studies supporting the existence of a CD mechanism specialized to detect motion-in-depth are debatable. Brooks (2002) found that the velocity aftereffect to approaching motion was significantly larger following RDS adaptation than URDS adaptation when the RDS adaptor had uncrossed disparity through the entire adaptation period but not when the RDS adaptor passed through a surrounding, similarly textured plane. The author interpreted this as evidence of CD adaptation, but the RDS stimulus had a well-defined disparity pedestal, while the mean disparity of the URDS stimulus was undefined and effectively zero. As a result, the difference could be due to more effective adaptation of stimuli that were separated from the fixation plane rather than to the presence of a CD signal. This could be because stimuli on a pedestal had increased visibility (Portfors & Regan, 1997), because there were persistent monocular occlusion regions, or simply because depth of adaptation varies with apparent depth or apparent speed (stimuli were presented with an uncrossed disparity pedestal, which geometrically predicts a higher speed than for stimuli at the fixation plane). Regan, Portfors, and Hong (1997) reported that adaptation to DRDS simulating back-and-forth motion-in-depth decreases the sensitivity to such motion (see also Regan et al., 1998). However, this aftereffect could have been caused by adaptation of static disparity detectors tuned to the adaptor's range of disparity, because they measured depth threshold in terms of peak-to-peak disparity. Similarly, Czuba et al. (2011, 2012) found an MAE in depth following adaptation to CD in DRDS. However, we have found no reliable aftereffect after adaptation to DRDS when we have controlled for the effects of vergence eye movements and depth adaptation (Sakano et al., 2012). In Experiments 1 and 3 of the present study, we found no significant difference in aftereffect between the RDS and URDS adaptation conditions accounting for possible ceiling effects. 

Taken together, we find no persuasive evidence for the contribution of adaptable CD mechanisms specialized to detect motion-in-depth or for the existence of the mechanism itself. 

Seeing two-frame motion-in-depth based on IOVD mechanisms

In the second experiment we used a novel stimulus that contained both coherent CD and IOVD cues. In this stimulus, CD was coherent and well defined, unlike in URDS, but unlikely to result in a negative aftereffect. Oscillating motion-in-depth has been shown to reduce sensitivity to oscillations of a similar frequency (Regan & Beverley, 1973), but the balanced nature of the stimulus should have not produced any motion aftereffect due to CD. The adaptation stimulus used in the second experiment of the present study did provide coherent two-frame apparent motion-in-depth, and this seemed sufficient to produce a robust motion aftereffect in depth. Thus, stereomotion mechanisms based on the temporal correlation of the monocular inputs can adapt even in the presence of coherent CD signals. These results suggest that the IOVD mechanisms contribute to the perception of motion-in-depth even if the motion is composed of only two frames. 

Also, in the third experiment we confirmed these findings with 12 naïve and untrained subjects. We found that an MAE in depth occurred in the majority of the subjects (seven of 12 subjects), although the proportion was lower than those in the RDS and URDS conditions. These results suggest that even in untrained observers, the IOVD mechanisms contribute to the perception of motion-in-depth even if the motion is composed of only two frames. 

To our knowledge, the only other study to use two-frame motion to examine binocular cues to motion-in-depth is by Nefs and Harris (2010), who reported results seemingly at odds with those found in Experiments 2 and 3 of the present study. Their results suggested that IOVD cues were very weak or ineffective in two-frame stimuli (presented once, not alternately as in our experiment), whereas we found substantial MAE to two-frame apparent motion-in-depth. In their main experiment, Nefs and Harris presented two-frame sequences where an RDS or DRDS stimulus jumped from either a large crossed disparity on the first frame to a large uncrossed disparity on the second or vice versa. For the URDS version, the dot displacement in each half image was equivalent to half the disparity in the binocularly correlated sequences. Such a large change in depth was easily discernable when the stimulus had coherent disparity (RDS or DRDS) and was visible for a long time before and after the step in depth (the authors referred to this as the “stimulus frame time”). Thus, to measure stimulus strength, they varied the stimulus frame time and the coherence to vary the strength of the stimulus. The coherence was the percentage of dots that were signal dots, with the remainder noise dots that were randomly placed independently on the first and second frames. The main measure was the coherence threshold for discriminating the direction of motion at each frame time for each stimulus type. Nefs and Harris found that only one observer could judge the direction of motion-in-depth in the URDS case and only at near 100% coherence. In contrast, with DRDS and RDS stimuli subjects could judge the direction of the depth displacement, particularly when the frames were presented for 250 ms or more, and with increasing tolerance to lack of coherence as frame time increased. 

The use of a forced-choice discrimination task and the dependence on frame rate, however, strongly suggest that subjects were judging either the initial depth of the stimulus (which could be used to perform the task in the RDS and DRDS cases but not the URDS case) or the difference in static depth at the start and beginning of the trial. Another consideration suggesting this interpretation is the fairly large disparity step (19.8 arcmin), which might have exceeded the range of local motion processing mechanisms (Boulton & Baker, 1993). A second experiment presenting the stimuli on a depth pedestal suggested that subjects were in part using the depth sign of the initial frame to perform the task. In the second experiment, the observers could undertake a similar initial position or displacement strategy, although they would need to judge depth magnitude as well as sign. Nefs et al. (2010) also used the similar stimuli and tasks, although the motion was continuous rather than two-frame, and thus the subjects could have judged the motion direction based on the depth sign of the first half of motion-in-depth in the RDS and DRDS conditions, but not in the URDS condition, which the authors referred to as the “TCRDS condition.” 

Thus, in the experiments of Nefs and Harris (2010) and Nefs et al. (2010), observers were probably using depth or depth displacement to perform the threshold task. In our Experiments 2 and 3, the adaptor was of alternation in depth and we were measuring suprathreshold aftereffects. A clear and strong motion aftereffect was produced by the modest-amplitude, two-frame apparent motion sequence. The disparity signal was always clearly present in the stimulus, and arguably the CD signal was stronger than the IOVD signal, since it was present on every frame change, whereas IOVD was only coherent on alternate frame changes. Thus a clear effect of an adaptable IOVD mechanism could be demonstrated in the presence of balanced, rather than degraded and incoherent, changing disparity even in a (repeated) two-frame motion sequence. 

On individual differences

As discussed previously, in the third experiment we repeated conditions from the first and second experiments with 12 naïve and untrained subjects and found that MAE in depth occurred in the majority of the subjects in all adaptation conditions. These results are consistent with those of Experiments 1 and 2 and thus rule out the possibility that the results of those experiments only reflect the characteristics of the visual system of selected subjects who are well experienced in stereoscopic motion-in-depth. 

Nevertheless, two subjects did not report an aftereffect in any adaptation conditions in Experiment 3. This result is consistent with those of Nefs et al. (2010) and Watanabe et al. (2008), who concluded that there are individual differences in the sensitivity of the IOVD mechanism. 

Similarly, in the two-frame adaptation condition of Experiment 3, three additional subjects reported no aftereffect. This lack of the aftereffect also might also be attributed to lower sensitivity to IOVD (or prior for monocular motion) in these five subjects compared with the other seven subjects. The effects of sensitivity to IOVD were expected to be most pronounced for the two-frame condition, as the duty cycle of adaptation was low (50%; one of two frame transitions) and interleaved with the frame transition in which the IOVD signal was random and presumably disruptive to adaptation. In contrast, in the RDS and URDS conditions the duty cycle was high (88.9%; 16 of 18 frame transitions) and there might have been less disturbance because of the blank frame. 

On unintended binocular matching in URDS

In agreement with the idea that URDS stimulates the IOVD and not the CD mechanism, Shioiri, Matsumiya, and Matsubara (2012) simulated the response of disparity energy neurons to URDS stimuli and confirmed that these model neurons do not respond to such incoherent disparity signals in a coherent manner. However, it could be argued that CD was contained not only in the RDS adaptor but also in the URDS adaptor in Experiments 1 and 3 because of persistent binocular matching of chance correspondences (Allison & Howard, 2011). Such a “dynamic disparity” signal would rely on persistence of matching, not instantaneous matching of frames. However, even if persistent matching enabled extraction of a CD signal in the URDS stimuli, only a certain proportion of well-matched dots should have contained CD, while in the RDS, all dots should have except for some unpaired dots presented close to the left and right edges of the planes. Thus, the effects of adaptation of the CD signal should still manifest as a difference in aftereffect between the RDS and URDS conditions. 

Existence and characteristics of adaptable IOVD mechanisms

When combined with previous reports of motion-in-depth produced by binocularly uncorrelated stimuli (Allison et al., 1998; Grafton, Nefs, & Harris, 2009; Rokers, Czuba, Cormack, & Huk, 2011; Shioiri et al., 2000), effects of motion adaptation on perceived motion-in-depth (Brooks, 2002; Czuba et al., 2011; Sakano et al., 2012; Shioiri et al., 2009), and correlation between characteristics of monocular motion perception and those of stereomotion (Brooks & Stone, 2004, 2006), these experiments provide compelling evidence for the existence and role of a motion-in-depth mechanism sensitive to IOVD. Such an IOVD adaptation process may not be wholly binocular, since after adaptation to monocular lateral motion, a binocularly viewed test pattern reportedly produces combined apparent motion-in-depth and lateral motion (Fernandez & Farell, 2006; Shioiri et al., 2009). These results suggest that monocular processes for motion-in-depth based on IOVD are adaptable and responsible, at least in part, for the depth component of the MAE. However, the preponderance of evidence suggests that the adaptable IOVD mechanisms are binocular and that a significant amount of adaptation takes place after binocular combination. Shioiri et al. (2009) found that while lateral MAEs were spatial-frequency selective, MAEs in depth for stereomotion were not. Similarly, velocity information based on color and luminance appears to be combined prior to IOVD processing, suggesting that the IOVD estimation is performed at a higher stage of motion processing (Shioiri, Yoshizawa, et al., 2012). Finally, binocular adaptation produces an MAE with a depth component (diagonal motion-in-depth) when viewing a monocular test pattern, suggesting that adaptation occurs beyond the site of binocular combination (Sakano et al., 2012). 

Conclusion

The role of IOVD cues appears to predominate over changing disparity in stereomotion adaptation experiments, despite the theoretical and empirical evidence that predicts less cue conflict with monocular motion-in-depth cues for “pure CD” in DRDS stimuli (Allison & Howard, 2000; Brooks & Stone, 2010). Experiment 1 suggests that this predominance cannot simply be explained by saturation of adaptable motion-in-depth mechanisms, and Experiment 2 shows that adaptation to IOVD can be demonstrated in the presence of coherent and well-defined CD signals in a repeated two-frame sequence. These experiments, combined with previous studies, suggest that the contribution of IOVD to the perception of motion-in-depth is substantial even in a two-frame motion, while that of CD is very limited (if any), at least at the stages of direction-selective mechanisms subject to an aftereffect phenomenon. 

Acknowledgments

This work was supported in part by grants from the Province of Ontario (Premier's Research Excellence Award), the Canadian Foundation for Innovation, and NSERC (Canada). Portions of this study have been previously reported in abstract form (Sakano & Allison, 2006, 2007). 

Commercial relationships: none. 

Corresponding author: Yuichi Sakano. 

Email: [e-mail]. 

Address: Universal Communication Research Institute, National Institute of Information and Communications Technology, Keihanna Science City, Kyoto, Japan.