Asymmetric interaction between motion and stereopsis revealed by concurrent adaptation

Abstract

Although contingent aftereffects between motion and stereopsis have been referred to as behavioral evidence for the joint processing of the two features, the reciprocal nature of encoding the two features has not been systematically studied. Using a novel form of concurrent adaptation, we probed the perception of direction- and disparity-defined coherent surfaces in parallel before and after adaptation to a stimulus that moved in a single direction at a particular binocular disparity. Contrary to earlier findings, we found a strong asymmetry between motion and stereopsis: the detection of disparity signal after adaptation was more impaired when the test stimulus was moving in the adapted direction than in the non-adapted direction, whereas the test disparity hardly affected the detection of coherent motion. However, motion adaptation became dependent on disparity when we added another surface that was moving in the opposite direction at the opposite sign of disparity to those of the original adaptor, as in previous studies of contingent aftereffects. The observed asymmetric contingency between motion and disparity adaptation urges the reinterpretation of previously reported contingent aftereffects and suggests a corresponding asymmetry between neural mechanisms devoted to processing of motion and stereopsis in human visual cortex.

Introduction

Motion and stereoscopic depth are two indispensable features for the localization of dynamic objects in three-dimensional space and time. Computational algorithms for processing these two features must resolve a common problem, dubbed ‘correspondence problem’. Optical inputs must be matched across images in succession over time to register motion velocity. Similarly, the images from the two eyes must be compared to encode stereoscopic depth. This commonality has been referred to as a theoretical ground for joint processing of motion and disparity (Qian & Andersen, 1997). The visual system indeed appears to benefit from joint processing of motion and disparity in numerous incidences. Motion and disparity information help each other to integrate and segregate otherwise ambiguous signals (van Ee & Anderson, 2001; von Grünau, Dube, & Kwas, 1993). The combination of the two features appears to be required to account for various perceptual phenomena or performances, including hyper stereoacuity for objects moving at high speed (Morgan & Castet, 1995), Pulfrich effects (Anzai, Ohzawa, & Freeman, 2001; Qian & Andersen, 1997), and figure-ground segregation (Bradley & Andersen, 1998; Bradley, Chang, & Andersen, 1998). As neural substrates for joint processing of direction and disparity, electrophysiological studies have reported neurons tuned for both direction and disparity in cats (Anzai et al., 2001) and in monkeys (DeAngelis & Newsome, 1999; Grunewald & Skoumbourdis, 2004; Pack, Born, & Livingstone, 2003; Roy, Komatzu, & Wurtz, 1992). 

Psychophysical evidence for the presence of joint processing of motion and disparity came predominantly from contingent aftereffects. A prolonged exposure to two surfaces moving oppositely at different disparities induced motion aftereffects (MAEs) in the direction opposite to that of motion paired with the given disparity during adaptation (Anstis & Harris, 1974; Sohn & Seiffert, 2006; Verstraten, Verlinde, & Fredericksen, 1994). The direction-contingent depth aftereffect was also reported using Structure from Motion (SfM) stimuli (Nawrot & Blake, 1989, 1991). 

Although these studies imply the presence of joint processing, it does not provide detailed descriptions on how motion and stereo information interact. In particular, it is not clear how much of joint and non-joint mechanisms respectively contribute to the perception of stimuli composed of the two features. Another related question is whether the influences between the two features are reciprocally balanced in terms of the magnitude and processing precedence. To answer these questions, we carried out psychophysical experiments employing a ‘concurrent adaptation’ procedure. Before and after adaptation to a single stimulus defined jointly by direction and disparity ( Figure 1A), we probed the ability of observers to detect motion-defined and disparity-defined surfaces ( Figures 1B and 1C). This protocol enabled us to assess the relative contributions of the three potential mechanisms to the processing of a stimulus jointly defined by direction and disparity: (1) direction-selective but disparity-nonselective, (2) disparity-selective but direction-nonselective, and (3) jointly selective to direction and disparity. We found a strong asymmetry in contingent adaptation: disparity adaptation was highly dependent on motion information whereas motion adaptation was hardly affected by disparity information. 

Figure 1

 

Schematic diagrams of experimental stimuli and conditions. A: Observers adapted to moving dots at crossed or uncrossed disparity (Leftward motion at crossed is shown). In the actual stimulus, white dots were presented against black background. At the center fixation point, a box containing segmented horizontal lines was presented for the gap detection task. The gap in the bottom line is shown here. B: An example of a single trial for motion detection. C: Test stimuli and conditions. Four test stimuli are shown. For simplicity, each tester is drawn as containing 100% coherence both for motion and depth. The label of each condition is shown under the table in the right side. The first and second letters in the condition name indicate whether the task-relevant and task-irrelevant features, respectively, are the same as the adaptor. ‘SS’: both features are the same as the adaptor (leftward at crossed). ‘SD’: task-relevant feature is the same as the adaptor but task-irrelevant is not. ‘DS’: task-relevant feature is different from the adaptor but task-irrelevant is the same. ‘DD’: both features are different from the adaptor.

Materials and methods

icipants

One of the authors and three naïve observers participated in single-surface adaptation experiments. Only three observers participated in the stereo adaptation without motion and double-surface experiments. All observers had normal or corrected-to-normal visual acuity and stereo vision. All participants gave written consent, approved by Seoul National University Ethics Committee. 

Stimuli during adaptation

Dichoptic stimuli were viewed through a mirror haploscope that was mounted on a head-chin rest to generate stable perception of stereoscopic depth. As an adapting stimulus, 110 white dots moving coherently in a single direction (left or right) were randomly distributed within an imaginary circular aperture of 2.48° radius against a black background. Dots were anti-aliased with subpixel resolution using a 2-D Gaussian filter with σ of .07°. The maximum contrast of each dot was 100%. The stimulus display was linearized using an 8-bit lookup table and the mean luminance was 38 cd/m ². In single-surface adaptation experiments, all dots drifted at the same speed (5°/sec) with an asynchronous, limited lifetime of 100 ms at the same depth plane (crossed or uncrossed disparity of 0.2°). Each dot disappeared after its own lifetime and then reappeared at a random position within the aperture. There were four adaptor types: leftward at front, rightward at front, leftward at far, and rightward at far. In the stereo adaptation without motion experiments, dots were identical to those in the concurrent adaptation experiments except that dots remained stationary during the lifetime (static condition) or they moved in random directions (random motion condition). In the double-surface adaptation experiment, 110 dots were placed at a crossed disparity of 0.2° and the other 110 dots at an uncrossed disparity of 0.2°. The two groups of dots moved oppositely, one to the left and the other to the right. 

Stimuli during tests

The dots were identical to the adapting dots except for contrast. The maximum contrast of each dot was ∼1% for the low contrast condition and ∼3% for the high contrast condition (marked gray and black symbols, respectively, in Figures 2, 3, 5, and 6). Two brief intervals were sequentially presented during the test, the noise and the signal. In the motion detection sessions, all dots were presented at the same disparity as the adapting stimulus or at the same amount of disparity with the opposite sign. The signal interval contained coherent dots moving in a single direction and noise dots moving in random directions, whereas the noise interval contained only randomly moving dots. All dots in the test stimulus for stereo detection were moving in a single direction identical or opposite to the adaptor. The noise interval contained only noise dots whose depths were scattered randomly over the disparity range of −0.3° to +0.3°. In the signal interval, coherent dots, which were located at a single disparity of either +0.2° or −0.2°, were embedded in noise dots. Since coherent dots were placed within the depth range of noise dots and spatially scattered in a 2-dimensional space, observers could not perform the task by simply detecting a few dots placed at the closest or farthest depth and were forced to identify a coherent, global depth plane amid the cloud of noise dots. The particular range of disparity (−0.3° to +0.3°) was chosen carefully by conducting a pilot experiment, where we confirmed that observers were able to reliably detect a 100 % coherent depth plane with the duration of 0.5 seconds. We used four types of testers, which differed in terms of whether their task-relevant and task-irrelevant features matched those of the adaptor ( Figure 1C). In the first type, both the task-relevant and task-irrelevant features were the same as the adaptor (‘SS’ in Figure 1C). In the second, the task-relevant feature was the same as the adaptor but the other feature was not (‘SD’ in Figure 1C). In the third, the task-relevant feature was different from the adaptor but the task-irrelevant one was the same (‘DS’ in Figure 1C). In the last, both features were different from those of the adaptor (‘DD’ in Figure 1C). 

Experimental procedures

An experimental session consisted of four pre- and post-adaptation blocks. Each block contained 28 trials (seven levels of coherence × four test conditions: ‘SS’, ‘SD’, ‘DS’, and ‘DD’). In pre-adaptation blocks, observers viewed two intervals of test stimuli and reported which interval contained coherent motion or disparity signal by pressing one of two keys (2 IFC). Each stimulus interval lasted for 500 ms, and the inter-stimulus interval (ISI) was 300 ms ( Figure 1B). The inter-trial interval was 1500 ms, during which observers had to make a response. Post-adaptation blocks started with a 40 sec of initial adaptation, and 4 s of top-up adaptation preceded each test trial. To ensure the stability of fixation and promote accurate binocular alignment of the two eyes during adaptation, we employed a ‘gap detection’ task (Sohn & Seiffert, 2006). In the initial adaptation, every 4 sec, a lined box appeared for 500 ms around the fixation point (the enlarged inset in Figure 1A). Each horizontal line was broken into four segments, two of which were randomly selected to be presented to one eye and the other two to the other eye. There was always one segment missing either in the upper or in the bottom lines. Observers performed a one-back task by reporting whether the position of the present gap was the same as the position of the gap that appeared 4 sec ago. During top-up adaptation, the box appeared at 1.5 sec and at 3.5 sec from the start of the stimulus. Observers reported whether the position of the gap was the same between the first and the second boxes. After adaptation, 0.5 sec of blank screen with a fixation point was presented before the test stimulus appeared. 

Data analysis

Thresholds in different viewing conditions were separately estimated using a hybrid technique, in which a constant stimuli method was applied within a session and a staircase method between sessions. In the starting session, the proportion of coherent dots in the signal interval varied across seven different levels, which was chosen out of a predetermined set of 30 coherence levels. In the following three sessions, based on the performance after each session, we continued to adjust the range of coherence levels within the predetermined set such that the average percent correct was roughly around 75%. We obtained a psychometric curve from four sessions of data for each viewing condition and estimated a coherence threshold for 75% performance by fitting the cumulative Gaussian function to the curve. We repeated this procedure 1000 times by re-generating the data sets using a bootstrap method (Efron & Tibshirani, 1986), and obtained the mean threshold and an estimation error (SE). We performed this procedure for pre- and post adaptation conditions at each tester type, adaptor type, and contrast level in each observer. Then, we calculated adaptation index (AI) defined below.  

$\mathrm{A}\mathrm{d}\mathrm{a}\mathrm{p}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{I}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{x}(\mathrm{A}\mathrm{I})=\frac{(T{H}_{post}-T{H}_{pre})}{(T{H}_{post}+T{H}_{pre})},$

(1)

where TH _pre and TH _post indicate thresholds estimated before and after adaptation, respectively. An AI can have a value between negative one and positive one. A positive AI indicates an increase in detection thresholds after adaptation and a negative means decreased thresholds. In this study, we call the latter type of adaptation ‘null adaptation’, as opposed to the conventional adaptation effects characterized by increased detection thresholds for the adapted feature. In order to evaluate the contingency of adaptation on the task-irrelevant feature information, we normalized the difference between AIs from testers with the adapted and non-adapted irrelevant features, and named it contingency index (CI).  

$\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}\mathrm{I}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{x}(\mathrm{C}\mathrm{I})=\frac{(A{I}_{same}-A{I}_{diff})}{\sqrt{\frac{S{E}_{same}^2+S{E}_{diff}^2}{2}}}$

(2)

where AI _same is the AI from the condition where the task-irrelevant feature was the same as the adaptor and AI _diff is the AI from the test condition where the task-irrelevant feature was different from the adaptor. SE is standard error of estimation for each condition. A positive CI indicates that an AI is greater when the tester contained the task-irrelevant feature that was presented during adaptation than when it contained non-adapted feature. A negative CI means the opposite, and a CI near zero means no contingency between the two features. 

Results

In the current study, following adaptation to an identical adaptor defined by direction and disparity, we measured the detection performance for the direction-defined and disparity-defined coherent surfaces. Note that the motion and stereopsis tasks share the identical task structure (‘discriminating a surface of coherent signals from noise’) and were both performed before and after adaptation, allowing direct comparisons between perceptual changes in the two features after adaptation. For quantitative comparison, we normalized the amount and the sign of adaptation effects across conditions by computing adaptation index (AI) for each pair of pre- and post-adaptation thresholds. A positive AI indicates an increase in detection thresholds after adaptation (conventional adaptation) and a negative means a decrease (null adaptation). We obtained thresholds at more than one level of test contrast (marked as black and gray symbols in Figures 2, 3, 5, and 6) because adaptation effects might vary in magnitude depending on the contrast of test stimuli due to changes in contrast gain (Kohn & Movshon, 2003). Since we did not find any systematic difference between the results at different contrast levels, we merged results from different contrast conditions for statistical tests. 

Figure 2

 

Detection thresholds and adaptation effects in the motion task. Data from different adaptor types, contrast levels of testers, and observers are plotted together. The shapes of symbols indicate different types of adaptor: RN for the adaptor moving to the rightward at the near disparity; LN for moving leftward at near disparity, RF for moving rightward at far disparity, and LF for moving leftward at far disparity. The brightness of symbols indicates the contrast level of tester stimuli: black for high contrast and gray for low contrast. A–D. Pairs of pre- and post-adaptation detection thresholds for four different tester conditions: A, same direction and same disparity as the adaptor (‘SS’); B, different direction and same disparity (‘DS’); C, same direction and different disparity (‘SD’); D, different direction and different disparity (‘DD’). The arrows and numbers indicate average thresholds for pre- and post-adaptation in each condition. E–F. Adaptation indices for the non-adapted disparity (ordinate) plotted against those for the adapted disparity (abscissa): E, AIs from the condition where the test direction was the same as the adaptor; F, AIs from the condition where it was opposite to the direction of the adaptor. The histograms are projected frequency distributions of AIs along horizontal (‘SS’ or ‘DS’) or vertical axes (‘SD’ or ‘DD’). High and low contrast conditions are described as stacked bars, each corresponds to black and gray portions of the bars. The bars in color indicate AIs that were significantly different from zero with the 95% confidence interval of estimation (red for ‘conventional adaptation’ and blue for ‘null adaptation’). The arrow indicates the average of the distribution. Error bars are standard error (SE) estimated by a bootstrap procedure (see Materials and methods).

Figure 3

 

Detection thresholds and adaptation effects in the stereo task. Figure formats and notations are the same as in Figure 2. A–D: Plots of 75% detection thresholds for disparity. Each plot represents four different experimental conditions, A: same disparity and same direction as the adaptor (‘SS’). B: different disparity and same direction (‘DS’). C: same disparity and different direction (‘SD’). D: different disparity and different direction (‘DD’). E and F show adaptation indices for the same (E) and different (F) disparities.

Single surface adaptation: Little disparity contingency in direction-selective adaptation

In each tester condition, we obtained a total of 19 pairs of motion detection thresholds from pre- and post-adaptation conditions from four different observers. When the target direction was the same as the adaptor, motion detection threshold substantially increased ( p < .05 for Wilcoxon's matched pairs test) following adaptation regardless of whether the disparity of the tester, which was irrelevant to the motion detection, was the same as that of the adaptor (‘SS’, Figure 2A) or not (‘SD’, Figure 2C). When these elevations in detection threshold were translated into AIs, the mean AI did not significantly differ between ‘SS’ (.13) and ‘SD’ (.11) conditions ( p = .69 for Wilcoxon's matched pairs test, Figure 2E). When the target direction was opposite to that of the adaptor, there was a tendency that detection threshold decreased following adaptation (note the data points under the diagonal line in Figures 2B and 2D), indicating the presence of null adaptation. The average AIs from the ‘DS’ (−.05) and ‘DD’ (−.07) conditions were negative and were not significantly different from each other ( p = .15 for Wilcoxon's matched pairs test, Figure 2F). The overall trend of positive AIs for the adapted direction and negative AIs for the direction opposite to the adaptor was consistently observed in the data from individual observers ( Supplementary Figure 1A). 

We evaluated the contingency of motion adaptation on the task-irrelevant disparity information by comparing adaptation effects from the testers presented at the adapted disparity and those from the testers at the non-adapted disparity. In doing so, we normalized the difference between AIs by taking into account their estimation error and named it contingency index (CI). (see Materials and methods). CI provides a robust means to assess the magnitude and reliability of observed differences in adaptation effects especially when the size of adaptation substantially varies across different adaptors or observers as in our data. A positive CI indicates that an AI is greater at the adapted disparity than at the non-adapted disparity, a negative CI means the opposite, and a CI near zero means no contingency between the two features. We plotted CIs against the pooled AI of the two disparity conditions in Figures 6A and 6B, respectively for adapted (average CI = .32) and non-adapted directions (average CI = .44). The average CIs were not significantly different from zero ( p = .49 and .09 for Wilcoxon's signed rank test), meaning no contingency in motion adaptation on disparity. None of the AI pairs from the two disparity conditions was significantly different from each other in the adapted direction and only one (out of nineteen) showed significant difference (marked as red in Figure 6B) in the non-adapted direction. This lack of disparity contingency of motion adaptation was not relevant to the size of adaptation effect, which was indicated by no significant correlation between the pooled AI and the magnitude of CI ( r = .11 and .32 for adapted and non-adapted directions. p > .05 for both conditions). The results indicate that in motion adaptation, there is no or negligible, if any, contingency on disparity. 

Single surface adaptation: Strong direction contingency in disparity-selective adaptation

Figures 3A– 3D show stereo detection thresholds from pre- and post-adaptation conditions for each of the four tester conditions. When both the disparity and direction of the tester were the same as those of the adaptor, the detection threshold increased after adaptation (‘SS’, Figure 3A; p < .01 for Wilcoxon's matched pairs test). In contrast, when the same disparity target was moving in the opposite direction to the adaptor, the thresholds significantly decreased (‘SD’, Figure 3C; p < .05 for Wilcoxon's matched pairs test). The similar tendency was observed when the test disparity was different from that of the adapting stimulus; conventional adaptation effects shown as increased detection thresholds following adaptation to the stimulus in the adapted direction (‘DS’, Figure 3B; p < .01 for Wilcoxon's matched pairs test) and null adaptation shown as decreased detection thresholds to the non-adapted direction (‘DD’, Figure 3D; p < .05 for Wilcoxon's matched pairs test). Note that regardless of the target disparity during the test, the sign of adaptation was determined completely by whether the direction of the tester matched to that of the adaptor, even though the tester direction was irrelevant to the task here. Such data patterns are shown as positive AIs with the adapted direction (mean AI = .09 for ‘SS’ in Figure 3E; mean AI = .07 for ‘DS’ in Figure 3F) and negative AIs with the non-adapted direction (mean AI = −.06 for ‘SD’ in Figure 3E; mean AI = −.05 for ‘DD’ in Figure 3F). This strong tendency of direction-dependent disparity adaptation was consistent across individual observers ( Supplementary Figure 1B) and confirmed by the positive average CIs for both the adapted (3.35, Figure 6C) and non-adapted (2.72, Figure 6D) disparities ( p < .01 for Wilcoxon's signed rank test). For both disparity conditions, in nearly 50% of the AI pairs (9 out of 19), the AI from the tester moving in the adapted direction was significantly larger than that from the tester in the opposite direction (marked as red in Figures 6C and 6D). The direction contingency of disparity adaptation became more pronounced as the size of AI increased, evidenced by the high positive correlation between AIs and CIs ( r = .89 and .82 for adapted and non-adapted disparity conditions. p < .01). This indicates that the contingency was reliably observed whenever substantially large adaptation was generated by a given adaptor. From these results, we conclude that the stereo detection was impaired with the adapted direction and improved with the opposite direction, regardless of the target disparity to be detected. 

Robust disparity-selective adaptation without coherent motion signal

The results of the first two experiments showed that direction-selective adaptation did not depend on the disparity of adapting stimuli, whereas disparity-selective adaptation was contingent upon the direction of adapting stimuli. While this asymmetry may be taken as the inherent precedence of motion over stereo information processing, an alternative explanation can account for the observed asymmetry. Suppose that the adapting stimulus used in our study failed to induce substantial disparity adaptation and was optimal only for generating motion adaptation. Then, the performance in either the motion or stereo task would be hardly affected by the disparity of testing stimuli as observed in our data. 

To exclude the possibility that our findings of asymmetric contingent adaptations were simply caused by the lack of disparity adaptation, we conducted a control experiment. We repeated the stereo detection experiment using adapting and test stimuli defined only by binocular disparity without any coherent motion signal. Since the adaptor lacking coherent motion signal cannot generate direction-selective adaptation, any changes in detection threshold occurring after adaptation should be attributed to disparity adaptation. The three out of four observers in the first two experiments participated. The stimuli and procedure were exactly the same as those in the ‘stereo detection’ experiment except how the positions of dots were updated over time. In the ‘static’ condition, dots remained stationary for 100 ms, which was the same limited lifetime as that in the concurrent adaptation experiments, and then reappeared in a random position inside the circular display. Such stimuli elicit the percept of asynchronously twinkling dots without carrying any coherent motion. In the ‘random motion’ condition, each dot moved with the same lifetime and speed as before. However, the directions of dots were equally spread over 360 degrees, and each dot maintained its direction throughout the lifetime. In both conditions, adaptation elevated detection thresholds for the adapted disparity (Wilcoxon's matched pairs test, p < .05; left two bars in Figures 4A and 4C) but did not generate any significant difference in thresholds for the non-adapted disparity ( p = .06 for both conditions; right two bars in Figures 4A and 4C). Conventional adaptation effects (positive AI) were evident for the adapted disparity (white bars in Figures 4B and 4D), and the sizes of disparity-selective AIs were not significantly different from those for ‘SS’ condition of concurrent adaptation (in both ‘static’ and ‘random motion’ conditions, p > .05 for Wilcoxon's matched pairs test across three observers). 

Figure 4

 

Stereo adaptation without coherent motion signal. A and C: Average thresholds for 75% performance in stereo detection before (white bars) and after (gray bars) adaptation: A for the static condition and C for the random motion condition. The error bars indicate standard error (SE) of the mean across different adaptors (at near or far disparity) and three observers. The asterisk indicates statistically significant difference between pre- and post-adaptation thresholds: p < .05 for Wilcoxon's matched pairs test. B and D: Average adaptation indices from three observers are shown for stereo adaptation with static (B) and with random motion (D). White bars are AIs from the conditions where the tester disparity was the same as the adapting disparity and gray bars are when the target disparity was different from the adaptor. Error bars indicate SE estimated by a bootstrap.

The considerable amount of disparity-selective adaptation makes the possibility unlikely that our adapting stimuli were not optimal enough to promote strong disparity adaptation per se. Furthermore, considerable null adaptation (negative AI) was observed for the non-adapted disparity (gray bars in Figures 4B and 4D). These results undoubtedly indicate that disparity signals in our stimuli were by themselves strong enough to readily promote adaptation, but became limited as an adaptor when presented in conjunction with coherent motion signal. 

Double surface adaptation: Disparity contingency in direction-selective adaptation

While our finding of the direction dependency in stereo detection following concurrent adaptation is consistent with the previous studies that showed direction-contingent depth aftereffects in SfM stimuli (Nawrot & Blake, 1989, 1991), little stereo dependency in motion detection is rather perplexing given previous studies that reported significant disparity-contingent directional aftereffects (Anstis & Harris, 1974; Sohn & Seiffert, 2006; Verstraten et al., 1994). One notable difference between these studies and the current study is that the previous studies employed two oppositely moving surfaces at different disparities as an adaptor. We conjectured that such coexistence of double opposing surfaces abolishes disparity-independent motion adaptation and selectively promotes disparity-contingent motion adaptation. Since adaptation by disparity-independent motion mechanisms would be cancelled between opposing directions in such an adaptor, only adaptation by the motion mechanism that is also tuned for disparity would be observed. On the other hand, in the adapting stimuli consisting of a single surface like the one used in our study, since there is no potential suppression or cancellation between opposing stimulus features, both types of adaptation would be observed and the contribution of the jointly tuned mechanism would be relatively minimal. 

Our interpretation led us to two predictions with the addition of a double-opposing surface to our original adaptor: first, direction-selective adaptation will be more contingent on disparity, and, second, the size of direction-selective adaptation itself will be reduced because the added surface cancels out opposite directional adaptation. We repeated the motion detection experiment, but this time with two adapting surfaces placed on top of each other. The adapting stimulus contained two groups of dots that were moving oppositely at crossed and uncrossed disparities, respectively. Test stimuli were the same as those in the experiment with a single-surface adaptor. Note that, since both the two values of direction and disparity in test stimuli were presented as the adaptor, the condition where the test direction was non-adapted (‘DS’ and ‘DD’) did not exist. Thus, we categorized four types of test stimuli into two conditions, ‘SS’ and ‘SD’. For example, when the adaptor was a leftward motion at crossed and rightward at uncrossed disparities, the ‘SS’ condition includes test stimuli with leftward motion at the crossed disparity and rightward at the uncrossed. Rightward at crossed and leftward at uncrossed disparities fall in the ‘SD’ condition. 

The results supported both of our predictions. We found increased disparity dependency in motion adaptation compared to that in the single surface experiment. We plotted thresholds from post-adaptation against those from pre-adaptation ( Figures 5A and 5B) and AIs from ‘SD’ condition against those from ‘SS’ ( Figure 5C). The increase in detection thresholds after adaptation was significant ( p < .01 for Wilcoxon's matched pairs test) for ‘SS’ condition but not ( p = .63) for ‘SD’ condition. Accordingly, AIs were significantly larger than zero in the ‘SS’ condition (mean AI = .07, p < .01) but not in ‘SD’ condition (mean AI = .01, p = .4). Wilcoxon's matched pairs test showed the AIs from the ‘SS’ and ‘SD’ conditions were significantly different ( p < .05). For direct comparison between CIs of the single- and double-surface experiments, examine Figures 6E to 6A. The average CI substantially shifted in positive direction in the double surface experiment (1.1, the arrow in Figure 6E; The average CI was significantly larger than zero. p < .05 for Wilcoxon signed rank test), as compared to the single surface experiment (.32, the arrow in Figure 6A; The average CI was not significantly different from zero. p = .49 for Wilcoxon signed rank test). Also, the number of AI pairs that showed significant differences between the ‘SS’ and ‘SD’ conditions increased to 7 with double surfaces (red triangles in Figure 6E), from zero with a single surface (no colored triangles in Figure 6A). Lastly, the linear relationship between the size of pooled AI and the magnitude of CI, which was the characteristic feature of contingent adaptation in stereo detection, was evident ( r = .5, p < .05) in the double surface experiment but not in the single surface experiment. 

Figure 5

 

Results from motion adaptation to double opposing surfaces. The figure format and symbols are same as in Figures 2A, 2C, and 2E. A–B: Plots of 75% detection thresholds for the testers with the adapted disparity (A) and with the different disparity (B). C: AI's for motion detection with double surfaces.

Figure 6

 

Contingency between motion and stereo adaptation. In each panel, contingency indices are plotted against pooled AIs and summarized in histograms on top. A pooled AI is an overall magnitude of adaptation in a given condition and was obtained by taking the squared root of the sum of the squared AIs from the condition where the task-irrelevant feature was the same as the adaptor and that where it was different. The colored symbols both in plots and histograms are AI pairs that showed significant difference with the 95% confidence interval under bootstrap-generated data sets; red for larger AI from the condition where the task-irrelevant feature was the same as the adaptor, and blue for larger AI from the condition where the task-irrelevant feature differed from the adaptor. The correlation coefficient between the pooled AI and CI with its significance (*, p < .05 and **, p < .01) is shown in each panel. The arrow in each histogram is the average CI. A–B. Motion adaptation. C–D. Stereo adaptation. E. Motion adaptation with double surface adaptor.

In order to examine our second prediction on the overall magnitude of adaptation, we compared the magnitudes of AIs from the ‘SS’ condition between the double- and single-surface experiments. Among the 19 AIs in the single surface experiment, except two data points from the observer (SK) who participated only in the single-surface experiment, 17 AIs had matched adaptor types and contrast levels in double-surface experiment. As predicted, the average AI for the ‘SS’ condition in double-surface experiments (.08) was smaller than that in single-surface experiments (.14), although the difference between the average AIs turned out to be marginally insignificant (Wilcoxon's matched pairs test, p = .055) based on a small sample size of 17. The within-adaptor comparison between single- vs. double-surface adaptors is shown in Supplementary Figure 2. In about half pairs (8 out of 17), AI from the single-surface experiment was significantly larger than that from the double-surface experiment with 95% confidence intervals under the bootstrap-generated distribution. In summary, we conclude that the adaptor of double opposing surfaces increased disparity contingency of direction-selective adaptation but decreased the overall magnitude of motion adaptation. 

Discussion

The most prominent finding in our study is the asymmetrical interaction between motion and stereopsis in adaptation. Concurrent adaptation impaired the performance of detecting disparity-defined coherent surfaces when tester dots were moving in a direction identical to the adaptor, but enhanced the performance when the tester was moving in the opposite direction. Such strong contingency of disparity adaptation on direction was in sharp contrast with little contingency of motion adaptation on disparity. This asymmetry in direction and disparity adaptation is not likely to be due to the lack of disparity adaptation. First, the disparity range we used in the current study was carefully selected based on the known disparity selectivity in humans. Although the width of disparity tuning in some neurons of monkey MT is as large as 1 deg (DeAngelis & Newsome, 1999), humans are able to detect much finer disparities down to 2 arc min (Blackmore & Julesz, 1971; Stevenson, Cormack, Schor, & Tyler, 1992) but often lose good perception of depth when the disparity is larger than 1 deg (Backus, Fleet, Parker, & Heeger, 2001). The disparity range used in the current study is considered as optimal for human perception of disparity-defined surfaces and has been reported to generate substantial adaptation effects (Stevenson et al., 1992). Nevertheless, we performed a set of control experiments using adaptors without coherent motion and successfully demonstrated a substantial amount of disparity-selective adaptation in our stimuli. 

The strong asymmetry between motion and stereopsis in adaptation has not been previously reported and may be considered as a conflict to the previous reports of disparity-selective directional aftereffects (Anstis & Harris, 1974; Verstraten et al., 1994; Sohn & Seiffert, 2006). We showed that this discrepancy was due to the use of different adapting stimuli. By pitting against two antagonistic values in each feature domain during adaptation, previous studies had effectively demonstrated the existence of joint processing of two features, but cannot provide a fair quantitative description of mutual influences between motion and stereopsis. 

Motion precedence over disparity in processing dynamic 3-D surfaces

In observers' task in our study, where they had to detect a surface of coherent dots embedded in noise dots, direction or disparity signals must be pooled over a large space. One possible interpretation of the asymmetry in contingency is that the neural mechanism underlying the integration of local motion signals precedes that underlying the integration of local disparity signals somewhere along the hierarchy in the construction of global surfaces. 

Motion precedence over disparity in global integration is not surprising given what we know of how the visual system encodes the two features. Direction selectivity is found as early as at the level of retina in rabbits (Barlow, Hill, & Levick, 1964; Taylor, He, Levick, & Vaney, 2008), and in primate V1, motion is processed by the monocular as well as binocular neurons (Hubel & Wiesel, 1968). On the other hand, information of binocular disparity is not available before V1, where signals from the two eyes are first combined. In addition, it is known that the basic characteristics of direction-selectivity in V1 are well preserved in a subpopulation of MT neurons that receive projections from V1 (Movshon & Newsome, 1996) whereas disparity-selectivity in the primary visual area and that observed in extrastriate areas exhibit qualitative differences (Cumming & DeAngelis, 2001). A recent study on monkeys (Ponce, Lomber, & Born, 2008) suggests that direction selectivity of neurons in MT is dependent largely on direct input from V1 whereas disparity selectivity in MT requires an involvement of V2/V3 at intermediate processing stages. The authors inspected changes in direction and disparity tuning of MT neurons while inactivating indirect pathways (V2 and V3) from V1 to MT. The directional tuning curves of MT neurons were well preserved whereas their disparity tuning curves were substantially distorted during inactivation of V2 and V3. In line with these reports, our findings suggest that the processing of motion information is completed earlier and affects the process of encoding disparity information. 

We are not arguing here that motion integration occurs entirely independent of disparity information. For example, local cancellation between directionally opposed motion signals paired within a small region is dependent upon whether or not opposed motion signals are presented at the same disparity (Qian, Andersen, & Adelson, 1994). This implies that the fine-spatial-scale interaction between motion signals is affected by joint relationship between disparity and direction. We underscore that the perceptual task in our study taps the process of constructing coherent global surfaces out of local signals, rather than the encoding of local motion and disparity signals. Only globally coherent directional signals, but not local random motion, affected detection performance of disparity-defined surfaces after adaptation (stereo adaptation experiment without coherent motion). Such dominance of motion over disparity in perception of global surfaces in our study is also consistent with previous studies using non-adaptation paradigms (Hibbard, Bradshaw, & DeBruyn, 1999; Lankheet & Palmen, 1998; Muller, Lankheet, & van de Grind, 2004) that reported strong motion dominance over disparity information in spatial integration tasks. 

Potential neural substrates for the asymmetry

In the cortical pathways processing motion and stereopsis, we conjecture MT as the most probable neural locus responsible for the observed asymmetry in contingency. Neurons in MT are known to integrate local inputs over a larger region of visual space than do their cortical afferents (Croner & Albright, 1999), which has been reported both for motion (Movshon & Newsome, 1996) and disparity (DeAngelis & Uka, 2003). Our finding that disparity-selective adaptation was contingent upon co-located global coherent motion, but not upon local random motion, supports the idea that MT plays a major role in the extraction of feature-defined surfaces in the current study. 

Our concurrent adaptation paradigm was designed to measure the extent to which three potential neural populations contribute to the encoding of surfaces defined jointly by direction and disparity: nonjoint direction-selective, nonjoint disparity-selective, and joint selective units ( Figure 7A, from left to right). Our results from single-surface adaptor experiment ( Figure 7B) are qualitatively close to the pattern of adaptation predicted by the nonjoint direction-selective units (the leftmost panel in Figure 7A). This motion dominance in concurrent adaptation may be explained by the imbalance between the three groups of neural population in MT. In MT, nearly all neurons are direction-selective (Albright, 1984; Maunsell & van Essen, 1983a; Snowden, Treue, & Andersen, 1992; Zeki, 1974) whereas only 60 (Maunsell & van Essen, 1983b)–90% (DeAngelis & Uka, 2003; Palanca & DeAngelis, 2003) of neurons are tuned for disparity. Also, the degree of selectivity is greater for direction than for disparity. In addition, MT clearly has a subregion of direction-selective cells with poor disparity tuning and a subregion of jointly selective cells, but little of disparity-selective cells with poor direction selectivity (DeAngelis & Newsome, 1999). 

Figure 7

 

Hypothetical and observed contingency tables of adaptation effects. The four cells in each 2 × 2 contingency table represent categorical test conditions, which are the combinations out of adapted and non-adapted directions (columns) and disparities (rows). The adaptor corresponds to the top left cell (adapted in both direction and disparity). Color describes adaptation effects normalized within each unit (A) or experimental condition (B). Hue indicates the sign of adaptation, red for conventional adaptation and blue for null adaptation. Brightness indicates the magnitude of adaptation, 0 for no adaptation and 1 for the largest adaptation effect in a given unit of neural population (A) or experimental condition (B). A. Hypothetical adaptation effects that will be observed in three different neural populations; nonjoint direction-selective unit, nonjoint disparity-selective unit, and jointly selective unit, from left to right. B. The observed adaptation effects from the motion detection experiment (left) and from the disparity detection experiment (right). The number in each cell is the AI averaged across data from all of the observers (arrows in Figures 2E, 2F, 3E, and 3F). For visual comparison with the hypothetical adaptation effects in A, we computed the normalized AIs, which are indicated by color, by dividing the averaged AIs by the largest average AI in a given contingency table.

This disproportion in functional architecture of MT is consistent with our findings in many aspects. First, it explains the asymmetry in contingency in the single-surface adaptation. Since a single surface adapts both non-joint direction-selective units and jointly selective units in MT, motion detection, which can be performed based on contributions from either units, is not dependent much on the disparity of stimuli (‘motion detection’ in Figure 7B). In contrast, because disparity detection requires contributions mostly from the jointly selective units, it would be highly dependent on the direction of stimuli (‘stereo detection’ in Figure 7B). Next, double opposing surfaces would adapt only jointly selective units in MT because non-joint direction-selective units do not respond to opposing directions of motion due to mutual suppression (Bradely, Qian, & Andersen, 1995). Under this condition, motion detection as well as disparity detection is highly contingent upon each other because performing the motion task is now determined by the contribution from the jointly selective units. Lastly, the single-surface stimulus adapts both non-joint direction-selective units and jointly selective units whereas the double surfaces adapt only jointly selective units, resulting in the total amount of direction-selective adaptation being greater for the former. In our double-surface experiment, the average direction-selective AI for the adapted direction and disparity was only 61% of direction-selective AI with the single-surface adaptor, which is comparable to the reported 68% of disparity-selective neurons in MT from an early study (Maunsell & van Essen, 1983b). 

While the above explanation based on the imbalance between heterogeneous MT populations can serve as an underlying mechanism for the observed asymmetry, there may be another explanation. Different tuning properties of MT neurons for direction and disparity may also explain the observed asymmetry. In our study, disparity selective adaptation was probed by the two disparity levels that are only apart by 0.4 deg whereas direction selective adaptation by opposite directions. The two test directions in the current study may be different enough to activate separate populations of MT neurons due to relatively narrow direction tuning curves, but the disparity difference in the current study be rather narrower than that can be distinctively represented by some of MT neurons, whose disparity tuning width can be as large as 1 deg (DeAngelis & Newsome, 1999). Even though we demonstrated that the amount of perceptually measured disparity-selective adaptation was significant and comparable to that of direction-selective adaptation, the near and far disparity in the present study may have adapted a subpopulation of MT neurons with similar strength given the disparity tuning width of MT neurons. This wide tuning property for disparity may have contributed to the weak disparity contingency in direction selective adaptation that we observed. The explanation based on different tuning properties for direction and disparity is not contradictory nor mutually exclusive with the imbalance hypothesis suggested above. Instead, they may well exist in parallel and contribute to the asymmetric interaction between the two features. 

Concluding remarks

The concurrent adaptation paradigm enabled us to quantitatively assess the relative influence of two features and infer the processing precedence, which have not been addressed by previous studies. Our proposed explanation of the asymmetric contingency effects based on the disproportion of joint and non-joint feature-selective neural populations is applicable to previously reported contingent aftereffects in other feature domains, where two opposing values in each feature were employed as an adaptor: orientation and color (McCollough, 1965); color and motion (Favreau, Emerson, & Corballis, 1972). Our findings warn us to exercise cautions when interpreting contingent aftereffects as evidence for neural mechanisms for reciprocal processing between two features. Our concurrent adaptation procedure and analysis offer a valid and effective means of studying interactions between multiple features. Furthermore, the experimental results may provide a prediction for neural substrates underlying adaptive changes in perception of stimuli defined by multiple features in any future electrophysiological or imaging studies. 

Supplementary Materials

Supplementary Figure 1 - Supplementary Figure 1 

Supplementary Figure 1. Average AIs for individual observers. Average AIs from four test conditions and four observers are shown for motion (A) and stereo (B) adaptation. The error bars indicate SE (across individual AIs). 

Supplementary Figure 2 - Supplementary Figure 2 

Supplementary Figure 2. Pair-wise comparison between single vs. double surface adaptors. AIs from ‘SS’ condition of the double surface adaptor experiment are plotted against those from ‘SS’ condition of the single surface adaptor. The meaning of the symbol shapes is the same as figures in the main text. 

Acknowledgments

We thank Adriane E. Seiffert for helpful comments on the earlier version of the manuscript. This work was supported by the Cognitive Neuroscience Research program of the Korea Science and Engineering Foundation (M10644020001-06N4402-00110) to S.-H. L. and the Korean Research Foundation Grant funded by the Korean Government (MOEHRD, Basic Research Promotion Fund) (KRF-2007-354-H00002) to W. S. 

Commercial relationships: none. 

Corresponding author: Sang-Hun Lee. 

Email: [e-mail]. 

Address: Department of Brain and Cognitive Sciences, Seoul National University, 588 Gwanak-ro, Gwanak-gu, Seoul 151-742, Republic of Korea.