Motion processing with two eyes in three dimensions

Abstract

The movement of an object toward or away from the head is perhaps the most critical piece of information an organism can extract from its environment. Such 3D motion produces horizontally opposite motions on the two retinae. Little is known about how or where the visual system combines these two retinal motion signals, relative to the wealth of knowledge about the neural hierarchies involved in 2D motion processing and binocular vision. Canonical conceptions of primate visual processing assert that neurons early in the visual system combine monocular inputs into a single cyclopean stream (lacking eye-of-origin information) and extract 1D (“component”) motions; later stages then extract 2D pattern motion from the cyclopean output of the earlier stage. Here, however, we show that 3D motion perception is in fact affected by the comparison of opposite 2D pattern motions between the two eyes. Three-dimensional motion sensitivity depends systematically on pattern motion direction when dichoptically viewing gratings and plaids—and a novel “dichoptic pseudoplaid” stimulus provides strong support for use of interocular pattern motion differences by precluding potential contributions from conventional disparity-based mechanisms. These results imply the existence of eye-of-origin information in later stages of motion processing and therefore motivate the incorporation of such eye-specific pattern-motion signals in models of motion processing and binocular integration.

Introduction

The eyes of primates are forward-facing and horizontally offset. As a consequence, an object moving directly toward or away from an observer yields horizontally opposite directions of motion on the two retinae. In addition to the changes in binocular disparity that result from these opposite retinal motions, visual percepts of 3D motion are in part determined by a mechanism that extracts interocular velocity differences (IOVDs) per se. Although there is an increasing body of evidence suggesting a strong and unique contribution of IOVDs (Brooks, 2002a, 2002b; Brooks & Stone, 2004, 2006; Czuba, Rokers, Huk, & Cormack, 2010; Fernandez & Farell, 2005; Rokers, Cormack, & Huk, 2008; Shioiri, Kakehi, Tashiro, & Yaguchi, 2009; Shioiri, Nakajima, Kakehi, & Yaguchi, 2008; Shioiri, Saisho, & Yaguchi, 2000), it is unclear how motion signals used to make this interocular comparison fit into the known visual hierarchy (Regan & Gray, 2009). We investigated whether the IOVD mechanism operated upon early stages of motion processing typically ascribed to primary visual cortex (V1), or on later stages of processing that extract the motions of patterns, which is typically ascribed to extrastriate areas like the middle temporal area (MT) and related motion processing structures (Khawaja, Tsui, & Pack, 2009; Movshon, Adelson, Gizzi, & Newsome, 1985; Perrone & Thiele, 2002; Simoncelli & Heeger, 1998). 

Plaid stimuli have been used to characterize the hierarchical steps involved in 2D motion processing (Adelson & Movshon, 1982; Movshon et al., 1985; Welch, 1989) and 3D depth processing (Delicato & Qian, 2005; Farell, 2003). Superimposing two sinusoidal gratings drifting in different directions produces a plaid pattern that appears to move in a single coherent direction. “Component motion” neurons in V1 respond maximally when either one of the gratings moves in a preferred direction, orthogonal to the cell's preferred spatial orientation. These neurons have small receptive fields and are thought to function as 1D motion detectors. In contrast, “pattern motion” neurons in areas like MT respond maximally whenever the plaid pattern moves in the cell's preferred direction, regardless of the direction of the individual gratings that constitute the plaid (Movshon et al., 1985; Rodman & Albright, 1989). These pattern motion cells have larger spatial receptive fields than cells in V1 and function as 2D motion detectors. 

In addition to the notion that V1 neurons extract component motions, the standard hierarchy of the visual system also dictates that neurons in V1 combine monocular inputs into a single binocular stream (Hubel & Wiesel, 1968), which can thus be thought of as effectively “cyclopean” (Carney & Shadlen, 1993; Shadlen & Carney, 1986). Because interocular velocity differences must (by definition) be extracted prior to complete binocular combination, these two assertions imply that interocular velocity differences must be derived from early component motions, rather than later pattern motion signals. 

Contrary to this prediction, in a series of psychophysical experiments described below, we repeatedly found evidence for a strong contribution of eye-specific pattern motions to 3D motion percepts. In the Results section, we describe the dependencies of 3D motion direction discrimination on the relative directions of both component and pattern motions present in dichoptic motion displays. Each experiment in this sequence presents an increasingly specific requirement that the visual system rely on eye-specific pattern motions to compute 3D motion (as distinct from component motions and/or changes in disparity over time). This culminates in a novel “dichoptic pseudoplaid” visual stimulus that lacks conventional binocular matches—but which still yields 3D motion percepts when it contains global pattern motions that are horizontal and opposite in the two eyes. In the Discussion section, we then review why the overall pattern of results cannot be explained by known disparity-based mechanisms. Together, these results support the conclusion that the brain extracts 3D motion by comparing eye-specific 2D motion signals, using eye-of-origin information widely believed to be unavailable at later stages of visual motion processing. 

Methods

Observers

A total of 7 observers participated in the experiments. All had normal or corrected-to-normal vision. Four observers (the authors, males aged 26–44) were experienced psychophysical observers in both motion and depth experiments. One of the authors (B.R.) and three naive observers participated in a second set of 2AFC experiments. One of the naive observers was an otherwise experienced psychophysical observer. The other 2 naive observers had no history of performing visual psychophysics whatsoever. All naive 3 observers showed similar effects at the individual level, although the more experienced naive observer tended to show larger modulations as a function of our experimental manipulation (yet smaller than the expert author subject). Observers were included based on the criterion that they could easily judge the direction of 3D motion for a large (12.5 deg) grating that drifted in opposite horizontal directions in the two eyes (one potential naive observer was excluded on these grounds). 

Experiments were undertaken with the written consent of each observer, and all procedures were approved by The University of Texas at Austin Institutional Review Board. All data were collected at UT Austin, and all observers were recruited from the UT Austin community. 

Apparatus

All experiments were performed on a Quad Core Intel Mac Pro with an NVidia GeForce 8800 GT GPU, running Matlab (The Mathworks) and the Psychophysics Toolbox (Brainard, 1997). The stimuli were presented on two 35.0 cm × 26.3 cm CRT displays (ViewSonic G90fB, one for each eye; 75 Hz, 1280 × 1024 pixels) at a viewing distance of 90 cm (21.2 × 16.3 deg of visual angle). Left- and right-eye half-images were combined using a mirror stereoscope. The luminances of the two displays were linearized using standard gamma-correction procedures, and the mean luminance was 50.6 cd/m². 

Stimuli

All stimuli were presented within an annular aperture (1 deg inner and 6.25 deg outer radii) in a background of 1/f noise (which made it easy for the observers to maintain vergence). Additionally, a small, square fixation mark was placed at the center of the display, which had both horizontal and vertical nonius lines on its perimeter. 

The grating and conventional plaid stimuli were spatially apertured (2-dimensional Gaussian, sigma 2.5 deg, centered on the fixation point) and temporally apertured (positive half of a raised cosine, 500 ms half-period, spanning the stimulus duration). Each grating was presented at 10% Michelson (unapertured nominal) contrast, unless otherwise specified. The overall starting (and ending) phase of the drifting gratings was randomized across trials with respect to the Gaussian apertures. On each trial, all gratings started and ended at π binocular phase disparity (i.e., began completely out of phase in the two eyes so that the direction of disparity was ambiguous). No feedback concerning performance was given. 

The single-grating stimulus consisted of a single 2 cycles/deg sinusoid drifting at 2 cycles/s in opposite directions in the two eyes. The relatively slow monocular speed (1 deg/s in each eye) corresponds to moderately brisk 3D motion, ranging from 35.2 (toward) to 57.2 (away) cm/s for purely horizontal monocular motions. From trial to trial, grating orientation (and hence, direction) was pseudorandomly selected from a set of directions spanning 0 to 360 deg in 15 deg increments, with 0 defined (by arbitrary convention) as horizontal rightward motion in the right eye (the left eye was always 180 deg opposite in orientation/direction). The data shown in Figures 1B and 5A were collected using a single 10% contrast grating. We also performed this experiment using a 20% contrast grating (thereby equating the net contrast energy to the 2-component plaids used in Experiments 2 and 3) and obtained identical results. 

Figure 1

 

Three-dimensional motion direction discrimination when viewing oppositely moving gratings in the two eyes. (A) Illustration of the 3D motion percept and dichoptic grating stimulus. All observers viewed gratings drifting in opposite directions in the two eyes (L, left eye; R, right eye). Expert observers performed a rating scale 3D direction discrimination task as grating orientation (and hence direction) was varied from trial to trial (“away” or “toward”, at either high, low, or zero confidence). Gratings always moved in opposite directions in the 2 eyes. By convention, we labeled the condition illustrated here (rightward horizontal motion in the right eye) as “0 deg.” Trials with other directions of motion were created by rotating both eyes' gratings in 15 deg increments. This maintained a matched stimulus orientation in both eyes on each trial, while also yielding 180 deg opposite directions of motion. (B) Three-dimensional motion direction discrimination resulting from viewing a grating moving in opposite direction in the two eyes. Y-axis plots the mean 3D motion rating (signed direction and confidence, in a signal detection rating scale task; see Figure 5B for illustration of ROC curves and d′ estimates from this rating scale data). X-axis denotes the direction of grating motion in the right eye (graphically indicated by the icons below the plot; left eye motion was always 180 deg opposite). The stimulus supported the clearest 3D direction discrimination (away or toward) when the grating drifted horizontally (i.e., at 0 and 180 deg, respectively) but not when it drifted vertically (90, 270 deg). Black curve indicates a sinusoidal fit to the data. This curve closely matches a reference unit-amplitude cosine function with a peak at 0 deg (gray line). Each data point (black dot) indicates the mean response combined across subjects. The sinusoidal relationship between monocular motion direction and 3D motion rating indicates that the 3D motion discrimination performance was proportional to the horizontal portion of the monocular motions. In this and all following figures, similar patterns were observed in each individual. Error bars depict 95% confidence intervals on each point and are sometimes smaller than the plotting symbols. The gray solid triangle above the upper x-axis indicates right eye motion direction for which the 3D motion direction discrimination performance (of “away” motion) was predicted to be highest based on the horizontal fraction of grating motion (0 deg). The open black triangle indicates the best fit peak from the data (2.2 deg).

In the plaid experiments, we characterized the 3D direction discrimination performance produced by plaids moving in opposite directions in the two eyes. First, we presented “Type I” plaids consisting of two superimposed drifting gratings in each eye (each with the same parameters described above), with orientations that differed by 120 deg (components oriented ±60 deg relative to the pattern motion direction; Figure 2A). The contrast values of the two component gratings were added to produce the plaid. The same range of directions and number of trials were employed as in the grating experiment (and in all following experiments). 

Figure 2

 

Three-dimensional motion sensitivity when viewing dichoptic plaids with opposite pattern motions in the two eyes. (A) Illustration of the Type I plaid stimulus. Observers viewed two superimposed gratings separated by 120 deg (black arrows) forming a Type I plaid (pattern motion direction, red arrows) moving in opposite direction in each eye (L, left; R, right). The condition illustrated here corresponds to the 0 deg condition in lower panels (i.e., rightward pattern motion in the right eye; left eye motions were always 180 deg opposite). (B) Three-dimensional motion direction discrimination performance when viewing a Type I plaid moving in opposite direction in the two eyes. Format similar to Figure 1B; 3D motion direction discrimination performance (y-axis) as a function of pattern motion direction in the right eye (x-axis). Three-dimensional motion performance (mean rating; as in Figure 1B) was highest when the pattern motion was horizontal (0, 180 deg) and very weak when either of the components moved horizontally (−60, 60, 120, 240 deg). Black curve indicates a best-fit sinusoid; gray curve indicates a reference unit-amplitude cosine function identical to Figure 1B. Icons below the plot indicate the directions of right eye motion. Symbols above the upper x-axis indicate right eye motion directions that would predict the highest 3D performance (for “away” motion), based on component motion (±60 deg; gray ticks), pattern motion (0 deg; red solid triangle), and best fit to the data (−0.5 deg; open black triangle), suggesting a dependence of the 3D percept on the plaid pattern motion direction. (C) Illustration of the Type II plaid stimulus. Observers viewed a Type II plaid moving in opposite direction in the two eyes. Component motions associated with the two superimposed gratings are indicated with black arrows; pattern motion of the plaid (by IOC) is indicated by the red arrows; vector average of the components is indicated by the blue arrows. The condition illustrated here corresponds to the “0 deg” point in the next panel (i.e., 0 deg average component direction in the right eye; left eye motions were always opposite). Temporal frequency of the faster component was twice that of the slower component. (D) Three-dimensional motion direction discrimination performance when viewing oppositely moving Type II plaids in the two eyes. X-axis indicates the average direction of the two components (i.e., 0 deg indicates the stimulus schematized in (A) above). Three-dimensional motion discrimination performance varied as a function of direction. The fitted peak of the perceptual tuning curve fell at 23.0 deg (black open triangle above plot; bounding 95% confidence interval, [20 25] deg), which is well past the vector average direction (5.1 deg; blue triangle), outside the range of directions spanned by the individual components (±15 deg; gray ticks), and shifted toward the IOC direction (51.2 deg; red solid triangle). Thus, the amount of shift cannot be explained by a mechanism based on individual component contributions (or on their vector average) alone and indicates a contribution of pattern motion. Icons below the plot depict the directions of right eye motion.

We also characterized 3D direction discrimination performance produced by “Type II” plaids (Figure 2C). We again presented two drifting gratings in each eye; this time, however, the components were separated by 30 deg (oriented ±15 deg) and the temporal frequency of one of the gratings was increased to 4 cycles/s. The two components thus had the same spatial frequencies but speeds that differed by a factor of 2. This yields a pattern motion distinct from the individual components, strongly biased toward the “intersection of constraints” (“IOC”) direction (Adelson & Movshon, 1982), in this case 51.2 deg away from the mean component direction. 

In the pseudoplaid experiments, we characterized 3D direction discrimination performance generated by spatially isolated component motions that specified opposite pattern motions in the two eyes. We constructed the stimulus by presenting 28 small non-overlapping drifting gratings within small stationary Gaussian apertures (standard deviation, 0.1 deg), creating Gabors that were considerably smaller than 1 deg in diameter (i.e., ±3 SDs corresponds to a 0.6 deg diameter). Gabors were presented at 20% Michelson contrast (nominal maximum at aperture center), 2 cycles/deg spatial frequency within a 500-ms raised cosine temporal aperture. We presented half (14) of the Gabors in one eye, and the other half in the other. All drifting Gabors were compatible with a single pattern motion direction in each eye, and the pattern motion was in opposite directions between the eyes. 

In the 2-component “pseudoplaid” condition (Figure 3A), each of the Gabors was randomly assigned one of two (±45 deg) orientations (akin to spatially sampling the two components of a 90 deg Type I plaid) and drifted at 2 cycles/s in opposite directions in the two eyes. 

Figure 3

 

Three-dimensional motion sensitivity when viewing dichoptic pseudoplaids with opposite pattern motions in the two eyes. (A) Illustration of the 2-component pseudoplaid stimulus. Observers viewed fields of spatially separated Gabor elements in the left (L) and right (R) eyes. In each eye's pseudoplaid, Gabors were oriented at 90 deg relative to one another (i.e., akin to a ±45 deg Type I plaid but with the components represented as spatially separated Gabor elements instead of overlapping gratings) and randomly distributed in space. Inset top left shows a magnified view of a single Gabor element. Critically, Gabors in the left eye (i.e., red circle) were separated by at least 1.4 deg (edge to edge) from any Gabors in the corresponding right eye's half-image (crossed red circle; see Methods section for more details). Gabors in the left eye's half-image drifted in opposite directions to those in the right eye (black arrows, only some arrows shown for clarity, in the actual stimulus all Gabors drifted within their stationary Gaussian envelope). The condition illustrated here corresponds to the 0 deg condition in lower panels (i.e., rightward global pattern motion (red arrow) in the right eye). (B) Velocity-space representation of the 2-component pseudoplaid stimulus. Velocity vectors for the left and right eyes' views of the 0 deg pseudoplaid stimulus. Horizontal axis, horizontal velocities; vertical axis, vertical velocities. Black arrows, component motions, representing the Gabor component arrows overlaid on the stimulus illustration shown in (A) in velocity space. Dashed lines depict 1D motion constraint lines; red arrows, global pattern motion. (C) Three-dimensional motion direction discrimination when viewing a 2-component pseudoplaid drifting in opposite direction in the two eyes. X-axis indicates the global pattern direction (i.e., 0 deg corresponds to the stimulus shown in (A) above). Three-dimensional motion direction discrimination performance varied as a function of global pattern direction, with a peak near 0 deg (95% confidence interval on fitted peak location, [0, 9] deg). Three-dimensional motion discrimination performance was low when either pseudoplaid component drifted horizontally (−45, 45, 135, 225 deg). Instead, performance peaked when the pattern motion was horizontal (0, 180 deg), demonstrating a primary dependence on interocular pattern motion and not local component motions. Icons below the plot indicate the directions of right eye motion. Symbols above the upper x-axis depict key elements: Right eye motion direction predicting highest performance (i.e., discrimination of “away” motion) based on either component motion (±45 deg; gray ticks) or pattern motion (0 deg; red solid triangle); best fit to the data (4.6 deg; open black triangle). (D) Illustration of the multi-component pseudoplaid stimulus. Observers viewed a stimulus identical to the one shown in (A), except that the orientations of all Gabors were fully randomized (i.e., uniformly distributed throughout all possible orientations) while their individual speeds were tailored to be consistent with a single pattern motion velocity (by IOC). We call this version of the dichoptic pseudoplaid stimulus “multi-component” simply because it contains multiple compatible component motions. Stimulus depicted here corresponds to the 0 deg condition (i.e., rightward pattern motion in the right eye; left eye was always opposite). For clarity, motions of only some of the Gabors are indicated (black arrows); all elements drifted in the actual stimulus. (E) Velocity-space representation of the multi-component pseudoplaid stimulus. Similar format to (B). Black arrows indicate various component motions, corresponding to a range of orientations as depicted in (D). In this figure, only some of the arrows are shown for clarity, the actual stimulus specified 14 component motions in each eye, drawn randomly from a uniform distribution. Dashed lines depict 1D component motion constraint lines. Red arrow indicates global pattern motion as obtained by intersection of constraints. To be consistent with a single pattern motion, all component velocities produced by the randomly oriented Gabors have to fall on a circle in velocity space (gray circle). (F) Three-dimensional motion direction discrimination when viewing the multi-component pseudoplaid stimulus. Format similar to (C). X-axis depicts the global pattern motion direction (i.e., 0 deg corresponds to the stimulus shown in (A) above). Performance (y-axis) varied as a function of global pattern direction, with a peak near 0 deg (95% confidence interval on fitted peak location, [−9, 0] deg). Icons below the plot depict the directions of right eye motion. Symbols above the upper x-axis indicate key elements: Right eye direction predicting highest performance (i.e., discrimination of “away” motion) based on either component motion (continuous gray band) or pattern motion (0 deg; red solid triangle); best fit to the data (−3.5 deg; open black triangle).

In the multi-component “pseudoplaid” condition (Figure 3C), we oriented the gratings in each aperture randomly, but all component velocities were compatible with a single (IOC) velocity (this is akin to spatially sampling a complex 2D object with multiple orientations, moving in a single direction). Temporal frequency was 2sinθ cycles/s, where θ is the angle of the grating relative to the pattern motion direction, so that when the Gabor carrier orientation was exactly perpendicular to the pattern motion direction, the temporal frequency was at its maximum of 2 cycles/s. 

Strict constraints were placed on location of individual Gabors within these pseudoplaid stimuli in order to preclude possible binocular (or pattern motion) integration at the scale of conventional V1 mechanisms (Van Essen, Newsome, & Maunsell, 1984). The center of each drifting Gabor was separated by at least 2 deg from the center of any other Gabors in either the contralateral or ipsilateral eye (i.e., throughout the cyclopean view). In combination with the 0.1 deg SD of the Gaussian envelope of each Gabor, and the 8-bit resolution of our displays, this amounted to ∼1.4 deg edge-to-edge dichoptic and monocular spacing, as the rendered contrast of each Gabor was effectively zero at ±3 SDs (i.e., ±0.3 deg). 

Although the 2AFC task is in very common use, we originally used the 5-point rating scale task because it allowed us to see systematic effects of direction with stimulus strengths well above conventional psychophysical threshold (e.g., d′ ≫ 2). Likewise, we had originally focused on expert observers because of the large number of experimental conditions, coupled with our desire to make rather fine-grained quantitative inferences about the location of peaks. To assess performance in naive observers and using a 2AFC task, we performed additional measurements with two issues in mind. First, we expected that the naive (non-expert) observers might be generally less sensitive (and perhaps more variable) than the expert observers: we therefore collected data from one expert observer as a reference. Second, we needed to modify some stimulus parameters to bring 2AFC performance closer to threshold (thereby avoiding the ceiling effects that may be bypassed in the rating scale task): thus for conventional grating and plaid stimuli (e.g., gratings, type I plaids, and type II plaids), presentation time was reduced to 250 ms. Accordingly, temporal frequency was doubled (to 4 cycles/s from 2 cycles/s) to maintain whole temporal cycles during presentation. 

In each experimental condition, all drifting components could thus start and end every trial at a half-cycle of binocular phase disparity, drifting for exactly one or two cycles during each trial. Each trial thus began and ended at an ambiguous disparity (owing to the periodic nature of the stimuli), which could be perceived as either near or far. Instantaneous starting, ending, as well as average disparities therefore could not serve as potential cues for task performance. 

Procedure and task

The stereoscope was initially adjusted so that the vergence demand was appropriate for the viewing distance given a typical interocular distance. Prior to each session, each observer made further minor adjustments so that the nonius markers were aligned both horizontally and vertically, and vergence was comfortable. Observers were instructed to maintain fixation for the duration of each experimental session, and the four expert observers were all experienced in monitoring vergence during 3D motion psychophysics, minimizing the possibility of unintended binocular matches. 

Before participating in the experiments, the naive observers were introduced to demonstration versions of the stimuli and task, viewing them using active stereo shutter glasses (NVIDIA) and a 120-Hz DLP projector (DepthQ). The observers were instructed to try to discriminate the global 3D direction (toward/away) of pseudoplaid stimuli, ignoring the frontoparallel motions of the individual Gabor elements. After these preliminary demonstrations, the naive observers were then assisted in adjusting the stereoscope in the main apparatus, and they did not report any discomfort or problems with fusion of the surround. 

For the expert observers, on each trial, the stimulus was presented for a single 500-ms interval, and the observer responded via a key press. For each of the 4 observers, 30 repetitions of each of 24 directions were pseudorandomly distributed across 1 or 2 runs. Observers performed a signal detection 5-point confidence rating direction discrimination task (toward or away; the 5 potential responses corresponded to high confidence away, low confidence away, neutral/ambiguous, low confidence toward, high confidence toward). This can also be thought of as a 2AFC task, where the response on each trial is assigned one of three confidence levels (high, low, totally ambiguous). The main advantage of this technique is that it simultaneously captures multiple criteria and can thus be used to quickly generate an ROC curve (see Figure 5B), hence separating bias and sensitivity. 

We acquired similar results with 3 naive observers (and one of the authors) using a simpler 2-alternative forced-choice (2AFC) paradigm, using stimuli closer to 3D motion detection threshold. The temporal parameters were changed to a shorter 250-ms presentation at a faster 4 cycles/s. Some ceiling effects are evident for the experienced observer in the 2AFC data, but even with saturated accuracy levels that distort the sinusoidal dependence on direction, the location of the peaks of his curve were still consistent with all the other results. These ceiling effects were not a factor for the expert observers, since they reported motion direction and associated confidence on a 5-point scale, rather than making a 2AFC. Indeed, we incorporated confidence ratings to avoid such ceiling effects for the most effective stimulus conditions (Figure 5B demonstrates that the results from the signal detection confidence rating task continued to follow systematic modulations for d′ values well above 3). We note that both 2AFC and multiple-level rating scale tasks can be analyzed to extract standard quantities like receiver operating characteristic (ROC) curves and d′ (Green & Swets, 1966; Macmillan & Creelman, 2005). 

Data analysis

For the expert observers, we encoded each observer's 5-point responses on a linear rating scale from 1 (away, high confidence) to −1 (toward, high confidence); intermediate responses of 0.5 indicated low confidence away and −0.5 indicated low confidence toward, and a 0 rating indicated zero toward/away confidence. We plotted the mean rating as a function of direction (defining horizontal right-eye motion as “0 deg”). All statistics were estimated using a bootstrap procedure, resampling the data from each run and from each subject 1000 times; this propagates inter-run and inter-subject variability into all estimates of variance. Error bars on each data point represent 95% confidence intervals, equivalent to ±2 standard errors of the mean. These 3D motion perceptual tuning curves are shown in Figures 1B, 2B, 2D, 3C, and 3F. (We also derive ROC curves and d′ for these data, as shown in Figure 5B.) 

To quantify the dependence of perceived 3D motion on direction, we fit the perceptual tuning curves (i.e., mean rating as a function of direction) with a sinusoidal function (with amplitude and phase as free parameters), minimizing RMS error (black curves, Figures 1B, 2B, 2D, 3C, and 3F). The fitted amplitude specifies the height at the peak of the curve, and the fitted (cosine) phase indicates the location of the peak of the curve. We estimated 95% confidence intervals on these fitted parameters using a bootstrap (1000 iterations). We then compared the phase confidence interval (i.e., the confidence interval about the estimated location of the peak of the curve) with relevant component and pattern motion direction predictions for each experiment. For example, we could test whether the confidence interval about the fitted peak contained the expected peak location based on horizontal pattern motions, as well as whether it excluded the predicted peak based on alternatives (such as the vector average of the component motions). 

For the naive observers who performed a 2AFC direction discrimination task (toward versus away), we simply measured the proportion of “away” judgments as a function of direction, and then performed the same analyses as for the rating scale data. 

As expected, the naive observers were not as skilled in adjusting the mirror stereoscope to be in perfect alignment (as compared to the expert observers). Although they confirmed good binocular fusion and maintenance of vergence, it was likely that slight optical misalignments (i.e., mixtures of pan and tilt) produced images that were slightly rotated relative to ground truth (i.e., the “0 deg” condition was actually not perfectly vertical). We estimated this misalignment for each observer using their responses in the drifting grating experiment. Using the peak of this baseline condition as the corrected “0 deg,” we then subtracted this value to shift the curves in subsequent conditions. This correction was only on the order of a few degrees at most, and we note that it does not affect the pattern of results across conditions. 

Rating scale tasks, such as the one used in our expert observer experiments, are well suited to conventional signal detection receiver operatic characteristic analysis. For the ROC analysis (Figure 5B), we used standard n-afc rating scale procedures (Green & Swets, 1966). Each response boundary (e.g., “<0” vs. “≥0”) reflects a criterion, thus yielding 4 criteria from our 5-point scale. We then tallied the proportion of hits and false alarms under a specific criterion for a given absolute stimulus direction (which was hypothesized to correspond to horizontal IOVD signal strength). This resulted in 4 data points per ROC curve. The results of this analysis are shown in Figure 5B for the single-grating data to illustrate the utility of the rating scale method and its relationship to 2AFC. 

In an additional control experiment, we verified that expert observers did not rely on joint information on eye of origin of stimulus elements and monocular direction of motion. The observers performed a 2AFC direction discrimination task on a dichoptic multiple-grating stimulus. This stimulus was similar to a dichoptic pseudoplaid, except all stimulus elements had the same orientation within each monocular view (as before, they moved in opposite directions in the two eyes; Figure 4A). Observers judged 90 trials each in two motion conditions (vertical motion or horizontal motion). For the vertical motion, observers judged whether the motion in the right eye was upward or downward (as an assay of their ability to perform the posited joint eye-of-origin/direction-of-motion task); for the horizontal motion, they judged whether the 3D motion was “toward” or “away” (as an assay of their ability to discriminate 3D motion direction). We computed the proportion of times observers reported the stimulus as moving away (horizontal condition) or upward in the right eye (for the vertical condition). 

Figure 4

 

Control experiment to rule out possible joint eye-of-origin and monocular direction-of-motion discrimination. (A) Stimuli used to rule out joint eye-of-origin/direction-of-motion discrimination. All expert observers participated in a control experiment to address the concern that performance in the 3D motion direction discrimination task could have resulted from observers discriminating the direction of motion in one eye and mapping that to the correct 3D direction response (despite never receiving feedback). The 4 expert observers performed a 2AFC direction discrimination task on a dichoptic multiple-grating stimulus. This stimulus was similar to a dichoptic pseudoplaid, except all stimulus elements had the same orientation within each monocular view (as before, they moved in opposite directions in the two eyes). For the vertical motion (left panel), observers judged whether the motion in the right eye was upward or downward (as an assay of their ability to perform the posited joint eye-of-origin/direction-of-motion task); for the horizontal motion (right panel), they judged whether the 3D motion was “toward” or “away” (as an assay of their ability to discriminate 3D motion direction). (B) Proportion correct for vertical and horizontal motion discrimination tasks. As described in (A), motion of all stimulus elements was either vertical (left group of 4 points; each observer is a separate plotting symbol) or horizontal (right group). Symbols plot the average proportion correct from 90 trials per direction (error bars showing 95% confidence intervals). For all observers, accuracy was nearly perfect for the horizontal (3D direction) condition but close to chance for the vertical (joint eye-of-origin/direction-of-motion) condition. Observers could have used the component and/or pattern motion in this stimulus to perform the joint eye-of-origin/direction-of-motion task, but the data show little support for the use of either. These results are consistent with the modulation in performance as a function of direction in the main experiments. Most importantly, the far higher levels of 3D direction discrimination accuracy show that any sort of alternative strategy based on 2D (monocular) direction discrimination supported by simultaneous eye-of-origin discrimination is unlikely to account for our main findings.

Results

We performed a series of experiments in which observers judged 3D motion direction as a function of the monocular directions of either component (1D) or pattern (2D) motions. The initial baseline experiment, using a single 1D grating in each eye, serves to explain the general methods and logic of the approach. The following experiment, using a Type I plaid in each eye, extends this logic to 2D dichoptic motions. Then, the remaining experiments (Type II plaids and dichoptic pseudoplaids) represent the key results that characterize the eye-specific motion signals used to estimate 3D motion. 

Baseline experiment: Dichoptic gratings

We first characterized 3D motion direction discrimination performance as observers viewed a single grating that drifted in opposite directions in the two eyes (Figure 1A). These single-grating data served as a baseline for the rest of the experiments that employed multiple component motions in each eye. We varied grating orientation from trial to trial, and on each trial, the expert observer judged the direction of perceived 3D motion (indicating either motion toward or away, at one of three levels of confidence in a signal detection rating scale task; Figure 1B; throughout the following experiments, we plot the mean rating and refer to this quantity as “direction discrimination performance”). On each trial, the right and left eyes' gratings always moved in the polar opposite direction from one another (i.e., 180 deg apart). 

Three-dimensional motion direction discrimination performance was highest (i.e., correctly judged as “away” or “toward” with the highest levels of confidence) when the grating drifted horizontally (0, 180 deg), intermediate at intermediate directions, and observers were unable to reliably discriminate 3D motion direction when the grating's motion was completely vertical (90, 270 deg). This perceptual tuning curve was well characterized by a sinusoidal function, which we use throughout the following experiments as a descriptive fit that allows us to estimate the location of the perceptual sensitivity peak with respect to monocular direction. In this experiment with a single grating in each eye, the 95% confidence interval on the location of this peak spanned [0, 3] deg. The apparent cosine phase suggests that 3D motion sensitivity was proportional to the horizontal portion of the 2D grating motion vectors. 

Dichoptic type I plaid experiment: Introducing the idea of eye-specific pattern motions

We then measured the dependence of 3D motion direction discrimination as observers viewed conventional drifting plaids in each eye. In the Type I plaid condition, both eyes' plaids were composed of two superimposed gratings rotated by ±60 deg (Figure 2A). Figure 2B shows observers' 3D motion direction discrimination, plotted as a function of the plaid pattern motion direction (in the right eye). Direction discrimination performance peaked when the pattern direction was horizontal (0 and 180 deg; [−2, 1] deg, 95% CI on peak location)—such that component grating directions were at ±60 deg from horizontal. In contrast, when either one of the component grating motions was horizontal (i.e., when the pattern motion was at −60, 60, 120, or 240 deg), direction discrimination performance was low. The fact that the perceptual tuning curve was unimodal, instead of containing peaks at ±60 deg, indicates that discrimination performance did not derive from a winner-take-all mechanism that operated solely upon the most horizontal component motion. Of course, the sum of two sinusoid response curves with phases at ±60 deg is simply a sinusoid of the same frequency, with a phase at 0 deg. Thus, a mechanism that perfectly summed the horizontal components of the disparities—or the velocities for that matter—would produce sinusoidal response curves centered at 0 deg. The additional experiments that follow rule out a wide set of disparity- and component motion-based accounts of these initial results. 

Dichoptic type II plaid experiment: A first test for a unique contribution of eye-specific pattern motions

To more directly pit pattern motion against component motion (and corresponding temporally changing horizontal disparities), we presented drifting Type II plaids (Adelson & Movshon, 1982) in each eye (Figure 2C). In these plaids, the pattern motion falls outside the range of the two component motions and is not the simple result of their vector average (or sum). We generated Type II plaids by superimposing a pair of component gratings with a small angle between them (±15 deg), moving at different velocities (1 and 2 deg/s). Under our viewing conditions (and consistent with prior work; Adelson & Movshon, 1982; Welch, 1989; Yo & Wilson, 1992), the resulting pattern motion direction in each eye was well approximated by the “intersection of constraints” (IOC) direction—the single 2D velocity geometrically consistent with both component motions. 

Figure 2D shows 3D motion direction discrimination performance, plotted as a function of the mean component motion direction (right eye; left eye motions were 180 deg opposite). If 3D motion direction discrimination depended solely on the interocular comparison of the vector average (or sum) of component motion signals, the peaks of the tuning curve would fall near 0 deg (VA, blue triangle; 5.1 deg) and certainly within the ±15 deg range of directions spanned by the component motions (vertical gray tick marks). Instead, the peak of the curve (open black triangle; [20, 25] deg, 95% CI) was considerably shifted toward the IOC direction (red triangle; 51.2 deg), reliably outside the component motion range (±15 deg). Because the shift toward IOC was significant but not complete, it is certainly possible that 3D motion percepts reflected a mixture of 2D pattern and 1D component motions. 

So far, the patterns of 3D motion direction discrimination resulting from dichoptic Type I and Type II plaids suggest a parsimonious explanation based on a mechanism that computes IOVDs using eye-specific pattern motions: the experiment that follows introduces a less conventional stimulus that compellingly supports this explanation. 

Although we observed reliable direction discrimination performance in the absence of feedback (consistent with observers' subjective reports of solid cyclopean percepts for horizontal and near-horizontal retinal motions), binocular rivalry can certainly occur in dichoptic viewing (Liu, Tyler, & Schor, 1992) and might be related to the weaker direction discrimination performance we measured as monocular motions approached vertical. 

Although these results require some sort of 2D computation prior to the 3D motion mechanism, there is still the possibility that some of these results depend upon 2D disparity matches, rather than 2D velocities. Some evidence exists for two-dimensional binocular matching, instead of simple 1D horizontal disparity extraction (Farell, Chai, & Fernandez, 2009). Although the primacy and relevance of such 2D mechanisms is not yet clear, in principle such 2D disparity processing could complicate interpretations based on (the geometrically equivalent) interocular pattern motions. We therefore sought to design a stimulus that retained opposite pattern motions in the two eyes but that avoided binocular overlap of constituent elements to sidestep rivalry and/or 2D disparity processing. 

Dichoptic pseudoplaid experiments: Evidence for eye-specific pattern motions

To perform a more stringent test for IOVDs based on eye-specific pattern motions, we capitalized on the fact that the spatial scale of extrastriate pattern motion mechanisms in primates is significantly larger than that of component motion mechanisms in V1 (Movshon et al., 1985; Van Essen et al., 1984). The resulting “dichoptic pseudoplaid” stimulus—shown in Figure 3A—contains opposite pattern motion signals in the two eyes that exceeded the spatial scale of component motion mechanisms. Scattered Gabor elements (i.e., individual, drifting sinusoidal gratings within small, stationary Gaussian apertures; Amano, Edwards, Badcock, & Nishida, 2009; Majaj, Carandini, & Movshon, 2007) were arranged to produce a global pattern motion signal in each eye that could only be recovered by integration over multiple elements across space (Clark & Bradley, 2008; Watamaniuk & Sekuler, 1992; Watamaniuk, Sekuler, & Williams, 1989). Crucially, the elements in one eye's view were not matched binocularly—a Gabor in one eye was always farther than 1.4 deg (edge to edge, where “edge” indicates the location at which our Gabors fell to the background luminance) from any other Gabor (in either the same or the other eye). This spacing ensured that no significant processing beyond the extraction of monocular component motion could occur in single units within primary visual cortex given known measures of primate V1 receptive fields at these eccentricities (Van Essen et al., 1984). Similar to the conventional plaid stimuli, each eye's stimulus contained 2 distinct (±45 deg) component motions, producing a pattern motion different from that of the individual Gabor elements (schematized in Figure 3B). In addition, as in the earlier grating and plaid experiments, pattern motion direction was always 180 deg opposite in the two eyes. 

Observers again discriminated 3D motion direction as we varied the pattern motion direction (Figure 3C). Three-dimensional motion direction discrimination performance was highest when the global pattern motion in each eye was horizontal ([0, 9] deg, 95% CI on peak location). When pattern motion in each eye was vertical, observers were no longer able to discriminate 3D motion direction. Not surprisingly, discrimination performance was reduced relative to those observed using the binocularly paired conventional plaids, given that this dichoptic pseudoplaid stimulus consisted only of a small number of sparse, monocularly visible Gabor elements. What is surprising is that observers perceived 3D motion at all—instead of a constellation of randomly oriented monocular (or rivalrous) Gabors. This demonstrates that an interocular comparison of pattern motions (in the absence of conventional binocular disparities) is sufficient to yield percepts of 3D motion. 

The shape of the perceptual tuning curve argues against the observers' reliance on unintended binocular overlap of elements (perhaps due to vergence eye movements or unexpectedly large but finely tuned orientation receptive fields). Had this been the case, percepts of 3D motion should be strongest when one of the components moved horizontally—but performance under such conditions (±45 deg) was in fact relatively low (Figure 3C). Moreover, the stimulus had a 1/f central and peripheral textured background, which provided a powerful anchor for vergence across a range of visual scales. 

As a final challenge to interocular pattern motion mechanisms, we further generalized the stimulus so that each drifting Gabor had an independent, random orientation uniformly selected over all 180 deg of possible orientations (Figure 3D). We then assigned each Gabor a velocity consistent with a single, global, pattern motion velocity (Amano et al., 2009; Majaj et al., 2007)—opposite in the two eyes (Figure 3E). This “multi-component pseudoplaid” simulates a single moving object with different dominant orientations at different spatial locations (such as a zebra), viewed dichoptically through apertures that differentially occlude the 2 eyes' views (such as foliage close to one's face). As in the previous experiment, 3D motion direction discrimination performance peaked when the pattern motions were horizontal and opposite (Figure 3F; [−9, 0] deg, 95% CI), suggesting that the randomly oriented component motion signals are combined into a single pattern motion signal for each eye, prior to the extraction of 3D motion. As with the conventional gratings and plaids, the patterns of results with dichoptic pseudoplaids were confirmed in naive subjects and using a 2AFC direction discrimination task (Figures 6C and 6D). 

Control experiment: Ruling out joint eye-of-origin and monocular direction-of-motion discrimination

We conducted a control experiment to explicitly test whether observers might have performed the 3D motion task by covertly performing a joint eye-of-origin and direction-of-motion discrimination—essentially knowing which eye saw what direction of 2D motion, and then mapping that correctly on to a 3D direction response despite the absence of any feedback (see Figure 4A). In separate runs, we had the four expert observers view the multi-component dichoptic pseudoplaids when all monocular motions were either purely vertical (and opposite in the two eyes) or horizontal (and opposite in the two eyes). When the motions were vertical, observers attempted to perform a joint eye-of-origin and direction-of-motion task (i.e., “which eye saw upward motion?”). When the motions were horizontal, observers performed a standard 2AFC 3D motion direction discrimination (“toward or away?”). Accuracy on the vertical task tested observers' ability to jointly discriminate monocular direction and eye of origin—and performance in all observers was close to (and often statistically indistinguishable from) chance. Accuracy on the horizontal task was near perfect—far higher than would be predicted from the vertical eye-specific direction-of-motion performance—leaving little possibility that observers based their responses on anything other than the perceived direction of 3D motion. If observers had relied on a cognitive strategy or had simply closed one eye, they should have performed equally well in both conditions. Although one logical possibility remains—that observers can perform such a joint eye-of-origin and direction-of-motion task for horizontal motions (such as in the main experiments) but not for vertical motions (as in this control)—we think this is not likely enough to warrant further consideration unless future data support such an odd proposition. 

Generalization of effects to naive observers and to a different task

We also confirmed that our key results (i.e., a dependence of 3D motion judgments parsimoniously explained by interocular comparisons of eye-specific pattern motions) could also be obtained from non-expert, naive subjects and in a simpler two-alternative forced-choice (2AFC) task. We therefore repeated the full set of dichoptic conditions (conventional gratings, Type I plaids, and Type II plaids, as well as both the two-component and multiple-component pseudoplaids) in 3 naive observers (one was an experienced psychophysical observer; the other two had no prior experience doing any sort of visual psychophysics) as they performed a 2AFC 3D motion direction discrimination task (indicating simply whether the stimulus appeared to move toward or away). 

We observed a similar pattern of results in these 2AFC data collected in naive observers. The results of the conventional grating calibration experiment are shown in Figure 5A. All observers showed a sinusoidal modulation of direction discrimination accuracy as a function of monocular direction, with a peak near horizontal (95% confidence interval, [−2, 3] deg). Figure 5B is an ROC plot that relates these data to those from the corresponding rating scale task (refer back to Figure 1B). This plot shows that sensitivity in the 2AFC task (individual square points) was highest for horizontal grating motions (darkest squares, d′ ∼ 2) and fell off for increasingly vertical grating motions (lighter squares, approaching d′ of zero). The rating scale data are represented by connected curves and follow the same dependence on grating direction (higher for horizontal, lower for vertical), although the overall range of sensitivity spanned is wider and extends far above threshold (horizontal d′ > 5). 

Figure 5

 

Naive observer results from a 2AFC task when viewing dichoptic gratings and their relationship to the experienced observer rating scale results in ROC space. (A) Three-dimensional direction discrimination accuracy in a 2AFC task, resulting from viewing a grating drifting in opposite direction in the two eyes for 3 naive and 1 expert observers. X-axis shows motion direction in the right eye (as in Figure 1). Y-axis shows proportion of trials judged as moving “away” (the other trials were thus judged as “toward”); perfect performance would correspond to all trials between −90 and 90 judged as “away”, and no trials between 90 and 270 judged as “away” (i.e., all trials judged as “toward”). Gray points show individual observer data. Error bars indicate the 95% confidence interval (data point darkness reflects amount of previous psychophysical experience). Darkest points indicating the one expert/author observer, and lightest points indicating an observer with no previous psychophysical experience. Black solid line reflects best cosine fit to the data from the three naive observers, i.e., not including the data from the expert/author observer. For reference, the gray solid line depicts the unit cosine function. The symbols above the upper x-axis indicate right eye motion direction for which the proportion of 3D “away” judgments would be largest if dependent on the horizontal aspect of the component motion (0 deg; gray solid triangle), and the best fit peak from the data (0 deg, 95% confidence interval, [−2, 3] deg; open black triangle—in this case, superimposed on gray triangle), replicating the pattern of results reported for the experienced observers. (B) Single grating ROC curves for the 4 expert observers (derived from their rating scale data) and 2AFC data from 3 naive observers. The abscissas and ordinates give the z-scores corresponding to the proportion of false alarms and hits, respectively (converting to z-scores linearizes the ROC curves assuming equal variance Gaussian noise on the decision axis). The use of a rating scale in expert observers allowed us to obtain reliable responses across a wide range of sensitivities (d′ ranges from 0 to >5), while avoiding ceiling and floor effects. Contours of equal d′ are given by the grayscale transitions. Circles connected by dashed lines show the averages across the expert observers calculated from the rating scale data, with darker symbols denoting stimulus motion closer to horizontal (“Absolute motion direction” in legend indicates absolute angular deviation of monocular motion direction from horizontal). Estimated d′ and area under the curve both change systematically with motion direction. Squares show the corresponding 2AFC data for the 3 naive observers in the same ROC space. All three were close to chance for vertical motion and achieved a d′ greater than 2 for pure horizontal motion, with monotonically increasing performance in between.

The results from the other (plaid and pseudoplaid) experiments are shown in Figure 6. Across all conditions, 3D motion direction discrimination sensitivity was highest when the pattern motion in each eye was horizontal (just as in the rating scale data). This was true for both the conventional Type I and Type II plaids (Figures 6A and 6B, respectively; 95% confidence intervals on the peak were [−3, 4] and [12, 20] deg) as well as for the two-component and multiple-component pseudoplaids (Figures 6C and 6D; 95% confidence intervals were [−1, 8] and [−7, 7] deg). All of these 2AFC data can be explained by an interocular velocity difference mechanism that operates upon pattern motions for each eye, just as in the rating scale experiments. 

Figure 6

 

Naive observer 2AFC results for plaids and pseudoplaids. (A) Three-dimensional direction discrimination accuracy resulting from viewing oppositely moving Type I plaids in the two eyes for 3 naive (and 1 expert) observers. Formatting as for Figures 5A, 2, and 3. Symbols above the upper x-axis indicate predicted location of peak discrimination accuracy, based on either component motion (±60 deg; gray ticks) or pattern motion (0 deg; red solid triangle). Best fit to the data (0.1 deg, 95% confidence interval, [−3, 4] deg; open black triangle) indicates a dependence of accuracy on the plaid pattern motion direction. (B) Three-dimensional direction discrimination accuracy resulting from viewing oppositely moving Type II plaids in the two eyes for 3 naive (and 1 expert) observers. Formatting as above. Symbols above the upper x-axis indicate predicted location of peak discrimination accuracy, based on either component motion (vector average 5.1 deg; blue solid triangle) or pattern motion (intersection of constraints (IOC) 51.2 deg; red solid triangle). The fitted peak of the perceptual tuning curve fell at 16.1 deg (black open triangle above plot; bounding 95% confidence interval, [12, 20] deg). (C) Three-dimensional direction discrimination accuracy resulting from viewing the 2-component pseudoplaid stimulus for 3 naive (and 1 expert) observers. Formatting as above. Three-dimensional motion direction discrimination accuracy depended on global pattern motion direction, even when individual stimulus elements were spaced so that the classical receptive field of a V1 only received input from one of the eyes. Peak accuracy at 1.1 deg (95% confidence interval on fitted peak location, [−1, 8] deg). (D) Three-dimensional direction discrimination accuracy resulting from viewing the multi-component pseudoplaid stimulus for 3 naive (and 1 expert) observers. Formatting as above. Three-dimensional motion direction discrimination accuracy depended on global pattern direction, even when individual stimulus elements were randomly oriented, with the constraint that each individual element's motion was compatible with a single global pattern motion direction (otherwise identical to the 2-component pseudoplaid). Peak at 0.3 deg (95% confidence interval on fitted peak location, [−7, 7] deg).

The only quantitative difference between the original rating scale results and the these results is that the 2AFC confidence interval for Type II plaids ([12, 20] deg) excludes the predicted peak based on mean component direction (0 deg) or vector average (5.1 deg) but does not exclude one of the component motion vectors (15 deg). The rating scale confidence interval for expert observers did not contain either component motion vector. As for expert observers, the subsequent results for the naive observers with pseudoplaid stimuli can be used to establish the contribution of eye-specific pattern motion signals. Thus, these results overall further generalize the applicability of eye-specific pattern motions to a wider range of 3D motion phenomena and might support the relative utility of using rating scale tasks rooted in signal detection theory. 

One final anecdote was also noted by the experimenters. Although all the expert/author observers reported rich 3D motion phenomenology when viewing the dichoptic pseudoplaids (as well as the other stimuli), the naive observers expressed varying strengths of subjective percepts—they initially claimed not to be able to see 3D motion in the more complex conditions (such as the pseudoplaids). One naive observer quickly came to report subjective experiences of 3D motion, one admitted to gaining some sort of an appreciation over time, and the other remained dubious to the end. Yet behavioral data from each of the 3 naive observers clearly demonstrate that they were able to discriminate 3D motion direction with good sensitivity (peaking at about d′ = 2), despite an apparent dissociation from their phenomenology. Although this issue deserves more rigorous investigation, we note at this point that the somewhat mercurial perceptual experiences generated by the dichoptic pseudoplaids may hint at mechanisms that do not necessarily support strong perceptual experiences (at least without practice) but may still guide action effectively. Furthermore, rich subjective experiences of 3D motion from animated demonstrations of these stimuli may require patience from some observers. There may also be significant individual differences in the degree of reliance on the IOVD cue (Harris, Nefs, & O'Hare, 2010). Movie 1 contains a demonstration of the pseudoplaid stimulus (along with an interpretive audio track). 

 

Movie 1

 

Demonstration of the pseudoplaid stimulus. Through either cross- or free-fusing one half of the image can be presented to one eye, and the other half presented to the other eye. Use your hands to block the undesired other half of the image in each eye for a cleaner percept. The movie presents a sequence of conditions building up to the novel pseudoplaid stimulus, which produces a percept of motion toward or away from the observer in the absence of binocular matches and thus in the absence of binocular disparity. Refer to the audio track within the movie for more details.

Discussion

Our results motivate serious consideration of the proposition that 3D motion is computed by extracting the velocity difference between the pattern motion seen by each eye. Such a mechanism has some intriguing corollaries. First, interocular pattern motion differences per se support percepts of 3D motion: changing retinal disparities, as traditionally conceived, may be sufficient to yield percepts of 3D motion but are not necessary. Second, the binocular comparison (or “matching”) that underlies the computation of these pattern-based interocular velocity differences occurs at a spatial scale much larger than that of a classical V1 receptive field. Finally, these large-scale pattern motions must also be “monocular” in the sense that a binocular comparison of them must be appropriately signed for the perception of “toward” versus “away” motion through depth. 

Whenever observers dichoptically view complex spatiotemporal patterns, it is necessary to consider multiple potential sources of binocular and monocular information in interpreting the resulting perceptual sensitivities. The constellation of these results, most strongly demonstrated using the pseudoplaid stimuli, rules out explanations based on mechanisms other than pattern-motion-based IOVDs. First, one might wonder whether our results could be explained simply by sensitivity to changes in horizontal disparity. It is of course true that a horizontal disparity-based mechanism could explain the cosine dependence of discrimination performance in our preliminary grating experiment and also, perhaps, in the Type I experiment. However, such a mechanism would not explain the fact that Type II plaid sensitivity peaked outside the range of directions spanned by the individual components. Furthermore, conventional binocular disparities are extracted on the scale of V1 receptive fields and thus could not be used to account for the results of our dichoptic pseudoplaid experiments. 

Second, psychophysical and physiological experiments have revealed sensitivity to disparities that are larger than those processed in V1 (Schor, Edwards, & Pope, 1998; Takemura, Inoue, Kawano, Quaia, & Miles, 2001). However, sensitivities to these large disparities have been shown to primarily reflect matching of temporally abrupt contrast envelopes, with little or no contributions of the orientation-specific information within the envelopes (Wilcox & Allison, 2009). In contrast, our pseudoplaid Gabor elements were presented with gradual temporal onsets and within stationary envelopes. Because all 3D motion information was conveyed by the orientation and direction content within these envelopes, and not by the envelopes themselves, it is unlikely that even unconventionally large disparity mechanisms could explain our dichoptic pseudoplaid results. In particular, there is no way to explain the dependence on the direction of the pattern motion by appealing to disparities among the envelopes. The pseudoplaid data could be explained by suggesting a mechanism that compares the horizontal components of the velocities in each eye, but such a mechanism would be incompatible with the data from the preceding (conventional) Type II plaid experiment. 

Finally, there is some evidence suggesting that binocular disparities can be extracted by comparing 2D spatial (mis)matches between the two eyes, instead of simply extracting the pure 1D horizontal component. Although such a mechanism could support the perceptual sensitivity we observed in our first set of grating and conventional plaid experiments, there were no systematic 2D matches between the left and right eyes' views in the dichoptic pseudoplaid experiments. In short, the dichoptic pseudoplaid stimulus provides such powerful evidence for an IOVD computation based on pattern motions because no known or posited disparity mechanism appears capable of extracting anything systematic from it. Furthermore, observers did not receive feedback, mitigating concerns that they may have arbitrarily mapped a particular eye's direction to a particular 3D motion response (a control experiment further weighs against this possibility, Figure 5B). Finally, we ensured that the starting, ending, and average disparities in our grating and plaid experiments were uninformative with respect to 3D direction, mitigating concerns that observers could have used a trivial static disparity cue. 

One brief prior study investigated 3D motion percepts when viewing dichoptic gratings and Type I plaids (Wright & Gurney, 1992). Consistent with the results in our grating and conventional plaid experiments, they reported that 3D speed matches for dichoptic gratings followed the horizontal component of the velocity, and that stable motion-in-depth percepts were generated by dichoptic Type I plaids, even when component grating orientations were close to vertical. They interpret their results as consistent with dependence on the pattern motion, and we agree with this interpretation. They also reported increasing perceived speed with increasingly near-vertical motion, while we report decreasing 3D motion discrimination performance as monocular motions approached vertical. These results are not inconsistent. Although we have not directly measured motion through depth discrimination as a function of pattern speed, we note that robust percepts of 3D motion occur at relatively slow pattern velocities—consistent with the binocular viewing geometry (wherein very fast environmental speeds in depth actually give rise to fairly slow projected speeds on the two retinae). 

The notion of eye-specific pattern motion processing within large receptive fields has both computational and ecological appeals. Computationally, our results demonstrate that 3D motion signals are built directly from 2D pattern motion signals (and not from a larger number of ambiguous 1D signals). Ecologically, our results demonstrate that the visual system can use a global motion signal for each eye to compute the 3D direction of an object or surface, thus bypassing the traditional—and traditionally difficult—binocular matching problem. Thus, the visual system can still compute 3D direction even when different parts of an object or surface may be occluded for each eye, possibly at the expense of some spatial resolution. 

These psychophysical results raise many questions and possibilities concerning how interocular pattern motions are computed in visual circuitry. One possibility is that an extrastriate area such as MT explicitly computes eye-specific pattern motions and/or their interocular difference (Zeki, 1974). However, individual neurons in MT lose their pattern motion sensitivity when 2-component pseudoplaids are placed within their receptive fields (Majaj et al., 2007), as well as when the individual component gratings making up a plaid are presented to separate eyes (Tailby, Majaj, & Movshon, 2010). However, these stimuli differ in several regards (Gabor sizes, densities, speeds, etc.). Known physiology thus does not conclusively tell us whether MT would exhibit pattern motion selectivity to our dichoptic pseudoplaids. 

Interocular comparison of pattern motion signals could be implemented explicitly in a pair of monocular pathways running through extrastriate dorsal cortex. Of course, this seems problematic given reports of only modest ocular dominance in extrastriate motion-sensitive cortical areas, such as MT (DeAngelis & Newsome, 1999; Maunsell & Van Essen, 1983). On the other hand, the interocular comparison of pattern motion could still exploit these small degrees of ocularity (Sabatini, Solari, Andreani, Bartolozzi, & Bisio, 2001). This explanation assumes that, at the population level, small monocular biases are not ignored as “wiring noise” but are instead used to recover eye-of-origin information. Alternatively, interocular comparisons of pattern motions could be performed in the same step as the integration of multiple 1D signals; this possibility could be implemented without the existence of monocular neurons that are pattern motion selective. 

We should note that our psychophysical stimuli (especially the dichoptic pseudoplaids) were different in key regards from the stimuli typically used to probe MT in electrophysiological experiments. For example, the vast majority of what we know about the function of area MT (such as pattern motion integration and binocularity) has been assessed using relatively fast frontoparallel speeds within the spatial receptive field of individual neurons (e.g., Maunsell & Van Essen, 1983). It remains logically possible that interocular pattern motion signals may only be evident when these neurons are tested using slower monocular speeds (like in our psychophysical stimuli), which are more consistent with the retinal projections of many ecologically valid 3D motions (Czuba et al., 2010). Likewise, our psychophysical stimuli has a larger spatial extent than a typical MT receptive field, and thus may have engaged surround mechanisms that are not as well understood (Raiguel, Van Hulle, Xiao, Marcar, & Orban, 1995; Xiao, Raiguel, Marcar, & Orban, 1997, 1998). 

Alternatively, despite the fact that MT is considered a key stage in the computation of 2D pattern motion, other brain regions may be required to perform the motion integration demonstrated in our psychophysical results. Prior work has suggested degrees of pattern motion selectivity in V1 neurons (Guo, Benson, & Blakemore, 2004; Tinsley et al., 2003), although debate surrounds details of experimental assessments of pattern motion selectivity (Movshon, Albright, Stoner, Majaj, & Smith, 2003; Pack, Berezovskii, & Born, 2001). Assuming this selectivity does exist, it is possible that pattern-based interocular velocity differences are computed in V1, although this still leaves open the question of how these pattern motions could be extracted at spatial scales larger than the classical receptive fields of V1 neurons. This alternate explanation must remain speculative until more is revealed about spatiotemporal integration in V1 neurons. 

Another possibility is that eye-specific pattern motions might be extracted in a subcortical “blindsight” pathway that sends signals directly to dorsal extrastriate regions, including area MT (Barbur, Watson, Frackowiak, & Zeki, 1993; Berman & Wurtz, 2010; ffytche, Guy, & Zeki, 1995; Standage & Benevento, 1983). Motion perception preferentially survives in blindsight, and a tectofugal pathway might thus subserve some aspect of real-world (3D) motion perception. 

Although it is difficult to psychophysically pinpoint the neural location of a computation with absolute certainty, our results do reveal that the nervous system somehow accomplishes a 3D motion computation that cannot be easily explained by the use of either early, monocular motion signals or any known or posited disparity mechanisms. The results motivate further study and use of the dichoptic pseudoplaid stimulus in both psychophysics and physiology. For example, one pressing question is whether the pattern motion computations involved in processing pseudoplaids are similar to those for conventional plaids. This is a topic of current study in our laboratory, involving the comparison of Type I and Type II pseudoplaids, as well as the consideration of spatial and temporal parameters that may affect the pattern motion computations (Takeuchi, 1998). 

More broadly, these results suggest a fresh perspective for thinking about a range of previous findings. Several studies have demonstrated a role for interocular velocity differences in 3D motion perception (Beverley & Regan, 1973; Brooks, 2002a; Fernandez & Farell, 2006; Harris & Rushton, 2003; Rokers, Cormack, & Huk, 2009; Shioiri et al., 2000), as well as contributions of monocularly occluded stimuli (Brooks & Gillam, 2006, 2007), but these studies did not directly investigate the level of the motion processing hierarchy at which this computation occurs. (One recent study did report that the IOVD mechanism operates on eye-specific motion signals that are broadband in spatial frequency, suggestive of a later stage of spatiotemporal integration (Shioiri et al., 2009).) Meanwhile, a separate line of work has suggested monocular contributions to 2D pattern motion perception (Alais, Burke, & Wenderoth, 1996; Alais, van der Smagt, Verstraten, & van de Grind, 1996; Burke & Wenderoth, 1993) but did not consider the potential functional role of these signals in 3D motion perception. It may be possible to parsimoniously integrate these seemingly disparate lines of work with a single appeal to eye-specific pattern motions. Given that the brain appears to compute and compare eye-specific pattern motions for recovering 3D motion, the challenge now is to understand how and where such a computation occurs (Likova & Tyler, 2007; Rokers et al., 2009). 

Supplementary Materials

Abstract (Dutch version) - PDF 

Acknowledgments

This work was supported by National Science Foundation (NSF) CAREER Award BCS-0748413 to A. C. Huk, NSF Grant IIS-0917175 to L. K. Cormack, and Netherlands Organisation for Scientific Research (NWO) Veni Grant 016.108.047 to B. Rokers. 

Commercial relationships: none. 

Corresponding author: Bas Rokers. 

Email: [e-mail]. 

Address: Experimental Psychology, Utrecht University, Heidelberglaan 2, 3584 CS Utrecht, The Netherlands.