Adaptive pooling of visual motion signals by the human visual system revealed with a novel multi-element stimulus

Abstract

The two-dimensional (2D) trajectory of visual motion is usually not directly available to the visual system. Local one-dimensional (1D) sensors initiate processing but can only restrict the solution to a set of speed and direction combinations consistent with the 2D trajectory. These 1D signals are then integrated across orientation and space to compute 2D signals. Both motion integrations are thought to occur in higher cortical areas, but it remains unclear whether 1D signals are integrated over orientation and space simultaneously (1D pooling process), or instead are integrated locally with the resulting 2D signals then spatially integrated (2D pooling process). From psychophysical responses to novel global-motion stimuli comprised of numerous Gabor (1D) or Plaid (2D) elements, here we show that the human visual system adaptively switches between 1D pooling and 2D pooling depending on the input. When local 2D signals cannot be determined, the visual system shows effective 1D pooling that approximately follows the intersection of constraints rule. On the other hand, when local 2D signals are available, the visual system shows 2D pooling that approximately follows the vector average rule. Spatial motion integration therefore exhibits great flexibility when estimating complex optic flows in natural scenes.

Introduction

In the primate visual system, motion signals are initially detected by direction-selective simple cells in cortical-area V1 with receptive fields tuned for orientation and restricted in spatial extent to small areas. In order to extract the veridical motion of spatially extended objects and global-motion flow, pooling of these local-motion signals across orientation and space is required. This pooling is thought to occur in higher cortical areas, e.g., area V5/MT (Born & Bradley, 2005; Britten, 2004; Kohn & Movshon, 2004; Movshon, Adelson, Gizzi, & Newsome, 1985; Newsome, Britten, & Movshon, 1989; Priebe, Lisberger, & Movshon, 2006; Snowden, 1994). While numerous studies have examined the nature of motion pooling across both orientation and space, there is still no consensus regarding how the visual system implements this pooling, specifically whether it does so using one or two pooling stages. This study directly investigates this issue using novel stimuli designed for the purpose. 

Integration of motion signals across different orientations is needed to solve the aperture problem (Adelson & Movshon, 1982; Fennema & Thompson, 1979; Marr & Ullman, 1981). Due to narrow orientation tuning of V1 neurons (half amplitude bandwidth ranges from 10° to 180° but with a mode of 30° in Macaque; De Valois, Yund, & Hepler, 1982), a single neuron cannot uniquely specify the true 2D vector (direction and speed) of a moving object. Rather, it can only indicate a family of possible solutions consistent with the true 2D motion. The true 2D motion can be determined by pooling 1D signals across orientation. Plaid stimuli, which consist of overlapping component 1D gratings differing in orientation, have been typically used to investigate pooling across orientation (Adelson & Movshon, 1982; Movshon et al., 1985; Wilson & Kim, 1994). The evidence of cross-orientation pooling for plaid stimuli has been found in area V5/MT using electrophysiology (Movshon et al., 1985; Smith, Majaj, & Movshon, 2005) and brain imaging (Huk & Heeger, 2002). A central computational question is whether the integration follows the mathematically valid intersection of constraints (IOC) rule (Adelson & Movshon, 1982; Weiss, Simoncelli, & Adelson, 2002), or other rules, including the vector average (VA), which would be more useful for animate objects that vary in shape during motion (Kim & Wilson, 1993; Wilson, Ferrera, & Yo, 1992; Wilson & Kim, 1994). 

Studies that have used stochastic random-dot kinematograms have focused on how such information is integrated across space (Bex & Dakin, 2002; Britten, Shadlen, Newsome, & Movshon, 1993; Edwards & Badcock, 1994; Newsome et al., 1989). These stimuli are designed in such a way that spatial integration of the motion signals generated by the dots gives rise to a coherent global-motion percept. This global-motion percept is visible when even less than 10% of the dots are moving in the global direction and the rest of the dots are moving in noise directions. It has been shown that the neural response of many V5/MT cells is tuned to the global-motion signal intensity (Britten et al., 1993; Newsome et al., 1989; Salzman, Britten, & Newsome, 1990). 

Motion integrations across orientation and space are not separate problems. For example, to compute the 2D motion of an object from local motions of various contours belonging to the object, the visual system has to integrate 1D local-motion signals across orientation and space. Using more elaborated stimuli, such as multi-aperture plaids, several studies have demonstrated the perception of global motion by spatial integration of non-overlapping 1D motions (Lorenceau, 1998; Lorenceau & Shiffrar, 1992; Lorenceau & Zago, 1999; Mingolla, Todd, & Norman, 1992; Rubin & Hochstein, 1993; Takeuchi, 1998). A possible mechanism that can account for these “1D pooling phenomena” is the “1D pooling process,” in which 1D motion signals are integrated across orientation and space at the same time (“1D by 1D” hypothesis; Figure 1a). Simoncelli and Heeger (1998) have proposed a model in which cross-orientation integration, following the IOC rule, is implemented in V1–MT/V5 connections. With an optional assumption of simultaneous cross-space integration, the model can adequately predict the responses of MT/V5 neurons to the plaid motions, regardless of whether 1D components are spatially overlapping or non-overlapping (1D pooling phenomenon). In addition, the same model can predict the neural responses to global random-dot kinematograms, where local motions are carried by 2D patterns (2D pooling phenomenon). This suggests a possibility that a single one-stage integration process may be responsible for motion integration across orientation and space: i.e., both 1D and 2D pooling phenomena may be mediated by the 1D motion pooling process [“1D by 1D” hypothesis (Figure 1a) and “2D by 1D” hypothesis (Figure 1b)]. 

Figure 1

 

Possible mechanisms underlying motion pooling tested in this study. (a, b) A 1D pooling process that integrates 1D local motion signals across orientation and space following an IOC-like rule. (c, d, e) A 2D pooling process that integrates 1D motion signals across orientation at each location then integrates the resulting 2D local motion signals over space following a VA-like rule. The 1D pooling mechanism can explain not only the spatial pooling of 1D motions (1D pooling phenomena; e.g., local Gabor motions) (a) but also the spatial pooling of 2D motions (2D pooling phenomena; e.g., local Plaid or dot motions). The title “1D by 1D” implies “explanation of 1D pooling phenomenon by 1D pooling process.” Similarly, the 2D pooling process can explain not only 2D pooling phenomena (e), but also 1D pooling phenomena with an additional assumption that local 2D motion signals are computed via local pooling of adjacent 1D motion signals (c), or that local 2D motion signals are approximated by orthogonal vectors of 1D motion signals (d). Our question is whether there is the 1D pooling process only (a + b), the 2D pooling process only (c/d + e), or both are adaptively used depending on the stimulus (a + e). Our results support the last hypothesis.

An alternative possibility is “2D pooling process” in which motion signals are first integrated across orientation over a small region, and then the resulting 2D local motion signals are globally integrated over space ( Figures 1c– 1e). The critical difference from the 1D pooling process is that the aperture problem is solved locally before spatial pooling. In support of this hypothesis, recent electrophysiological findings suggest that the spatial range of cross-orientation integration of MT/V5 neurons is relatively small (Majaj, Carandini, & Movshon, 2007; Rust, Mante, Simoncelli, & Movshon, 2006). Theoretically, the 2D pooling process can explain global-motion perception not only when local motion is conveyed by a 2D pattern such as a dot (i.e., 2D pooling phenomenon), but also when local motions are conveyed by 1D pattern as in the components creating multi-aperture plaid stimuli (1D pooling phenomenon). These possibilities are respectively called “2D by 2D” (Figure 1e) and “1D by 2D” (Figures 1c and 1d) hypotheses. As shown by Figure 1, there are two versions of the “1D by 2D” hypothesis. One (“semi-local 1D pooling” hypothesis; Figure 1c) is to compute 2D motion signals from local processing of adjacent 1D signals falling within a small spatial area for cross-orientation integration. The other (“orthogonal vector” hypothesis; Figure 1d) is to use orthogonal vectors of 1D local motion as local 2D motion signals—when no other motion or form cues are available, a 1D pattern, e.g., a static Gabor with a moving carrier, is seen to move in the direction orthogonal to the carrier. This suggests that orthogonal motion is the default solution of the aperture problem for 1D patterns. A 2D pooling process could then treat these 1D signals (which still contain direction and speed information) in the same manner as (true) 2D signals. 

Given that both 1D and 2D pooling processes can account for the perceptual phenomena when either 1D or 2D motion patterns are presented (1D and 2D pooling phenomenon), it remains an open question as to which pooling process the human visual-system uses. It is also possible that the human motion system may flexibly use 1D or 2D pooling solutions depending on which one the stimulus allows to be useful—1D pooling is used when local information only represents a 1D component of the object's motion so that the aperture problem cannot be solved locally, while 2D pooling is used when local information is rich enough to solve the aperture problem. This hypothesis assumes that the choice of stimulus shapes the spatial range over which 1D local motion is pooled, and then changes the type of motion signals that can be usefully integrated over a large spatial scale. This possibility is functionally elegant, but the underlying computation must be more complex than is needed for the first two possibilities. 

To address the mechanism of motion pooling, this study introduces a series of novel motion stimuli that allow us to control the amount of information (1D or 2D) available at each spatial location. The stimulus for the 1D pooling phenomenon consists of numerous, spatially distributed, stationary Gabor elements with a drifting-carrier grating (Global-Gabor stimulus). While the orientation of the sine-wave carrier in each Gabor is randomly assigned, the drift rate of each carrier (i.e., its orthogonal motion) can be made consistent with an IOC-determined global 2D velocity. Under these conditions, observers see rigid pattern motion. As with global dot-motion stimuli, a portion of the Gabors can be made into noise elements by giving them drift rates that are incompatible with the global 2D motion. To investigate the 2D pooling phenomenon, a comparable stimulus was used, in which the local Gabors were replaced by local Plaids. In this Global-Plaid stimulus, each local element consisted of two sine-wave gratings windowed by a stationary Gaussian. In a series of psychophysical studies, we examined whether the underlying computation in motion integration is 1D or 2D pooling by independently manipulating the proportion of 1D and 2D motion signals included in the stimulus. We also examined whether the rule of motion signal integration was IOC or VA, since IOC pooling is an integration rule for 1D motion components, while, arguably, VA is an integration rule for 2D motion vectors (even when it is applied to orthogonal motions of 1D patterns). 

Specifically, in order to investigate the characteristics of 1D motion pooling, we measured the threshold signal ratio required for the perception of Global-Gabor motion direction. We also estimated the range of spatial pooling and tested motion transparency. These experiments were intended to test and exclude the hypothesis that Global-Gabor motion can be explained by the “1D by 2D: Semi-local 1D pooling” hypothesis. To see whether the Global-Gabor motion is based on IOC pooling of 1D component motions (consistent with the “1D by 1D” hypothesis) or VA pooling of orthogonal 2D motion vectors (consistent with the “1D by 2D: Orthogonal vector” hypothesis), we measured the perceived speed of Global-Gabor motion since the two hypotheses predict different perceived speeds. We also measured the perceived global-motion direction for two types of Global-Gabor motion made of two orientation components—one was made of orientation components that were symmetric about the signal 2D vector and had different relative densities (type I stimulus), and the other was a so-called type II stimulus (Ferrera & Wilson, 1990) in which the orthogonal vectors were both located on one side of the signal 2D vector. To investigate the characteristics of 2D motion pooling (i.e., to distinguish the “2D by 1D” and “2D by 2D” hypotheses), we measured the threshold signal ratio required for the perception of the Global-Plaid motion direction, with signal and noise components in either separate Plaid elements or in the same elements. Motion transparency was tested in the same way. Finally, we introduced a unique stimulus, Global-Mirror-Plaid motion (discussed below), to demonstrate genuine spatial pooling of local 2D motion signals. 

Our findings showed that the human visual system spatially integrates 1D local-motion signals (aperture problem not solved locally) using the IOC rule (“1D by 1D”; Figure 1a) and integrates 2D local-motion signals (local solution of the aperture problem) following the VA rule (“2D by 2D”; Figure 1e). Additionally, once the two 1D sine waves that comprise the local plaids are combined at the local level to produce the 2D signal, they cannot be decoupled at the global level, i.e., their 1D component signals cannot be independently combined at the global level (not “2D by 1D”; Figure 1b). These results indicate the presence of both 1D and 2D pooling processes in the human visual system. The present findings suggest that spatial motion integration is a complex and elegant system, having great flexibility for the purpose of estimating complex motion flows in natural scenes. 

Global-Gabor motion: Evidence for 1D motion pooling process

To investigate the computation underlying pooling of local 1D motion signals across orientation and space, we evaluated the psychophysical performance of human observers when detecting Global-Gabor motion ( Figure 2). The orientation of the sine-wave carrier in each Gabor in the Global-Gabor stimulus was randomly assigned but the drift rate was made consistent with the IOC-defined global 2D vector. That is, its drift speed varied as a sine function of the angle between the carrier orientation and the 2D vector. When the carriers of all Gabors move in this manner, observers see rigid-body motion in the global 2D direction ( Movie 1). In order to see such a coherent motion percept, motion information has to be spatially integrated across Gabor patches. This is consistent with previous reports of 1D motion pooling phenomena (Lorenceau, 1998; Lorenceau & Shiffrar, 1992; Lorenceau & Zago, 1999; Majaj et al., 2007; Mingolla et al., 1992; Rubin & Hochstein, 1993; Takeuchi, 1998), although having a large number of random orientation elements stabilized the global motion percept. The use of Gaussian windows and a relatively low contrast (ten times direction discrimination threshold) reduced the contribution of line terminators to the motion percept and peripheral viewing enhanced the global pooling (Lorenceau & Zago, 1999; Takeuchi, 1998). 

Figure 2

 

A depiction of the stimulus layout used for the current Global-Gabor motion experiments. See the text for details.

 

Movie 1

 

Global-Gabor motion with 100% downward signals. The movie playing speed is slower than that of the original stimulus for demonstration purpose. Format: QuickTime (H.264).

Noise tolerance

The perception of coherent motion for Global-Gabor motion is consistent with, but is not sufficient to prove, the existence of a 1D pooling process (“1D by 1D”; Figure 1a). One concern is that when all the Gabor elements are given 1D motions consistent with a common 2D vector, it is possible that the aperture problem is solved locally by integrating adjacent Gabors, and then the resulting 2D signals are integrated over space (“1D by 2D: Semi-local 1D pooling”; Figure 1c). To evaluate the effectiveness of spatial pooling of 1D motion signals for Global-Gabor motion perception, we measured the proportion of signal elements needed to identify the signal direction. As with global dot-motion stimuli, noise can be added, in this case by giving some of the Gabors drift rates that are incompatible with the IOC solution ( Figure 3, Movie 2). If Global-Gabor motion has high noise tolerance, the idea of semi-local 1D pooling followed by global 2D pooling would be dismissed. 

Figure 3

 

Noise tolerance of Global-Gabor motion. (a) An enlarged view of a portion of the Global-Gabor stimulus. (b) Diagram showing how signal Gabors (S) and noise Gabors (N) are made. A circle represents a Gabor patch. A black line indicates the orientation of luminance modulation. A gray arrow indicates the 2D vector given to the patch and the red arrow shows the 1D component vector in the orthogonal direction. (c) A plot of the percent correct when identifying global motion direction from eight possible alternatives displayed as a function of the signal ratio. The 56.25% point of the fitted Logistic function was computed as the threshold signal ratio (red circle). Observer: SN.

 

Movie 2

 

Global-Gabor motion with 50% downward signals and 50% noise.

Methods

For all our psychophysical experiments, visual stimuli were generated using a ViSaGe (Cambridge Research System, Cambridge) and presented on a 21-inch CRT (SONY GDM-F500) at a frame rate of 100 Hz. Observers viewed the monitor at a distance of 52.3 cm, and their head was stabilized by a chin rest. Two of the authors (KA and SN) and one naive subject (AM) participated in the experiments. 

Unless otherwise noted, a Global-Gabor motion stimulus ( Figure 2) consisted of 768 Gabor patches arranged in a regular 1° × 1° grid ¹ ( Figure 3a). The stimulus was presented within a circular field of 32° in diameter. Since 1D motion pooling deteriorates in the central visual field (Takeuchi, 1998), Gabors were not presented in the central field of 6° in diameter. In each Gabor patch, the envelope was a stationary Gaussian, and the carrier was a randomly oriented 2 c/deg sinusoid that drifted at the speed consistent with a given 2D vector of 2 deg/s. Stimulus exposure duration was 200 ms, and the observer viewed the stimulus while maintaining fixation on the central bull's-eye. To ensure the visibility of the stimulus, we measured the carrier contrast threshold for identifying the direction of a 100% signal Global-Gabor motion stimulus, in which all patches share a common 2D vector. The carrier contrast of the main experiments was set at 10 times the threshold (1.4%, 2.3%, and 1.8% for observers KA, SN, and AM, respectively). 

In order to measure the threshold signal ratio for direction identification, we presented a stimulus consisting of X% signal Gabors and 100 − X% noise Gabors ( Figure 3b). The method of constant stimuli was employed. For signal elements, the direction of the 2D vector (common for all the signal elements) was chosen from 8 possibilities (including cardinal directions, 45° step). For noise elements, the direction of the 2D vector was randomly chosen for each element from 72 possibilities (5° step, including the signal direction). ² The observers were instructed to indicate which of the 8 alternatives the perceived motion direction corresponded to. This procedure ensures only a 12.5% possibility of correctly guessing the global direction without proper spatial integration. Each signal ratio was presented 20 times. A logistic function was fitted to the obtained psychometric function, and the contrast producing 56.25% (100/8 + 50 × 7/8%) correct responses was estimated as the threshold ( Figure 3c). The 95% confidence interval of the estimated threshold was computed by the bootstrap method (Efron & Tibshirani, 1994). 

Results

The threshold signal percentage (for 56.25% correct identification) was as low as 12.2–14.4% [see Figure 3c for the psychometric function of a single observer and Figure 4 (Gabor) for the thresholds of all observers]. When the percentage of signal was ≥35%, the direction identification was nearly perfect. This suggests that local 1D motion signals are effectively integrated over space, in agreement with the “1D by 1D” hypothesis, but not with the “1D by 2D: Semi-local 1D pooling” hypothesis. 

Figure 4

 

The threshold signal percentage for the four stimulus conditions: Global-Gabor motion (see Figure 3) and Global-Plaid motion with global noise (full and half pattern density) and with local noise (see Figure 11). Error bars represent 95% confidence intervals.

Range of spatial pooling

To determine how pooling of 1D motion signals is influenced by distance, we examined how the detectability of global motion direction was affected by the inter-patch separation. If the “1D by 2D: Semi-local pooling” hypothesis is correct, global motion perception should be lost with relatively small patch separations. To increase the patch separation to N, we presented only one Gabor every N lines both horizontally and vertically. In other words, we erased ( N − 1) lines for every N lines without changing the original regular Gabor grid structure. This reduced the density to 1/ N ². N varied from 1 to 10, corresponding to 1–10 deg separation. The spatial position of the presentation lines was changed randomly for every trial. All the Gabor elements were signal, giving a common 2D vector chosen from 8 directions. The results ( Figure 5) show that global direction judgment is nearly perfect until the adjacent patches are separated by more than 4–6 deg. Thus, the range of spatial pooling of 1D motion signals can be fairly large, in agreement with the “1D by 1D” hypothesis. 

Figure 5

 

The percentage of correct identifications of global motion direction with 100% signal Global-Gabor motion is plotted as a function of patch separation (center-to-center distance).

Motion transparency

When one half of the Gabors moved in a given 2D direction, and the other half the opposite direction, the observers could see two global motions at the same time ( Movie 3, Figure 6a). This is a phenomenon similar to motion transparency seen with random-dot kinematograms (Braddick, 1993; Edwards & Greenwood, 2005; Greenwood & Edwards, 2006; Qian, Andersen, & Adelson, 1994). The perception of motion transparency with Global-Gabor motion further requires that 1D motion signals are integrated over a wide spatial extent, providing additional evidence against the idea of “1D by 2D: Semi-local 1D pooling” (Figure 1c). To objectively demonstrate this intriguing phenomenon, we measured the proportion of signal elements (for one direction) required to discriminate a two-direction signal stimuli from stimuli containing pure noise and just one-direction signal (Figure 6). 

 

Movie 3

 

Global-Gabor motion with 50% downward signals and 50% upward signals.

Figure 6

 

Motion transparency of Global-Gabor motion. (a) Diagram showing how two-signal Gabors (S1 and S2) and noise Gabors (N) are made. For each stimulus presentation, the number of signals was either zero (N), one (N + S1), or two (N + S1 + S2), and the observer had to identify both the number and the direction of the signal components. (b, c) The percentage of correct identifications for one-signal stimuli (b) or for two-signal stimuli (c) is plotted as a function of the signal ratio. The correct response for the two-signal stimuli is regarded as indicating the observer's perception of motion transparency. The 50.0% point of the fitted function was computed as the threshold signal ratio (red circle). Observer: SN.

Methods

To measure the threshold signal ratio for transparent motion perception in which two opposing motion directions were seen simultaneously, we presented three types of stimuli in random order in equal numbers of trials—(0) no signal: N 100%; (1) one signal: S1 X% + N 100 − X%; and (2) two signals: S1 X% + S2 X% + N 100 − 2 X%, where N: Noise; S1: Signal #1; and S2: Signal #2. S1 was randomly chosen from eight directions. S2 was opposite to S1. In each trial, observers had to identify the motion type (the number of signals), and when they judged that it was either a one- or two-signal stimulus, they had to identify motion direction from eight possible directions. For the condition containing two signal directions, the observer only had to correctly indicate one of the directions. A correct trial required correct performance on both tasks. This procedure allowed observers to reduce the miss rates for one and/or two-signal stimuli by reducing their criterion for the presence of signals in the first task. To prevent this strategy and to reduce the false alarm rate for no-signal stimuli, incorrect one- or two-signal responses were marked by auditory feedback. No feedback was given when observers made a no-signal response to one- or two-signal stimuli. A given signal ratio was presented 10 times each for both the one- and two-signal conditions. A Logistic function was then fitted to the obtained psychometric function, and the signal ratio corresponding to the 50% correct point was used as the threshold value. It should be noted that the expected lowest performance was 0%, not the chance level of direction judgment (100/8% for one-signal stimuli and 100/4% for two-signal stimuli) since observers should have chosen the “no-signal” option when they did not perceive a signal direction. 

Results

The results indicated that the observers could reliably discriminate the two-direction signal stimuli from stimuli containing pure noise and only one-direction signal [see Figure 6b for the psychometric functions of a single observer and Figure 7 (Gabor) for the two-signal thresholds of all observers]. The threshold to correctly identify the two-signal condition was ∼20%. These transparency thresholds are higher than those obtained for the unidirectional condition ( Figure 4), which is consistent with earlier findings that have shown higher signal levels are required to see motion transparency (Edwards & Greenwood, 2005). 

Figure 7

 

The threshold signal percentage for perception of motion transparency (two-signal stimuli) for the three stimulus conditions: Global-Gabor (Figure 6), Global-Plaid (Figure 11), and Global-Mirror-Plaid (Figure 12) stimuli. Error bars indicate the 95% confidence interval.

We also observed that the motion transparency remained visible even when patches for two signals were regularly alternated in the grid. These observations suggest that 1D motion signals are not only simply integrated over space but are also effectively pooled in a manner consistent with the direction when rigid-motion direction solutions are available. 

Perceived speed

Effective spatial pooling of motion signals in Global-Gabor motion, as described above, is not consistent with the 2D pooling process with “semi-local 1D pooling” ( Figure 1c). However, in order to conclude the presence of a true 1D pooling process in the human visual system, we need to exclude another possibility. As described in Figure 1d (“1D by 2D: Orthogonal vectors”), another potential way to solve the aperture problem for 1D local-motion signals is to assign a 2D vector to these 1D signals in the direction orthogonal to their edges, and then spatially integrate these signals using VA or another, similar 2D pooling process. The next three experiments were designed to distinguish this potential 2D pooling process from a true 1D pooling process. 

One way to determine which of these two pooling processes is being used by the visual system is to investigate the perceived speed of the resultant 2D motion. This is because, while the two processes would often generate the same, or similar, perceived direction of motion, they predict different perceived speeds. Our numerical computation indicates that mathematical VA of orthogonal 2D vectors of 100% signal Global-Gabor motion should result in global-motion perception at, on average, half the speed. We therefore compared the perceived speed of the following three stimuli ( Figure 8a). G: Global-Gabor motion in which all the 1D local movements were consistent with one signal vector ( Movie 1). P1: Global-Plaid motion (consisting of 2D Plaid elements instead of Gabor elements, see the next section entitled “Global-Plaid motion: Evidence for 2D motion pooling process”) in which all the 1D and 2D local movements were consistent with the same signal vector ( Movie 4). VA of local 2D vectors (or IOC of 1D component motions) predicts global motion perception at a veridical speed. P2: Global-Plaid motion in which local 2D motions of Plaid patches were given in such a way to simulate the orthogonal local vectors of Gabor patches in G ( Movie 5). In this case, the direction and speed of 2D Plaid vectors were variable across patches. VA of local 2D vectors of P2 predicts a global-motion percept of half the speed, and IOC has no rigid solutions for P2. If Global-Gabor motion is computed via 2D pooling of orthogonal local vectors (the orthogonal 1D component is treated like a 2D signal), G should appear more similar to P2 than to P1. On the other hand, if G is mediated by IOC-based 1D pooling, the apparent speed as well as the high rigidity of global motion should be similar for G and P1 while P2 should move more slowly and less rigidly. 

Figure 8

 

Speed matching. (a) Stimulus component diagram. P1: Global-Plaid motion in which all the 1D and 2D local movements were consistent with the same signal vector. G: Global-Gabor motion in which all the 1D local motions were consistent with one signal vector. P2: Global-Plaid motion in which local 2D motions of Plaid patches were specified in order to simulate the orthogonal local vectors of Gabor patches in G. A gray arrow indicates the 2D vector given to the patch, and the red arrow shows the 1D component vector in the orthogonal direction. A blue arrow indicates the 2D vector obtained by the IOC computation of local 1D components, which always corresponds to the gray arrow in the present cases. (b) The apparent speeds of the three stimuli estimated by speed matching to a moving checkerboard. The speed of the signal vector (and that of IOC solution of G) was 2 deg/s, while the VA of 1D orthogonal components was 1 deg/s. Error bars indicate the 95% confidence interval.

 

Movie 4

 

Global-Plaid motion with 100% downward signals.

 

Movie 5

 

Global-Plaid motion with each 2D element simulating orthogonal motion of the element in Global-Gabor motion.

Methods

After the presentation of one of the stimuli (test stimulus, G, P1 or P2), we presented a checkerboard stimulus (checker size: 2.1 deg, checker orientation: 0 deg, contrast: 50%) moving rigidly for 200 ms within a 16-deg field in the same direction as the test stimulus. Subjects were instructed to indicate whether the test stimulus was faster or slower than the checkerboard stimulus. The point of subjective equality was measured by the staircase method (1 down, 1 up), in which the speed of checkerboard pattern was increased after a “slow” response by a factor of the fourth root of 2 while it was decreased after a “fast” response by the same factor. The experiment continued until six reversals of the checkerboard speed (each reversal corresponds to a response change), and the perceived speed was considered as the average of the last four reversal points. Four measurements were conducted for each of the three stimuli. We used a coarse checkerboard pattern as the reference stimulus because it has a broad spatial-frequency spectrum and its speed was easy to judge. Note, however, given that a checkerboard is nothing more than a complex plaid, we did not assume that the perceived checkerboard-speed was veridical, we merely considered the relative speed relationship between the three test stimuli. 

Results

Speed matching to a common comparison stimulus indicated that the apparent speeds of G and P1 were indeed similar, and close to that of the signal 2D vector given to the stimuli, while the apparent speed of P2 was only about half that speed ( Figure 8b). These findings suggest that, when presented with the 1D signals in the Global-Gabor stimulus, the visual system does not compute the average of the local 2D vectors but rather a solution requiring proper (i.e., consistent with IOC) integration of 1D local signals, across orientation and space. Also, as we will discuss later, these results are consistent with the notion of VA pooling for Global-Plaid stimuli. 

IOC vs. VA

There has been a long discussion about the principle of motion integration across orientation. Two major candidates are intersection of constraints (IOC) (Adelson & Movshon, 1982; Weiss et al., 2002) and vector average (VA) solutions (Adelson & Movshon, 1982; Wilson et al., 1992). The contributions of second-order motion (Cropper, Badcock, & Hayes, 1994; Derrington, Badcock, & Holroyd, 1992; Kim & Wilson, 1993; Wilson et al., 1992; Wilson & Kim, 1994), 2D feature motion (Alais, Wenderoth, & Burke, 1997; Derrington, Allen, & Delicato, 2004; Nakayama & Silverman, 1988; Pack & Born, 2001), and boundary orientation (Badcock, McKendrick, & Ma-Wyatt, 2003) can be ignored in the case of Global-Gabor motion. The evidence is mixed as to whether 1D motion pooling phenomena follows the IOC rule or the VA rule (Lorenceau, 1998; Mingolla et al., 1992; Rubin & Hochstein, 1993). The present finding of valid speed perception for Global-Gabor motion suggests that spatial integration of local 1D motion signals follows the IOC rule. The result is consistent with the finding by Lorenceau (Lorenceau, 1998) made with rotating line stimuli. 

However, VA models of Plaid perception have been developed mainly to account for the perception of motion direction, not motion speed (Kim & Wilson, 1993; Wilson et al., 1992; Wilson & Kim, 1994). In fact, there has been little serious discussion of whether the integration process computes vector average or sum, although it is obvious that vector sum overestimates global speed for multiple element stimuli (Khuu & Badcock, 2002). Although our data suggest that the VA rule cannot account for the perceived motion speed for 1D pooling phenomena (though it can for 2D pooling phenomena), we could not exclude the possibility that a modified VA model might provide a consistent account for the results shown in Figure 8. To assess our argument that Global-Gabor motion is mediated by a 1D pooling process following the IOC rule, rather than by a 2D pooling process following the VA rule, we carried out two additional experiments. 

Changing orientation composition

The IOC rule predicts that the component orientations in the calculation can vary without affecting the result provided the speeds are adjusted to match the same 2D vector. To test this directly, and by extension the hypothesis that the IOC rule is being implemented, we used Global-Gabor motion consisting of Gabors set to one of two possible orientations (O1 and O2, 90° difference; Figure 9a). The drift speeds of these Gabors were consistent with a common 2D vector (−40° away from O1, and +50° from O2). We randomly changed the target vector together with O1 and O2 and measured the apparent direction of global motion while changing the relative density of O1 and O2 elements. The IOC solution should always be in the target direction independent of the number of contributing components. On the other hand, VA is a linear operation so if motion integration is the VA of local orthogonal vectors, we would expect that the perceived global motion direction should linearly change from the orthogonal direction of O1 to that of O2 as the density of O2 elements changed from 0% to 100%. 

Figure 9

 

Changing orientation composition. (a) Stimulus diagram. (b) Perceived global motion direction as a function of the proportion of Gabor elements with a given orientation [red and green arrows in (a)], whose orthogonal motion direction was about 45 deg away from the signal 2D vector [gray arrows in (a)]. The rest of Gabor elements had an orientation whose orthogonal motion direction was about −45 deg. The direction of +1.0 (−1.0) corresponds to the orthogonal direction of the O1 (O2) component. The proportion was varied from 0 to 0.5, and the data for the proportion with 0.5 to 1 were plotted by flipping the measured data. Global motion was seen in the signal direction (plotted as 0 deg on the y-axis) for a wide range of mixture ratios of the two orientations, as predicted from IOC. The error bars indicate ±1 SE.

Methods

The stimulus consisted of 192 Gabors, each subtending 2° × 2°, having one of two possible orientations (O1 and O2). The stimulus was presented within a circular field of 32° in diameter. Gabors were not presented in the central field of 12° in diameter. The speed of all Gabors was consistent with a randomly selected common 2D vector. After the presentation of a moving stimulus, a line originating from the center of the screen appeared. Observers rotated the line until its orientation matched the perceived direction of motion. The matching was repeated 10 times for each condition. For measurement of the effect of relative density, X% of the Gabors had O1, and 100 − X% O2. The normal vectors of the O1 and O2 components were +50 and −40 deg or −40 and +50 away from the 2D vector. Spatial frequency was 2.8 c/deg (but we obtained the similar results when the frequency was 0.7 c/deg). The exposure duration was 200 ms. 

Results

The results ( Figure 9b, Movie 6) showed that perceived motion direction remained nearly identical to the IOC direction across a wide density change (between ∼10 and ∼90%), supporting the prediction from IOC. 

 

Movie 6

 

Two orientation Global-Gabor motion with the O2 ratio of 0%, 100%, 50%, 10%, and 90%. The perceived global motion is downward for the last three ratios.

Type II Global-Gabor motion

Type II plaids are stimuli that have been considered as providing critical tests for IOC or VA. In these plaids the two component orthogonal vectors are present on the same side of the true 2D vector (Adelson & Movshon, 1982; Lorenceau, 1998; Mingolla et al., 1992; Rubin & Hochstein, 1993; Wilson et al., 1992). We created type II Global-Gabor motion stimuli made of two similar orientation components (red and green arrows in Figure 10a). 

Figure 10

 

Type II Global-Gabor motion. (a) Stimulus diagram. (b) The perceived direction of type II Global-Gabor motion plotted as a function of stimulus exposure duration for two spatial frequencies of the carrier of the Gabors. The perceived direction was biased towards the IOC direction at longer exposure durations. Subject AM could not reliably judge motion direction for 0.7 c/deg, 100 ms exposure. Error bars indicate the 95% confidence interval.

Methods

The stimulus consisted of 192 Gabors, each subtending 2° × 2°, each having one of two possible orientations (O1 or O2). The speed of all Gabors was consistent with a randomly selected common 2D vector. The relative Gabor density ( X) was 50%. The normal vectors of O1 and O2 components were on the same side of the signal 2D vector, 40 deg and 70 deg away, respectively. Spatial frequency was 1.4 or 5.6 c/deg, and the exposure duration was 100, 200 or 300 ms. 

Results

The perceived motion direction was close to the VA when the exposure duration was very brief (100 ms). When the exposure duration was increased, however, the perceived direction was gradually shifted towards the direction of IOC beyond the range bounded by the two component vectors ( Figure 10b and Movie 7). Similar developments towards the valid 2D direction are also reported for other stimuli (Cropper et al., 1994; Lorenceau, Shiffrar, Wells, & Castet, 1993; Pack & Born, 2001; Yo & Wilson, 1992). The effect of exposure duration was less obvious for one of the three observers (AM), but even for this observer, the perceived direction was significantly away from the VA prediction. Although the perceived direction was significantly deviated from the IOC solution even at the best condition we used, this may be explained by a model that assumes a Bayesian IOC solution with slow speed preference (Weiss et al., 2002). It should be also noted that observers could see some non-rigidity between the motions of the two orientation components, in agreement with the idea that the visual system did not reach a perfect IOC solution in this case. In sum, the perceived direction of type II Global-Gabor motion suggests a significant contribution by IOC-based pooling of 1D motion signals, particularly as duration increases. 

 

Movie 7

 

Type II Global-Gabor motion. O1, O2, and O1 + O2.

Several studies have investigated the integration rule for non-overlapping 1D motions in type II stimuli (Lorenceau, 1998; Mingolla et al., 1992; Rubin & Hochstein, 1993) but they have found conflicting results. On the one hand, Mingolla et al. (1992) showed that the perception of type II multiple aperture motion was more consistent with VA than IOC predictions (as we found at short durations, but their stimulus presentation was 2 s). Rubin and Hochstein (1993) more convincingly showed that the perceived direction of multiple moving lines seen in an aperture supported VA pooling. However, on the other hand, Lorenceau (1998) reported that the perceived path of a figure defined by connected line segments and moving behind apertures was consistent with the IOC rule. Lorenceau ascribed the difference between his finding and the previous two reports (Mingolla et al., 1992; Rubin & Hochstein, 1993) to the presence of a figural cue for natural contour integration—unlike in the previous studies, in his display, the orientations and positions of the line segments were consistent with an interpretation that those segments were connected with L junctions behind apertures. This figural cue facilitates IOC motion integration (Lorenceau & Alais, 2001; Lorenceau & Zago, 1999; McDermott, Weiss, & Adelson, 2001). The balance between IOC and VA pooling that occurs in type II stimuli may depend upon the degree of configural information in the stimulus. That is, the degree to which the orientations and positions of the line/contour segments in the stimulus is consistent with a rigid-body moving behind the viewing apertures. The greater the configural information, the more the pooling is IOC dominated. That our results are midway between the IOC and VA solutions could thus be due to the presence of this configural information in our stimulus, specifically at high spatial-frequencies. For example, the greater number of bars in the sine wave of each Gabor increases the probability that some of them are consistent with this configural information. We are currently investigating this possibility. 

To summarize this section, our findings with Global-Gabor motion support the 1D motion-pooling process in which local 1D motion signals are simultaneously integrated across orientation and space (“1D by 1D”; Figure 1a). 

Global-Plaid motion: Evidence for 2D motion pooling

The question addressed in this section is whether the 1D pooling process alone is sufficient to explain all the motion-pooling phenomena, including when local motions are carried by 2D patterns (“2D by 1D”; Figure 1b). As described below, several findings obtained with Global-Plaid motion reject this idea, indicating that the visual system also pools 2D motion signal over space (i.e., “2D by 2D”; Figure 1e). 

Global and local noise tolerance

According to the 1D motion-pooling model, the global-motion percept should be dependent only on the 1D motion components within the integration area. We tested this prediction using Global-Plaid motion stimuli ( Figure 11). Global-Plaid motion is identical to Global-Gabor motion except that local elements are 2D Plaid patches ( Movie 4). Unless otherwise noted, within each patch, the orientations of the two 1D components (sinusoids) were orthogonal to each other. As in the case of Global-Gabor motion, we evaluated the performance of Global-Plaid motion perception by measuring the proportion of 1D signal components required to identify signal direction. In the global-noise condition ( Movie 8), signal Plaid patches were given a common 2D vector (gray and blue arrows in S + S in Figure 11), while noise patches were given random 2D vectors (gray and blue arrows in N + N in Figure 11b). In terms of 1D motion components, each Plaid patch contained either two signal components, or two noise components. In the local-noise condition ( Movie 9), on the other hand, a signal component was always paired with a noise component (S + N). When all the Plaid patches were pairs of signal and noise, the proportion of signal was 50%. Further noise was added by replacing some Plaids with purely noise pairs. If the performance of global-motion perception depends only on the composition of 1D motions as predicted by the 1D motion-pooling model, there should be little difference between the thresholds obtained under the two noise conditions of Global-Plaid motion and the threshold obtained with Global-Gabor motion. 

Figure 11

 

Noise tolerance of Global-Plaid motion. (a) An enlarged view of a portion of Global-Plaid motion. (b) A diagram showing stimulus construction. S + S: Two 1D motion components (red arrows) are both signal and create a common 2D vector (gray and blue arrow). This combination makes a global-noise condition. N + N: Both 1D and 2D components are noise. S + N: One component is signal (indicated by common gray arrows) and the other is noise (indicated by randomly directed gray arrows). Blue arrows indicate the resulting local 2D vectors. This combination makes a local-noise condition. (c) The percentage of correct identifications of global motion direction from the eight possible alternatives is plotted as a function of the signal ratio, separately for Global-Plaid motion with global noise (full density and half density), and with local noise. The 56.25% point of the fitted function was computed as the threshold signal ratio (red circle). Observer: SN.

 

Movie 8

 

Global-Plaid motion with 50% global noise. 50% of the patches were given a downward signal, while the other patches were given random 2D vectors.

 

Movie 9

 

Global-Plaid motion with 50% local noise. Each 2D element consisting of a downward signal 1D component and noise 1D component.

Methods

A Global-Plaid motion stimulus consisted of 768 patches arranged in a regular 1° × 1° grid. Each patch was a Plaid composed of two orthogonal 2 c/deg Gabors. The orientation of each patch was randomly chosen. The speed of 2D local motion was 2 deg/s. The contrast of each component grating was 5 times the direction identification threshold for 100% Global-Gabor motion. 

The experimental procedures to measure noise tolerance were identical to those for Global-Gabor motion, except for the following points. Noise was added in two ways. In the global-noise condition, at each patch, the signal component was paired with a signal component while the noise component was paired with a noise component. Gabor components of the signal Plaids moved in a common 2D signal direction randomly chosen from eight possible directions while those of noise Plaids moved in a random 2D direction. Both full and half density stimuli (768 and 384 patches, respectively) were tested. In the local-noise condition, the signal component was locally paired with the noise component. A signal + noise Plaid consisted of a component moving in a common 2D signal direction and one moving in a random direction, while a pure noise Plaid consisted of two components moving in random directions. The maximum signal ratio was 50%. 

Results

The results indicate that the threshold was significantly lower for the global-noise condition of Global-Plaid motion than for Global-Gabor motion [see Figure 11c for the psychometric function of a single observer and Plaid (Global noise) condition in Figure 4 for the thresholds of all observers]. This cannot be ascribed to the difference in the total number of 1D components in the display since the threshold for Global-Plaid motion remained nearly the same even when the density of Plaid patches was halved [ Figure 11d, Half plaid (Global noise) condition in Figure 4]. On the other hand, the global-motion perception was much harder under the local-noise condition [ Figure 11e, Plaid (Global noise) condition in Figure 4]. The signal threshold was about two times as high as that for global noise condition and even higher than that for Global-Gabor motion. These results suggest that interactions between 1D motions, such as integration among signals and masking of signal by noise, are more efficient at the same location than across different locations. 

Motion transparency

The priority of local interaction over global interaction was also indicated by transparency-type stimuli. When opposing 2D motion signals were given to different sets of Plaid patches (global pairing, Movie 10), one could see clear motion transparency ( Figure 12c). Objective performance for detecting two signal directions was better than that for Global-Gabor motion ( Figure 7). However, when the two directions were given to two 1D components within each patch (local pairing, Movie 11), one could not see motion transparency at all (no data, since coherent motion could not be perceived even at the maximum signal intensity). This is similar to previous studies (Curran & Braddick, 2000; Qian et al., 1994) that local pairing of opposite motions suppressed motion transparency, although they paired 2D dot motions while we paired 1D motions. This observation again suggests that interactions between 1D motions are more efficient at the same location than across different locations. 

 

Movie 10

 

Global-Plaid motion with 50% downward signals and 50% upward signals (global pairing).

Figure 12

 

Motion transparency of Global Plaid motion (global pairing). (a) Stimulus construction diagram. (b, c) The percentage of correct identifications of both the number and the direction of signal components is plotted as a function of the signal ratio. (b) Results for one-signal stimuli. (c) Two-signal stimuli. The red circle depicts the 52.65% correct point on the fitted function. Observer: SN.

 

Movie 11

 

Global-Plaid motion with each 2D element consisting of a downward signal 1D component and an upward signal 1D component (local pairing).

Global-Mirror-Plaid motion

The 1D motion pooling account of Global-Plaid motion (“2D by 1D”; Figure 1b), at least in its simplest form, cannot predict priority of local interaction over global interaction. However, it could be argued that a part of the local priority may be explained by an additional assumption that the strength of interaction between 1D motion signals is reduced as a function of distance. While we do not argue against this assumption, we believe that a 2D pooling mechanism is necessary since we also found a 2D pooling phenomenon that can be explained by the 2D pooling process (“2D by 2D”; Figure 1e) but appears impossible to explain in terms of 1D motion pooling process. The stimulus consists of 1D motion components with a uniform direction distribution and a constant speed. Within each Plaid patch, the orientation difference is not limited to 90°. Instead, the two 1D motion components are always mirror symmetric with regard to an axis that is common for all the patches (see Figure 13b). As a result, the local 2D motion vector (blue arrows), regardless of whether it is determined by IOC, VA (including the second-order motion component) or other algorithms, will be always in parallel with the symmetry axis, although the sign of the direction and the speed is variable among patches. We termed this Global-Mirror-Plaid motion ( Movie 12). For example, when the symmetry axis was vertical, a local 1D component orientated at a given angle clockwise from horizontal was paired with that oriented at the same angle anticlockwise. The two components had the same sign of motion direction with regard to the vertical symmetry axis. The resulting Mirror-Plaids had 2D vectors either upward or downward, but not in the other directions. Pooling 1D component motions predicts no coherent global motion, whereas pooling of 2D local motion vectors predicts opposing global motions along the symmetry axis as transparent motions. 

Figure 13

 

Motion transparency of Global-Mirror-Plaid motion. (d) An enlarged view of a portion of Global-Mirror-Plaid motion. (a) Stimulus construction diagram. (b, c) The percentage of correct identifications of both the number and direction of signal components plotted as a function of the signal ratio. (b) Results for one-signal stimuli. (c) Two-signal stimuli. The red circle depicts the 52.65% correct point on the fitted function. Observer: SN.

 

Movie 12

 

Global-Mirror-Plaid motion with each local 2D element being mirror symmetric about the vertical axis.

Methods

As in the experiment on motion transparency, three types of stimuli were presented in random order each in 33.3% of the trials: 0: N 100%; 1: S1 X% + N 100 − X%; and 2: S1 X% + S2 X% + N 100 − 2 X%, where N: Noise; S1: Signal #1; and S2: Signal #2. All elements were Mirror Plaids. Each 1D component was a randomly oriented 2 c/deg sinusoid that drifted at the orthogonal speed of 2°/s. The 2D motion speed was determined by 1D component orientation relative to the symmetry axis. The 2D motion direction, determined by the symmetry axis orientation and the sign of components' motion direction, was randomly determined for N, matched with a given signal direction for S1, and matched with the opposite direction for S2. The noiseless two-signal stimulus was the original Global-Mirror Plaid motion, the elements of which had the same symmetry axis, random 1D component orientations, and random direction signs. The noiseless one-signal stimulus consisted only of Mirror Plaids with positive direction signs. The experimental procedures were identical with that used for the transparency of Global-Gabor/Plaid motion, and subjects indicated both the number of motion signals (0, 1, 2) and their direction. 

Results

The results showed that the observers could reliably discriminate two-direction signal stimuli (i.e., Global-Mirror-Plaid motion) from comparable stimuli containing pure noise and one-direction signal ( Figures 7 and 13), indicating that observers could simultaneously see two opposing global motions along the symmetry axis as transparent motions of Global-Mirror-Plaid motion. This finding provides strong evidence that the visual system involves a 2D motion pooling computation. 

Perceived speed

Finally, we consider whether the integration rule for 2D motion pooling is VA or something like it (Khuu & Badcock, 2002). As reported in the last section, we measured the perceived speed of two types of Global-Plaid motion. In one stimulus (P1 in Figure 8a), all the 1D and 2D local movements were consistent with the same signal vector. In the other (P2), local 2D motions of Plaid patches were given in such a way as to simulate the orthogonal local-vectors of Gabor patches in G. The obtained results (Figure 8b, P2) are consistent with the hypothesis that integration of local 2D motion signals follows the VA rule. In agreement with the 50% reduction of the perceived speed of P2, the mathematical VA of local 2D vectors in P2 has 50% of the original signal speed. Perception of valid 2D global motion for P1 can be explained by the VA-based spatial pooling of validly computed local 2D vectors. Another possibility is that 1D motion signals included in P1 are spatially integrated by the IOC rule, but the local priority, shown in particular by the strong masking effects of local noise (Figure 11), suggests that once 1D motion signals are combined and the aperture problem is solved locally, those 1D signals cannot be used again for a global-pooling computation. 

Discussion

Here we have introduced a new theoretical framework and a new method for analyzing the processes supporting the spatial integration of local-motion signals to form a global-motion percept. The aim of the present study was to determine whether the spatial pooling consists of a single stage involving simultaneous integration of 1D local signals across orientation and space (1D motion-pooling process) or an additional stage that integrates locally computed 2D signals across space (2D motion-pooling process). While it has been previously shown that both the local motions of 1D patterns and 2D patterns can be spatially integrated into a coherent global motion percept (Lorenceau & Shiffrar, 1992; Mingolla et al., 1992; Newsome et al., 1989), it remained open whether the underlying computation was 1D motion pooling, 2D motion pooling, or both. Our findings are summarized as follows:

1.  
   The threshold signal ratio for direction judgment of Global-Gabor motion was quite low at 12–15%. Direction judgment of 100% signal Global-Gabor motion remained intact even when the adjacent patches were separated by 4–6 deg. These findings suggest an efficient and wide-range spatial pooling of 1D motion signals. In addition, when one half of the Gabors moved in a given 2D direction, and the other half the opposite direction, the observers could see the two global motions at the same time as transparent motions. The results are consistent with the “1D by 1D” hypothesis (Figure 1a), but not with the “1D by 2D: Semi-local pooling” hypothesis (Figure 1c).
2.  
   The perceived speed of Global-Gabor motion was nearly the same as that of Global-Plaid motion sharing the same global 2D velocity and much faster than the perceived speed of Global-Plaid motion that had local 2D vectors simulating the local orthogonal vectors of Global-Gabor motion. The perceived direction of Global-Gabor motion consisting of two symmetric 1D components (type I) was nearly veridical across a wide range of relative-orientation densities. The perceived direction of type II Global-Gabor motion was significantly shifted away from the VA prediction towards the direction of the IOC prediction. These findings support the suggestion that 1D motion integration in Global-Gabor motion perception is based on the IOC rule. These results are consistent with the “1D by 1D” hypothesis ( Figure 1a), but not with the “1D by 2D: orthogonal vector” hypothesis ( Figure 1d).
3.  
   The threshold signal ratio for direction judgment of Global-Plaid motion was lower than the threshold of Global-Gabor motion when the noise was global (presented as separate Plaid elements), while higher than the threshold of Global-Gabor motion when the noise was local (paired with the signal within an element). Motion transparency was clearly seen when opposing 2D motion signals were given to different sets of Plaid patches, while not seen at all when the two directions were given to two 1D components within each patch. These findings suggest that interactions between 1D motions are more efficient at the same location than across different locations, in agreement with the idea that local 1D motion components in Global-Plaid motion are integrated locally before integrated over space. The perception of Global-Mirror-Plaid motion, which cannot be explained by pooling of 1D motion signals, strongly support the pooling of 2D motion signals. The perceived speed of Global-Plaid motion is consistent with predictions from VA. These results are consistent with the “2D by 2D” hypothesis ( Figure 1e) but not with the “2D by 1D” hypothesis ( Figure 1b).

Our results suggest that the human motion-processing system uses both types of pooling computations, but the method observed depends on the information contained in the stimulus. Motion integration is the IOC-based 1D pooling process when the local motions are ambiguous 1D signals (“1D by 1D”), but VA-based 2D pooling process when the local motions are unambiguous 2D signals (“2D by 2D”). This shows that the pooling of local-motion signals by the visual system is highly adaptive and that it uses the most appropriate combination rule for the given stimulus. When dealing with 1D signals, i.e., when the aperture problem has not been solved, combining the signals over space via the IOC rule leads to a mathematically correct solution for the global 2D motion of a rigid object. However, when the aperture problem can be solved locally, each local signal can be regarded as an estimate of the actual (global) 2D motion. Then, the best estimate of that global 2D signal would be obtained by taking the average (VA) of the local estimates, which is what the system does. 

Although our findings ( Figures 4 and 5) indicate that the range of 1D pooling can extend over several degrees or more, it is a mistake to think that the pooling of local 1D motion signals always occurs over a large, global, spatial scale in order to extract a 2D signal. If the information is available at a local scale, as is the case with the Global-Plaid stimuli, then pooling seems to occur only over a small spatial extent. The choice of stimulus shapes the spatial range over which 1D local motion is pooled and then changes the type of motion signals that can be usefully integrated over a large spatial scale. Additionally, once the two 1D sine waves that comprise the local plaids are combined at the local level to produce the 2D signal, they cannot be decoupled at the global level, i.e., their 1D component signals cannot be independently combined at the global level. It has been known that the spatial range of motion integration is not a fixed constant value but dependent on such factors as retinal eccentricity and stimulus contrast (De Bruyn, 1997; Lorenceau & Zago, 1999; Pack, Hunter, & Born, 2005; Takeuchi, 1998). The present study did not use central vision and/or high contrast that would impair motion integration. Future studies need to address how the stimulus-dependent integration processes identified in the present study interact with these and other stimulus factors to determine the final range of spatial integration. 

The present psychophysical study is intended to establish the computational mechanism of motion perception. Our findings suggest that spatial motion-integration is a more complex and elegant computation than previously thought. This is indicated not only by a flexible change of integration strategy but also by the perception of motion transparency for Global-Gabor motion. This is computationally a challenging case where two rigid global solutions are obtained by spatial pooling of local-motion signals within the same spatial region. It has also been suggested that these motion-pooling processes also take into account figural information when determining the degree of pooling of spatially adjacent motion signals (Lorenceau & Alais, 2001; McDermott et al., 2001). The effects of contrast and eccentricity on motion integration might also reflect a strategy of the visual system to optimally estimate the velocity field (De Bruyn, 1997; Weiss et al., 2002). Presumably these complex and flexible computations are necessary for the visual system to process complex optic-flows in natural scenes. We hope our findings will lead to developments of machine vision algorithms as intelligent as human motion vision. To account for the computation suggested by the present findings in terms of a mathematical algorithm, we modified the latest statistical IOC model (Weiss et al., 2002) into a two-stage process (Amano & Nishida, 2006). In brief, 1D motion signals are locally integrated across orientation (if a rigid solution is possible), then all the candidates for 2D velocity extracted at each location are pooled over space, and the global motion vector is computed from the point of maximum likelihood. We found that this model successfully accounted for all the phenomena reported in the present paper. 

Given the finding of a new computation mechanism, a next question is its neural implementation. At present, it remains an open question as to how this adaptive motion pooling is implemented in neural circuitry. A recent electrophysiological study failed to find integration of spatially separated plaids by monkey MT/V5 neurons (Majaj et al., 2007). One might consider this result to imply that 1D spatial pooling does not take place at or before this area. However, this study simplified the stimulus into a grating pair that did not yield perceptual integration. It has been shown that the same neuron can change the way motion is integrated depending on the stimulus (Huang, Albright, & Stoner, 2007). It would be of interest to see how those neurons respond to the Global-Gabor motion stimuli. In future studies, the set of stimuli we introduced in this study should be a useful tool for analyzing the neural mechanism of motion integration. 

We have also shown that spatial integration of 1D motion signals follows the IOC rule, at least under some stimulus conditions. When 1D motion signals are spatially overlapped as in the standard Plaid stimuli, a similar IOC computation may be also carried out. Bowns and Alais (2006) found that adaptation to a motion in the VA direction made type II plaids appear to move in the IOC direction, and vise versa. This suggests that the visual system can adaptively use both IOC and VA methods even for spatially overlapped Plaid stimuli. We however do not exclude the contributions of other factors including movements of features generated by nonlinear interaction of 1D stimuli (Alais et al., 1997; Badcock & Derrington, 1987; Derrington et al., 2004; Derrington & Badcock, 1992; Kim & Wilson, 1993; Wilson et al., 1992; Wilson & Kim, 1994). It should be also noted that spatial integration of 2D motion vectors may not strictly follow the VA rule. It has recently been reported that VA cannot accurately predict the global motion direction of random dots when the direction distribution is very broad and asymmetric (Webb, Ledgeway, & McGraw, 2007). 

In addition to the apparent direction and speed that we measured in this study, apparent rigidity is an important attribute that characterizes the perception of global motion. IOC is an integration rule that assumes the rigidity of moving objects, while VA is a rule applicable to non-rigid motion. We interpreted a non-rigid movement of P2, in addition to reduced speed, as indicating VA integration ( Figure 8), and a non-rigid movement of type II Global-Gabor motion as indicating a failure to find a perfect IOC solution. We however do not simply consider that rigid motion implies IOC integration and non-rigid motion implies VA integration. For instance, we think it likely that rigid motion of 100% signal Global-Plaid motion (P1) results from VA pooling of local 2D vectors, as noted above, and non-rigid movement of type II Global-Gabor motion is significantly contributed by IOC-based processing. It is also worth mentioning that the perception of non-rigid motion implies that residual local motion information that cannot be ascribed to the global motion is available to the visual system. Full understanding of global-motion perception should also reveal the mechanism underlying this aspect of the phenomena. 

Conclusion

The human visual motion system adaptively switches between 1D (IOC-based) motion pooling and 2D (VA-based) pooling depending on the input. Spatial motion-integration exhibits great flexibility when estimating complex optic flows in natural scenes. 

Acknowledgments

Supported by the Japan Society for the Promotion of Science (KA) and by the Australian Research Council through the ARC Centre of Excellence for Visual Science #CE0561903 (ME) and #DP0666206 (DRB). 

Commercial relationships: none. 

Corresponding author: Kaoru Amano. 

Email: [e-mail]. 

Address: The University of Tokyo, 5-1-5, Kashiwanoha, Kashiwa-shi, Chiba 277-8561, Japan. 

Footnotes

Footnotes

1  The size of each Gabor was made roughly comparable to the size of V1 receptive fields (Dow, Snyder, Vautin, & Bauer, 1981). Note however that it is not critical for our argument to assume that each Gabor element stimulates a single local motion detector since the aperture problem cannot be solved properly even when more than one Gabor elements stimulate the same detector.

Footnotes

2  The directions of the noise elements were randomly chosen from a uniform distribution covering the entire 360 degrees. This means that on any given trial, some of them were consistent with the signal direction. Due to the ambiguity of 1D local motions, the proportion of such noise elements was significantly larger in the Global-Gabor motion than in traditional global motion stimuli consisting of local 2D motions (including Global-Plaid motion). This is however not a problem for our purpose. Given the uniform distribution of the noise directions, those noise elements that had directions consistent with the global direction would have been balanced out by similar numbers of noise elements moving in all of the other directions, such that at 0% signal intensity, the observers did not perceive any global motion. Global motion perception is dependent on the directional bias given by the addition of signal elements on top of directionally balanced noise elements.