Asymmetric global motion integration in drifting Gabor arrays

Abstract

Abstract

Abstract:

Abstract  We examined how ambiguous motion signals are integrated over space to support the unambiguous perception of global motion. The motion of a Gaussian windowed drifting sine grating (Gabor) is consistent with an infinite number of grating velocities. To extract the consistent global motion of multi-Gabor arrays, the visual system must integrate ambiguous motion signals from disparate regions of visual space. We found an interaction between spatial arrangement and global motion integration in this process. Linear arrays of variably oriented Gabor elements appeared to move more slowly, reflecting suboptimal integration, when the direction of global translation was orthogonal to the line as opposed to along it. Circular arrays of Gabor elements appeared to move more slowly when the global motion was an expansion or contraction rather than a rotation. However, there was no difference in perceived speed for densely packed annular arrays for these global motion pattern directions. We conclude that the region over which ambiguous motion is integrated is biased in the direction of global motion, and the concept of the association field, held to link like elements along a contour, needs to be extended to include global motion computation over disparate elements referencing the same global motion.

Introduction

Visual motion provides a rich source of information about the movement of objects in the world and about the observer's own movement (Koenderink & van Doorn, 1991; Longuet-Higgins & Prazdny, 1980). In order to correctly extract this information, motion signals must be correctly segmented and integrated (Braddick, 1993). At a local level, even the motion of a single object becomes ambiguous due to the aperture problem (Marr & Ullman, 1981; Wallach, 1996). This ambiguity arises as any motion of an extended contour or luminance gradient parallel to its orientation will produce no change in the image. The component of motion parallel to the orientation is rendered undetectable, and so we can refer to this local, ambiguous motion as a 1-D motion. Although 1-D movement does not determine the underlying 2-D translation, it does impose certain geometrical constraints on the set of potential solutions to the 2-D velocity (Adelson & Movshon, 1982). The set of all possible 2-D motions that could generate a given 1-D motion lies on a line in velocity space, and the set of all 1-D local motions that could arise from a rigidly translating 2-D contour lies on a circle through the origin in velocity space. 

Amano, Edwards, Badcock, and Nishida (2009) have shown that human observers can integrate motion signals from multi-Gabor arrays to produce a robust and accurate estimation of global motion. Each Gabor element has a different orientation and hence a different 1-D motion but references the same underlying 2-D motion. Extracting the correct underlying motion given the geometry of the array becomes a simple matter of finding the intersection of constraints for multiple 1-D signals. It is still a matter of debate whether the visual system makes full use of the geometric constraint. Other methods of motion integration have been proposed based on the vector average or sum (Bowns & Alais, 2006; Yo & Wilson, 1992), feature tracking (Bowns, 1996), or mechanisms based on combining information from first- and second-order motion channels (Derrington, Badcock, & Holroyd, 1992). A more recent analysis points to the harmonic vector average as a viable integration scheme (Johnston & Scarfe, 2013). Although the exact method of integration is still under investigation, the need for motion integration over space is undeniable. It should be noted that, here, by motion integration, we are generally referring to the process by which the inherent ambiguity of 1-D signals is resolved to recover the underlying 2-D motion. This is different from motion integration used in reference to the unambiguous motion of 2-D stimuli, such as dots, by which it describes the process of assigning one overall 2-D motion to a set of moving elements, which may or may not have the same physical 2-D motion (Dakin, Mareschal, & Bex, 2005; Webb, Ledgeway, & Rocchi, 2011). 

For rigid frontoparallel translations, the motion constraints depend solely on the speed and direction of the 1-D velocity components and are independent of location in the visual field (Jasinschi, Rosenfeld, & Sumi, 1992; Schunck, 1989), i.e., we can throw away position information and still compute the global motion. However, for more complex situations, such as multiple objects, global rotations and expansions, and transparent motion, the spatial arrangement of motion signals must be taken into account (Durant, Donoso-Barrera, Tan, & Johnston, 2006; Zemel & Sejnowski, 1998). Accordingly, a number of motion-processing models make explicit use of the spatial distribution of motion signals (Grossberg, Mingolla, & Viswanathan, 2001; Liden & Pack, 1999). These models tend to assume a small, isotropic pooling region in which motion signals from both line segments and line terminators are grouped primarily on the basis of their location. 

The effect of stimulus configuration on detection of contours has been extensively studied (Field, Hayes, & Hess, 1993). For static Gabor elements, detection performance increases with the number of elements, element alignment, and similarity of phase and spatial frequency (Bex, Simmers, & Dakin, 2003). Hayes (2000) showed that detection was based on perceived alignment rather than physical alignment by introducing motion-based position shifts into the elements. The detection of these contours is thought to be accomplished by biased connections between local detectors favoring spatially aligned neurons tuned to similar orientations often referred to as an association field (Field et al., 1993). However, longer range interactions also need to be considered (Loffler, 2008). 

Bex et al. (2003) have shown that the human visual system can correctly integrate local direction signals into moving contours. Their array elements consisted of Laplacian dots to ensure the grouping cue was provided by the direction of motion rather than the contour orientation. They found that the direction of motion of each local element, relative to the spatial arrangement of the contour, had little effect on the contour's detectability. However, because the stimuli used in these experiments were band-pass filtered dots, which have well-defined 2-D movements, the visual system can, in theory, extract the direction and speed of motion of each element unambiguously. Amano et al. (2009) have shown global motion integration can be quite different for 1-D (Gabor) and 2-D (plaid) elements. In particular, 2-D plaid elements chosen to have the same velocity as the normal component of a corresponding set of Gabor array elements do not appear to cohere effectively into a single moving surface, and the perceived speed is much lower. The expected average normal component velocity of a global Gabor array is half the speed of the true global velocity. If the local motion of a Gabor array with a single global speed is integrated perfectly, for example, by means of an intersection of constraints (IOC) operation, the perceived speed should match the true global speed. If perceived speed is less than the true global velocity, this implies less than perfect integration. 

We examined 1-D motion integration over space by measuring the perceived 2-D motion of multielement Gabor arrays by comparison with locally unambiguous plaid arrays. In the first experiment, we found that the perceived speeds of Gabor arrays depended on the spatial arrangement of the individual elements relative to their single global motion direction. When global motion was parallel to the direction of spatial orientation of linear Gabor arrays, the perceived speed of motion was greater than when the global motion was orthogonal to the spatial orientation of the array. In the second experiment, we found similar results with more complex motion, i.e., rotation and expansion. For circular Gabor arrays, the perceived speed of global rotation was seen as faster than expanding Gabor arrays when measured against the corresponding plaid arrays. This is again consistent with motion orthogonal to the spatial arrangement being perceived as slower than motion parallel to the global contour, indicating enhanced motion integration for global motion aligned with the spatial configuration. 

Other studies have found a difference in perceived speed between translation, rotation, and expansion (Bex & Makous, 1997; Bex, Metha, & Makous, 1998; Geesaman & Qian, 1996). In a final control experiment, we used larger multielement arrays and found no integration-dependent reductions in perceived speed for these global motion patterns. The results of Experiments 1 and 2 therefore point toward a reduction in the degree of motion integration when the direction of global motion is orthogonal to the spatial arrangement of the 1-D motions. We conclude that a higher-order property, namely global motion, that is not available to neurons early in the visual pathway, must be influencing how the information from these neurons is integrated. 

General methods

Stimuli were generated on a PC (Dell Precision 380; Dell, Bracknell, UK) and displayed using a Visage system (Cambridge Research Systems, Rochester, UK) on a Mitsubishi Diamond Plus 230SB monitor (1280 × 800 pixels, 100 Hz refresh; Mitsubishi, Tokyo, Japan). Viewing distance was 57 cm and maintained using a chin rest. Two of the authors and four naïve subjects participated in the experiments. All had normal or corrected-to-normal vision. In all experiments, stimuli consisted of arrays of Gabor or plaid patches (single or two superimposed sine gratings modulated by a Gaussian). The number, position, orientation, and drift speeds of the patches were experiment-specific parameters to be described below. 

The array elements were defined as  

 

where j = 1 or 2 for a Gabor or plaid, respectively. C = 50% is the contrast, σ = 0.4° is the standard deviation of the Gaussian window, a and b determine the orientation and the spatial frequency (1 c/°) of the sine grating, and ω is the temporal frequency. This function was truncated to limit the patch to a 1° square region (|x|,|y| < 1° from the center of the patch). The amplitude of the Gaussian envelope was less than 1% at a distance of 1° from each patch center (1° = 2.5 standard deviations), ensuring no visible boundaries at the patch edges. The temporal frequency, ω, was determined by the sine wave grating orientation such that the drift speed of the grating was consistent with a given global motion. When the underlying 2-D motion has speed s and direction φ, the drift speed of a sine grating of orientation θ is given by s × cos(θ − φ). Motion sequences comprised 27 frames, each shown twice, at a frequency of 100 Hz, giving a 540-ms sequence at a resulting frame rate of 50 Hz. Subjects fixated a bull's-eye in the center of the screen in all experiments. 

Experiment 1

In the first experiment, we addressed the issue of whether or not the integration regions in global motion processing are isotropic. Amano et al. (2009) showed that the perceived speed of a plaid array with the same local velocity components as a comparison Gabor array appeared to move more slowly. This is because, for global Gabor arrays that are consistent with a single common velocity, the drift speed of each Gabor is less than or equal to the global speed. The visual system can only extract the true global speed if the movement of individual Gabors is correctly integrated. This observation indicates that the perceived speed of Gabor arrays can be used as a proxy measure of how well elements integrate to provide a single global percept. 

Other potential measures of motion coherence include subjective assessments or direction judgments (Alais, Burke, & Wenderoth, 1996; Amano et al., 2009; Takeuchi, 1998). We used speed judgments rather than direction judgments as they provide a quantitative measure of coherence for any distribution of Gabor orientations. If motion integration results from spatially undifferentiated integration over a spatially isotropic region and is independent of global motion direction, then we should see no difference in perceived speed when we vary the direction of global motion. Any differences observed can be attributed to differences in motion coherence. 

Methods

One of the authors and two naïve subjects participated in Experiment 1. Stimuli consisted of linear arrays of Gabor or plaid patches (see Figure 1A). A total of 16 patches were shown in two parallel lines ±8° from fixation. Horizontal and vertical lines were tested in separate blocks. Patch separation was either 2.29° or 3.43°; therefore the lines extended ±8° or ±12° from the midline. Gabor orientations were chosen at random from one of 10 equally spaced angles from 0° to 170° from vertical. The two components of each plaid were orthogonal and chosen at random to be either 0° and 90° or 45° and 135° from vertical. The global direction of motion was pseudorandomly chosen from one of the four cardinal directions, ensuring global motion direction was always parallel or orthogonal to the arrangement of motion elements. The global speed of the Gabor stimulus was fixed at 3°/s. Perceived speed was measured using the method of constants. There were 10 test speeds. The global speed of the comparison plaid arrays varied exponentially between 1.82°/s (=3e^−0.5) and 4.95°/s (=3e^+0.5). Trials consisted of 540-ms movie sequences of Gabor and plaid stimuli presented in random order and separated by a 2-s interval in which only the fixation target was shown. Within each trial, the global direction of the Gabor and plaid arrays were always identical. Subjects reported which of the two motion stimuli appeared to move faster in a binary choice paradigm. A psychometric function (cumulative Gaussian) was fitted to the data and the 50% point taken as the point of subjective equality (PSE). Variances were estimated via bootstrapping. 

Figure 1

 

(A) Left: An example of the (horizontal) linear Gabor array stimulus used in Experiment 1. Right: Results of speed matching for linear Gabor arrays relative to linear plaid arrays. (B) Center-to-center patch separation was 2.29°. (C) Center-to-center patch separation was 3.43°. Note, in both cases, the linear arrays were located 8° from fixation. Open symbols are results for three individuals; filled squares are group means. Error bars are 1 SE of these means. Dashed line indicates veridical speed, 3°/s.

Results

The results of Experiment 1 for three observers are shown in Figure 1. Perceived speed in degrees per second is on the ordinate. Veridical speed was 3°/s in all cases. There is a clear distinction between the parallel motion (i.e., global motion parallel to the linear arrangement of Gabors) and orthogonal motion conditions. For the narrow (2.29°) patch separation condition (Figure 1B), the perceived speeds for parallel motions with horizontal and vertical lines of Gabors were 3.00°/s (SE = 0.056°/s) and 2.96°/s (0.054°/s), respectively. Corresponding values for orthogonal motions were 2.57°/s (0.116°/s) and 2.70°/s (0.068°/s). This equates to a reduction in perceived speed of 10%–16% for orthogonal versus parallel global motion. For the wide (3.43°) patch separation condition (Figure 1C), the results are similar for two observers, and there is no difference between parallel and orthogonal motion for observer AB. 

Experiment 2

For the linear arrays, each element provides an estimate of global translation. However, for motion fields, such as expansion or rotation, both local velocity and position need to be specified to generate a global percept. Neurons in the medial superior temporal (MST) area that are sensitive to expansion and rotation have considerably larger receptive fields than those in the middle temporal (MT) area (Desimone & Ungerleider, 1986). This may allow integration of motion signals over a larger area and hence an increase in accuracy of motion perception. However, if motion integration in MST is based on integrating estimates of local translation, then we may expect the reduction in perceived speed seen in Experiment 1 to be seen in perceptions of more complex motion stimuli. 

Methods

In Experiment 2, observers again judged the relative speed of Gabor and plaid arrays. However, the stimuli differed from Experiment 1 in terms of the type of global motion present and the spatial arrangement of the individual elements. Motion elements were placed on a 2° × 2° square grid and confined to lie within an annulus between 10° and 12° from fixation (see Figure 2A). The global motions presented were the four cardinal directions in “spiral space” (Graziano, Andersen, & Snowden, 1994): clockwise rotation, anticlockwise rotation, expansion, and contraction. Global rotation is therefore approximately parallel to the local arrangement of elements, and expansion and contraction are orthogonal. In each trial, observers were sequentially shown an array of Gabors and an array of plaids, each moving in the same global motion pattern with order randomized. The global reference speed of each Gabor was always 3°/s, and the plaid speed was chosen via the method of constants from 10 exponentially spaced speeds from 3 × e^−0.5 = 1.82°/s to 3 × e^+0.5 = 4.95°/s. Observers indicated by button press which stimulus appeared to move faster. A psychometric function (cumulative Gaussian) was fitted to the data and the 50% point taken as the PSE. Variances were estimated via bootstrapping. 

Figure 2

 

(A) An example of the circular Gabor array stimulus used in Experiment 2. (B) Results of speed matching for three individuals (open symbols) and the group mean (filled squares). Error bars denote 1 SE of the mean. Dashed line indicates veridical speed, 3°/s.

Results

The results of Experiment 2 for three observers are shown in Figure 2B. Perceived speed of the elements for rotation is 3.15°/s (SE = 0.059°/s) and for expansion/contraction is 2.85°/s (SE = 0.060°/s); both of these are significantly different from veridical (p < 0.01). The reduction in perceived speed for expansion/contraction relative to rotation is again consistent with motion parallel to a contour being perceived as faster than orthogonal motion. The spatial arrangement and velocity distribution of these stimuli are identical. The global motion direction is simply rotated by 90°. We should consider whether the increased speed for rotation seen in this experiment is due to spatial coalignment of the global motion direction and the spatial configuration of the circle or, alternatively, whether rotation may appear faster than expansion or contraction in general. Because we measure perceived speed of Gabors relative to the speed of plaid elements, any such general effects of motion direction should cancel in the comparison. Nevertheless, as a further check that the differences in perceived speed were due to the configuration rather than global motion direction per se, we measured perceived speed for rotation, spiral motion, expansion, and contraction in dense arrays. In this case, each Gabor is completely surrounded by other Gabors, apart from those elements on the boundary, and no configural alignment effect would be expected. 

Experiment 3

Experiment 2 suggests that global motion perception is mediated by a mechanism that is biased toward motion along a contour. Differential effects have been found for speed perception of various global motion types with expansion perceived as faster (Bex & Makous, 1997; Bex et al., 1998; Geesaman & Qian, 1996). These studies used either random dot kinematograms or gratings with the global motion always orthogonal to the grating. Although we are comparing Gabors and plaid elements, a procedure that should eliminate any general effects of global motion direction on perceived speed, we wished to clarify whether there were any residual differential effects that might explain the spatial configuration effect. We repeated the procedure of Experiment 2 for dense annular arrays of Gabor and plaid elements that do not spatially constrain motion integration to act along a contour. 

Methods

The experimental paradigm was much the same as that in Experiment 2. Gabor and plaid elements were again positioned on a notional 2° × 2° grid and confined to an annulus about fixation. The annulus now covered a region from 6° to 16° from fixation, giving a total of 176 motion elements (see Figure 3A). Observers again viewed sequentially displayed arrays of Gabor and plaid elements moving with a specific global motion type drawn from spiral space: 0° and 180° being expansion and contraction, ±90° being rotations, and ±30°, ±45°, and ±60° denoting various spirals. Note that speed varies with eccentricity in these types of global motion stimuli, so the speed of Gabor element motion was chosen to give an average translational speed of 3°/s. The plaid speed was exponentially varied around this speed as in the previous experiments. Observers again indicated by button press which stimulus appeared to move faster. A psychometric function (cumulative Gaussian) was fitted to the responses and the 50% point taken as the PSE; variances were estimated via bootstrapping. 

Figure 3

 

(A) An example of the dense annular array used in Experiment 3. (B) Results of speed matching for five individuals (open symbols) and the group means (filled squares). The abscissa denotes the spiral pitch of the global motion (0° = expansion/contraction, 90° = rotation). The ordinate denotes the perceived speed of the Gabor array relative to the plaid array. Error bars denote 1 SE of the mean. Dashed line indicates veridical speed, 3°/s.

Results

The results for five observers are shown in Figure 3 with perceived speed on the ordinate and spiral pitch on the abscissa. We can see that the perceived speed of the Gabor arrays did not differ across the types of motion pattern. The results of Experiment 2 cannot be explained simply in terms of residual difference between Gabor and plaid arrays for different directions of global motion processing and are therefore due to the spatial arrangement of motion elements relative to their global motion. 

General discussion

We have shown that the integration of 1-D motion depends on the relationship between the global direction of motion and the spatial layout of ambiguous motion signals, in that perceived speed, an indicator of motion integration, is greater when the global motion is in the direction of aligned Gabor array elements as compared to when the global motion is orthogonal to the aligned elements. 

Local motion analyzers are thought to have a length-width ratio of around one when measured psychophysically through patterns of spatial summation for motion detection and direction discrimination (Anderson & Burr, 1991; Watson & Turano, 1995), indicating that there is no intrinsic spatial summation bias at the level of the analysis of low-level motion signals. However, at this level, the relevant signals result from spatiotemporal variation in image luminance rather than variation in image velocity. 

Nakayama, Silverman, MacLeod, and Mulligan (1985) investigated thresholds for deviation from uniform motion fields. They found that, for shear and compression fields at high motion field spatial frequencies, the amplitude thresholds for the detection of motion in compression fields were lower than for shear fields. They interpreted this as evidence for greater summation of motion signals in the direction orthogonal to the local image motion. In this paradigm, the individual dots are visible, and because the moving elements are 2-D, the individual dot motion is unambiguous. However, note, they did not find any differences in perceived velocity of shear patterns at high spatial frequencies. 

Watamaniuk, McKee, and Grzywacz (1995) showed that extended trajectories of single dots in random noise were readily detectable. The detection enhancement for trajectories of longer than 100 ms cannot be explained by local motion energy mechanisms (Verghese & McKee, 2002; Verghese, Watamaniuk, McKee, & Grzywacz, 1999). One proposal is that motion signals are enhanced through excitatory links from similar motion signals occurring at prior locations along the motion trajectory (Grzywacz, Watamaniuk, & McKee, 1995); an alternative related proposal is that the early part of the trajectory acts as a cue, allowing improved detection of the later part (Verghese & McKee, 2002). This general approach has recently gained some neurophysiological support. Guo et al. (2007) reported that some V1 cell responses to an oriented bar are enhanced if the stimulus forms part of an apparent motion trajectory. Indeed these V1 cells showed a tuned response to the orientation of the bar prior to the arrival of the bar within the cells' classical receptive field. Others have argued that enhanced detection of motion trajectories can be explained by the spatiotemporal smearing of motion signals in the visual cortex, otherwise known as motion streaks (Edwards & Crane, 2007; Geisler, 1999). However, we do not believe this mechanism could account for our findings as the Gabor and plaid patches have physically static envelopes, and so the temporal integration required to produce motion streaks would simply serve to slightly lower the contrast of the stimulus. 

Evidence for motion grouping in the direction of a motion contour comes from studies of motion-defined contour detection (Ledgeway & Hess, 2002; Ledgeway, Hess, & Geisler, 2005), which have shown lower thresholds for detecting contours defined by random dot moving elements when the motion was in the direction of the contour. Sensitivity declined with rotation of the local motion signals away from the contour even though the signals were similar in direction and magnitude. Ledgeway and Hess interpreted their results as reflecting the operation of a motion-based association field, associating direction and position, similar in concept to the association field proposed by Field et al. (1993) to account for orientation-based spatial contour integration. Anatomical support for the association field concept comes from the pattern of local connections in V1 between neurons with similar preferences (Lund, Angelucci, & Bressloff, 2003). Long-range clustering of connections has also recently been reported in V5/MT for neurons with similar direction preferences (Ahmed, Cordery, McLelland, Bair, & Krug, 2012). 

The new observation reported here is that local 1-D motion signals whose normal components differ markedly show greater coherence when the global motion is in the direction of the contour. We can reject isotropic motion pooling as if there were a simple isotropic pooling region for the integration of local velocity signals; then apparent speed should not depend upon the spatial arrangement of the motion elements within the pooling region. A static geometry for anisotropic pooling is also unlikely as we see enhanced integration for circular as well as linear arrays. Our data indicate that the neural integration process linking motion and position needs to be more sophisticated than previously suggested as both the motion facilitation in extended trajectories and the association field proposal requires the local motion vectors to be aligned. In the case of the Gabor elements, veridical speed requires the elements be combined to provide the global motion solution through an IOC calculation or, if unbiased, a harmonic vector average computation (Johnston & Scarfe, 2013). This level of sophistication in neural processing might be achieved by V5/MT cells as Huang, Albright, and Stoner (2007) have shown that the direction tuning of V5/MT cells to ambiguous moving edges can be influenced by the unambiguous motion of a corner outside the classical receptive field. Responses to motion carried by unambiguous moving dots were inhibited by the same unambiguous information in the extraclassical receptive field. 

To some extent, the motion-direction configuration benefit is surprising in that a moving contour of an object is as likely to be orthogonal to the direction of motion as aligned with it. We believe the computation advantage must relate to sequential processing as a contour element will likely fall on a spatial location in the direction of movement. One possibility is that receptive fields of global motion–sensing neurons may be biased dynamically, i.e., a cell that responds strongly to global horizontal motion, say, might pool information over a region that is larger in the horizontal direction than the vertical. The most likely implementation is that the motion-integration process searches ahead based on the normal component of local motion, and if another consistent element is present, the new information is integrated into the local estimate of global speed and direction, supporting integration along aligned contours in a manner reminiscent of the association field (Field et al., 1993; Ledgeway et al., 2005). This might require some interaction between motion coding in multiple areas involved in motion integration, such as area MT/V5 and areas that project to it (Bayerl & Neumann, 2004; Scarfe & Johnston, 2011). 

Acknowledgments

The authors thank the BBSRC for support. 

Commercial relationships: none. 

Corresponding author: Andrew Thomas Rider. 

Email: [e-mail]. 

Address: University College London, London, UK.