Motion-induced position shifts in global dynamic Gabor arrays

Abstract

Objects in motion appear shifted in space. For global motion stimuli we can ask whether the shift depends on the local or global motion. We constructed arrays of randomly oriented Gaussian enveloped drifting sine gratings (dynamic Gabors) whose speed was set such that the normal component of motion was consistent with a single global velocity. The array appears shifted in space in the direction of the global motion. The size of the shift is the same as for arrays of uniformly oriented dynamic Gabors that are moving in the same direction at the same global speed. Arrays made up of vertically oriented gratings whose speeds were set to the horizontal component of the random array elements were shifted less far. This shows that motion-induced position shifts of coherently moving surface patches are generated after the completion of the global motion computation.

Introduction

The human visual system needs to localize objects that move in space. However the motion of an object or motion within an object can cause a misperception of its position or of the position of objects presented nearby (Arnold, Thompson, & Johnston, 2007; Baldo & Klein, 1995; Bressler & Whitney, 2006; De Valois & De Valois, 1991; Durant & Johnston, 2004; Durant & Zanker, 2009; Fu, Shen, & Dan, 2001; Kanai, Sheth, & Shimojo, 2004; Linares, López-Moliner, & Johnston, 2007; McGraw, Whitaker, Skillen, & Chung, 2002; Mussap & Prins, 2002; Nishida & Johnston, 1999; Patel et al., 2000; Shim & Cavanagh, 2004; Whitaker, McGraw, & Pearson, 1999; Whitney, 2002; Whitney & Murakami, 1998). In this paper we focus on the apparent change in position of objects containing motion. 

A number of theories have been put forward to account for this shift. Ramachandran and Anstis (1990) demonstrated that the boundary of a patch containing random dot motion shifts in the direction of motion. They proposed that the spatial location coded by direction selective cells might be shifted in the direction of motion to anticipate the spatial displacement of objects in motion. However they left open how spatial location might be encoded. De Valois and De Valois (1991) showed that the envelope of a Gabor (a Gaussian-windowed sine grating) appears shifted in the direction of motion of the sine-wave carrier. The classical view is that position is coded by the location of activity in a cortical map. The motion-induced spatial shift provides a challenge to this view. As De Valois and De Valois (1991) pointed out, it is unlikely that striate neurons with different receptive field centers would be activated by different directions of motion in a given region of visual space. 

However, Fu, Shen, Gao, and Dan (2004) have shown that the responsivity function of cat primary visual cortical complex cell shifts in the opposite direction to a moving stimulus within their receptive field. They argue that this receptive field shift means that the moving input is seen as advanced relative to the receptive field's static position and that this adaptation explains the De Valois and De Valois (1991) contrast envelope shift. Although there is a shift in responsivity it is not clear that the boundaries of the receptive field are shifted. An alternative interpretation is that there is simply a change in responsivity or gain throughout the receptive field. The effect of this change would also be that the response of the cell would be advanced for movement relative to static flicker. This is because the leading edge of a moving bar would generate the same response before it arrives at the receptive field center as a static flickering pattern would generate if placed in the receptive field center. It is not clear at present whether this response property is a cause of the spatial shift or a consequence of the mechanism underlying the shift. For example anticipation in the retina (Berry, Brivanlou, Jordan, & Meister, 1999) as a result of reductions in contrast gain behind a motion edge could cause a shift in the population response in the direction of motion. This should bias the responsivity of a cortical cell in the direction opposite to the motion as was described by Fu et al. (2004). A similar motion-dependent shift in responsivity in macaque V4 neurons has also been reported by Sundberg, Fallah, and Reynolds (2006). 

An alternative, related, low-level model proposes that enveloped motion appears shifted due to a contrast enhancement at the leading motion edge and a contrast decrement at the trailing edge (Arnold et al., 2007; Whitney et al., 2003). Arnold et al. (2007) demonstrated a decrease in contrast threshold at the leading edge of motion in a Gaussian enveloped drifting sine grating. The leading edge of a Gaussian blurred grating appeared shifted relative to an abutting sharp-edged grating, however the internal structure appeared to remain aligned. This was taken as evidence that the DeValois and DeValois effect reflected a change in the visibility of the envelope at the margins rather than a shift in spatial location. Note that since observers judged relative displacement one cannot discount an additional spatial shift of the internal structure in both the blurred and sharp-edged stimuli. The BOLD response in primary visual cortex also appears shifted in the opposite direction to motion in the image (Whitney et al., 2003). This was interpreted as reflecting inhibition at the trailing edge of windowed motion; however, Liu, Ashida, Smith, and Wandell (2006) have more recently proposed that this is a result of a motion–direction bias in responsivity. 

Nishida and Johnston (1999) using a polar angle grating (windmill grating), which has the advantage of not containing a boundary in the direction of motion, showed that a motion trajectory appears shifted forward in space. The motion aftereffect also generated displacements in spatial location (see also Snowden, 1998). Since primary visual cortex has the most precise spatial cortical representation they proposed that the position shift might reflect feedback from extrastriate motion areas (such as V5/MT to V1). McGraw, Walsh, and Barrett (2004) reported that transmagnetic stimulation (TMS) over area V5/MT caused a reduction in the adaptation-induced position shift using drifting carrier Gabors, whereas TMS over area V1 did not. They interpreted this as evidence that the neural substrate for the position shift was in MT or above in the visual hierarchy. However additional support for this argument is needed since the position code in V1 may simply be robust to TMS. The fMRI evidence does indicate some change in neural processing in V1 although the detailed accounts of the observations remain controversial. 

Ramachandran and Anstis (1990) showed that the magnitude of their motion-induced spatial shift depended upon whether the target area appears in front or behind a background. They saw this as evidence that higher level processes such as those contributing to figure–ground assignment can modulate the spatial shift. This role for grouping processes is supported by evidence that the interpretation of bistable motion can alter perceived motion drag (Shim & Cavanagh, 2004). Position shifts are also found in second-order stimuli such as contrast-defined or motion-defined contours (Bressler & Whitney, 2006; Durant & Zanker, 2009) and global motion random dot displays (Mussap & Prins, 2002). In addition, Edwards and Badcock (2003) reported a perceptual shift in perceived depth for expanding and contracting motion fields consistent with the conclusion that the objects appear shifted in the direction of motion in 3D space. 

Recently, Hisakata and Murakami (2009) and Mather and Pavan (2009) have independently shown that the spatial shift for a plaid pattern is similar to that of a Gabor pattern oriented orthogonally to the direction of motion but larger in magnitude to each of the components measured separately. This indicates that the spatial shift depends on processing subsequent to 2D motion computation. However, V5/MT neurons respond differently to superimposed gratings and spatially separate grating (pseudoplaids) in their receptive fields (Majaj, Carandini, & Movshon, 2007). One interpretation of this is that 2D motion is not accomplished simply by the combination of the outputs of 1D motion processing. Hisakata and Murakami (2009) show that the spatial shift can arise from global Gabor arrays but the magnitude of the spatial shift in random-oriented arrays and parallel arrays with the same global motion were not compared directly. 

Dynamic global Gabor arrays provide a means of investigating mid-level motion mechanisms and their role in spatial vision. Individual static envelope dynamic carrier Gabor patches are seen to move in the direction normal to their carrier orientation, and their envelopes are also perceived as shifted in this direction. However, randomly oriented dynamic Gabors that all move with a drift speed that is consistent with a single global motion can cohere into a single surface, which appears to move in the global motion direction (Amano, Edwards, Badcock, & Nishida, 2009). Motion information is integrated across space and orientation to give a perception of rigid global translation. We can ask whether the interaction between perceived position and motion takes place before or after this global pooling stage. To appear as rigid global motion the speed of each element should be a cosine function of the angle between its normal component and the global motion direction. Suppose each Gabor element is perceived as shifted in the normal direction that is a distance proportional to the speed of drift then we would expect only those elements with carriers that move in the global motion direction to be maximally shifted in the global motion direction. The other elements should not appear shifted at all or shifted to a lesser degree in random directions. On this basis the aggregate shift should be smaller than the shift for an array containing Gabor elements orientated orthogonally to the global motion direction. 

The first experiment demonstrates that the magnitude of the position shift for an array of randomly oriented dynamic Gabors, whose speeds are chosen to conform to a single global horizontal motion, is the same as that found for a dynamic Gabor array in which all the Gabors are oriented orthogonally to the global motion direction. The second experiment shows that the position shift of the array is larger than the aggregate shift of the individual elements in the direction of the global motion. In the third experiment we show that an array of vertically oriented Gabors with the same horizontal velocity components as displayed in the first experiment produce a smaller position shift than in Experiment 1. In this case the local motion is not consistent with a single 2D global motion. We conclude that in the case of Gabor arrays the position shift reflects the velocity of the global motion computation. 

General methods

Stimuli were generated on a PC (Dell Precision 380) running Matlab R2008a and displayed via a Cambridge Research Systems ViSaGe stimulus generator on a Mitsubishi Diamond Plus 230SB monitor with a frame rate of 100 Hz. Viewing was binocular from a distance of 57 cm. Head stability was maintained by a chin rest. Observers were instructed to fixate a bull's-eye in the center of the display throughout the experiment. A total of seven subjects took part in some or all experiments, including one of the authors and 6 naive but experienced observers. All reported normal or corrected-to-normal vision. Some data were collected while the first author was in Japan (NTT Communication Science Laboratories) using a similar CRS setup. 

Stimuli consisted of 98 Gabor patches (drifting sinusoidal luminance gratings windowed by stationary 2D Gaussian functions) arranged in two 7 by 7 square grids, one above and one below fixation (see Figure 1). The center of each patch ranged from 4 to 10 degrees from fixation in the vertical direction and between −3 and +3 degrees in the horizontal direction. The spatial frequency of the sine gratings was fixed at 2 cpd; contrast was 50% and the standard deviation of the Gaussian function was 0.2 degree with each patch restricted to a 1 degree square. Movie sequences consisted of 27 frames, each shown twice at a frame rate of 100 Hz, giving a total stimulus duration of 540 ms and an update rate of 50 Hz. 

Figure 1

 

The dimensions of the stimuli used in these experiments.

The apparent position of the array was measured using a method of constants binary choice task. On each trial the sine gratings in the top array of Gabors drifted either to the left or to the right of the vertical (chosen at random), with the lower array always drifting in the opposite direction. Initial phases of each sine grating were chosen at random. The two arrays were physically offset in 9 steps by between −8 and 8 pixels (±20 arcmin) in the horizontal direction. A random amount of horizontal jitter (between −5 and 5 pixels (±12.5 arcmin)) was added to both arrays in order to ensure that the task could not be completed by comparing one array with the fixation spot. The relative position of both arrays had to be considered. On each trial an offset was chosen at random and subjects had to indicate by button press whether the top array appeared to be positioned to the right or left of the lower array. A psychometric function (cumulative Gaussian) was fitted to each individual's data and the 50% point taken as the perceived position shift. We report the displacement of one array rather than the difference in position of the two arrays. Estimates of standard deviations were obtained via bootstrapping. 

Experiment 1

In order to test whether the perceived shift was determined by the global motion of the array or by an aggregate shift of the elements, the first experiment investigated whether there was any difference in the perceived shift of arrays composed of vertically oriented dynamic Gabors and randomly oriented dynamic Gabors when both had the same global horizontal motion. If the position shift is solely due to low-level mechanisms we would expect the random condition, having less average horizontal motion, to be shifted considerably less than the parallel condition. 

Methods

A total of seven subjects participated in Experiment 1, including one author and six subjects who all had significant experience in psychophysical tasks but were naive as to the purpose of this study. In one condition the orientation of every sine grating was set to vertical (parallel condition, see Figure 2), and in the other condition (random condition, see Figure 2) the orientation of each grating was chosen at random from one of 18 orientations ten degrees apart, ranging from 0 to 170 degrees from vertical. The speed of the elements in the random orientation array was set to the global motion speed times the cosine of the angle between the global motion direction and the direction normal to the contours of the Gabor. Each session consisted of a single condition, with twenty repetitions of each of the 9 horizontal offsets (180 trials in total). Each observer conducted two sessions of both conditions. 

Figure 2

 

The perceived position shift for 7 subjects in the parallel and random conditions of Experiment 1. Error bars denote 1 SE. Open symbols are individual's data, filled symbols and solid line denote means. (Left) An example of the arrays used in the parallel condition of Experiment 1. (Right) An example of the random condition of Experiment 1.

Results

Figure 2 shows the perceived position shift of the seven observers and the mean of these, for the two conditions. All subjects show a spatial shift in the expected direction although there is some variation in the magnitude of the shift between subjects. There is no difference between the shifts in the parallel condition and in the random condition. The mean shift for the parallel condition was 8.25 arcmin ( SE = 0.70) and for the random condition it was 7.39 arcmin ( SE = 0.70), p = 0.22. The magnitude of the shift for vertically aligned gratings is close to that reported by De Valois and De Valois (1991) for individual Gabors at the minimum eccentricity used here (4 degrees; approximately 8 arcmin, see Figure 2 of De Valois & De Valois, 1991). 

Discussion

The results of Experiment 1 shows that the perceived shift of an array of drifting Gabors depends on the global velocity field and not what would be expected from shifts in the local elements. This does not necessarily rule out the possibility of interaction between motion and position at a more local low-level motion measurement stage, but it would then require a mechanism that adjusts the output of any such process in light of the global motion information. There was no obvious spatial disorganization of the array, which suggests a coordinated shift of the elements as well as the boundaries of the array. 

Experiment 2

Previous experiments on the position shift of Gabor stimuli have used single dynamic Gabor patches and measured the shift orthogonal to the grating. In the random condition of Experiment 1 we used several different orientations but instructed subjects to only make judgments about the position of the arrays in the horizontal direction. To assess the expected perceived horizontal displacement of the elements we constructed global arrays containing Gabors with a single, non-vertical, orientation. The local and global motions of these arrays are identical. We measured the horizontal shift of these arrays. We would expect the horizontal shift to reflect the horizontal component of motion. Previous reports have provided various results about how the size of spatial shifts vary with motion parameters. Nishida and Johnston (1999) found the shift in the center of a motion trajectory to be proportional to velocity. Whitaker et al. (1999) reported that the change in position of the boundary of radial gratings was proportional to the square root of the speed. The shift of single Gabors has been reported as tuned to temporal frequency (De Valois & De Valois, 1991). Chung, Patel, Bedell, and Yilmaz (2007) found that the form of the displacement by velocity function can depend on duration, speed, spatial frequency and visual eccentricity. For durations of around 500 ms and an eccentricity of 4 degrees of visual angle, displacement is a saturating function of speed (Chung et al., 2007). Hisakata and Murakama (2009) report a small but significant increase in position shift of a Gabor array with global motion speed although they also report individual subject differences. 

Methods

Three subjects who took part in the first experiment also took part in the second experiment, including one author. Orientations of 0, 22.5, 45, 67.5 and 90 degrees from horizontal were tested in separate blocks, to avoid any possible order effects. Within each block the orientation angle of every Gabor was chosen as clockwise or anticlockwise of horizontal at random on a trial-by-trial basis. Arrays were given the same underlying horizontal drift speed as in Experiment 1, so that the 90 degree condition was identical to the parallel condition in Experiment 1 and the 0 degree condition simply comprised static horizontal Gabors. The drift speed of the intermediate orientations was proportional to gcos( θ), where g is the global motion speed, and θ is the orientation between the direction normal to the Gabor and the global motion direction. The horizontal component is gcos ²( θ) (see Figure 3). All other parameters were as in Experiment 1. 

Figure 3

 

The perceived horizontal shift for various orientations of the Gabor arrays. The black line indicates the linear fit, the dashed line is the sine fit and the dotted line is the sine ² fit. Note that all fits were constrained to pass through (0, 0). The horizontal component of a velocity consistent with rightward motion is shown in the diagram.

Results

Figure 3 shows the perceived shift in the horizontal direction against orientation for three subjects. An outlier was removed in the 67.5 degree condition; therefore this mean is based on data for two subjects. The shift increases monotonically with orientation, from no shift at 0 degrees (mean = −0.05 arcmin, SE = 0.2) to 8.05 arcmin ( SE = 0.29) at 90 degrees. While these data are well fitted by a straight line (black line, adjusted R-squared = 0.981) it is also reasonably well fit by a sine-squared function (dotted line, adjusted R-squared = 0.908). However, if we assume a square root law between position shift and the horizontal speed component we get a closer fit to the spatial shift data (dashed line, adjusted R-squared = 0.987). Thus the data suggest that the horizontal spatial shift for oriented Gabors is consistent with a shift in the horizontal direction, which is proportional to the square root of the velocity, or equivalently the horizontal component of a shift in the direction of motion, which is proportional to the square root of velocity. 

Discussion

The results of Experiment 2 suggest that when asked to judge the horizontal offset of arrays of Gabors subjects can perform the task well and the shift is approximately proportional to the square root of the horizontal component of the motion. This confirms the square root relationship between velocity and the magnitude of the shift reported by Whitaker et al. (1999). The arrays were not perceived as moving horizontally. Therefore the results of Experiment 2 are best explained by a shift in the direction normal to the Gabor orientation with subjects reporting on the horizontal component of this shift. Viewed in isolation the individual orientations are perceived as shifted in the amount expected from a local analysis, but when placed in the context of a global horizontal translation the shift is considerably greater than we might expect from, say, averaging the horizontal components. 

Experiment 3

It is clear from Experiment 2 that arrays of parallel Gabors appear shifted in the horizontal direction by an amount approximately proportional to the square root of the horizontal component of the drift velocity. In the random condition of Experiment 1 on an element-by-element basis we would expect individual Gabors to be shifted by different amounts. However the array did not appear to manifest the spatial disorganization that this would imply. It is possible the patches are indeed shifted by different amounts but global perception is captured by the motion-induced shift of the vertically aligned Gabors, which, having the highest drift speed, will be shifted the most (see also Hisakata and Murakami, 2009 for a discussion of this point). To test this we used arrays of Gabors that had the same distribution of horizontal velocities as in the random condition of Experiment 1. In this case the horizontal distribution of velocities is the same as in Experiment 1, but because the orientation is uniform the array in not consistent with a single 2D velocity. 

Methods

The same three observers participated in Experiment 3 as in Experiment 2. Stimuli were composed of vertically aligned Gabors and the drift speeds, rather than being constant as in the parallel condition of Experiment 1, were chosen from the same distribution of horizontal components in the random condition, i.e., gcos ²( θ), where g is the global drift speed used in Experiment 1, and θ is chosen at random from the 18 orientations used in the random condition. All other methods are as in Experiment 1. 

Results and discussion

The stimuli did not appear perceptually coherent, as might be expected given the broad range of velocities, and the fact the velocity distribution did not have a single global solution. Figure 4 shows the perceived position shift, averaged across the three subjects, compared with the perceived shift for these same three subjects in the two conditions of Experiment 1. There is little difference between the shifts in Experiment 1 (mean ( SE): parallel condition = 8.05 ( SE = 0.29), random condition = 8.39 ( SE = 1.08)). The shift in Experiment 3 = 5.64 ( SE = 0.25) is present but is considerably smaller. So while there is an identical distribution of horizontal components in this experiment and the random condition of Experiment 1, we do not see a comparable shift. 

Figure 4

 

The magnitude of the position shift of the two conditions in Experiment 1 on the left and that of Experiment 3 on the right.

General discussion

We asked whether apparent spatial shifts in global motion arrays could be accounted for by the aggregate shifts of the local elements. The data clearly point to the need to incorporate a global motion stage in any explanation of the apparent shift of the array. This global shift cannot easily be accounted for by local shifts in responsivity in the retina or visual cortex (Berry et al., 1999; Fu et al., 2004; Sundberg et al., 2006). It is also difficult to account for the shift on the basis of the visibility of the local envelopes (Arnold et al., 2007; Whitney et al., 2003) although spatial shift might occur at various levels of the visual system and these mechanisms may play a role in the perceived location of single elements. 

The spatial shift in Experiment 3 was much less than in the random orientation condition of Experiment 1 even though the horizontal components of motion were equivalent. This indicates that the spatial shift is greater when the array determines a particular global motion. However there was still a significant shift in the direction of the average motion of the array even though it did not appear to cohere. This may be due to some patchy coherence in the array for velocities that are similar and close together or some mixture of global and local effects. Nevertheless it indicates that perfect coherence is not essential to the generation of apparent spatial displacement of the array. Amano et al. (2009) report that a plaid array with velocities set to match the velocity components in the global motion direction of a coherent Gabor array appears to move more slowly than the Gabor array. Therefore the perceived speed of the Gabor array in Experiment 3 should also appear less than in Experiment 1, which might mediate the reduction in apparent displacement. 

It is generally considered that area V5/MT plays a significant role in solving the aperture problem (Born & Bradley, 2005; Huang, Albright, & Stoner, 2007; Huk & Heeger, 2001; Majaj et al., 2007; Movshon, Adelson, Gizzi, & Newsome, 1985; Pack & Born, 2001; Perrone & Krauzlis, 2008; Rust, Mante, Simoncelli, & Movshon, 2006; Smith, Majaj, & Movshon, 2005). Recently, Mather and Pavan (2009) have shown that drifting plaids are perceptually shifted in space in a manner that is consistent with either the IOC or the vector sum of the shifts elicited by the individual component gratings. This has been confirmed by Hisakata and Murakami (2009) who went on to show that the shift arising from spatially segregated oriented gratings (pseudoplaid pattern) is generally larger than expected from the average shift of the individual components. In Experiment 1 we showed that the shift seen with these kinds of arrays is equal to that expected by an IOC construction, and hence that the shift is not only larger than expected from the local motion but crucially depends on global motion. The maximal spatial shift must come after the resolution of the aperture problem for spatially separate components. However, it remains open whether velocity signals computed at the global motion stage, perhaps in MT, are fed back to the representation of elements at an earlier stage in visual processing. If so this might explain why coherent dynamic Gabor arrays still look spatially regular. Ramachandran and Anstis (1990) demonstrated that motion-induced shifts appeared larger for surfaces that appeared as figure rather than ground. If spatial shifts are to play a functional role in the representation of the position of moving surfaces and objects then the observation that the magnitude of the shift depends on the computation of global motion, which attributes diverse local measures to a single global cause, makes perfect sense. 

Conclusion

Global arrays made up of Gaussian-windowed drifting sine gratings appear shifted in space in the direction of motion, as is the case for single Gabor elements. There is some degree of shift for arrays that move incoherently in a particular direction, perhaps indicating a role for local mechanisms. The shift for arrays made up of randomly oriented Gabors whose carrier speed is set to be consistent with a single global motion is equal to that of a uniformly oriented array indicating that the shift in these arrays must have been generated after the computation of global motion. 

Acknowledgments

Supported by the BBSRC and EPSRC. 

Commercial relationships: none. 

Corresponding author: Andrew Rider. 

Email: [e-mail]. 

Address: Cognitive, Perceptual and Brain Sciences Research Department, 26 Bedford Way, London, WC1H 0DS, UK.