Evidence for reciprocal antagonism between motion sensors tuned to coarse and fine features

Abstract

Early visual processing analyses fine and coarse image features separately. Here we show that motion signals derived from fine and coarse analyses are combined in rather a surprising way: Coarse and fine motion sensors representing the same direction of motion inhibit one another and an imbalance can reverse the motion perceived. Observers judged the direction of motion of patches of filtered two-dimensional noise, centered on 1 and 3 cycles/deg. When both sets of noise were present and only the 3 cycles/deg noise moved, judgments were reversed at short durations. When both sets of noise moved, judgments were correct but sensitivity was impaired. Reversals and impairments occurred both with isotropic noise and with orientation-filtered noise. The reversals and impairments could be simulated in a model of motion sensing by adding a stage in which the outputs of motion sensors tuned to 1 and 3 cycles/deg and the same direction of motion were subtracted from one another. The subtraction model predicted and we confirmed in experiments with orientation-filtered noise that if the 1 cycle/deg noise flickered and the 3 cycles/deg noise moved, the 1 cycle/deg noise appeared to move in the opposite direction to the 3 cycles/deg noise even at long durations.

Introduction

This paper characterizes an apparently catastrophic perceptual failure that occurs when the visual system is required to interpret motion signals extracted from fine image features in the presence of nonconflicting information extracted from coarse image features. Models of human visual motion sensing have long assumed that the basic motion sensor is selective for spatial frequency, orientation, and location (Adelson & Bergen, 1985; Watson & Ahumada, 1985) so that the detection of the motion of fine and coarse image features, such as the raising of an eyebrow or the movement of a head, is supported by the same processing operations carried out in detectors tuned to different spatial frequencies and orientations. Experiments using spatial summation and masking support the view that motion sensors in the human visual system are similar at coarse and fine scales, being selective for spatial frequency with localized receptive fields (Anderson & Burr, 1987, 1989, 1991; Anderson, Burr, & Morrone, 1991). 

Given that signals from different spatial scales have been separately analyzed, it is theoretically possible to combine the results in ways that facilitate some tasks at the expense of others. The results of experiments in which human observers are required to discriminate the direction of motion of very brief stimuli suggest that this occurs: Humans make systematic errors in discriminating the direction of motion of stimuli that contain gratings of two different spatial frequencies (Derrington, Fine, & Henning, 1993; Derrington & Henning, 1987a; Henning & Derrington, 1988). However, these results are difficult to interpret because the grating patterns that were used are one-dimensional and periodic and their combinations contain predictable features that might be expected to generate anomalous responses (Badcock & Derrington, 1985; Burton, 1973; Henning, Hertz, & Broadbent, 1975). For this reason, we decided to investigate how the visual system synthesizes motion signals across spatial scales by requiring observers to discriminate the direction of motion of spatial frequency band-limited but aperiodic stimuli: We used patches of moving noise filtered to contain different bands of spatial frequencies. 

Methods

General

All stimuli were presented on a gamma-corrected 19-in. monitor (Mitsubishi Diamond pro 2020U) under the control of an Apple Macintosh G5 running Matlab (MathWorks Ltd.) using the Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). The monitor had a resolution of 1,024 × 768 pixels (horizontal × vertical) with vertical frame rate of 120 Hz and a mean luminance of 45.3 cd/m². The stimuli were presented in white mode at the center of the monitor screen in a square of 20 cm per side and were viewed at a distance of 143 cm subtending an area of 8° × 8°. The remainder of the screen was at mean luminance. The display spatial resolution was 64 pixels per degree of visual angle. 

Subjects

Three human subjects, two males (IS and AD) and one female (ER), took part in the experiments. The subject ER was not aware of the purpose of the study. All subjects had normal or corrected-to-normal refraction and normal visual acuity and viewed the screen binocularly with natural pupils. The experiments were carried out in a dark room and a chin rest (UHCOTech HeadSpot) was used to stabilize the subject's head and to control the observation distance. To minimize tracking eye movements, the subjects were instructed to maintain fixation on a small cross (0.25° × 0.25°) in the center of the screen before presenting the stimuli. Experimental procedures were approved by the Psychology Ethics Committee of Newcastle University. 

Stimuli

Digital images with 512 × 512 pixels with 8-bit range were constructed using Matlab. In Experiments 1 and 2, anisotropic noise and isotropic noise stimuli were used, respectively (see examples in Figures 1a and 2a). The equation of a complex moving noise is described as follows:  

$L(x,y,t)=L_0\left\{\begin{array}{l}1+m\left(t\right)\exp (-\frac{x^2+y^2}{2{\sigma }_{xy}^2})\\ \times \left[\begin{array}{l}{m}_1\left(t\right)\times n_1(x-{\upsilon }_{1}t,y)\\ +m_2\left(t\right)\times n_2(x-{\upsilon }_{2}t,y)\end{array}\right]\end{array}\right\},$

(1)

where n ₁ and n ₂ are the filtered Gaussian noises (anisotropic or isotropic) with peak spatial frequencies of 1 and 3 cycles/deg, respectively (the equations of the filters are given below); L ₀ is the mean luminance, in cd/m ²; σ _xy is the spatial standard deviation, in deg ( σ _xy = 2°); m is the Michelson contrast as a function of time given by m( t) = exp {− t ² / (2 σ _t ²)}, where σ _t is the temporal standard deviation; υ ₁ and υ ₂ are the velocities of each noise, in deg/s; and m ₁ and m ₂ are the contrasts which are chosen to ensure that both filtered noises have equal contrast energy. 

Figure 1

 

Results from Experiment 1. (a) Example of orientation-filtered noise. The image is the sum of two octave-wide bands of noise centered on 1 and 3 cycles/deg. The orientation-filtered noise has an orientation bandwidth of 30° and a spatial Gaussian window with σ = 2°. The RMS contrasts were 0.0374 for each scale and 0.051 for the combined image. (b) Fourier amplitude spectrum. (c) Thick line: amplitude spectrum profile. Thin line: profile of the Gabor filters used to construct the noise. (d–f) Horizontal direction discrimination performance for three observers as a function of the duration of the temporal Gaussian window (duration = 2 σ _t). Open circles, stationary 1 cycle/deg noise with moving 3 cycles/deg noise; open squares, moving 1 cycle/deg noise with moving 3 cycles/deg noise; open triangles, moving 1 cycle/deg noise with stationary 3 cycles/deg noise; solid triangles, moving 1 cycle/deg noise; black circles, moving 3 cycles/deg noise. Moving components had a fixed speed of 4 deg/s. There were 25 observations per point per subject. The shaded area marks the 95% confidence limits of the mean performance expected by chance, assuming binomial variability (±1.96 σ).

Figure 2

 

Results from Experiment 2. (a) Example of isotropic-filtered noise. The image is the sum of two octave-wide bands of isotropic noise centered on 1 and 3 cycles/deg. The image was windowed by a two-dimensional spatial Gaussian function with σ = 2°. The RMS contrasts were 0.0748 for each scale and 0.108 for the combined image. (b) Fourier amplitude spectrum. (c) Thick line: amplitude spectrum profile. Thin line: profile of the Gabor filters used to construct the noise. (d–f) Horizontal direction discrimination performance for three observers as a function of the duration of the temporal Gaussian window (duration = 2 σ _t). Open circles, stationary 1 cycle/deg noise with moving 3 cycles/deg noise; open squares, moving 1 cycle/deg noise with moving 3 cycles/deg noise; open triangles, moving 1 cycle/deg noise with stationary 3 cycles/deg noise; solid triangles, moving 1 cycle/deg noise; black circles, moving 3 cycles/deg noise. Moving components had a fixed speed of 4 deg/s. There were 25 observations per point per subject. The shaded area marks the 95% confidence limits of the mean performance expected by chance, assuming binomial variability (±1.8 σ).

To construct the anisotropic noise, we made a two-dimensional Gaussian white noise and then this noise was filtered using the following anisotropic Gaussian filter  

$\begin{array}{cc}& \left|H\right(u,v\left)\right|=\left[\begin{array}{l}\exp \{-2{\pi }^2{\sigma }_u^2(u-{\rho }_0{)}^2\}\\ +\exp \{-2{\pi }^2{\sigma }_u^2(u+{\rho }_0{)}^2\}\end{array}\right]\\ \\ & \times \exp \{-2{\pi }^2{\sigma }_v^2{v}^2\}\end{array},$

(2)

where the spreads of the Gaussian filter σ _u and σ _v were obtained by the following equations  

${\sigma }_u=\frac{\sqrt{\log \left(2\right)}(1+2^B)}{{\rho }_0\sqrt{2}\pi (2^B-1)},$

(3)

 

${\sigma }_v=\frac{\sqrt{\log \left(2\right)}}{{\rho }_0\sqrt{2}\pi \tan (\alpha /2)},$

(4)

where B = 1 octave (full width at half-height); α = 30° (full width at half-height); and the center frequency ρ ₀ of the filter was 1 cycle/deg for the low frequency noise and 3 cycles/deg for the high frequency noise. Then, we need to calculate the contrast ( m ₁ and m ₂) for the filtered images I to equate both images (low and high) in energy. To obtain the value of m, we only need to know the root mean square contrast ( c _RMS) value for each noise; in this case, the RMS contrast was 0.0374. The equation (Serrano-Pedraza & Sierra-Vazquez, 2006) used was 

$m=\frac{I_0{\mathrm{c}}_{\mathrm{R}\mathrm{M}\mathrm{S}}}{\sqrt{a-b^2}+{\mathrm{c}}_{\mathrm{R}\mathrm{M}\mathrm{S}}(I_0-b)},$

(5)

with I₀ = 128, where 

$a=\frac{\sum \sum I^2(x,y)}{N^2},$

(6)

 

$b=\frac{\sum \sum I(x,y)}{N^2}.$

(7)

 

To construct the isotropic noise, we made a two-dimensional Gaussian white noise and then this noise was filtered using the following isotropic Gaussian filter  

$\left|H\right(u,v\left)\right|=\exp \{-2{\pi }^2{\sigma }_{}^2{(\sqrt{u^2+v^2}-{\rho }_0)}^2\},$

(8)

where the spread, σ, of the Gaussian filter was obtained by the equation  

$\sigma =\frac{\sqrt{\log \left(2\right)}(1+2^B)}{{\rho }_0\sqrt{2}\pi (2^B-1)},$

(9)

where B = 1 octave (full width at half-height) and the center frequency ρ ₀ of the filter was 1 cycle/deg for the low frequency noise and 3 cycles/deg for the high frequency noise. The contrast m of each noise was calculated as described above and the RMS contrast was 0.0748 for each noise (low and high). 

In Experiment 3, two types of anisotropic noise were used. One of them is described by Equation 1, the second is as described above but with the low frequency component flickering at 3 Hz. The equation of this second type of noise moving is as follows  

$L(x,y,t)=L_0\left\{\begin{array}{l}1+m\left(t\right)\exp (-\frac{x^2+y^2}{2{\sigma }_{xy}^2})\\ \times \left[\begin{array}{l}{m}_1\left(t\right)\times \cos (2\pi 3t)\times n_1(x-{\upsilon }_{1}t,y)\\ +m_2\left(t\right)\times n_2(x-{\upsilon }_{2}t,y)\end{array}\right]\end{array}\right\},$

(10)

where symbols have the same meaning as in the Equation 1. 

To construct different moving noises, we constructed movies of 60 samples ( Experiments 1 and 2) or 120 samples ( Experiment 3). In each frame, each sample of noise had the same contrast energy. A different stochastic noise sample was used in each trial. 

Procedure

In all experiments, each trial started with a fixation cross displayed at the center of the screen using a Gaussian temporal envelope with standard deviation of 80 ms truncated to give an overall duration of 500 ms. 

In Experiments 1 and 2, five different types of stimuli were used. For both anisotropic and isotropic noises, we used three complex stimuli: stationary 1 cycle/deg noise with moving 3 cycles/deg noise; moving 1 cycle/deg noise with moving 3 cycles/deg noise both with the same motion direction; moving 1 cycle/deg noise with stationary 3 cycles/deg noise; and two simple stimuli: moving 1 cycle/deg noise and moving 3 cycles/deg noise. Moving components had a fixed speed of 4 deg/s. The complex stimuli were displayed using a temporal Gaussian envelope with a standard deviation of σ _t ∈ {12.5, 17.67, 25, 35.35, 50, 70.71, 100} ms, the simple stimuli were displayed with a standard deviation of σ _t ∈ {12.5, 25, 50, 100} ms. The temporal envelope was truncated to obtain the overall duration of 500 ms. The motion direction, left or right, was randomized and the observer's task was to indicate, by pressing a mouse button, the direction they saw on each presentation. A new trial was initiated only after the observer's response, thus the experiment proceeded at a pace determined by the observer. For each stimulus and duration, 25 presentations were required. No feedback about the correctness of responses was provided. 

In Experiment 3, two different types of anisotropic stimuli were used. A moving 1 cycle/deg noise with variable speed added to a moving 3 cycles/deg noise with fixed speed of 4 deg/s; and a 1 cycle/deg noise flickering at 3 Hz and moving with variable speed and added to a moving 3 cycles/deg noise with fixed speed of 4 deg/s. Stimuli of the first type were displayed using a temporal Gaussian envelope with a standard deviation of σ _t ∈ {12.5, 25, 50, 100} ms, stimuli of the second type were displayed with a standard deviation of σ _t ∈ {100, 200} ms. The temporal envelope was truncated to obtain the overall duration of 1 s. 

The motion direction of the 3 cycles/deg noise, left or right, was randomized and the observer's task on each presentation was to indicate, by pressing a mouse button, the direction of the 1 cycle/deg noise component. 

The speed of the 1 cycle/deg noise component was altered by an adaptive staircase with fixed step of 0.5 deg/s for standard deviations of 50 and 100 ms, 1 deg/s for 25 ms, and 4 deg/s for 12.5 ms. Three interleaved staircases of 40 trials were running for each stimulus and duration. 

A cumulative normal distribution function was fitted by maximum likelihood (Watson, 1979) to the proportion of correct responses taking the three staircases together. The 50% point of the fitted psychometric function was taken as the cancellation speed (see Figure 4). 

Statistical analysis

The confidence intervals of Figure 5 were obtained by simulating 2,000 cancellation experiments using the same adaptive procedure as the subjects. The random numbers were obtained using the algorithm of Wichmann and Hill (1982). A cumulative normal distribution function with the subject's parameters, estimated from the experiments, was used as a model of the observer in the simulations (see Figure 4). In each simulation of the experiments, the same normal cumulative function was fitted by maximum likelihood (Watson, 1979) in order to obtain the 50% point of the psychometric function as the cancellation speed. The central 90% range was obtained from the distribution of these 2,000 estimated cancellation speeds. 

The confidence intervals of Figure 6 were obtained as described above but in this case, instead of using the adaptive procedure, the method of constant stimuli was used because it was the method used to obtain the cancellation speeds using the model of motion discrimination. 

Results

Experiments 1 and 2: direction discrimination using filtered noise: perceptual reversals

Figure 1 illustrates the catastrophic failures in motion perception that occur when fine scale, vertically oriented moving features are presented together with coarse scale features. Panels a, b, and c show a sample of the orientation-filtered noise stimulus together with the two-dimensional spatial frequency spectrum and the horizontal profile of its amplitude spectrum. Panels d, e, and f show results from three different observers, each of which produces the same general pattern of responding. Each plot shows performance of one observer discriminating the direction of motion of five different combinations of fine scale (3 cycles/deg) and coarse scale (1 cycle/deg) noise, plotted against the duration for which the stimulus was presented. 

For every observer and for every stimulus, performance is effectively perfect at the longest durations. The interesting variations occur at the shorter durations. Solid symbols show performance when only one kind of noise is present and provide a baseline for interpreting the results when both sets of noise are present. 

For all three observers, the solid symbols show the expected pattern of results: Performance is close to chance at the shortest durations, improves with increasing duration and is perfect at all durations from 100 ms upward. 

For two of the observers, performance is slightly better with the higher spatial frequency (circles). This difference is expected because the higher spatial frequency produces a higher temporal frequency, which, other things being equal, should result in a greater difference between rightward and leftward motion signals. 

When both sets of noise are present, very different results are obtained depending on whether the high spatial frequency noise moves. When only the low spatial frequency noise moves, performance is comparable to that obtained with only one set of noise present, indeed it is slightly better in two of the observers. However, when the high spatial frequency noise moves, if the low frequency noise is static, all observers respond as if they see reversed motion; however, if the low frequency noise is moving (in which case it moves in the same direction and at the same speed as the high frequency noise), performance is much worse and is below chance at the shortest durations. The fact that reversed motion sometimes occurs when both noise bands move is very surprising because either band, if it were present alone, would be seen to move correctly. 

Neither the impairment in discrimination that occurs when the fine scale features move with the coarse scale features moving nor the reversal that occurs when they move with the coarse scale features static would be predicted from current models of motion perception, although both phenomena have been observed using sinusoidal gratings instead of oriented noise (Derrington et al., 1993; Derrington & Goddard, 1992; Derrington & Henning, 1987b; Henning & Derrington, 1988). The fact that both the gratings that have been used previously and the vertically oriented noise we used here consist either exclusively or predominantly of vertically oriented features means that motion analysis is biased toward producing directions of motion close to a horizontal axis because vertical features primarily stimulate motion sensors tuned to leftward or rightward motion. We therefore felt it would be informative to carry out a similar experiment using isotropic noise filtered into the same two spatial frequency bands as those used in Experiment 1. The results of this experiment are shown in Figure 2. Figures 2a–2c show a sample of the isotropic noise, its two-dimensional spatial frequency spectrum, and the profile of the spectrum along the horizontal axis. The difference between these and the corresponding panels in Figure 1 is quite striking. The dark and the light blobs in Figure 2a are much less elongated than the vertical blobs in Figure 1a and show no tendency to lie at any particular orientation. The spectral components in Figure 2b are scattered throughout two annular patches, indicating that the components cover all orientations but are limited to two narrow ranges of spatial frequencies corresponding to the radii of the annuli, whereas those in Figure 1b are clustered in paired patches close to the horizontal axis indicating that the components are all close to vertical in orientation and are clustered around the same two spatial frequencies. The spectral profiles in Figures 1c and 2c carry no information about orientation because they are one-dimensional and so they are very similar to one another. 

Despite these differences in the stimuli, Figures 2d– 2f show that the most distinctive feature of the results, the reversal in direction that occurs when fine scale components move and coarse scale components are static, is retained in all three observers although the results with anisotropic noise show slightly stronger reversals and the reversals when both noise bands move are slightly stronger too. In spite of these small differences, the results with isotropic noise indicate clearly that stimulus orientation is unlikely to be crucial for the surprising perceptual reversals and makes it possible to examine a potential explanation using space–time plots and spatiotemporal frequency spectra. 

An explanation of the perceptual reversals using space–time plots and spatiotemporal frequency spectra

Figure 3a shows a space–time plot of a sample from the stimulus set whose motion is most consistently reversed, static 1 cycle/deg noise added to moving 3 cycles/deg noise, presented for a duration of 25 ms. The plot shows the temporal profile of the stimulus on the vertical axis and the horizontal spatial profile on the horizontal axis. The stimulus appears as narrow, horizontally aligned patch of vertical stripes. Close inspection reveals that there are two sets of stripes coarse, low spatial frequency stripes, which truly are vertical and finer high spatial frequency stripes, which are oriented obliquely upward to the right. This corresponds to the fact that the low spatial frequency components of the stimulus are stationary and the high spatial frequency components are moving rightward. Figure 3b shows the space–time plot of a stimulus that is identical except for its duration, which is 200 and whose motion is always seen correctly. The plot of the longer duration stimulus extends much farther along the time axis and it is easier to see that the coarse stripes are vertical and the fine stripes are oblique. If we examine the frequency domain representations of these two space–time plots, which are shown in Figures 3d and 3e, we can see where the reversals in perceived direction might originate. 

Figure 3

 

Space–time plots and spatiotemporal frequency spectra of stimuli with moving fine scale components and static or flickering coarse scale components. (a) Space–time ( X– T) plot of a stationary 1 cycle/deg noise added to a moving 3 cycles/deg noise with a duration of 25 ms. (b) X– T plot of a stationary 1 cycle/deg noise added to a moving 3 cycles/deg noise with duration of 200 ms. (c) X– T plot of a 1 cycle/deg noise flickering at 3 Hz added to a moving 3 cycles/deg noise with duration of 200 ms. Moving components had a fixed speed of 4 deg/s. (d) Spatiotemporal spectrum (U-TF) of image (a). (e) U-TF plot of image (b). (f) U-TF plot of image (c). Components in quadrants 1 and 3 represent rightward motion and those in quadrants 2 and 4 represent leftward motion.

Figure 3d shows the spatiotemporal frequency spectrum of the briefly presented stimulus, which is perceived to move backward, and Figure 3e shows the spectrum of the longer duration stimulus, which is perceived veridically. Spatial frequency components are reflected symmetrically about the ordinate, with those that are static plotted at zero temporal frequency and those that are moving plotted on a line that is rotated away from the zero temporal frequency line with the angle of the rotation corresponding to the velocity. Thus, stimuli that are moving to the right plot in the first and third quadrants and those that are moving to the left plot in the second and the fourth quadrants. These features are easy to see in Figure 3e in which the low frequency noise components are closely clustered on the zero temporal frequency line and the high frequency components plot on a line whose slope corresponds to 4 Hz per cycles/deg or 4 deg/s. They are less easy to see in Figure 3d because the very short duration causes the components to be smeared along the temporal frequency axis. This smearing allows us to generate a possible explanation of the illusory direction reversal at short durations. The explanation depends on two assumptions. 

The first assumption is that the temporal smearing is sufficiently large that the static low spatial frequency noise stimulates motion sensors corresponding to opposite directions of motion, but not so large that the moving high spatial frequency noise does so. The width of the temporal frequency spectrum is inversely proportional to the stimulus duration, so for any given velocity there will be a duration such that the activation of the low spatial frequency sensors by temporal smearing approximately matches the activation of the high spatial frequency sensors by the motion. This looks plausible from Figure 3d, in which the high spatial frequency components plot almost entirely within the first and third quadrants whereas the low spatial frequency components are smeared to high and low temporal frequencies. It follows from this assumption that the brief stimulus with moving high spatial frequency components will generate not one but three sets of motion signals; there will be two (illusory) oppositely directed signals at low spatial frequency and a single, correctly perceived signal at high spatial frequency. 

It is important to be clear that the temporal smearing on its own will not result in an overall reversed motion signal in either spatial frequency band. In the high spatial frequency band, the signal is in the correct direction and in the low spatial frequency band the signals corresponding to opposite directions are equally balanced. In this respect, our stimulus is unlike the fluted square wave illusion discussed by Adelson and Bergen (1986; see their Figure 4). In that case, when a square wave moves rightward, the fundamental frequency of the wave has motion energy in quadrants 1 and 3 (see our Figure 3), and the stimulus is perceived moving rightward. However, the third harmonic has motion energy in quadrants 2 and 4 (see our Figure 3), and if the fundamental is removed, the square wave without the fundamental seems to move to the left, not to the right. In the fluted square wave, but not in our stimulus, the physical motion energy is in the direction that corresponds to the percept and so the motion energy model predicts the percept. We need to make a further assumption in order to explain the perceived motion of our stimuli. 

The second assumption, which is the crucial feature of our model, is that low spatial frequency and high spatial frequency signals corresponding to the same direction of motion will inhibit one another. The consequence of this assumption is that the veridical motion signal generated by the moving high spatial frequency components is both canceled by and cancels the illusory low spatial frequency motion signal corresponding to the correct direction of motion. After this cancellation, we would expect a clear percept of motion in the wrong direction to be produced by the remaining uncanceled illusory motion signal generated by low spatial frequency components. Although this may sound implausible, it is testable because it allows us to make an experimental prediction and to develop a model to test it. 

The experimental prediction is that if we take a long duration stimulus, like that illustrated in Figure 3b, we should be able to produce a similar illusion if we can make the temporal frequency spectrum of the low spatial frequency components broader. 

Figures 3c and 3f show the space–time plot and the spatiotemporal frequency spectrum of a candidate stimulus. It is a long duration stimulus in which the high spatial frequency components move and the low spatial frequency components flicker. In the next section, we test whether this stimulus produces a comparable motion reversal. 

Experiment 3: canceling illusory reversed motion

Figure 4 shows how we tested the existence and measured the strength of the motion reversals in stimuli with flickering low spatial frequency components. The flickering components were made to move and their speed was adjusted from trial to trial using an adaptive “staircase” controlled by the observer's report of their direction of motion. 

Figure 4

 

Sample of procedure to obtain the cancellation speed. (a) Sample of the track to obtain the cancellation speed for one observer (AD) using orientation-filtered noise (two components centered in 1 cycle/deg flickering at 3 Hz and 3 cycles/deg) with duration of the Gaussian temporal window of 200 ms. Three staircases of 40 trials were interlaced. The speed of the 1 cycle/deg flickering component is plotted on the ordinate; negative speed values indicate that the 1 cycle/deg component was moving in opposite direction to the 3 cycles/deg component; positive speed values indicate that both components were moving in the same direction. The 3 cycles/deg component always had a constant speed of 4 deg/s. The speed of the 1 cycle/deg component was reduced when the motion discrimination was correct and both components were moving in the same direction or when the motion discrimination was incorrect and the two components were moving in opposite directions. Otherwise, the speed was increased. On the right is shown a histogram with the number of times each speed was presented. (b) Direction discrimination performance for the 1 cycle/deg flickering component as a function of its speed. Filled circles, results obtained from panel (a); line, cumulative Normal psychometric function fitted by maximum likelihood. The estimated cancellation speed for a probability of 0.5 was 2.525 deg/s. (c) Histogram of the 2,000 estimated cancellation speeds obtained implementing simulations of the same experiment and assuming the fitted psychometric function as the model of the observer. In the upper part of the histogram, the circle lies at the mean of distribution and the horizontal line covers the central 90% range.

Three staircases were run in parallel as shown in Figure 4a, and the results were used to compile the psychometric function shown in Figure 4b, which indicates that the speed that produced equally frequent reports of leftward and rightward motion was just over 2.5 deg/s. 

This cancellation speed is a measure of the strength of the illusory motion; the fact that it is positive indicates that, as we predicted, the flickering stimulus, like the briefly presented stimulus, produces an illusory motion reversal. 

Figure 5 shows, for the three observers who provided data for Figures 1 and 2, how the cancellation velocity varies with stimulus conditions. 

Figure 5

 

Results from Experiment 3. Cancellation speed using orientation-filtered noise as a function of the duration of the temporal Gaussian window (duration = 2 σ _t). Symbols represent the estimated cancellation speeds; vertical line through each symbol is the 90% confidence interval (see Figure 4). Open circles, cancellation speed for the 1 cycle/deg noise component added to a moving 3 cycles/deg noise with fixed speed of 4 deg/s. Open squares, cancellation speed for a 1 cycle/deg noise component flickering at 3 Hz added to a moving 3 cycles/deg noise with fixed speed of 4 deg/s.

To measure the cancellation velocity both sets of noise were presented steadily with the high frequency noise moving at 4 deg/s and the low frequency noise moving at a speed determined by the observer's reports. 

The circles plot cancellation velocity as a function of stimulus duration. As would be expected from the spatiotemporal frequency spectra shown in Figure 3 and from the fact that observers do not report reversed motion at long durations, when the low frequency components move without flickering, the cancellation velocity falls with increasing duration. When the duration reaches 200 ms, the cancellation velocity is less than 0.5 deg/s for all observers. However, as we predicted from Figure 3f, when the low frequency noise flickers, it appears to move in the opposite direction to the high frequency noise unless it actually moves in the same direction. The squares show cancellation velocities measured using long duration stimuli in which the low spatial frequency noise flickered at 3 Hz and the high frequency noise moved at 4 deg/s. In all cases, this raised the cancellation velocity to at least 2 deg/s. 

Model simulations

We developed a simple model (see 1) to test our proposal that the impaired motion discrimination and the reversals in perceived direction we describe here reflect inhibitory interactions between motion sensors tuned to high spatial frequencies and those tuned to low spatial frequencies. The model calculates the response of motion energy sensors (Adelson & Bergen, 1985; Watson & Ahumada, 1985) tuned to 1 and 3 cycles/deg and to all orientations at a grid of positions covering the stimulus. Each spatial frequency channel calculates the half-wave rectified difference between its response and that of the channel tuned to the other frequency. The overall output is based on the channel with the bigger response. 

Figure 6 shows simulated psychometric functions generated from the calculated responses of the model to the stimuli used in all experiments. Responses to orientation-filtered noise are shown in Figure 6a; responses to isotropic noise are shown in Figure 6b; and cancellation velocities, which are predicted from the low frequency channel's response, are shown in Figure 6c. 

Figure 6

 

Model simulations. (a) Model predictions of Experiment 1, calculated using the same samples of orientation-filtered noise as were used in the experiment. (b) Model predictions of the Experiment 2, calculated using the same samples of isotropic noise as were used in the experiment. Open circles, stationary 1 cycle/deg noise with moving 3 cycles/deg noise; open squares, moving 1 cycle/deg noise with moving 3 cycles/deg noise; open triangles, moving 1 cycle/deg noise with stationary 3 cycles/deg noise; solid triangles, moving 1 cycle/deg noise; black circles, moving 3 cycles/deg noise. Moving components had a fixed speed of 4 deg/s. Points are means of 25 noise samples ± SEM. (c) Model predictions of Experiment 3, calculated using the same stimuli (orientation-filtered noises). Open circles, cancellation speed for the 1 cycle/deg noise component added to a moving 3 cycles/deg noise with fixed speed of 4 deg/s. Open squares, cancellation speed for a 1 cycle/deg noise component flickering at 3 Hz added to a moving 3 cycles/deg noise with fixed speed of 4 deg/s. Symbols represent the estimated cancellation speeds; vertical line through each symbol is the 90% confidence interval. Model details are given in 1.

The model responses show the same basic features as the results. Figure 6a shows that the model's estimate of the direction of motion of high spatial frequency orientation-filtered noise reverses at short durations when the noise is presented with static low spatial frequency noise. It also shows that direction discrimination is impaired when the low frequency noise moves with the high frequency noise. Figure 6b shows that similar reversals occur with isotropic noise but that the impairment in performance when both sets of noise move together is less severe than with orientation-filtered noise, which is similar to the psychophysical results. Figure 6c replicates both the decline in cancellation velocity with duration that occurs when the low frequency noise is presented steadily and the increase that occurs when it is flickered. 

Discussion

The results show that when a brief moving stimulus contains both fine and coarse features, motion of the fine features causes potentially catastrophic perceptual failures. The worst failures occur when the coarse features are static because then the motion percept is reversed. However, even when the coarse features move in step with the fine features, perception is substantially impaired and may be reversed under some conditions. The modeling demonstrates that simple subtractive interactions between motion sensors tuned to high and to low spatial frequencies can account for this pattern of errors. 

Although they are very clear, these results raise important questions both about the characteristics of the failures and about their significance for visual processing. Perhaps the most obvious questions are why these catastrophic failures have not been observed before and whether, given their novelty, we can be sure that the failures occur in the visual system rather than in the display equipment. Second, and most important, we need to consider whether there are visual tasks that are facilitated by these interactions: Does the visual system gain any benefit from them? Finally, it seems curious that we do not get a similar pattern of perceptual failure when high frequencies are static and low frequencies move. 

Fortunately the obvious question has an obvious answer. Although there are many examples in the literature of studies of interactions between stimuli of different spatial frequencies, including some, to be discussed below, that suggest antagonistic interactions between different spatial frequencies, nobody has used such brief stimuli to study interactions between different spatial frequencies in motion perception. The interactions only lead to catastrophic errors with very brief stimuli: Reversals only occur at the very shortest stimulus durations and performance is always good if the stimulus duration is greater than 100 ms. There are several good reasons that most people refrain from using such brief stimuli: Until relatively recently, equipment capable of generating the displays was very expensive; there was no reason to expect that anything interesting would happen with very short displays; and naive subjects find it difficult to work reliably with such brief displays. We ourselves chose to use very brief display presentations because we wanted to reduce the possibility that subjects would track the stimuli. 

In order to confirm that the errors and the reversals arise within the visual system and are not caused by failures in the display equipment, we repeated the crucial stimuli from Experiment 1 with the viewing distance doubled. The effect is that we double the spatial frequency of the stimulus viewed by the subject while reusing the same movies that were used in Experiment 1. The results are shown in Figure 7, which shows for two observers that the effect of raising the spatial frequency is to restore veridical motion perception, raising direction discrimination performance above chance. 

Figure 7

 

Results of control experiment. Each panel shows the results from one observer as two plots of direction discrimination performance against stimulus duration. The left half of the plot shows results obtained at a viewing distance of 143 cm, at which the center frequencies of the noise were 1 and 3 cycles/deg; these results are replotted from Figures 1d and 1e. The right half of the plot shows results obtained using the same images viewed from 286 cm, at which distance the high frequency noise is centered on 6 cycles/deg and the low frequency noise is centered on 2 cycles/deg and the speed of motion is 2 deg/s. Triangles, high (H) frequency noise static, low (L) frequency noise moving; circles, high frequency noise moving, low frequency noise static; squares, both noises moving. Other details are as in Figure 1.

This both confirms that the reversals arise within the visual system and demonstrates that they are dependent on the absolute spatial frequencies of the stimuli. 

Before speculating on whether there might be visual tasks that benefit from the coarse–fine antagonism in motion perception, it is helpful to consider other, potentially related cases of interactions between stimulus components of different spatial frequencies in motion perception. In the literature, we can find two types of interaction between scales, motion capture, and motion induction. In the first type of interaction, the motion of a low frequency pattern captures the high frequency pattern and makes it appear to move in the same direction that the low frequency pattern is moving (Ido, Ohtani, & Ejima, 1997; Ramachandran & Cavanagh, 1987; Yo & Wilson, 1992). In the second type of interaction, the motion of a high frequency pattern makes the low frequency pattern appear to move in the opposite direction (Henning & Derrington, 1988; Levi & Schor, 1984; Murakami & Shimojo, 1995; Reinhardt-Rutland, 1988). Our results show this second type of interaction, although they are more severe than other examples. For example, in the cancellation experiment, we measured the speed of the motion of a low frequency pattern in order to cancel the motion induced when the high frequency pattern moved at a fixed speed. Our model takes into account both types of interaction by including a stage of reciprocal interaction between channels tuned to high and low spatial frequencies. 

We know that the phenomenon is general: Everyone of the 10–15 subjects we have tested formally and informally show qualitatively similar effects to those in Figure 1. However, as Figure 1 shows, there are clearly individual differences in the strength of the perceptual impairment: Two of the three observers show significant reversals in motion perception even when both sets of noise move together. It is possible that at least part of this variation in the severity of the perceptual impairment reflects individual differences in the optimal spatial parameters of the antagonism. It would be helpful to establish the extent to which there is a real variation in the strength of the antagonism and whether it is correlated with any other aspect of perceptual performance. 

We have not attempted to optimize the parameters for each subject and are just beginning systematic exploration of the effect of varying the spatial frequency and visual field location of stimulus components. Measurements of performance as a function of duration, like those in Figures 1 and 2, provide only limited information and it is encouraging that estimates of cancellation speed, which give extra information and can be carried out with a wider range of stimuli, show the same pattern of individual differences as the duration measures. Stimulus conditions that result in reversed motion for a particular observer when low frequency components move give rise to cancellation speeds greater than 4 deg/s in that observer. Measurements of cancellation speed using grating stimuli suggest that there is a relatively wide range of spatial frequency combinations that give rise to measurable antagonistic interactions between spatial frequencies with a “crossover” between high spatial frequency and low spatial frequency components at about 2 cycles/deg (Henning & Derrington, 1988). 

The kind of perceptual task that would be facilitated by inhibitory interactions between sensors would be the detection of relative motion, and the idea has a long history in the context of detecting relative motion between different parts of the visual field in order to discount eye movements (Regan, 1986) or detecting differences in motion between an object and its surroundings in order to segment the visual image (Regan & Beverley, 1984). However, we know from work with sinusoidal gratings that the antagonism between spatial frequencies is much stronger when they are superimposed, as in our experiments, than when they are adjacent: The cancellation speed of a 1 cycle/deg grating is reduced by about 50% when the 3 cycles/deg grating that makes it appear to move is presented in a strip above and below the 1 cycle/deg grating rather than superimposed on it (Henning & Derrington, 1988). 

There are also aspects of image interpretation that might be facilitated by the availability in the visual system of relative motion signals. One that springs to mind here is the detection of object rotations: The rotation of a solid object with a textured surface would be signaled by differential motion between the fine scale features of the surface texture and the coarse scale features of the object body. 

The last question to consider is why a symmetrical reciprocal inhibition between low spatial frequency and high spatial frequency motion sensors should produce an asymmetrical pattern of errors. Performance—both of human observers and of our simple model—reverses when the low spatial frequency noise is static but not when the high spatial frequency noise is static. It is not possible to say definitively what happens, but it is likely that the asymmetry of the illusion is related to the fact that speed and spatial frequency are inversely related. The consequence of this inverse relationship is that the temporal frequencies associated with the velocity of our moving stimuli are about three times as high for the high spatial frequency noise as they are for the low spatial frequency noise. The high temporal frequency associated with the higher spatial frequency has the effect that, even with the temporal smear induced by the briefest presentation, the moving noise contains almost no energy in the quadrant corresponding to motion in the opposite direction of motion (see Figure 3d). The same would not be the case at the lower spatial frequency, which contains substantial energy in the “wrong” quadrants (Derrington & Goddard, 1989). In order for the stimulus motion to generate a comparable imbalance in the distribution of energy of the low spatial frequency components to that shown for the high spatial frequency components in Figure 3d, it would be necessary to use three times the speed. Furthermore, in the hypothetical case that the inhibitory processes we have modeled were to induce an imbalance in the internal representation of the spatiotemporal spectrum of the high spatial frequency components, the associated speed would be three times less. 

Appendix A

Model

To explain our results, we use a widely accepted model of the motion sensor (Adelson & Bergen, 1985) adding a differential processing stage which combines the outputs of sensors tuned to coarse and fine scales. The model contains spatial weighting functions and temporal impulse response functions. The spatial weighting function was a two-dimensional Gabor function (Watson & Ahumada, 1985): 

$f(x,y)=\gamma \left({\rho }_0\right)\times \exp \{-\frac{{\widehat{x}}^2}{2{\sigma }_x^2}-\frac{{\widehat{y}}^2}{2{\sigma }_y^2}\}\cos (2\pi {\rho }_0\widehat{x}+{\phi }_0),$

(A1)

where 

$\widehat{x}=(x-x\prime )\cos \left({\theta }_0\right)+(y-y\prime )\sin \left({\theta }_0\right),$

(A2)

 

$\widehat{y}=-(x-x\prime )\sin \left({\theta }_0\right)+(y-y\prime )\cos \left({\theta }_0\right).$

(A3)

 

The frequencies of the sensors were ρ ₀ ∈ {1, 3} cycles/deg. In the model below, we use the terms LF for the 1 cycle/deg sensors and HF for the 3 cycles/deg sensors. The function γ is the gain of the sensor, where γ (1) = 0.15 and γ (3) = 1. The spreads of the Gaussian function σ _x and σ _y were obtained by the equations  

${\sigma }_x=\frac{\sqrt{\log \left(2\right)}(1+2^B)}{{\rho }_0\sqrt{2}\pi (2^B-1)},$

(A4)

 

${\sigma }_y=\frac{\sqrt{\log \left(2\right)}}{{\rho }_0\sqrt{2}\pi \tan (\alpha /2)},$

(A5)

where B = 1 octave (full width at half-height) and α = 30° (full width at half-height). The locations of the sensors were x′ ∈ {−2°, −1°, 0°, 1°, 2°} and y′ ∈ {−1.5°, 0°, 1.5°}. The total number of locations was a combination of x′ × y′ = 15 locations. The orientations of the sensors were θ ₀ ∈ {−60°, −30°, 0°, 30°, 60°, 90°}. The model uses a quadrature pair of sensors f ₁ and f ₂. For f ₁ ( x, y), the phase was φ ₀ = π/2 rad; and for f ₂ ( x, y), the phase was φ ₀ = 0 rad. 

The temporal impulse response functions chosen, h ₁( t) and h ₂( t), were a quadrature pair. The equation of the slower function h ₂( t) was taken from Adelson and Bergen (1985) 

$h\left(t\right)={(kt)}^n\exp (-kt)\times \{1/n!-{(kt)}^2/(n+2)!\},$

(A6)

where k = 0.09 and n = 3. The faster function, h₁(t), was the quadrature pair of h₂(t), calculated in the frequency domain by using the Hilbert transform (Watson & Ahumada, 1985). 

The model starts by calculating the responses of a set of motion energy sensors to the stimulus. The set comprises sensors with two different center frequencies (1 and 3 cycles/deg) selective to six different orientations (−60°, −30°, 0°, 30°, 60°, 90°) located at 15 different locations. The response of a sensor with location i and orientation j was calculated from the inner product of the stimulus with the sensor spatial weighting function and the convolution of the stimulus with the temporal impulse response function  

$A_{ij}\left(t\right)=h_1\left(t\right)*{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\mathrm{I}(x,y,t)\times f_{1_{ij}}(x,y)\mathrm{d}x\mathrm{d}y,$

(A7)

 

${A^{\prime }}_{ij}\left(t\right)=h_2\left(t\right)*{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\mathrm{I}(x,y,t)\times f_{1_{ij}}(x,y)\mathrm{d}x\mathrm{d}y,$

(A8)

 

$B_{ij}\left(t\right)=h_1\left(t\right)*{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\mathrm{I}(x,y,t)\times f_{2_{ij}}(x,y)\mathrm{d}x\mathrm{d}y,$

(A9)

 

${B^{\prime }}_{ij}\left(t\right)=h_2\left(t\right)*{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\mathrm{I}(x,y,t)\times f_{2_{ij}}(x,y)\mathrm{d}x\mathrm{d}y.$

(A10)

 

The operation of each sensor was as described by Adelson and Bergen (1985) but we calculate the total energy integrating across time 

${\mathrm{L}}_{ij}=\int {\left(A_{ij}\right(t)-B{\prime }_{ij}(t\left)\right)}^2+{\left(A{\prime }_{ij}\right(t)+B_{ij}(t\left)\right)}^2\mathrm{d}t,$

(A11)

 

${\mathrm{R}}_{ij}=\int {\left(A_{ij}\right(t)+B{\prime }_{ij}(t\left)\right)}^2+{\left(A{\prime }_{ij}\right(t)-B_{ij}(t\left)\right)}^2\mathrm{d}t.$

(A12)

 

Then, the response was pooled across n locations of the same sensor with the same orientation  

${\mathrm{L}}_j=\sum _{i=1}^n\left[{\mathrm{L}}_{ij}\right],{\mathrm{R}}_j=\sum _{i=1}^n\left[{\mathrm{R}}_{ij}\right].$

(A13)

 

Next, there was a subtraction and half-wave rectification between sensors with low (LF) and high (HF) spatial frequency with the same orientation  

${\mathrm{L}}_{\mathrm{L}{\mathrm{F}}_j}=\lfloor {\mathrm{L}}_{\mathrm{L}{\mathrm{F}}_j}-{\mathrm{L}}_{\mathrm{H}{\mathrm{F}}_j}\rfloor ,{\mathrm{R}}_{\mathrm{L}{\mathrm{F}}_j}=\lfloor {\mathrm{R}}_{\mathrm{L}{\mathrm{F}}_j}-{\mathrm{R}}_{\mathrm{H}{\mathrm{F}}_j}\rfloor ,$

(A14)

 

${\mathrm{L}}_{\mathrm{H}{\mathrm{F}}_j}=\lfloor {\mathrm{L}}_{\mathrm{H}{\mathrm{F}}_j}-{\mathrm{L}}_{\mathrm{L}{\mathrm{F}}_j}\rfloor ,{\mathrm{R}}_{\mathrm{H}{\mathrm{F}}_j}=\lfloor {\mathrm{R}}_{\mathrm{H}{\mathrm{F}}_j}-{\mathrm{R}}_{\mathrm{L}{\mathrm{F}}_j}\rfloor .$

(A15)

 

Then, each response was pooled across m orientations with a cosine weighting  

${\mathrm{L}}_{\mathrm{L}\mathrm{F}}=\sum _{j=1}^m\cos \left({\theta }_j\right)\times {\mathrm{L}}_{\mathrm{L}{\mathrm{F}}_j},{\mathrm{R}}_{\mathrm{L}\mathrm{F}}=\sum _{j=1}^m\cos \left({\theta }_j\right)\times {\mathrm{R}}_{\mathrm{L}{\mathrm{F}}_j},$

(A16)

 

${\mathrm{L}}_{\mathrm{H}\mathrm{F}}=\sum _{j=1}^m\cos \left({\theta }_j\right)\times {\mathrm{L}}_{\mathrm{H}{\mathrm{F}}_j},{\mathrm{R}}_{\mathrm{H}\mathrm{F}}=\sum _{j=1}^m\cos \left({\theta }_j\right)\times {\mathrm{L}}_{\mathrm{H}{\mathrm{F}}_j}.$

(A17)

 

After this pooling, the psychophysical response was calculated using the sensors, LF or HF, that had the greater difference between left and right  

$\max \left(|{\mathrm{L}}_{\mathrm{L}\mathrm{F}}-{\mathrm{R}}_{\mathrm{L}\mathrm{F}}||{\mathrm{L}}_{\mathrm{H}\mathrm{F}}-{\mathrm{R}}_{\mathrm{H}\mathrm{F}}|\right).$

(A18)

 

Next, the direction index (DI) was calculated using the following function  

$\mathrm{D}\mathrm{I}=\frac{R-L}{R+L},-1\le \mathrm{D}\mathrm{I}\le 1.$

(A19)

 

Finally, the DI was transformed into proportion of correct responses using a normal cumulative distribution function (using a linear transformation of the DI gave a similar result).  

$P\left(\mathrm{R}\right|\mathrm{R})=0.5\times [1+\mathrm{e}\mathrm{r}\mathrm{f}\mathrm{(}\mathrm{D}\mathrm{I}\mathrm{/}\mathrm{\left(}\mathrm{0.3}\sqrt{\mathrm{2}}\right)\left)\right],$

(A20)

where  

$\mathrm{e}\mathrm{r}\mathrm{f}\left(x\right)=\frac{\mathrm{2}}{\sqrt{\pi }}{\int }_0^x\exp [-t^2]\mathrm{d}t.$

(A21)

 

To obtain the results of Figures 6a and 6b, the same movies used in the psychophysical Experiments 1 and 2 were used in the simulations. The only difference was that movies that had shown leftward motion were played backward so that the direction of the motion stimuli was always rightward. The final proportion of correct direction discriminations was the mean of the model output for the 25 movies for each experimental condition. To obtain the results of Figure 6c, we used the same stimuli used in the psychophysical Experiment 3 in the simulations. In this case, instead of using the sensor with the greater difference between left and right (LF or HF) to determine the response, we always took the response of the LF channel because in the psychophysical task the subject had to attend only to the low spatial frequency component. For the model computations, the direction of the high frequency component stimuli was always rightward and the direction of the low frequency component was leftward. The method of constant stimuli was used instead of using adaptive staircases. In the computations, we used seven speeds (0, 1, 2, 3, 4, 5, 6 deg/s) for the low frequency component (flickering at 3 Hz or without flicker), and for the high frequency component the speed was always 4 deg/s. For each one of the seven speeds and duration, we obtained a proportion correct response which was the mean of the model output for the 25 different movies. The cancellation speed for each duration ( Figure 6c) was obtained by means of fitting a cumulative normal function to the seven proportion correct responses as a function of the speed in order to obtain the 50% point of the psychometric function (see Statistical analysis section). 

Acknowledgments

This research was supported by Supported by a grant from the Wellcome Trust. Experiments were carried out at Newcastle University. 

Commercial relationships: none. 

Corresponding author: Andrew M. Derrington. 

Email: [e-mail]. 

Address: Department of Psychology, University of Kent, Canterbury, Kent, CT2 7NP, United Kingdom.