Characterizing motion contour detection mechanisms and equivalent mechanisms in the luminance domain

Abstract

Motion-defined contours are ecologically important cues to object boundaries in complex fields of optic flow. We designed a novel stimulus in which the velocities of randomly positioned dots are defined by a 2D Gabor function, resulting in a motion-defined pattern with a clear orientation. We found that the number of correct responses in a vertical/horizontal orientation discrimination task increases and saturates with size of the Gabor envelope at around 4–5° full width at half height. The number of correct responses decreases with higher spatial frequency of the Gabor patterns. The best performance occurs at 0.1 cycles/degree, when only a single contour is visible. Using elliptical Gabor stimuli, we found that accuracy is higher if the patch is elongated along the contours (rather than orthogonal to them), confirming the existence of an elongated detector mechanism for a single contour. We compared tuning properties for motion-defined Gabor patterns with sparsely defined luminance Gabor patterns and found similar results, but only at low sampling densities. The nature of the information and the strength of the signal influence the properties of luminance contour detection mechanisms, whereas motion contour detection may be limited by the sparse visual representation of the motion field.

Introduction

In a visual scene, discontinuities over space can be characterized by various attributes, such as luminance, color, and texture as well as motion. Contours often indicate the existence of object boundaries and are therefore crucial to meaningful scene segmentation. Contours defined by motion alone can be ecologically important when other visual cues are not informative, e.g. for breaking camouflage. Humans can readily detect and localize boundaries between different moving surfaces (Burr, Mckee, & Morrone, 2006; Watamaniuk, Flinn, & Stohr, 2003). Previously in the luminance domain two-dimensional Gabor patterns have been used to probe the performance of the human visual system in response to spatial changes in luminance over space (Hoekstra, van der Goot, van den Brink, & Bilsen, 1974; Watson, Barlow, & Robson, 1983). These patterns are defined by a sinusoidal grating (the carrier) limited by a Gaussian envelope and are optimal in terms of the encoding of spatial position and frequency (Daugman, 1988). Similarly, we can use a motion-defined Gabor pattern to examine how the visual system responds to changes in motion over space. This is useful for investigating receptive field size and spatial frequency tuning for motion-defined contours. Gabor pattern defined motion stimuli contain continuous changes in velocity which lead to non-rigid motion within the pattern. They can also vary both in speed and direction (see Figure 1), both of which provide a cue to the location of motion-defined contours (Durant & Zanker, 2008). 

Figure 1

 

A schematic illustration of the stimulus. In (a) the velocities of dots are shown by arrows, with the length of the arrow representing speed. White upward arrows show upward movement and black downwards arrows show downwards movement (all dots in the motion-defined Gabor condition are black). The dotted curves show the threshold below which random motion directions are chosen and the circular limits of the Gabor function, below which the lower threshold is reached. In (b) the velocity is shown on the y axis and space along the x axis, along the horizontal dashed line of (a), to show the exact shape of the velocity profile across space. Full width at half height (fwhh) of the Gabor is also illustrated.

An important feature of motion-defined contours is that their extraction must occur “higher” in the visual processing hierarchy because first local motion must be extracted, and then patches of different local motion need to be grouped in a further global processing step (Braddick, 1993; Zanker, 1993). By examining the shape of these higher-level, global filters we may begin to understand the specific processing mechanisms at this stage of visual analysis. It is interesting to consider whether the principles of contour extraction resemble the properties of what we know of the lower-level luminance contour processing stage that operates directly on retinal luminance information (Hubel & Weisel, 1968). 

In the luminance domain it has been found that the human visual system increases in sensitivity to the spatial variation of luminance (i.e. luminance contrast) with the spatial extent of a stimulus and then levels out, suggesting a limited area over which luminance is integrated (Hoekstra et al., 1974; Syväjärvi, Näsänen, & Rovamo, 1999). The well-established human contrast sensitivity function demonstrates that for luminance contours under photopic conditions when static stimuli are used, humans are most sensitive at around 2–6 cycles/degree (c/deg), dropping off at lower and higher spatial frequencies (De Valois & De Valois, 1990). These results are for so-called “first order” stimuli, for which contours can be detected by linear luminance spatial frequency tuned filters. The motion-defined Gabor patterns we describe here may be thought of as second order stimuli, where average luminance remains constant across space and so the variation in pattern remains undetectable to linear luminance filters. A large class of stimuli fall under this category, e.g. contrast, orientation and other textural features can be modulated to produce contours. These second-order stimuli require two stages of filtering to be detected (Sutter, Sperling, & Chubb, 1995). Similarly motion-defined patterns would require an initial motion detection stage, followed by a second stage to be detected. Second order contour detection has been previously examined with Gabor patterns defined by changes in contrast (Sutter et al., 1995) and changes in orientation (Kingdom, Keeble, & Moulden, 1995). Both looked at the effect of the size of the Gabor patterns as well as the spatial frequency of the contrast or orientation modulation. They found band-pass tuning for spatial frequency, with a peak at much lower spatial frequencies than typically found for luminance-defined patterns—around 0.1 c/deg. When the size of the Gabor was varied these experiments found scale invariance, meaning that the same pattern of results remain as viewing distance is changed; in other words spatial frequency tuning depends not on the absolute retinal spatial frequency (c/deg), but rather the object related spatial frequency (cycles/object), a common result found using other types of second order stimuli e.g. (Landy & Bergen, 1991). It has also been found that the spatial relationship between the carrier (i.e. the pattern that is being modulated) and the envelope (the properties of the modulation) plays a crucial role, in particular their relative orientations can have differential effects at low and high spatial frequencies (Dakin & Mareschal, 2000). We will consider how spatial variation in motion compares with spatial variation in contrast and orientation. 

The size and spatial frequency tuning of human motion detection units have also been elucidated using sine waves (Anderson & Burr, 1989; Kelly, 1979) as well as Gabor patterns (Anderson & Burr, 1987). This work found that units sensitive to lower spatial frequencies had larger receptive fields and found that spatial frequency tuning extended from as low as 0.03 c/deg to 15 c/deg. Correspondingly, receptive field sizes vary from 0.05 deg to 4.8 deg. These are the properties of motion detectors, which must form the input for any detectors that are tuned for discontinuities in the motion field. Our acuity for motion-defined contours has also been measured in the past using dense random noise dots whose motion was modulated according to square wave and sine wave patterns (Burr et al., 2006; van Doorn & Koenderink, 1982; Watson & Eckert, 1994). Most of the studies agree that motion contour detection is best at low spatial frequencies. It must be noted that it is difficult to examine the low frequency end of the tuning for motion contours, as we are limited by the size of standard displays, scaling with respect to eccentricity and ultimately the extent of visual space. Some studies however did find band-pass characteristics for motion detection with sinusoidal motion-defined gratings (Sachtler & Zaidi, 1995; Watson & Eckert, 1994). At high spatial frequencies, motion contours become transparent, i.e. a different type of segregation manifests itself, suggesting that local/global motion processes are still operating but to different effect (van Doorn & Koenderink, 1982; Zanker, 2001). None of these studies, however, examine the size of these putative motion contour detector units. Burr et al. ( 2006) measured accuracy for localizing a single motion-defined contour and found resolution to be much poorer than for luminance and more significantly only 1.1–1.5 times better than the motion-defined grating resolution, suggesting motion edges are not as accurately coded as luminance edges. 

It is possible that poor performance for motion contour detection is due to motion being more sparsely sampled over space, because of larger receptive fields. This might imply that motion contour detection suffers from a worse signal-to-noise ratio, although the relationship between receptive field size and sampling density is not straightforward, as larger but more densely packed overlapping receptive fields need not yield lower sampling density. 

Previous work on the detection of luminance-defined Gabor patches degraded through noise has shown different results than for unimpaired Gabor patterns, with performance dropping off with an increased number of cycles, peaking when only one cycle within the envelope was presented (Kersten, 1984)—however, the saturation with increasing size with constant carrier spatial frequency is not altered by the presence of noise (Syväjärvi et al., 1999). Furthermore, spatial integration appears to differ in the periphery where one might argue the visual scene is also more sparsely sampled due to larger receptive fields (Manahilov, Simpson, & McCulloch, 2001). 

In the present work we aim to investigate the mechanisms underlying motion-defined contour detection. We are interested in the tuning properties of the operators in the visual system for these stimuli. We will also investigate the parallels that can be drawn between motion- and luminance-defined contours. We will compare the performance on an orientation detection task for motion-defined contours with results for corresponding stimuli in the luminance domain. We will test how the similarity between the two domains changes as a function of dot density. Instead of introducing noise into the stimulus we reduce the amount of information present, by showing both densely and sparsely sampled luminance patterns over space to compare with the results from the motion domain. The implications of these results will be discussed in terms of motion as a spatially sparsely sampled distribution both due to the information present in the stimulus and the limitations of the visual system and to the extent that this might apply to second order stimuli in general. 

Experiment 1: Motion-defined patterns

Methods

Stimuli

Stimuli were presented on a 21 inch EIZO Flexscan T662-T CRT monitor, controlled by a Cambridge Systems ViSaGe graphics card, programmed in Visual C++.NET. The refresh rate of the screen was 60 Hz and the screen resolution was 656 × 493 pixels. A central blue fixation point was presented on a light gray background (73 cd/m ²) in the center of the screen. Black (0.5 cd/m ²) moving dots 0.05° (1 pixel) in width and height were presented. The dots had a limited lifetime (this minimizes motion streak cues and prevents tracking of individual dots as well as avoiding wrap-around issues), each one was present for 3 frames of motion (50 ms) and then randomly re-assigned to a new position. The local speed value defined the distance (number of pixels) a dot moved between frames. Maximum dot speed was 3 pixel/frame (= 10°/s). Positions were calculated at sub-pixel accuracy and rounded to the nearest pixel position. The value for a 2D Gabor function centered at the fixation point was calculated for each pixel coordinate (normalized to a maximum value of 1). These values determined the velocity of any given dot at a given coordinate, which were randomly positioned within a square area (12.5° × 12.5°) around the fixation target (see Figure 1, and Movie 1 for a demo without the fixation point). A positive value of the Gabor function could represent upwards or rightwards motion, a negative value the opposite motion direction. The speed was the value of the Gabor function multiplied by 3 in pixels/frame, resulting in a maximal speed of 10°/s. If the calculated speed at a given coordinate was less than 0.3 pixels/frame (1°/s), the velocity was set to a random velocity drawn from 0.3 to 3 pixels/frame (1°/s–10°/s). Gabor patterns could be oriented horizontally or vertically and motion could be parallel (shearing patterns) or orthogonal (compression/expansion patterns) to the Gabor orientation, yielding four conditions. These conditions were randomly interleaved within a block and averaged to avoid a correct response being achieved by responding to axis of motion alone. The Gabor pattern phase was randomly chosen to be 0° or 180°, so that there was always a central contour, but the direction of motion defining it varied within the same condition (for instance, in the orthogonal condition, the central contour could be defined by expanding or contracting motion with equal probability). Stimuli were presented for 15 frames (250 ms) in all of these experiments apart from Experiment 1d. In Experiment 1d a staircase procedure was used (see Procedure) in which the duration of the stimulus (number of frames) was manipulated to find the orientation detection threshold at different spatial frequencies and different dot densities. 

 

Movie 1

 

A demo of a motion-defined Gabor pattern, there are two cycles of alternating upwards and downwards motion visible. The pattern is oriented vertically, and the motion is parallel to the contours. Larger dots are used for the demo than in the experiment to clearly illustrate the motion.

Participants

Three well-practiced participants who all had some knowledge of the aims of the experiment participated (first author SD, AM, TH). Additionally, 6 participants naive to the purpose of the study who had never seen the stimulus before participated. Participants all had normal or corrected-to-normal vision. The experiments were approved by the Royal Holloway Department of Psychology Ethics committee. 

Procedure

Participants made use of a chin rest to maintain a steady head position at 57 cm from the screen. In a single interval 2AFC contour orientation detection task, observers were asked to judge whether the motion-defined Gabor pattern was oriented horizontally or vertically, ignoring the direction of the motion defining the pattern. Some example stimuli were presented before the experiment to allow participants to familiarize themselves with the task. For Experiments 1a– 1c each data point was the number correct from 32 judgments (8 judgments were made on each of the four conditions: horizontal/vertical orthogonal/parallel). For the practiced observers (SD, AM and TH) each data point was measured four times and mean and standard error of mean expressed in terms of proportion correct were plotted. For the naive observers each data point was measured once and the mean of the group and the standard error of mean of the group expressed in terms of proportion correct were plotted. In Experiment 1d a standard simple up–down staircase procedure (Levitt,1971) was used that altered the stimulus duration up or down a number of frames according to whether the response was correct for a contour randomly chosen to be vertical or horizontal. All spatial frequencies were interleaved and blocks with different dot densities were presented in a pseudo-random order. The threshold was calculated from the mean of the last 5 reversals from 7 and the standard error of this mean was used in calculations. The maximum number of frames presented was 60 frames and the minimum was 1. For each spatial frequency at each dot density subjects completed two interleaved staircases, one containing horizontal motion and one containing vertical motion, ensuring that a correct judgment could not be achieved by using the direction of motion alone as a cue. In Experiment 1e each data point was the proportion correct out of 10 responses measured four times for each participant and the mean and standard error of mean expressed in terms of proportion correct were plotted. 

Results

Experiment 1a and 1b: Envelope size

In the first experiment we varied the spatial extent of the Gabor pattern (measured as full width at half height (fwhh) which is determined by the standard deviation of the Gaussian envelope). In Experiment 1a we used a low spatial frequency of the carrier (0.1 c/deg), which resulted in a single central boundary being visible at all sizes. In Experiment 1b we used a higher spatial frequency (0.5 c/deg), which led to a pattern that increased in the number of boundaries with size (see illustrations of stimuli in Figure 2). We measured the ability to detect the orientation of the motion-defined contours by calculating the proportion of correct responses (judging horizontal versus vertical orientation of contour). The number of correct answers was averaged over the four conditions (vertical/horizontal orientation, parallel/orthogonal motion—see Methods) for each envelope size, in order to discount any systematic biases that might occur in association with the axis of motion that was forming the motion-defined contours. 

Figure 2

 

The Gabor patterns below the graphs illustrate the spatial distribution of opposite motions represented by opposite luminance polarities. Speed is represented by the luminance difference from gray. Where the area shown is gray the dot motion was randomly chosen from the range of motions in the area of the Gabor pattern. (a) Proportion correct of horizontal/vertical for motion-defined patterns as a function of the size of the Gaussian envelope of the Gabor pattern with a carrier spatial frequency = 0.1 c/deg. The gray line is the fitted logistic function, from which the 90% of saturation point was used in Experiment 1c (SD: 5.1°, AM: 3.6°, group 3.6°). Number of dots: SD, AM = 12, group = 100. Standard error of mean (SEM) bars calculated from individuals over four repetitions and for the group are the error of the group mean. (b) Proportion correct of horizontal/vertical judgment as a function of the size of the Gaussian envelope of the Gabor pattern, with a carrier spatial frequency = 0.5 c/deg. A logistic function, fitted as in (a), determines the 90% of saturation points SD: 3.5°, AM: 4.1°, TH: 4.8°. All three participants were presented with 60 dots. SEM bars calculated over four repetitions.

For all participants the number of dots was chosen such that they would not be able to be 100% correct at any stimulus condition, in order to avoid ceiling effects. In Experiment 1a, the two observers SD and AM who also participated in pilot studies were presented with 12 dots and each data point was the average of four blocks (see Methods), while the rest of observers who had never seen the stimulus before and were naive to the purpose of the study, were presented with 100 dots and were only measured once, so as to avoid any effect of learning over four repetitions. As the naive observers showed exactly the same pattern as the practiced observers we assumed that reducing the dots has the effect of decreasing the level of performance equally at all conditions without changing the shape of the performance curve i.e. shifting the curves shown uniformly down the y-axis—an assumption we tested in Experiment 1d. This difference in the number of dots needed for a similar performance between observers shows a marked learning effect for these stimuli. 

In Experiment 1b, having established that naive observers show a similar pattern despite worse overall performance, we used only three well-practiced observers, who were presented with 60 dots each to achieve similar level of best performance as with the lower spatial frequency, again for four blocks, averaged over the four conditions as before. 

The results for Experiments 1a and 1b are shown in Figures 2a and 2b respectively. Varying the size of the envelope at low carrier frequency in Experiment 1a, we found that detection improved with the increasing envelope size and leveled off at around 4°–5° fwhh. This suggests that local motion signals are integrated over a relatively large area contributing to the detection of the contour orientation, as compared to the largest integration areas of around 2.5° at full spatial extent found in the luminance domain (Robson & Graham, 1981). Practiced observers were able to detect the contours with only 12 dots. Naive observers show the same pattern of results at a higher dot density. At low carrier frequency, increasing size increased the area of uniform motion either side of and along the single central contour (see insets in Figure 2a). At the higher spatial frequency of the carrier in Experiment 1b the three practiced observers tested required substantially more dots (60) to achieve similar performance levels as with the low frequency carrier. However, again the ability to detect the correct orientation increased with increasing envelope size and then leveled off at around 4°–5° fwhh. In this case increasing envelope size does not increase the size of the lobes of uniform motion, but does increase the number of contours present and the area of uniform motion along the contour (see insets in Figure 2b). 

We fitted a saturating function (a logistic function of the form: f( x) =

$\frac{r_{\max }}{1-e^{\frac{(\frac{x-0.5}{2}-r_{50})}{r_{\mathrm{s}\mathrm{l}\mathrm{o}\mathrm{p}\mathrm{e}}}}}$

, where r _max = saturation level, r ₅₀ = 50% point of r _max, and r _slope = the slope of the function) to the results from Experiment 1a and found 90% of the maximum of the function. This gave us an estimate of the size of area over which signals are integrated. We found values of SD: 5.1° fwhh; AM: 3.6° fwhh, group: 3.6° fwhh for the low spatial frequency condition and values of SD: 3.5°, AM: 4.1°, TH: 4.8° fwhh for the high spatial frequency condition. 

Experiment 1c and 1d: Carrier spatial frequency

In Experiment 1c we examined the effect of varying carrier spatial frequency at constant envelope size (the number of contours that are visible within a Gabor patch of constant size). The 90% of maximum performance estimate from Experiment 1a was rounded to the nearest size that we measured and used as the near optimal size for testing the effect of spatial frequency on orientation detection for the same participants (fwhh: SD = 5.4°, AM = 4.0°, TH = 4.0°). Keeping the same number of dots as in Experiment 1a, we measured the proportion of correct answers as the spatial frequency of the Gabor carrier varied, with the envelope size fixed. At the lowest spatial frequency only one contour was present and at each step, additional contours appeared and the lobes narrowed. The results of this experiment, and an illustration of the stimulus, are shown in Figure 3a. We found that the ability to detect the orientation of the motion contours decreased with spatial frequency. Again, the same pattern of results occurred for trained and naive observers, despite the number of dots presented being different for individuals. The observation that best performance always occurs at the lowest frequency measured (0.07 c/deg), suggests that a single motion contour is sufficient for detection, and that further contours interfere with, rather than enhance detection. This would suggest that when we measured the effect of size for a higher spatial frequency in Experiment 1b ( Figure 2b), the improvement with increasing size was not due to the additional contours, and that it probably is due to increasing the length of the central contour (and indeed, the same pattern of improvement was found for a single contour in Experiment 1a). 

Figure 3

 

Motion stimulus conditions illustrated by luminance as in Figure 2. (a) Proportion correct of horizontal/vertical judgments with SEM error bars for motion-defined patterns as a function of the spatial frequency of the carrier of Gabor pattern, for envelope as found from Experiment 1a (see Figure 2a), rounded to the nearest envelope size measured: SD: 5.4°, AM: 4.0°, group 4.0°. Number of dots: SD, AM = 12, group = 100. (b) Threshold duration for correct horizontal/vertical judgments for Gabor patterns defined by sparse luminance dots as a function of spatial frequency. Fixed spatial envelope at 4.7° fwhh. In the top panel each of 5 participants' data with SEMs are shown. In the lower panel the mean threshold duration over participants and the inverse of the mean threshold duration for comparison with part (a) of this figure are shown.

So far we operated at low signal levels, by choosing low levels of dot density, to avoid saturation effects. However, with dot densities as low as 12 pixels per image for experienced observers, it is obvious that the number of motion-defined cycles is rather limited. We were also interested in finding the implications of using different numbers of dots for different observers as was necessitated by the previous method. In Experiment 1d we used a staircase method that enabled us to simultaneously test participants on the same scales, and consider the effect of dot density. We tested with 12 and 100 dots again to ensure the results from Experiment 1c are reproduced with this different method and then also tested higher densities of 520 and 1500 dots. We fixed the envelope size at fwhh = 4.7°. 

Five participants took part in Experiment 1d and their results are shown in Figure 3b. There was some variability as already noted between subjects, dependent on level of practice. Similar patterns remain as found for the 12 dot and 100 dot conditions as before. With increasing dot density we find greater spatial resolution, but for all dot densities the best average performance is found at the lowest spatial frequency, consistent with a preference for a single motion border. With all dot densities performance drops down after 1 c/deg. The results show that all functions share a common low-pass tuning. Our method of measuring the staircase forces an upper limit of a 60 frame duration. A threshold of around 60 frames suggests that participants were not able to perform the task, as we can see for example for most of the points in the 12 dot condition for observer LM ( Figure 3b). 

Experiment 1e: Envelope shape

The results so far are not fully conclusive as to the effect of more contours present in the pattern. As we increase the size of a Gabor pattern (and hence the number of contours in the high spatial frequency pattern), there is an improvement in detection rate, but we have also seen that increasing carrier spatial frequency at a fixed size of envelope reduces detection rate. To test whether the improvement with increasing size of a high spatial frequency Gabor is due to the increase in length of the central contour, or to the additional amount of motion-defined contour information, we varied the spatial extent of the Gabor independently in the vertical and horizontal spatial dimensions in Experiment 1e. We measured four different conditions, which are illustrated in Figure 4. The “small” condition was the usual Gabor pattern with a fwhh of 1.2° extended the same size in both directions, in the “big” condition the Gabor pattern had a fwhh of 5.9° also circular in boundary shape. The “narrow” pattern had fwhh of 1.2° orthogonal to the contours and 5.9° parallel to contours, and in the “wide” condition this was the other way round, i.e. the Gabor was elongated orthogonal to the contours. Spatial frequency was fixed at 0.5 c/deg, the same as in Experiment 1b. We measured orientation detection performance for these four conditions over all the different types of possible motion (motion orthogonal/parallel to contours, central expansion/contraction) for three experienced observers SD, TH and AM with the same amount of dots as in Experiment 1b (60). We analyzed separately the conditions in which the central contour was expanding or contracting, but the type of motion that defined the central contour did not seem to greatly influence the results, and hence they were averaged over all types of motion, as shown in Figure 4 for all three observers. For each observer there is a significant effect over the four conditions (ANOVA results SD: F _3,12 = 161.0, p < 0.0005; TH: F _3,12 = 156.5, p < 0.0005; AM: F _3,12 = 95.3, p < 0.0005) and a Tukey post-hoc test reveals that there is a significant difference between the two elongated conditions for all three subjects at the 0.05 level. Clearly there is no improvement from the “small” to the “wide” condition and similarly from “narrow” to the “big” condition. In fact for observers TH and AM additional contours significantly reduce the proportion of correct responses. It is clear in this case that it matters whether the extra information is along the central contour or in the form of additional contours, and also any other information away from the central contour does not contribute to our ability to detect the orientation of the contour. This suggests the existence of an elongated edge detector mechanism that integrates information along the motion-defined contour and extends uniformly in space so that additional contours inhibit performance. 

Figure 4

 

Proportion correct of horizontal/vertical judgments for motion-defined patterns with the spatial extent of the Gabor pattern varied differently in the two spatial dimensions. Envelope sizes: small—1.2° in both directions; narrow—1.2° orthogonal to contours, 5.9° parallel to contours; wide—1.2° parallel to contours, 5.9° orthogonal to contours, big—5.9° in both directions. Spatial frequency 0.5 c/deg. Dashed lines on graphs show performance level at chance. SEM bars calculated for averages from four repetitions. Motion stimulus conditions illustrated by the luminance patterns as in Figure 2. All three participants were presented with 60 dots per image.

Experiment 2—Luminance-defined patterns

How do the results we have found for motion-defined patterns compare with what is known about responses to Gabor patterns in the luminance domain? When comparing to experiments involving continuous luminance-defined contours, the pattern of saturation with the size of the stimulus is similar to what has been observed in the luminance domain in the past, but saturation with luminance-defined Gabor patterns is observed at around 2.5° full width for a truncated grating, which is 1.25° at half width (Hoekstra et al., 1974; Syväjärvi et al., 1999) and around 1° fwhh for a Gabor pattern (Foley, Varadharajan, Koh, & Farias, 2007)—a much smaller size than the 4–5° fwhh saturation point we found for motion-defined Gabor patterns. The typical contrast sensitivity function that illustrates human sensitivity to luminance changes at various spatial frequencies has a peak at approximately 5 c/deg and drops off for both higher and lower spatial frequencies (De Valois & De Valois, 1990). We found the best performance for motion contour orientation detection at the lowest measured spatial frequency of 0.1 c/deg (much lower than with continuous luminance patterns) and did not observe a reduction of performance at the low spatial frequency end, which may partly be due to the physical limit on the lowest spatial frequency that we were able to test using a standard CRT. 

On the other hand, the comparison of limited lifetime and sparsely defined motion patterns with luminance patterns that are continuous in space and time, might not be considered entirely meaningful. It therefore should be more informative to compare the perception of motion-defined Gabor patterns with Gabor patterns that are defined in the luminance domain by a similarly sparse representation of limited lifetime luminance dots. In the second set of experiments we tested luminance-defined Gabor patterns generated from isolated dots which had limited lifetime, in an effort to match the dynamical properties and density of local information that needs to be spatially and temporally integrated over each trial. The limited lifetime dots do produce random motion signals, but not in any way that would result in a cue to the orientation of the luminance-defined contours. 

Methods

Stimuli

The same apparatus and stimuli were used as in Experiment 1, except the Gabor pattern now determined the luminance of the dots ranging from white (146 cd/m ²) to black (0.5 cd/m ²), with the background a light gray as before (73 cd/m ²). In areas where the Gabor pattern had absolute values less then 0.1 (the gray area in the previous diagrams), the luminance of the dots was chosen randomly from the same luminance range as above. Dots were not moving, but had a limited lifetime of 3 frames. The stimulus duration was always 15 frames (250 ms), apart from Experiment 2d, for which duration was again manipulated in a staircase. See Movie 2 for a demo of this dynamic and sparse luminance-defined Gabor pattern, without the fixation point used in the experiments. 

 

Movie 2

 

A demo of the dynamic and sparse luminance-defined Gabor pattern. The pattern is oriented vertically. As before there are two spatial cycles visible. Larger dots are shown in the demo than were used in the experiment.

Participants

Three well-practiced observers (SD, AM and TH) and 7 observers naive to the purpose of the experiment (different ones from Experiment 1) who had never seen the stimulus before were tested. 

Procedure

The task was again to report the vertical or horizontal orientation of the contours. The proportion of correct responses for horizontal and vertical Gabor patterns was recorded separately for the two orientations and averaged, after no difference in performance was found for the different orientations. For Experiments 2a– 2c each data point was the number correct from 32 judgments (8 judgments were made on each of the four conditions: horizontal/vertical orthogonal/parallel). For the practiced observers (SD, AM, and TH) each data point was measured four times and mean and standard error of mean expressed in terms of proportion correct were plotted. For the naive observers each data point was measured once and the mean of the group and the standard error of mean of the group expressed in terms of proportion correct were plotted. In Experiment 2d a standard simple up–down staircase procedure (Levitt,1971) was used that altered the stimulus duration up or down a number of frames according to whether the response was correct for a randomly chosen vertical/horizontal contour. All spatial frequencies were interleaved and blocks with different dot densities were presented in a pseudo-random order. The threshold was calculated from the mean of the last 5 reversals from 7 and the standard error of this mean was used in calculations. The maximum number of frames presented was 60 frames and the minimum was 1. A single staircase was completed for each spatial frequency at each dot density for each subject. In Experiment 2e each data point was the proportion correct out of 10 responses measured for times for each participant and mean and standard error of mean expressed in terms of proportion correct were plotted. 

Results

Experiment 2a and 2b: Envelope size

In Experiment 2a we measured the low spatial frequency (0.1 c/deg) condition and used the same number of dots (SD: 12, AM:12, naive group: 100) as in Experiment 1a and got very similar results with increasing size ( Figure 5a) as with the motion-defined stimuli. The 90% points of the saturation levels were found again by fitting the logistic function and were similar to Experiment 1a (fwhh SD: 4.1°, AM 4.9°, group 4.9°). In Experiment 2b for the higher spatial frequency (0.5 c/deg) condition however ( Figure 5b), more dots were required in the luminance condition for the three practiced observers (84 dots), than in the motion condition to achieve the same level of performance. With this increased number of dots in the higher spatial frequency condition measured in Experiment 2b we get an increasing, saturating function as in the motion condition (compare Figure 5b with Figure 2b). Beyond the overall resemblance of these functions, there are some small differences, for AM notably the performance level saturates at smaller envelope sizes than in Experiment 2a and for TH the results are fairly noisy. The 90% point of the saturation levels were SD: 4.2°. AM: 2.0°, TH: 5.0°, similar to the high spatial frequency motion condition in Experiment 1b, apart from for AM. 

Figure 5

 

Proportion correct of horizontal/vertical judgments for Gabor patterns defined by sparse luminance dots. Keeping spatial frequency fixed and varying envelope size. Refer to Figure 2 for a schematic illustration of the stimuli, the luminance of the Gabor patterns represents the spatial luminance distribution from which the dots are sparsely sampled. If a dot is from the gray area its luminance is randomly chosen. SEM bars shown. (a) Spatial frequency = 0.1 c/deg. 90% points of saturation from the fitted logistic function were SD: 4.1°, AM: 4.9°, group: 4.9°. Number of dots: SD, AM = 12, group = 100. (b) Spatial frequency = 0.5 c/deg. Fitted 90% of saturation were SD: 4.2°. AM: 2.0°, TH: 5.0°. Number of dots = 84 for all participants.

Experiment 2c and 2d: Carrier spatial frequency

We used the 90% point of the fitted maximum from Experiment 2a to choose a fixed envelope size at which we varied carrier spatial frequency in Experiment 2c (rounded to the nearest size we measured, except for AM, we chose one size larger as by inspection of the graph from AM in Figure 5a we were not confident that saturation had been reached by the point given by the fit; sizes used were SD: 4.7, AM 5.9, group: 4.7). The participants and dot numbers were the same as in Experiment 2a (SD:12, AM:12, group:100). A remarkably similar pattern occurs as with the motion-defined contours in Experiment 1c, with a clear drop in performance as spatial frequency increases (compare Figures 3a and 6a). This is at odds with the usual psychophysical contrast sensitivity function (De Valois & De Valois, 1990), which suggests that humans detect continuous luminance patterns best at medium spatial frequencies (of around 5 c/deg), dropping off both at the high and the low end. By defining the luminance pattern sparsely in time and space the characteristics of the detection mechanism appear to change from when the pattern is continuous. 

Figure 6

 

(a) Proportion correct of horizontal/vertical judgments for Gabor patterns defined by sparse luminance dots. Varying spatial frequency and keeping envelope size fixed, (fwhh SD: 4.7°, AM: 5.9°, group: 4.7°, number of dots SD: 12, AM: 12; group: 100). Refer to Figure 3a for illustrations of the spatial distribution of the luminance of the dots. Standard error of mean (SEM) bars calculated from individuals over four repetitions and for the group are the error of the group mean. (b) Threshold duration for correct horizontal/vertical judgments for Gabor patterns defined by sparse luminance dots as a function of spatial frequency. Fixed spatial envelope at 4.7° fwhh. In the top panel each of 5 participants' data with SEMs are shown. In the lower panel the mean threshold duration over participants and the inverse of the mean threshold duration for comparison with part (a) of this figure are shown.

In Experiment 2d we again used a staircase method that enabled us to simultaneously test all participants on the same scales, and consider the effect of dot density, in particular enabling us to move from a sparse representation closer to the continuous patterns typically used to measure contrast sensitivity as described above. We tested with 12 and 100 dots again to ensure the results from Experiment 2c are reproduced with this different method and then also tested higher densities of 520 and 1500 dots. We fixed the envelope size at fwhh = 4.7°. 

Five participants took part in Experiment 2d and their results are shown in Figure 6b. Again there was some variability between participants. Similar patterns are found for the 12 dot and 100 dot conditions as before, with peak performance for the single contour. However in this case with increasing dot density we find a different pattern of performance than with the motion-defined contours. In the two high dot density conditions we see distinct spatial frequency band-pass tuning emerging. As dot density increases, so the peak of the performance curve moves towards higher spatial frequencies. With 1500 dots we find a peak at around 2 c/deg, the lower end of estimates for the peak of contrast sensitivity found for continuous luminance patterns (De Valois & De Valois, 1990). We can see a clear continuum here between the sparse conditions where spatial frequency tuning resembles the motion domain and as the peak shifts we find a difference between the two domains emerging. Again, a threshold of around 60 frames suggests that participants were not able to perform the task, as we can see for example for most of the points in the 12 dot condition for observer LC and AS ( Figure 6b). Similarly, here we observe that the minimum duration of 1 frame causes ceiling effects for high dot densities for observer SD and AS (i.e. they are able to perform the task with a single frame). 

Experiment 2e: Envelope shape

As a similar pattern is observed for sparsely defined luminance contours as for motion contours in the two conditions above, in Experiment 2e we also investigated our ability to detect differently shaped sparse luminance patterns, elongated in different directions relative to the contours, as we did for motion contours in Experiment 1e. The same four shapes consisting of “small,” “narrow,” “wide” and “big” were tested (see Figure 4 for an illustration). As in Experiment 1 the “small” condition was the usual Gabor pattern with a fwhh of 1.2° extended the same size in both directions, in the “big” condition the Gabor pattern had a fwhh of 5.9° also circular in boundary shape. The “narrow” pattern had fwhh of 1.2° orthogonal to the contours and 5.9° parallel to contours, and in the “wide” condition this was the other way round, i.e. the Gabor was elongated orthogonal to the contours. Spatial frequency was fixed at 0.5 c/deg, the same as in Experiment 1b. The same number of dots (84) was used as in Experiment 2b initially, but for TH, as we can see from Figure 5b, the performance with this many dots and a 5.9° fwhh envelope Gabor pattern at a spatial frequency of 0.5 c/deg (the same as the “big” condition) envelope is still not very high, so we increased the number of dots to 120. The results of this experiment are shown in Figure 7. 

Figure 7

 

Proportion correct of horizontal/vertical judgments for Gabor patterns defined by sparse luminance dots. The spatial extent of the Gabor pattern was varied differently in the two spatial dimensions and spatial frequency was fixed at 0.5 c/deg. Envelope sizes: small—1.2° in both directions; narrow—1.2° orthogonal to contours, 5.9° parallel to contours; wide—1.2° parallel to contours, 5.9° orthogonal to contours, big—5.9° in both directions. Dashed lines on graphs show performance level at chance. See Figure 4 for an illustration of the spatial luminance distribution of the dots forming the patterns. Number of dots SD: 84, AM: 84, TH: 120. SEM bars calculated for averages from four repetitions.

The overall pattern of results is not as clear-cut as for motion contours. Although two participants show a similar pattern of results as before, and a significant main effect of shape was observed for all three participants (ANOVA results SD: F _3,12 = 6.63, p < 0.01; AM: F _3,12 = 4.62, p < 0.05; TH: F _3,12 = 6.84, p < 0.01), a Tukey post-hoc test reveals no significant difference between the “narrow” and the “wide” conditions, only between “small” and “big” for SD and between “big” and all other conditions for TH at the p < 0.05 level. There is no indication in any of the observers of a suppressive effect when introducing more contours. There is however, some indication for a similar asymmetry between information along the contour and information over space away from the contour, suggesting that in the presence of a stimulus containing a sparse luminance pattern the shape of the contour detector may be similarly elongated along a single contour, as we suggested for the motion domain. 

General discussion

The aim of this work was to investigate the properties of the underlying mechanisms that enable us to detect a discontinuity between areas of uniform motion, by using a motion-defined Gabor pattern made of limited lifetime random dots moving at a speed determined by the Gabor function (see Figure 1). How does the detection of contours in this domain compare with what we know about the detection of luminance-defined contours and the properties of motion detectors upon which such contour detection must rely? First we found that motion-defined contours are processed on a large scale. We showed that the ability to detect the orientation of the motion-defined Gabor pattern grows with larger envelopes and only saturates at 4–5° of fwhh of the Gaussian envelope, regardless of the spatial frequency (see Figure 2). We found that not only do additional motion contours not aid orientation detection, but when we varied spatial frequency at a fixed size the best performance occurred at the lowest spatial frequency measured, where only a single motion contour was present and dropped off with increasing spatial frequency, as more motion contours became visible (see Figure 3). This suggests that detectors that respond to changes in the motion field over space are specialized for detecting motion edges rather than fulfilling the role of spatial frequency analyzers that may be used to reconstruct some kind of motion texture surface, the role often ascribed to Gabor-like receptive fields. The data agree with psychophysical studies using moving gratings and uniform random dot motion to investigate the properties of motion detection mechanisms. Large motion sensitive receptive field sizes, of above 9° diameter in humans have been found (Anderson & Burr, 1987; Watamaniuk & Sekuler, 1992) (5° fwhh corresponds to a diameter of about 8.5°) and motion sensitivity for moving luminance gratings with spatial frequencies as low as 0.03 c/deg (Anderson & Burr, 1989). However this differs from the results of Watson and Eckert (1994) who did find a decline in sensitivity for low spatial frequency motion-defined gratings in a similar range that we measured. Their stimulus was quite different to ours as they used dense random noise dot fields and manipulated what they called motion contrast to measure detection thresholds for motion contours: They used overlapping upwards and downwards motion where the contrast for each was varied out of phase, so that in the high motion contrast condition this would appear as motion stripes. However, in the low motion contrast condition both types of motion would be visible in one location to some extent, possibly causing a percept of transparency. As mentioned, the motion signal detection task they used does not clarify whether segregated or transparent motion was seen. Similarly in the study by Sachtler and Zaidi ( 1995) where band-pass tuning was found for dense sinusoidal motion-defined gratings (although not square-wave ones), the task was to detect “motion” versus “no motion,” which does not require detecting the motion boundaries per se. 

Transparency can be thought of as a different process to seeing segregated motion; indeed MT responses are shown to vary differently with parameters such as dot density when two motions are present at once in the same visual area than when only a single motion is present (Snowden, Treue, Erickson, & Andersen, 1991). None of the participants reported seeing our stimulus as transparent. This is as we would expect as although the dots had opposite directions, they also had differing speeds, which can make grouping into transparent layers more difficult. The possible perception of transparency in the Watson & Eckert study might be one explanation of the differing results, alternatively they may be caused by the different speeds used—their fixed motion speed was only 0.94°/s, whereas we had peak speeds of 10°/s. 

By using elongated Gabor patterns we found that the improved performance with increasing size is not due to additional information from over the whole pattern, but instead the increasing length of the central contour and the motion signals along it. In fact, we find that the presence of additional contours to a small extent inhibits the ability to detect the orientation of a motion-defined contour. This indicates the existence of an elongated motion contour detector with a large RF with uniform areas of opposite direction preference either side of the central contour causing the impairment from “narrow” to “big” and “small” to “wide” seen in Experiment 1e. Furthermore it reveals no evidence of a further mechanism that combines motion contours. This further suggests that in order to detect the orientation of these motion-defined contours we are not using the same pattern analysis techniques that have been suggested as useful properties of using a Gabor-type filter which is circularly shaped (Clausi & Jernigan, 2000; Jain, Ratha, & Lakshmanan, 1997). This makes intuitive sense as motion pathways are not considered useful in the detailed analysis of local texture, because of the large receptive field sizes, but rather give large grain information on changes over time. We can think of motion-defined contours as the spatial positions at which such large grain changes in spatiotemporal structure occur. This also may be relevant when considering the properties of other second-order stimuli as it has been found that there appears to be no further mechanism for linking second order contours together across space (Hess, Ledgeway, & Dakin, 2000). 

The relationship between the size of the pattern and the ability to detect the orientation of a motion-defined Gabor resembles previous results for the detection of continuously luminance-defined Gabor patterns in as much as we observe saturation over a spatial area (Hoekstra et al., 1974; Syväjärvi et al., 1999). However, the low-pass tuning for spatial frequency we observed with motion-defined Gabor patterns differs from the band-pass contrast sensitivity found for continuous luminance gratings (De Valois & De Valois, 1990; De Valois, Morgan, & Snodderly, 1974). We do need to bear in mind that we are limited as to how low we can go when measuring spatial frequency, reduction of performance levels may occur at lower spatial frequency levels. Many previous studies have considered the type of elongated Gabor stimuli we used here in the luminance domain (Foley et al., 2007; Meese & Hess, 2007; Polat & Tyler, 1999), with mixed results, but from this literature there is no overwhelming evidence for a difference between the collinearly and orthogonally extended stimuli. Moreover, these studies generally resulted in worse detection rates or sensitivity than for circular stimuli, even when matched for area. In the motion domain we find clear differences between the two elongated conditions ( Figure 4). These are in fact, if anything, more detectable than their circular counterparts, when elongated along the contour. This further confirms the different role played by the motion contour detector, which is not so involved in local spatial frequency analysis and is specialized to pick up information along a single border. 

Our results agree more with results found from the detection of 2 ^nd order stimuli defined by spatial changes in orientation or contrast rather than luminance. These show spatial summation over large areas and peak tuning can be as low as 0.1 c/deg depending on the properties of the pattern that is being modulated (Kingdom et al., 1995; Sutter et al., 1995). They do however in some cases show band-pass tuning, but this is dependent on the properties of the pattern that is being manipulated. However, it has also been suggested that when first-order artifacts have been accounted for the spatial frequency tuning curves for different types of second-order stimuli become similar and are low-pass (Schofield & Georgeson, 2003). It is hard to determine how the filtering that is the first stage of this analysis compares with the motion extraction that must be the first stage of the motion contour analysis. In order to address this in more detail in future it would be interesting to manipulate the properties of the motion e.g. different peak speeds or directions and also to vary the viewing distance to see if size invariance holds with motion-defined contours also. In this work we have focused on the similarities between motion- and luminance-defined contours. Interestingly however, 2^nd order patterns have also shown that the density of the patterns is not crucial in determining the peak spatial frequency (Kingdom et al., 1995), as we have found with motion-defined Gabor patterns, suggesting similar comparisons between second-order stimuli and sparsely defined first-order stimuli can be made. We have found some similarities between the size and spatial frequency tuning for motion-defined contours with those of second-order contours defined by contrast or orientation. This could be due to these stimuli feeding into the same second-order stages after they are extracted by separate first-order mechanisms—or equally may simply reflect the fact that these types of stimuli are useful cues to similar types of relevant information in the visual scene requiring similar tuning properties, but could still be processed independently at all stages. Our data do not distinguish between these two possibilities. 

We then considered what would happen to the detection of luminance-defined contours if we reduced these from continuous patterns to more sparsely sampled stimuli represented only in places by the luminance of dots. We found that at the highest density of dots tested the results for performance of orientation detection of the contours resembled previous results of detection performance with continuous luminance-defined contours, a band-pass shape with a peak at much higher spatial frequency than for motion-defined contours ( Figure 6b). However, as we reduced the spatial sampling we found that the pattern of performance with luminance-defined Gabor patterns resembled that of motion-defined contours ( Figures 5 and 6). This is interesting as it suggests that the shape of the response changes with low densities, now preferring the lowest spatial frequency single contour and aggregating over a large spatial area, although there seemed to be some integration of contour information away from the central contour when we contrasted the elongated patterns with the circular patterns. The shape of the spatial frequency tuning curve obtained with low dot densities in the luminance domain resembles the spatial frequency tuning curve at all densities in the motion domain. As noted before all the luminance-defined contour stimuli do contain spurious motion signals caused by the limited lifetime dots, however if this “flicker” were to change the spatial frequency tuning of contour detection, we should see this effect at all dot densities. The different spatial frequency tuning curve shapes at different densities suggest that tuning properties are dependent on the amount of information present in the stimulus. On the other hand the fact that the same spatial frequency tuning curve shape is found for all densities for motion-defined contour, shows that this detection process may be limited by the sampling of the motion system, not the stimulus alone. Our results with the sparse luminance-defined Gabor patterns suggests that the distinction may not entirely be between the motion and the luminance domain, but rather between sparsely and densely represented visual stimuli. 

It is plausible that spatial maps of motion are more sparsely represented in the visual system than luminance maps. MT and MST are thought to be associated with extracting global motion and are known to have larger receptive fields than V1 at corresponding eccentricities in macaques (Van Essen, Maunsell, & Bixby, 1981). For coherent motion detection both spatial and temporal integration needs to take place, requiring large receptive fields, possibly resulting in a more sparsely defined motion field than that of luminance. It has also been shown in macaques that MT and MST response increases sharply with a small number of dots, leveling out at a density of 0.25 dots/deg², after which dot density does not have much effect on response (Duffy & Wurtz, 1991; Snowden et al., 1991). The lowest dot density we used was 0.96 dots/deg². If motion contour integration relies on these higher level motion areas then the dot densities we used was not the limiting factor for integration size and spatial frequency resolution. However, neurophysiological studies in macaques and human fMRI data investigating tuning for such kinetic contours remains equivocal as to the areas involved. In macaques area V4 (Mysore, Vogels, Raiguel, & Orban, 2006) and the inferior temporal (IT) cortex (Sáry, Vogels, Kovács, & Orban, 1995) have been implicated and in humans there is some debate about whether a specialized area ‘KO’ for these contours exists (Van Oostende, Sunaert, Van Hecke, Marchal, & Orban, 1997), but also areas V3 and V5/MT (Smith, Greenlee, Singh, Kraemer, & Hennig, 1998; Zeki, Perry, & Bartels, 2003) have not been ruled out. Although there has been some evidence for response to motion-defined contours in V1 and V2, it has been argued that this is a result a feedback from higher motion specialized areas (Orban, 2008). 

With both type of contours it should be noted that we also found a considerable learning effect, leading to variable performance amongst observers according to practice, although as we saw, without affecting the shape of the tuning functions. This may reflect increased efficiency of information integration with learning which could be investigated in the future. 

In conclusion we have shown that motion contour detection relies on large integration areas and operates at low spatial frequencies, preferring a single motion boundary, regardless of the density of motion signals. When comparing this performance with sparsely sampled luminance contour detection it appears that limiting the amount of information available in the stimulus can affect the properties of the neural mechanisms for luminance contour detection. If we limit visual test patterns to sparsely defined stimuli we observe strikingly similar patterns and levels of performance for motion- and luminance-defined contours. For motion-defined contours the sparseness of the representation may be inherent in types of mechanisms needed to calculate global motion. This suggests that similar principles may underlie the detection of sparse luminance-defined contours and motion-defined contours even though the mechanisms operate at different processing levels of the visual system. 

Acknowledgments

We would like to thank Andrew Meso, Tim Holmes, and Frouke Hermens for their useful comments and suggestions, as well as all the participants. This work is supported by EPSRC grant number EP/C015061/1. 

Commercial relationships: none. 

Corresponding author: Szonya Durant. 

Email: [e-mail]. 

Address: Department of Psychology, Royal Holloway University of London, Egham, TW20 0EX, Surrey, UK.