Two mechanisms for optic flow and scale change processing of looming

Abstract

The detection of looming, the motion of objects in depth, underlies many behavioral tasks, including the perception of self-motion and time-to-collision. A number of studies have demonstrated that one of the most important cues for looming detection is optic flow, the pattern of motion across the retina. Schrater et al. have suggested that changes in spatial frequency over time, or scale changes, may also support looming detection in the absence of optic flow (P. R. Schrater, D. C. Knill, & E. P. Simoncelli, 2001). Here we used an adaptation paradigm to determine whether the perception of looming from optic flow and scale changes is mediated by single or separate mechanisms. We show first that when the adaptation and test stimuli were the same (both optic flow or both scale change), observer performance was significantly impaired compared to a dynamic (non-motion, non-scale change) null adaptation control. Second, we found no evidence of cross-cue adaptation, either from optic flow to scale change, or vice versa. Taken together, our data suggest that optic flow and scale changes are processed by separate mechanisms, providing multiple pathways for the detection of looming.

Introduction

The perception of visual expansion, or looming, is fundamental to determining how we move through the world, and how objects move around us. Looming information is conveyed through several visual cues, including optic flow, angular expansion and binocular disparity. Optic flow has long been considered a primary cue for the detection of looming, particularly during self-motion, for human observers (Gibson, 1950; Warren, 1998). Recently, it has been proposed that observers also can detect looming from changes in spatial frequency over time, termed “scale change”, in the absence of motion (Schrater, Knill, & Simoncelli, 2001). 

The perception of scale changes as a cue for looming stems from the association between spatial frequency and depth: as a textured object approaches, the scale of its textures increases, resulting in a decrease of the spatial frequencies of those textures. This relationship has been demonstrated in static stimuli as the ability to use spatial frequencies to perceive surface tilt despite confounded binocular disparity cues (Tyler & Sutter, 1979). Schrater et al. (2001) were the first to demonstrate that observers could estimate the rate of expansion of a visual stimulus based solely on the change in spatial frequency content. They showed that subjects could accurately adjust the speed of an expanding random dot field to match the rate of change of spatial frequencies, indicating that they perceived scale changes proportionally to the expansion rate of optic flow stimuli. Schrater et al. (2001) also showed that prolonged viewing of scale change stimuli induced perceived after-effects, suggesting that observers used a perceptual processing mechanism for scale changes, rather than cognitive strategies. 

Their results demonstrated the ability of observers to detect expansion from scale changes in the absence of optic flow. This raises the question of whether the visual system uses a single mechanism for the detection of both optic flow and scale change, or whether separate mechanisms exist for detecting expansion from these two cues. The fact that observers could detect scale changes implies that subjects were sensitive to changes in spatial frequency over time. The detection of optic flow and scale changes may involve a shared mechanism if both mechanisms draw upon the same low-level feature or local motion detectors, or if outputs from both mechanisms are combined into a cue-invariant looming perception. 

The existence of separate pathways for the detection of optic flow and scale change is indirectly supported by psychophysical and physiological studies of the spatial frequency tuning properties of neurons in the motion processing system. Much of early visual processing is segmented into spatial frequency channels (Blakemore & Campbell, 1969; Campbell & Robson, 1968; Foster, Gaska, Nagler, & Pollen, 1985; Wilson, McFarlane, & Phillips, 1983) that provide a potential basis for the computation of scale changes. For motion detection, speed is specified as the ratio of temporal to spatial frequency, such that many combinations of spatial and temporal frequencies correspond to the same speed, with frequency components and speeds contributing at different levels of motion processing (McKee, Silverman, & Nakayama, 1986; Vaina et al., 2003). Global motion processing has been shown to be primarily sensitive to speeds, not spatial frequencies (Newsome, Gizzi, & Movshon, 1983; Perrone & Thiele, 2002) by pooling across spatial frequencies (Amano, Edwards, Badcock, & Nishida, 2009; Priebe, Cassanello, & Lisberger, 2003). While local motion detectors show narrow spatial frequency tuning, global motion involved in optic flow detection is broadband (Bex & Dakin, 2002), suggesting that global motion mechanisms are not tuned for spatial frequencies. Since detecting scale changes is based on the detection of spatial frequency content over time, it seems unlikely that global motion mechanisms would support cue-invariant looming processing, suggesting that the detection of optic flow and scale changes may require separate mechanisms. 

To determine whether the detection of looming from scale changes and optic flow are mediated by a single or separate mechanisms, we used an adaptation paradigm to measure whether humans employ the same sets of neurons for the detection of both cues. Adaptation is a paradigm for assessing the nature of basic visual mechanisms and representations by measuring the degree to which the perception of two stimuli relies on the same set of neurons. In selective adaptation, prolonged exposure to a stimulus renders the observer (and the neurons responding to the stimulus) less sensitive (Blakemore & Campbell, 1969). Adaptation to motion has been reported at both the behavioral level (Beverley & Regan, 1973), and the neuronal level in area MT (Kohn & Movshon, 2003, 2004; Petersen, Baker, & Allman, 1985; Van Wezel & Britten, 2002). Adaptation effects can also indicate the degree to which the same neurons are shared by two tasks: a reduction in performance following adaptation to a different stimulus can indicate that the two tasks share at least part of their neural circuitry, for example, in direction discrimination (Levinson & Sekuler, 1975) and 1st and 2nd order motion (Nishida, Ledgeway, & Edwards, 1997) tasks. 

We used an adaptation paradigm similar to that used by Kohn and Movshon (2003) and Nishida et al. (1997) to investigate cross-stimulus adaptation effects on processing of optic flow and scale change. We measured performance on both tasks after prolonged adaptation to scale change, optic flow or a static image baseline condition. Within-stimulus adaptation effects (performance on optic flow after adaptation to optic flow and performance on scale change after adaptation to scale change) were considered as control conditions to determine whether our stimuli produced adaptation related performance changes. We expected that for within-stimulus adaptation there would be a reduction in performance for both cue conditions. Cross-stimulus adaptation (i.e. performance on scale change after adapting to optic flow, and vice versa) measured the degree to which the two tasks share a common neural substrate. A lack of cross-cue adaptation implies separate mechanisms, while bidirectional adaptation supports a single, common, cue-invariant looming mechanism. 

Our results showed strong within-cue adaptation, demonstrating that the stimuli successfully produced adaptation. However, when adapting to and testing with different cue types, we found no evidence of cross-cue adaptation. This trend persisted when adapting to the specific speeds used to generate the test stimuli, and when adapting to the perceived speed of the stimuli (which was reduced when we added noise). We suggest that these results indicate that the perception of optic flow and scale changes are mediated by separate mechanisms, indicating that the human visual system may utilize multiple mechanisms for the detection of motion in depth. 

Methods

Subjects (n = 9, seven male, two female, ages 18–30, mean 22 ± 4, including two of the authors, FC and KR) were tested with artificial looming stimuli containing either optic flow or scale changes. Stimuli were presented for 700 ms simulating motion either towards or away from the observer. To reduce cues at the edge of the stimuli, a sigmoidal contrast window was applied to every frame of all stimuli (see Figure 2). The result was a circular stimulus of radius 10.5°, with maximum contrast between 0 and 4.75° eccentricity and then smoothly fading to the background luminance (and 0% contrast) between 4.75 and 10.55° eccentricity. The stimulus size and contrast filter were the same size for all frames so that there was no change in size during the simulated motion. Subjects were placed 60 cm from a CRT display, and used a chin rest to ensure correct viewing distance and to stabilize head movement. Throughout testing trials, subjects fixated a square mark presented in the center of the display. 

Test stimuli

Both optic flow and scale change stimuli were created based on manipulations of a bandpass looming stimulus. The looming stimulus was first created from a white noise image passed through a finite impulse response filter (center frequency 2 cycles/deg, bandwidth 0.4 cycles/degree, order n = 50). Looming was simulated by zooming in on the filtered white noise image. The rate of the zoom was chosen consistent with motion at a specific simulated speed of motion-in-depth (chosen as 3 m/s, which was detectable at near-100% rates for both optic flow and scale change, based on preliminary results). This looming stimulus contained both optic flow (image features moved radially outward), and scale changes (the spatial frequency content decreased as the image expanded, seen as an increase in the size of texture features). 

Optic flow stimuli were derived from the looming stimuli by filtering each frame with a constant bandpass filter. Filtering did not alter the phase components of each frame, resulting in a stimulus sequence containing expanding motion (optic flow) devoid of any changes in spatial frequency (Figures 1A–1C). 

Figure 1

 

Illustration of optic flow (A–C) and scale change (D–F) stimuli. (A) and (D) show a horizontal cross-section of the optic flow and scale change stimuli respectively over time. (B) and (E) show motion vector field plots for each stimulus. (C) and (F) show the upper and lower cutoff frequencies of the passband over time for each stimulus. In the optic flow condition, the pattern of textures expand outward without a change in spatial frequencies, while the scale change stimuli contain no structured motion but do show a change in feature size as determined by a drop in spatial frequencies.

Stimuli were generated to contain only scale changes by using each frame of the looming stimulus as a spatial frequency filter applied to a set of randomly generated white noise images (one per frame of the stimulus). This created a sequence that contained the same spatial frequencies as the looming sequences, but since each frame was derived from a different white noise image, there was no correlation among frames, and therefore no optic flow (Figures 1D–1F). 

To control for the difficulty of the test stimuli, we created noise stimuli that contained neither optic flow nor scale changes, by iteratively filtering white noise images with a constant bandpass filter and selecting the resulting sequences to match the distribution of local speeds. Test stimuli were created as the weighted super-position of noise with either optic flow or scale change stimuli (we will refer to the relative weight of the test stimuli as the coherence), and were normalized to match the luminance distribution present in the original, noiseless stimuli. Lower coherence values had the effect of reducing observer performance to a range where the task was difficult, but in which adaptation effects would be visible if present, regardless of the speed used to create the stimuli. 

In both optic flow and scale change conditions, observers performed a 2AFC task and reported whether the stimulus simulated an approaching or receding trajectory Test stimulus difficulties were chosen as the coherence that resulted in approximately 75–80% detection rates for direction of motion, based on preliminary data from each observer. For test stimuli created based on a simulated motion in depth of 3 m/s, this varied between 18 and 30% coherence for optic flow stimuli, and between 25 and 60% for scale change stimuli. This difference reflects the relative difficulty in detecting looming from scale changes compared to optic flow. 

Adaptation stimuli

Subjects were tested with both optic flow and scale change stimuli in three conditions: (1) adaptation to optic flow, (2) adaptation to scale changes, and (3) adaptation to a dynamic control (null) stimulus. The null adaptation stimuli were identical to the noise stimuli that were added to the test stimuli, and contained neither optic flow nor scale changes. Adaptation to these null stimuli was used to measure baseline performance and ensure that changes in performance could not be explained by adaptation to dynamic noise within specific spatial frequencies bands. Optic flow and scale change adaptation periods consisted of a test stimulus shown expanding and contracting in a loop. The use of both expansion and contraction in the adaptation periods was intended to adapt sensitivity to both directions of motion, and to reduce response biases arising due to directional motion aftereffects. All test and adaptation stimuli were generated using a simulated speed of motion-in-depth of 3 m/s. 

Paradigm

The adaptation paradigm consisted of motion-in-depth discrimination for optic flow and scale change stimuli interleaved within a testing block following prolonged adaptation (see Figure 2). Each testing run started with a 40 sec adaptation period (“full adaptation”). The adaptation period was shown every 20 trials, with a “top-up”, 12 sec, adaptation period presented every five trials. This paradigm, adapted from (Kohn & Movshon, 2003), has been shown to reduce the required testing time while still producing strong adaptation effects. Fifty optic flow and scale change test stimuli were interleaved within a testing block, so that a full test block consisted of 100 trials. Each test run contained only a single adaptation type (static, optic flow or scale change), and subjects performed each run four times, for 200 total trials per adaptation-test combination. Subjects were given a break from testing of at least 5 min before switching adaptation stimuli to reduce long-term adaptation effects. The sequence of adaptation blocks was determined pseudo-randomly across subjects to ensure that subjects did not all perform the test conditions in the same order. 

Figure 2

 

Adaptation paradigm for scale change and optic flow detection. Full adaptation periods lasting 40 sec were presented every 20 trials, with a shorter, top-up (12 sec) adaptation period presented every 5 trials. Test trials included both optic flow and scale change conditions, but adaptation periods contained only a single test type per testing block.

Results

For each test stimulus, adaptation effects were measured as performance (percent correct) on the optic flow (OF) or scale change (SC) adaptation condition minus performance on the static adaptation condition. A negative adaptation value indicates that performance was worse following cue adaptation than in the static adaptation condition, suggesting that adaptation in that condition caused a reduced observer's sensitivity to the test stimulus. Four adaptation combinations were considered: within-cue adaptation (adapt OF-test OF, and adapt SC-test SC), and cross-cue adaptation (adapt OF-test SC, adapt SC-test OF). The within-cue adaptation conditions served as control conditions to ensure that our stimuli and paradigm were capable of revealing adaptation effects, while the cross-cue adaptation effects were of interest in determining the relationship between optic flow and scale change processing mechanisms. 

Within-cue adaptation effects (Figure 3A) were analyzed to ensure that the stimuli were effective in reducing performance when tested with the same cue type. Significant within-cue adaptation effects were found for both cue types. Detection of motion-in-depth from optic flow dropped by an average of 11.2% (standard error across observers SEM = 2.9%) following adaptation to optic flow (t = −3.89, p = 0.004, df = 8). Similarly, scale change detection dropped by 8.9% (SEM = 1.9%) following scale change adaptation (t = −4.62, p = 0.002, df = 8). These results show that both optic flow and scale change stimuli were successful in causing adaptation. 

Figure 3

 

Summary of individual results for adaptation effects. (A) Within-cue adaptation (circles are adapt OF-test OF, squares are adapt SC-test SC), and (B) cross-cue adaptation (circles are adapt SC-test OF, squares are adapt OF-test SC). In both plots, the x-axis indicates performance on the optic flow or scale change task in blocks adapting to a null stimulus, with adaptation to the specified stimulus cue on the y-axis. Circles indicate performance in detecting looming from optic flow, and squares are performance in detecting looming from scale change. Each data point is the performance for one observer. The dotted line indicates expected results if adaptation had no effect on performance; points below the line a decrease in performance following cue adaptation.

To determine whether optic flow and scale change share a neural processing stage, we measured cross-cue adaptation effects (i.e. testing optic flow after adaptation to scale changes, and vice versa; individual results are shown in Figure 3B). We found that after adapting to scale change, performance on the optic flow condition did not change significantly (0.9% improvement, SEM = 1.1%, t = 0.93, p = 0.4, df = 8). Performance on the scale change task was similarly unaffected following adaptation to optic flow (0.3% decrease, SEM = 1.8%, t = 0.19, p > 0.8, df = 8). These results (group averages summarized in Figure 4) imply that neither optic flow nor scale change cues elicited significant cross-cue adaptation effects. 

Figure 4

 

Summary of adaptation results averaged across subjects. Data are change in performance from a null adaptation condition to performance following adaptation to optic flow (dark gray) and scale change (light gray).

When we added noise to the test stimuli for controlling the difficulty of the task (so that the adaptors and test stimuli were generated using the same speeds) subjects reported that this had the effect of reducing the perceived speed of the stimuli. To be certain that we were adapting the perceptual mechanisms, we performed a control test in which we estimated the perceived speed of the noisy test stimuli, and used adaptors matched to these speeds (see 1). When adapting to stimuli matched to the perceived speed of the test stimuli, the magnitude of within-cue adaptation effects changed, but the data showed no evidence of cross-cue adaptation. This confirmed our initial finding that adaptation effects are limited to within-cue conditions. 

Discussion

We used an adaptation paradigm to address the independence of the mechanisms used to process looming from optic flow and scale change cues. Our results demonstrate that both cues became significantly adapted after prolonged viewing, resulting in decreased detection rates. An investigation of cross-cue adaptation performance showed that neither scale change nor optic flow detection was significantly affected by adaptation to the other cue (as shown in Figure 4). We propose that this result implies that the detection of scale changes and optic flow are mediated by separate mechanisms. 

One additional fundamental difference between scale change and optic flow cues is that while optic flow relies on changes in luminance over time, scale change cues are absent of any luminance-defined motion. This dichotomy in the motion system has been long studied as the difference between 1st order (luminance-defined) and 2nd order (contrast-defined) motion (Chubb & Sperling, 1988). The lack of cross-cue adaptation effects in our current study mirrors results from studies which have shown a dissociation between 1st and 2nd order motion processing, both behaviorally (Clifford & Vaina, 1999; Ledgeway & Smith, 1994, 1997; Nishida et al., 1997; Pavan, Campana, Guerreschi, Manassi, & Casco, 2009; Vaina & Cowey, 1996; Vaina, Makris, Kennedy, & Cowey, 1998) and in several neuroimaging studies (Ashida, Lingnau, Wall, & Smith, 2007; Vaina & Soloviev, 2004). The results we have presented also support findings from 1st and 2nd order optic flow processing that the dissociation of mechanisms underlying the detection of translational motion extend to the detection of looming (Badcock & Khuu, 2001). 

The possibility that scale changes are processed independently from optic flow suggests that they may provide an alternate, compensatory mechanism for the perception of looming when optic flow is obscured, or unavailable, such as in the case of patients selectively impaired on optic flow or in conditions when luminance does not provide a reliable cue. 

Appendix A

In preliminary data collection, subjects noted that reducing the coherence had the effect of reducing the perceived speed of the stimulus. This motivated us to repeat the adaptation experiment with adaptors matched to the perceived speed of the partially coherent test stimuli, rather than matched to the underlying, physical speed. In the main experiment, we matched stimuli based on the physical characteristics (the change in position and spatial frequency for optic flow and scale change respectively induced by a constant speed of simulated motion in depth) since integration would be most advantageous when using cues arising from the same physical stimulus. However, if inputs were combined later in the visual processing hierarchy, it is conceivable that it is the perceived, rather than physical, speed that is used for integration. This could imply that the speeds used in the main experiment were not appropriate for detecting adaptation effects. 

We first estimated from two subjects the perceived speed of the partially coherent scale change and optic flow stimuli using a 2AFC task comparing the noisy stimuli to fully coherent optic flow and scale change reference stimuli. In preliminary testing, we found that using the threshold level coherences was problematic since subjects had difficulty reporting the speed of stimuli whose motion they could not readily detect. We instead focused on supra-threshold stimuli (50% and 100% coherence) and measured the probability that a 50% coherent test stimulus was perceived faster than a 100% coherence reference as a function of the simulated speed of motion in depth used when generating the test stimulus. By fitting a sigmoid to the data and estimating the point of subjective equality (PSE, the test stimulus speed at which there was an equal probability that the test and reference would be perceived faster), we found that for within-cue comparisons, the perceived speed was well explained as the weighted average of the signal (optic flow or scale change) and noise components (Figure A1A). That is, a 50% test stimulus containing 50% motion at 3 m/s, and 50% noise (0 m/s) was perceived at approximately 1.5 m/s, suggesting the signal and noise components were being averaged perceptually. For cross-cue comparisons, scale change stimuli were perceived approximately 20% than optic flow stimuli generated using the same simulated speed of motion in depth, as indicated by a shift in the PSE (Figure A1B). 

Figure A1

 

Perceived speed of partially coherent test stimuli. (A) Comparison of a 50% test to a 100% reference stimulus (moving at 3 m/s) of the same stimulus type (filled symbols for optic flow, open symbols for scale change). (B) Comparison of a 50% test to a 100% reference stimulus (moving at 3 m/s) of the opposite stimulus type. Filled symbols (OF/SC) indicate an optic flow reference and scale change test stimulus, open symbols (SC/OF) indicate a scale change reference and optic flow test. The solid curve fit applies to optic flow reference data, and the dashed curve fit shows the scale change reference. The dash horizontal line indicates 50%, at which the reference and test stimuli were perceived as moving equally fast. Circles and squares reflect data from two subjects (including one of the authors, FC) on the task.

Using these relationships, we repeated the adaptation experiment in 3 subjects, reducing the adaptor speed to match the perceived speed of the test stimuli (Figure A2). As in the main experiment, we found within-cue adaptation effects for optic flow and scale change. For two subjects, the effect of the optic flow adaptor had a much stronger effect than in the original experiment, reducing their performance to near chance. The effect of scale change adaptation, however, was reduced. This was likely because the scale change adaptor was much weaker (and closer to the speed detection threshold). However, we found no cross-cue adaptation effects for ether scale changes nor optic flow, indicating that even when using adaptors matched to the perceived speed of the stimuli, there appears to be a separation of pathways underlying the processing of these cues. 

Figure A2

 

Performance of 3 observers for optic flow and scale change detection following adaptation to a stimulus matched to the perceived speed of the test stimuli.

Acknowledgments

We thank Jonathan Victor and the anonymous reviewers of the paper for feedback and suggestions regarding the stimuli used here. This work was supported by NIH grant R01NS064100 to LMV. 

Commercial relationships: none. 

Corresponding author: Lucia M. Vaina. 

Email: [email protected]. 

Address: Brain and Vision Research Laboratory, Department of Biomedical Engineering, Boston University, 44 Cummington Street, Boston, MA 02134, USA.