Bilocal visual noise as a probe of wide field motion computation

Abstract

Using an apparent visual motion stimulus with motion energies limited to specific separations in space and time, we study the computational structure of wide-field motion sensitive neurons in the fly visual brain. There is ample experimental evidence for correlation-based motion computation in many biological systems, but one of its central properties, namely that the response is proportional to the product of two bilocal signal amplitudes, remains to be tested. The design of the apparent motion stimuli used here allows us to manipulate the amplitudes of the bilocal input signals that serve as inputs to the computation. We demonstrate that the wide-field motion response of H1 and V1 neurons indeed shows bilinear behavior, even under contrast sign reversal, as predicted. But the response also varies inversely with contrast variance, an effect not described by the correlator operation. We also quantify the correlator contributions for different spatial and temporal separations. With suitable modification, the apparent motion stimuli used here can be applied to a broad range of neurophysiological as well as human psychophysical studies on motion perception.

Introduction

The visual system has evolved neural circuits that allow the extraction of motion information from spatiotemporal patterns of light. Attempts at resolving the functional computational structure of these circuits have led to the development of several models for motion detection. The Hassenstein-Reichardt correlator (HRC) model, originally developed to explain the optomotor response of a beetle, postulates that motion detection arises from correlation of time delayed light signals from two spatial points (Hassenstein & Reichardt, 1956; Reichardt, 1961). Barlow and Levick (1965) proposed a mechanism for direction-selective computation in retinal ganglion cells that combines time-delayed excitatory and inhibitory inputs from the presynaptic bipolar cells. The motion energy model describes a spatiotemporal filter approach to understanding motion perception in humans (Adelson & Bergen, 1985). Despite the apparent mechanistic differences, these models share an irreducible computational principle for motion detection, namely the nonlinear combination of light signals measured at spatiotemporal offsets. 

The direction of motion computed from a moving visual pattern should be invariant to the sign of its contrast. This simple requirement directly implies that the computation of motion from a measurement of spatiotemporal contrast cannot be linear and, indeed, the lowest order polynomial interaction with this invariance is bilinear (Hassenstein & Reichardt, 1956; Potters & Bialek, 1994; Reichardt, 1961). This second order interaction also is the basis for computing correlation, and many psychophysical and neurophysiological studies therefore have designed stimuli based on variants of correlated random noise, such as dynamic visual noise (Morgan & Ward, 1980) and Julesz-type patterns (Braddick, 1974; van de Grind, Koenderink, & van Doorn, 1986; van Doorn & Koenderink, 1982a, 1982b; van Doorn, Koenderink, & van de Grind, 1985), to assess perceptual thresholds for motion detection in humans. A similar approach was used in the study by Britten, Shadlen, Newsome, and Movshon (1992), which involved the use of moving random dots with limited lifetime for studying how neurons in the MT (middle temporal) region of monkeys process motion. One advantage of these types of stimuli is that they lack spatial structures such as edges and large correlated patterns, which eliminates any response that could have been evoked specifically by these structures. That makes these stimuli particularly amenable for testing computational principles underlying motion perception. 

The computation of stimulus correlation is implemented directly in the HRC model, which also has been one of the most widely used models for understanding neural and behavior responses to motion in flies (Borst & Egelhaaf, 1989). For example, the response properties of wide-field motion sensitive neurons in the lobula plate support a correlation type mechanism similar to the HRC scheme (Egelhaaf, Borst, & Reichardt, 1989). More recent work on mutant Drosophila (Gal4-UAS system) has identified specific neurons in the medulla region that correlate signals derived from neighboring spatial locations (Schnell, Raghu, Nern, & Borst, 2012). The reverse-phi illusion, which induces robust optomotor response in flies, can also be explained using a correlation-type nonlinearity (Tuthill, Chiappe, & Reiser, 2011). Although these findings lend credence to the correlation scheme, it remains unclear to what extent correlation-based nonlinearities dominate visual motion computation under more complex visual conditions experienced during natural flight. It is known, for example, that variability in the spatial structure of the input, the speed of movement, and contrast substantially modulate gain and time-course of the response to motion (Egelhaaf & Borst, 1989; Maddess & Laughlin, 1985; O'Carroll, Barnett, & Nordstrom, 2011). Furthermore, there is ample evidence for various adaptive mechanisms in motion processing (Brenner, Strong, Koberle, Bialek, & de Ruyter van Steveninck, 2000; de Ruyter van Steveninck, Zaagman, & Mastebroek, 1986; Fairhall, Lewen, Bialek, & de Ruyter Van Steveninck, 2001; Harris, O'Carroll, & Laughlin, 1999; Maddess & Laughlin, 1985). 

The fly, by virtue of its compound eye and retinotopic visual organization and because of easy access to its motion sensitive neurons, is an attractive system for testing models of biological motion detection. The array of lenses and their associated ommatidia in the compound eye sample visual space in an approximately hexagonal angular raster with a well-characterized angular point spread function (Braitenberg, 1967; Land, 1997; Smakman, van Hateren, & Stavenga, 1984; Stavenga, 2003). The retinotopic columnar organization of the visual pathway preserves the geometry of the visual field as signals propagate through various neuropils. The movement-sensitive neurons in the lobula plate integrate local motion signals over large areas of the visual field, and produce action potentials or graded depolarizations. 

In this paper we are interested in how the computation of movement is implemented in the fly visual system. One limitation in studying this question with conventional moving patterns is that their correlation functions are continuous in space and time. Here we use apparent motion stimuli formed by random noise with a discontinuous space–time correlation function designed to have nonzero values only at selected space–time separations (and, necessarily, also at the origin). The construction of this stimulus is described in the Methods section, along with a calculation of the correlator response. Fly motion-sensitive cells strongly respond to such stimuli, and for human observers they generate a clear percept of coherent motion superposed on a flickering background. These stimuli were presented to the fly in a hexagonal pixel arrangement at a pitch that was adjusted to the fly's sampling raster. Recordings were made from the lobula plate tangential neurons H1 and V1, which integrate local inputs over a large visual field to produce spiking response to motion in the horizontal and vertical directions, respectively (Hausen, 1976). The response of these neurons reveals that at constant flicker contrast, wide-field motion response varies bilinearly with the input signals. This confirms that motion detection involves a multiplicative nonlinearity, in accordance with the HRC construction. However, we find that, to good approximation, the neural response is normalized to the background flicker, countering the predictions of a strict HRC model. This result suggests that the visual system may scale its computational algorithm depending on signal-to-noise ratio of the input (de Ruyter van Steveninck, Bialek, Potters, & Carlson, 1994; Potters & Bialek, 1994). 

Methods

Fly preparation

Wild-type blowflies (Calliphora vicina) were captured in traps set outdoors. They were then transferred inside where they were housed in an enclosed arena and supplied with sugar, water, tomato juice, and dry protein. The temperature was controlled at 21°C and humidity maintained at approximately 60%. The ambient light was set to alternate between on and off with a 12-hr cycle. To prepare the fly for the experiment, its wings and legs were immobilized with wax, after which it was placed in a plastic holder such that its head protruded out. At the back of the head, an incision was made with a razor blade and a small piece of integument was surgically removed. Excess fat and some air sac membrane were removed from the superficial layers, and a muscle in the ventrolateral region was cut to prevent large twitches and their associated electrical interference during extracellular recording. The proboscis was kept free for fluid intake during the experiment. The holder containing the fly was placed on a goniometer platform, which allowed initial adjustments to the azimuth and elevation angles of the fly's eye. A stand-mounted movable Nikon SMZ460 optical microscope (Nikon Instruments Inc., Melville, NY) was used to view the fly from the back. We used both male and female flies for our experiments. The responses were cross-validated against at least four flies, and the results shown here correspond to a single H1 or V1 neuron with the clearest response behavior. 

Electrophysiology

Extracellular recordings were made from H1 and V1 spiking neurons (Hausen, 1976) in the right brain hemisphere, contralateral to the eye receiving the visual input. For H1 (V1), head orientation was adjusted to minimize response to vertical (horizontal) motion. Tungsten microelectrodes (FHC Inc., Bowdoin, ME) with tip diameter 1 μm and 3 MΩ impedance were used to record differential voltage, which was subsequently low pass filtered by an amplifier and discretized by a window discriminator (World Precision Instruments, Inc., Sarasota, FL). The discriminator pulses were time-stamped at 10 μs resolution by a National Instruments PCI-6259 Data Acquisition Card (National Instruments Corporation, Austin, TX) and stored in the computer for analysis. Recordings were only made from clear isolated spikes with amplitude exceeding 250 μV, such that the amplitude remained at least five times above the baseline voltage fluctuations. To prevent tissue desiccation, the fly was fed sugar solution in between experiments. 

Visual stimulation

Display screen

Visual stimuli were displayed at a frame rate of 500 Hz on a Tektronix 608 CRT (cathode ray tube) standard phosphor (P31) monitor (Tektronix, Beaverton, OR) with maximum radiance of 150 mW/sr · m⁻². At mean radiance this produces of order 5 × 10⁴ absorbed photons per second in each photoreceptor. The image on the screen consists of 827 pixels arranged in a hexagonal grid that extends 38° along the horizontal direction and 44° along the vertical direction. The frame covers approximately 8% of the total angular visual field and 15% of the angular field of H1 (Krapp & Hengstenberg, 1997). Angular distortions resulting from projection on a flat 2-D screen were corrected to first order by increasing the local raster spacing for pixels farther away from the center, following a gnomonic projection (Coxeter, 1969). The distance between the screen and the fly was adjusted to match the projection of the screen pixel raster onto the angular pitch of the fly's ommatidial raster (see section on Nyquist experiment under Results). 

P31 phosphor luminance is known to persist over several milliseconds and therefore, in principle, could give rise to stimulus artifacts that induce response unrelated to the actual stimulus. This issue has been raised on several occasions (Westheimer, 1993; Wolf & Deubel, 1997) targeted at studies in which the visual system is sensitive to temporal fluctuations in light occurring at the time scale over which the phosphor luminance decays (Zele & Vingrys, 2005). Therefore, we measured the phosphor luminance profile of a single pixel on the Tektronix 608 CRT screen using a pin photodiode. We found that the pixel luminance decreases to 8% of its maximum in 151 μs, 4% of its maximum in 279 μs, and 1.4% of its maximum in 569 μs (Supplementary Figure 1A, B). At 2 ms, the residual luminance is 0.04%. Note that the decay of residual luminance has a long tail that persists for several milliseconds. This leads to temporal averaging of luminance signals causing a reduction in the stimulus contrast. Using the average pixel luminance over the 2 ms period (Supplementary Figure 1A) and the maximum and minimum luminance at the onset and at the end of the 2 ms period, respectively, we find that the upper bound of the overall effect on contrast (according to Michelson contrast definition) is 1.24%. So, for all practical purposes, the reduction in contrast is negligible compared to the actual stimulus contrast, and therefore unlikely to impact the neural response to motion. 

Apparent motion stimuli

Variants of apparent motion stimuli have been used extensively in psychophysical motion perception studies. Julesz-like patterns were used by Braddick (1974) and dynamic random dot stimuli by Morgan and Ward (1980), both in human psychophysical studies. The response of MT neurons in monkey was characterized based on dynamic random dot stimulus displays (Britten et al., 1992). Moving spatial white noise patterns with fast alternations in space or time were employed to study computational aspects of motion detection in humans (van Doorn & Koenderink, 1982a, 1982b). The lack of spatial higher order structures in these stimuli means that motion response is induced solely by spatiotemporal correlations of light intensities. Our stimuli share features of these, but are based on discontinuous steps in space and time, and allow the independent setting of contrast for the two inputs to motion computation. 

In its most elementary form, a visual motion stimulus is associated with a light signal at one location and its delayed repetition at another location. Thus, a pair of points separated by a distance Δr⃗, showing identical temporal intensity sequences over time with a delay Δt between them, contains motion information consistent with velocity v⃗ = Δr⃗ / Δt. Such a bilocal signal would be detected by a correlator with an input span of Δr⃗. By tiling the visual field with a superposition of such bilocal signals, we create a wide-field version of this stimulus. As the basis for the stimuli in this study, we use spatiotemporal white noise by drawing a time series of spatial 2-D random values, I₀(r⃗,t), from an identically independent distribution (IID). Here, r⃗ and t represent the spatial coordinate and time, respectively. The spatiotemporal autocorrelation function of I₀(r⃗,t) is a delta function in space and time:  

 

 

To generate wide field apparent motion, we take I₀(r⃗,t) and add to it an “echo” of itself with displacements Δr⃗ in space and Δt in time:  

 

using weights m₁ and m₂ for the original and its echo. The autocorrelation of this combined pattern is,  

 

 

The first term represents a peak of height + , at location (r⃗, t) = (0,0). The second term represents a pair of peaks, located at (Δr⃗, Δt) and (−Δr⃗, −Δt), with identical heights m₁ · m₂ (see Figure 1). The central peak describes the random flicker component in the stimuli, with flicker contrast variance of + . The pair of satellite peaks carries motion signal of strength m₁ · m₂, with apparent velocity v⃗_app = Δr⃗ / Δt. Displaying I(r⃗,t) as a space–time sequence of intensities produces a strong impression of wide-field motion superposed on a flickering background. For brevity, we will refer to the product term m₁ · m₂ as “stimulus motion energy” and the term + as “stimulus flicker energy.” Here, energy is defined in units of the product of two contrast parameters, consistent with the definition originally proposed by Adelson and Bergen (1985). The stimulus parameters are chosen to have zero mean intensity and maximum contrast. The values for I₀ are drawn from a binary distribution such they are either −1/2 or +1/2. The constraint |m₁ + m₂| ≤ 2 ensures that m₁ and m₂ always have real values and contrast lies in the range [−1, 1]. For clarity, we omit from our equations an additive constant mean intensity, numerically equal to 1. We can ignore this constant term since it does not contain motion information and does not play any role in our analyses.  

Figure 1

 

Space-time correlation function of the apparent motion stimuli used in this study. (A) Idealized condition, with delta correlation in both space and time, plotted for one space dimension, namely X. The vertical red lines located at (Δt, Δx) and (−Δt, −Δx) represent nonzero motion energy and the vertical blue line located at (0, 0) represents nonzero flicker energy. The flicker energy is twice the motion energy for the condition m₁ = m₂. The four quadrants of the energy space are denoted by I, II, III, and IV. (B) Motion energy is negative for the condition m₁ = −m₂. (C, D) In practice, patterns are displayed for a finite time T_p, which in our experiments is an integer multiple of the 2 ms frame time. For most experiments a longer value of T_p is used, typically a few frame intervals, because this is matched better to the integration time of fly photoreceptors. The example in (C) shows the triangular profile of the autocorrelation function Λ(t) (see Equation 5) for T_p = 6 ms. Our stimuli avoid the situation in (D) where T_p > Δt, because that causes some of the motion energy to leak into the adjacent quadrants. The temporal offset is given by Δt. The space, time, and stimulus energy are represented by the x, y, and z axes, respectively.

At a frame time of 2 ms, intensity transitions are substantially faster than the photoreceptor integration time (Juusola, Kouvalainen, Jarvilehto, & Weckstrom, 1994; Payne & Howard, 1981; Weckstrom, 1989). Therefore, to drive the photoreceptor response to the light fluctuations effectively, stimuli are displayed with a “persistence time,” denoted by T_p, with values close to the photoreceptor integration time, typically T_p = 4 − 8 ms. T_p determines the time for which a particular random sample I₀ remains on the screen, which can be formalized as a rectangle function (Bracewell, 2000).  

 

 

The temporal autocorrelation of the sample I₀ is obtained by convolving the rectangle function Π(t) with itself, resulting in a triangle function Λ(t) (Bracewell, 2000):  

 

 

The spatial part of the autocorrelation function remains unchanged as δ(r⃗). With finite T_p the cross-correlation of intensity (Equation 2) is now:  

 

 

As before (Equation 3), the cross-correlation has three peaks, but the peaks are now centered on the triangle functions which are nonzero within the time window 2T_p, as illustrated in Figure 1. Note that each of the two components of I(r⃗, t), that is, the original and its echo, persist for a time T_p. Our setup, however, allows us to manipulate the values of Δt in increments of 2 ms frame time, independently of the choice of T_p. In our experiments we found an overall increase in the response magnitude with values of T_p in the 4–8 ms range, consistent with the idea that increasing T_p increases the effective contrast for the fly. We did not notice any qualitative changes in other response properties with values of T_p in the 2–8 ms range (not shown). Supplementary movies MVI_m0.2_f0.4, MVI_m0.2_f2.0, MVI_m0.8_f2.0, and MVI_m0.8_f1.6_pos_neg, show different stimulus realizations obtained by manipulating the contrast parameters {m₁, m₂} and the time offset Δt (see Supplementary Materials). 

Spatial alignments

Alignment of the head with respect to the axes of the screen was done for each fly individually, using the regular hexagonal pixel grid displaying bar patterns (contrast = 1, spatial wavelength = 5°) moving at 50°/s. The response to preferred motion was maximized and the response to null motion minimized, resulting in an azimuth angle of about 35°. The roll angle was adjusted to match the responses to motion in the upward and downward directions. This alignment procedure was carried out separately for H1 and V1 neurons, prior to the actual experiments. 

The hexagonal raster of pixels displayed on the screen (see Figure 6A) has a nearest neighbor distance of Δr ≅ 1.8 mm. The column to column separation Δx = Δr · cos(30°) ≅ 1.5 mm defines the angular spacing in the horizontal direction (x axis in Figure 6A) and the row to row separation Δy = Δr / 2 ≅ 0.9 mm defines the angular spacing in the vertical direction (y axis in Figure 6A). The ommatidial raster of the fly's eye samples space in a hexagonal array with angular spacing Δφ; its precise value varies from fly to fly, but is typically of the order Δφ ≅ 1.5°. With these parameters, the pitch of the stimulus array matches the fly's angular spacing at a fly-to-screen distance of D = Δr / tan⁻¹(Δφ) ≅ 70 mm. The photoreceptor point spread function (PSF; Snyder, Stavenga, & Laughlin, 1977) admits spatial frequencies higher than the raster's Nyquist limit, leading to aliasing of high spatial frequencies. This effect can be measured in motion sensitive neurons when the fly is presented with moving patterns at high enough spatial frequencies to induce a reverse response (Götz, 1964). We use this effect to adjust the visual stimulus to the fly's sampling raster. The horizontal and the vertical projections of interommatidial separation are (see also Figure 3):  

 

 

 

 

The standard hexagonal pixel arrangement does not allow us to display patterns with short enough wavelengths. Therefore, specifically for this experiment, the pixel arrangement was reformatted to a rectangular lattice with an intercolumn separation Δl of exactly one quarter of the horizontal projection: Δl = Δx / 4. For H1, the fly's motion response was tested using bar patterns of spatial wavelength λ_s equal to 6, 8, and 12 lines, or λ_s = 1.5 · Δx, 2 · Δx, 3 · Δx, respectively. The correct match was found by adjusting the fly-to-screen distance such that there was no change in response to motion in the preferred and null directions for λ_s = 2 · Δx. For H1 recordings, the pattern was moved along the horizontal axis alternatingly in the preferred and null directions for 4 s each, and repeated across 50 trials. For V1, the rectangular pixel grid was rotated by 90°, with the spatial wavelengths appropriately scaled and the distance was adjusted while presenting vertical movement. Because the motion response of tangential cells decreases at high temporal frequencies (Hausen & Egelhaaf, 1989), the velocity of the stimulus for this experiment was kept relatively low, to about 6°/s. 

Results

Hassenstein-Reichardt correlator model

The HRC model has been extensively used to explain visual the motion response in flies and in other insects (Buchner, 1976; Clifford & Ibbotson, 2002; Hassenstein & Reichardt, 1956). In its simplest form (see Figure 2), the signal in each arm is delayed by the low-pass temporal delay filter G(t) and then multiplied by the instantaneous signal of the other arm. The signals from the mirror-symmetric arms after multiplication are subtracted to produce an antisymmetric, direction-selective output (Figure 2A). The wide-field response can be modeled by replicating the correlator structure over the visual field and integrating the local outputs over the entire array of detectors. The inputs of the correlator can span the nearest neighbor distance, but can also be separated by multiple raster spacings, and in general, the motion response can be thought of as a weighted sum of correlator populations with different spans. To mimic the fly's optics and make the model physiologically relevant, we introduce a spatial filter S(x) that represents the point-spread function (PSF) of the photoreceptors (Land, 1997; Snyder et al., 1977), a linear temporal filter F(t) representing the photoreceptor impulse response and a linear temporal delay filter G(t) (Figure 2A). Further details about the functional form and parametrization of filters can be found in the section, Correlator response to apparent motion stimuli (below). More filters can, in principle, be added, for example in the cross-arms. However, because filtering in the model is treated as linear, the properties of the cross-arm filter can be accounted for by appropriate compensations of F(t) and G(t). We expect the model to be rich enough to describe correlator behavior in our experiment, and therefore refrain from using more complex elaborations of the HRC model (van Santen & Sperling, 1985). 

Figure 2

 

Schematics of a 1-D correlator model. (A) The elementary correlator consists of two mirror-symmetrical subunits with span equal to integer number of Δφ ≈ 1.5°. The light input is represented by I(x,t) and the Gaussian PSF (σ = 0.5°) of the photoreceptors at locations 1 and 2 are represented by S₁ and S₂, respectively. The other components of the correlator circuit are, F(t): lognormal temporal prefilter and G(t): first-order exponential delay filter. The multiplicative nonlinearity, spatial summation, and correlator output are denoted by x, ∑, and R, respectively. (B) An array of correlators with different spans. Unit span is equal to Δφ (see also Poggio & Reichardt, 1973).

Figure 3

 

Spatiotemporal receptive field of the model correlator. The 3-D profile shows the receptive field of a 2 unit model correlator as a function of space (η) and time (ζ) for (A) time constant τ = 10 ms, and for (B) time constant τ = 300 ms of the delay filter G(t) (see Equation 19). For this plot, the photoreceptor temporal filter F(t) is replaced by a log-normal function with parameters K = 1, t_p = 15 ms, s = 0.23 (Payne & Howard, 1981), and the delay filter G(t) is replaced by a first order exponential. The solid gray curve to the left and to the right represent the antisymmetric spatial and antisymmetric temporal part of the receptive field. The z axis represents the weighting factor. Excitatory response is shown in red (quadrants I and III) and suppressive response is shown in blue (quadrants II and IV).

Time and space average of the correlator response

Here we derive an expression for the time-averaged response of the model correlator (Figure 2) suitable for describing the response to our apparent motion stimuli. Because wide-field motion-sensitive neurons integrate signals over a large array of local detectors, their mean rates can be described by the time-averaged response of one correlator times the number of correlators over which the signals are integrated (Figure 2B). To keep the derivation simple, we use the simplest version of the HRC model, consisting of a spatial filter S that represents the photoreceptor PSF, a temporal filter F that represents the impulse response of the photoreceptor, and a delay filter G that is approximated by a first-order exponential (Figure 2A). For ease of notation, we also limit space to one dimension. Light intensity as a function of space and time is given by I(x,t); the PSF of the correlator centered at x = 0 is given by S(x), and PSFs of the correlators centered at x = −Δφ / 2 and x = Δφ / 2 are given by S₁ and S₂, respectively. Here Δφ is the span, or the angular distance between the two correlator inputs. The linear spatial and temporal filters transform the signals as follows:  

 

 

 

where ⊗ represents convolution, H is defined as the convolution of G and F, and the indices of u, v, and S identify the two correlator inputs (Figure 2A). The output, R, of the correlator after the multiplication stage is the difference of two products defined as R₊ and R₋, respectively:  

 

where “·” represents multiplication and R is a function of x and t. By averaging the response over time and space, we obtain the equivalent wide-field averaged response of the correlator system. If we just focus on one branch of the correlator:  

 

 

Writing this in terms of all the convolution integrals and rearranging orders of integration, we get:  

 

 

 

 

 

where Θ_II is the stimulus spatiotemporal autocorrelation function. Substituting ζ = ξ₁ − ξ₂, ξ = ξ₂ and η = τ₁ − τ₂, τ = τ₂, we get:  

 

 

Defining the spatial self-overlap of the point spread function as  

 

we write the full correlator response as:  

 

 

Thus, the averaged response of the correlator is an integral over space and time of the stimulus autocorrelation multiplied by a purely temporal term (first set of braces), and a purely spatial term (second set of braces). The correlator is thus described by a separable spatiotemporal “receptive field” and its averaged output to the stimulus is given by the inner product of that receptive field with the stimulus autocorrelation. Note that the spatial term of the receptive field is antisymmetric, while the temporal term is not. Because the stimulus autocorrelation is a symmetric function under joint space and time reversal: Θ_II(ζ,η) = Θ_II(−ζ,−η), the calculated response of the correlator is always antisymmetric in both time and space separately, and thus symmetric under joint exchange of space and time. This implies that we get the same outcome for the inner product if we define the receptive field Γ_corr(ζ,η) as a sum of its regular and time-reversed parts:  

 

and with this definition we have:  

 

 

An example of Γ_corr(ζ,η) is shown in Figure 3, along with its spatial and temporal components. Note that this derivation is not limited to the stimuli we use in our experiment, but applies generally to wide field motion stimuli with constant velocity. 

Correlator response to apparent motion stimuli

To illustrate the spatiotemporal response properties of the correlator receptive field (Equation 19), we provide specific functional forms for the filters, derived from experimental data. The spatial filter S(x) is approximated by a Gaussian of standard deviation σ = 0.51° (Smakman et al., 1984), the temporal filter F(t) is approximated by a log-normal function K · exp[−{log(t / t_p)}² / 2s²] with parameter values t_p = 15 ms and s = 0.23 chosen from photoreceptor measurements (Howard, Dubs, & Payne, 1984; Payne & Howard, 1981), and the temporal delay filter G(t) is approximated by a first order exponential (1 / τ)e^−t/τ (Srinivasan, 1983), where τ can range from few tens of milliseconds to hundreds of milliseconds (de Ruyter van Steveninck et al., 1986; Harris et al., 1999).  

The time-averaged response of the correlator as a function of the temporal offset Δt, exhibits an initial transient, followed by a peak and an exponential decay (Figure 3A, B). A small value of time constant τ = 10 ms is associated with a sharper transient and smaller peak latency (Figure 3A) compared to a larger time constant τ = 300 ms (Figure 3B). Note that the correlator's response qualitatively matches the responses of H1 and V1 to change in Δt (see Figure 6D, E). A positive response in quadrants I and III of the energy space indicates excitation from preferred motion, while negative response in quadrants II and IV indicates suppression from null motion. The temporal response profile is symmetric about the vertical axis (Figure 3A, B, gray projection to the right) because the filters in the two correlator arms are identical. This, however, need not be true for an actual biological correlator, where different gain values in the two arms based on increment or decrement of signal may lead to asymmetric response curves (Borst & Egelhaaf, 1990). The spatial response profile is a difference of two Gaussians centered respectively at the location of the two detectors, where each Gaussian represents the convolution of the photoreceptor PSFs (Figure 3A, B, gray projection to the left). Together, these define the 3-D landscape of the full receptive field of the correlator Γ_corr(ζ,η). 

The correlation structure of the apparent motion stimuli in our experiments, as defined by Equation 6, is essentially a pair of delta functions defined by space and time offsets Δr⃗ and Δt and amplitudes m₁, m₂. So, by measuring the response of the cell as a function of Δr⃗ and Δt, keeping m₁ · m₂ constant, we essentially sample the surface of the correlator receptive field at those locations, as long as the persistence time T_p remains small enough. If we vary m₁ and m₂ the response is predicted to scale in proportion to m₁ · m₂. 

Nyquist experiment

According to the sampling theorem, a discretely sampled waveform accurately represents the original waveform only if the sampling frequency if higher than twice the highest frequency present in the waveform. This limiting frequency is called the Nyquist frequency (Nyquist, 1928; Shannon, 1949). For lower sampling frequencies, aliasing leads to artifacts in the sampled waveform. The fly visual system is also susceptible to this problem. The hexagonal array of ommatidia samples the visual image discretely with a sampling frequency of 1 / Δφ, where Δφ is the angular spacing between neighboring ommatidia. For blowfly, this spacing is of the order of ≈1.5°. If the stimulus contains frequencies higher than 1 / (2Δφ), it leads to aliasing, and this effect can be measured in motion sensitive neurons that elicit reverse response when the fly is presented with moving patterns at high enough spatial frequencies (Götz, 1964). We exploit this response behavior to map the visual stimulus to the fly's sampling raster. 

To match the receptor spacing to the spacing of the pixels, we used a rectangular pixel grid (see Methods) and vertical bar patterns with spatial wavelengths of 6, 8, and 12 lines respectively for experiments on H1 (see Figure 4 upper panel) with line spacing Δl = Δx / 4. The fly-to-screen distance was adjusted so that the pattern with spatial wavelength of 8 lines produced equal responses to preferred and null directions of horizontal movement. This criterion resulted in an adjusted eye to screen distance (D) of about 78 mm with a standard deviation of 5 mm across four flies tested. From the spacing of lines on the screen (Δl ≈ 0.45 mm), we compute Δφ_x tan⁻¹(4 · Δl / D) ≅ 1.3°. As an additional check, and also to verify the quality of the fly's image, we used spatial wavelengths of 6 and 12 lines. In this condition, the neuron is expected to show a reverse response for λ_s = 1.5 · Δx (6 lines) because of aliasing and a normal response for λ_s = 3 · Δx (12 lines), which is what we found. H1 produced a reverse and a normal response corresponding to these wavelengths, with rates differing by a factor of approximately 5 in both cases (Figure 4). Test for Nyquist response was also conducted on V1 using the same pixel grid but rotated 90° about its center. 

Figure 4

 

Spatial Nyquist frequency test. Bar patterns of three spatial wavelengths, λ_s = 1.5·Δx (6 lines), λ_s = 2·Δx (8 lines), and λ_s = 3·Δx (12 lines), at contrast 1.0 were presented. The horizontal projection of the interommatidial separation is Δφ_x. The responses to motion in the preferred and null directions are shown by hatched bars and solid gray bars, respectively. The Nyquist response can be seen for the case with λ_s = 1.5·Δx. Velocity was maintained at 6°/s for all spatial wavelengths. Error bars represent SD of the firing rate obtained from 50 trials.

Response does not depend on pixel-to-photoreceptor matching

Even if the pitch of the pixel raster on screen matches the pitch of the ommatidial sampling raster (Figure 4) that does not guarantee that the photoreceptor optical axes are exactly aligned with the pixel positions. A mismatch might result in a change in the amplitude of the light signal at each photoreceptor location, which may subsequently change the response to apparent motion. To test this, we presented apparent motion along −30°, +30°, and +90° directions and measured the response of H1 at different absolute locations of the pixel grid (see lower panel of Figure 5). 

Figure 5

 

Response dependence on absolute location of pixel grid. The plot shows the response mean and the response variation around mean, of H1, for four different directions grid shift: along −30°, +30°, 90° (y axis), and 0° (x axis). The full grid consists of 827 pixels. The original and shifted grid locations are exemplified by the hexagonal pattern of black and gray dots respectively, in the bottom panel. Shifts along the x axis are made in steps of (Δx ≈ 1.3°)/4 (blue dotted line), while shifts along the other three directions are made in steps of (Δr ≈ 1.5°)/4 (red dotted line). Four consecutive shifts are made to cover one period (i.e., Δx along x axis and Δr along other directions). The color codes represent the six directions of apparent motion (see legend). The error bars represent SD of mean firing rate obtained from four grid locations, corresponding to each direction of grid shift. The dotted lines represent the response averaged over the four different directions of grid shift.

Figure 6

 

Spatial sensitivity profile and response tuning to apparent velocity. (A) The gray dots in the schematic represent the location of spatial correlations in units of the fly's sampling raster. Horizontal and vertical projections of Δφ ≈ 1.5° are Δφ_x and Δφ_y (Equations 7, 8). Angles are measured with respect to the x axis, where x and y axes define the lab coordinate system. (B) Excess firing rate of H1 is shown for correlations between the central pixel (color-coded black), and pixels located at ±30° (1, ±1), at 0° (2, 0) and at ±15° (3, ±1). (C) Excess firing rate of V1 is shown for correlations between the central pixel, and the pixels located at ±60° (±1, 1), at 0° (0, 2) and at ±30° (±1, 3). Here, x and y axes are in units of pixel separation on the screen: Δx, Δy. The gray vertical bar shows the range of response. The SD of the firing rate is <10% of the mean rate. Contrast is set to maximum and Δt is set to 8 ms. Response of (D) H1 for spatial offsets (Δx, Δy) = (1, ±1), (2,0), (3, ±1), and (E) V1 for spatial offsets (Δx, Δy) = (±1, 1), (0, 2), (±1, 3), as a function of the temporal offset Δt. The green and purple circles represent the response summed over the two locations: Δy = ±1 for odd column number for H1, and Δx = ±1 for even row number for V1. The orange circles represent the response obtained from the locations: Δy = 0 for even column number for H1, and Δx = 0 for even row number for V1. Error bars represent SD of the mean firing rate calculated over 50 trials. The solid lines represent nonlinear least-square fits using the temporal receptive field of the model correlator (Equation 19) with parametrized F(t) and G(t). The parameter values that minimized the least square error are chosen for the fit. The correlator contributions to the wide-field response of (F) H1 and (G) V1, are shown as a function of Δt. Similar to D and E, the green and purple curves represent the summed contributions from the two locations lying on the same vertical line (Δy = ±1) for H1 and on the same horizontal line (Δx = ±1) for V1. The indices of β correspond to the locations described in D and E. The frame persistence, T_p, was set to 6 ms.

The responses of H1 to velocities along +30° and −30° were higher than the responses to velocities along −150° and +150°, respectively, by at least a factor of 2.5, for all the directions of grid shift (Figure 5, red, blue filled circles). The response to downward motion was slightly stronger than the response to upward motion (Figure 5, green filled circles). Because H1 is sensitive to horizontal motion, the response to vertical motion can be attributed to stimulus flicker and spontaneous discharge of H1. For a particular motion direction, the standard deviation of the responses corresponding to different directions of grid shift is less than 10% of the mean responses. Together, these results indicate that the mean spiking response is independent of the precise match between pixels and photoreceptors. As a more sensitive test, we investigated whether the pixel size impacts response dependence on grid shift. For this, we reduced the pixel spot size to about a quarter of the original size by refocusing the electron beam. While this resulted in a 60% decrease in intensity and a subsequent reduction in the overall response (results not shown), the response characteristics displayed in Figure 5 did not show any significant change. Based on this result, exact one-to-one alignment does not seem to be a prerequisite for the interpretation of our data. Hence, there seems to be no benefit to achieving exact alignment. Although optical methods for alignment can be used (Franceschini, 1975), we chose not to apply them because they are intrusive, and substantially limit the lifetime of the fly in the experiment. 

Spatial sensitivity map

It has been suggested previously that contribution of signal interaction to global motion response is confined to correlators with a span of up to a few neighboring receptor elements (Buchner, 1976; Pick & Buchner, 1979). Here, we quantify the contribution of local interactions at different spans for both H1 and V1 by varying the spatial offsets between correlated signals in the stimulus. The contrast is set to maximum and the temporal offset is set to Δt = 8 ms. Each trial consists of 2.5 s of apparent motion in the preferred direction (i.e., +X for H1 and −Y for V1), followed by 2.5 s of pure flicker. To generate the pure flicker condition, we set Δt to 200 ms, which is experimentally equivalent to Δt → ∞ (see stimulus movie clips in Supplementary Materials). For clarity, we shall use units of column separation (Δφ_x ≅ 1.3°) and row separation (Δφ_y ≅ 0.75°) to refer to the spatial offsets in the (x, y) coordinate system (Figure 6A), which in matched conditions correspond to the (Δx, Δy) values of the raster displayed on screen. 

The excess firing rate of H1 to apparent motion is highest for spatial correlation between pixels at (Δx, Δy) = (1, ±1), with no noticeable difference in response for the two values of Δy. This indicates that the largest contribution to the motion response comes from nearest-neighbor interactions (Figure 6B). The excess firing rate drops from 29 spikes/s to 21 spikes/s as the column separation increases from 1 to 2. At a column separation of 3, the excess firing rate drops to 8 spikes/s. For larger column separations, the response was indistinguishable from the pure flicker response. For V1, the excess firing rate rises to a maximum of 25 spikes/s at (Δx, Δy) = (0, 2), (Figure 6C). For the nearest neighbor pixels oriented at ±60° about the vertical axis, that is, with (Δx, Δy) = (±1, 1), the excess firing rate reduces to 17 spikes/s. With Δy = 3, the response further diminishes to 14 spikes/s. This indicates that the strongest excitation to V1 comes from nearest-neighbor interactions with velocity in the preferred direction (downward). 

In the stimulus conditions described thus far, we primarily changed Δr⃗, keeping Δt constant. But apparent velocity depends on both Δr⃗ and Δt. Further, the HRC model predicts a response dependence on Δt, also formalized in Equation 20 where the stimulus autocorrelation term Θ_II depends on Δt. To examine the temporal response properties, we chose the following set of values for Δt: 4 ms, 8 ms, 12 ms, 16 ms, and 20 ms for each spatial offset described in Figure 6B and C. To fit the response values, we used the temporal part of the correlator receptive field, Γ_corr(ζ,η) (see Equations 19 and 20) with filters F(t) and G(t) parametrized by a log-normal function and an exponential decay function described earlier in the section, Correlator response to apparent motion stimuli. As a function of Δt, the excess firing rate of both H1 and V1 neurons rises to a peak value and then decays (Figure 6D, E). For the three curves, the initial rise is slightly steeper for V1 than for H1, which can be attributed to a difference in response gain between V1 and H1 and a stronger excitation from correlated signals separated by a smaller absolute distance (Δy = 1 ≈ 0.75° compared to Δx = 1 ≈ 1.3°). For both H1 and V1, the dominant responses correspond to the nearest-neighbor spatial correlations, that is, with (Δx, Δy) = (±1, 1) for H1 and with (Δx, Δy) = {(1, ±1),(0, 2)} for V1. As noted earlier, the shape of the response curves in Figure 6D and E reflects the temporal response profile predicted by the HRC model (Götz, 1972; Hausen & Egelhaaf, 1989), as described by Equation 19. 

Note that our experiments tested only a limited range of Δt values because the response at large Δt becomes weak and quite hard to measure. Figure 6 shows that the response rates peak at Δt = 12 ms and 8 ms for H1 and V1 respectively, equivalent to apparent speeds of |v⃗_app| = 1.3° / 0.012s ≅ 110°/s and |v⃗_app| = 1.5° / 0.008s ≅ 190°/s. This roughly corresponds to the velocities that generate peak rates for conventional moving patterns in many lab conditions. For larger Δt, the rate decreases as Δt increases; in other words, for large values of Δt the rate is an increasing function of |v⃗_app|. The decrease in response for the smaller values of Δt is presumably due to the finite bandwidths of the various filters in the system. Based on measurements of the photoreceptor response under similar illumination conditions (not shown), photoreceptor filtering alone will cause a decrease in rate for values of Δt below 6 ms. Additional filtering operations may shift that criterion to the somewhat higher values of Δt we find here. 

In the preceding experiments, we tacitly assumed that light is emitted by a point-source and is received by a point-detector. In practice, however, the light input to the photoreceptor is a weighted function of light intensities over a small region of visual space. The weighting function is given by the PSF of the lens-photoreceptor system (Smakman et al., 1984). This spatial smearing will affect the wide-field motion response of H1 and V1 neurons by decreasing optical contrast, and introducing crosstalk of the optical signal in neighboring photoreceptors. 

The response of a wide field motion sensitive cell can be thought of as sum of contributions of correlator subsets with different spatial spans and temporal properties. Those contributions may have different weights depending on span and stimulus delay time, and those weights can be determined by presenting the stimulus at different space and time offsets. In fact, the weights would be directly proportional to the measured rate if the system did not optically blur the images. In the presence of optical blurring we need to apply a correction, and to extract the “true” contribution of the correlators to the motion response, we must first quantify the optical contribution to the motion response. The PSF is approximated as a Gaussian of σ = 0.51° (Land, 1997; Smakman et al., 1984), and the optical contribution from the neighboring locations to a correlator of given span is found by the overlap between the area of a pixel given by P(ζ) (which is a disk with diameter ≈0.30°) and the correlator's spatial receptive field. The latter is described by the difference between two convolved PSFs separated in space, expressed as the spatial part, {Ψ_SS(ζ + Δφ) − Ψ_SS(ζ − Δφ)}, of the correlator “receptive field” (see Equation 19 and also Figure 3, gray curve on the left). The optical contribution of a pixel pair at stimulus offset s to a correlator with span Δφ_i,j is then defined as  

 

where Ψ_PP is the self-overlap of the pixel disk. If β_i,j represents the actual unknown contribution of that set of correlators to the neural response, then the total response can be formulated as:  

 

where indices (i,j) correspond to spatial locations in units of (Δφ_x, Δφ_y). In the analysis we limit the set of spans to {(i,j)} = {(1,1),(1,–1),(2,0),(3,1),(3,−1)} for H1 and {(i,j)} = {(−1,1),(1,1),(0,2),(−1,3),(1,3)} for V1 (see Figure 6). The offsets s_i,j used in the experiment correspond to the same raster points as given in these expressions, but now in units of (Δx, Δy). To find the β_i,j(Δt), we treat them as fitting parameters in a standard linear least squares minimization of χ², defined as:  

 

 

By substituting in Equation 23 the excess firing rate of H1 to a set of stimuli with Δt = 12 ms, we obtain the following correlator contributions for the five different spans: β_1,1 = 35%, β_1,–1 = 34%, β_2,0 = 16%, β_3,1 = 7% and β_3,−1 = 8%. Thus, the two nearest-neighbor correlators at (1,±1) together contribute about 70%, the next–nearest-neighbor correlator at (2,0) contributes about 15%, while the next–next–nearest-neighbor correlators at (3,±1) together contribute about 15%, to the overall motion response. When we plot the values of β_i,j(Δt) in Figure 6F and D, we find that, due to the correction, the true summed contribution from the two nearest-neighbor correlators rises above 60 spikes/s, which is higher than twice the measured average response of H1 (Figure 6F). The contributions from the second–, and the two third–nearest-neighbor correlators do not show significant deviation from the corresponding measured responses. Similarly, for V1, the correlator contributions are higher than the corresponding measured responses (Figure 6G, E). The response in both cases peaks at 8–12 ms and then decreases monotonically with Δt, characteristic of the model correlator's response. These results corroborate previous findings that the nearest-neighbor correlator provides the dominant input to motion response (Buchner, 1976), and shows that optical blurring due to finite PSF impacts the overall magnitude and quality of response to motion. 

Bilinear dependence on contrast

The HRC model postulates that response is proportional to the product of signals arriving from two spatially distinct locations, where one signal is delayed with respect to the other (Hassenstein & Reichardt, 1956). Previous studies that have reported a correlation type mechanism for motion detection, have noted a linear dependence of the response on the square of stimulus contrast, at least for low contrast values (Egelhaaf et al., 1989; Haag, Denk, & Borst, 2004). This behavior is a corollary of the more general prediction of bilinear behavior. Because our stimuli allow us to manipulate m₁ and m₂ independently, we can investigate that prediction more directly, and of course, a bilinear dependence of response on signal amplitudes would provide a more direct validation of a multiplicative nonlinearity. 

We independently varied the parameters m₁ and m₂ such that in one case the flicker energy changes but the motion energy remains constant at 0.2 (Figure 7A), and vice versa in the other case with flicker energy constant at 1.0 (Figure 7B). For each of these cases, the excess firing rate was obtained from two reference conditions: (1) null motion (Figure 6, blue color) and (2) pure flicker (Figure 7, orange color). The pure flicker condition was approximated by setting Δt = 200 ms, whereas null motion was generated by altering the sign of either Δt or Δx. 

Figure 7

 

Response of H1 changes with modulations in m₁ and m₂. (A) The flicker energy (solid line) is constant while the motion energy (dashed line) changes. (B) The motion energy (dashed line) is constant while the flicker energy (solid line) changes. The change in flicker and motion energies is linear in both cases. Response is recorded using two reference stimulus conditions: (1) motion in the null direction (blue dashed line; i.e., v₋, in the even segments), and (2) pure flicker (orange dashed line; i.e., v₀, in the odd segments of trial). The dual color-coded y axes correspond to the two conditions. Response of H1 averaged over 50 trials to (C) stimulus conditions shown in A and (D) shown in B. The excess rate to pure flicker and half of excess rate to null motion are shown color coded orange and blue respectively. The mean rate is r and subscripts +, −, and 0 refer to preferred motion, null motion, and pure flicker conditions. The values of other stimulus parameters were: Δx = 2, Δt = 8 ms, and T_p = 6 ms.

We see that the excess firing rate rises approximately linearly with increase in motion energy (Figure 7C). This indicates that motion response varies bilinearly with the signal amplitude. However, with increasing flicker energy at constant motion energy, the firing rate shows a steady decrease (Figure 7D). This behavior deviates from the prediction of the HRC model that motion response should depend only on the motion term m₁ · m₂ and not on the flicker term + . We also observe a slight difference in response gain for the two reference conditions. A slightly steeper blue curve relative to the orange curve (Figure 7C) means that with increasing motion energy, suppression from null motion becomes stronger, whereas response to pure flicker remains largely unchanged. This causes a larger excess rate for the null motion condition. In contrast, when the motion energy remains constant at a low value of 0.2 (Figure 7D), the response is largely dominated by flicker, which causes an approximately equal change in the response to null motion and pure flicker, thus keeping the response gain equal for the two conditions.  

Response dependence on contrast polarity reversal

A change in the direction of perceived motion as a result of contrast polarity reversal has been previously demonstrated in psychophysical studies (Anstis, 1970; Anstis & Mather, 1985), as well as in Reichardt and Hassenstein's (1956) original work on the behavior of Chlorophanus. This method of generating negative motion is qualitatively different from the conventional method; in the latter, we change the sign of either Δx or Δt, shifting the locus of peak motion energy from quadrant I to II and from quadrant III to IV (Figure 1). By reversing the polarity of contrast, the locus does not change, but the sign of the motion energy flips from positive to negative (see Figure 1B and Adelson & Bergen, 1985). 

For the experiment, we changed the sign of either m₁ or m₂, but not both, in alternate segments of a trial (Figure 8A). This leads to a change in the sign of the motion energy m₁ · m₂ but of course does not affect the flicker energy + . H1 shows a clear reduction in firing rate when m₁ · m₂ < 0 compared to the pure flicker response (last segment of Figure 8B). This reduction is of the same order, though somewhat smaller in magnitude than the excitatory response to m₁ · m₂ > 0 (Figure 8B). Note that this behavior is in qualitative agreement with the prediction of bilinearity. The response also shows a linear dependence on the stimulus motion energy, where the slope for positive motion energy is steeper than for negative motion energy (Figure 8C, filled circles). This indicates that the visual system responds to stimuli with positive and negative motion energies with different flicker normalizations. To differentiate the effect of motion reversal caused by contrast polarity reversal from reversal in the direction of apparent velocity, we changed the sign of Δt in the stimuli keeping the sign of m₁ · m₂ unchanged. This leads to a slight reduction in slope of the response curves for both positive (Δt > 0) and negative (Δt < 0) apparent velocities (Figure 8C, open circles), relative to the curves with contrast reversal (Figure 8C, filled circles). The response of V1 to contrast polarity reversal showed characteristics that were similar to those obtained from H1 (results not shown).  

Figure 8

 

Wide-field response to reversal of contrast. (A) The stimulus condition is set to reverse the sign of m₁ and keep the sign of m₂ same, in every even segment of a trial. The motion energy decreases linearly (dashed line) while the flicker energy remains constant at 1 (solid line). In the last segment, Δt is set to 200 ms (m₁ = 0.5, m₂ = 0.5), to generate an approximately pure flicker condition. (B) The PSTH shows the trial averaged response of H1 to change in stimulus conditions described in A. (C) Steady state response is computed by discarding the initial transient (approx. 200 ms). The filled blue circles and filled orange circles represent response for odd and even segments of trial respectively. The sign of Δx and Δt remains unchanged here. The open blue circles and open orange circles represent responses to negative and positive apparent velocity (v_app), respectively. Here, the signs of m₁ and m₂ remain unchanged (shown by the top x axis). The gray lines represent linear fit to the data, obtained separately for the two contrast conditions and v_app conditions. The error bars represent SD of firing rate across 50 trials.

Motion response depends on flicker strength

Wide-field flicker is known to influence both neuronal (Borst & Egelhaaf, 1987; Nordstrom, Moyer de Miguel, & O'Carroll, 2011) and behavioral (Tuthill, Nern, Rubin, & Reiser, 2014) responses to motion stimuli. Flicker can also reduce contrast gain, although by a magnitude that is smaller than that induced by motion (Harris, O'Carroll, & Laughlin, 2000). Our stimulus allows us to manipulate motion and flicker energies independently, within certain bounds, and here we examine the effect of both. 

We solve a quadratic equation involving the terms m₁ · m₂ and + with constraints (see Methods), to determine a set of real positive values for m₁ and m₂. The allowed range of flicker and motion energies ([0, 2] and [-1, 1] respectively), covers a large part of the motion-flicker space (Figure 9A, gray region). The absolute bound for allowed and nonallowed values is set by the constraint |m₁ + m₂| ≤ 2 (Figure 9A). The response curves for H1 show a notable decrease with increasing flicker energy, and a clear increase with an increase in motion energy (Figure 9B). The curves in Figure 9B demonstrate both that the response increases with increasing motion energy, and decreases with increasing flicker energy. To describe the response dependence on m₁ · m₂, we define a radius r and an angle θ to represent the locus of m₁ and m₂ in the polar coordinate plane (Figure 9C).  

 

 

 

 

Figure 9

 

Response depends on motion and flicker components in stimuli. (A) The plot represents the 2-D space of flicker and motion energy. The range for flicker is set to [0, 2] and for motion is set to [0, 1]. The gray line at the top right hand corner represents the boundary separating the regions where values of m₁ and m₂ are allowed (i.e., m₁ + m₂ < 2) and not allowed (i.e., m₁ + m₂ > 2). The gray dashed line meets the flicker axis at value 4 and motion axis at value 2. The gray triangular shaded region represents the experimentally accessible region of motion-flicker space (i.e., where m₁ and m₂ have real values). For each value of m₁ × m₂, a set of linearly increasing values of + is calculated. The color codes represent the 4 levels of motion energy within the accessible space. (B) Excess firing rate is plotted as a function of the flicker energy, for each value of motion energy, shown color coded. The excess rate is obtained by subtracting the null motion response from the preferred motion response (50 trials, trial time = 2.5 s each for preferred and null motion). The error bars represent SD of the mean firing rate. Other stimulus parameters were set at: Δx = 2, Δt = 8 ms, and T_p = 6 ms. (C) Graphical representation of the product term (area in gray) and the quadrature term (radius in blue) using an m₁, m₂ coordinate system. The angle (θ) relates m₁ and m₂. (D) Color coded response curves, are plotted as a function of the angle θ. The solid gray curve with black markers represents scaled sin(2θ) / 2 or, the product m₁ × m₂ in terms of the angle θ.

The motion and flicker terms respectively are then given by:  

 

 

 

 

Plotting the response and the product term as function of θ = tan⁻¹(m₂ / m₁), we observe that the response curves virtually overlap with each other and with the scaled sin(2θ) / 2 curve (Figure 9D). Because the flicker energy is different for different points on a particular curve, the overlap of the different response curves implies that the response scales inversely with the flicker energy (see also Figure 9B). The observation that the response curves follow the sin(2θ) / 2 curve shows that the response is also proportional to the motion energy. Part of this result is consistent with the predictions of the HRC model, in that the neural response varies linearly with the product term m₁ · m₂. But the other part suggests that the neuron is sensitive to the background flicker, based on which it adjusts its motion response. Computationally, this implies that the fly visual system uses a flicker-normalized correlation for movement detection at least across the set of conditions explored in these experiments. Conceptually, this can be understood as a response dependence not on pure motion energy, but on the ratio of motion to flicker. 

Discussion

Conventional visual motion stimuli typically have continuous autocorrelation functions. By construction, the apparent motion stimulus used in our experiments is discontinuous in both space and time. It shares that property with phi-motion stimuli, but whereas phi-motion stimuli are bilocal, our stimulus can cover a large area. Conventional bilocal phi-motion stimuli are typically based on functions with long correlations, such as square wave or sinusoidal fluctuations. Tiling space with such stimuli therefore introduces intensity correlations all across the visual field. Because our stimulus is based on random functions with nonzero correlations at specific offsets only (Equation 2), we can tile the visual field without introducing additional correlations. This makes these stimuli very suitable for studying wide field motion sensitive neurons and the computational structure that gives rise to their response. 

Motion detection, in particular in flies, has generally been identified with a correlation operation described by the HRC model (Hassenstein & Reichardt, 1956). The central concept of this model is the multiplication of delayed output of one channel with the nondelayed output of the other channel. There is a large body of work describing various computational aspects of the motion-processing pathway, in the fly as well as in other organisms. The lack of spatiotemporal continuity allows us interrogate some of the computational properties in a more direct way than is possible with conventional stimuli: We can target the spatial offsets and temporal delays of stimulus correlations, which allows us to probe directly the space and time aspects of the computation. The amplitudes of the contrast signals for the two arms of the correlator can be independently set, making it possible to test the bilinear character of the computation. Finally, we can manipulate the variance of the flicker. The price for these additional flexibilities is that the stimulus cannot be interpreted as representing natural conditions, so some care should be taken in interpreting the experimental results. 

The HRC model does not impose any restrictions on the spatial separation between correlated signals. However, for natural visual scenes, one might expect decreasing levels of correlation for larger separations due, for example, to inherent scene variability and motion orthogonal to the correlator axis. That would make it counterproductive to compute motion over too large a correlator span. The contributions from correlators of different spans have been quantified, though necessarily indirectly, using moving sinusoidal bar patterns (Buchner, 1976). Using correlated noise stimuli with delta correlations at specified space–time offsets, we found that neural response to motion is dominated by the nearest-neighbor interactions with perhaps some contribution from next-nearest neighbors (Figure 6). Interactions separated by more than three receptor units evoke negligible motion response in the conditions of our experiment. The weights of these relative contributions may very well depend on stimulus brightness (Heisenberg & Buchner, 1977), and it has been suggested that the system could increase spatial acuity at the cost of spatial sensitivity (Pick & Buchner, 1979) at higher light levels. Similar mechanisms also exist in the vertebrate retina, where signals converge on to bipolar cells from relatively large receptive fields at low light levels and smaller fields at high light levels (Sterling, 2004). These questions can be addressed in more detail by applying apparent motion stimuli at different light levels. 

The linear dependence of response on the square of contrast is a hallmark of the HRC model (Reichardt, 1961). This behavior can be seen as a corollary of the more general property of bilinearity, predicted by the HRC model. Our data clearly show that, over the range tested, the computation in the fly indeed has a bilinear character (Figure 7C). However, as implemented in the fly, this is a relative phenomenon in the sense that the response is normalized by the contrast flicker variance (Figure 7D, Figure 9). It is important to note that this is valid for the limited range of contrasts tested here: Clearly, this normalization must break down in the limit of very low levels of contrast variance. Currently we are testing this for a wider range of stimulus conditions. 

The observation of flicker normalization gives rise to two different sorts of questions, one about implementation and the other about the possible functional utility. From our experiments we can say very little about the implementation. Our data are consistent with the possibility that signals are scaled to flicker standard deviation very early on in the visual pathway, before the nonlinear element of the correlator, since the information is available locally in the visual field. But it is also conceivable that flicker information is relayed via a pathway parallel to the motion pathway (Schnell et al., 2012), and that scaling to the variance takes place after the nonlinear interaction. Our data do not allow us to draw conclusions about whether the variance measurement is pooled over a spatially extended domain. This question may be addressed in principle by experiments with inhomogeneous spatial distributions of flicker variance. There is evidence that the scaling mechanism is not instantaneous, but takes some time (on the order of a few hundred milliseconds) to adapt (de Ruyter van Steveninck & Laughlin, 1996; additional preliminary observations). As to the possible functional utility, it is well known that correlator output depends monotonically on stimulus velocity over a certain range, but depends on a host of other stimulus variables as well. The scaling normalization observed here removes at least one confounding variable from the correlator response, and results in a nearly contrast independent motion estimator (Potters & Bialek, 1994). 

Illusory motion stimuli have been a powerful tool for probing neural computations underlying motion detection. The response to a version of reverse-phi motion was in fact one of the key observations in the development of the HRC model (Reichardt, 1961). More recently, the reverse-phi illusion (Anstis, 1970), which involves contrast reversal in moving images, has provided clues to the type of nonlinear interactions between ON and OFF channels in the Drosophila visual system (Clark, Bursztyn, Horowitz, Schnitzer, & Clandinin, 2011). By construction, our apparent motion stimuli allow independent contrast reversals of the signals arriving at the detectors. A change in response from excitatory to inhibitory when the sign of m₁ or m₂ changes from positive to negative (Figure 8C), indicates that the neural system identifies this as a reversal in the direction of motion. There are slight differences in response gain to positive and negative motion energy. These may be due to the fact that the mechanisms that “measure” flicker and motion energy have finite integration times, which implies that they must average the energy around the peaks of the stimulus autocorrelation function over finite areas in the space–time plane (Figure 1A, B). It is conceivable that these areas overlap somewhat and because of the resulting “leaking” of energy, may lead to different values for the weighting of the normal and reversed stimulus and flicker energies. If the sign of Δx or Δt is changed, but contrast is not reversed, only the direction of perceived velocity changes. In this case, we also observe a change in gain for H1, which may be explained by a differential strength of excitation and suppression to motion observed in earlier work (Borst & Egelhaaf, 1990). 

Several studies have explored possibilities of modified HRC models that better predict naturalistic image velocity (Dror, O'Carroll, & Laughlin, 2001; Straw, Rainsford, & O'Carroll, 2008). Natural stimuli may be subject to large and fast dynamic fluctuations in their statistical properties, and many of these studies have proposed that adaptive filters are part of the visual processing pathway, including perhaps the correlator itself (see also Maddess & Laughlin, 1985; de Ruyter van Steveninck et al., 1986). In future work we will use generalizations of the stimuli described here, for example incorporating specific statistical nonstationarities in space and time, to study some of those adaptive strategies. 

Acknowledgments

We thank Dr. Shiva R. Sinha for help with developing the stimulus-generating interface and for providing valuable inputs on the project. 

Commercial relationships: none. 

Corresponding author: Rob de Ruyter van Steveninck.