May 2016
Volume 16, Issue 7
Open Access
Article  |   May 2016
Bilocal visual noise as a probe of wide field motion computation
Author Affiliations
Journal of Vision May 2016, Vol.16, 8. doi:10.1167/16.7.8
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Suva Roy, Rob de Ruyter van Steveninck; Bilocal visual noise as a probe of wide field motion computation. Journal of Vision 2016;16(7):8. doi: 10.1167/16.7.8.

      Download citation file:


      © 2017 Association for Research in Vision and Ophthalmology.

      ×
  • Supplements
Abstract

Using an apparent visual motion stimulus with motion energies limited to specific separations in space and time, we study the computational structure of wide-field motion sensitive neurons in the fly visual brain. There is ample experimental evidence for correlation-based motion computation in many biological systems, but one of its central properties, namely that the response is proportional to the product of two bilocal signal amplitudes, remains to be tested. The design of the apparent motion stimuli used here allows us to manipulate the amplitudes of the bilocal input signals that serve as inputs to the computation. We demonstrate that the wide-field motion response of H1 and V1 neurons indeed shows bilinear behavior, even under contrast sign reversal, as predicted. But the response also varies inversely with contrast variance, an effect not described by the correlator operation. We also quantify the correlator contributions for different spatial and temporal separations. With suitable modification, the apparent motion stimuli used here can be applied to a broad range of neurophysiological as well as human psychophysical studies on motion perception.

Introduction
The visual system has evolved neural circuits that allow the extraction of motion information from spatiotemporal patterns of light. Attempts at resolving the functional computational structure of these circuits have led to the development of several models for motion detection. The Hassenstein-Reichardt correlator (HRC) model, originally developed to explain the optomotor response of a beetle, postulates that motion detection arises from correlation of time delayed light signals from two spatial points (Hassenstein & Reichardt, 1956; Reichardt, 1961). Barlow and Levick (1965) proposed a mechanism for direction-selective computation in retinal ganglion cells that combines time-delayed excitatory and inhibitory inputs from the presynaptic bipolar cells. The motion energy model describes a spatiotemporal filter approach to understanding motion perception in humans (Adelson & Bergen, 1985). Despite the apparent mechanistic differences, these models share an irreducible computational principle for motion detection, namely the nonlinear combination of light signals measured at spatiotemporal offsets. 
The direction of motion computed from a moving visual pattern should be invariant to the sign of its contrast. This simple requirement directly implies that the computation of motion from a measurement of spatiotemporal contrast cannot be linear and, indeed, the lowest order polynomial interaction with this invariance is bilinear (Hassenstein & Reichardt, 1956; Potters & Bialek, 1994; Reichardt, 1961). This second order interaction also is the basis for computing correlation, and many psychophysical and neurophysiological studies therefore have designed stimuli based on variants of correlated random noise, such as dynamic visual noise (Morgan & Ward, 1980) and Julesz-type patterns (Braddick, 1974; van de Grind, Koenderink, & van Doorn, 1986; van Doorn & Koenderink, 1982a, 1982b; van Doorn, Koenderink, & van de Grind, 1985), to assess perceptual thresholds for motion detection in humans. A similar approach was used in the study by Britten, Shadlen, Newsome, and Movshon (1992), which involved the use of moving random dots with limited lifetime for studying how neurons in the MT (middle temporal) region of monkeys process motion. One advantage of these types of stimuli is that they lack spatial structures such as edges and large correlated patterns, which eliminates any response that could have been evoked specifically by these structures. That makes these stimuli particularly amenable for testing computational principles underlying motion perception. 
The computation of stimulus correlation is implemented directly in the HRC model, which also has been one of the most widely used models for understanding neural and behavior responses to motion in flies (Borst & Egelhaaf, 1989). For example, the response properties of wide-field motion sensitive neurons in the lobula plate support a correlation type mechanism similar to the HRC scheme (Egelhaaf, Borst, & Reichardt, 1989). More recent work on mutant Drosophila (Gal4-UAS system) has identified specific neurons in the medulla region that correlate signals derived from neighboring spatial locations (Schnell, Raghu, Nern, & Borst, 2012). The reverse-phi illusion, which induces robust optomotor response in flies, can also be explained using a correlation-type nonlinearity (Tuthill, Chiappe, & Reiser, 2011). Although these findings lend credence to the correlation scheme, it remains unclear to what extent correlation-based nonlinearities dominate visual motion computation under more complex visual conditions experienced during natural flight. It is known, for example, that variability in the spatial structure of the input, the speed of movement, and contrast substantially modulate gain and time-course of the response to motion (Egelhaaf & Borst, 1989; Maddess & Laughlin, 1985; O'Carroll, Barnett, & Nordstrom, 2011). Furthermore, there is ample evidence for various adaptive mechanisms in motion processing (Brenner, Strong, Koberle, Bialek, & de Ruyter van Steveninck, 2000; de Ruyter van Steveninck, Zaagman, & Mastebroek, 1986; Fairhall, Lewen, Bialek, & de Ruyter Van Steveninck, 2001; Harris, O'Carroll, & Laughlin, 1999; Maddess & Laughlin, 1985). 
The fly, by virtue of its compound eye and retinotopic visual organization and because of easy access to its motion sensitive neurons, is an attractive system for testing models of biological motion detection. The array of lenses and their associated ommatidia in the compound eye sample visual space in an approximately hexagonal angular raster with a well-characterized angular point spread function (Braitenberg, 1967; Land, 1997; Smakman, van Hateren, & Stavenga, 1984; Stavenga, 2003). The retinotopic columnar organization of the visual pathway preserves the geometry of the visual field as signals propagate through various neuropils. The movement-sensitive neurons in the lobula plate integrate local motion signals over large areas of the visual field, and produce action potentials or graded depolarizations. 
In this paper we are interested in how the computation of movement is implemented in the fly visual system. One limitation in studying this question with conventional moving patterns is that their correlation functions are continuous in space and time. Here we use apparent motion stimuli formed by random noise with a discontinuous space–time correlation function designed to have nonzero values only at selected space–time separations (and, necessarily, also at the origin). The construction of this stimulus is described in the Methods section, along with a calculation of the correlator response. Fly motion-sensitive cells strongly respond to such stimuli, and for human observers they generate a clear percept of coherent motion superposed on a flickering background. These stimuli were presented to the fly in a hexagonal pixel arrangement at a pitch that was adjusted to the fly's sampling raster. Recordings were made from the lobula plate tangential neurons H1 and V1, which integrate local inputs over a large visual field to produce spiking response to motion in the horizontal and vertical directions, respectively (Hausen, 1976). The response of these neurons reveals that at constant flicker contrast, wide-field motion response varies bilinearly with the input signals. This confirms that motion detection involves a multiplicative nonlinearity, in accordance with the HRC construction. However, we find that, to good approximation, the neural response is normalized to the background flicker, countering the predictions of a strict HRC model. This result suggests that the visual system may scale its computational algorithm depending on signal-to-noise ratio of the input (de Ruyter van Steveninck, Bialek, Potters, & Carlson, 1994; Potters & Bialek, 1994). 
Methods
Fly preparation
Wild-type blowflies (Calliphora vicina) were captured in traps set outdoors. They were then transferred inside where they were housed in an enclosed arena and supplied with sugar, water, tomato juice, and dry protein. The temperature was controlled at 21°C and humidity maintained at approximately 60%. The ambient light was set to alternate between on and off with a 12-hr cycle. To prepare the fly for the experiment, its wings and legs were immobilized with wax, after which it was placed in a plastic holder such that its head protruded out. At the back of the head, an incision was made with a razor blade and a small piece of integument was surgically removed. Excess fat and some air sac membrane were removed from the superficial layers, and a muscle in the ventrolateral region was cut to prevent large twitches and their associated electrical interference during extracellular recording. The proboscis was kept free for fluid intake during the experiment. The holder containing the fly was placed on a goniometer platform, which allowed initial adjustments to the azimuth and elevation angles of the fly's eye. A stand-mounted movable Nikon SMZ460 optical microscope (Nikon Instruments Inc., Melville, NY) was used to view the fly from the back. We used both male and female flies for our experiments. The responses were cross-validated against at least four flies, and the results shown here correspond to a single H1 or V1 neuron with the clearest response behavior. 
Electrophysiology
Extracellular recordings were made from H1 and V1 spiking neurons (Hausen, 1976) in the right brain hemisphere, contralateral to the eye receiving the visual input. For H1 (V1), head orientation was adjusted to minimize response to vertical (horizontal) motion. Tungsten microelectrodes (FHC Inc., Bowdoin, ME) with tip diameter 1 μm and 3 MΩ impedance were used to record differential voltage, which was subsequently low pass filtered by an amplifier and discretized by a window discriminator (World Precision Instruments, Inc., Sarasota, FL). The discriminator pulses were time-stamped at 10 μs resolution by a National Instruments PCI-6259 Data Acquisition Card (National Instruments Corporation, Austin, TX) and stored in the computer for analysis. Recordings were only made from clear isolated spikes with amplitude exceeding 250 μV, such that the amplitude remained at least five times above the baseline voltage fluctuations. To prevent tissue desiccation, the fly was fed sugar solution in between experiments. 
Visual stimulation
Display screen
Visual stimuli were displayed at a frame rate of 500 Hz on a Tektronix 608 CRT (cathode ray tube) standard phosphor (P31) monitor (Tektronix, Beaverton, OR) with maximum radiance of 150 mW/sr · m−2. At mean radiance this produces of order 5 × 104 absorbed photons per second in each photoreceptor. The image on the screen consists of 827 pixels arranged in a hexagonal grid that extends 38° along the horizontal direction and 44° along the vertical direction. The frame covers approximately 8% of the total angular visual field and 15% of the angular field of H1 (Krapp & Hengstenberg, 1997). Angular distortions resulting from projection on a flat 2-D screen were corrected to first order by increasing the local raster spacing for pixels farther away from the center, following a gnomonic projection (Coxeter, 1969). The distance between the screen and the fly was adjusted to match the projection of the screen pixel raster onto the angular pitch of the fly's ommatidial raster (see section on Nyquist experiment under Results). 
P31 phosphor luminance is known to persist over several milliseconds and therefore, in principle, could give rise to stimulus artifacts that induce response unrelated to the actual stimulus. This issue has been raised on several occasions (Westheimer, 1993; Wolf & Deubel, 1997) targeted at studies in which the visual system is sensitive to temporal fluctuations in light occurring at the time scale over which the phosphor luminance decays (Zele & Vingrys, 2005). Therefore, we measured the phosphor luminance profile of a single pixel on the Tektronix 608 CRT screen using a pin photodiode. We found that the pixel luminance decreases to 8% of its maximum in 151 μs, 4% of its maximum in 279 μs, and 1.4% of its maximum in 569 μs (Supplementary Figure 1A, B). At 2 ms, the residual luminance is 0.04%. Note that the decay of residual luminance has a long tail that persists for several milliseconds. This leads to temporal averaging of luminance signals causing a reduction in the stimulus contrast. Using the average pixel luminance over the 2 ms period (Supplementary Figure 1A) and the maximum and minimum luminance at the onset and at the end of the 2 ms period, respectively, we find that the upper bound of the overall effect on contrast (according to Michelson contrast definition) is 1.24%. So, for all practical purposes, the reduction in contrast is negligible compared to the actual stimulus contrast, and therefore unlikely to impact the neural response to motion. 
Apparent motion stimuli
Variants of apparent motion stimuli have been used extensively in psychophysical motion perception studies. Julesz-like patterns were used by Braddick (1974) and dynamic random dot stimuli by Morgan and Ward (1980), both in human psychophysical studies. The response of MT neurons in monkey was characterized based on dynamic random dot stimulus displays (Britten et al., 1992). Moving spatial white noise patterns with fast alternations in space or time were employed to study computational aspects of motion detection in humans (van Doorn & Koenderink, 1982a, 1982b). The lack of spatial higher order structures in these stimuli means that motion response is induced solely by spatiotemporal correlations of light intensities. Our stimuli share features of these, but are based on discontinuous steps in space and time, and allow the independent setting of contrast for the two inputs to motion computation. 
In its most elementary form, a visual motion stimulus is associated with a light signal at one location and its delayed repetition at another location. Thus, a pair of points separated by a distance Δr⃗, showing identical temporal intensity sequences over time with a delay Δt between them, contains motion information consistent with velocity v⃗ = Δr⃗ / Δt. Such a bilocal signal would be detected by a correlator with an input span of Δr⃗. By tiling the visual field with a superposition of such bilocal signals, we create a wide-field version of this stimulus. As the basis for the stimuli in this study, we use spatiotemporal white noise by drawing a time series of spatial 2-D random values, I0(r⃗,t), from an identically independent distribution (IID). Here, r⃗ and t represent the spatial coordinate and time, respectively. The spatiotemporal autocorrelation function of I0(r⃗,t) is a delta function in space and time:    
To generate wide field apparent motion, we take I0(r⃗,t) and add to it an “echo” of itself with displacements Δr⃗ in space and Δt in time:  using weights m1 and m2 for the original and its echo. The autocorrelation of this combined pattern is,    
The first term represents a peak of height Image not available + Image not available , at location (r⃗, t) = (0,0). The second term represents a pair of peaks, located at (Δr⃗, Δt) and (−Δr⃗, −Δt), with identical heights m1 · m2 (see Figure 1). The central peak describes the random flicker component in the stimuli, with flicker contrast variance of Image not available + Image not available . The pair of satellite peaks carries motion signal of strength m1 · m2, with apparent velocity v⃗app = Δr⃗ / Δt. Displaying I(r⃗,t) as a space–time sequence of intensities produces a strong impression of wide-field motion superposed on a flickering background. For brevity, we will refer to the product term m1 · m2 as “stimulus motion energy” and the term Image not available + Image not available as “stimulus flicker energy.” Here, energy is defined in units of the product of two contrast parameters, consistent with the definition originally proposed by Adelson and Bergen (1985). The stimulus parameters are chosen to have zero mean intensity and maximum contrast. The values for I0 are drawn from a binary distribution such they are either −1/2 or +1/2. The constraint |m1 + m2| ≤ 2 ensures that m1 and m2 always have real values and contrast lies in the range [−1, 1]. For clarity, we omit from our equations an additive constant mean intensity, numerically equal to 1. We can ignore this constant term since it does not contain motion information and does not play any role in our analyses.  
Figure 1
 
Space-time correlation function of the apparent motion stimuli used in this study. (A) Idealized condition, with delta correlation in both space and time, plotted for one space dimension, namely X. The vertical red lines located at (Δt, Δx) and (−Δt, −Δx) represent nonzero motion energy and the vertical blue line located at (0, 0) represents nonzero flicker energy. The flicker energy is twice the motion energy for the condition m1 = m2. The four quadrants of the energy space are denoted by I, II, III, and IV. (B) Motion energy is negative for the condition m1 = −m2. (C, D) In practice, patterns are displayed for a finite time Tp, which in our experiments is an integer multiple of the 2 ms frame time. For most experiments a longer value of Tp is used, typically a few frame intervals, because this is matched better to the integration time of fly photoreceptors. The example in (C) shows the triangular profile of the autocorrelation function Λ(t) (see Equation 5) for Tp = 6 ms. Our stimuli avoid the situation in (D) where Tp > Δt, because that causes some of the motion energy to leak into the adjacent quadrants. The temporal offset is given by Δt. The space, time, and stimulus energy are represented by the x, y, and z axes, respectively.
Figure 1
 
Space-time correlation function of the apparent motion stimuli used in this study. (A) Idealized condition, with delta correlation in both space and time, plotted for one space dimension, namely X. The vertical red lines located at (Δt, Δx) and (−Δt, −Δx) represent nonzero motion energy and the vertical blue line located at (0, 0) represents nonzero flicker energy. The flicker energy is twice the motion energy for the condition m1 = m2. The four quadrants of the energy space are denoted by I, II, III, and IV. (B) Motion energy is negative for the condition m1 = −m2. (C, D) In practice, patterns are displayed for a finite time Tp, which in our experiments is an integer multiple of the 2 ms frame time. For most experiments a longer value of Tp is used, typically a few frame intervals, because this is matched better to the integration time of fly photoreceptors. The example in (C) shows the triangular profile of the autocorrelation function Λ(t) (see Equation 5) for Tp = 6 ms. Our stimuli avoid the situation in (D) where Tp > Δt, because that causes some of the motion energy to leak into the adjacent quadrants. The temporal offset is given by Δt. The space, time, and stimulus energy are represented by the x, y, and z axes, respectively.
At a frame time of 2 ms, intensity transitions are substantially faster than the photoreceptor integration time (Juusola, Kouvalainen, Jarvilehto, & Weckstrom, 1994; Payne & Howard, 1981; Weckstrom, 1989). Therefore, to drive the photoreceptor response to the light fluctuations effectively, stimuli are displayed with a “persistence time,” denoted by Tp, with values close to the photoreceptor integration time, typically Tp = 4 − 8 ms. Tp determines the time for which a particular random sample I0 remains on the screen, which can be formalized as a rectangle function (Bracewell, 2000).    
The temporal autocorrelation of the sample I0 is obtained by convolving the rectangle function Π(t) with itself, resulting in a triangle function Λ(t) (Bracewell, 2000):    
The spatial part of the autocorrelation function remains unchanged as δ(r⃗). With finite Tp the cross-correlation of intensity (Equation 2) is now:    
As before (Equation 3), the cross-correlation has three peaks, but the peaks are now centered on the triangle functions which are nonzero within the time window 2Tp, as illustrated in Figure 1. Note that each of the two components of I(r⃗, t), that is, the original and its echo, persist for a time Tp. Our setup, however, allows us to manipulate the values of Δt in increments of 2 ms frame time, independently of the choice of Tp. In our experiments we found an overall increase in the response magnitude with values of Tp in the 4–8 ms range, consistent with the idea that increasing Tp increases the effective contrast for the fly. We did not notice any qualitative changes in other response properties with values of Tp in the 2–8 ms range (not shown). Supplementary movies MVI_m0.2_f0.4, MVI_m0.2_f2.0, MVI_m0.8_f2.0, and MVI_m0.8_f1.6_pos_neg, show different stimulus realizations obtained by manipulating the contrast parameters {m1, m2} and the time offset Δt (see Supplementary Materials). 
Spatial alignments
Alignment of the head with respect to the axes of the screen was done for each fly individually, using the regular hexagonal pixel grid displaying bar patterns (contrast = 1, spatial wavelength = 5°) moving at 50°/s. The response to preferred motion was maximized and the response to null motion minimized, resulting in an azimuth angle of about 35°. The roll angle was adjusted to match the responses to motion in the upward and downward directions. This alignment procedure was carried out separately for H1 and V1 neurons, prior to the actual experiments. 
The hexagonal raster of pixels displayed on the screen (see Figure 6A) has a nearest neighbor distance of Δr ≅ 1.8 mm. The column to column separation Δx = Δr · cos(30°) ≅ 1.5 mm defines the angular spacing in the horizontal direction (x axis in Figure 6A) and the row to row separation Δy = Δr / 2 ≅ 0.9 mm defines the angular spacing in the vertical direction (y axis in Figure 6A). The ommatidial raster of the fly's eye samples space in a hexagonal array with angular spacing Δφ; its precise value varies from fly to fly, but is typically of the order Δφ ≅ 1.5°. With these parameters, the pitch of the stimulus array matches the fly's angular spacing at a fly-to-screen distance of D = Δr / tan−1φ) ≅ 70 mm. The photoreceptor point spread function (PSF; Snyder, Stavenga, & Laughlin, 1977) admits spatial frequencies higher than the raster's Nyquist limit, leading to aliasing of high spatial frequencies. This effect can be measured in motion sensitive neurons when the fly is presented with moving patterns at high enough spatial frequencies to induce a reverse response (Götz, 1964). We use this effect to adjust the visual stimulus to the fly's sampling raster. The horizontal and the vertical projections of interommatidial separation are (see also Figure 3):     
The standard hexagonal pixel arrangement does not allow us to display patterns with short enough wavelengths. Therefore, specifically for this experiment, the pixel arrangement was reformatted to a rectangular lattice with an intercolumn separation Δl of exactly one quarter of the horizontal projection: Δl = Δx / 4. For H1, the fly's motion response was tested using bar patterns of spatial wavelength λs equal to 6, 8, and 12 lines, or λs = 1.5 · Δx, 2 · Δx, 3 · Δx, respectively. The correct match was found by adjusting the fly-to-screen distance such that there was no change in response to motion in the preferred and null directions for λs = 2 · Δx. For H1 recordings, the pattern was moved along the horizontal axis alternatingly in the preferred and null directions for 4 s each, and repeated across 50 trials. For V1, the rectangular pixel grid was rotated by 90°, with the spatial wavelengths appropriately scaled and the distance was adjusted while presenting vertical movement. Because the motion response of tangential cells decreases at high temporal frequencies (Hausen & Egelhaaf, 1989), the velocity of the stimulus for this experiment was kept relatively low, to about 6°/s. 
Results
Hassenstein-Reichardt correlator model
The HRC model has been extensively used to explain visual the motion response in flies and in other insects (Buchner, 1976; Clifford & Ibbotson, 2002; Hassenstein & Reichardt, 1956). In its simplest form (see Figure 2), the signal in each arm is delayed by the low-pass temporal delay filter G(t) and then multiplied by the instantaneous signal of the other arm. The signals from the mirror-symmetric arms after multiplication are subtracted to produce an antisymmetric, direction-selective output (Figure 2A). The wide-field response can be modeled by replicating the correlator structure over the visual field and integrating the local outputs over the entire array of detectors. The inputs of the correlator can span the nearest neighbor distance, but can also be separated by multiple raster spacings, and in general, the motion response can be thought of as a weighted sum of correlator populations with different spans. To mimic the fly's optics and make the model physiologically relevant, we introduce a spatial filter S(x) that represents the point-spread function (PSF) of the photoreceptors (Land, 1997; Snyder et al., 1977), a linear temporal filter F(t) representing the photoreceptor impulse response and a linear temporal delay filter G(t) (Figure 2A). Further details about the functional form and parametrization of filters can be found in the section, Correlator response to apparent motion stimuli (below). More filters can, in principle, be added, for example in the cross-arms. However, because filtering in the model is treated as linear, the properties of the cross-arm filter can be accounted for by appropriate compensations of F(t) and G(t). We expect the model to be rich enough to describe correlator behavior in our experiment, and therefore refrain from using more complex elaborations of the HRC model (van Santen & Sperling, 1985). 
Figure 2
 
Schematics of a 1-D correlator model. (A) The elementary correlator consists of two mirror-symmetrical subunits with span equal to integer number of Δφ ≈ 1.5°. The light input is represented by I(x,t) and the Gaussian PSF (σ = 0.5°) of the photoreceptors at locations 1 and 2 are represented by S1 and S2, respectively. The other components of the correlator circuit are, F(t): lognormal temporal prefilter and G(t): first-order exponential delay filter. The multiplicative nonlinearity, spatial summation, and correlator output are denoted by x, ∑, and R, respectively. (B) An array of correlators with different spans. Unit span is equal to Δφ (see also Poggio & Reichardt, 1973).
Figure 2
 
Schematics of a 1-D correlator model. (A) The elementary correlator consists of two mirror-symmetrical subunits with span equal to integer number of Δφ ≈ 1.5°. The light input is represented by I(x,t) and the Gaussian PSF (σ = 0.5°) of the photoreceptors at locations 1 and 2 are represented by S1 and S2, respectively. The other components of the correlator circuit are, F(t): lognormal temporal prefilter and G(t): first-order exponential delay filter. The multiplicative nonlinearity, spatial summation, and correlator output are denoted by x, ∑, and R, respectively. (B) An array of correlators with different spans. Unit span is equal to Δφ (see also Poggio & Reichardt, 1973).
Figure 3
 
Spatiotemporal receptive field of the model correlator. The 3-D profile shows the receptive field of a 2 unit model correlator as a function of space (η) and time (ζ) for (A) time constant τ = 10 ms, and for (B) time constant τ = 300 ms of the delay filter G(t) (see Equation 19). For this plot, the photoreceptor temporal filter F(t) is replaced by a log-normal function with parameters K = 1, tp = 15 ms, s = 0.23 (Payne & Howard, 1981), and the delay filter G(t) is replaced by a first order exponential. The solid gray curve to the left and to the right represent the antisymmetric spatial and antisymmetric temporal part of the receptive field. The z axis represents the weighting factor. Excitatory response is shown in red (quadrants I and III) and suppressive response is shown in blue (quadrants II and IV).
Figure 3
 
Spatiotemporal receptive field of the model correlator. The 3-D profile shows the receptive field of a 2 unit model correlator as a function of space (η) and time (ζ) for (A) time constant τ = 10 ms, and for (B) time constant τ = 300 ms of the delay filter G(t) (see Equation 19). For this plot, the photoreceptor temporal filter F(t) is replaced by a log-normal function with parameters K = 1, tp = 15 ms, s = 0.23 (Payne & Howard, 1981), and the delay filter G(t) is replaced by a first order exponential. The solid gray curve to the left and to the right represent the antisymmetric spatial and antisymmetric temporal part of the receptive field. The z axis represents the weighting factor. Excitatory response is shown in red (quadrants I and III) and suppressive response is shown in blue (quadrants II and IV).
Time and space average of the correlator response
Here we derive an expression for the time-averaged response of the model correlator (Figure 2) suitable for describing the response to our apparent motion stimuli. Because wide-field motion-sensitive neurons integrate signals over a large array of local detectors, their mean rates can be described by the time-averaged response of one correlator times the number of correlators over which the signals are integrated (Figure 2B). To keep the derivation simple, we use the simplest version of the HRC model, consisting of a spatial filter S that represents the photoreceptor PSF, a temporal filter F that represents the impulse response of the photoreceptor, and a delay filter G that is approximated by a first-order exponential (Figure 2A). For ease of notation, we also limit space to one dimension. Light intensity as a function of space and time is given by I(x,t); the PSF of the correlator centered at x = 0 is given by S(x), and PSFs of the correlators centered at x = −Δφ / 2 and x = Δφ / 2 are given by S1 and S2, respectively. Here Δφ is the span, or the angular distance between the two correlator inputs. The linear spatial and temporal filters transform the signals as follows:   where ⊗ represents convolution, H is defined as the convolution of G and F, and the indices of u, v, and S identify the two correlator inputs (Figure 2A). The output, R, of the correlator after the multiplication stage is the difference of two products defined as R+ and R, respectively:  where “·” represents multiplication and R is a function of x and t. By averaging the response over time and space, we obtain the equivalent wide-field averaged response of the correlator system. If we just focus on one branch of the correlator:    
Writing this in terms of all the convolution integrals and rearranging orders of integration, we get:    where ΘII is the stimulus spatiotemporal autocorrelation function. Substituting ζ = ξ1ξ2, ξ = ξ2 and η = τ1τ2, τ = τ2, we get:    
Defining the spatial self-overlap of the point spread function as  we write the full correlator response as:    
Thus, the averaged response of the correlator is an integral over space and time of the stimulus autocorrelation multiplied by a purely temporal term (first set of braces), and a purely spatial term (second set of braces). The correlator is thus described by a separable spatiotemporal “receptive field” and its averaged output to the stimulus is given by the inner product of that receptive field with the stimulus autocorrelation. Note that the spatial term of the receptive field is antisymmetric, while the temporal term is not. Because the stimulus autocorrelation is a symmetric function under joint space and time reversal: ΘII(ζ,η) = ΘII(−ζ,−η), the calculated response of the correlator is always antisymmetric in both time and space separately, and thus symmetric under joint exchange of space and time. This implies that we get the same outcome for the inner product if we define the receptive field Γcorr(ζ,η) as a sum of its regular and time-reversed parts:  and with this definition we have:    
An example of Γcorr(ζ,η) is shown in Figure 3, along with its spatial and temporal components. Note that this derivation is not limited to the stimuli we use in our experiment, but applies generally to wide field motion stimuli with constant velocity. 
Correlator response to apparent motion stimuli
To illustrate the spatiotemporal response properties of the correlator receptive field (Equation 19), we provide specific functional forms for the filters, derived from experimental data. The spatial filter S(x) is approximated by a Gaussian Image not available of standard deviation σ = 0.51° (Smakman et al., 1984), the temporal filter F(t) is approximated by a log-normal function K · exp[−{log(t / tp)}2 / 2s2] with parameter values tp = 15 ms and s = 0.23 chosen from photoreceptor measurements (Howard, Dubs, & Payne, 1984; Payne & Howard, 1981), and the temporal delay filter G(t) is approximated by a first order exponential (1 / τ)et/τ (Srinivasan, 1983), where τ can range from few tens of milliseconds to hundreds of milliseconds (de Ruyter van Steveninck et al., 1986; Harris et al., 1999).  
The time-averaged response of the correlator as a function of the temporal offset Δt, exhibits an initial transient, followed by a peak and an exponential decay (Figure 3A, B). A small value of time constant τ = 10 ms is associated with a sharper transient and smaller peak latency (Figure 3A) compared to a larger time constant τ = 300 ms (Figure 3B). Note that the correlator's response qualitatively matches the responses of H1 and V1 to change in Δt (see Figure 6D, E). A positive response in quadrants I and III of the energy space indicates excitation from preferred motion, while negative response in quadrants II and IV indicates suppression from null motion. The temporal response profile is symmetric about the vertical axis (Figure 3A, B, gray projection to the right) because the filters in the two correlator arms are identical. This, however, need not be true for an actual biological correlator, where different gain values in the two arms based on increment or decrement of signal may lead to asymmetric response curves (Borst & Egelhaaf, 1990). The spatial response profile is a difference of two Gaussians centered respectively at the location of the two detectors, where each Gaussian represents the convolution of the photoreceptor PSFs (Figure 3A, B, gray projection to the left). Together, these define the 3-D landscape of the full receptive field of the correlator Γcorr(ζ,η). 
The correlation structure of the apparent motion stimuli in our experiments, as defined by Equation 6, is essentially a pair of delta functions defined by space and time offsets Δr⃗ and Δt and amplitudes m1, m2. So, by measuring the response of the cell as a function of Δr⃗ and Δt, keeping m1 · m2 constant, we essentially sample the surface of the correlator receptive field at those locations, as long as the persistence time Tp remains small enough. If we vary m1 and m2 the response is predicted to scale in proportion to m1 · m2
Nyquist experiment
According to the sampling theorem, a discretely sampled waveform accurately represents the original waveform only if the sampling frequency if higher than twice the highest frequency present in the waveform. This limiting frequency is called the Nyquist frequency (Nyquist, 1928; Shannon, 1949). For lower sampling frequencies, aliasing leads to artifacts in the sampled waveform. The fly visual system is also susceptible to this problem. The hexagonal array of ommatidia samples the visual image discretely with a sampling frequency of 1 / Δφ, where Δφ is the angular spacing between neighboring ommatidia. For blowfly, this spacing is of the order of ≈1.5°. If the stimulus contains frequencies higher than 1 / (2Δφ), it leads to aliasing, and this effect can be measured in motion sensitive neurons that elicit reverse response when the fly is presented with moving patterns at high enough spatial frequencies (Götz, 1964). We exploit this response behavior to map the visual stimulus to the fly's sampling raster. 
To match the receptor spacing to the spacing of the pixels, we used a rectangular pixel grid (see Methods) and vertical bar patterns with spatial wavelengths of 6, 8, and 12 lines respectively for experiments on H1 (see Figure 4 upper panel) with line spacing Δl = Δx / 4. The fly-to-screen distance was adjusted so that the pattern with spatial wavelength of 8 lines produced equal responses to preferred and null directions of horizontal movement. This criterion resulted in an adjusted eye to screen distance (D) of about 78 mm with a standard deviation of 5 mm across four flies tested. From the spacing of lines on the screen (Δl ≈ 0.45 mm), we compute Δφx tan−1(4 · Δl / D) ≅ 1.3°. As an additional check, and also to verify the quality of the fly's image, we used spatial wavelengths of 6 and 12 lines. In this condition, the neuron is expected to show a reverse response for λs = 1.5 · Δx (6 lines) because of aliasing and a normal response for λs = 3 · Δx (12 lines), which is what we found. H1 produced a reverse and a normal response corresponding to these wavelengths, with rates differing by a factor of approximately 5 in both cases (Figure 4). Test for Nyquist response was also conducted on V1 using the same pixel grid but rotated 90° about its center. 
Figure 4
 
Spatial Nyquist frequency test. Bar patterns of three spatial wavelengths, λs = 1.5·Δx (6 lines), λs = 2·Δx (8 lines), and λs = 3·Δx (12 lines), at contrast 1.0 were presented. The horizontal projection of the interommatidial separation is Δφx. The responses to motion in the preferred and null directions are shown by hatched bars and solid gray bars, respectively. The Nyquist response can be seen for the case with λs = 1.5·Δx. Velocity was maintained at 6°/s for all spatial wavelengths. Error bars represent SD of the firing rate obtained from 50 trials.
Figure 4
 
Spatial Nyquist frequency test. Bar patterns of three spatial wavelengths, λs = 1.5·Δx (6 lines), λs = 2·Δx (8 lines), and λs = 3·Δx (12 lines), at contrast 1.0 were presented. The horizontal projection of the interommatidial separation is Δφx. The responses to motion in the preferred and null directions are shown by hatched bars and solid gray bars, respectively. The Nyquist response can be seen for the case with λs = 1.5·Δx. Velocity was maintained at 6°/s for all spatial wavelengths. Error bars represent SD of the firing rate obtained from 50 trials.
Response does not depend on pixel-to-photoreceptor matching
Even if the pitch of the pixel raster on screen matches the pitch of the ommatidial sampling raster (Figure 4) that does not guarantee that the photoreceptor optical axes are exactly aligned with the pixel positions. A mismatch might result in a change in the amplitude of the light signal at each photoreceptor location, which may subsequently change the response to apparent motion. To test this, we presented apparent motion along −30°, +30°, and +90° directions and measured the response of H1 at different absolute locations of the pixel grid (see lower panel of Figure 5). 
Figure 5
 
Response dependence on absolute location of pixel grid. The plot shows the response mean and the response variation around mean, of H1, for four different directions grid shift: along −30°, +30°, 90° (y axis), and 0° (x axis). The full grid consists of 827 pixels. The original and shifted grid locations are exemplified by the hexagonal pattern of black and gray dots respectively, in the bottom panel. Shifts along the x axis are made in steps of (Δx ≈ 1.3°)/4 (blue dotted line), while shifts along the other three directions are made in steps of (Δr ≈ 1.5°)/4 (red dotted line). Four consecutive shifts are made to cover one period (i.e., Δx along x axis and Δr along other directions). The color codes represent the six directions of apparent motion (see legend). The error bars represent SD of mean firing rate obtained from four grid locations, corresponding to each direction of grid shift. The dotted lines represent the response averaged over the four different directions of grid shift.
Figure 5
 
Response dependence on absolute location of pixel grid. The plot shows the response mean and the response variation around mean, of H1, for four different directions grid shift: along −30°, +30°, 90° (y axis), and 0° (x axis). The full grid consists of 827 pixels. The original and shifted grid locations are exemplified by the hexagonal pattern of black and gray dots respectively, in the bottom panel. Shifts along the x axis are made in steps of (Δx ≈ 1.3°)/4 (blue dotted line), while shifts along the other three directions are made in steps of (Δr ≈ 1.5°)/4 (red dotted line). Four consecutive shifts are made to cover one period (i.e., Δx along x axis and Δr along other directions). The color codes represent the six directions of apparent motion (see legend). The error bars represent SD of mean firing rate obtained from four grid locations, corresponding to each direction of grid shift. The dotted lines represent the response averaged over the four different directions of grid shift.
Figure 6
 
Spatial sensitivity profile and response tuning to apparent velocity. (A) The gray dots in the schematic represent the location of spatial correlations in units of the fly's sampling raster. Horizontal and vertical projections of Δφ ≈ 1.5° are Δφx and Δφy (Equations 7, 8). Angles are measured with respect to the x axis, where x and y axes define the lab coordinate system. (B) Excess firing rate of H1 is shown for correlations between the central pixel (color-coded black), and pixels located at ±30° (1, ±1), at 0° (2, 0) and at ±15° (3, ±1). (C) Excess firing rate of V1 is shown for correlations between the central pixel, and the pixels located at ±60° (±1, 1), at 0° (0, 2) and at ±30° (±1, 3). Here, x and y axes are in units of pixel separation on the screen: Δx, Δy. The gray vertical bar shows the range of response. The SD of the firing rate is <10% of the mean rate. Contrast is set to maximum and Δt is set to 8 ms. Response of (D) H1 for spatial offsets (Δx, Δy) = (1, ±1), (2,0), (3, ±1), and (E) V1 for spatial offsets (Δx, Δy) = (±1, 1), (0, 2), (±1, 3), as a function of the temporal offset Δt. The green and purple circles represent the response summed over the two locations: Δy = ±1 for odd column number for H1, and Δx = ±1 for even row number for V1. The orange circles represent the response obtained from the locations: Δy = 0 for even column number for H1, and Δx = 0 for even row number for V1. Error bars represent SD of the mean firing rate calculated over 50 trials. The solid lines represent nonlinear least-square fits using the temporal receptive field of the model correlator (Equation 19) with parametrized F(t) and G(t). The parameter values that minimized the least square error are chosen for the fit. The correlator contributions to the wide-field response of (F) H1 and (G) V1, are shown as a function of Δt. Similar to D and E, the green and purple curves represent the summed contributions from the two locations lying on the same vertical line (Δy = ±1) for H1 and on the same horizontal line (Δx = ±1) for V1. The indices of β correspond to the locations described in D and E. The frame persistence, Tp, was set to 6 ms.
Figure 6
 
Spatial sensitivity profile and response tuning to apparent velocity. (A) The gray dots in the schematic represent the location of spatial correlations in units of the fly's sampling raster. Horizontal and vertical projections of Δφ ≈ 1.5° are Δφx and Δφy (Equations 7, 8). Angles are measured with respect to the x axis, where x and y axes define the lab coordinate system. (B) Excess firing rate of H1 is shown for correlations between the central pixel (color-coded black), and pixels located at ±30° (1, ±1), at 0° (2, 0) and at ±15° (3, ±1). (C) Excess firing rate of V1 is shown for correlations between the central pixel, and the pixels located at ±60° (±1, 1), at 0° (0, 2) and at ±30° (±1, 3). Here, x and y axes are in units of pixel separation on the screen: Δx, Δy. The gray vertical bar shows the range of response. The SD of the firing rate is <10% of the mean rate. Contrast is set to maximum and Δt is set to 8 ms. Response of (D) H1 for spatial offsets (Δx, Δy) = (1, ±1), (2,0), (3, ±1), and (E) V1 for spatial offsets (Δx, Δy) = (±1, 1), (0, 2), (±1, 3), as a function of the temporal offset Δt. The green and purple circles represent the response summed over the two locations: Δy = ±1 for odd column number for H1, and Δx = ±1 for even row number for V1. The orange circles represent the response obtained from the locations: Δy = 0 for even column number for H1, and Δx = 0 for even row number for V1. Error bars represent SD of the mean firing rate calculated over 50 trials. The solid lines represent nonlinear least-square fits using the temporal receptive field of the model correlator (Equation 19) with parametrized F(t) and G(t). The parameter values that minimized the least square error are chosen for the fit. The correlator contributions to the wide-field response of (F) H1 and (G) V1, are shown as a function of Δt. Similar to D and E, the green and purple curves represent the summed contributions from the two locations lying on the same vertical line (Δy = ±1) for H1 and on the same horizontal line (Δx = ±1) for V1. The indices of β correspond to the locations described in D and E. The frame persistence, Tp, was set to 6 ms.
The responses of H1 to velocities along +30° and −30° were higher than the responses to velocities along −150° and +150°, respectively, by at least a factor of 2.5, for all the directions of grid shift (Figure 5, red, blue filled circles). The response to downward motion was slightly stronger than the response to upward motion (Figure 5, green filled circles). Because H1 is sensitive to horizontal motion, the response to vertical motion can be attributed to stimulus flicker and spontaneous discharge of H1. For a particular motion direction, the standard deviation of the responses corresponding to different directions of grid shift is less than 10% of the mean responses. Together, these results indicate that the mean spiking response is independent of the precise match between pixels and photoreceptors. As a more sensitive test, we investigated whether the pixel size impacts response dependence on grid shift. For this, we reduced the pixel spot size to about a quarter of the original size by refocusing the electron beam. While this resulted in a 60% decrease in intensity and a subsequent reduction in the overall response (results not shown), the response characteristics displayed in Figure 5 did not show any significant change. Based on this result, exact one-to-one alignment does not seem to be a prerequisite for the interpretation of our data. Hence, there seems to be no benefit to achieving exact alignment. Although optical methods for alignment can be used (Franceschini, 1975), we chose not to apply them because they are intrusive, and substantially limit the lifetime of the fly in the experiment. 
Spatial sensitivity map
It has been suggested previously that contribution of signal interaction to global motion response is confined to correlators with a span of up to a few neighboring receptor elements (Buchner, 1976; Pick & Buchner, 1979). Here, we quantify the contribution of local interactions at different spans for both H1 and V1 by varying the spatial offsets between correlated signals in the stimulus. The contrast is set to maximum and the temporal offset is set to Δt = 8 ms. Each trial consists of 2.5 s of apparent motion in the preferred direction (i.e., +X for H1 and −Y for V1), followed by 2.5 s of pure flicker. To generate the pure flicker condition, we set Δt to 200 ms, which is experimentally equivalent to Δt → ∞ (see stimulus movie clips in Supplementary Materials). For clarity, we shall use units of column separation (Δφx ≅ 1.3°) and row separation (Δφy ≅ 0.75°) to refer to the spatial offsets in the (x, y) coordinate system (Figure 6A), which in matched conditions correspond to the (Δx, Δy) values of the raster displayed on screen. 
The excess firing rate of H1 to apparent motion is highest for spatial correlation between pixels at (Δx, Δy) = (1, ±1), with no noticeable difference in response for the two values of Δy. This indicates that the largest contribution to the motion response comes from nearest-neighbor interactions (Figure 6B). The excess firing rate drops from 29 spikes/s to 21 spikes/s as the column separation increases from 1 to 2. At a column separation of 3, the excess firing rate drops to 8 spikes/s. For larger column separations, the response was indistinguishable from the pure flicker response. For V1, the excess firing rate rises to a maximum of 25 spikes/s at (Δx, Δy) = (0, 2), (Figure 6C). For the nearest neighbor pixels oriented at ±60° about the vertical axis, that is, with (Δx, Δy) = (±1, 1), the excess firing rate reduces to 17 spikes/s. With Δy = 3, the response further diminishes to 14 spikes/s. This indicates that the strongest excitation to V1 comes from nearest-neighbor interactions with velocity in the preferred direction (downward). 
In the stimulus conditions described thus far, we primarily changed Δr⃗, keeping Δt constant. But apparent velocity depends on both Δr⃗ and Δt. Further, the HRC model predicts a response dependence on Δt, also formalized in Equation 20 where the stimulus autocorrelation term ΘII depends on Δt. To examine the temporal response properties, we chose the following set of values for Δt: 4 ms, 8 ms, 12 ms, 16 ms, and 20 ms for each spatial offset described in Figure 6B and C. To fit the response values, we used the temporal part of the correlator receptive field, Γcorr(ζ,η) (see Equations 19 and 20) with filters F(t) and G(t) parametrized by a log-normal function and an exponential decay function described earlier in the section, Correlator response to apparent motion stimuli. As a function of Δt, the excess firing rate of both H1 and V1 neurons rises to a peak value and then decays (Figure 6D, E). For the three curves, the initial rise is slightly steeper for V1 than for H1, which can be attributed to a difference in response gain between V1 and H1 and a stronger excitation from correlated signals separated by a smaller absolute distance (Δy = 1 ≈ 0.75° compared to Δx = 1 ≈ 1.3°). For both H1 and V1, the dominant responses correspond to the nearest-neighbor spatial correlations, that is, with (Δx, Δy) = (±1, 1) for H1 and with (Δx, Δy) = {(1, ±1),(0, 2)} for V1. As noted earlier, the shape of the response curves in Figure 6D and E reflects the temporal response profile predicted by the HRC model (Götz, 1972; Hausen & Egelhaaf, 1989), as described by Equation 19
Note that our experiments tested only a limited range of Δt values because the response at large Δt becomes weak and quite hard to measure. Figure 6 shows that the response rates peak at Δt = 12 ms and 8 ms for H1 and V1 respectively, equivalent to apparent speeds of |v⃗app| = 1.3° / 0.012s ≅ 110°/s and |v⃗app| = 1.5° / 0.008s ≅ 190°/s. This roughly corresponds to the velocities that generate peak rates for conventional moving patterns in many lab conditions. For larger Δt, the rate decreases as Δt increases; in other words, for large values of Δt the rate is an increasing function of |v⃗app|. The decrease in response for the smaller values of Δt is presumably due to the finite bandwidths of the various filters in the system. Based on measurements of the photoreceptor response under similar illumination conditions (not shown), photoreceptor filtering alone will cause a decrease in rate for values of Δt below 6 ms. Additional filtering operations may shift that criterion to the somewhat higher values of Δt we find here. 
In the preceding experiments, we tacitly assumed that light is emitted by a point-source and is received by a point-detector. In practice, however, the light input to the photoreceptor is a weighted function of light intensities over a small region of visual space. The weighting function is given by the PSF of the lens-photoreceptor system (Smakman et al., 1984). This spatial smearing will affect the wide-field motion response of H1 and V1 neurons by decreasing optical contrast, and introducing crosstalk of the optical signal in neighboring photoreceptors. 
The response of a wide field motion sensitive cell can be thought of as sum of contributions of correlator subsets with different spatial spans and temporal properties. Those contributions may have different weights depending on span and stimulus delay time, and those weights can be determined by presenting the stimulus at different space and time offsets. In fact, the weights would be directly proportional to the measured rate if the system did not optically blur the images. In the presence of optical blurring we need to apply a correction, and to extract the “true” contribution of the correlators to the motion response, we must first quantify the optical contribution to the motion response. The PSF is approximated as a Gaussian of σ = 0.51° (Land, 1997; Smakman et al., 1984), and the optical contribution from the neighboring locations to a correlator of given span is found by the overlap between the area of a pixel given by P(ζ) (which is a disk with diameter ≈0.30°) and the correlator's spatial receptive field. The latter is described by the difference between two convolved PSFs separated in space, expressed as the spatial part, {ΨSS(ζ + Δφ) − ΨSS(ζ − Δφ)}, of the correlator “receptive field” (see Equation 19 and also Figure 3, gray curve on the left). The optical contribution of a pixel pair at stimulus offset s to a correlator with span Δφi,j is then defined as  where ΨPP is the self-overlap of the pixel disk. If βi,j represents the actual unknown contribution of that set of correlators to the neural response, then the total response can be formulated as:  where indices (i,j) correspond to spatial locations in units of (Δφx, Δφy). In the analysis we limit the set of spans to {(i,j)} = {(1,1),(1,–1),(2,0),(3,1),(3,−1)} for H1 and {(i,j)} = {(−1,1),(1,1),(0,2),(−1,3),(1,3)} for V1 (see Figure 6). The offsets si,j used in the experiment correspond to the same raster points as given in these expressions, but now in units of (Δx, Δy). To find the βi,jt), we treat them as fitting parameters in a standard linear least squares minimization of χ2, defined as:    
By substituting in Equation 23 the excess firing rate of H1 to a set of stimuli with Δt = 12 ms, we obtain the following correlator contributions for the five different spans: β1,1 = 35%, β1,–1 = 34%, β2,0 = 16%, β3,1 = 7% and β3,−1 = 8%. Thus, the two nearest-neighbor correlators at (1,±1) together contribute about 70%, the next–nearest-neighbor correlator at (2,0) contributes about 15%, while the next–next–nearest-neighbor correlators at (3,±1) together contribute about 15%, to the overall motion response. When we plot the values of βi,jt) in Figure 6F and D, we find that, due to the correction, the true summed contribution from the two nearest-neighbor correlators rises above 60 spikes/s, which is higher than twice the measured average response of H1 (Figure 6F). The contributions from the second–, and the two third–nearest-neighbor correlators do not show significant deviation from the corresponding measured responses. Similarly, for V1, the correlator contributions are higher than the corresponding measured responses (Figure 6G, E). The response in both cases peaks at 8–12 ms and then decreases monotonically with Δt, characteristic of the model correlator's response. These results corroborate previous findings that the nearest-neighbor correlator provides the dominant input to motion response (Buchner, 1976), and shows that optical blurring due to finite PSF impacts the overall magnitude and quality of response to motion. 
Bilinear dependence on contrast
The HRC model postulates that response is proportional to the product of signals arriving from two spatially distinct locations, where one signal is delayed with respect to the other (Hassenstein & Reichardt, 1956). Previous studies that have reported a correlation type mechanism for motion detection, have noted a linear dependence of the response on the square of stimulus contrast, at least for low contrast values (Egelhaaf et al., 1989; Haag, Denk, & Borst, 2004). This behavior is a corollary of the more general prediction of bilinear behavior. Because our stimuli allow us to manipulate m1 and m2 independently, we can investigate that prediction more directly, and of course, a bilinear dependence of response on signal amplitudes would provide a more direct validation of a multiplicative nonlinearity. 
We independently varied the parameters m1 and m2 such that in one case the flicker energy changes but the motion energy remains constant at 0.2 (Figure 7A), and vice versa in the other case with flicker energy constant at 1.0 (Figure 7B). For each of these cases, the excess firing rate was obtained from two reference conditions: (1) null motion (Figure 6, blue color) and (2) pure flicker (Figure 7, orange color). The pure flicker condition was approximated by setting Δt = 200 ms, whereas null motion was generated by altering the sign of either Δt or Δx
Figure 7
 
Response of H1 changes with modulations in m1 and m2. (A) The flicker energy (solid line) is constant while the motion energy (dashed line) changes. (B) The motion energy (dashed line) is constant while the flicker energy (solid line) changes. The change in flicker and motion energies is linear in both cases. Response is recorded using two reference stimulus conditions: (1) motion in the null direction (blue dashed line; i.e., v, in the even segments), and (2) pure flicker (orange dashed line; i.e., v0, in the odd segments of trial). The dual color-coded y axes correspond to the two conditions. Response of H1 averaged over 50 trials to (C) stimulus conditions shown in A and (D) shown in B. The excess rate to pure flicker and half of excess rate to null motion are shown color coded orange and blue respectively. The mean rate is r and subscripts +, −, and 0 refer to preferred motion, null motion, and pure flicker conditions. The values of other stimulus parameters were: Δx = 2, Δt = 8 ms, and Tp = 6 ms.
Figure 7
 
Response of H1 changes with modulations in m1 and m2. (A) The flicker energy (solid line) is constant while the motion energy (dashed line) changes. (B) The motion energy (dashed line) is constant while the flicker energy (solid line) changes. The change in flicker and motion energies is linear in both cases. Response is recorded using two reference stimulus conditions: (1) motion in the null direction (blue dashed line; i.e., v, in the even segments), and (2) pure flicker (orange dashed line; i.e., v0, in the odd segments of trial). The dual color-coded y axes correspond to the two conditions. Response of H1 averaged over 50 trials to (C) stimulus conditions shown in A and (D) shown in B. The excess rate to pure flicker and half of excess rate to null motion are shown color coded orange and blue respectively. The mean rate is r and subscripts +, −, and 0 refer to preferred motion, null motion, and pure flicker conditions. The values of other stimulus parameters were: Δx = 2, Δt = 8 ms, and Tp = 6 ms.
We see that the excess firing rate rises approximately linearly with increase in motion energy (Figure 7C). This indicates that motion response varies bilinearly with the signal amplitude. However, with increasing flicker energy at constant motion energy, the firing rate shows a steady decrease (Figure 7D). This behavior deviates from the prediction of the HRC model that motion response should depend only on the motion term m1 · m2 and not on the flicker term Image not available + Image not available . We also observe a slight difference in response gain for the two reference conditions. A slightly steeper blue curve relative to the orange curve (Figure 7C) means that with increasing motion energy, suppression from null motion becomes stronger, whereas response to pure flicker remains largely unchanged. This causes a larger excess rate for the null motion condition. In contrast, when the motion energy remains constant at a low value of 0.2 (Figure 7D), the response is largely dominated by flicker, which causes an approximately equal change in the response to null motion and pure flicker, thus keeping the response gain equal for the two conditions.  
Response dependence on contrast polarity reversal
A change in the direction of perceived motion as a result of contrast polarity reversal has been previously demonstrated in psychophysical studies (Anstis, 1970; Anstis & Mather, 1985), as well as in Reichardt and Hassenstein's (1956) original work on the behavior of Chlorophanus. This method of generating negative motion is qualitatively different from the conventional method; in the latter, we change the sign of either Δx or Δt, shifting the locus of peak motion energy from quadrant I to II and from quadrant III to IV (Figure 1). By reversing the polarity of contrast, the locus does not change, but the sign of the motion energy flips from positive to negative (see Figure 1B and Adelson & Bergen, 1985). 
For the experiment, we changed the sign of either m1 or m2, but not both, in alternate segments of a trial (Figure 8A). This leads to a change in the sign of the motion energy m1 · m2 but of course does not affect the flicker energy Image not available + Image not available . H1 shows a clear reduction in firing rate when m1 · m2 < 0 compared to the pure flicker response (last segment of Figure 8B). This reduction is of the same order, though somewhat smaller in magnitude than the excitatory response to m1 · m2 > 0 (Figure 8B). Note that this behavior is in qualitative agreement with the prediction of bilinearity. The response also shows a linear dependence on the stimulus motion energy, where the slope for positive motion energy is steeper than for negative motion energy (Figure 8C, filled circles). This indicates that the visual system responds to stimuli with positive and negative motion energies with different flicker normalizations. To differentiate the effect of motion reversal caused by contrast polarity reversal from reversal in the direction of apparent velocity, we changed the sign of Δt in the stimuli keeping the sign of m1 · m2 unchanged. This leads to a slight reduction in slope of the response curves for both positive (Δt > 0) and negative (Δt < 0) apparent velocities (Figure 8C, open circles), relative to the curves with contrast reversal (Figure 8C, filled circles). The response of V1 to contrast polarity reversal showed characteristics that were similar to those obtained from H1 (results not shown).  
Figure 8
 
Wide-field response to reversal of contrast. (A) The stimulus condition is set to reverse the sign of m1 and keep the sign of m2 same, in every even segment of a trial. The motion energy decreases linearly (dashed line) while the flicker energy remains constant at 1 (solid line). In the last segment, Δt is set to 200 ms (m1 = 0.5, m2 = 0.5), to generate an approximately pure flicker condition. (B) The PSTH shows the trial averaged response of H1 to change in stimulus conditions described in A. (C) Steady state response is computed by discarding the initial transient (approx. 200 ms). The filled blue circles and filled orange circles represent response for odd and even segments of trial respectively. The sign of Δx and Δt remains unchanged here. The open blue circles and open orange circles represent responses to negative and positive apparent velocity (vapp), respectively. Here, the signs of m1 and m2 remain unchanged (shown by the top x axis). The gray lines represent linear fit to the data, obtained separately for the two contrast conditions and vapp conditions. The error bars represent SD of firing rate across 50 trials.
Figure 8
 
Wide-field response to reversal of contrast. (A) The stimulus condition is set to reverse the sign of m1 and keep the sign of m2 same, in every even segment of a trial. The motion energy decreases linearly (dashed line) while the flicker energy remains constant at 1 (solid line). In the last segment, Δt is set to 200 ms (m1 = 0.5, m2 = 0.5), to generate an approximately pure flicker condition. (B) The PSTH shows the trial averaged response of H1 to change in stimulus conditions described in A. (C) Steady state response is computed by discarding the initial transient (approx. 200 ms). The filled blue circles and filled orange circles represent response for odd and even segments of trial respectively. The sign of Δx and Δt remains unchanged here. The open blue circles and open orange circles represent responses to negative and positive apparent velocity (vapp), respectively. Here, the signs of m1 and m2 remain unchanged (shown by the top x axis). The gray lines represent linear fit to the data, obtained separately for the two contrast conditions and vapp conditions. The error bars represent SD of firing rate across 50 trials.
Motion response depends on flicker strength
Wide-field flicker is known to influence both neuronal (Borst & Egelhaaf, 1987; Nordstrom, Moyer de Miguel, & O'Carroll, 2011) and behavioral (Tuthill, Nern, Rubin, & Reiser, 2014) responses to motion stimuli. Flicker can also reduce contrast gain, although by a magnitude that is smaller than that induced by motion (Harris, O'Carroll, & Laughlin, 2000). Our stimulus allows us to manipulate motion and flicker energies independently, within certain bounds, and here we examine the effect of both. 
We solve a quadratic equation involving the terms m1 · m2 and Image not available + Image not available with constraints (see Methods), to determine a set of real positive values for m1 and m2. The allowed range of flicker and motion energies ([0, 2] and [-1, 1] respectively), covers a large part of the motion-flicker space (Figure 9A, gray region). The absolute bound for allowed and nonallowed values is set by the constraint |m1 + m2| ≤ 2 (Figure 9A). The response curves for H1 show a notable decrease with increasing flicker energy, and a clear increase with an increase in motion energy (Figure 9B). The curves in Figure 9B demonstrate both that the response increases with increasing motion energy, and decreases with increasing flicker energy. To describe the response dependence on m1 · m2, we define a radius r and an angle θ to represent the locus of m1 and m2 in the polar coordinate plane (Figure 9C).     
Figure 9
 
Response depends on motion and flicker components in stimuli. (A) The plot represents the 2-D space of flicker and motion energy. The range for flicker is set to [0, 2] and for motion is set to [0, 1]. The gray line at the top right hand corner represents the boundary separating the regions where values of m1 and m2 are allowed (i.e., m1 + m2 < 2) and not allowed (i.e., m1 + m2 > 2). The gray dashed line meets the flicker axis at value 4 and motion axis at value 2. The gray triangular shaded region represents the experimentally accessible region of motion-flicker space (i.e., where m1 and m2 have real values). For each value of m1 × m2, a set of linearly increasing values of Image not available + Image not available is calculated. The color codes represent the 4 levels of motion energy within the accessible space. (B) Excess firing rate is plotted as a function of the flicker energy, for each value of motion energy, shown color coded. The excess rate is obtained by subtracting the null motion response from the preferred motion response (50 trials, trial time = 2.5 s each for preferred and null motion). The error bars represent SD of the mean firing rate. Other stimulus parameters were set at: Δx = 2, Δt = 8 ms, and Tp = 6 ms. (C) Graphical representation of the product term (area in gray) and the quadrature term (radius in blue) using an m1, m2 coordinate system. The angle (θ) relates m1 and m2. (D) Color coded response curves, are plotted as a function of the angle θ. The solid gray curve with black markers represents scaled sin(2θ) / 2 or, the product m1 × m2 in terms of the angle θ.
Figure 9
 
Response depends on motion and flicker components in stimuli. (A) The plot represents the 2-D space of flicker and motion energy. The range for flicker is set to [0, 2] and for motion is set to [0, 1]. The gray line at the top right hand corner represents the boundary separating the regions where values of m1 and m2 are allowed (i.e., m1 + m2 < 2) and not allowed (i.e., m1 + m2 > 2). The gray dashed line meets the flicker axis at value 4 and motion axis at value 2. The gray triangular shaded region represents the experimentally accessible region of motion-flicker space (i.e., where m1 and m2 have real values). For each value of m1 × m2, a set of linearly increasing values of Image not available + Image not available is calculated. The color codes represent the 4 levels of motion energy within the accessible space. (B) Excess firing rate is plotted as a function of the flicker energy, for each value of motion energy, shown color coded. The excess rate is obtained by subtracting the null motion response from the preferred motion response (50 trials, trial time = 2.5 s each for preferred and null motion). The error bars represent SD of the mean firing rate. Other stimulus parameters were set at: Δx = 2, Δt = 8 ms, and Tp = 6 ms. (C) Graphical representation of the product term (area in gray) and the quadrature term (radius in blue) using an m1, m2 coordinate system. The angle (θ) relates m1 and m2. (D) Color coded response curves, are plotted as a function of the angle θ. The solid gray curve with black markers represents scaled sin(2θ) / 2 or, the product m1 × m2 in terms of the angle θ.
The motion and flicker terms respectively are then given by:     
Plotting the response and the product term as function of θ = tan−1(m2 / m1), we observe that the response curves virtually overlap with each other and with the scaled sin(2θ) / 2 curve (Figure 9D). Because the flicker energy is different for different points on a particular curve, the overlap of the different response curves implies that the response scales inversely with the flicker energy (see also Figure 9B). The observation that the response curves follow the sin(2θ) / 2 curve shows that the response is also proportional to the motion energy. Part of this result is consistent with the predictions of the HRC model, in that the neural response varies linearly with the product term m1 · m2. But the other part suggests that the neuron is sensitive to the background flicker, based on which it adjusts its motion response. Computationally, this implies that the fly visual system uses a flicker-normalized correlation for movement detection at least across the set of conditions explored in these experiments. Conceptually, this can be understood as a response dependence not on pure motion energy, but on the ratio of motion to flicker. 
Discussion
Conventional visual motion stimuli typically have continuous autocorrelation functions. By construction, the apparent motion stimulus used in our experiments is discontinuous in both space and time. It shares that property with phi-motion stimuli, but whereas phi-motion stimuli are bilocal, our stimulus can cover a large area. Conventional bilocal phi-motion stimuli are typically based on functions with long correlations, such as square wave or sinusoidal fluctuations. Tiling space with such stimuli therefore introduces intensity correlations all across the visual field. Because our stimulus is based on random functions with nonzero correlations at specific offsets only (Equation 2), we can tile the visual field without introducing additional correlations. This makes these stimuli very suitable for studying wide field motion sensitive neurons and the computational structure that gives rise to their response. 
Motion detection, in particular in flies, has generally been identified with a correlation operation described by the HRC model (Hassenstein & Reichardt, 1956). The central concept of this model is the multiplication of delayed output of one channel with the nondelayed output of the other channel. There is a large body of work describing various computational aspects of the motion-processing pathway, in the fly as well as in other organisms. The lack of spatiotemporal continuity allows us interrogate some of the computational properties in a more direct way than is possible with conventional stimuli: We can target the spatial offsets and temporal delays of stimulus correlations, which allows us to probe directly the space and time aspects of the computation. The amplitudes of the contrast signals for the two arms of the correlator can be independently set, making it possible to test the bilinear character of the computation. Finally, we can manipulate the variance of the flicker. The price for these additional flexibilities is that the stimulus cannot be interpreted as representing natural conditions, so some care should be taken in interpreting the experimental results. 
The HRC model does not impose any restrictions on the spatial separation between correlated signals. However, for natural visual scenes, one might expect decreasing levels of correlation for larger separations due, for example, to inherent scene variability and motion orthogonal to the correlator axis. That would make it counterproductive to compute motion over too large a correlator span. The contributions from correlators of different spans have been quantified, though necessarily indirectly, using moving sinusoidal bar patterns (Buchner, 1976). Using correlated noise stimuli with delta correlations at specified space–time offsets, we found that neural response to motion is dominated by the nearest-neighbor interactions with perhaps some contribution from next-nearest neighbors (Figure 6). Interactions separated by more than three receptor units evoke negligible motion response in the conditions of our experiment. The weights of these relative contributions may very well depend on stimulus brightness (Heisenberg & Buchner, 1977), and it has been suggested that the system could increase spatial acuity at the cost of spatial sensitivity (Pick & Buchner, 1979) at higher light levels. Similar mechanisms also exist in the vertebrate retina, where signals converge on to bipolar cells from relatively large receptive fields at low light levels and smaller fields at high light levels (Sterling, 2004). These questions can be addressed in more detail by applying apparent motion stimuli at different light levels. 
The linear dependence of response on the square of contrast is a hallmark of the HRC model (Reichardt, 1961). This behavior can be seen as a corollary of the more general property of bilinearity, predicted by the HRC model. Our data clearly show that, over the range tested, the computation in the fly indeed has a bilinear character (Figure 7C). However, as implemented in the fly, this is a relative phenomenon in the sense that the response is normalized by the contrast flicker variance (Figure 7D, Figure 9). It is important to note that this is valid for the limited range of contrasts tested here: Clearly, this normalization must break down in the limit of very low levels of contrast variance. Currently we are testing this for a wider range of stimulus conditions. 
The observation of flicker normalization gives rise to two different sorts of questions, one about implementation and the other about the possible functional utility. From our experiments we can say very little about the implementation. Our data are consistent with the possibility that signals are scaled to flicker standard deviation very early on in the visual pathway, before the nonlinear element of the correlator, since the information is available locally in the visual field. But it is also conceivable that flicker information is relayed via a pathway parallel to the motion pathway (Schnell et al., 2012), and that scaling to the variance takes place after the nonlinear interaction. Our data do not allow us to draw conclusions about whether the variance measurement is pooled over a spatially extended domain. This question may be addressed in principle by experiments with inhomogeneous spatial distributions of flicker variance. There is evidence that the scaling mechanism is not instantaneous, but takes some time (on the order of a few hundred milliseconds) to adapt (de Ruyter van Steveninck & Laughlin, 1996; additional preliminary observations). As to the possible functional utility, it is well known that correlator output depends monotonically on stimulus velocity over a certain range, but depends on a host of other stimulus variables as well. The scaling normalization observed here removes at least one confounding variable from the correlator response, and results in a nearly contrast independent motion estimator (Potters & Bialek, 1994). 
Illusory motion stimuli have been a powerful tool for probing neural computations underlying motion detection. The response to a version of reverse-phi motion was in fact one of the key observations in the development of the HRC model (Reichardt, 1961). More recently, the reverse-phi illusion (Anstis, 1970), which involves contrast reversal in moving images, has provided clues to the type of nonlinear interactions between ON and OFF channels in the Drosophila visual system (Clark, Bursztyn, Horowitz, Schnitzer, & Clandinin, 2011). By construction, our apparent motion stimuli allow independent contrast reversals of the signals arriving at the detectors. A change in response from excitatory to inhibitory when the sign of m1 or m2 changes from positive to negative (Figure 8C), indicates that the neural system identifies this as a reversal in the direction of motion. There are slight differences in response gain to positive and negative motion energy. These may be due to the fact that the mechanisms that “measure” flicker and motion energy have finite integration times, which implies that they must average the energy around the peaks of the stimulus autocorrelation function over finite areas in the space–time plane (Figure 1A, B). It is conceivable that these areas overlap somewhat and because of the resulting “leaking” of energy, may lead to different values for the weighting of the normal and reversed stimulus and flicker energies. If the sign of Δx or Δt is changed, but contrast is not reversed, only the direction of perceived velocity changes. In this case, we also observe a change in gain for H1, which may be explained by a differential strength of excitation and suppression to motion observed in earlier work (Borst & Egelhaaf, 1990). 
Several studies have explored possibilities of modified HRC models that better predict naturalistic image velocity (Dror, O'Carroll, & Laughlin, 2001; Straw, Rainsford, & O'Carroll, 2008). Natural stimuli may be subject to large and fast dynamic fluctuations in their statistical properties, and many of these studies have proposed that adaptive filters are part of the visual processing pathway, including perhaps the correlator itself (see also Maddess & Laughlin, 1985; de Ruyter van Steveninck et al., 1986). In future work we will use generalizations of the stimuli described here, for example incorporating specific statistical nonstationarities in space and time, to study some of those adaptive strategies. 
Acknowledgments
We thank Dr. Shiva R. Sinha for help with developing the stimulus-generating interface and for providing valuable inputs on the project. 
Commercial relationships: none. 
Corresponding author: Rob de Ruyter van Steveninck. 
References
Adelson E. H., Bergen J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 2, 284–299.
Anstis S. M. (1970). Phi movement as a subtraction process. Vision Research, 10, 1411–1430.
Anstis S. M., Mather G. (1985). Effects of luminance and contrast on direction of ambiguous apparent motion. Perception, 14, 167–179.
Barlow H. B., Levick W. R. (1965). The mechanism of directionally selective units in rabbit's retina. Journal of Physiology, 178, 477–504.
Borst A., Egelhaaf M. (1987). Temporal-modulation of luminance adapts time constant of fly movement detectors. Biological Cybernetics, 56, 209–215.
Borst A., Egelhaaf M. (1989). Principles of visual motion detection. Trends in Neurosciences, 12, 297–306.
Borst A., Egelhaaf M. (1990). Direction selectivity of blowfly motion-sensitive neurons is computed in a two-stage process. Proceedings of the National Academy of Sciences, USA, 87, 9363–9367.
Bracewell R. N. (2000). The Fourier transform and its applications (3rd ed.). Boston, MA: McGraw Hill.
Braddick O. (1974). A short-range process in apparent motion. Vision Research, 14, 519–527.
Braitenberg V. (1967). Patterns of projection in the visual system of the fly. I. Retina-lamina projections. Experimental Brain Research, 3, 271–298.
Brenner N., Strong S. P., Koberle R., Bialek W., de Ruyter van Steveninck R. R. (2000). Synergy in a neural code. Neural Computation, 12, 1531–1552.
Britten K. H., Shadlen M. N., Newsome W. T., Movshon J. A. (1992). The analysis of visual motion: A comparison of neuronal and psychophysical performance. Journal of Neurosciences, 12, 4745–4765.
Buchner E. (1976). Elementary movement detectors in an insect visual system. Biological Cybernetics, 24, 85–101.
Clark D. A., Bursztyn L., Horowitz M. A., Schnitzer M. J., Clandinin T. R. (2011). Defining the computational structure of the motion detector in Drosophila. Neuron, 70, 1165–1177.
Clifford C., Ibbotson M. (2002). Fundamental mechanisms of visual motion detection: Models, cells, and functions. Progress in Neurobiology, 68, 409–437.
Coxeter H. S. M. (1969). Introduction to geometry (2nd ed.). New York: Wiley.
de Ruyter van Steveninck R. R., Bialek W., Potters M., Carlson R. H. (1994, October). Statistical adaptation and optimal estimation in movement computation by the blowfly visual system, Paper presented at Systems, Man, and Cybernetics. IEEE International Conference on Humans, Information, and Technology, San Antonio, TX.
de Ruyter van Steveninck R. R., Laughlin S. B. (1996). Light adaptation and reliability in blowfly photoreceptors. International Journal of Neural Systems, 7, 437–444.
de Ruyter van Steveninck R. R., Zaagman W., Mastebroek H. (1986). Adaptation of transient responses of a movement-sensitive neuron in the visual system of the blowfly Calliphora erythrocephala. Biological Cybernetics, 54, 223–236.
Dror R. O., O'Carroll D. C., Laughlin S. B. (2001). Accuracy of velocity estimation by Reichardt correlators. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 18, 241–252.
Egelhaaf M., Borst A. (1989). Transient and steady-state response properties of movement detectors. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 6, 116–127.
Egelhaaf M., Borst A., Reichardt W. (1989). Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly's nervous system. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 6, 1070–1087.
Fairhall A. L., Lewen G. D., Bialek W., de Ruyter Van Steveninck R. R. (2001). Efficiency and ambiguity in an adaptive neural code. Nature, 412, 787–792.
Franceschini N. (1975). Sampling of the visual environment by the compound eye of the fly: Fundamentals and applications. In Snyder A. Menzel R. (Eds.) Photoreceptor optics (pp. 98–125). Berlin, Germany: Springer-Verlag.
Götz, K. G. (1964). Optomotory investigation of the visual system of some eye mutations of the Drosophila fruit-fly. Kybernetik, 2 (2), 77–91.
Götz K. G. (1972). Principles of optomotor reactions in insects. Bibliotheka Ophthalmologica, 82, 251–259.
Haag J., Denk W., Borst A. (2004). Fly motion vision is based on Reichardt detectors regardless of the signal-to-noise ratio. Proceedings of the National Academy of Sciences, USA, 101, 16333–16338.
Harris R. A., O'Carroll D. C., Laughlin S. B. (1999). Adaptation and the temporal delay filter of fly motion detectors. Vision Research, 39, 2603–2613.
Harris R. A., O'Carroll D. C., Laughlin S. B. (2000). Contrast gain reduction in fly motion adaptation. Neuron, 28 (2), 595–606.
Hassenstein B., Reichardt W. (1956). Systemtheoretische Analyse Der Zeit, Reihenfolgen Und Vorzeichenauswertung Bei Der Bewegungsperzeption Des Ruesselkaefers Chlorophanus [Translation: System-theoretical analysis of the evaluation of time, sequence and sign in movement perception by the weevil Chlorophanus]. Zeitschrift Fuer Naturforschung Part B-Chemie Biochemie Biophysik Biologie Und Verwandten Gebiete, 11 (9–10), 513–524.
Hausen K. (1976). Functional characterization and anatomical identification of motion sensitive neurons in lobula plate of blowfly Calliphora erythrocephala. Zeitschrift Fur Naturforschung C-a Journal of Biosciences, 31 (9-10), 629–633.
Hausen K., Egelhaaf M. (1989). Neural mechanisms of visual course control in insects. In Stavenga D. Hardie R. (Eds.) Facets of vision (pp. 391–424): Berlin, Germany: Springer.
Heisenberg, M., Buchner E. (1977). Role of retinula cell types in visual behavior of Drosophila melanogaster. Journal of Comparative Physiology, 117, 127–162.
Howard J., Dubs A., Payne R. (1984). The dynamics of phototransduction in insects—A comparative study. Journal of Comparative Physiology, 154, 707–718.
Juusola M., Kouvalainen E., Jarvilehto M., Weckstrom M. (1994). Contrast gain, signal-to-noise ratio, and linearity in light-adapted blowfly photoreceptors. Journal of General Physiology, 104, 593–621.
Krapp H. G., Hengstenberg R. (1997). A fast stimulus procedure to determine local receptive field properties of motion-sensitive visual interneurons. Vision Research, 37, 225–234.
Land M. F. (1997). Visual acuity in insects. Annual Review of Entomology, 42, 147–177.
Maddess T., Laughlin S. B. (1985). Adaptation of the motion-sensitive neuron H1 is generated locally and governed by contrast frequency. Proceedings of the Royal Society Series B: Biological Sciences, 225, 251–275.
Morgan M. J., Ward R. (1980). Conditions for motion flow in dynamic visual noise. Vision Research, 20, 431–435.
Nordstrom K., Moyer de Miguel I., O'Carroll D. C. (2011). Rapid contrast gain reduction following motion adaptation. Journal of Experimental Biology, 214, 4000–4009.
Nyquist H. (1928). Certain topics in telegraph transmission theory. Transactions of the American Institute of Electrical Engineers, 47, 617–644.
O'Carroll D. C., Barnett P. D., Nordstrom K. (2011). Local and global responses of insect motion detectors to the spatial structure of natural scenes. Journal of Vision, 11 (14): 20, 1–17, doi:10.1167/11.14.20. [PubMed] [Article]
Payne R., Howard J. (1981). Response of an insect photoreceptor: A simple log-normal model. Nature, 290, 415–416.
Pick B., Buchner E. (1979). Visual movement detection under light- and dark-adaptation in the fly, Musca domestica. Journal of Comparative Physiology, 134, 45–54.
Poggio T., Reichardt W. (1973). Considerations on models of movement detection. Kybernetik, 13 (4), 223–227.
Potters M., Bialek W. (1994). Statistical mechanics and visual signal processing. Journal De Physique I, 4 (11), 1755–1775.
Reichardt W. (1961). Autocorrelation: A principle for the evaluation of sensory information by the central nervous system. In Rosenblith W. A. (Ed.) Sensory communication (pp. 303–318). New York: Wiley.
Schnell, B., Raghu S. V., Nern A., Borst A. (2012). Columnar cells necessary for motion responses of wide-field visual interneurons in Drosophila. Journal of Comparative Physiology A, 198, 389–395.
Shannon C. E. (1949). Communication in the presence of noise. Proceedings of the Institute of Radio Engineers, 37 (1), 10–21.
Smakman J. G. J., van Hateren J. H., Stavenga D. G. (1984). Angular sensitivity of blowfly photoreceptors: Intracellular measurements and wave-optical predictions. Journal of Comparative Physiology A, 155, 239–247.
Snyder A. W., Stavenga D. G., Laughlin S. B. (1977). Spatial information capacity of compound eyes. Journal of Comparative Physiology, 116, 183–207.
Srinivasan M. (1983). The impulse response of a movement-detecting neuron and its interpretation. Vision Research, 23, 659–663.
Stavenga D. G. (2003). Angular and spectral sensitivity of fly photoreceptors. I. Integrated facet lens and rhabdomere optics. Journal of Comparative Physiology A, 189 (1), 1–17.
Sterling P. (2004). How retinal circuits optimize the transfer of visual information. In Chalupa L. M. Werner J. S. (Eds.) The visual neuroscience (Vol. 1. pp. 234–259). Cambridge, MA: MIT Press.
Straw, A. D., Rainsford T., O'Carroll D. C. (2008). Contrast sensitivity of insect motion detectors to natural images. Journal of Vision, 8 (3): 32, 1–39, doi:10.1167/8.3.32. [PubMed] [Article]
Tuthill J. C., Chiappe M. E., Reiser M. B. (2011). Neural correlates of illusory motion perception in Drosophila. Proceedings of the National Academy of Sciences, USA, 108, 9685–9690.
Tuthill J. C., Nern A., Rubin G. M., Reiser M. B. (2014). Wide-field feedback neurons dynamically tune early visual processing. Neuron, 82, 887–895.
van de Grind W. A., Koenderink J. J., van Doorn A. J. (1986). The distribution of human motion detector properties in the monocular visual field. Vision Research, 26, 797–810.
van Doorn A. J., Koenderink J. J. (1982a). Spatial properties of the visual detectability of moving spatial white noise. Experimental Brain Research, 45, 189–195.
van Doorn A. J., Koenderink J. J. (1982b). Temporal properties of the visual detectability of moving spatial white noise. Experimental Brain Research, 45, 179–188.
van Doorn A. J., Koenderink J. J., van de Grind W. A. (1985). Perception of movement and correlation in stroboscopically presented noise patterns. Perception, 14, 209–224.
van Santen J. P., Sperling G. (1985). Elaborated Reichardt detectors. Journal of the Optical Society of America A, 2, 300–321.
Weckstrom M. (1989). Light and dark adaptation in fly photoreceptors: Duration and time integral of the impulse response. Vision Research, 29, 1309–1317.
Westheimer G. (1993). Phosphor persistence in oscilloscopic displays. Vision Research, 33, 2337–2338.
Wolf W., Deubel H. (1997). P31 phosphor persistence at photopic mean luminance level. Spatial Vision, 10, 323–333.
Zele A. J., Vingrys A. J. (2005). Cathode-ray-tube monitor artefacts in neurophysiology. Journal of Neuroscience Methods, 141, 1–7.
Figure 1
 
Space-time correlation function of the apparent motion stimuli used in this study. (A) Idealized condition, with delta correlation in both space and time, plotted for one space dimension, namely X. The vertical red lines located at (Δt, Δx) and (−Δt, −Δx) represent nonzero motion energy and the vertical blue line located at (0, 0) represents nonzero flicker energy. The flicker energy is twice the motion energy for the condition m1 = m2. The four quadrants of the energy space are denoted by I, II, III, and IV. (B) Motion energy is negative for the condition m1 = −m2. (C, D) In practice, patterns are displayed for a finite time Tp, which in our experiments is an integer multiple of the 2 ms frame time. For most experiments a longer value of Tp is used, typically a few frame intervals, because this is matched better to the integration time of fly photoreceptors. The example in (C) shows the triangular profile of the autocorrelation function Λ(t) (see Equation 5) for Tp = 6 ms. Our stimuli avoid the situation in (D) where Tp > Δt, because that causes some of the motion energy to leak into the adjacent quadrants. The temporal offset is given by Δt. The space, time, and stimulus energy are represented by the x, y, and z axes, respectively.
Figure 1
 
Space-time correlation function of the apparent motion stimuli used in this study. (A) Idealized condition, with delta correlation in both space and time, plotted for one space dimension, namely X. The vertical red lines located at (Δt, Δx) and (−Δt, −Δx) represent nonzero motion energy and the vertical blue line located at (0, 0) represents nonzero flicker energy. The flicker energy is twice the motion energy for the condition m1 = m2. The four quadrants of the energy space are denoted by I, II, III, and IV. (B) Motion energy is negative for the condition m1 = −m2. (C, D) In practice, patterns are displayed for a finite time Tp, which in our experiments is an integer multiple of the 2 ms frame time. For most experiments a longer value of Tp is used, typically a few frame intervals, because this is matched better to the integration time of fly photoreceptors. The example in (C) shows the triangular profile of the autocorrelation function Λ(t) (see Equation 5) for Tp = 6 ms. Our stimuli avoid the situation in (D) where Tp > Δt, because that causes some of the motion energy to leak into the adjacent quadrants. The temporal offset is given by Δt. The space, time, and stimulus energy are represented by the x, y, and z axes, respectively.
Figure 2
 
Schematics of a 1-D correlator model. (A) The elementary correlator consists of two mirror-symmetrical subunits with span equal to integer number of Δφ ≈ 1.5°. The light input is represented by I(x,t) and the Gaussian PSF (σ = 0.5°) of the photoreceptors at locations 1 and 2 are represented by S1 and S2, respectively. The other components of the correlator circuit are, F(t): lognormal temporal prefilter and G(t): first-order exponential delay filter. The multiplicative nonlinearity, spatial summation, and correlator output are denoted by x, ∑, and R, respectively. (B) An array of correlators with different spans. Unit span is equal to Δφ (see also Poggio & Reichardt, 1973).
Figure 2
 
Schematics of a 1-D correlator model. (A) The elementary correlator consists of two mirror-symmetrical subunits with span equal to integer number of Δφ ≈ 1.5°. The light input is represented by I(x,t) and the Gaussian PSF (σ = 0.5°) of the photoreceptors at locations 1 and 2 are represented by S1 and S2, respectively. The other components of the correlator circuit are, F(t): lognormal temporal prefilter and G(t): first-order exponential delay filter. The multiplicative nonlinearity, spatial summation, and correlator output are denoted by x, ∑, and R, respectively. (B) An array of correlators with different spans. Unit span is equal to Δφ (see also Poggio & Reichardt, 1973).
Figure 3
 
Spatiotemporal receptive field of the model correlator. The 3-D profile shows the receptive field of a 2 unit model correlator as a function of space (η) and time (ζ) for (A) time constant τ = 10 ms, and for (B) time constant τ = 300 ms of the delay filter G(t) (see Equation 19). For this plot, the photoreceptor temporal filter F(t) is replaced by a log-normal function with parameters K = 1, tp = 15 ms, s = 0.23 (Payne & Howard, 1981), and the delay filter G(t) is replaced by a first order exponential. The solid gray curve to the left and to the right represent the antisymmetric spatial and antisymmetric temporal part of the receptive field. The z axis represents the weighting factor. Excitatory response is shown in red (quadrants I and III) and suppressive response is shown in blue (quadrants II and IV).
Figure 3
 
Spatiotemporal receptive field of the model correlator. The 3-D profile shows the receptive field of a 2 unit model correlator as a function of space (η) and time (ζ) for (A) time constant τ = 10 ms, and for (B) time constant τ = 300 ms of the delay filter G(t) (see Equation 19). For this plot, the photoreceptor temporal filter F(t) is replaced by a log-normal function with parameters K = 1, tp = 15 ms, s = 0.23 (Payne & Howard, 1981), and the delay filter G(t) is replaced by a first order exponential. The solid gray curve to the left and to the right represent the antisymmetric spatial and antisymmetric temporal part of the receptive field. The z axis represents the weighting factor. Excitatory response is shown in red (quadrants I and III) and suppressive response is shown in blue (quadrants II and IV).
Figure 4
 
Spatial Nyquist frequency test. Bar patterns of three spatial wavelengths, λs = 1.5·Δx (6 lines), λs = 2·Δx (8 lines), and λs = 3·Δx (12 lines), at contrast 1.0 were presented. The horizontal projection of the interommatidial separation is Δφx. The responses to motion in the preferred and null directions are shown by hatched bars and solid gray bars, respectively. The Nyquist response can be seen for the case with λs = 1.5·Δx. Velocity was maintained at 6°/s for all spatial wavelengths. Error bars represent SD of the firing rate obtained from 50 trials.
Figure 4
 
Spatial Nyquist frequency test. Bar patterns of three spatial wavelengths, λs = 1.5·Δx (6 lines), λs = 2·Δx (8 lines), and λs = 3·Δx (12 lines), at contrast 1.0 were presented. The horizontal projection of the interommatidial separation is Δφx. The responses to motion in the preferred and null directions are shown by hatched bars and solid gray bars, respectively. The Nyquist response can be seen for the case with λs = 1.5·Δx. Velocity was maintained at 6°/s for all spatial wavelengths. Error bars represent SD of the firing rate obtained from 50 trials.
Figure 5
 
Response dependence on absolute location of pixel grid. The plot shows the response mean and the response variation around mean, of H1, for four different directions grid shift: along −30°, +30°, 90° (y axis), and 0° (x axis). The full grid consists of 827 pixels. The original and shifted grid locations are exemplified by the hexagonal pattern of black and gray dots respectively, in the bottom panel. Shifts along the x axis are made in steps of (Δx ≈ 1.3°)/4 (blue dotted line), while shifts along the other three directions are made in steps of (Δr ≈ 1.5°)/4 (red dotted line). Four consecutive shifts are made to cover one period (i.e., Δx along x axis and Δr along other directions). The color codes represent the six directions of apparent motion (see legend). The error bars represent SD of mean firing rate obtained from four grid locations, corresponding to each direction of grid shift. The dotted lines represent the response averaged over the four different directions of grid shift.
Figure 5
 
Response dependence on absolute location of pixel grid. The plot shows the response mean and the response variation around mean, of H1, for four different directions grid shift: along −30°, +30°, 90° (y axis), and 0° (x axis). The full grid consists of 827 pixels. The original and shifted grid locations are exemplified by the hexagonal pattern of black and gray dots respectively, in the bottom panel. Shifts along the x axis are made in steps of (Δx ≈ 1.3°)/4 (blue dotted line), while shifts along the other three directions are made in steps of (Δr ≈ 1.5°)/4 (red dotted line). Four consecutive shifts are made to cover one period (i.e., Δx along x axis and Δr along other directions). The color codes represent the six directions of apparent motion (see legend). The error bars represent SD of mean firing rate obtained from four grid locations, corresponding to each direction of grid shift. The dotted lines represent the response averaged over the four different directions of grid shift.
Figure 6
 
Spatial sensitivity profile and response tuning to apparent velocity. (A) The gray dots in the schematic represent the location of spatial correlations in units of the fly's sampling raster. Horizontal and vertical projections of Δφ ≈ 1.5° are Δφx and Δφy (Equations 7, 8). Angles are measured with respect to the x axis, where x and y axes define the lab coordinate system. (B) Excess firing rate of H1 is shown for correlations between the central pixel (color-coded black), and pixels located at ±30° (1, ±1), at 0° (2, 0) and at ±15° (3, ±1). (C) Excess firing rate of V1 is shown for correlations between the central pixel, and the pixels located at ±60° (±1, 1), at 0° (0, 2) and at ±30° (±1, 3). Here, x and y axes are in units of pixel separation on the screen: Δx, Δy. The gray vertical bar shows the range of response. The SD of the firing rate is <10% of the mean rate. Contrast is set to maximum and Δt is set to 8 ms. Response of (D) H1 for spatial offsets (Δx, Δy) = (1, ±1), (2,0), (3, ±1), and (E) V1 for spatial offsets (Δx, Δy) = (±1, 1), (0, 2), (±1, 3), as a function of the temporal offset Δt. The green and purple circles represent the response summed over the two locations: Δy = ±1 for odd column number for H1, and Δx = ±1 for even row number for V1. The orange circles represent the response obtained from the locations: Δy = 0 for even column number for H1, and Δx = 0 for even row number for V1. Error bars represent SD of the mean firing rate calculated over 50 trials. The solid lines represent nonlinear least-square fits using the temporal receptive field of the model correlator (Equation 19) with parametrized F(t) and G(t). The parameter values that minimized the least square error are chosen for the fit. The correlator contributions to the wide-field response of (F) H1 and (G) V1, are shown as a function of Δt. Similar to D and E, the green and purple curves represent the summed contributions from the two locations lying on the same vertical line (Δy = ±1) for H1 and on the same horizontal line (Δx = ±1) for V1. The indices of β correspond to the locations described in D and E. The frame persistence, Tp, was set to 6 ms.
Figure 6
 
Spatial sensitivity profile and response tuning to apparent velocity. (A) The gray dots in the schematic represent the location of spatial correlations in units of the fly's sampling raster. Horizontal and vertical projections of Δφ ≈ 1.5° are Δφx and Δφy (Equations 7, 8). Angles are measured with respect to the x axis, where x and y axes define the lab coordinate system. (B) Excess firing rate of H1 is shown for correlations between the central pixel (color-coded black), and pixels located at ±30° (1, ±1), at 0° (2, 0) and at ±15° (3, ±1). (C) Excess firing rate of V1 is shown for correlations between the central pixel, and the pixels located at ±60° (±1, 1), at 0° (0, 2) and at ±30° (±1, 3). Here, x and y axes are in units of pixel separation on the screen: Δx, Δy. The gray vertical bar shows the range of response. The SD of the firing rate is <10% of the mean rate. Contrast is set to maximum and Δt is set to 8 ms. Response of (D) H1 for spatial offsets (Δx, Δy) = (1, ±1), (2,0), (3, ±1), and (E) V1 for spatial offsets (Δx, Δy) = (±1, 1), (0, 2), (±1, 3), as a function of the temporal offset Δt. The green and purple circles represent the response summed over the two locations: Δy = ±1 for odd column number for H1, and Δx = ±1 for even row number for V1. The orange circles represent the response obtained from the locations: Δy = 0 for even column number for H1, and Δx = 0 for even row number for V1. Error bars represent SD of the mean firing rate calculated over 50 trials. The solid lines represent nonlinear least-square fits using the temporal receptive field of the model correlator (Equation 19) with parametrized F(t) and G(t). The parameter values that minimized the least square error are chosen for the fit. The correlator contributions to the wide-field response of (F) H1 and (G) V1, are shown as a function of Δt. Similar to D and E, the green and purple curves represent the summed contributions from the two locations lying on the same vertical line (Δy = ±1) for H1 and on the same horizontal line (Δx = ±1) for V1. The indices of β correspond to the locations described in D and E. The frame persistence, Tp, was set to 6 ms.
Figure 7
 
Response of H1 changes with modulations in m1 and m2. (A) The flicker energy (solid line) is constant while the motion energy (dashed line) changes. (B) The motion energy (dashed line) is constant while the flicker energy (solid line) changes. The change in flicker and motion energies is linear in both cases. Response is recorded using two reference stimulus conditions: (1) motion in the null direction (blue dashed line; i.e., v, in the even segments), and (2) pure flicker (orange dashed line; i.e., v0, in the odd segments of trial). The dual color-coded y axes correspond to the two conditions. Response of H1 averaged over 50 trials to (C) stimulus conditions shown in A and (D) shown in B. The excess rate to pure flicker and half of excess rate to null motion are shown color coded orange and blue respectively. The mean rate is r and subscripts +, −, and 0 refer to preferred motion, null motion, and pure flicker conditions. The values of other stimulus parameters were: Δx = 2, Δt = 8 ms, and Tp = 6 ms.
Figure 7
 
Response of H1 changes with modulations in m1 and m2. (A) The flicker energy (solid line) is constant while the motion energy (dashed line) changes. (B) The motion energy (dashed line) is constant while the flicker energy (solid line) changes. The change in flicker and motion energies is linear in both cases. Response is recorded using two reference stimulus conditions: (1) motion in the null direction (blue dashed line; i.e., v, in the even segments), and (2) pure flicker (orange dashed line; i.e., v0, in the odd segments of trial). The dual color-coded y axes correspond to the two conditions. Response of H1 averaged over 50 trials to (C) stimulus conditions shown in A and (D) shown in B. The excess rate to pure flicker and half of excess rate to null motion are shown color coded orange and blue respectively. The mean rate is r and subscripts +, −, and 0 refer to preferred motion, null motion, and pure flicker conditions. The values of other stimulus parameters were: Δx = 2, Δt = 8 ms, and Tp = 6 ms.
Figure 8
 
Wide-field response to reversal of contrast. (A) The stimulus condition is set to reverse the sign of m1 and keep the sign of m2 same, in every even segment of a trial. The motion energy decreases linearly (dashed line) while the flicker energy remains constant at 1 (solid line). In the last segment, Δt is set to 200 ms (m1 = 0.5, m2 = 0.5), to generate an approximately pure flicker condition. (B) The PSTH shows the trial averaged response of H1 to change in stimulus conditions described in A. (C) Steady state response is computed by discarding the initial transient (approx. 200 ms). The filled blue circles and filled orange circles represent response for odd and even segments of trial respectively. The sign of Δx and Δt remains unchanged here. The open blue circles and open orange circles represent responses to negative and positive apparent velocity (vapp), respectively. Here, the signs of m1 and m2 remain unchanged (shown by the top x axis). The gray lines represent linear fit to the data, obtained separately for the two contrast conditions and vapp conditions. The error bars represent SD of firing rate across 50 trials.
Figure 8
 
Wide-field response to reversal of contrast. (A) The stimulus condition is set to reverse the sign of m1 and keep the sign of m2 same, in every even segment of a trial. The motion energy decreases linearly (dashed line) while the flicker energy remains constant at 1 (solid line). In the last segment, Δt is set to 200 ms (m1 = 0.5, m2 = 0.5), to generate an approximately pure flicker condition. (B) The PSTH shows the trial averaged response of H1 to change in stimulus conditions described in A. (C) Steady state response is computed by discarding the initial transient (approx. 200 ms). The filled blue circles and filled orange circles represent response for odd and even segments of trial respectively. The sign of Δx and Δt remains unchanged here. The open blue circles and open orange circles represent responses to negative and positive apparent velocity (vapp), respectively. Here, the signs of m1 and m2 remain unchanged (shown by the top x axis). The gray lines represent linear fit to the data, obtained separately for the two contrast conditions and vapp conditions. The error bars represent SD of firing rate across 50 trials.
Figure 9
 
Response depends on motion and flicker components in stimuli. (A) The plot represents the 2-D space of flicker and motion energy. The range for flicker is set to [0, 2] and for motion is set to [0, 1]. The gray line at the top right hand corner represents the boundary separating the regions where values of m1 and m2 are allowed (i.e., m1 + m2 < 2) and not allowed (i.e., m1 + m2 > 2). The gray dashed line meets the flicker axis at value 4 and motion axis at value 2. The gray triangular shaded region represents the experimentally accessible region of motion-flicker space (i.e., where m1 and m2 have real values). For each value of m1 × m2, a set of linearly increasing values of Image not available + Image not available is calculated. The color codes represent the 4 levels of motion energy within the accessible space. (B) Excess firing rate is plotted as a function of the flicker energy, for each value of motion energy, shown color coded. The excess rate is obtained by subtracting the null motion response from the preferred motion response (50 trials, trial time = 2.5 s each for preferred and null motion). The error bars represent SD of the mean firing rate. Other stimulus parameters were set at: Δx = 2, Δt = 8 ms, and Tp = 6 ms. (C) Graphical representation of the product term (area in gray) and the quadrature term (radius in blue) using an m1, m2 coordinate system. The angle (θ) relates m1 and m2. (D) Color coded response curves, are plotted as a function of the angle θ. The solid gray curve with black markers represents scaled sin(2θ) / 2 or, the product m1 × m2 in terms of the angle θ.
Figure 9
 
Response depends on motion and flicker components in stimuli. (A) The plot represents the 2-D space of flicker and motion energy. The range for flicker is set to [0, 2] and for motion is set to [0, 1]. The gray line at the top right hand corner represents the boundary separating the regions where values of m1 and m2 are allowed (i.e., m1 + m2 < 2) and not allowed (i.e., m1 + m2 > 2). The gray dashed line meets the flicker axis at value 4 and motion axis at value 2. The gray triangular shaded region represents the experimentally accessible region of motion-flicker space (i.e., where m1 and m2 have real values). For each value of m1 × m2, a set of linearly increasing values of Image not available + Image not available is calculated. The color codes represent the 4 levels of motion energy within the accessible space. (B) Excess firing rate is plotted as a function of the flicker energy, for each value of motion energy, shown color coded. The excess rate is obtained by subtracting the null motion response from the preferred motion response (50 trials, trial time = 2.5 s each for preferred and null motion). The error bars represent SD of the mean firing rate. Other stimulus parameters were set at: Δx = 2, Δt = 8 ms, and Tp = 6 ms. (C) Graphical representation of the product term (area in gray) and the quadrature term (radius in blue) using an m1, m2 coordinate system. The angle (θ) relates m1 and m2. (D) Color coded response curves, are plotted as a function of the angle θ. The solid gray curve with black markers represents scaled sin(2θ) / 2 or, the product m1 × m2 in terms of the angle θ.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×