Contrast sensitivity of insect motion detectors to natural images

Abstract

How do animals regulate self-movement despite large variation in the luminance contrast of the environment? Insects are capable of regulating flight speed based on the velocity of image motion, but the mechanisms for this are unclear. The Hassenstein–Reichardt correlator model and elaborations can accurately predict responses of motion detecting neurons under many conditions but fail to explain the apparent lack of spatial pattern and contrast dependence observed in freely flying bees and flies. To investigate this apparent discrepancy, we recorded intracellularly from horizontal-sensitive (HS) motion detecting neurons in the hoverfly while displaying moving images of natural environments. Contrary to results obtained with grating patterns, we show these neurons encode the velocity of natural images largely independently of the particular image used despite a threefold range of contrast. This invariance in response to natural images is observed in both strongly and minimally motion-adapted neurons but is sensitive to artificial manipulations in contrast. Current models of these cells account for some, but not all, of the observed insensitivity to image contrast. We conclude that fly visual processing may be matched to commonalities between natural scenes, enabling accurate estimates of velocity largely independent of the particular scene.

Introduction

One useful task a visual system may perform is to provide reliable estimates of velocity. By definition, such responses must be insensitive to other stimulus parameters such as contrast or spatial content, providing similar responses to stimuli of equal velocity but differing in other characteristics. Yet, estimates of the absolute magnitude of velocity in both flies (Buchner, 1984) and humans (Thompson, 1982) are confounded when viewing luminance gratings. Such simple stimuli, however, do not reflect the complex spatio-temporal luminance patterns experienced in the natural world, which have broadband but non-random statistics (Dong & Atick, 1995; Tolhurst, Tadmor, & Chao, 1992; van Hateren, 1997) and contain marked variations in contrast (van der Schaaf & van Hateren, 1996). It is thus possible that results obtained with simple stimuli are insufficient to predict responses of a complex, non-linear motion detection system in natural conditions. Furthermore, motion-induced change in response to motion (motion adaptation) is a prominent feature of biological motion detectors (Clifford & Ibbotson, 2002; Maddess & Laughlin, 1985), but the effects on neural responses to naturalistic stimuli are unclear. 

We therefore investigated the accuracy of responses of fly motion detecting neurons at estimating the velocity of natural images of varied contrast under conditions minimizing and maximizing motion adaptation. Although it is possible to record responses in outdoor settings (Lewen, Bialek, & de Ruyter van Steveninck, 2001), the experimenter cannot easily adjust individual stimulus parameters independently of others (Egelhaaf, Grewe, Kern, & Warzecha, 2001). Therefore, under laboratory conditions, we displayed moving panoramic natural images to flies in a configuration that mimicked pure yaw rotations of the animal (Figure 1A). The purely rotational structure of this stimulus may be particularly behaviorally relevant for our experimental animal, the hoverfly Eristalis tenax, because this species frequently hovers with minimal translation while shifting their gaze through yaw rotations (Geurten, Braun, Kern, & Egelhaaf, 2007). 

Figure 1

 

Determination of HS cell response to natural image motion. (A) Visual stimuli approximate input experienced during yaw movement in a natural setting while intracellular recordings were performed. (B) Three natural panoramic images varying threefold in contrast and representative HS cell responses to their motion. (C) Natural panoramic images after artificial contrast scaling by 0.25. Responses are reduced in magnitude compared to unscaled images, but the inherent threefold variation in contrast has little effect on response. All traces are single trials from an individual HS cell. Horizontal bar represents onset of motion (42°/s, 1 s) from mean luminance background.

Stimuli were displayed while recording intracellularly from HS (horizontal system) cells, individually identifiable motion detecting interneurons sensitive to horizontal movement (Hausen, 1982). The large receptive fields of these cells are well characterized by a model that spatially pools many local, retinotopic elementary motion detectors (EMDs) (Single & Borst, 1998) in which two spatially adjacent and differentially time-filtered channels carrying contrast information are multiplied (Egelhaaf, Borst, & Reichardt, 1989). Such a correlator was originally proposed by Hassenstein and Reichardt in 1956 to model optomotor reactions of a beetle and remains influential today as a model for visual motion detection in many animals (Clifford & Ibbotson, 2002), including flies (Haag, Denk, & Borst, 2004) and humans (Adelson & Bergen, 1985; van Santen & Sperling, 1985). The basic correlator model, due to the multiplication between two contrast signals, predicts that response magnitude varies in proportion to the square of contrast. Fly HS cells show just such a quadratic relationship between contrast and response (membrane potential) at low contrasts when viewing sine-wave gratings, but increasing contrast further produces limited additional response, presumably due to saturation of the neural elements (Dvorak, Srinivasan, & French, 1980; Egelhaaf & Borst, 1989; Srinivasan & Dvorak, 1980). By using a purely rotational stimulus, where the spatial and temporal structure of motion is simple, the output of HS cells under these conditions is expected be a spatially averaged integration of individual EMDs responses to the same stimulus. 

Methods

Animals and electrophysiology

Experiments were performed on male hoverflies ( E. tenax) collected from the wild in and around Adelaide, South Australia. Flies were waxed to a viewing platform facing the display with the head tilted forward such that a small window could be cut in the cuticle through which electrodes were inserted and guided into the lobula plate. Intracellular recordings were done with glass capillary sharp electrodes filled with 2 mol l ⁻¹ KCl with tip resistances of 20–40 MΩ. All recordings were done from the left lobula plate. HS cells were identified by their preferred direction, their characteristic rumble on the audio monitor, their receptive field location, and their lack of distinct IPSPs that characterizes CH cells. 

Stimuli

To produce stimuli that filled the large receptive field of HS cells, a CRT computer monitor (640 × 480 pixels, 200 Hz refresh, mean luminance 41 cd/m ²) displayed a ∼100° wide window showing a portion of a rotating, 360° horizontal panorama (using freely available Vision Egg software by author A.D.S.; Straw, Warrant, & O'Carroll, 2006). From the position of the fly, the flat screen displayed perspective-corrected stimuli, ensuring accurate presentation of angular velocity across the large receptive fields of HS cells (Figure 1A). (The angular velocity of the cylindrical panorama was constant, and thus angular velocity on the eye varied with the cosine of elevation angle.) Each panorama was generated from twelve individual photographs taken at 30° intervals rotated about the nodal point of the camera lens (Fujix DS 300 digital camera, Apple Quicktime VR Authoring Studio software). The images output from this software were 8192 pixels wide and were rescaled to 2048 pixel width for presentation using bicubic interpolation (Adobe Photoshop) (see Auxiliary files). 

Artificial contrast scaling on the natural images was performed using the contrast parameter of the SpinningDrum class of the Vision Egg stimulus generation software. Due to its use of OpenGL for realtime image generation and scaling, contrast scaling is implemented as texture operations with the video card hardware. Ultimately, however, the value for each pixel is determined by a process equivalent to I _final = C( I _{raw_image} − 0.5) + 0.5, where pixel value, I, is a floating point number from 0 to 1 representing minimum and maximum intensity. Thus, for raw images with mean intensity of 0.5, mean luminance is constant while measures of contrast will vary. 

The Gaussian windowed sinusoidal gratings (Gabor patches) of Figure 2 were generated as described in our previous paper (Straw et al., 2006). Briefly, sinusoidal gratings moving in the preferred direction of the neuron were windowed by a small (Gaussian SD = 7.1°) patch and displayed frontally (azimuth 0.0°, elevation 30°). 

Figure 2

 

Contrast–response relationship to a moving sinusoidal grating in a Gaussian window (Gabor patch, SD = 7.1°). A 1-s constant velocity motion step (5 Hz, 0.1 cycles/°) was presented from a mean luminance background. (A) Representative responses to individual presentations measured intracellularly. The solid bar indicates the period in which contrast was nonzero. (B) Contrast–response relationship plotted on a logarithmic contrast axis. Responses were measured as in Figures 3 and 4 as the mean membrane potential from 50 to 150 ms after motion onset. The error bars show standard deviation (6 trials, data from a single HS cell).

Trials to measure motion-adapted responses used an initial, 3-s period of motion at near-optimal velocity (100°/s) to produce strong responses that induce significant motion adaptation in similar experiments (Harris, O'Carroll, & Laughlin, 2000). An example of such motion adaptation is evident in Figure 3A as the decline in response following stimulus onset. After this initial period, the moving image progressed through a sequence of test velocity periods interleaved with periods of 100°/s adapting velocity to maintain a strongly motion-adapted state. Responses to this stimulus were hyperpolarized after stimulus offset and are nearly constant during the adapting period, consistent with the presence and maintenance of a strong, motion-adapted state (Figure 3A). Furthermore, the pattern of monotonically increasing and then decreasing test velocities was chosen to maximize any history-dependent effects, which would indicate a change in the state of adaptation. However, similar responses to each test velocity, regardless of presentation order in a increasing or decreasing series, indicate that the state of motion adaptation was largely constant (Figure 3A). While transitions between different velocities in this stimulus were not designed to mimic any specific sequence that would be experienced in nature, they nevertheless permitted us to analyze responses to the full range of speeds that might be experienced during a variety of natural behaviors and thus derive velocity tuning of these neurons. This is of particular interest given recent evidence that these neurons may play a role in detection of translatory optical flow, in addition to yaw rotation (Kern, van Hateren, Michaelis, Lindemann, & Egelhaaf, 2005). Temporal anti-aliasing (motion blur) was used to avoid artifacts that would otherwise be caused by discrete temporal sampling, such as “ghosting.” 

Figure 3

 

Velocity–response relationships measured using a rapid, motion-adapting stimulus protocol. (A) Representative response to the rapid stimulus protocol. The 3-s initial ‘adapting period’ of motion is followed by a series of monotonically increasing then decreasing test velocities, which are interleaved with a constant adapting velocity (average of 4 trials from single neuron). (B) Velocity–response relationships measured for natural images of natural and artificial environments (3 trials per curve from single cell, error bars show SEM). (C) As in panel B, but with artificial contrast scaling of 0.25 (4 trials per curve from single cell, error bars show SEM). Thick lines show images in Figures 1 and 3. Dashed lines show images taken in indoor environments.

Measuring contrast

Defining a contrast metric for naturalistic images is non-trivial because a single number is used to characterize the statistics of a large number of luminance samples over two dimensions. We calculated four different metrics of contrast for each image. The first metric ( C _HS) is based on responses of an elaborated correlator model using physiologically realistic spatial low-pass and temporal band-pass prefiltering, with a low-pass filter on one arm of the correlator (parameters and model from Dror, O'Carroll, & Laughlin, 2001, Appendix A and Equation 3). The maximum steady-state response to each image (by definition, at the optimum velocity) varied nearly 10-fold across the images and, due to the multiplicative nature of the correlator model, represents a signal proportional to the square of contrast for this physiologically realistic model neuron. Thus, the square root of these values was taken as C_HS. The second contrast metric (C_std/mean) was the standard deviation of pixel luminance values divided by the mean of pixel luminance values (van Hateren, 1997). Third (C_MAD) was the mean absolute deviation (MAD) of pixel luminance values. Fourth (C_RGC) was the average absolute value of the results of a physiologically inspired model lateral geniculate nucleus cell producing contrast estimates (Tadmor & Tolhurst, 2000) at 1000 randomly sampled locations for each image. Although color images were used for experiments, all contrast measurements used only the green image channel data as luminance values because this corresponds most closely with the spectral sensitivity of Eristalis motion detectors (Srinivasan & Guy, 1990). 

The contrast values for our set of 9 images calculated with the different contrast metrics (see Table 1) had correlation coefficients ranging from 0.80 to 0.97. All measured contrast metrics scaled exactly with our artificial contrast-scaling factor; artificially scaling contrast by 0.25 resulted in contrast values exactly one fourth their original value. 

Table 1

 

Contrast of natural images used in study.

Image	Contrast metrics
		C HS	C std/mean	C MAD	C RGC
A	1.00	1.24	0.92	0.28
B	0.43	0.83	0.74	0.19
C	0.45	0.81	0.71	0.18
D	0.41	0.82	0.71	0.22
E	0.45	0.74	0.60	0.11
F	0.44	0.64	0.52	0.15
G	0.28	0.57	0.52	0.12
H	0.31	0.45	0.35	0.13
I	0.25	0.40	0.32	0.10

For the Gabor patches of Figure 2, contrast was measured using the standard Michelsen contrast formula C = ( L _max − L _min) / ( L _max + L _min). Note that this measure would be inappropriate for use on naturalistic images (Peli, 1990) because one or two pixels would be sufficient to determine contrast in this case. 

Results

Similar magnitude responses to images of varying contrast

Figure 1B shows intracellularly recorded responses of a single HS neuron to rotational motion of 3 natural scenes and our main result: Contrary to the predictions of the basic correlator model, the responses are remarkably similar in each case despite the large difference in contrast of the images. Contrast of these images varied threefold when estimated with a physiologically realistic model of the hoverfly visual system and three additional metrics (see Table 1, Methods). Dror et al. (2001) proposed that saturation of contrast signals in early visual processing reduces or eliminates sensitivity to contrast by normalizing inputs to EMDs. To test the contribution of saturation, we artificially reduced contrast while maintaining mean luminance. A scaling factor that reduces image contrast is predicted to reduce pre-EMD neural responses and thus minimizes effects of saturation on motion detector output. Figure 1C shows an example using a contrast-scaling factor of 0.25. While responses to contrast-scaled natural images are weaker than those to un-attenuated images, indicating a reduction in saturation, they are surprisingly similar to each other for the different images. Thus, saturation is unlikely to explain the observed invariance in response to natural images of varying contrast. 

To establish that our experimental setup itself was not somehow responsible for this contrast invariance and also to establish that Eristalis HS cells show the contrast–response relationship typical for other fly species when viewing sine-wave gratings, we performed experiments measuring response amplitude as a function of contrast for Gaussian-weighted sine-wave gratings (Gabor patches), as shown in Figure 2. The contrast–response relationship measured on our setup with Eristalis shows a similar quadratic rise in response at low contrasts and saturation at high contrasts as measured in the blowflies Lucilia sericata (Dvorak et al., 1980; Srinivasan & Dvorak, 1980) and Calliphora vicina (Egelhaaf & Borst, 1989). For an extensive analysis of the contrast sensitivity of Eristalis HS cells in response to sinusoidal gratings, including an analysis of the temporal and spatial tuning across the surface of the eye in the sexually dimorphic manner, see Straw et al. (2006), in which we performed experiments with the same apparatus and in some cases during the same recording sessions used for the present study. 

Contrast invariance found under strongly motion adapted conditions

Could motion adaptation contribute to the observed invariance of motion detector responses to natural images? Motion adaptation operates on behaviorally relevant time scales (Fairhall, Lewen, Bialek, & de Ruyter Van Steveninck, 2001), appears intrinsic to the motion detection mechanism in flies (Borst, Flanagin, & Sompolinsky, 2005), and involves at least two mechanisms in HS cells, a contrast gain reduction and a shift of mean output level (Harris et al., 2000). Furthermore, one component of contrast gain reduction may operate in an instantaneous, feed-forward fashion (Borst, Egelhaaf, & Haag, 1995). One possibility is that independence to contrast could be achieved if gain were reduced to high contrast stimuli based on the recent history of image statistics in a way that normalizes responses across scenes of varied contrast. To test responses under strongly motion-adapted conditions, we used a protocol in which high velocity adapting stimulus periods were interleaved temporally with test periods (Figure 3A). As an indirect measure of the state of adaptation, we also analyzed responses during the adapting periods (see Methods). As shown in Figure 3B, for all 9 images tested, HS cells responded with similar magnitude to test velocities up to 100–200°/s in a monotonically increasing manner. This speed is similar to the maximum angular velocity of the eyes between rapid turns (saccades) in blowflies (van Hateren & Schilstra, 1999). (During saccades, retinal angular velocity may be much higher (van Hateren & Schilstra, 1999).) Beyond this velocity optimum, response magnitudes decline with increasing velocity, but remain similar in magnitude across images. The similarity in response magnitude across images indicates that the particular image used and thus its contrast has little effect in the magnitude of the response under strongly adapted conditions. 

The relative invariance of responses to natural images of varied contrast during the motion adapting stimulus does not result from saturation of neural elements to the contrast of natural stimuli because similar results are obtained if contrast of the stimuli is artificially reduced to relieve saturation within the system (e.g., Figure 3C). At these artificially scaled contrasts, the maximum response magnitude is decreased, yet responses remain similar across all images from outdoor scenes. Interestingly, the two images taken in indoor environments evoke stronger responses at this artificially reduced contrast (Images E and I). This is surprising because these two images fall in the lower half of the range of contrasts tested. Although the data from only two artificial images are inconclusive on the issue, these results raise the possibility that the insensitivity of HS cell response to natural images relies on physiological mechanisms matched to properties of natural, outdoor scenes. This would be an interesting avenue for future research. To summarize, an artificial reduction in contrast reduces the overall magnitude of the neural response, indicating a relief from saturation (and consistent with predictions of correlator-like motion detector models), but saturation in early vision cannot explain the neuron's ability to encode image velocity at different contrast levels. 

Contrast invariance does not depend on strong motion adaptation

Although the data in Figure 3 show that natural image invariance is observed in the presence of powerful motion adaptation, they do not prove that motion adaptation is responsible. We also observed this natural image invariance of HS responses in experiments where care was taken to minimize the effects of motion adaptation ( Figure 4A). These experiments were performed after at least 6.5 s of exposure to a blank, mean-luminance screen. Furthermore, the analysis was performed only on early portions of responses. (The time constant of the fundamental motion-detection operation of these cells is thought to be about 35 ms (Harris, O'Carroll, & Laughlin, 1999), and the data shown are the average of responses from 50 to 150 ms after stimulus onset.) Figure 4, which also includes responses to motion in the anti-preferred direction, shows that responses under conditions of minimal motion adaptation have a similar shape and optimum velocity to those from strongly motion-adapted neurons. Unless motion adaptation operates on very fast time scales similar to motion detection itself, our data suggest that contrast gain reduction induced by motion adaptation is unlikely to explain the insensitivity of HS cell response to natural images of varied contrast. Because we find invariance in responses to natural images of varying contrast in both our strongly and minimally adapted stimulus conditions, we do not expect our finding to be constrained to the particular sequence of velocities, and hence adaptation state, selected for these experiments. Similar velocity tuning before and after a strong motion stimulus suggests that motion adaptation cannot be accurately modeled as a shortening of the time constant of EMD delay filters (Harris et al., 1999). 

Figure 4

 

Velocity–response relationships obtained with unscaled and artificially contrast-scaled natural images. (A) Responses in minimally motion-adapted conditions. For a given contrast-scaling factor, contrast of image had little effect on output but the scale factor determined the overall response magnitude. (B) Responses in adapted conditions. Responses are decreased in magnitude and less symmetrical to preferred-direction (PD) versus anti-preferred direction (Anti-PD) motion but peak at similar velocities and exhibit similar invariance to contrast of images. (Results averaged from 4 and 5 animals in the PD and Anti-PD directions in panel A and 5 and 4 animals in the PD and Anti-PD directions in panel B.) Error bars show SEM.

Discussion

Our results show that, across nine natural images tested, contrast of the specific image has little effect on responses of fly HS neurons. This is surprising when 4 different measures of image contrast (see Methods) show that the set of images selected span a large, approximately threefold, range of contrasts ( Table 1). Indeed, one contrast metric ( C _HS), based on a Hassenstein–Reichardt correlator model designed to simulate the HS neuron responses, shows a near 10-fold range in output level to the same set of images. Although this model lacks additional non-linearities thought to be present in the fly motion detection circuitry upstream of HS cells, our experiments show that neither saturation nor prior exposure to motion (and thus strong motion adaptation) provide an obvious explanation for this apparent independence to image contrast. The primary effect of our motion adaptation paradigm appears to be a reduction in amplitude of responses to natural images without otherwise altering velocity tuning. We have recently shown that further elaborated models are able to capture some, but not all, of the insensitivity to contrast (O'Carroll, Shoemaker, & Straw, 2005) observed here. In particular, linear high-pass spatial filtering in conjunction with any of four non-linear processes reduced variation in simulated responses to natural images. These non-linear processes, which reduced variation in response each individually and in combination with the high-pass filtering, were motion adaptation, early vision saturation, correlator saturation, and a form of gain control. In no case, however, did the simulation results approach the level the invariance with respect to contrast of the neural responses. 

Although our experiments using rotational stimuli sought to emulate conditions where hoverflies hover (with minimal translational motion), these animals also engage in flight with significant translational components. To what extent are the responses properties described here thought to be relevant for more complex patterns of motion? Blowfly HS cell responses to stimuli emulating more complex movement trajectories have been investigated (Boeddeker, Egelhaaf, Lindemann, & Zeil, 2005; Egelhaaf, Karmeier, Kern, & van Hateren, 2006; Kern et al., 2005; Lindemann, Kern, Michaelis, Meyer, van Hateren, & Egelhaaf, 2003), and the conclusions from these experiments are that HS cells respond both to translation and to rotation-evoked visual motion, with translation-evoked responses dominating the response period between saccades, particularly the lower frequency components of the response. Although our experiments do not directly test hoverfly HS responses under such conditions, the results are directly behaviorally relevant for periods in which the hoverfly is hovering without translation. Furthermore, our finding of relative insensitivity to contrast of natural images appears to be a general property and may be expected to affect responses to translational movement. 

Experiments on speed regulation of freely flying fruit flies (David, 1982) as well as centering behavior (Kirchner, Lehrer, Srinivasan, & Zhang, 1991) and visually mediated odometry (Si, Srinivasan, & Zhang, 2003) in honeybees show that these insects are capable of regulating forward velocity independent of pattern texture, suggesting that these behaviors rely on motion detectors fundamentally different than a correlation-based mechanism with its sensitivity to pattern texture and contrast (Srinivasan & Zhang, 1997; Srinivasan, Zhang, & Chandrashekara, 1993). Furthermore, recent experiments have shown that altered pattern contrast does not affect forward velocity regulation in bees (Baird, Srinivasan, Zhang, & Cowling, 2005). Is it possible that neurons with contrast sensitivity properties similar to those described here could serve these behaviors? In these behavioral experiments, insects flew in tunnels of constant spatial frequency, measured in linear terms. On the retina, the animals saw a range of spatial frequencies in angular terms in addition to frequencies created by edges and other imperfections of the experimental setups. Could the range of frequencies present in the behavioral experiments, combined with the reduced contrast sensitivity for broadband stimuli described here, mean that HS cells subserve this behavior? Although the present experiments are not sufficient to draw conclusions, they show that, when viewing naturalistic stimuli, HS cells are capable of signaling velocity with responses less sensitive to contrast than previously reported. 

Further work exploring the physiological mechanisms underlying the observed contrast (in)sensitivity to natural images combined with an investigation of the neural substrate of the motion detection operations underlying the behavioral results from the literature will be necessary to address whether cells with properties similar to HS might play a role in behaviors such as speed regulation, centering response, and visually mediated odometry. Nevertheless, it appears flies are capable of more accurately estimating real-world velocities than previously thought. 

Supplementary Materials

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Figure S1 - Figure S1 

Acknowledgments

This work was supported through a Predoctoral Fellowship from the Howard Hughes Medical Institute to A.D.S. and grants from the United States Air Force Office of Scientific Research/Asian Office for Aerospace Research and Development. We thank Rebekah Smith for some of the digital photography, Russell Brinkworth for helpful comments on the manuscript, and the Botanical Gardens of Adelaide for allowing us to collect Eristalis. 

Commercial relationships: none. 

Corresponding author: Andrew D. Straw. 

Email: [e-mail]. 

Address: California Institute of Technology, Mailcode 138-78, Pasadena, CA 91125, USA.