Modeling a space-variant cortical representation for apparent motion

Abstract

Abstract

Abstract:

Abstract  Receptive field sizes of neurons in early primate visual areas increase with eccentricity, as does temporal processing speed. The fovea is evidently specialized for slow, fine movements while the periphery is suited for fast, coarse movements. In either the fovea or periphery discrete flashes can produce motion percepts. Grossberg and Rudd (1989) used traveling Gaussian activity profiles to model long-range apparent motion percepts. We propose a neural model constrained by physiological data to explain how signals from retinal ganglion cells to V1 affect the perception of motion as a function of eccentricity. Our model incorporates cortical magnification, receptive field overlap and scatter, and spatial and temporal response characteristics of retinal ganglion cells for cortical processing of motion. Consistent with the finding of Baker and Braddick (1985), in our model the maximum flash distance that is perceived as an apparent motion (D_max) increases linearly as a function of eccentricity. Baker and Braddick (1985) made qualitative predictions about the functional significance of both stimulus and visual system parameters that constrain motion perception, such as an increase in the range of detectable motions as a function of eccentricity and the likely role of higher visual processes in determining D_max. We generate corresponding quantitative predictions for those functional dependencies for individual aspects of motion processing. Simulation results indicate that the early visual pathway can explain the qualitative linear increase of D_max data without reliance on extrastriate areas, but that those higher visual areas may serve as a modulatory influence on the exact D_max increase.

Introduction

The human visual system has highly nonuniform sampling of visual input in both the space and time domains. This nonuniform, or space-variant, sampling greatly affects human perception of motion from a visual scene. These space-variant factors include cortical magnification, receptive field overlap, receptive field scatter, and retinal ganglion spatiotemporal response characteristics. Despite the pivotal role that these sampling features play in the perception of motion, relatively little is known about how the individual space-variant components of the visual processing pathway affect the perception of motion. Our study teases apart the role of cortical magnification by constructing a neural, mathematical model that incorporates a number of space-variant sampling components in their simplest form. We tie the corresponding relationship between the model output and the motion percept by constraining the model to the psychophysical function D_max. 

For a given experimental setup, the psychometric function D_max is a measure of the maximum stimulus displacement that yields a percept of motion. D_min is directly analogous for the minimal stimulus displacement. Baker and Braddick (1985) found that D_min and D_max increase nearly linearly as a function of eccentricity up to at least 10°. The cause of this linear increase, however, is not known. Larger receptive fields in the periphery might account for larger D_max by accumulating broader motion signals within the required integration time to perceive motion. It could be counterhypothesized, however, that faster peripheral processing via the magnocellular pathway could decrease the integration time necessary to combine motion signals and thus decrease D_max in the periphery. A major question, then, is whether early visual processing (retina through V1) can account for the observed change in D_max across eccentricities or whether this change requires higher level visual processing. 

Previous models used to explain D_max data

Previous neural models have successfully explained a number of psychophysical findings regarding D_max. Eagle (1996) describes a model “in which direction discrimination is based on the nearest-neighbor matching of zero-crossings in the output of a single-spatial-filter bandpass in both spatial frequency and orientation;” the author uses this model to account for D_max psychophysics data for spatially broadband motion patterns, arguing that D_max is determined by the coarsest spatial filter activated by each stimulus. Glennerster (1998) use an implementation of the MIRAGE model (Watt & Morgan, 1985) to account for data on D_max as a function of stimulus dot density. Morgan (1992) considered data on increasing D_max as a function of increasing stimulus element size and argued that such data could be explained using a model where a spatial filtering step removes fine detail in a stimulus pattern before motion processing occurs. Tripathy, Shafiullah, and Cox (2012) describe a model based on a set of Reichardt-type local detectors whose radius of their catchment areas scale proportionately with the displacement that they are tuned to detect; this model is able to account for D_max data where correspondence noise is a major contributing factor. 

To the knowledge of the authors, however, no previous model has explained D_max data showing that D_max increases nearly linearly as a function of eccentricity. The model presented herein elucidates this data by taking into account space variant factors present in the early visual system. How the visual system processes motion across a wide range of biological visual parameters, including cortical magnification, receptive field overlap and scatter, and spatial and temporal response characteristics of retinal ganglion cells are key to understanding how D_max changes at different eccentricities. By using a point spread function to compute motion signals and a discriminability index based on the model cortical activity difference between continuous and discrete inputs to determine D_max, our model is able to show that the space-variant factors present in the early visual system (retina to V1) are sufficient to account for increasing D_max as a function of eccentricity. 

Space variance in the visual system

The human visual system has evolved to process light input that enters the eye in a highly nonuniform manner. Perhaps the most notable aspect of this nonuniformity is the clarity with which we perceive objects projecting to our fovea versus periphery. This difference in clarity arises from both the sampling density of retinal receptors as well as the subsequent neural processing. Motion processing is also greatly affected by the same space-variant sampling processes, but in more subtle ways. It is not so clear that the fovea should be better suited to process or perceive motion despite its greater spatial resolution; indeed, motion processing is perhaps the one case where periphery vision may be better suited than the fovea (Westheimer, 1983, Baker & Braddick 1985). Why our visual system has evolved to be so dominantly space-variant is unknown, but a back-of-the-envelope computation from Bonmassar and Schwartz (1997) estimates that for the human brain to maintain foveal resolution throughout the entire field-of-view, the brain would have to weigh upwards of a few metric tons. Additional research has indicated that space-variance in the visual system may be useful for more than simplifying computational complexity. Space variance has been a key component in systems designed for segmentation (Mishra & Aloimonos, 2009), time-to-contact estimation (Tistarelli & Sandini, 1993), and robot navigation (Engel et al., 2009). 

In this study we aim to determine what, if any, aspects of the early visual system's space variance are responsible for the observed increase in D_max as a function of eccentricity. In the following sections we describe several space-variant properties present in the human visual system and how they relate to one another in the context of cortical sampling of the retinal image. 

Cortical magnification factor

Cortical magnification (CM) is a measure of how much cortical length is dedicated to process a stimulus of a given visual angle. In most space-variant visual systems, the amount of cortical surface dedicated to processing the central visual field is far greater than for the peripheral field. In humans this variance is generally about two orders of magnitude (Daniel & Whitteridge, 1961). The quantitative measure of CM is called the cortical magnification factor (CMF) and is typically measured in millimeters of cortical length per degree of visual angle (Figure 1). The exact measure of CMF differs between cortical areas. Area V2, for example, has more enhanced foveal representation than V1; as much as 50% of Marmoset monkey V2 is dedicated to the central 5° of the visual field (Rosa, Fritsches, & Elston, 1997). Also, compared to V1, area V4 (which is believed to be involved in fine shape and texture analysis) has very little area dedicated to the periphery whereas other areas, such as V6 (which is believed to be responsible for analyzing self-motion), show a more gradual decline in the amount of cortical area dedicated to the periphery (Daniel & Whitteridge, 1961). 

Figure 1

 

Two-DG staining on Macaque V1. Left: Color-coded visual input. Right: Color-coded stain of Macaque V1 after being shown the input on the left. Note that, per unit area, much more cortical area is dedicated to the fovea compared to the periphery. This allocation of visual processing resources is known as cortical magnification and is expressed through the cortical magnification factor, or the ratio of millimeters of cortical length per degree of visual angle. [Figure adapted from Tootell et al. (1982) with permission].

To the knowledge of the authors no model to date has incorporated CMF into determining the maximal displacement for perceived motion. The present work explores the functional role of higher CMF in the foveal region compared to the periphery. It is not trivially clear whether a greater CMF in the fovea would contribute to lower D_max in this region. 

Receptive field size

The concept of a receptive field is used to describe individual neurons. The receptive field for a particular neuron is defined as the region in stimulus space that alters the firing of that neuron. For visual cortical neurons this correlates to the region on the retina wherein a change in the stimulus affects the firing rate of that neuron. Receptive fields have complex shapes but are often taken to be near-circular for quantitative purposes. The receptive field size for visual cortical neurons is most often measured in degrees of visual angle and increases with stimulus eccentricity. Note that this property is distinct from cortical magnification, as it is theoretically possible to have nearly any combination of cortical magnification and receptive field size for a given sheet of cortical neurons. Since receptive field sizes increase as a function of eccentricity (Figure 2) while CMF decreases, it seems reasonable to think that visual cortical architecture contains some inherent constant factor. Others have assumed that this constant factor is the product of CMF and receptive field size, that some have called point image size, to conflicting results (Hubel & Wiesel, 1974; Dow, Snyder, Vautin, & Bauer, 1981; Van Essen et al., 1984; McIlwain, 1986). More recent studies, however, have yielded additional credence to the hypothesis that the point image size in V1 is indeed constant or very slightly-increasing (Harvey & Dumoulin, 2011; Palmer et al., 2012). A related hypothesis is that receptive field overlap might be constant (Bolduc & Levine, 1998). While not mathematically equivalent, the assumption of constant receptive field overlap combined with a linear increase/decrease in receptive field size/CMF, respectively, can yield a near-constant point image size for a range of constant overlap constants. Our model uses this assumption of constant receptive field overlap in order to constrain the relationship between CMF and receptive field size. 

Figure 2

 

Schematic comparison of receptive field sizes across eccentricities for different cortical regions. (Figure taken from Freeman and Simoncelli, 2011, with permission).

Receptive field overlap

Receptive field overlap refers to the portion of adjacent receptive fields that share the same stimulus space. Assuming near-circular receptive fields, receptive field overlap can be computed knowing the receptive field size and centroid locations for each point in the visual field. For primates, biological evidence suggests any given region of the retina can be part of as many as 35 ganglion cell receptive fields (Braccini, Gambardella, Sandini, & Tagliasco, 1982; Bolduc & Levine, 1998). This number appears to be relatively constant throughout the entire visual field, thus yielding a large constraint that can be applied when determining the model parameter values for CMF and RF size. Our model incorporates this overlap factor in order to constrain both the point image size, as well as the model's cortical columnar structure. 

Cortical receptive field scatter

The primary visual cortical surface is topographically organized. That is, physically adjacent neurons in the cortex will have adjacent receptive field centroids. This topographical organization means that as one moves through a cortical column all neurons would be expected to have the same centroid location. To a first order approximation this assumption holds true. There is, however, still a measurable deviation of receptive field centroid locations even within one column. The amount of deviation of receptive field centers as one moves down a cortical column is called cortical receptive field scatter (Figure 3). 

Figure 3

 

“Overlap of striate receptive fields recorded in pairs of vertical penetrations displaced 2 mm from one another on the cortical surface. For each pair, dashed lines represent receptive fields from one penetration; solid lines represent receptive fields from the other penetration. One pair of penetrations was made in the central foveal representation (15-foot eccentricity), the other in peripheral representation (110-foot eccentricity).” Used with permission from Dow et al. (1981).

There is debate as to whether this scatter has a functional role in refining acuity perception or is simply added noise. While the model presented here does not seek to answer what exact functional role for cortical receptive field scatter might serve, it does, however, seek to place an upper limit on the amount of added noise that such cortical scatter could place on cortical motion processing in V1. 

Receptive field centroid density

Referring to a model of overlapping, circular receptive fields with well-defined centers, the receptive field centroid density is the density of receptive fields per unit area (2D case) or per unit length (1D case) measured in retinal coordinates. As changing the RF centroid density would alter model cortical magnification values, such a change would affect D_max. For the purposes of this study, RF centroid density is a derived measure that can be computed from known CMF and receptive field size values, along with the additional data constraint of constant or near-constant point image size (Harvey & Dumoulin, 2011; Palmer et al., 2012) derived from a constant receptive field overlap factor. 

Spatiotemporal response

The previous visual sampling parameters have all been defined in the spatial domain. The response of actual neurons, however, is also a function of time. That is, their responses are more accurately described in a spatiotemporal domain wherein the output of a neuron to a particular spatial stimulus changes over time. The exact nature of the spatiotemporal response is important when modeling speed-tuned neurons. Adelson and Bergen (1985) describe how to set up linearly separable spatiotemporal filters that are functionally equivalent to Reichardt detectors and posit a neural implementation for their filters. Our model extends this spatiotemporal description to allow each component, spatial and temporal, to vary independently as a function of eccentricity. By adding in this eccentricity-dependence we are able to more accurately track the spatiotemporal response of model neurons across V1 as the input stimuli moves across many degrees of visual space. 

Apparent motion

Apparent motion describes a number of related phenomena wherein motion is perceived from one or more static images. Apparent motion can be subdivided by the manner in which the motion is perceived and the stimulus used, such as illusory motion, beta motion, and phi motion. In the case of multi-image stimuli it is common to use single flashes of light or arrays of random dots. 

In the simplest case, a two-flash stimulus is used to produce a motion percept. Classically, there are three main stimulus parameters that dictate when a two-flash stimulus will yield apparent motion. These parameters were characterized by Wertheimer's student Korte in what have become known as Korte's laws, which describe the relationships between stimulus parameters over some ranges to stay at threshold for perceiving apparent motion: 

1.  
   Separation versus Intensity. Intensity must increase as the separation distance increases (and vice versa).
2.  
   Rate versus Intensity. Intensity must increase as the flicker rate decreases (and vice versa).
3.  
   Separation versus Rate. Flicker rate must decrease as the separation distance increases (and vice versa).

While the classic definition of D_max and D_min are meant to be constant for a given flicker rate and intensity, Baker and Braddick (1985) have also shown that they are both proportional to visual eccentricity. 

Determining D_max

D_max and its counterpart, D_min, are perceptual functions. In other words, they measure the dependence of a perceptual phenomenon (the perception of motion) as a function of input spatial separation in a two-flash stimulus paradigm. Baker and Braddick (1985) measured D_max and D_min using a two-frame random dot stimulus. In this setup the subject is shown two frames of random dots wherein a single region of the image is replicated and shifted from frame to frame. The subject's task is to respond to the direction of motion of the replicated region (see Figure 4). 

Figure 4

 

Input schematics for determining D_max psychophysically. The subject is shown the two random-dot kinematogram frames in sequence and asked which direction the replicated region moved (up or down). The maximal displacement for which the subject is able to correctly respond above a set percentage is called D_max. D_min is directly analogous for the small displacements.

Baker and Braddick (1985) measured D_max and D_min and found that both measures increased linearly as a function of eccentricity (Figure 5). Psychometric experiments that are used to determine D_min and D_max must then provide a link between the perceptual phenomena they wish to discover and the psychometric data which they produce. Baker and Braddick (1985) provide this link by asking subjects to rate a perceptual measure (motion clarity) on a graded scale. Plotting this motion clarity measure versus performance yields a sigmoidal diagram. By selecting the center point of this sigmoid as a performance cutoff one can approximately define an analytic link between the performance-based measure observed in the psychometric study and the perception of motion. 

Figure 5

 

D_max data taken from Baker and Braddick (1985), plotting D_max as a function of eccentricity.

The modeling study presented here continues in a similar vein; while our input stimulus follows a paradigm that can be directly linked back to apparent motion, our measurement of D_max follows an activity-difference metric. By measuring the Euclidean distance between model V1 network activity profiles we obtain a direct relationship between the input stimulus and the distinguishable network activity which can be used to derive a percept of motion. By defining a cutoff threshold for this Euclidean distance measure we effectively produce a performance-based metric. Aligning the model performances with that from Baker and Braddick (1985) we are able to derive a direct conversion between network model activity and the perceptual measure, D_max. 

What is the source of the increase in D_max as a function of eccentricity?

Behavioral data from the Baker and Braddick (1985) psychophysical experiments make it clear that there is an eccentricity-dependence in the visual system's processing of motion. Physiologically there are many sources of space variance in the visual system (Table 1) that could all contribute to this eccentricity dependence, and it is not trivially clear which of these physiological factors are necessary to explain the behavioral data. Because these factors interact together in a very nonlinear manner, teasing apart the role of each one individually will prove difficult to do through psychophysical and physiological methods. Instead, we construct a model of the early visual system that incorporates each space-variant factor in Table 1. We tie the model V1 activity output to D_max behavioral data through the use of a discriminability index that measures the difference in model V1 activity between the cases of discrete versus continuous input. The results of this modeling effort give credence to the idea that the increase in D_max as a function of eccentricity can be explained by early visual space-variant factors alone. Furthermore, behavioral data obtained by Todd and Norman (1995) that show an increase in D_max as a function of the number of input frames is not able to be explained by the current model (refer to the Discussion section Role of Higher Level Visual Areas). Thus the model also hints that the number-of-frames effect expounded by Todd and Norman (1995) might require processing from higher level visual areas (extrastriate areas) that is not present in the early visual system in order to be explained. 

Table 1

 

List of sources of space-variance in the early visual system.

Sources of space-variance in the early visual system
Cortical magnification factor
Receptive field size
Receptive field overlap
Cortical receptive field scatter
Spatiotemporal response

The rest of this paper is broken down as follows: the Methods section breaks down the major model components and how the model incorporates each of the individual space-variant factors; the Results section shows the model is able to fit D_max psychophysical data showing that D_max increases nearly linearly as a function of eccentricity while the model fails to fit D_max psychophysical data showing that D_max also increases with the number of input frames; finally a Discussion section summarizes the main findings of this paper and what the model data supports in terms of the early visual system's roles in space-variant motion processing. 

Methods

The following section outlines the model, which includes three main neural correlate stages: (1) input sampling layer (corresponding to the retina), (2) shunting inhibition layer (corresponding to LGN/V1), and (3) a cortical layer (corresponding to V1). 

Figure 6 gives a graphical representation of the model stages. The input stimulus (shown in Figure 8) to the model is a paired-flash stimulus, which is sampled by a number of spatially-offset retinal fields with different scales. The purpose of multiple, off-set retinal sampling fields is to allow the model's LGN/V1 neurons to receive input from retinal ganglion neurons with randomly-selected offsets in their receptive fields. This sampling mechanism allows the model to incorporate a measure of cortical receptive field scatter. The retinal layer is connected to a model LGN layer described by a feed-forward center-surround network which incorporates shunting inhibition (refer to Equation 3) in order to assure that the model remains normalized throughout a broad range of possible input intensities. The model V1 cortical architecture contains individual layers wherein each layer receives input from a different, offset retinal sampling field, thus incorporating cortical receptive field scatter. Model V1 averages activity across a single column, resulting in a cortical activity profile. D_max is computed by generating a difference measure between the activity profile generated from a continuous input and the activity profile generated from the paired-flash stimulus (refer to the Discriminability indices section). 

Figure 6

 

Model overview. The model consists of five main stages detailed in the following sections: (1) model input stimuli, (2) retinal sampling layer, (3) feed-forward center-surround layer (corresponding to LGN/V1), (4) V1 cortical columnar layer, and (5) aggregate cortical activity.

Figure 7

 

Two-flash stimulus paradigm. The stimulus consists of two frames separated by a blank. Stimulus onset asynchrony (SOA): the time from the onset of the first frame to the onset of the second frame; interstimulus interval (ISI): the time from the offset of the first frame to the onset of the second frame.

Figure 8

 

Example of two-flash input stimuli presented to model.

Model input

We simulate our network following the 1D flash stimulus paradigm shown below. 

Figure 7 shows an example input to our model. The two input nodes receive dot (impulse) inputs while all other nodes receive the same, nonzero background activity. 

While we tie our model's output to the psychometric-to-perception link provided by Baker and Braddick (1985), it is important to note the distinction between our input and that used in their experiment. Our stimulus is a two-flash input directly applicable to Korte's laws of apparent motion, wherein motion would be regarded as a perceptual phenomenon. The experimental setup used by Baker and Braddick (1985) consisted of random dot stimuli inputs under a performance-based psychometric task (refer to Figure 4). This experimental design removes the results from direct applicability to perceptual motion via Korte's laws in two ways: (1) the performance-based metric must be carefully calibrated to the perception of motion, and (2) there is the conflicting factor of background noise. The first factor has been discussed previously, but selecting an appropriate stimulus is important for our study for the second factor as well. Since the model is focused on early visual processing stages and does not rely on any direct motion processing, it is important that the final network activity not require any additional filtering or higher level visual processing to remove noise effects as these effects would corrupt the Euclidean difference metric used to determine the D_max. 

Spatiotemporal receptive fields

Model retinal sampling follows the overlapping circular receptive field model described in Yamamoto, Yeshurun, and Levine (1996) (Figure 9). A central region in the retinal sampling field is defined to be the fovea. All receptive fields in this region have the same spatiotemporal properties with receptive field size r_f. The overlap constants are also held constant in this region such that the RF density and cortical magnification factor are constant. An iterative scheme is then used to create a space-variant sampling method one ring at a time using the following equation:  

 

 

 

where r_n is the radius of the n^th RF in the peripheral direction (note that in this notation r_n=0 = r_f), c_n is the centroid location of the n^th RF (c_n=0 = c_f), α is a parameter that describes the ratio of RF radius to eccentricity (Figure 10), and ω is a parameter that describes the amount of overlap between RFs (Figure 11). 

Figure 9

 

Model sampling. The 2D sampling paradigm from Yamamoto et al. (1996) is adapted to a 1D input space. The fovea is taken to be the central 1° of visual space; within this region receptive field characteristics are constant. Starting from the edge of the fovea moving towards the periphery, receptive field diameter increases linearly in accordance with known data (Freeman and Simoncelli, 2011).

Figure 10

 

Displaying the degree of fan-out, determined by the parameter α.

Figure 11

 

Displaying the degree of overlap, determined by the parameter ω.

Cortical layers exhibit feed-forward center-surround excitation/inhibition

The output from the retinal layer feeds into the model LGN, which is described by a feed-forward competitive shunting network (Figure 12). Shunting inhibition is used to assure that the network remains normalized throughout a range of possible input intensities. The LGN is analytically described by:  

 

where I is an input vector, x is the associated output activation vector, n is the number of units in both the input and output layers, t is time, B and D are shunting parameters, and C_ki and E_ki are Gaussian kernels. Note that the decay-rate parameter A_i, the rise-time parameter τ_i, and the Gaussian functions C_ki and E_ki are all a function of eccentricity. C_ki and E_ki are defined as:  

 

 

 

where μ and ν are the variances of the Gaussians, (k–i) is the distance from the center of the receptive field centroid of node i to node k, and C and E are parameters. For the eccentricity-dependent parameters A_i and τ_i we tried many different functional forms and find that simple linear functions still allow enough flexibility in the model to accurately fit both the physiological and psychophysical data.  

 

 

 

where the individual parameter values can be found in Table 2. 

Figure 12

 

Diagram showing center-surround excitation/inhibition. A one-layer shunting network is used for normalization of the input space. Note that while the spatial connectivity in the cortical domain (bottom half of the diagram) is constant this connectivity is spatially skewed in the input domain (top half of the diagram. The result is that activity across the stimulus space in the far peripheral areas (in retinal coordinates) spreads wider through the model cortical layer than activity in foveal areas. Note that this effect is due solely to cortical magnification and is independent of the spatiotemporal characteristic differences between foveal and peripheral cortical neurons.

Table 2

 

Model parameters. Notes: B, C, D, E: shunting parameters. μ, v: variances of the Gaussian kernels C_ki and E_ki, respectively. m_A, b_A slope: offset for retinal spatial eccentricity-dependence. m_τ, b_τ slope: offset for retinal temporal eccentricity-dependence. α: degree of receptive field fan-out. ω: degree of overlap. r_f: foveal radius. r₁: foveal receptive field size. r_p: field-of-view radius of the periphery. δ_i: scatter offset factor.

Parameter	Value
B	10
C	3
D	10
E	0
μ, v	0.1
mA	0.2
bA	0.01
mτ	0.6
bτ	0.03
α	0.25
ω	0.5
rf	1.0
r1	0.25
rp	85
δi	0.1

Cortical columns incorporate cortical RF scatter

In order to simulate cortical scatter, multiple cortical layers are created with offset δ_j. That is, each RF in the j^th retinal layer has a centroid location c_n + δ_j and RF size r_n (Figure 13). 

Figure 13

 

Model cortical scatter schematic. Model V1 receptive fields are sampled from offset retinal locations. Cortical columns are generated by combining (averaging) V1 neuron activities from neurons with RFs that are offset in retinal coordinates. All neurons within a column have a RF centroid taken in the range c_n + δ_j, where c_n is the centroid location of the n^th neuron and δ_j is a model parameter (Table 2).

The output of the LGN layer feeds into model V1, which sums the activity of each LGN input across all cortical scatter locations. That is,  

 

 

Cortical activity profiles

The model output to a single input stimulus is one described by “fat and fast” cortical activity in neurons with peripheral RFs and “slim and slow” cortical activity in neurons with foveal RFs (Figure 14). Because peripheral neurons have larger spatiotemporal parameter constants (dictated by m_A and m_τ) a single peripheral input stimulus will result in a broader, faster cortical response compared to cortical neurons with foveal receptive fields. While these response characteristics could be useful for spreading motion-selective neural activity, peripheral neuron activity also decays much more rapidly than their foveal counterparts. The interplay between fast, broad peripheral network activation and acute, slow foveal network activation plays a large role in determining the model D_max. 

Figure 14

 

Schematic showing the spatiotemporal response characteristics for model cortical neurons to single, unit amplitude input stimuli at different eccentricities.

Discriminability indices

Once the model network activity is computed a measure of D_max must be derived. In order to determine D_max a difference measure is computed and a threshold is established to produce a value of D_max at varying eccentricities. The difference measure used is a straight Euclidean metric between the network activity resulting from a two-flash stimuli against the network activity resulting from a continuous stimuli input. That is, the model system is first shown an example of continuous motion and the resulting network activity is recorded. If this activity is similar enough (in a Euclidean sense) to a discrete, flash input then that input is said to produce a motion percept. If the activity difference is greater than some threshold, however, then that input is said to produce a discrete flash percept. Thus, if the network activity difference is below some threshold (selected to give an appropriate range of D_max values) then the network is determined to produce a motion percept. Figure 15 shows the difference measure for a wide range of interstimulus distances and for multiple eccentricities. 

Figure 15

 

Schematic showing the discriminability measure (normalized Euclidean distance) over a range of interstimulus distances (ISD) for multiple eccentricities. In order to fit psychophysical measurements from Baker and Braddick (1985) an appropriate cutoff is selected to produce an appropriate range for D_max.

Results

Basic observations of the phenomenon

For our initial explorations, we sought to make sure that the model would yield eccentricity-dependent results based on standard model parameters and would not yield eccentricity-dependent results using non-variant space parameters. 

The parameter α is used to determine the receptive field layout (in accordance with Equation 1); additionally for this basic condition the space variance of the network time constant (τ), decay rate (A), and Gaussian kernels (C_ki and E_ki) were all set proportional to α. That is, the slope of the τ, A, C_ki, and E_ki as a function of eccentricity is proportional to α so that when α = 0 the aforementioned parameters are not space-variant. 

We determine the exact values of D_max by selecting a difference measure threshold for Figure 16. Figure 17 shows D_max as a function of eccentricity after selecting a discriminability index of 0.5 for different alpha values (i.e., the slope of the eccentricity dependence for α, τ, A, C_ki, and E_ki). It is clear from Figure 17 that D_max eccentricity dependence increases as the eccentricity dependence (from all applicable parameters) increases as well. 

Figure 16

 

Model discriminability difference measures as a function of eccentricity for difference ISDs for (a) α = 0 and (b) α = 0.6. (a) The Euclidean distance measure varies erratically for a given ISD at multiple eccentricities in the absence of any space-variant factors. (b) When cortical magnification is present, however, a “layered” structure of the Euclidean difference measure forms across increasing eccentricity as one holds ISD constant. While the exact parameter value chosen for the discriminability index is arbitrary this Figure shows that regardless of the exact discriminability parameter value (and thus the exact values of D_max) the qualitative monotonicity of the eccentricity dependence will be present only in the space-variant case.

Figure 17

 

D_max as a function of eccentricity for the D_max threshold 0.5.

Constant overlap factor

In the following sections we look at individual space-variant factors and their effect on the eccentricity dependence of D_max. 

The model assumes near-constant overlap between receptive fields. That is, for any point in the retinal space there is a constant number of cortical receptive fields that contain that particular point. Bolduc and Levine (1998) use a constant overlap parameter of 35 receptive fields for each point in retinal space. We follow this precedence in determining our default model parameters. 

In order to test the importance of this assumption to the model we vary the overlap parameter while maintaining its constancy across eccentricities. The results can be seen in Figure 18. 

Figure 18

 

Displaying D_max as a function of eccentricity for varying overlap factor. Note that there is no obvious increase in D_max as a function of increased overlap factor.

Note that changing the amount of RF overlap in the absence of other space-variant factors has no pronounced effect on the linear increase of D_max as a function of eccentricity (see Figure 18). For this reason we believe that above an absolute lower limit (two or three receptive fields) the exact overlap parameter is not as important as the assumption of its constancy. 

Number of frames

Todd and Norman (1995) studied how the number of frames presented to an observer changes D_max at multiple eccentricities (Figure 19). In their experiment observers were required to identify the shapes of moving targets and discriminate regions of motion from regions of uncorrelated noise. The maximal range for which an observer could perform a particular task above a set accuracy threshold was taken to be D_max. 

Figure 19

 

D_max as a function of the number of frames for two observers. Figure taken from Todd and Norman (1995) with permission.

The increase in D_max as a function of increasing number of frames appears to be a robust phenomenon. Despite this robustness, however, little is known about its mechanism. In order to determine whether this phenomenon can be explained from lower level visual processes alone we tested the model to an increasing number of frames, finding the associated D_max for the last [pair of] frames in each trial. The model results can be found in Figure 20. 

Figure 20

 

Model D_max as a function of the number of input frames. Note that there is negligible change in D_max as a function of the number of frames. As network activity builds up in the model increasing frame-pairs more easily excite the network above threshold, thus increasing D_max. This effect is offset, however, by the rapid decay of network activity. As shown above, in model simulations using biologically plausible parameters, the rapid decay of network activity was by far the more prominent effect. There was only a very slight increase in D_max as a function of number of frames.

As seen in Figure 20, there is no observable change in model activity with an increase in number of frames shown. The cause of this result is directly related to the fact that the network activity dies down far too quickly to allow for any network activity buildup between frames. Since the temporal model parameters are fit to biological limits found in the early primate visual system it is likely that higher level processing (such as that from area MT) is required in order to obtain the number-of-frames effect on D_max. 

Magno versus parvocellular pathways

In addition to the spatial space-variant properties of the early visual pathway the temporal characteristics are also key in determining the limits of motion perception. In order to test whether eccentricity-dependence of D_max could be related to the temporal response properties of retinal cells as a function of eccentricity, we vary the response time, τ (Figure 21). 

Figure 21

 

D_max as a function of eccentricity while varying τ.

The data show that as τ increases to higher values D_max increases until it reaches a plateau, after which D_max ceases to increase further. The concavity of the eccentricity-dependence curves for varying τ changes as well. This result shows that the network loses the ability to distinguish high speeds of motion for large τ. 

Discussion

The goal of this paper is to explore the importance of individual space-variant visual features in forming cortical motion percepts. These key features include retinal spatiotemporal sampling characteristics, cortical magnification, constant receptive field overlap, and number of input frames. By constructing and testing a model of the early human visual system that incorporates these features we are able to determine a set of minimal requirements for increasing D_max/D_min as a function of eccentricity and the potential role of higher level visual processes motion processing. 

Minimum requirement for the increase in D_max as a function of eccentricity

Baker and Braddick (1985) showed that D_max and D_min increase linearly as a function of eccentricity. The neural substrates responsible for this increase, however, have remained elusive despite the many space-variant aspects of the visual system that are known. One major question that arises is whether the space-variancy present in motion-processing areas, such as MT, are directly responsible for increase in D_max, or if the perceptual phenomena can be entirely accounted for in the early visual stream. 

In the model presented here we show that the combination of faster magnocellular processing, larger peripheral receptive field sizes, and increasing cortical magnification factor is enough to account for current D_max psychometric data. That is, this subset of the early visual stream is sufficient to account for the observed D_max increase as a function of eccentricity in its most basic qualitative form. 

Does cortical receptive field scatter play a role in D_max?

The role of receptive field cortical scatter has been debated. Classically, this scatter is simply considered to be a source of noise to the visual system. Recent research, however, speculates that it could be used to refine visual measurements (Engel et al., 2009, Mishra & Aloimonos, 2009, Tistarelli & Sandini, 1993). Difficulty in determining the exact contribution of cortical RF scatter is due to the complex columnar connections through V1 and the limited data detailing the phenomenon. Despite these difficulties, we are able to determine an order-of-magnitude approach geared toward ascertaining whether RF scatter is important in determining D_max. Under the assumption that cortical receptive field scatter functions as a low pass filter on the retinal input we use a simple columnar model to show that the relative contribution towards D_max due to receptive field scatter is minimal. Further work would be needed to determine whether cortical scatter can serve a refinement role for visual features. 

Role of higher level visual processes

One requirement found from the model in order to obtain increasing D_max as a function of eccentricity is the cortical magnification factor. Cortical magnification, however, does not stop at V1. It is highly possible, then, that cortical magnification to higher visual areas (V1 to V2, etc.) may play a significant role in determining the exact value of D_max at any given eccentricity. It is still not clear, however, what role, if any, motion-specific areas such as MT play on D_max. While it may not be requisite that they be involved in order for a linearly-increasing D_max as a function of eccentricity, the current model can only place constraints on the sufficiency of the early visual system, not in the exclusion of any higher visual area. 

Todd and Norman (1995) found that increasing the number of frames increases D_max. While the model presented here cannot account for this data, one hypothesis is that extrastriate areas may be requisite to explain the increase of D_max as the number of presented frames increases. Our model, bound by parameter limits found in the early visual system, shows a negligible increase in D_max as a function of the number of frames. That is, D_max stays virtually the same even as more frames are shown to the model. One possibility we propose that would account for the model results is that higher level visual areas may be able to pool visual input across larger time scales than can be pooled in the early visual system. Larger integration times can arise due to additional low pass filtering in the temporal domain due to additional synaptic delays in interareal connections or from larger temporal constants present in extrastriate cellular responses. 

Conclusion

D_max and D_min are the maximum and minimum displacements, respectively, to which apparent motion can be observed for a particular stimulus. Thus it follows that while D_max might be the result of long-range motion processing D_min is much more apt to be an acuity measurement. Baker and Braddick (1985) posited that D_min could be the result of the early visual pathway while D_max could be the result of higher level visual processing. The picture drawn here, however, would seem to indicate a more complex arrangement. We posit that the basic D_max eccentricity dependence may well be found entirely within the early visual system, but that the exact discriminability may be modulated by higher visual areas operating on additional features (e.g., motion). 

Acknowledgments

This research was supported in part by CELEST (NSF SBE-0354378 and OMA-0835976), Office of Naval Research (ONR N00014-11-1-0535), and AFOSR FA9550-12-1-0436. 

Commercial relationships: none. 

Corresponding author: Arash Yazdanbakhsh. 

Email: [e-mail]. 

Address: Center for Computational Neuroscience and Neural Technology and Program of Cognitive and Neural Systems Boston University Boston, MA, USA.