**Sensory neurons represent stimulus information with sequences of action potentials that differ across repeated measurements. This variability limits the information that can be extracted from momentary observations of a neuron's response. It is often assumed that integrating responses over time mitigates this limitation. However, temporal response correlations can reduce the benefits of temporal integration. We examined responses of individual orientation-selective neurons in the primary visual cortex of two macaque monkeys performing an orientation-discrimination task. The signal-to-noise ratio of temporally integrated responses increased for durations up to a few hundred milliseconds but saturated for longer durations. This was true even when cells exhibited little or no adaptation in their response levels. These observations are well explained by a statistical response model in which spikes arise from a Poisson process whose stimulus-dependent rate is modulated by slow, stimulus-independent fluctuations in gain. The response variability arising from the Poisson process is reduced by temporal integration, but the slow modulatory nature of variability due to gain fluctuations is not. Slow gain fluctuations therefore impose a fundamental limit on the benefits of temporal integration.**

*Macaca mulatta*, one

*M. nemestrina*) were trained to perform an orientation-discrimination task. They were seated with their heads stabilized in a dimly lit, sound-isolated room in front of a gamma-corrected CRT monitor. We presented drifting sinusoidal gratings of varying orientation at an eccentricity of approximately 5° of visual angle for a duration of 500 ms. The stimulus was positioned within the neuron's receptive field. Stimulus orientation varied over a range of 30°. The animals judged the orientation of the stimulus relative to a discrimination boundary which was chosen to correspond to the steepest part of the neuron's orientation-tuning function (estimated by visually inspecting the orientation tuning function measured in a fixation task prior to the decision-making task). After the stimulus had disappeared, the animals communicated their decision via a saccadic eye movement toward one of two choice targets. Stimuli were presented in random order and repeated at least 10 times (for the data presented here, mean = 43 repeats). The animals' behavioral orientation acuity approximated that of human observers tested under similar conditions (Goris et al., 2017).

^{2}), the squared ratio of the mean to the standard deviation of the spike count of a single neuron within a given temporal window. We chose to use SNR

^{2}because a homogeneous Poisson process predicts linear growth in this quantity with time. We computed the

*momentary response*by counting spikes in a sliding 10-ms window and the

*integrated response*by counting spikes in a window of 10, 20, 30 ms, etc., beginning at stimulus onset.

^{2}, whereby the prediction is derived under the assumption that response variability is temporally uncorrelated (see Results). We classified relative reliability as statistically significant if it fell outside of the central 95% of the expected null distribution, computed from 1,000 randomly permuted data sets.

*adaptation*) on temporal integration. We computed the relative strength of the reliability transient by taking the ratio of the maximal SNR

^{2}during the first 100 ms to the mean SNR

^{2}over the remaining 400 ms.

^{2}, the ratio of the squared mean to the variance of the spike count, which we hereafter term

*reliability*. Under our experimental conditions, orientation tuning is stable over time (Mazer, Vinje, McDermott, Schiller, & Gallant, 2002). It is therefore reasonable to harvest the encoded stimulus information by accumulating spikes over the entire duration of the stimulus. In particular, if the mean and variance across trials were identical for all time bins and the spike counts were uncorrelated across time bins, then accumulated reliability would grow linearly with duration. In reality, however, mean and variance usually fluctuate within a trial (Churchland et al., 2010). For neuron 1, reliability measured within 10-ms intervals peaked just after response onset, then quickly dropped and remained low for the rest of the stimulus epoch (Figure 1A, top panel). As a result, if spikes are temporally uncorrelated, reliability of the accumulated response should grow rapidly at first and thereafter at a slower rate (Figure 1A, dashed curve in the bottom panel). For this particular neuron, integrating responses over time yields a reliability that closely follows this prediction (Figure 1A, orange curve in the bottom panel).

^{2}of the spike count, accumulated over discrete bins, is

*N*is the spike count in the

_{i}*i*th time bin,

*T*is the number of time bins being accumulated, and

*E*[·], var[·], and cov[·,·] are the mean, variance, and covariance (respectively) of the binned spike counts. If the counts are uncorrelated, the second term in the denominator is zero and reliability will grow with time. But if responses are temporally correlated, reliability will grow more slowly.

*neurometric*analysis differs from ours in that it critically relies on a comparison of neuronal responses across stimulus conditions, whereas we have so far only considered responses within a single stimulus condition. We wondered whether the effects of temporal integration would be comparable for both measures of signaling capacity. For each neuron, we computed a family of neurometric functions, using different temporal integration intervals (see Methods). Figure 2A shows three such functions, averaged across all neurons recorded from one animal. Prior to averaging, we converted neurometric performance to

*d*′ (Green & Swets, 1966), a statistic whose variance does not depend on performance level. After averaging, we converted

*d*′ back into choice proportion. A steeper slope of the neurometric function corresponds to an increase in neuronal sensitivity for stimulus orientation, and is equivalent to a decrease in orientation-discrimination threshold. As can be seen, increasing integration time from 50 to 232 ms systematically increases the slope of the neurometric function (Figure 2A). However, as was the case for SNR

^{2}(Figure 1C), increasing the integration window beyond 232 ms yielded little further benefit (Figure 2B). This behavior deviates from the traditional prediction that neuronal sensitivity should improve in proportion to the square root of integration time (Figure 2B, dashed curve). Given the similar behavior of both statistics, we hereafter exclusively focus on SNR

^{2}.

*N*is spike count in the

_{i}*i*th time bin,

*μ*is the mean stimulus-driven spike count in that bin, and

_{i}*g*is a stochastic gain signal for that bin, with mean 1 and variance

_{i}*p*< 0.001,

*t*test), but the effects of temporal integration are quite diverse. We found the account of the modulated Poisson model for this diversity to be consistent with the data: Relative reliability decreases with gain fluctuations (Figure 3D,

*r*

_{s}= −0.61,

*p*< 0.001, Spearman correlation).

^{2}during the first 100 ms to the mean SNR

^{2}over the remaining 400 ms. As can be seen in Figure 3E, the association between both factors was weak (

*r*

_{s}= −0.22,

*p*< 0.01, Spearman correlation), probably too weak to be meaningful.

*r*

_{s}= −0.37,

*p*< 0.001, Spearman correlation).

*r*

^{2}= 0.60,

*p*< 0.001).

*T*is small, reliability grows with

*T*at a rate of

*μ*. But when

*T*is large, the denominator is dominated by the gain fluctuations term, and reliability saturates at a level of

*μ*and

*r*= −0.37,

_{s}*p*< 0.001, Spearman correlation). Predictions from the modulated Poisson model also accounted for a substantial proportion of the variance in relative reliability in MT (Figure 6D,

*r*

^{2}= 0.31,

*p*< 0.001). However, model performance for MT was worse than for V1. For the majority of MT neurons, the model underestimated the benefits of temporal integration (Figure 6D), presumably because the saturation of reliability with integration time was less complete than for neurons in V1 (Figure 6A). This might be due to the fact that the moving dot stimuli were not repeated exactly across trials (and thus some response variability arises from temporally independent stimulus fluctuations), or it might signify that the gain varied somewhat within each 2-s trial. Despite this, many features of the MT data were qualitatively consistent with both the V1 data and the modulated Poisson model, including a shift in the factors controlling reliability over the course of the trial (Figure 6E).

*encoding*of sensory information. Previous studies have suggested that the limited benefits of temporal integration in behavioral tasks may arise during the

*decoding*of sensory signals, either through imperfect (“leaky”) integration (Grice, 1972; McClelland, 1979; Busemeyer et al., 2006) or through a policy preference to minimize response latency (Kiani et al., 2008; Drugowitsch et al., 2012). In contrast, our analysis provides evidence for the idea that elementary limitations in sensory encoding may be critical—it may not matter whether integration is leaky or organisms have an urgent need to respond fast, if sensory integration is intrinsically limited.

*Science*, 273, 1868–1871.

*The Journal of Physiology*, 141, 337–350.

*Nature Neuroscience*, 18, 1656–1663.

*Comptes Rendus de Séances de La Société de Biologie*, 37, 493–495.

*The Journal of Neuroscience*, 12, 4745–4765.

*Vision Research*, 41, 1891–1899.

*Neural Network*, 19, 1047–1058.

*Nature Reviews Neuroscience*, 13, 51–62.

*Journal of Neurophysiology*, 99, 1366–1379.

*Nature Neuroscience*, 13, 369–378.

*The Journal of Neuroscience*, 11, 3612–3628.

*Neuron*, 82, 235–248.

*The Journal of Neuroscience*, 36, 1775–1789.

*Journal of Vision*, 15 (16): 8, 1–12, https://doi.org/10.1167/15.16.8. [PubMed] [Article]

*Vision Research*, 35, 2723–2730.

**'**s law for contrast.

*Journal of the Optical Society of America A*, 3, 52–61.

*Nature Neuroscience*, 17, 858–865.

*Psychological Review*, 120, 472–496.

*The Journal of Neuroscience*, 37, 5195–5203.

*Nature Neuroscience*, 14, 239–245.

*Signal detection theory and psychophysics*. New York: Wiley.

*Perception & Psychophysics*, 12, 103–107.

*Proceedings of the National Academy of Sciences, USA*, 112, E6973–E6982.

*Neuron*, 76, 962–975.

*The Journal of Neuroscience*, 28, 3017–3029.

*Neuron*, 87, 644–656.

*Journal of Neurophysiology*, 77, 24–42.

*Proceedings of the National Academy of Sciences, USA*, 99, 1645–1650.

*Psychological Review*, 86, 287–330.

*Nature Neuroscience*, 17, 1410–1417.

*The Journal of Neuroscience*, 24, 3210–3222.

*eLife*, 4, e08998.

*The Journal of Neuroscience*, 16, 1486–1510.

*Nature Neuroscience*, 8, 220–228.

*Vision Research*, 23, 775–785.

*The Journal of Neuroscience*, 23, 3515–3530.

*The Journal of Neuroscience*, 10, 3543–3558.

*Vision Research*, 19, 515–522.

*Nature*, 447, 1075–1080.

*Eye movements and vision*. New York: Plenum Press.

*Nature*, 370, 140–143.

*Neuron*, 89, 369–383.

^{2}as a function of integration time may be written (by substituting Equation 2 into Equation 1) as

*T*is the number of time bins being accumulated,

*μ*is the mean stimulus-driven spike count in the

_{i}*i*th time bin, and

*g*is a stochastic gain signal for that bin, with mean 1.0 and variance

_{i}^{2}is an absolute measure of reliability. We also considered relative reliability of integrated activity for two models of neural response statistics (see Figure 4A). We define relative reliability as the ratio of the reliability in the presence of rate fluctuations to the reliability with no such fluctuations. For the modulated Poisson model, in which rate fluctuations arise from a multiplicative source, relative reliability is