Abstract
Computations that are suboptimal for laboratory tasks may help reveal the ecologically relevant tasks that perception may be optimized for. We examined how humans perform spatial averaging, a fundamental computation that is useful for integrating noisy local signals into more stable global estimates. We previously reported that human performance is substantially less accurate than the ideal observer in spatial averaging tasks in both the luminance and stereoscopic domains. In our tasks, the sample mean of the stimulus defines the correct choice, so the ideal observer computes its estimate from a simple average across the stimulus. Ideal observers without internal noise can perform the task without error. The observed patterns of human suboptimality could not be accounted for by fixed suboptimal receptive fields or simple forms of internal noise. Indeed, humans made consistent errors across repeated presentations of identical stimuli that could only be explained with nonlinear mechanisms. Here, we examine the nature of these nonlinearities, and determine whether they are sensitive or insensitive to spatial patterns in the stimuli. In two double-pass experiments, human observers judged the average luminance or average stereoscopic depth of nine adjacent horizontal bars relative to a reference surface. In the first experiment, observers responded to repeated presentations of identical stimuli. In the second experiment, observers responded to repeated presentations that were spatially shuffled. Across three observers in both the luminance and stereoscopic tasks, spatial shuffling reduces choice consistency by 60% on average. Thus, both pattern-insensitive and pattern-sensitive nonlinearities are at work. The pattern-insensitive nonlinearity implements a form of outlier down-weighting; the pattern-sensitive nonlinearity encodes the spatial interaction among nearby signals. These compulsory nonlinear mechanisms, while suboptimal in our laboratory task, may reflect an integration strategy that improves performance in more natural contexts cluttered with object boundaries and illumination variation.