Abstract
Vision is often faced with noisy and uncertain input. A central tenet of many models is that the visual system resolves this uncertainty by combining information in an optimal way. For instance, when prior stimuli are more reliable than present ones, vision is biased towards the past, a strategy consistent with the ideal observer. Recent work on serial dependence, however, challenges this notion, showing that the bias towards prior stimuli is independent of their reliability. Here we demonstrate that serial dependence can even lead to opposite effects from those predicted by the ideal observer. Participants reproduced the average orientation of an ensemble of 36 Gabors shown in the periphery of the visual field. We manipulated uncertainty by intermixing ensembles with high and low orientation variability. Serial dependence, quantified as the bias towards the average orientation shown on the preceding trial, was larger when the uncertainty was high on both the current and, surprisingly, the previous trial. In a further experiment, we introduced performance feedback and manipulated its reliability across blocks, while keeping the stimulus uncertainty constant. Serial dependence was evident exclusively in blocks of trials where participants received mostly poor performance feedback. Together, our findings indicate that serial dependence is ‘state-dependent’: observers combine current and prior stimuli when the perceived difficulty of the task increases, such as during streaks of uncertain stimuli or in the presence of poor performance feedback. We argue that such phenomena, apparently incompatible with a simple ideal observer, can only be explained by taking the internal states of the observer into account, rather than focusing exclusively on the uncertainty in the stimulus.