Abstract
Task performance is determined not only by the amount of task-relevant "signal" present in our brains, but also by the presence of "noise", which can arise from multiple sources, such as trial variability (variability under seemingly identical conditions) and nuisance variability (variability that results from changes in the world that are irrelevant for a task). While trial variability is known to be approximately described by a Poisson process, nuisance variability is much less well-understood. To investigate the relative impact of trial and nuisance variation on performance for one complex visual task, we recorded neural responses in inferotemporal cortex (IT) as two monkeys performed an "invariant delayed match to sample task" that required them to sequentially view images and identify when a "target match" appeared despite variation in the objects' positions, sizes and background contexts. Consistent with a representation of target match information in IT, a linear population read-out of the same images presented as target matches versus as distractors (invariant to nuisance variables such as object identity) performed robustly and this IT read-out systematically misclassified conditions when the monkeys made errors. To determine the relative impact of different sources of variability on IT target match performance, we reformulated a measure of single-neuron task performance (d') as a function of the firing rate modulation attributed to signal, trial variability and nuisance variation. Somewhat counterintuitively, we found that the modulations resulting from a subset of nuisance variables were larger than trial variability; however, the overall impact of nuisance variation on d' was much smaller. This is because the impact of nuisance variation depended on the pooled (average) variance, rather than the sum, introduced by all nuisance parameters. These results reveal that even in the presence of large sources of nuisance variation, trial variability can be the primary factor in limiting task performance.
Meeting abstract presented at VSS 2016