Abstract
In feature learning, uncertainty about feature values is reduced. Selective attention can help this, implying that agents should focus attention more under greater expected uncertainty about action outcomes. Little work tests this “attention-for-learning” prediction, and in particular it is unknown whether attention-for-learning is sensitive to the degree to which uncertainty can actually be reduced. Here we tested the attention-for-learning hypothesis in a naturalistic learning task that manipulated both reducible and irreducible forms of uncertainty, and quantified the strength of selective attention using attention-augmented reinforcement learning (RL) models.
Human participants performed a 2-AFC object selection task in which multidimensional objects with a particular feature were more likely to be rewarded. Reducible uncertainty was manipulated between blocks by having objects vary along either two or five possible feature dimensions (different arms, body shapes, patterns, textures, or colors). Irreducible uncertainty took the form of different reward probabilities, either 0.70 or 0.85. As expected, when either form of uncertainty was higher, response times were longer, learning was slower, and asymptotic performance was lower. On blocks where one form of uncertainty was high and the other was low, these performance measures did not differ. However model results show that this similar performance was the result of different mechanisms. Specifically, when reducible uncertainty was high and irreducible uncertainty was low, participants had narrower attentional focus and greater exploratory biases than in the opposite condition.
These results demonstrate that attention flexibly adjusts to the specific type of decision uncertainty. When faced with high levels of reducible uncertainty, attention becomes more focused and exploration increases, but the reverse is true for irreducible uncertainty, even when the resulting behaviour is highly similar. Taken together, these findings provide quantitative evidence for flexible adjustment of attention during learning to specific types of experienced uncertainty.