Abstract
The problem of visual recognition entails matching the sensory input with the stored knowledge of semantic information. This process involves a feed-forward component where visual input is processed to extract features characterizing different objects. It is also presumed to involve an opposite processing pathway where high-level features propagate back providing prediction on the kind of visual stimulus presented. This top-down pathway appears to be useful particularly for tasks like processing degraded stimuli. The process by which the forward and backward pathways integrate leading to visual perception is still largely unknown. Here, using a deep neural network (DNN) trained to recognize objects as a proxy for hierarchical neural representations (Horikawa & Kamitani, 2015), we demonstrate that top-down processing pathway attempts to supplement missing visual features. We first trained multivoxel fMRI decoders to predict DNN features of multiple layers for stimulus images. The trained decoders were then used to analyze independent fMRI data collected while viewing pairs of normal and degraded images. Degraded images were created by blurring original images using averaging filters, and by binarizing slightly blurred images using thresholding. We found that decoded features from fMRI responses to degraded images were more correlated to DNN features calculated from the original images than from the degraded (presented) ones. This was especially salient in the lower level DNN representations. We also found that the task of categorizing the visual stimuli increased the correlation difference especially in higher visual areas. These results suggest the operation of the top-down pathway as it attempts to supplement the missing information in degraded images. The effect of the categorizing task may indicate how giving a prior guides the exploration efforts towards successful perception. This DNN-based brain decoding approach may reveal the interactions between the bottom-up and the top-down pathways, providing empirical evidence for existing and novel theoretical models.
Meeting abstract presented at VSS 2017