A critical test of deep convolutional neural networks’ ability to capture recurrent processing using visual masking.

Abstract

Recurrent processing is a crucial feature in human visual processing supporting perceptual grouping, figure-ground segmentation, and recognition under challenging conditions. Most researchers agree on a need to incorporate recurrent processing in deep convolutional neural networks, but the computations and implementation of recurrent processing remains unclear. In this paper, we tested the ability of deep residual networks (ResNet) to capture recurrent processing, as ResNets’ computations have been shown to be equivalent to unrolled time steps of standard recurrent neural networks. We used ResNets of varying depths (to model varying levels of recurrent processing) to explain (electroencephalography) brain activity within a visual masking paradigm. A total of 62 humans and 50 ResNets completed an object recognition task. We replicated effects from an earlier study showing that deeper networks performed similarly to humans under undisrupted viewing conditions (i.e. unmasked); whereas, shallower networks performed similar to humans in disrupted viewing conditions (i.e. masked). ResNets were also able to capture the difference in brain activity between unmasked and masked trials, showing a larger processing peak at ~180ms from stimulus onset in unmasked trials compared to masked trials. Last but not least, deeper networks (ResNets-10, 18 and 34) also explained more variance in brain activity compared to shallower networks (ResNets-4 and 6). Thus, we conclude that recurrent processing in ResNets captures parts of recurrent processes in humans.