Neural model for the representation of static and dynamic bodies in cortical body patches

Abstract

The visual recognition of body poses and body actions is a fundamental visual function for many social species, especially primates. The detailed underlying neural circuitry is currently not well understood. We propose a physiologically-inspired hierarchical neural model for the recognition of static and moving bodies that attempts to account for electrophysiological and fMRI data. METHODS: Our model combines a feed-forward deep neural network (VGG-19) with a neurodynamical model that reproduces the activation dynamics at the single-cell level in the visual and premotor cortex (Giese and Poggio, 2003). The lower levels of the visual hierarchy were modeled by the layers up to the Conv5.1 layer of VGG-19 (pretrained on ImageNet). These outputs were further processed by a subnetwork consisting of radial basis function units in order to model invariance properties of real cortical neurons in body-selective patches. We propose a recurrent neural network (neural field) (Giese and Poggio, 2003; Fleischer et al., 2013) as the mechanism that accounts for the observation that some body-selective neurons show temporal sequence-selectivity. RESULTS: We tested the model using static images of body poses as well as real videos of macaques and humans performing different actions, which were also used in electrophysiological and fMRI experiments by our consortium. The model successfully recognizes the actions of humans and macaques in the stimuli, including the videos. Like real neurons, the model shows (partial) generalization to silhouette stimuli after being trained on realistic stimuli. It predicts a traveling pulse of activation in populations of sequence-selective body-selective neurons, which is strongly attenuated if the sequential order is reversed. CONCLUSIONS: The model provides an account for several aspects of recently acquired data on the neural representation of bodies in body-selective single units present in body patches in the macaque visual cortex.