Abstract
The dorsal and ventral streams of visual processing are known to be interconnected, but there has been surprisingly little investigation of how these systems interact on a cellular level. To explore this issue, we trained two monkeys to perform a discrimination task using animated, visually rich stimuli, and recorded from cells in inferior temporal cortex and the superior temporal sulcus while they performed this task. We presented the monkeys with eight rendered bipedal figures that could perform eight animated motions. The monkeys were required to respond according to the action being performed, regardless of the model's identity. Each action sequence lasted about 1.5 seconds, and started and ended with the same neutral pose. Learning curves indicated that the monkeys understood the task and were able to transfer their knowledge to new models performing the same actions. Using mutual information measures, we found that many of the recorded cells were selective with regard to which model was visible, while another large set of neurons conveyed information about the action being performed; these two sets had moderate overlap. Action-selective neural responses might arise from sensitivity to one or more the following: individual still images that form the action sequences, particular collections of images appearing within a short period of time, or the actions' trajectories. Model-selectivity did not increase after actions began, which suggests that selective responses to actions were not driven solely by single views. Further work is needed to determine whether and how much each mechanism contributes. We have shown that single temporal neurons can carry information about the visual objects that populate our world and about what they are doing.
James S. McDonnell Foundation and NIH R01-EY014681