Abstract
Problem. Evidence suggests that cells exist in cortical area STS which are selective to the motion of animate objects and show an increase in activity when characteristic poses of the corresponding temporal sequence are presented [Jellema & Perrett, Neurophysiologica 41, 2003]. In addition, activities in MT/MST are enhanced when still images with implied motion are presented [Kourtzi & Kanwisher, J. Cogn. Neurosci.13, 2000]. It is still unclear how and to which extent form and motion information contribute to the generation of the underlying mid-level visual representations. Method. We propose a neural model consisting of two separate processing pathways for segregated form and motion processing extending work by Giese & Poggio [Nat. Rev. Neurosci.4, 2003]. Prototypical form representations (area IT) and optical flow patterns (area MST) are learned using a competitive Hebbian learning scheme with a trace mechanism. The activities of form and motion prototypes converge in area STS where their temporal correlations are learned by a combined bottom-up and top-down learning. Top-down weights establish a prediction mechanism enabling the sequence selectivity of the STS cells. A modulatory mechanism which favors prototypes co-occuring with local minima in the motion energy is responsible for the selection of key poses in a sequence. For the signature of walking this condition is satisfied for strongly articulated body poses. Results and Conclusion. The proposed model is capable of learning action sequences and suggests convergent form-motion interactions in action and activity recognition. Learning is driven by realistic input sequences and their processing along the different model pathways. After learning, form-driven bottom-up input activation leads to increased activation of STS cells. Motion driven MST and STS cells show an activation even if there is only form information present in the signal thus giving an explanation of the neural basis of implied motion in vision.
Meeting abstract presented at VSS 2012