Abstract
Recently, we launched Project Prakash with the aim of experimentally exploring the development of visual object concepts in children, following sight onset (Sinha, VSS 2003). To complement these ongoing behavioral studies, we have embarked upon a computational project called Dylan (Dynamic input based Learning in Artificial and Natural systems). The goal of this project is to formulate a cascade of computational processes that together can discover objects in real-world video sequences without requiring pre-normalization of the inputs - a task that our human subjects are adept at performing with a few months of visual experience. While this is an extraordinarily difficult problem, and we are still far from a comprehensive solution, experimental data from infant studies and Project Prakash have allowed us to begin designing Dylan's basic computational architecture. The model comprises four stages: 1. Motion-guided orienting and region-trajectory analysis to determine which region assemblies to bind together, 2. Tracking assemblies to extract temporally extended appearance models (TEAMs) of dynamically transforming objects, 3. Statistical estimation of inter-TEAM correlations across time to infer their predictive dependencies, and 4. Object recognition in new inputs via TEAMs and their mutual correlational structure. This architecture emphasizes the role of dynamic information in the task of object learning, and is based on experimental data showing that motion cues are critical for accurate image parsing by infants as well as sight-restored children. The computations involved in motion-based region binding and tracking are detailed in presentations by Ostrovsky and Balas respectively. This presentation contextualizes all stages and describes the overall results, both successes and failures, of our current Dylan implementation.
The John Merck Scholars Award, The Alfred P. Sloan Foundation