Two views of the world: Active vision, attention, and perception and action loops in real-world interaction

Abstract

We report new findings using a novel method that seeks to describe the visual learning environment from a young child's point of view. The method consists of a multi-camera sensing environment consisting of two head-mounted mini cameras that are placed on both the child's and the parent's foreheads respectively. Two video streams from two first-person views were recorded simultaneously and as well as their head and hand movements in the interaction. Computational and data mining techniques were applied to detect various objects in the two views, and then discover reliable visual and perceptual patterns from these simultaneous views. In addition, Itti's saliency map algorithm was also employed to find saliency areas/pixels in terms of motion, orientation and intensity. The main result is that the adult's and child's views are fundamentally different in (1) the spatial distributions of hands and everyday objects in the child's visual field and where they are in the parent's field; (2) the salience of individual objects and hands in those two visual fields; and (3) the temporal dynamic structures of objects and hands in two views. Moreover, by correlating visual information with body movement data, we also found that the dramatic difference in two views is largely due to head and hand movements of both the young child and their social partner. In a recent study, we also recorded momentary eye movements from the parent which allowed us to pair the parent's visual attention with the child's visual attention, and further analyze how they jointly attended to the same objects and how one person's visual attention is dynamically coupled with the other person's visual attention. These findings have broad implications for how one studies and thinks about the critical role of embodied active vision and selective attention in various cognitive learning tasks and developmental processes.