Abstract
Categorical distinctions between images---for example shape, color, or object type---have been shown to boost accurate image recall over the short term. Do these distinctions offer a similar recall boost for crossmodal associations?
In this study, 37 participants detected changes in auditory-visual associations in between-category and within-category trial blocks. In both cases, they observed three sound-image pairs in sequence, then after a blank pause judged whether a test sound-image pair matched one of the earlier items. Only image categories were manipulated; all auditory input involved recognizable animal sounds.
Fifteen faces and 15 random polygons were compared for their contributions to crossmodal change detection accuracy. Half the participants performed category-internal change detection with the face set and distinct-category change detection with random polygons, convex shapes, and drawn objects. The other participants had category-internal trials with random polygons and faces in the distinct-category condition.
Although comparisons between gray convex shapes and black concave random polygons might seem more difficult than comparisons between the convex shapes and photographs of faces, crossmodal change detection both within and across categories was better with the random polygons than with the faces, a marginally significant interaction (F(3,2989)=2.5,p=.058). Participants performed two blocks of each experimental condition, and no change detection boost was observed in the later blocks, although reaction time did go down significantly, F(1,2882)=58.09,p<.001.
A recognition test was performed approximately 10min after the change detection protocol, after an unrelated lexical decision task. A technical issue reduced this data set to only 14 participants; members of the group that performed within-category change detection with faces showed better recall of particular facial expressions. Both groups struggled to recall the random polygons at better than chance levels. Thus, the polygons' characteristics that conferred greater crossmodal change detection accuracy---compared to associations with faces---did not correspond to better long-term recall.
Meeting abstract presented at VSS 2012