Abstract
While there is no doubt that word-specific and picture-specific operations contribute to language and scene understanding, respectively, the two domains are not completely separate. For example, words can prime object recognition and vice versa. Here we ask if conceptual knowledge of the typical locations of objects in scenes ("scene syntax") affects perceptual and lexical decisions. In two experiments, we presented a preview of an indoor-scene (500ms) followed by a location cue (500ms). Then either an image of an object (Exp.1) or a string of letters (Exp.2) appeared at the pre-cued location within the scene. Observers had to decide as quickly as possible, whether the object was visually distorted (Exp.1) or whether the letter string formed an English word (Exp.2). Objects/words could either be congruent with location in the scene (soap on sink), semantically incongruent (egg on sink), syntactically incongruent (soap on towel rack that is, the soap is appropriately semantically associated with the sink but it is in the wrong location, relative to the sink), or doubly-incongruent (egg on towel rack). In Exp.1, though the task involved only the objects perceptual characteristics, semantic incongruity impeded performance compared to fully consistent control scenes. Syntactic incongruity didn't impede performance. In contrast, in Exp.2, lexical decisions were impeded by both semantic and syntactic incongruity. That is, observers are slower to categorize "EGG" as a word in a semantically incongruent bathroom scene than in a kitchen and slower to accept "SOAP" as a word if presented in a syntactically incongruent position in that bathroom, compared to an appropriate location on top of the sink. We therefore argue that lexical processing is primed not only by semantic associations with scenes or objects, but also by syntactic understanding of the likely position of an object in a scene.
Meeting abstract presented at VSS 2014