Abstract
People can categorize scenes accurately and rapidly. Which visual properties do they use to categorize scenes with such efficiency? Here we provide conclusive evidence from computational analysis, behavioral testing, and decoding from neural activity that intact local structure of scenes is essential for human scene categorization. (1) We extracted structural properties of contours (orientation, length, and curvature) and contour junctions (types and angles) from line drawings of natural scenes. Of these properties, orientation contained the most information about scene category that can be exploited computationally. We found, however, that junction properties (requiring precise localization of contours, thus only available locally) generated prediction errors most similar to errors made by humans in a six-alternative forced-choice scene categorization task. (2) To further test their role in scene categorization we selectively perturbed junctions (by randomly shifting contours) and orientation (by randomly rotating the image). Participants categorized rotated scenes more accurately than contour-shifted scenes. More importantly, error patterns of rotated but not contour-shifted scenes correlated with error patterns of intact scenes. (3) How do these manipulations affect the neural representation of scene categories? Using functional magnetic resonance imaging we recorded brain activity of participants passively viewing intact, rotated, and contour-shifted scenes. We could decode viewed scene category from intact and rotated but not from contour-shifted scenes in the parahippocampal place area (PPA), retrosplenial cortex, and the occipital place area. Furthermore, decoding errors in PPA matched behavioral errors if and only if local structure was preserved, i.e., for rotated and intact scenes. We conclude that local structure is essential for scene categorization by humans. Disruption of local structure degrades scene categorization performance and affects category-specific neural activation patterns in PPA. The view that scene perception is chiefly determined by global scene properties needs to be revised in light of these results.
Meeting abstract presented at VSS 2014