Abstract
Many research questions involve determining whether the visual system represents two stimulus properties “independently” or “invariantly” versus “configurally” or “holistically”. General recognition theory (GRT) is an extension of signal detection theory that provides formal definitions of such concepts and allows researchers to dissociate perceptual from decisional factors in their study. Unfortunately, because GRT reduces the representation of each property to a single “perceptual evidence” variable, it cannot provide insight on exactly how the representations of two or more properties interact. Here, we link GRT to the linear-nonlinear observer model that is the basis of classification image techniques, to allow for the study of representational separability and configurality. We define template separability as a form of independence at the level of the perceptual templates assumed by this model, and link it to perceptual separability from the traditional GRT. Simulations show that their relation depends critically on stimulus factors, which should be taken into account when making conclusions about separability and configurality. Commonly used naturalistic stimuli, such as faces, readily produce patterns of interactivity in a GRT model even when there is no perceptual interaction in the underlying observer model. The theory can also account for reports of unexpected violations of separability found in the literature (e.g., between line orientation and size). Finally, we estimate classification images for several face identity and expression tasks (discrimination of identity, happiness, and sadness), and use them to re-interpret the results of GRT-based analyses. We show that our analyses using observer models offer two advantages over traditional GRT analyses of perceptual independence: (1) they provide information about external sources of interaction between properties, which are usually confused with true representational interactions, and (2) they provide precise information about how one stimulus property influences sampling of information about another when true interactions are at work.
Acknowledgement: Research reported here was supported by the National Institute Of Mental Health of the National Institutes of Health under Award Number R21MH112013. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.