December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Multisensory processing supports deep encoding of visual objects
Author Affiliations
  • Shea E. Duarte
    University of California, Davis
  • Joy J. Geng
    University of California, Davis
Journal of Vision December 2022, Vol.22, 4152. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Shea E. Duarte, Joy J. Geng; Multisensory processing supports deep encoding of visual objects. Journal of Vision 2022;22(14):4152.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Visual object recognition memory can be improved when an object is encoded alongside its characteristic sound (a dog and a bark). Recent research showed that this redundant auditory information specifically benefits recollection-based recognition memory, suggesting that the recognition enhancement is based on memory for specific details of the perceptually encoded event rather than general object familiarity (Duarte et al., 2021). While previous work has focused exclusively on audiovisual memory effects on individual objects, recollection-specific effects may be impacted by the presence of additional items in the visual field at encoding. In the present work, we investigated whether this recollection improvement is impacted by the presence of a second visual object at encoding. Participants performed an audiovisual encoding task in which pairs of visual objects were presented with a sound that was congruent with one of the objects or a control white-noise sound. Participants reported whether just one of the objects (indicated by a retroactive cue) would fit in a suitcase (Experiment 1) or whether the two items were related (Experiment 2). For both experiments, they performed a remember/know recognition task for each individual visual item after the perceptual encoding. Results from Experiment 1 (n=50) replicated the finding that recollection was improved for objects paired with congruent sounds at encoding relative to those paired with control sounds, even with an additional visual object present at encoding. Experiment 2 (n=50) showed that when participants are required to consider the relationship between the two visual items at encoding, the memory benefit of audiovisual processing is mitigated such that recollection and familiarity-based recognition are not different between conditions. These results suggest that multisensory processing supports visual memory by facilitating elaboration on an object’s identity, and that this facilitation is mitigated when elaboration on both an audiovisual and visual stimuli is required by task demands.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.