September 2015
Volume 15, Issue 12
Vision Sciences Society Annual Meeting Abstract  |   September 2015
A bias-free measure of crossmodal audiovisual action adaptation
Author Affiliations
  • Nick Barraclough
    Department of Psychology, University of York, Heslington, York, YO10 5DD, UK
  • Bruce Keefe
    Department of Psychology, University of York, Heslington, York, YO10 5DD, UK
  • Steve Page
    Department of Psychology, University of Hull, Cottingham Road, Hull, HU6 7RX, UK
Journal of Vision September 2015, Vol.15, 716. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Nick Barraclough, Bruce Keefe, Steve Page; A bias-free measure of crossmodal audiovisual action adaptation. Journal of Vision 2015;15(12):716.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Actions can be perceived through multiple sensory modalities including vision and audition. It is currently uncertain if audiovisual actions are coded multimodally or in discrete unimodal areas. In order to test if actions are coded multimodally, we used a crossmodal adaptation paradigm to test whether the perception of hand action sounds was influenced by prior adaptation to visual and auditory actions. We generated an action sound continuum between the sound of a hand knocking and slapping by blending the sounds between the two action extremes. Following adaptation to auditory knocks, subsequent ambiguous test sounds appeared more like slaps; following adaptation to auditory slaps, subsequent ambiguous test sounds appeared more like knocks. Smaller but significant auditory aftereffects were generated by adaptation to purely visual information, whilst visual adaptation to visually presented orthographic action information did not generate auditory aftereffects. We also tested whether crossmodal adaptation improved action sound sensitivity around the adapting stimulus to rule out the possibility of observer response bias (Morgan 2013). Just-noticeable differences (JNDs) were calculated for knock sounds after participants adapted to auditory, visual, and audiovisual knocks, or did not adapt. Participants completed a two-interval forced-choice (2-IFC) adaptive staircase procedure, in which they judged which sound was more like a knock. JNDs were derived from a psychometric function fitted to the data. Auditory, audiovisual, and importantly visual only adaptation improved sensitivity to knock sounds when compared to no adaptation. A further experiment confirmed that auditory sensitivity to knock sounds was most enhanced following adaptation to visual knock actions compared to visual slap actions. These results suggest that adaptation to actions, irrespective of the adaptor modality can selectively enhance mechanisms underlying the coding of actions to improve perceptual sensitivity. Such crossmodal action aftereffects may result from adaptation of multimodal neurons tuned to specific actions.

Meeting abstract presented at VSS 2015


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.