September 2011
Volume 11, Issue 11
Vision Sciences Society Annual Meeting Abstract  |   September 2011
Footstep sounds increase sensitivity to point-light walking when visual cues are weak
Author Affiliations
  • James Thomas
    Rutgers University, Newark
  • Maggie Shiffrar
    Rutgers University, Newark
Journal of Vision September 2011, Vol.11, 693. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      James Thomas, Maggie Shiffrar; Footstep sounds increase sensitivity to point-light walking when visual cues are weak. Journal of Vision 2011;11(11):693.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Multisensory neurons in superior colliculus display inverse effectiveness (IE); responses to multimodal stimuli are greater when the most effective unisensory stimulus produces a weak response (e.g. Meredith & Stein, 1983). Recently, fMRI evidence for inverse effectiveness has been found in superior temporal sulcus (Stevenson & James, 2009), a region implicated in multisensory integration (e.g. Barraclough et al., 2009) and human action perception (e.g. Blake & Shiffrar, 2008). While the relationship between neural IE and multisensory perception and performance remains unclear, and evidence suggests that not all multimodal processes adhere strictly to the IE rule (Holmes, 2007), behavioral studies have shown that the effect of auditory cues on visual processing is greater when the visual signal is relatively weak (e.g., Collignon et al., 2008).

Previously, we found that sensitivity to point-light walkers improves when paired with footstep sounds. Because sounds' influence on vision depends upon the strength of the visual signal, we hypothesized that heard footsteps should increase detection sensitivity only when the visual stimulus is weak. To test this hypothesis, we analyzed data from Thomas & Shiffrar (2010) to see whether the detectability of the visual signal determined the degree to which footstep sounds improved detection of point-light walkers. We obtained percent correct for each movie presented in silence (range 33–100%). We then split all movies into two groups based on difficulty. A 3 × 2 mixed measures ANOVA with sound condition as the between-subjects factor (silence, tones, footsteps) and difficulty as the within-subjects factor (easy vs. hard) revealed a main effect of sound, difficulty and a sound X difficulty interaction. Subsequent analyses revealed that footsteps improved visual sensitivity only for “hard” movies, supporting our hypothesis. A follow-up experiment replicated and extended these results.

The Simons Foundation: Award #94915. 

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.