September 2005
Volume 5, Issue 8
Vision Sciences Society Annual Meeting Abstract  |   September 2005
Seeing music, hearing gestures
Author Affiliations
  • Michael Schutz
    University of Virginia
  • Michael Kubovy
    University of Virginia
Journal of Vision September 2005, Vol.5, 649. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Michael Schutz, Michael Kubovy; Seeing music, hearing gestures. Journal of Vision 2005;5(8):649.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Musicians disagree whether it is possible to produce notes of varying durations on percussion instruments. According to some, a longer physical gesture produces a longer note; others maintain gesture length in and of itself has no effect upon duration. In an attempt to reconcile these disparate viewpoints, we investigated the relative contribution of audio and visual information on perceived note duration.

Stroke preparation and release of single undamped short (S) and undamped long (L) notes performed by a world-renowned percussionist served as the base material. These were separated into visual (Sv, Lv) and auditory (Sa, La) components. Additionally, the auditory component of artificially damped notes (Da) provided extremely short auditory information. We generated audio-visual stimuli by crossing the two visual (Sv, Lv) and three auditory (Sa, La, Da) conditions. Subjects were asked to rate the duration of notes presented in audio-alone (A), and audio-visual (AV) conditions. In AV, subjects were instructed to make duration ratings based only on auditory information. To discourage subjects from ignoring visual information all together, we included a second question as to the level of agreement between audio and visual components.


  1. In the A condition, duration ratings of undamped notes (Sa, La) did not differ, although Sa and La were both rated longer than Da.

  2. In the AV condition, ratings of undamped notes were affected by visual (SvSa vs. LvSa), but not auditory (SvSa vs. SvLa) information.

  3. As measured by d', discrimination between damped and undamped auditory was weaker in AV (SvSa vs. SvDa; LvLa vs. LvDa) than A (Sa vs. Da; La vs. Da).

We will propose a reconciliation between two implications of these data: (1) Although the attempt to produce longer notes with longer gestures is an acoustic failure, it is a perceptual success due to the visual influence on perceived duration. (2) The presence of visual information reduces auditory discriminablity.

Schutz, M. Kubovy, M. (2005). Seeing music, hearing gestures [Abstract]. Journal of Vision, 5(8):649, 649a,, doi:10.1167/5.8.649. [CrossRef]

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.