June 2007
Volume 7, Issue 9
Free
Vision Sciences Society Annual Meeting Abstract  |   June 2007
Audiovisual congruence and the processing of synchrony in swing groove drumming
Author Affiliations
  • Melanie Russell
    Department of Psychology, University of Glasgow
  • Karin Petrini
    Department of Psychology, University of Glasgow
  • Phil McAleer
    Department of Psychology, University of Glasgow
  • Davide Rocchesso
    Department of Computer Science, University of Verona
  • Sofia Dahl
    Cognitive and Systematic Musicology Lab, School of Music, Ohio State University
  • Carl Haakon Waadeland
    Department of Music, Norwegian University of Science and Technology
  • Federico Avanzini
    Department of Information Engineering, University of Padova
  • Frank Pollick
    Department of Psychology, University of Glasgow
Journal of Vision June 2007, Vol.7, 874. doi:10.1167/7.9.874
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Melanie Russell, Karin Petrini, Phil McAleer, Davide Rocchesso, Sofia Dahl, Carl Haakon Waadeland, Federico Avanzini, Frank Pollick; Audiovisual congruence and the processing of synchrony in swing groove drumming. Journal of Vision 2007;7(9):874. doi: 10.1167/7.9.874.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Single cell data from macaque suggest special processing of the sights and sounds of biological actions (Kohler, Keysers, et al, Science 2002). Recently Arrighi, Alais & Burr (JOV, 2006) have examined this hypothesis using judgments of perceptual synchrony of audio and visual streams of conga drumming as well as with synthetic audio and visual streams. The perception of audiovisual temporal synchrony provides a window on how these two different sensory modalities are integrated. To further investigate the perception of audiovisual synchrony from human movements we first obtained 3D motion capture data (240 Hz) of a drummer performing swing groove drumming with an accent on the first beat at 60, 90 and 120 BPM. This movement data was converted into visual point light displays with points representing the shoulder, elbow, wrist, hand and 2 points on the drumstick and output as 60 Hz quicktime movies. Matching synthetic sounds were obtained by a simulation of the first 25 modes of a circular membrane that took as input the time and impact velocity of a strike and output the resulting audio signal. In one experimental condition the estimated impact velocity was provided to the algorithm and in the other condition the average impact velocity of all strikes was used. This allowed production of one audiovisual stream with a natural covariation of drummer motion and sound present, and another audiovisual stream with this covariation absent. For both the covariation present and absent conditions we presented observers with audiovisual displays of varying levels of positive and negative asynchrony between audio and visual streams and collected forced-choice judgments of synchrony. Our preliminary results indicate that the novel manipulation of eliminating the natural covariation between physical movement and audio signal influences observers' integration of sight and sound.

Russell, M. Petrini, K. McAleer, P. Rocchesso, D. Dahl, S. Haakon Waadeland, C. Avanzini, F. Pollick, F. (2007). Audiovisual congruence and the processing of synchrony in swing groove drumming [Abstract]. Journal of Vision, 7(9):874, 874a, http://journalofvision.org/7/9/874/, doi:10.1167/7.9.874. [CrossRef]
Footnotes
 Wellcome Trust, British Academy
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×