September 2018
Volume 18, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2018
Audiovisual combination with temporal correlation and time pressure
Author Affiliations
  • Robert Sekuler
    Volen Center, Brandeis University
  • Yile Sun
    Volen Center, Brandeis University
  • Timothy Hickey
    Computer Science, Brandeis University
Journal of Vision September 2018, Vol.18, 1131. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Robert Sekuler, Yile Sun, Timothy Hickey; Audiovisual combination with temporal correlation and time pressure. Journal of Vision 2018;18(10):1131.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

We built a video game to examine how temporal correlation and increasing time-pressure influenced audiovisual integration. We supplemented behavioral indices of integration with pulse rate measures and EEG from a frontal and temporoparietal electrodes in a wearable sensor band with. METHOD. Visually identical fish swam into view one at a time. Each oscillated sinusoidally in size —either at 5 or 8Hz. Most fish were accompanied by a sound that amplitude modulated —either at 5 or 8Hz. Modulation frequency was either matched to the frequency of visual oscillation (Congruent) or was mismatched to it (Incongruent). A Control condition omitted the sound. Fish of all kinds were randomly intermixed. Subjects were instructed to ignore sounds, categorizing fish solely on their frequency of visual oscillation. In Experiment One, each fish remained visible for two seconds or until the subject's response. The interval between successive fish was fixed at 1.6 sec. In Experiment Two, fish again remained visible for up to two seconds, but the interval between successive fish systematically decreased each 90 seconds, from 1.6 sec down to 400 msec. RESULTS. Experiment One. The Congruent condition produced significantly faster and more accurate responses than other conditions; Incongruent and Control conditions did not differ from one another. Theta band (4-7Hz) power from FP electrodes increased substantially ~300 msec after fish onset, but only for Incongruent fish. Experiment Two. Diminishing inter-fish intervals caused subjects to reduce the time spent observing the stimulus, which increased errors. Mean pulse rate increased modestly, but systematically with time-pressure. CONCLUSIONS. Audiovisual congruence boosted performance, but incongruence had no discernible behavioral effect. However, theta band EEG signals did differentiate conditions, perhaps because mismatched audio and visual signals demanded increased cognitive control. Reduced time between fish induced subjects to needlessly curtail observation times, a self-defeating strategy.

Meeting abstract presented at VSS 2018


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.