Abstract
We built a video game to examine how temporal correlation and increasing time-pressure influenced audiovisual integration. We supplemented behavioral indices of integration with pulse rate measures and EEG from a frontal and temporoparietal electrodes in a wearable sensor band with. METHOD. Visually identical fish swam into view one at a time. Each oscillated sinusoidally in size —either at 5 or 8Hz. Most fish were accompanied by a sound that amplitude modulated —either at 5 or 8Hz. Modulation frequency was either matched to the frequency of visual oscillation (Congruent) or was mismatched to it (Incongruent). A Control condition omitted the sound. Fish of all kinds were randomly intermixed. Subjects were instructed to ignore sounds, categorizing fish solely on their frequency of visual oscillation. In Experiment One, each fish remained visible for two seconds or until the subject's response. The interval between successive fish was fixed at 1.6 sec. In Experiment Two, fish again remained visible for up to two seconds, but the interval between successive fish systematically decreased each 90 seconds, from 1.6 sec down to 400 msec. RESULTS. Experiment One. The Congruent condition produced significantly faster and more accurate responses than other conditions; Incongruent and Control conditions did not differ from one another. Theta band (4-7Hz) power from FP electrodes increased substantially ~300 msec after fish onset, but only for Incongruent fish. Experiment Two. Diminishing inter-fish intervals caused subjects to reduce the time spent observing the stimulus, which increased errors. Mean pulse rate increased modestly, but systematically with time-pressure. CONCLUSIONS. Audiovisual congruence boosted performance, but incongruence had no discernible behavioral effect. However, theta band EEG signals did differentiate conditions, perhaps because mismatched audio and visual signals demanded increased cognitive control. Reduced time between fish induced subjects to needlessly curtail observation times, a self-defeating strategy.
Meeting abstract presented at VSS 2018