Abstract
People often examine textured surfaces by rubbing them, producing sounds characteristic of each texture. Because people move their fingers at a relatively constant speed, we predicted that modulation frequencies in vision and audition may be representationally associated. For example, if a texture is coarse, it will be low in spatial frequency and rubbing it will generate a sound with low-frequency amplitude modulation; if a texture is fine, it will be high in spatial frequency and rubbing it will generate a sound with high-frequency amplitude modulation. If these associations result in auditory-visual integration, one would predict that (1) observers should be able to consistently match each spatial frequency with an appropriate frequency of amplitude modulation, and (2) the ratio of the matched amplitude-modulation frequency to spatial frequency should be relatively constant, and it should agree with the typical speed of tactile exploration. Note that the matching should be based on physical rather than retinal spatial frequency because the sounds generated during tactile explorations depend on the former. As predicted, the matched visual and auditory modulation frequencies were strongly correlated, and the matching did not depend on viewing distance (55 or 110 cm) indicating its dependence on physical spatial frequency. Speeds of tactile exploration inferred from these matches were 2.8, 5.1, and 8.7 cm/s for gratings with physical spatial frequencies of 3.33, 1.25, and 0.33 cm-1, respectively. First, these inferred speeds are reasonable for tactile explorations. Second, the fact that the inferred speed increased for lower spatial frequencies is consistent with the fact that the speed of tactile exploration needs to be increased for slowly undulating textures to generate a reasonably high vibratory frequency for tactile receptors. Overall our results demonstrate an auditory-visual integration in texture perception that is likely to be mediated by the multisensory experience of examining surfaces.