Abstract
Previous research indicates that systematic music-to-color cross-modal associations in non-synesthetes are mediated by emotion (e.g., Palmer et al., 2013; Langlois et al., under review; Whiteford et al., VSS-2013). The present research asks whether specific musical features mediate cross-modal associations from music-to-texture by using single-line melodies that vary along highly controlled musical dimensions: register (low/high), mode (major/minor), note-rate (slow/medium/fast), and timbre (piano-sound/cello-sound). We also investigated whether these associations are mediated by emotional and/or geometric factors. First, 46 non-synesthetic participants picked the 3 most-consistent (and later 3 least-consistent) textures for each of 32 variations on a synthesized melody from a 4x7 array of black-and-white textures. Next, they rated each melody, and later each texture along 5 emotional dimensions (Happy/Sad, Angry/Not Angry, Agitated/Calm, Weak/Strong, Harmonious/Disharmonious), and a series of geometric dimensions (e.g., Simple/Complex, Sharp/Smooth, Granular/Fibrous, Curved/Straight, Separate/Connected). For each dimension, we computed Music-Texture-Associations (MTAs) as a weighted average of the ratings of the 3 textures chosen as going best/worst with each melody. Results indicated that cross-modal melody-to-texture associations were emotionally mediated, because the correlation between the emotional ratings of the music and the emotional MTAs of the chosen textures were so high (Angry/Not-Angry=.79, Calm/Agitated=.91, Active/Passive=.76, Harmonious/Disharmonious=.64). Unlike music-to-color associations, the Happy/Sad correlation (r=.31). was not significant. Melodies-to-texture associations were also mediated by shared geometric features (e.g., Sharp/Smooth=.96, .Curved/Straight=.92, Simple/Complex=.89, Granular/Fibrous=.80). Crucially, the cross-modal melody-to-texture associations corresponded to specific musical features. k-means clustering analyses revealed strong timbre and note-rate effects: cello melodies were paired with straight/sharp/fibrous textures, and piano melodies with curved/smooth/granular textures. Different textures were also chosen for different note-rates, with more granular/separate textures being chosen with slow note-rate melodies, and more fibrous/connected textures being chosen with faster note-rate melodies. These clusters show that the note-rate and timbre of a melody can be inferred reliably from visual texture correspondences alone.
Meeting abstract presented at VSS 2015