Abstract

Previous research has shown that auditory-visual cross-modal associations for complex music in non-synesthetes appear to be mediated by the emotional connotations of the stimuli (Palmer & Schloss, in review), rather than (or in addition to) exhibiting a direct sensory correspondence, as has been found for more basic auditory features such as pitch or loudness (Marks, 1978). In this study, we investigated a class of musical stimuli of intermediate complexity: namely, instrumental timbres. In a series of experiments, participants indicated what colors, edge contrasts, and onset dynamics in visual displays "went best" with the sound of 17 different common musical instruments (e.g., violin, piano, marimba). We found evidence for emotional mediation in non-synesthetes’ color choices, in that participants tended to choose colors whose emotional quality was similar to the rated emotional quality of the sound (e.g., "happy" looking colors, such as saturated red, were chosen for "happy" sounding instruments, such as a harpsichord). Systematic sound-to-sight correspondences were also evident for timbral attack and decay time in non-synesthetes’ choices of "best fitting" edge contrasts (low to high spatial frequencies) and dynamic onsets (temporal dynamics of contrast) of circular shapes. In contrast, timbre-color synesthetes, who chose the colors most similar to the colors they experienced to the sounds, reported color experiences qualitatively different from those made by non-synesthetes. In particular synesthetes showed consistently weaker effects of emotional mediation and greater effects of low-level perceptual features. These results indicate that, although both groups give consistent patterns of responses, synesthetic color experiences to instrumental sounds appear to be driven more strongly by sensory-perceptual features and less strongly by semantic and emotional factors than is typical of non-synesthetic color associations to those same sounds.

Meeting abstract presented at VSS 2013