The goal of this study was to test the nature of the memory advantage in synaesthesia. We compared four different types of synaesthetes (27 grapheme-colour, 21 sound-colour-, 25 grapheme-colour-and-sound-colour- and 24 sequence-space synaesthetes) to their matched controls. Recognition memory for three types of stimuli (music, words, colour) was tested. We anticipated a general advantage in memory for synaesthetes and potentially additional synaesthesia-specific benefits. The results showed a general advantage for synaesthesia. Further, a benefit for colour stimuli resulted for grapheme-colour synaesthetes and a benefit for music stimuli resulted for grapheme-colour-and-sound-colour synaesthetes, indicating synaesthesia-type specific effects. These results suggest different mechanisms for the explanation of the memory benefit for different types of synaesthesia such as synaesthesia-related colour expertise for grapheme-colour synaesthesia and additional encoding opportunities for grapheme-colour-and-sound-colour synaesthesia.

Copyright: © 2018 Lunke, Meier. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Expected results for the three possible mechanisms as mean differences between synaesthetic performance and performance of matched controls: a) an inducer-specific, b) a domain-specific and c) a general memory advantage of synaesthetes. GC = Grapheme-colour synaesthetes; SC = Sound-colour synaesthetes; GCSC = Grapheme-and-Sound-colour synaesthetes; SS = Sequence-Space synaesthetes.

Second, so far no study has included inducer-specific, concurrent-specific and unrelated material in a comparable way for different types of synaesthesia. In the present study, we thus compared four types of synaesthesia and matched controls for inducer-related, concurrent-related and synaesthesia-unrelated material in a recognition memory task. Grapheme-colour synaesthetes can be considered as the standard type of synaesthesia, sound-colour synaesthetes are based on a different inducer and grapheme-colour-and-sound-colour synaesthetes represent a multiple type of synaesthesia with different modalities. Additionally, we included sequence-space synaesthetes who have neither graphemes nor sound as inducer, but sequences, and no coloured concurrents, but spatial representations. All participants completed three episodic recognition memory tests, that is, one for words, one for music, and one for colour stimuli. Fig 1 illustrates the extent of the expected advantage compared to matched controls according to the three theories described above. As evidence for an inducer-specific advantage we expected a higher performance for grapheme-colour and grapheme-colour-and-sound-colour synaesthetes regarding word stimuli and for sound-colour synaesthetes and grapheme-colour-and-sound-colour synaesthetes regarding musical stimuli. As evidence for a domain-specific advantage we expected a higher performance for grapheme-colour-, sound-colour- and grapheme-colour-and-sound-colour synaesthetes regarding coloured stimuli and for their inducing stimuli respectively. If synaesthesia would show a general memory advantage, we expected higher performance in all types of stimuli.

To summarize, the reviewed results support a memory advantage for grapheme-colour and for time-space synaesthesia [ 3 ], [ 4 ]. However, they also implicate different benefits for different types of synaesthesia. At this point, no conclusion can be drawn which type of synaesthesia benefits from which of the aforementioned mechanisms–inducer-, concurrent- or domain-specificity or a general advantage. First of all, there is evidence that types of synaesthesia differ. Moreover, it is likely that types tested as grapheme-colour synaesthetes so far have been heterogeneous as well. While some studies did not describe their sample further, other authors included grapheme-colour synaesthetes with colour experiences for auditory graphemes or such with colour experiences for printed graphemes. Still others tested synaesthetes with colour experiences for sequential words like months [ 25 ], [ 26 ],[ 7 ], [ 9 ]. Some synaesthetes do associate their visual concurrents like colours or spatial arrangements and some do project them into space [ 27 ]. Additionally, many synaesthetes do not only show one distinct type of synaesthesia but also other, partly less pronounced types. Synaesthetes with coloured concurrents show for instance a significantly higher prevalence for sequence-space synaesthesia [ 28 ], [ 29 ]. It is thus particularly difficult to create homogenous and distinctive groups of synaesthesia types to detect the possible source of an advantage through the material.

Finally, Ward, Hovard, Jones and Rothen [ 4 ] used inducer-related, concurrent-related, and synaesthesia-unrelated memory tasks, based on words, non-words, scenes and abstract pictures (with and without colours), to compare grapheme-colour and lexical-gustatory synaesthetes. Grapheme-colour synaesthetes outperformed controls in all tasks, strongest for non-coloured fractals. Lexical-gustatory synaesthetes did not show any benefit. This speaks strongly against an inducer-specific benefit and supports a general advantage, but only for grapheme-colour synaesthetes. Further, it supports a mechanism related to coloured concurrents that appears to boost visual perception. Most importantly, it suggests that different types of synaesthesia have different memory advantages. This is consistent with the observation that different types of synaesthetes show preferences for different cognitive styles [ 21 ].

Neither dual-coding, implicit bi‐directionality, nor domain-expertise can account for a benefit for materials completely unrelated to synaesthesia [ 15 ],[ 16 ],[ 17 ]. It is possible, that the richer world of experiences with cascadically triggered associations in a broader semantic network accounts for this seemingly independent benefit, cf. [ 18 ], [ 19 ]. Such an explanation would be consistent with evidence from structural brain imaging suggesting a different, hyper‐connected network‐organization [ 20 ] and complemented by findings of general differences in cognitive style [ 21 ]. Accordingly, the memory advantage in synaesthesia would not necessarily directly be related to inducer and concurrent but rather to wider changes in the synaesthetic brain, related to perception and encoding in general [ 22 ], [ 23 ], [ 6 ]. This theory is in line with several group studies. Rothen and Meier [ 17 ] for instance found that grapheme-colour synaesthetes outperformed their control participants in inducer- and concurrent-related as well as synaesthesia-unrelated tasks of the Wechsler Memory Scale. Gross, Neargarder, Caldwell-Harris and Cronin-Golomb[ 24 ] on the other hand found an advantage for grapheme-colour synaesthetes only for a part of the inducer-related verbal and synaesthesia-unrelated tasks. These results rather support a general benefit but indicate that a benefit may only occur under certain circumstances. Pritchard, Rothen, Coolbear and Ward [ 16 ] tested recognition memory for shape-colour-location combinations. Grapheme-colour synaesthetes outperformed their control participants overall, strongest though when colour was the critical feature. This supports a concurrent expertise explanation. Bankieris and Aslin [ 15 ] compared grapheme-colour synaesthetes performance in learning colour-shape pairings. Synaesthetes outperformed controls which supports a more general memory advantage. However, neither Pritchard et al. [ 16 ] nor Bankieris and Aslin [ 15 ] included any inducer-related tasks.

Another possible explanation for an advantage in memory for both inducer- and concurrent-related material is the specific expertise for the synaesthetic domain. Synaesthetes might benefit from easier encoding and processing of synaesthesia-related stimuli such as colours, or more generally, visually presented stimuli [ 6 ], [ 4 ], [ 9 ]. Additionally, according to the implicit bi-directionality of synaesthesia, an association also exists between the concurrent and the inducer. This association might provide additional retrieval cues on an implicit level for concurrent-related stimuli as well [ 10 ], [ 11 ], [ 12 ], [ 13 ],[ 6 ], [ 14 ]. Such a domain-specific account is in line with results by Yaro and Ward [ 9 ], who found that grapheme-colour synaesthetes outperformed their control participants in the inducer-related Rey auditory-verbal learning task and in concurrent-related colour perception and colour recognition tasks.

According to dual-coding theory the opportunity to encode a stimulus via two pathways increases the chances of remembering it compared to a stimulus that was coded only via one pathway [ 5 ]. The additional memory code (i.e., the synaesthetic concurrent) results in a stronger representation, additional retrieval cues, and accordingly, in a performance advantage for synaesthetes compared to non‐synaesthetes. Naturally, different types of synaesthesia would have benefits in different tasks according to their inducer. Several group studies do support the hypothesis of an inducer-specific benefit. Simner, Mayo and Spiller [ 3 ] found superior memory for inducer-related autobiographical and public dates in time-space synaesthetes. Radvansky, Gibson and McNerney [ 7 ] found superior memory performance for inducer-related words in grapheme-colour synaesthesia. In contrast, Rothen and Meier [ 8 ] did not find an inducer-related memory advantage in grapheme-colour synaesthetes in a free recall test of digit matrices. As, these studies did exclusively test memory for inducer-specific material in one single type of synaesthesia, they cannot inform whether potential benefits would occur for types of synaesthesia with different inducers.

However, only very few studies have considered memory in other types of synaesthesia, and no study so far has used inducer- and concurrent-specific as well as synaesthesia-unrelated material [ 3 ],[ 4 ]. The goal of the present study was to systematically compare different types of synaesthesia with different inducer-concurrent pairings regarding a general, an inducer- and a concurrent-related advantage in memory. Theoretically, several mechanisms might lead to a synaesthetic memory advantage and these might differ in different types of synaesthesia. A memory advantage for material related to the inducer would be consistent with dual‐coding theory as the concurrent experience leads to an encoding via two pathways [ 5 ]. Another possible explanation is the experience-driven expertise for materials of the synaesthetic domain that should be reflected in a memory advantage for stimuli related to inducer and/or concurrent. A third possibility is a general memory advantage through a broader and more connected semantic network in which information can be easily integrated [ 2 ],[ 6 ]. In the following we outline these three possible mechanisms and the evidence supporting each.

Synaesthesia is a rare phenomenon in which the perception of ordinary stimuli (referred to as inducers) triggers untypical experiences (referred to as concurrents). Many different types of inducer-concurrent pairings exist. For example, in grapheme‐colour synaesthesia a digit (e.g., “5”) may trigger a specific colour experience (e.g., “blue). In sound-colour synaesthesia, music, tones and sounds trigger colour experiences. In sequence-space synaesthesia, sequences like days of the week or months elicit visuospatial representations. In case- and group studies grapheme-colour synaesthetes have shown cognitive benefits in many memory tasks (for reviews see [ 1 , 2 ]).

Method

Participants We recruited 102 synaesthetes who had completed the Synaesthesia-Check on our website (www.synaesthesie.unibe.ch) previously, and who had reported consistent colours for words, digits, letters, musical tones, instruments or melodies and/or spatial representations for digits, days, months or years. 102 healthy control-participants matched for age, gender and education were recruited, cf. [30]. The study was approved by the Ethics committee of the Human Sciences Faculty of the University of Bern (#2013-1-272903). Five participants experienced technical problems during different tasks and were therefore eliminated together with their matched controls resulting in a sample size of 97 synaesthetes and 97 control participants. Of the synaesthetes, 27 were grapheme-colour synaesthetes (25 female and 2 male), 21 were sound-colour- (14 female and 7 male), 25 were grapheme-colour-and-sound-colour (21 female and 4 male) and 24 sequence-space synaesthetes (23 female and 1 male). Nine synaesthetes and nine controls were left handed. Table 1 shows the demographic characteristic of all groups. PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Table 1. Age, education and verbal intelligence of each type of synaesthesia and the respective control group. https://doi.org/10.1371/journal.pone.0203055.t001 One-factorial ANOVAs showed no difference between synaesthetes and controls for age, F(1, 192) = 0.04, MSE = 288.90, p = .853, η p 2 < .01 and years of primary education F(1, 192) = 1.34, MSE = 2.50, p = .248, η p 2 < .01. However, there was an effect of Verbal IQ which was due to sequence-space synaesthetes who showed a higher score than their matched controls (t(46) = 2.29, SE = 2.72, p = .014). Prior to participation in the laboratory, participants with colour experiences for graphemes filled out an on-line measurement of consistency. They were presented with 36 black on white graphemes (A-Z, 0–9) in a random order and instructed to choose a colour out of 13 (black, dark blue, brown, dark green, grey, pink, purple orange, red, white, light blue, light green, yellow) or indicate no colour [32], [29]. The mean number of consistent digits and letters was M = 25.19 (SD = 6.97) which is above the cut-off of 20 used by Simner et al. [29] and Rothen and Meier [32]. Two participants did not fill out the on-line measurement. Six of the 52 participants with grapheme-colour synaesthesia scored lower than 20 consistent digits and letters. Notably, three of the remaining six participants had only digit-colour or word-colour synaesthesia. However, we did not exclude any participant as we relied on the subjective experiences of the participants. Notably, a re-analysis without these three participants gave the same pattern of results. For sound-colour synaesthesia, we originally considered using the Eagleman battery [33]. However, we noticed that in this battery, using a simple strategy (low tones/big instrument -> dark colours; high tones/small instruments -> bright colours) leads to a high consistency score (passing synaesthesia criterion). Moreover, sound-colour synaesthesia is a very heterogeneous phenomenon in which for some people pitch relates to colours, for others timbre relates to colours, for others tone intervals relate to colours, etc.. Thus, we decided to rely on the subjective self-reported experiences. For sequence space synaesthesia, participants who experienced spatial representations for sequences were instructed to draw these representations once at the beginning and once at the end of the experiment on-site (about 2h later). Drawings were categorized by an independent rater regarding consistency and complexity to confirm synaesthesia. One synaesthete was rated as inconsistent. However, this participant had indicated where variations occur in her experiences and was consistent in some other. We thus did not exclude her. At the beginning of the laboratory session, both synaesthetes and control participants were asked again whether they experienced any kind of synaesthesia. If any synaesthetic experiences were reported in addition to those described in the questionnaire, participants were–if possible–tested for consistency and either reassigned or excluded. If several types of synaesthesia were present, participants were asked which they would describe as the main type of synaesthesia. Of the grapheme-colour synaesthetes, thirteen reported having additional types of synaesthesia (sequence-space, other: person-smell and situation-smell). One grapheme-colour synaesthete was recategorised as grapheme-colour-and-sound-colour synaesthete. One control participant was recategorised as grapheme-colour synaesthete. Of the sound-colour synaesthetes, thirteen reported additional types of synaesthesia (sequence-space, grapheme-colour, other: ticker-tape, touch-colour, smell-form, two person/memory-colour, taste-colour/pictures). Three grapheme-colour-and-sound-colour synaesthetes were recategorised as sound-colour synaesthetes. Moreover, one control participant was recategorised as sound-colour synaesthete. Of the grapheme-colour-and-sound-colour synaesthetes, fourteen reported additional types of synaesthesia in the laboratory (sequence-space, other: ticker-tape, daytime-feeling, pain-colour, scene-taste/smell, sound-haptic). Three grapheme-colour synaesthetes were recategorised as grapheme-colour-and-sound-colour synaesthetes. Of the sequence-space synaesthetes, three reported additional types of synaesthesia in the laboratory (one grapheme-colour, one grapheme-colour-and-sound-colour, one person/experience-colour). One grapheme-colour and one grapheme-colour-and-sound-colour synaesthete were recategorised as sequence-space synaesthetes. Four control participants were recategorised as sequence-space synaesthetes. To obtain a medium effect size as reported in previous research, e.g. [17], for a repeated measures ANOVA with a three-levels within-subjects factor and 1-β = .80, the G*Power analysis proposed 19.13 participants per group [34]. We aspired to test at least 20 participants per group.

Material Word recognition. The word material was composed according to an earlier study [35] and consisted of high‐frequency and low‐frequency words, all concrete nouns selected from the vocabulary database of the University of Leipzig (http://wortschatz.uni-leipzig.de/). Two counter-balanced study and test lists were constructed. Twenty-four words of each frequency category were used in the study list together with 19 filler words (days and months), the remaining 24 words of each category were used as lures in the recognition test. Mean word-length was 5.42 letters. In the study phase, words were presented in a 32 point black Arial font on a white rectangle. A chart of colours was presented with every word. Each consisted of the 10 colours red, navy, yellow, green, black, white, light blue, purple, pink and brown, illustrated as 2x1.2cm big squares, arranged in an upright rectangle. In the recognition phase, words were presented in an 18 point courier new font centred on a white background. Music recognition. Each piece of music was selected from unfamiliar, rare recordings. They comprised music styles such as Classic, Jazz, Rock, Pop, Metal, Chinese, Indian, and Swiss folklore. Ten sec wav‐files were cut. Two counter-balanced study and test lists were constructed. Twenty-four pieces were used in the study list, 24 additional pieces were used as lures in the recognition test. Loudness was set to a comfortable level for the participants. Colour recognition. Colour patterns were selected for colour recognition. Half were selected from a synaesthesia catalogue (International Congress on Synesthesia, Science and Art, Granada, 2009) and half of the patterns were Mondrian style pictures, each consisting of four differently coloured squares. Two counter-balanced study and test lists were constructed. Twenty-four patterns (twelve of each category) were used in the study list, 24 patterns (twelve of each category) were used as lures in the recognition test. The pictures were presented centred on a white screen (50% height x 50% width) in the study as well as the recognition phase. Verbal intelligence. To assess verbal intelligence, a standardized vocabulary test was used. It consists of 42 trials, each composed of one target word and five distractor pseudo-words and the participant has to select the real world [31].

Apparatus On-site, participants were tested under controlled light conditions with an 85lux/watt lamp with 6400 calvin colour temperature and two standard interior lamps. Stimuli were presented with E-prime 1.2 (https://www.pstnet.com) on a standard 17 inch flat screen. Answers were given on a standard keyboard and sound was delivered via standard Sennheiser stereo headphones. Audio output was set at a comfortable level for the participant and remained unchanged during data collection. For one participant headphones were replaced by speakers due to her sensitivity of the ears.

Procedure Participants received a brief information about synaesthesia and signed a consent form. In the course of the acquisition of demographic data, synaesthetes who experience spatial representations for sequences sketched their representations. In the study phase, participants were first presented with a list of words on the screen one at the time. In addition to each word, a colour palette with 13 different colours was presented and the participant was instructed to select the colour that goes best with the particular word. Words were presented in randomized order for each participant. After a colour was selected the next word appeared immediately. For the music study phase, participants were asked to put on headphones. They were presented with short pieces of music for 10 sec each and were instructed to rate on a seven‐point scale how much they liked it. They were also asked whether they knew this particular piece of music immediately after each stimulus. Musical pieces were presented in randomized order for each participant. For the colour study phase, participants were presented with coloured patterns for 3 seconds each and they were asked to rate how much they liked each pattern on a seven-point Likert scale. Colour patterns were presented in randomized order for each participant. The three study phases were always fulfilled in the same order: words–music–pictures. After a filled retention interval of approximately 60 minutes, including the vocabulary test [31], the memory test phase began. In the word recognition test phase, words were presented, one at the time, in randomized order for each participant at the center of the screen, in black on a white background. Participants were informed that some of the words were old words from the study phase and some were new words not presented before. They were instructed to indicate whether the word was old or new. After a “new” decision, the next word appeared immediately. An “old” decision was given by pressing “b” and a “new” decision by pressing “n”. After an “old” decision, participants were immediately asked to give a Remember/Know judgement. They were instructed to give a “remember” response when they were able to recollect the word from the study phase and to give a “know” response when they were not able to recollect the word, but nevertheless believed that they had seen it in the study phase. They were instructed to press key 1 after each response they had felt they exactly remembered the stimulus and key 2 whenever they had the feeling they remembered the stimulus. After a response was made, the next word appeared. In the music recognition test phase, participants were again asked to put on headphones. They were played pieces of music for 10 sec each and they were informed that some of the pieces had been played before in the study phase (old pieces) and some not (new pieces). The pieces were presented in randomized order for each participant. They were instructed to indicate whether a piece was old or new. Immediately after an “old” decision, participants were asked to give a Remember/Know judgement similar as in the word-recognition phase. In the colour recognition test, coloured patterns were presented, one at the time, in randomized order at the center of the screen. Participants were informed that some of the patterns were old patterns from the study phase and some were new patterns not presented before. Colour patterns were presented in randomized order for each participant. They were instructed to indicate whether a pattern was old or new. Immediately after an “old” decision, participants were asked to give a Remember/Know judgement as in the word-recognition phase. After the three tests were completed, sequence-space synaesthetes were asked to draw their sequences for a second time.