Is attention essential for inducing synesthetic colors? Evidence from oculomotor distractors

Abstract

In studies investigating visual attention in synesthesia, the targets usually induce a synesthetic color. To measure to what extent attention is necessary to induce synesthetic color experiences, one needs a task in which the synesthetic color is induced by a task-irrelevant distractor. In the current study, an oculomotor distractor task was used in which an eye movement was to be made to a physically colored target while ignoring a single physically colored or synesthetic distractor. Whereas many erroneous eye movements were made to distractors with an identical hue as the target (i.e., capture), much less interference was found with synesthetic distractors. The interference of synesthetic distractors was comparable with achromatic non-digit distractors. These results suggest that attention and hence overt recognition of the inducing stimulus are essential for the synesthetic color experience to occur.

Introduction

Synesthesia is the phenomenon in which stimulation of one sensory modality leads to reports of “extra” automatic, involuntary experiences in a second modality. It can occur within a given sensory modality or between any two sensory modalities, although some combinations are more common (e.g., musical sounds giving rise to colors) than others (e.g., smells giving rise to tactile sensations; Cytowic, 1993). Even though synesthesia has become a topic of considerable scientific interest, the mechanisms underlying the automatically triggered percept are as yet not fully understood. 

In this paper, we focus on color graphemic synesthesia, wherein specific colors are associated with viewing particular letters or numbers. On a variety of visual tasks, synesthetic colors behave like real colors in that they can, for example, promote visual grouping (Kim, Blake, & Palmeri, 2006) and can trigger the Stroop effect (Cohen Kadosh & Henik, 2006; Dixon, Smilek, Cudahy, & Merikle, 2000; Mattingley, Rich, Yelland, & Bradshaw, 2001; Mills, Boteler, & Larcombe, 2003). At the same time, there are other instances where synesthetic colors do not influence performance on visual tasks whereas real colors significantly impact performance on those tasks (Edquist, Rich, Brinkman, & Mattingley, 2006; Gheri, Chopping, & Morgan, 2008; Hong & Blake, 2008; Rothen & Meier, 2009). These negative results may say something about salience of synesthetic colors or about the level of processing at which they arise. Alternatively, one could construe these negative results as evidence against the truly perceptual in nature of synesthesia, a view endorsed by some (Gheri et al., 2008). 

One unanswered question concerning color graphemic synesthesia concerns the extent to which visual attention is necessary for a synesthetic percept to occur. It might be that a synesthetic color is experienced only when the inducing stimulus receives focal attended. So far, studies that have employed visual search paradigms to examine the role of attention in synesthesia are largely inconsistent (Edquist et al., 2006; Laeng, Svartdal, & Oelmann, 2004; Palmeri, Blake, Marois, Flanery, & Whetsell, 2002; Sagiv, Heer, & Robertson, 2006; Smilek, Dixon, & Merikle, 2003). Some studies have shown that synesthetic color experiences may improve visual search performance by means of a smaller effect of set size on search time and faster responses with target-distractor pairs that induced hues of different color categories (Palmeri et al., 2002; Smilek et al., 2003). This suggests that color is bound to visual forms sufficiently early in processing to make target searches more efficient, maybe even pre-attentively. Other studies, however, have not observed this improvement (Edquist et al., 2006; Rothen & Meier, 2009; Sagiv et al., 2006) or have observed improvement only when the synesthetic target was close to the focus of attention (Laeng et al., 2004; Sagiv et al., 2006). These studies suggest that attention is necessary for integrating color and shape in synesthesia. 

To pursue this question of the role of visual attention in the induction of a synesthetic color experience, the current study has employed a different methodology that does not rely on subjective report. As attention and eye movements are generally considered to be strongly linked (e.g., Rizzolatti, Riggio, Dascola, & Umiltá, 1987; Van der Stigchel & Theeuwes, 2007), we used an eye movement paradigm called the “oculomotor distractor task” (Levy-Schoen, 1969; Walker, Deubel, Schneider, & Findlay, 1997) to gain more insight in the attentional processes underlying synesthetic color experiences. In this paradigm, a target and a single distractor are presented simultaneously and participants have to make an eye movement to the target while ignoring the distractor. When the distractor resembles the target, the distractor evokes strong interference (Ludwig & Gilchrist, 2002, 2003); the amount of erroneous eye movements directed to the distractor (i.e., “capture”) has been shown to be higher when the distractor has the same color as the target (Ludwig & Gilchrist, 2002). 

In the present experiment, synesthetes will be required to make an eye movement to a colored target, while a single task-irrelevant distractor will be presented in half of the trials. Several types of distractors will be used: first, colored graphemes sharing the same hue as the target; second, gray graphemes, inducing a synesthetic experience that is comparable in hue with the target (“synesthetic distractor”); third, colored abstract shapes sharing the same hue as the target; and fourth, gray abstract shapes. 

Because the percentage capture is higher when a distractor has the same hue as the target (Ludwig & Gilchrist, 2002, 2003), capture is expected to be higher for colored abstract shapes than for gray abstract shapes. Importantly, if the synesthetic color percept in this experiment is equivalent to the normal color perception, the percentage capture for a synesthetic distractor will be higher than for gray abstract shapes. 

Method

Participants

Eight synesthetes (one projector, seven associators), aged between 19 and 55 years old (mean age = 33.2 years, SD = 13.0 years), served as paid volunteers. Seven participants were female. All reported having normal or corrected-to-normal vision. They were naive as to the purpose of the experiment. All participants gave their informed consent prior to their inclusion in the study. 

Apparatus

Eye movements were registered by means of an infrared video-based eye tracker (SR Research Ltd, Canada). The Eyelink1000 system had a 1000-Hz temporal resolution and a spatial resolution of 0.5°. The left eye was recorded and analyzed. The participant's head was stabilized using a chin rest. The distance between monitor and chin rest was 65 cm. 

Stimuli and procedure

To determine which digits and corresponding synesthetic hues were to be used in the experiment, synesthetes started this experiment by indicating which exact synesthetic hues were associated with the digits 1 to 9 using digital image-editing software (see Figure 1A). This test was executed on a gray background (CIE xyY chromaticity coordinates of 0.285, 0.307, 9.6). After this test, the experimenter selected the two digits with the most prototypical red and green hues. These two digits and corresponding hues were used in the experiment. This test was repeated after the experiment to check for consistency of the answers. 

Figure 1

 

(A) Digit distractors used in the experiment. (B) Example of a trial sequence.

See Figure 1B for an illustration of the display sequence. All figures were presented on a gray background (same as above). The stimuli were either presented in the exact hue as the synesthetic experience, as indicated by the individual synesthetes in the synesthetic hue test, or in light gray (CIE xyY chromaticity coordinates of 0.285, 0.312, 39.0). Each trial started with the presentation of a light gray “plus” character (1.1° × 1.1°) in the center of the screen that was used as the fixation stimulus. After a random interval between 900 and 1300 ms the target (a colored disc with a diameter of 1.4°) appeared. The target was presented at one of six possible locations (1, 3, 5, 7, 9, or 11 o'clock) on an imaginary circle (radius of 10.8°) around central fixation point. The hue of the target was one of the two hues as determined by the synesthetic hue test. Therefore, on any given trial, the target was one of two colors. In half of the trials (144), a distractor (size = 1.4° × 1.4°) appeared, simultaneously with target onset. The distractor was either the digit associated with the target color (72) or an abstract stimulus (72). The distractor was positioned at one of six possible locations (2, 4, 6, 8, 10, 12 o'clock; i.e., 12 chromatic (6 digits, 6 non-digits) and 12 achromatic (6 digits, 6 non-digits) distractors per location) on the same imaginary circle as the target locations. The digit distractors were typical digital-type characters (stroke width = 0.14°) of which the center was positioned 10.8° from fixation. The color of the distractor was either light gray (“achromatic”) or the same hue as the target in that particular trial (“chromatic”). In contrast to the experiments of Ludwig and Gilchrist (2002, 2003), there were no trials in which the distractor appeared with a color different from the target, except for the trials in which the distractor was light gray. 

Participants were instructed to fixate the central fixation stimulus until target onset and to then move their eyes to the target location. It was stressed that one had to make a single accurate saccade toward the target element. Participants heard a short tone when the saccade latency was higher than 600 ms or shorter than 80 ms. 

The experiment consisted of a training session of 32 trials and an experimental session of 288 trials. The sequence of trials was randomized for each participant. 

After the experiment, two additional control experiments were run. The first control experiment was to test whether the digit distractors were identifiable at 10° from fixation. Synesthetes were asked to indicate the digit that was presented as a distractor while fixating the central fixation stimulus. The location of this digit was the same as the locations used for the distractors. Digits were presented for 300 ms. This control experiment consisted of 50 trials, and both digits used as distractors were presented. 

The second control experiment was to test whether a synesthetic percept could be obtained when presenting digits in the periphery, and whether the percept would be different from a synesthetic percept associated with viewing a centrally presented digit (Ramachandran & Hubbard, 2001). Again, synesthetes were asked to fixate the central fixation stimulus and report whether a synesthetic percept occurred. Digits were presented at 10° from fixation. The digits remained on screen for 300 ms, after which the verbal report was made to the experimenter. We realize that this test, unlike our eye movement technique, relies on subjective report, but the results nonetheless allow us to infer whether synesthetic impressions vary with retinal eccentricity in our participants. 

Data analysis

Saccade latency

We measured saccade latency to investigate whether the presence of a distractor resulted in the expected higher saccade latencies compared with when the distractor was absent (Walker et al., 1997). Saccade latency was defined as the interval between target onset and the initiation of a saccadic eye movement. Only correct saccades to the target location were analyzed. Eyelink software identified saccade initiation using a 22-deg/s velocity and an 8,000-deg/s² acceleration criterion. Per participant, saccade latencies lower than 80 ms and higher than 600 ms or more than two and a half standard deviations away from mean latency were removed from the analysis. Moreover, trials in which no saccade or in which the first saccade was too small (<3°) were excluded from analysis. A paired t-test was used to analyze the differences in latency for distractor absent versus present trials. In addition, in distractor present trials, an analysis of variance (ANOVA) with Distractor Color (achromatic vs. chromatic) and Distractor Identity (digit vs. non-digit distractor) as factors was performed to analyze possible latency effects. 

Capture

Regarding the percentage capture, the landing location of the first eye movement was computed. When the endpoint of the first saccade had an angular deviation of less than 15° from the center of the target or the distractor, the saccade was classified as landed on the target or the distractor, respectively. In other situations, the saccade was classified as an error and not analyzed. An analysis of variance (ANOVA) with Distractor Color (achromatic vs. chromatic) and Distractor Identity (digit vs. non-digit distractor) as factors was performed to analyze the percentage capture. 

The exclusion criteria resulted in a total loss of 16.6% of experimental trials (11.1% were trials in which participants did not adequately fixate the central fixation stimulus before target onset). In the first control experiment, trials with eye movements with an amplitude of more than 3° were excluded (9% of the trials). In the second control experiment, none of the trials were excluded; participants fixated the central fixation stimulus in all trials. 

Results and discussion

Saccade latency

To determine whether the distractor had an effect on latencies of saccades to the target, latencies on distractor present and distractor absent trials were compared. As expected, latencies on distractor present trials were longer (mean = 253.5 ms; SD = 27.4 ms) than on distractor absent trials (mean = 224.6 ms; SD = 20.5 ms; t(7) = 3.64; p < 0.01). This was confirmed by the individual subject analyses, which revealed that this effect was significant in 6 out of 8 synesthetes (see Table 1). 

Table 1

 

Individual statistics of the mean saccade latencies for the different conditions (in ms). Standard deviations are in brackets. Participant 5 is a projector; all other participants are associators.

n	Distractor absent	Distractor present	df	t	p
1	224 (67)	301 (96)	195	−7.22	<0.0001
2	247 (49)	273 (58)	257	−3.86	<0.001
3	243 (50)	267 (44)	244	−3.88	<0.001
4	192 (42)	234 (56)	230	−6.70	<0.0001
5	223 (37)	240 (42)	254	−3.41	<0.001
6	248 (64)	259 (86)	230	−1.18	0.24
7	207 (45)	212 (47)	147	−0.61	0.54
8	217 (60)	240 (73)	189	−3.10	<0.02
n	Digit distractor	Non-digit distractor	df	t	p
1	305 (75)	297 (86)	66	0.41	0.69
2	283 (57)	263 (56)	124	1.90	0.06
3	266 (45)	267 (45)	113	−0.10	0.92
4	228 (52)	241 (55)	102	−1.25	0.21
5	242 (45)	239 (44)	119	0.37	0.71
6	260 (73)	258 (86)	102	0.14	0.89
7	208 (42)	218 (54)	57	−0.83	0.41
8	243 (55)	236 (62)	81	0.52	0.60
n	Chromatic distractor	Achromatic distractor	df	t	p
1	304 (73)	300 (88)	66	0.21	0.83
2	270 (60)	275 (56)	124	−0.48	0.65
3	272 (48)	262 (41)	113	1.16	0.25
4	238 (56)	231 (53)	102	0.64	0.52
5	247 (46)	235 (43)	119	1.47	0.14
6	263 (74)	255 (87)	102	0.50	0.62
7	213 (53)	211 (43)	57	0.24	0.81
8	239 (54)	240 (63)	81	−0.04	0.97
n	Achromatic digit	Achromatic non-digit	df	t	p
1	317 (77)	281 (96)	33	1.21	0.23
2	283 (54)	268 (58)	64	1.08	0.28
3	261 (38)	263 (44)	57	−0.14	0.89
4	227 (50)	236 (56)	57	−0.63	0.53
5	236 (44)	234 (42)	63	0.26	0.79
6	268 (88)	246 (86)	50	0.91	0.37
7	204 (39)	219 (47)	32	−1.01	0.32
8	247 (55)	232 (73)	35	0.68	0.50
n	Chromatic digit	Chromatic non-digit	df	t	p
1	293 (72)	315 (74)	31	−0.84	0.41
2	283 (62)	258 (55)	58	1.59	0.12
3	272 (51)	272 (46)	54	0.12	0.99
4	228 (56)	247 (55)	43	−1.12	0.27
5	248 (46)	246 (47)	54	0.15	0.88
6	253 (55)	270 (85)	50	−0.85	0.40
7	212 (47)	216 (64)	23	−0.18	0.86
8	240 (55)	239 (55)	44	0.04	0.97

In the distractor present trials, no main effect of Distractor Identity ( F(1,7) = 0.26; p > 0.60) was found, indicating that latencies for digit and non-digit distractors were comparable. Additionally, there was no main effect of Distractor Color ( F(1,7) = 5.35; p > 0.05), indicating that chromatic and achromatic distractors had a similar effect on saccade latency. Crucially, no interaction between Distractor Identity and Distractor Color was obtained ( F(1,7) = 2.12; p > 0.10). Individual subject analyses confirmed these insignificant effects (see Table 1). 

In sum, this suggests that the presence of a distractor results in longer latencies, irrespective of the physical characteristics of the distractor (i.e., identity or chromaticity). 

Capture

For distractor present trials only, it was determined whether the percentage of trials in which an erroneous first eye movement was made to the distractor (i.e., “capture”) differed between the experimental factors. A main effect of Distractor Color ( F(1,7) = 18.83; p < 0.01) was found, showing that the percentage capture was larger for chromatic (mean = 22%; SD = 12%) than for achromatic distractors (mean = 13%; SD = 9%; see Figure 2). This is in line with previous findings demonstrating a larger percentage capture when the distractor has the same hue as the target (Ludwig & Gilchrist, 2002). There was no main effect of Distractor Identity (F(1,7) = 0.001; p > 0.90), indicating that digit and non-digit distractors caused a comparable percentage capture. 

Figure 2

 

The overall mean capture (± SEM) and the percentage capture for all eight synesthetes individually. Percentage capture was calculated as the percentage of trials in which the landing point of the first eye movement was positioned on the distractor. Note that more capture indicates stronger interference of the distractor.

If an achromatic digit distractor were perceived as having the same synesthetic hue as the target, an interaction between Distractor Identity and Distractor Color would be expected, because the effect of Distractor Color should be different for digit and non-digit distractors. However, no significant interaction was observed ( F(1,7) = 0.70; p > 0.40). Crucially, this appears to indicate that the achromatic digit distractor did not evoke more oculomotor competition resulting in more capture than the achromatic non-digit distractor. ¹ No individual subject analyses could be performed on percentage capture. In Figure 2, however, individual data are plotted; in five of the eight tested synesthetes, no effect of the synesthetic distractor can be observed, one of the synesthetes (Participant 6) shows an indication of more synesthetic capture, whereas two of the synesthetes (Participants 1 and 7) show a reverse of the effect expected by synesthetic capture. 

In the first control experiment, the synesthetes had to identify which digit was presented by using the numeric keyboard. Mean accuracy was 98.21% ( SD = 3.46%). We therefore concluded that the digit stimuli presented 10° away from fixation were clearly identifiable. 

In the second control experiment, all synesthetes subjectively reported that the synesthetic experience in periphery was identical to the synesthetic hue test. We therefore conclude that synesthetic color experiences can occur in periphery at this eccentricity, especially when attention is drawn toward the inducing stimuli. 

General discussion

In the current study, the role of visual attention in synesthesia was investigated using an oculomotor distractor task (Walker et al., 1997), in which an eye movement was to be made to a colored target, while ignoring a distractor. This distractor was either a digit or a non-digit distractor. The color of the distractor could be achromatic or identical to the target. The crucial comparison was whether the percentage capture by a gray digit distractor (synesthetic distractor) was higher than the percentage capture by a gray non-digit distractor. Capture by a distractor was caused by the distractor winning the competition over the target. While the task of the participant was to make an eye movement to a colored target, the presentation of the distractor evoked a strong sensory signal, which interfered with target selection. When this distractor matched the target template in terms of the physical color, the interference was even stronger, resulting in more erroneous eye movements (in line with Ludwig & Gilchrist, 2002, 2003). The present results showed that whereas physical color information did influence the target selection process, synesthetic color information did not influence this process; the percentage capture by a synesthetic distractor was comparable with the percentage capture by a gray non-digit distractor. In other words, a synesthetic color percept is not equivalent to “normal” color perception, in terms of the extent to which it influenced the target selection. This suggests that, on trials with a synesthetic distractor, the synesthetic color percept was either absent or not strong enough to distract eye movements away from the target in the same way as physically colored distractors. 

One might argue that the digit distractors presented 10 degrees from fixation were difficult to identify because of the reduced spatial resolution at this eccentricity, and hence no or a weaker synesthetic color percept occurred. In two control experiments, however, it was found that the digit distractors could be perceived at this eccentricity (Control Experiment 1; accuracy 98.21%) and that the resulting synesthetic hues were subjectively similar to the hues experienced with central vision (Control Experiment 2). All synesthetes reported that their synesthetic hues were located at or around the location of the digit, ensuring that the synesthetic hue was located at a task-relevant location. This is important because if synesthetes in the current experiment did not localize their hues, one would not expect a synesthetic color to capture and guide eye movements to a correct or a wrong location. The results of Control Experiment 2 therefore showed that this alternative explanation cannot account for our results. 

Another alternative explanation for the current results might be that eye movements were launched before the color of a synesthetic digit could have occurred since eye movement latencies were less than 300 ms on average. In other studies, however, it has been found that a synesthetic color experience was already reflected in visual area V4 around 122 ms after onset of the inducing stimulus (e.g., Beeli, Esslen, & Jäncke, 2008). Even though this early activity does not necessarily result in an early conscious synesthetic percept, it does show that activity at brain level related to a synesthetic percept occurs at an early stage, making it less likely that a synesthetic percept was formed after 300 ms. 

Many previous studies have reported comparable results between physically colored stimuli and synesthetically colored stimuli (Hubbard, Arman, Ramachandran, & Boynton, 2005; Palmeri et al., 2002; Ramachandran & Hubbard, 2001; Smilek, Dixon, Cudahy, Merikle, 2001; Wagar, Dixon, Smilek, & Cudahy, 2002). It is important to note that results of some of these studies are based only on subjective report (Hubbard et al., 2005; Ramachandran & Hubbard, 2001), have been found to be difficult to replicate (Smilek et al., 2001; see Sagiv et al., 2006), or suggest that not the color of the target, but the synesthesia elicited by the distractors attracts attention (Palmeri et al., 2002, Experiment 2). Nevertheless, the question arises when synesthetic color experience can influence behavior. The crucial factor might be the amount of attention allocated to the inducing grapheme. In the present study, the inducing grapheme was a task-irrelevant distractor because the exact identity of the distractor was not crucial to successfully perform the task. It was only important whether the element was a colored circle or not. Therefore, the amount of attention allocated to the inducing grapheme will most probably have been lower than in the studies that have found a behavioral effect of an inducing grapheme. A low amount of attention might simply not be enough to induce a synesthetic color experience. In line with this idea, Mattingley, Payne, and Rich (2006) have found that reducing the attentional resources available for processing an inducing grapheme decreases the synesthetic interference in a Stroop-like task. Moreover, it has been suggested that “overt recognition” of the inducing stimuli is crucial (Mattingley et al., 2001). Our results support this suggestion by showing that focused attention is important for the synesthetic percept to occur. 

More radically, one might argue that synesthetic reports are not perceptually based at all and that previous evidence can be explained by a combination of participants' as well as experimenter's biases, a view developed in some detail by Gheri et al. (2008). The absence of an effect of synesthetic color experiences in this study might be construed as evidence for this skeptical view. We are disinclined to interpret our results in this way for the following reason. Two recent studies in our laboratory (Gebuis, Nijboer, & van der Smagt, 2009a, 2009b) have demonstrated implicit number processes evoked by color, a finding that cannot be explained by participants' or experimenter's biases. In these studies, bi-directionality in synesthesia was investigated, using both a color–number and a number–color priming task. Interestingly, the magnitude of the congruency effects was similar in both priming tasks. This indicated that, even though colors do not elicit vivid number experiences, the percept of a color primed numerical processes to a similar extent as numbers primed color processing. These priming effects were also present in the P3b latency (parietal electrode site) and P3a amplitude (frontal electrode site) of the ERP data (Gebuis et al., 2009b). Both priming (especially the effects on the ERPs) and oculomotor distractor paradigms are immune to bias. The biggest difference between these studies is that the colors and the numbers are targets, and hence the focus of attention is in the ERP priming studies but not in the present eye movement study. Therefore, we are disinclined to interpret our present results as evidence that synesthesia is not real. 

In conclusion, this study is consistent with other papers suggesting a difference between synesthesia and actual physical color effects (e.g., Edquist et al., 2006; Gheri et al., 2008; Hong & Blake, 2008). In the case of our study, we conclude that these differences arise because attention must be directed to an achromatic letter before that letter can acquire its characteristic, synesthetic color. 

Authors Note

Both authors have put an equal amount of work in this project and therefore regard this as a shared first authorship. 

Acknowledgments

We would like to thank Maarten van der Smagt for his helpful comments on earlier versions of this manuscript and many useful discussions and Puck Imants and Sophie Heringa for helping testing the synesthetes. 

We also thank Randolph Blake and two anonymous reviewers for their useful comments. 

Commercial relationships: none. 

Corresponding author: Dr. Tanja C. W. Nijboer. 

Email: [e-mail]. 

Address: Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 2, 3584 CS, Utrecht, The Netherlands. 

Footnote

Footnotes

¹  Additionally, it was tested whether there was a difference between achromatic distractors presented close to the target (i.e., on a clock position 1 hour distant from the target location) versus those that were further away (i.e., all other locations). No main effects of Distance and Digit were found ( F(1,7) = 3.29, p > 0.10 and F(1,7) = .25, p > 0.60, respectively). Also, no significant interaction between Distance and Digit was found ( F(1,7) = 3.72, p > 0.10).