The role of one-shot learning in #TheDress

Abstract

#TheDress is remarkable in two aspects. First, there is a bimodal split of the population in the perception of the dress's colors (white/gold vs. black/blue). Second, whereas interobserver variance is high, intra-observer variance is low, i.e., the percept rarely switches in a given individual. There are two plausible routes of explanations: either one-shot learning during the first presentation of the image splits observers into two different, stable populations, or the differences are caused by stable traits of observers, such as different visual systems. Here, we hid large parts of the image by white occluders. The majority of naïve participants perceived the dress as black and blue. With black occluders, the majority of observers perceived the dress as white and gold. The percept did not change when we subsequently presented the full image, arguing for a crucial role of one-shot learning. Next, we investigated whether the first fixation determines the perceived color in naïve observers. We found no such effect. It remains thus a puzzling question where the source of variability in the different percepts comes from.

Introduction

Intuitively, the first amazing effect with the #TheDress is that a large number of people perceive it as white and gold even though the true colors of the dress are black and blue. Although the differences in the perceived and true colors are striking, apparent changes in color are common in many illusions, such as the checker shadow illusion (Adelson, 1995). These illusions are usually explained in terms of color constancy, where the luminance of the context plays the crucial role. Also in #TheDress, color constancy plays an important role (Brainard & Hurlbert, 2015; Gegenfurtner, Bloj, & Toscani, 2015; Winkler, Spillmann, Werner, & Webster, 2015; Witzel, Racey, & O'Regan, 2017). For example, the appearance of the dress can be changed by changing the background (Lafer-Sousa, Hermann, & Conway, 2015; Witzel et al., 2017). 

What makes #TheDress even more interesting is that the background is the same for all observers, still perception differs strongly, contrary to most color illusions where almost all observers perceive the colors in the same illusory way. On top of this, perception rarely changes, i.e., white/gold perceivers stay white/gold perceivers and the same is true for the (veridical) black/blue perceivers, contrary to ambiguous figures where perception easily can switch within one observer. About two-thirds of the population perceive the dress as white and gold but estimates differ (Lafer-Sousa et al., 2015; Moccia et al., 2016; Wallisch, 2016). In short, intra-individual variability is very low, whereas interindividual variability is high and bimodal: either white/gold or black/blue. 

It may be that the perceived color depends on largely invariant factors of each person, such as differences in the optical apparatus, genetics, personality, etc. The second hypothesis is that a one-shot learning mechanism causes the differences in perception during the first encounter of the dress, similar to an imprinting mechanism. This is reminiscent, for example, of the Dalmatian dog display. When one has learned to see the dog within the noisy background, one will always see it in future presentations, clear and stable. 

Here, we tested the one-shot learning hypothesis in naive observers who had never seen #TheDress. First, we covered the entire background and large parts of the dress by either white or black occluders, which strongly and differentially influenced the percepts of the observers. These percepts remained constant when the occluders were removed, strongly arguing for the involvement of one-shot learning in the perception of the dress. Next, we tested whether there are differences in the eye movement patterns of white/gold versus black/blue perceivers. We did not find any obvious pattern. Finally, we tested whether the luminance in a small neighborhood of the first fixation matters and found no effect either. 

Methods

Experiment 1

Observers

In Experiment 1, naïve observers, who had never seen the dress before, were needed. In Switzerland, finding such participants was difficult. We therefore conducted the experiment in Georgia where internet use, particularly in the older population, is less frequent. 

Sixty-seven observers participated in Experiment 1 (34 females, age 18–74 years). Observers signed a consent form and had normal or corrected-to-normal vision. Observers were paid for their participation. None of the observers was introduced to the goal of the experiment. We debriefed observers just after the short-term experiment (5 min). 

Stimuli and apparatus

The stimuli were presented on a LCD screen (ASUS VG248QE, Taipei, Taiwan; screen resolution 1920 × 1080 pixels) using Matlab software. Participants were seated 60 cm from the computer screen. Instructions were written in Georgian and the stimuli were presented at the center of the screen on a neutral gray background (30 cd/m²). We presented only a small part of the dress by occluding the background and large parts of the dress by adding either black (0.74 cd/m²) or white (101 cd/m²) occluders (total stimulus size 350 × 466 pixels; Figure 1a). The visible strip was 71 pixels wide. The full image of the dress was 225 × 300 pixels. 

Figure 1

 

Experiment 1. (a) The image of the dress was occluded by either black or white (not shown) occluders so that only a small part of the dress was visible. (b) The number of observers in each group. (c) Colors of the dress reported by naïve observers with either black or white occluders based on question 5. (d) Fraction of all observers for whom the percept did or did not change between the occluded and the full image of the dress.

Procedure

We first presented the occluded image of the dress. Then, (1) we asked observers to report the perceived colors and (2) whether they recognized the image. Then, we removed the occluders and presented the full image. (3) We asked observers to report the colors of the dress again and (4) if they had seen the image before the experiment. (5) In an AFC task, participants were asked to choose whether the dress appeared as either white/gold or black/blue. This question was introduced after 22 observers were tested already; for these 22 observers we used in (5) the colors named in (3); in case they did not name white/gold or black/blue we chose, for example, black/blue when they reported brown/blue because all observers who reported brown/blue in step 3 decided for black/blue in step 4; this was the case for only five naïve observers. (6) Observers who had seen the image previously were asked if it was the same color as they had seen before the experiment. 

Experiment 2a and b

Observers

Eleven observers (six females, age 31–67 years) participated in Experiment 2a and 51 observers (27 females, age 18–76 years) joined Experiment 2b. All observers were from Tbilisi, Georgia, and had not participated in Experiment 1. Informed consent was obtained from all participants. All observers had normal or corrected-to-normal vision. 

Stimuli and apparatus

We used the same set up as before. Eye movements were recorded using the EyeTribe tracker. The display was calibrated at the beginning of the experiment. 

Procedure

Experiment 2a:

The image of the dress was presented in the center of the screen on a gray background. Observers were able to freely explore the image for 5 s. Then, they were asked to report the two colors of the dress. Finally, they were asked whether they had seen #TheDress before. This question was asked at the end of the experiment to avoid any precedence effects. One observer had previously seen the dress and was excluded from the analysis. 

Experiment 2b:

First, observers fixated a red dot that subtended 0.264 degree of visual angle on a gray background. Then, the dress was presented while the fixation dot remained in place. The fixation dot was located at either the upper right or lower left corner (Figure 3a). We chose these two regions to test whether the luminance in the close neighborhood of the first fixation determines the perceived colors. The rationale is that fixating a bright region, such as in the top right of the image, leads to an assumed high illumination for the entire image. To obtain color constancy, this illumination is discounted for in the entire image and, hence, the colors of the dress appear to be biased toward darker colors, i.e., more black and blue. Conversely, if the first fixation is in the dark region at the bottom left, the assumed illumination would be darker, inducing a brighter percept, i.e., more akin toward white and gold. To support this rationale, we used a color constancy algorithm. First, we determined the average RGB values close to the fixation dots. Because these were very high and low values, respectively, we reduced the bright values and increased the dark values by about 50% (otherwise the image would appear totally black or white, respectively). Then, we divided the R, G, and B values of each pixel of the image by the R, G, and B values of this estimated assumed illumination to discount for the illumination. As expected, assuming a high illuminance (as in the top right corner of the dress image) leads to a darker output, more black and blue. Conversely, assuming a dark illumination (as in the bottom left) leads to a brighter output, more white and gold. It is important to note that these regions were not the target of the first saccades in Experiment 2a. We have chosen these extreme luminance regions on purpose to test whether the luminance at the first fixation could determine the percept. 

Figure 2

 

Experiment 2a. Top panel: Heat maps of the eye-tracking data. Bottom panel: First three fixations of the observers. Numbers indicate first, second, and third saccades. The larger a circle, the longer an observer fixated at the location. For clarity purposes, some fixations in the background are not shown. Left panel: white and gold perceivers. Right panel: black and blue perceivers. Photograph of the dress used with permission. Copyright Cecilia Bleasdale.

Figure 3

 

Experiment 2b. (a) Picture of the dress. The fixation points are at the upper right and lower left corners. Both points are shown in the figure but only one of them was presented during the experiment. The fixation points are larger in this figure than in the experiment proper. (b) Colors of the dress reported by naïve observers when the first fixation was either at the upper right (bright region) or lower left (dark region) corner of the picture of the dress. (c) Heat maps of the eye-tracking data split in four groups. Left panel: first fixation at the upper right corner. Right panel: first fixation at the lower left corner. Top panel: white and gold perceivers. Bottom panel: black and blue perceivers. Please note that the numbers of black and blue perceivers are low (four and five observers with first fixation at the upper right or lower left corner, respectively). Photograph of the dress used with permission. Copyright Cecilia Bleasdale.

We used a gaze-contingent display, i.e., the image appeared only if the observers were steadily fixating. The fixation point disappeared 1 s after the dress image presentation. Then, participants freely explored the image during an additional 5 s. They were then asked to report the two colors of the dress and whether they had seen the image before. Thirteen observers had previously seen the dress and were excluded from the analysis. 

Results

Experiment 1

We presented only a small part of the dress before showing the whole image (Figure 1a). We used either black or white occluders. Out of the 67 observers in total, 35 of the participants had not seen #TheDress before, 22 did see the dress before, and 10 had seen it before but did not recognize it when only the small stripes were visible (Figure 1b). 

Nineteen and 16 of the naïve participants, respectively, viewed the image with either black or white occluders. With black occluders, 82.4% of the observers turned into white and gold perceivers and with the white occluders, 85.7% of the participants perceived the dress as black and blue (Figure 1c). The proportions of white/gold and black/blue perceivers are significantly different (z score = 3.775; two-tailed p value < 0.01). 

When the occluders were removed, the percept did not change for the majority of observers (Figure 1d). Moreover, the non-naïve participants reported that it were the same colors as when they saw the dress picture for the very first time, not related to this experiment (except for four participants who did not recall what their first percept was). 

Experiment 2

In Experiment 2a, observers could freely explore the image of the dress. Heat maps and scan paths of the saccades show that there is no obvious pattern between white/gold and black/blue perceivers (Figure 2, top panel). The first fixations are highly variable between the observers (Figure 2, bottom panel). In Experiment 2b, we tested whether the luminance of the first fixation matters by forcing observers to first look at a bright (upper right corner) or dark (lower left corner) region of the picture (Figure 3a) with the rationale that the luminance in these regions may influence the assumed illumination of the entire image, thus, determining the perceived colors of the dress via color constancy mechanisms. When the fixation point was presented at the upper right position, 82.6% of the participants perceived the dress as white and gold and 17.4% as black and blue. When the first fixation was at the bottom left of the picture, 66.7% of the observers reported the dress as white and gold and 33.3% as black and blue (Figure 3b). There is no significant difference between the proportions of white/gold and black/blue perceivers in the two groups (z score = 1.127; two-tailed p value = 0.259). Hence, our results clearly argue against the hypothesis that the first fixation of the image matters because fixating the upper position should have led to a higher number of black and blue perceivers than in the group fixating the lower part. Next, we computed the scan path of the saccades following the first fixation and split the data in four groups depending on the position of the first fixation (upper right or lower left corner) and the perceived colors (Figure 3c). There seems to be no obvious pattern between the different groups of observers. 

Discussion

The most fascinating aspect of #TheDress is not that some people perceive the dress in the “wrong” colors white and gold, but that there is the stable and bimodal distribution of different color perceivers. If we all perceived the dress as white and gold, it would just be one more color illusion. We hypothesized that differences in the first exposure to #TheDress lead to one-shot learning, determining whether a person turns into a gold/white or black/blue perceiver. In Experiment 1, we removed all context and large parts of the dress by presenting black or white occluders. A posteriori, we split up the observers in three groups: people who had never seen the image before, people who had seen the image previously and observers who had seen it before but did not recognize the occluded image. The occluders had a large effect on the perceived colors. With black occluders, more than 80% of the naive participants turned into white/gold perceivers. Hence, we were able to induce the normally less frequent blue/black percept in the majority of naïve observers. With white occluders it was just the opposite. These results are very much in line with what one might expect from color constancy mechanisms: The dress seems brighter because of the dark flankers, or darker when surrounded by white flankers. We then removed the occluders and asked about the colors again. In all groups, there were almost no changes in color perception. Hence, there seems to be a one-shot learning mechanism in play, which leads to an imprinting of the color perceived. 

The results of Experiment 1 are well in line with previous studies, which have shown that context and color constancy can temporarily change the colors of the dress even after its first encounter (Brainard & Hurlbert, 2015; Gegenfurtner et al., 2015; Lafer-Sousa et al., 2015; Winkler et al., 2015; Witzel et al., 2017). Embedding the dress in a scene with unambiguous illumination cues biased the perception of the observers towards the one predicted by the illumination, for example, by using Color Contrast Cube images (Lafer-Sousa et al., 2015). Witzel et al. (2017) placed the dress in different natural world scenes. The perception of the color of the dress was determined by the scene interpretation (dark beach vs. bright cave scene) even though the local contrast was opposite to the assumed illumination of the scene. Thus, the interpretation of the scene, not the color contrast, biased the perceived colors of the dress. 

What causes one-shot learning? We thought that the first fixation may matter since viewing strategies play an important role in color perception (Cornelissen & Brenner, 1995; Golz, 2010; Granzier, Toscani, & Gegenfurtner, 2012). Specifically, the time spent fixating different parts of an image partially explains color constancy effects (Cornelissen & Brenner, 1995). Artificially forcing viewing strategies influences color perception. However, no correlation was found between eye movements and color perception under natural viewing conditions (Granzier, Toscani, & Gegenfurtner, 2012). 

In Experiment 2a, we found no obvious differences in the heat maps and scan paths of the white/gold versus black/blue perceivers when naïve observers were viewing #TheDress freely. In Experiment 2b, we explicitly tested whether the luminance of the first fixation matters, for example, by determining the assumed illumination. We found very little evidence for such a mechanism. Whether people were exposed to the top right part of the image or the lower left part did not strongly change the ratio of (white-gold)/(black-blue) observers. Hence, it seems that context plays a crucial role in the perception, but the luminance of the first fixation and likely subsequent eye movements do not play an important role. 

We analyzed the data in terms of age and gender and did not find any obvious differences. Eighty percent of the female and 67% of the male observers were white and gold perceivers. A total of 74% of participants between 18 and 50 years and 80% of those above 50 years perceived the dress as white and gold, respectively. 

We like to mention that the Georgian language contains six basic color words (red, yellow, black, white, green, and navy blue; Khomeriki, Lomashvili, & Kezeli, 2011) whereas for example in English there are 11 basic color terms (Berlin & Kay, 1991). However, results obtained here seem to be very similar to other countries. Particularly, the proportion of white and gold perceivers is similar. 

Our results show that the first percept — but not the first fixation — determines future color perception, similar to one-shot learning. It remains a puzzling question why with the full image more observers perceive the dress as white/gold even though eye movements seem to play no role. What are the sources of variability? We cannot exclude that covert attention plays a role or that complex eye movements patterns other than the first fixations matter. It may also well be that stable, individual differences, e.g., in the optical apparatus or personality, are important when the dress is presented fully. However, these differences play less a role when the context is strongly changed, such as presenting crude, large occluders as in Experiment 1. Hence, subtle differences in both the first encounter as well as between stable traits may explain the results. 

Acknowledgments

The authors are grateful to Janir Ramos da Cruz, Marc Lauffs, and Marc Repnow for technical support and to Sophie Geoghan for proofreading. This work was supported by a grant from the Swiss SystemsX.ch initiative and by the Swiss National Science Foundation grant 153001 (Basics of visual processing: from retinotopic encoding to non-retinotopic representations). 

Commercial relationships: none. 

Corresponding author: Leila Drissi Daoudi. 

Email: [e-mail]. 

Address: Laboratory of Psychophysics, Brain Mind Institute, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.