Clarifying signal detection theoretic interpretations of the Müller–Lyer and sound-induced flash illusions

Abstract

In a recent article, Witt, Taylor, Sugovic, and Wixted (2015) made several claims about the way the Müller–Lyer and sound-induced flash illusions should influence the signal detection theory criterion and sensitivity measures, c and d′. Here, we address some crucial conceptual inconsistencies in their simulation of the Müller–Lyer illusion and clarify a previous analysis of the sound-induced flash illusion from the literature that is misinterpreted in their discussion. Alternative signal detection theoretic interpretations of both illusions are offered.

Introduction

A recent article by Witt, Taylor, Sugovic, and Wixted (2015) reiterated and expanded on the important observation that the signal detection theoretic criterion measure, c, cannot distinguish between response biases and perceptual biases (Morgan, Hole, & Glennerster, 1990; Raslear, 1985; Witt et al., 2015). They pointed out a pervasive misconception in the psychophysics literature that changes in the criterion measure necessarily reflect changes in response bias, and they used several well-known perceptual illusions to illustrate how such criterion shifts could instead be interpreted as representing perceptual bias. Their manuscript thus serves as an important warning about the potential miscategorization of bias effects in signal detection theory (SDT) analyses. However, despite the value in this overarching message, some of the supporting arguments in Witt et al. (2015), particularly those related to the Müller–Lyer and sound-induced flash illusions, contain conceptual inconsistencies and other points of confusion regarding the correspondence between SDT measures and underlying perceptual processes. We attempt to clarify these points below. 

The Müller–Lyer illusion

The centerpiece of Witt et al. (2015) is a computer simulation of a behavioral experiment using the Müller–Lyer illusion. In brief, the Müller–Lyer illusion occurs when adding inward-facing or outward-facing tails to the ends of a horizontal line, which makes the line appear shorter or longer respectively. On each trial, a simulated observer is presented with either a short line (5 cm) or a long line (7 cm), with tails that are oriented either inwardly or outwardly. The description of the simulated effect of the illusion reads, “…on each trial, the perceived line length was drawn from a Gaussian distribution with the mean set to the sum of the actual line length plus the theorized influence of the tails (+13% for tails-out, −13% for tails-in)” (p. 4). The simulated observer then uses a fixed criterion at 6 cm to perform the task of judging whether the line is long or short. A hit occurs when the line is long (7 cm) and the observer responds that the line is long (perceived line length >6), and a false alarm occurs when the line is short (5 cm) and the observer responds that the line is long (perceived line length >6). The simulation is run with 10 observers, each of whom performs 1,000 trials (250 of each of four trial types: long tails-in, long tails-out, short tails-in, and short tails-out). Witt et al. (2015) reported that when using the resulting simulated behavioral data to compute the SDT measures d′ and c, they find a large change in c, but no change in d′ between the tails-in and tails-out conditions. This, they argue, provides evidence for the fact that a perceptual effect can be reflected in the criterion measure and not in d′. 

The critical point to notice in the description of their simulation that makes their reported result unexpected is that the effect of the Müller–Lyer illusion is described as being modeled in accordance with the Weber–Fechner law, where the magnitude of the illusion is proportional to the size of the visual stimulus. This relationship is intuitive and has been supported empirically in humans (Tudusciac & Nieder, 2010). However, if this is the case, then the distance between the means of the two perceived length distributions will be different between the different illusory conditions (Figure 1). If we then consider that all simulated distributions have a constant standard deviation of 1.2 cm, we should expect the results of the simulation to show a change in d′ between the tails-in and tails-out conditions. Indeed, when we ran this simulation, a two-tailed, paired-samples t test showed a significant difference in d′ between these two conditions (d′ tails-in = 1.43 ± 0.14, d′ tails-out = 2.02 ± 0.18, t(9) = 9.499, p < 0.001; Figure 2A). This raises the question of where the discrepancy lies between our simulation and that of Witt et al. (2015). Given that they reported a constant standard deviation for all simulated distributions, it follows from SDT that the only way to maintain a constant value of d′ while changing the criterion is to shift the two evidence distributions by the same constant. We modified our simulation to do this by shifting the means of the tails-out and tails-in distributions relative to the distributions in the no-tails condition by ±13% of 6 respectively, and the resulting SDT measures matched those reported in Witt et al. (d′ tails-in = 1.81 ± 0.13, d′ tails-out = 1.69 ± 0.18, t(9) = 1.55, p = 0.16; Figure 2B). 

Figure 1

 

Distributions of perceived line length under different illusion conditions in the Müller–Lyer simulation described in Witt et al. (2015). The distance between distribution means (vertical bars) changes with each condition. Tails-in and tails-out distributions are shifted left and right, respectively, by 13% of the means of the corresponding no-tails distributions. All distributions have a standard deviation of 1.2 cm. Note that the no-tails condition was not simulated in Witt et al. (2015), but is included here for illustration.

Figure 2

 

Results from the simulation of the Müller–Lyer illusion. (A) Modeling the effect of the illusion by shifting the two evidence distributions by the 13% of their respective means, as described in Witt et al. (2015). (B) Modeling the effect of the illusion by shifting the two evidence distributions by a constant. The results in (B) match those presented in Witt et al. (2015). *P < 0.001.

Based on the results above, we conclude that the most likely explanation for the discrepancy between the reported simulation design and the results in Witt et al. (2015) is that the authors did indeed model the effect of the Müller–Lyer illusion by shifting the two evidence distributions by the same constant. It is unclear whether the mistake was in the reporting or the running of the simulation. In the former case, it is left to be explained why the authors chose this less intuitive model of the illusion, for which empirical support in the literature is lacking. In the latter case, it needs to be clarified that when the illusion is simulated as was reported, there in fact is a change in d′ between the tails-in and tails-out conditions. In either case, the question is raised of whether or not the Müller–Lyer illusion provides an appropriate example of an experimental manipulation that can affect the criterion without affecting d′. This is not to say that the effect on the criterion should not be considered a perceptual bias. We do not wish to undermine this point, but rather to show that, in what may have been haste to provide more convincing evidence for an effect that supports the overarching message of their article, the authors have presented information that could potentially misguide future assumptions about the way the perceptual processes underlying the Müller–Lyer illusion are reflected in SDT measures. Given the pervasive use of this illusion in the study of visual perception, it seems crucial that this point be clarified. 

The sound-induced flash illusion

It is in the same vein that Witt et al. (2015) oversimplified a signal detection theoretic explanation of the sound-induced flash illusion, and in doing so, misrepresented the SDT analysis provided in Rosenthal, Shimojo, and Shams (2009). Their discussion of the sound-induced flash illusion centers on a comparison of the SDT measures computed for the one-beep and two-beep conditions presented in Rosenthal et al. (2009). Comparing across these two conditions shows exactly what the authors were looking for in their simulation of the Müller–Lyer illusion: a change in criterion, but no change in d′. Witt et al. (2015) criticized Rosenthal et al. (2009) for not comparing these conditions in their SDT analysis and for instead focusing on the change in d′ between the no-beep and two-beep conditions. This analysis is implied to be erroneous as Witt et al. (2015) stated, “Theoretically, the sound-induced flash illusion is an example of a perceptual bias, and therefore the illusion should present itself in the measure of c (or β), and not in d′” (p. 3). If we take this statement at face value, are we to believe that the change in d′ that occurs between the unisensory and multisensory conditions in Rosenthal et al. (2009) is not a manifestation of the sound-induced flash illusion? Clearly, this interpretation is unsatisfactory, as it is undeniable that the change in d′ is at least somehow related to the multisensory conditions that induce the illusion. 

The real issue here is that we should expect different changes in SDT measures depending on which conditions of the sound-induced flash illusion we choose to compare. This point is made clear by Figure 3, which provides an example of how d′ and c change between the three different auditory conditions (i.e., zero beeps, one beep, and two beeps) in Rosenthal et al. (2009). The fact that there is no significant change in d′ between the two multisensory conditions does not imply that the change in d′ between unisensory and multisensory conditions is somehow not reflective of the sound-induced flash illusion. The two comparisons are fundamentally different, and, in fact, it is intuitive that d′ should change between the latter and not the former. 

Figure 3

 

Signal detection measures approximated for each of three different auditory conditions in Rosenthal et al. (2009, experiment 1, feedback condition). Both d′ and c show significant change between each pair of conditions, except in the case of d' between the two multisensory conditions. Note that we have modeled shifts in the criterion as perceptual biases according to Witt et al. (2015).

The purpose of comparing d′ across the no-beep and two-beep conditions in Rosenthal et al. (2009) was to track the magnitude of the perceptual component of the illusion that is due to multisensory integration. A comparison between the one-beep and two-beeps conditions was not necessary for this purpose, and therefore it was left out of the analyses. This is not to say that the change in c between the two multisensory conditions cannot be reflective of perceptual processing, but only that this particular measure was not of primary interest to the goals of the study. We do agree, however, based on the central point of Witt et al. (2015), that considering shifts in the criterion as reflective of changes in perceptual processing could have potentially provided a more complete capture of the perceptual effect of the sound-induced flash illusion between unisensory and multisensory conditions. Yet it stands to be corrected that, contrary to what was stated in Witt et al. (2015), using d′ to track perceptual effects of the sound-induced flash illusion is not, in and of itself, an invalid method. 

Conclusion

In concluding, we wish to reiterate that the purpose of this response is not to devalue the central point of Witt et al. (2015) concerning the misinterpretation of the signal detection theoretic criterion measure. There is no doubt that this problem runs deep in the literature, and their article is thus an important warning to SDT users everywhere. Rather, our goal was simply to clarify specific supporting arguments that we found to be inconsistent, or that we believed were misrepresentative of previous work. The common thread in these arguments seems to be an overreliance on the criterion measure in terms of explaining the perceptual processes associated with each illusion, which may have arisen out of an eagerness on the part of the authors to provide support for the idea that the criterion measure can reflect perceptual bias in addition to response bias. We hope that showing the danger in making these types of oversimplifying assumptions provides a gentle warning of its own to readers and others using signal detection theoretic models of perception going forward. 

Acknowledgments

Commercial relationships: None. 

Corresponding author: J. D. Knotts. 

Email: [e-mail]. 

Address: University of California, Los Angeles, Los Angeles, CA, USA.