Abstract
Signal detection accuracy is susceptible to internal, sensory-driven noise and to external, stimulus-driven uncertainty. Combining multiple observers' judgments leads to well-known benefits for optimal integration (Sorkin & Dai, 1994), and lower but comparable benefits for majority-rule aggregation (Eckstein, Das, et al., 2012). However, such theoretical analyses and empirical studies use a signal of a single strength while real-life tasks often present a mixture of signal strengths, which makes some instances of the signal much easier to detect than others (e.g., some mammograms contain very large lumps while others contain very small lumps). Here, we simulated and conducted a detection task with a random mixture of high and low signal strengths. Each observer (N=20) participated alone in two conditions on separate days (counterbalanced). On each trial, observers responded on an 8-point confidence scale whether a Gaussian signal (σ=0.15o) was embedded in white noise (signal present 50% of the time). In the single strength condition, the signal always had 11% contrast (SNR=4). In the mixed strengths condition, half of the signals had 7% contrast (SNR=2.54) while the other half had 16% contrast (SNR=5.81). All observers viewed the same exact stimuli but in randomized order, and their individual d-primes were approximately the same for the two conditions (d'≈1.45). As expected, aggregating judgments across observers leads to higher d-primes; but these higher d-primes are 5% to 10% lower in the mixed strengths condition relative to the single strength condition (depending on the number of judgments that are combined). This is because observers' judgments during signal present trials have a heightened correlation in the mixed strengths condition relative to the single strength condition. These results highlight why the aggregations of judgments for real-life stimuli tend to generate less accuracy benefits than theoretically predicted and typically observed for synthetic stimuli where signal strength is usually constant.
Meeting abstract presented at VSS 2014