Abstract
Maximum likelihood difference scaling (MLDS) is a psychophysical method to estimate perceptual scales. Unlike other appearance-based methods such a matching or magnitude estimation, it relies on forced-choice judgments of stimulus differences and uses the resulting binary responses for scale estimation. In practice, several hundreds of trials are required in order to derive robust scale estimates. This restricts the use of the method because it limits experimenters to test few experimental conditions. It would thus be desirable to optimize data acquisition for MLDS in a similar spirit as adaptive methods optimize data acquisition for performance-based methods (e.g. staircases for discrimination experiments). Recently, Shooner and Mullen (2022) took a step in this direction by presenting a subset of comparisons which involved only the most informative comparisons. Their sampling scheme decreased the number of trials by about 50%, depending on the number of stimulus levels. Following these promising results here we use simulations to test under which conditions a selective subsampling of comparisons is a valid simplification of the procedure and whether it affects the method’s reliability. We test the effect of different sampling schemes on accuracy and precision of MLDS for different shapes of perceptual scales, amounts of observer’s internal noise and number of trials. We keep simulation parameters in a realistic range to reflect what an experimenter may encounter in practice. We observe that changes in the sampling scheme might lead to misestimates of the shape of the scale, particularly for strongly non-linear scales. Estimates of the scale maxima (which depend on estimates of observer’s noise), seem robust to the selective subsampling. In addition, we introduce a new type of visualization of the raw data collected in MLDS experiments, which can help experimenters to detect cases where a more efficient sampling scheme may be useful.