Abstract
In vision research we are often interested in understanding the mapping of complex physical stimuli to their perceptual dimensions. This mapping can be explored experimentally with multi-dimensional psychophysical scaling. One fruitful approach is the combination of triplet comparison judgments, asking if stimulus A or B is more similar to C, together with ordinal embedding methods. Ordinal embedding infers a point representation such that the distances in the inferred, internal perceptual space agree with the observer’s judgments. One fundamental problem of psychophysical scaling in multiple dimensions is, however, that the inferred representation only reflects perception if it has the correct dimensionality. Unfortunately, the methods to derive the “correct” dimensionality were thus far not satisfactory for noisy, behavioural data (e.g. no clear “knee” in the stress-by-dimension graph of multi-dimensional scaling). Here we propose a statistical procedure inspired by model selection to choose the dimensionality: Dimensionality can be tuned to prevent both under- and overfitting. The key elements are, first, measuring the scale’s quality by the number of correctly predicted triplets (cross-validated triplet accuracy). Second, performing a statistical test to assess if adding another dimension improves triplet accuracy significantly. In order to validate this procedure we simulated noisy and sparse judgments and assessed how reliably the ground-truth dimensionality could be identified. Even for high noise levels we are able to identify a lower-bound estimate of the perceived dimensions. Furthermore, we studied the properties and limitations of our procedure using a variety of behavioural datasets from psychophysical experiments. We conclude that our procedure is a robust tool in the exploration of new perceptual spaces and is able to help identify a lower bound on the number of perceptual dimensions for a given dataset. The identified dimensions and the resulting representation can then be related to perceptual processes in order to explore human vision.