Abstract
While it is well-established that human observers can use high-order image statistics to discriminate synthetic visual textures, it is unclear whether these statistics are used in real-world visual tasks. Here, we show that high-order statistics are needed to account for performance in radiologists' judgments of breast density, and we characterize the way in which they are used. To do this, we built generalized linear models of radiologists’ judgments of density (BIRADS a-b: fatty/scattered, vs. BIRADS c-d: heterogenous/dense) from 444 mammograms. Regressors consisted of statistical features extracted from interior regions of breast images: either just their spectral characteristics or also their local image statistics. The local statistics characterized the co-occurrence of pairs, triplets, and quadruplets of neighboring checks in a binarized image (Victor and Conte 2012), thus capturing information not present in the power spectrum. We computed these statistics over spatial scales ranging from the individual pixel (0.07 mm) to images downsampled by a factor of 32. Models were compared in terms of their ability to explain and predict radiologists’ judgments of low vs. high density mammograms. Inclusion of co-occurrence statistics improved performance substantially: for the explanatory model, the Akaike criterion was reduced from 346.3 to 174.2; for the predictive model, the area under the ROC curve was increased from 0.89 to 0.93 (p < 0.04). To analyze the contribution of local image statistics, we examined the performance of models that considered only a single scale. Scales corresponding to downsampling by a factor of 4 to 16 had the most impact. When local image statistics were recomputed from Gaussian surrogates that matched the original image spectra, the degree of improvement that they provided was reduced, confirming that they capture task-relevant aspects of the real images beyond the power spectrum and demonstrating their importance for real-world visual tasks.