Abstract
Humans have rapid access to global structural and statistical regularities which allows them to extract the “gist” of an image, central to efficient assessment and orienting in complex environments. This ability is most likely based on our experience with the regularities of the natural world. Mammograms can be thought of as a specialized class of scenes and radiologists as experts who have tuned their visual system to regularities in these unusual scenes. Consequently, we have found that the gist of the abnormal in radiographs, viewed only for 500 milliseconds, allows radiologists to detect the presence of disease independent of the locus of any lesion and up to 3 years before the onset of cancer. Using three different approaches to image analysis we aimed to determine the textural components which contribute to the gist signal. We first computed the mammographic power spectrum via the Fourier transform of patches drawn from the parenchyma. Secondly, we employed a radiomics toolkit to compute 90 different textural features related to cancer. Thirdly, we used Portilla-Simoncelli analysis to identify differences across different textural scales. For frequency data, we compared spectra directly,and using machine learning techniques identified frequencies which appear most important for dividing normal from abnormal data. Within the radiomic feature spaces, we used the same process to identify the most important radiomic features. As the textural meaning of these features is obfuscated, we synthesized artificial mammograms which display a heightened degree of important radiomic markers, and pass them through a deep neural network trained to classify mammographic patches. This reveals the contribution of these radiomic features to abnormality, and their textural impact. Finally, through Portilla-Simoncelli modeling, we find significant differences across scales. Together, these metrics are indicative of where in the texture the experts’ visual system is tuning to the gist of the abnormal.