Abstract
The perceived material of an object is inherently tied to its environmental affordances, whether we are avoiding a wet, slippery floor or handling a fragile, crystal vase. Moreover, the way we interact with an object also affects how we visually sample it, as our gaze is guided to behaviorally relevant features. We hypothesized that human gaze behavior during object viewing should therefore be guided by the object’s perceived material and, if so, visual sampling of the object should reflect regularities in image structure that perceptually define the material. To test this, we characterised the relationship between human gaze behaviour, image structure, and material perception by combining eye tracking and deep-learning-based gaze predictions for 924 rendered photorealistic object stimuli. These stimuli were complex glossy objects rendered in natural illumination fields with varying reflectance properties, leading to a wide range of material appearances such as plastic, clay, ceramics, fabric, etc. Using DeepGaze IIE to predict fixation patterns on these images, we found that these patterns do indeed differ between stimuli, independent of object shape. This suggests that surface properties affect how we visually sample objects. Further, these differences in gaze patterns correlated with differences in perceived material. Finally, we found that variations in contrast, clarity, size, and colour of specular reflections of the objects predicted differences in both perceived material and gaze behaviour, providing a direct link between image cues, viewing behaviour, and affordance-related information. In a series of follow-up analyses, we then tested these model-based predictions in eye tracking data. Collectively, our results support the notion that both object perception and viewing behaviour are shaped by the affordances of the things we look at, and that both are determined by regularities in image structure caused by the complex yet characteristic ways that light is scattered by materials with different surface properties.