Generative constraints on image cues for perceived gloss

Abstract

Abstract

Abstract:

Abstract  Image structure is generated by distinct physical sources that include 3-D surface geometry, surface reflectance, and the light field. A fundamental problem in midlevel vision involves understanding how, and how well, the visual system separates images into their underlying sources. A large body of recent work has shown that the perception of gloss exhibits significant dependencies on both 3-D surface geometry and the light field in which the surface is embedded. We recently proposed that these dependencies arise from the visual system relying on a restricted set of dimensions of specular image structure that are heuristically used to estimate surface gloss (Marlow, Kim, & Anderson, 2012). Here, we report a series of studies that manipulate surface geometry and the structure of the light field to prospectively test whether these image cues predict perceived gloss for a broad range of surface geometries and light fields. We identify generative constraints on each of our proposed cues that allow us to predict which cue will vary most for a given surface geometry, reflectance function, and light field. Our psychophysical data reveal that our generative manipulations succeed in promoting the significance of the intended cue, which in turn increases their predictive value in observers' judgments of gloss. Our results suggest that the perception of gloss in static, monocular images relies on a heuristic weighting of cues to specular image structure, and offer a unified explanation of why surface geometry and the light field modulate perceived gloss.

Introduction

Light is structured by the interaction between the light field and the 3-D shapes and reflectance functions of the materials that fill our environment. The reflectance functions of numerous materials can be reasonably well approximated by a two component reflectance function (such as smooth dielectric materials), which are presumed to correspond to two different perceived surface qualities. The diffuse component of reflectance is associated with the lightness or color of a surface, whereas the specular component is associated with a surface's glossiness. One of the fundamental problems in material perception involves understanding how the visual system extracts these different reflectance properties from the information available in the eyes. The computation of gloss is considered to be especially difficult because it has been shown to depend on physical variables that have nothing to do with a surface's intrinsic specular reflectance. In particular, the perception of gloss has been shown to vary as a function of both a surface's 3-D shape, and/or the particular illumination field in which it is embedded (Doerschner, Boyaci, & Maloney, 2010; Fleming, Dror, & Adelson, 2003; Ho, Landy, & Maloney, 2008; Motoyoshi & Matoba, 2012; Nishida & Shinya, 1998; Obein, Knoblauch, & Viénot, 2004; Olkkonen & Brainard, 2010, 2011; Pont & te Pas, 2006; te Pas & Pont, 2005; Vangorp, Laurijssen, & Dutré, 2007; Wendt, Faul, Ekroll, & Mausfeld, 2010; Wijntjes & Pont, 2010). 

We recently proposed an explanation for the effects of shape and illumination on perceived gloss (Marlow, Kim, & Anderson, 2012). The key insight shaping our explanation is that perceived gloss is modulated by image variables that are only roughly correlated with a surface's true specular reflectance. In particular, we argued that the visual system relies on the coverage, contrast, and sharpness of specular reflections to generate our experience of gloss. Each of these cues corresponds to a different aspect of the image structure created by specular reflections. Coverage refers to the proportion of a visible surface area occupied by specular reflections; contrast refers to the difference in luminance between a specular reflection and its surround¹; and sharpness refers to the slope of the luminance gradient at the edge of a reflection. Figure 1 depicts two materials with different specular reflectance functions that have the same shape. For the Ward (1994) model used herein (and extensively elsewhere), there are two physical parameters that determine the intensity and structure of specular reflections in the image: the proportion of incoming light reflected in the outgoing direction of the specular lobe and the spread of the specular lobe around the main specular direction (specular roughness). The high gloss image in Figure 1 has both a higher proportion of light reflected in the specular direction, as well as a more narrowly focused specular lobe than the low gloss image in the same figure. Note that the high gloss surface appears to be completely covered with specular reflections, whereas the low gloss surface appears to contain only a few isolated specular highlights. Note also that both the contrast and sharpness of specular reflections are greater for the high gloss surface than the low gloss surface. Our model asserts that objects will appear glossier if they generate images with higher specular coverage, sharpness, and contrast. 

Figure 1

 

Images of a high gloss surface (A) and low gloss surface (B) rendered with the same shape and light field. The second panel from the left depicts the locations of visible specular reflections, which cover the entire high gloss material, but only partially cover the low gloss material. The right side panels illustrate that the reflections also have higher contrast and sharpness for the high gloss surface than the low gloss surface.

Although our previous work demonstrated a very high correlation between perceived gloss and these images cues, our methods were limited by the fact that our analysis was purely correlational (Marlow et al., 2012). We constructed a set of stimuli with a fixed reflectance function and varied the light field and the local surface relief. Different groups of observers ordered the stimuli according to perceived gloss, or according to each of the three image cues shown in Figure 1. We found that both the gloss judgments and the cue judgments exhibited complex interactions between local surface relief and the structure of the light field. We modeled the interaction in the gloss judgments as a weighted arithmetic mean of the cue judgments. The best fit weighted specular coverage more heavily than specular contrast and sharpness, and accounted for 94% of the variance in the gloss judgments. This analysis demonstrated that the gloss judgments could be understood using these three cues, but offered no principled basis for the cue weightings. 

The experiments reported here address two limitations of our previous study. First, to overcome the retrospective concerns inherent in correlational studies, we sought to assess the predictive capacity of our cues by prospectively manipulating physical properties of surfaces that will vary the relative contributions of our proposed image cues. Second, our previous work only evaluated a single three-dimensional shape that varied along a single dimension of relief height. We therefore sought to determine whether our model could explain the dependence of perceived gloss on shape for a much broader range of shapes, including those that have been used previously by others (e.g., Fleming et al., 2003; Nishida & Shinya, 1998; Olkkonen & Brainard, 2011). Our model predicts that manipulating 3-D shape or the light field can alter the coverage, contrast, and/or sharpness of specular reflections, which in turn should modulate perceived gloss. Moreover, our model asserts that observers will rely on whichever cue most reliably distinguishes the different surfaces that they are asked to compare. If the shapes being compared have a high variability in the amount of specular coverage, but little variability in either sharpness or contrast, then observers' gloss judgments should be most strongly predicted by specular coverage. Conversely, if a set of shapes can be found that modulate sharpness more than coverage and contrast, then observers' judgments should be predominantly predicted by the relative sharpness of specular reflections in those images. In what follows, we use these predictions to design stimuli that modulate specific image cues that we have proposed in our model. 

Generative constraints on image cues

The key insight shaping the prospective tests of our model is the recognition that the proposed gloss cues depend on specific geometric properties of surfaces and their interaction with the light field. Our goal is to vary surface geometry in such a way that it will generate surfaces that are more highly covered by specular reflections, generate higher contrast specular reflections, or contain reflections that generate sharper gradients. To anticipate what follows, we show that specular coverage can be varied by manipulating the range and distribution of surface normals in a set of stimuli, whereas specular sharpness and contrast can be manipulated by varying the distribution of curvatures along a surface. 

Generative constraints on specular coverage

For a purely specular surface, the reflected light exhibits a very precise relationship between the angle that a surface is viewed and the direction(s) of the incoming illumination. The reflection of the brightest illumination direction will be restricted to surface normals that bisect that illumination direction and the viewing direction. Thus, if a surface is constructed that contains a very narrow range of surface normals satisfying this constraint, the surface will appear covered in specular highlights (Figure 2, top left). By comparison, if a surface has a broad range of surface normals (such as a sphere, for which the distribution of surface normals is uniform and spans the entire range of visible directions), and the main directions of illumination are relatively narrow, then a smaller proportion of the surface will appear covered in specular highlights (Figure 2, top right). 

Figure 2

 

Images of the same specular material molded into different three-dimensional shapes and placed in different illumination contexts. Specular coverage exhibits an interaction between illumination and shape. Coverage varies between illuminations for the shape with a unimodal frequency distribution of surface normals (i.e., the perturbed plane), whereas coverage varies relatively little between the illuminations for the shape with a uniform frequency of surface normals (i.e., the sphere).

The frequency distribution of surface normals also influences how stable the coverage cue is to changes in the main direction(s) of illumination. A narrow distribution of surface normals along a bumpy plane will only appear covered in specular reflections for viewing directions in which the surface normals bisect the viewing direction and one of the strongest illumination directions. In contradistinction, a sphere contains the full range of possible surface normals, which vary smoothly across its surface. Consequently, the area of a sphere covered by specular highlights is more stable to variations in the positions of the viewer and the strongest illumination directions. 

It should be noted, however, that the relationship between surface normals and specular coverage (as well specular sharpness and contrast) breaks down for highly specular surfaces, which produce sharp reflections of the entire light field. Highly specular surfaces behave as mirrors, forming sharp images of the world distorted by the geometry of the surface (see the high gloss surfaces in Figure 1). However, for moderate to low gloss levels, visible reflections will only be detectible from the reflection of the brighter regions of the light field, and coverage will vary in proportion to the frequency of surface normals that reflect bright regions of the light field. 

Generative constraints on specular sharpness

Specular sharpness also varies with the surface geometry and the light field. The geometry of a surface can be decomposed into three progressively smaller spatial scales: megascale, mesoscale, and microscale (Koenderink & Van Doorn, 1996). The megascale geometry refers to the overall shape of a surface, which is planar in the left of Figure 2 and spherical in the right of Figure 2. The mesoscale geometry refers to the shape perturbations visible across the megascale geometry, such as the bumps across the perturbed plane in Figure 2. The microscale geometry refers to the shape variations across imperceptibly small spatial scales, which includes both diffuse reflectance (modeled as Lambertian reflectance) and specular reflectance. Microscale geometry is not included in the shape meshes used in computer graphics, and its influence on the appearance of the surface is instead incorporated into the reflectance function of the surface. Increasing amounts of microscale roughness increasingly scatters (blurs) specular reflections, which broadens the specular lobe of the reflection function (for isotropic reflectance models such as Ward's). For surfaces with some degree of microroughness (i.e., where the specular component is not a pure mirror), specular sharpness will vary as a function of the rate of change of the surface normals (curvature), and the range of surface normals at the meso and megascales. Specular reflections will be sharpest in image regions that run parallel to local directions of high curvature, and will be most shallow (stretched) along directions of low curvature. For example, in Figure 2 (top), the sphere generates a blurrier specular reflection than the bumpy plane even though they have the same reflectance functions and are embedded in the same illumination. This difference in specular sharpness is due to differences in the distribution of surface curvatures along the sphere and the bumpy plane, which are higher along the bumpy plane due to its mesotexture than the sphere. More precisely, the sharpness of specular reflections depends not only on intrinsic surface curvature, but also on surface slant and the distance from which it is viewed (Fleming, Torralba, & Adelson, 2004, 2009). The sharpness of the specular reflections in the image depends on the gradients of surface normals projected into the image plane, not the curvature rates on the surface per se, and the spatial scale at which these gradients are sampled. 

It is also possible to eliminate or diminish effects of curvature on sharpness by choosing particular combinations of the light field, specular reflectance functions, and 3-D shapes. Specular gradients mirror changes in illuminance across the light field, so the appearance of specular gradients along a surface requires that such gradients exist in the light field. Eliminating the directional component of a light field will eliminate any relationship between surface curvature and specular sharpness. Indeed, several studies have observed that specular materials fail to appear glossy in diffuse light fields (Dror, Willsky, & Adelson, 2004; Klinker, Schafer, & Kanade, 1988; Pont & te Pas, 2006). Moreover, like coverage, the effect of surface curvature on specular sharpness also depends on the range of surface normals present in the image. A surface with a small range of surface normals (such as the bumpy plane) may not bisect the viewing direction and the main directions of illumination, reducing specular sharpness because no specular highlights are generated. 

Generative constraints on specular contrast

The contrast of specular reflections can also vary as a function of surface geometry and the light field. However, the precise nature of this dependence is hard to quantify because there is currently no single measure of contrast that generalizes across different image types. The brightest regions along a glossy surface will be generated by the specular reflection of the brightest illumination directions in the light field, and the darkest regions will be regions in shadow, in the darkest regions of shading, or surface regions that specularly reflect the darkest components of the light field (lowlights; see Kim, Marlow, & Anderson, 2012). Any measure of specular contrast will depend on the intensity of the specular highlights relative to the darkest visible surface regions, but perceived contrast also depends on the spatial scale at which these variations occur (due to the contrast sensitivity function). The 3-D shape can interact with the light field in complex ways that modulate the perceived contrast of specular reflections. Specular contrast will increase for surfaces with a higher degree of surface relief that gives rise to deeper shadows or darker shading (Pont & Koenderink, 2005), or by increasing the number of surface normals that generate specular highlights of the main illumination directions. Specular contrast will decrease if the light field and surface geometry are oriented such that the specular reflections of the brightest regions of a light field are reduced or eliminated. Specular contrast can also decrease if a relatively smooth surface is completely covered by bright specular reflections, which elevates mean luminance along the entire surface. Specular contrast will also increase with the slant of the surface if its bi-directional reflectance distribution function (BRDF) exhibits Fresnel effects. 

Overview of experiments

In the experiments reported below, we manipulate the distribution of surface normals and surface curvatures relative to real-world illumination to systematically vary the coverage, sharpness, and contrast of specular reflections along a surface. The goal is to prospectively test whether these cues predict how perceived gloss varies in response to interactions between 3-D shape and illumination for surfaces with a fixed reflectance function. The shapes and light fields were chosen such that different cues would vary more across one set of stimuli than another. Experiment 1 parametrically varied global shape by varying the amount of local changes in curvature of a smoothly perturbed, highly specular sphere (Figure 3). We predict that increasing the surface curvatures should increase both the sharpness and contrast of specular reflections, but have less of an effect on coverage because of the relatively broad range of surface normals present in all of these stimuli. Experiment 2 attempted to increase the role of specular coverage by replacing the perturbed spheres used in Experiment 1 with bumpy planes containing a smaller ranger of surface normals (Figure 4). We varied both the global geometry and mesostructure of the surface, and varied the main directions of illumination. The planar structure of these surfaces should increase the reliance on specular coverage and decrease reliance on specular sharpness, which should vary less than coverage for these surface geometries and illuminations. Experiment 3 used smoother versions of the stimuli used in Experiment 2 to modulate specular sharpness, which we predict will increase the effectiveness of sharpness at the expense of coverage (Figure 5). 

Figure 3

 

Experiment 1 stimuli. All of the stimuli share the same reflectance function (i.e., are made from the same green, glossy material), but are embedded in different light fields and each column depicts a different shape. The shapes increase in complexity and local surface curvature from left to right, which increases the sharpness of the reflections. The curvature for each visible surface point in the stimuli was measured, and the bar graph plots the mean surface curvature of each shape as a function of the number of times the spherical surface was perturbed.

Figure 4

 

Experiment 2 stimuli. The stimuli have identical reflectance functions, but differ in their surface geometry and illumination. The surface is a perturbed plane that varies in relief stretch (between 0.45 and 7.2 cm) and has one of two poses: frontoparallel to the observer, or inclined by 45°. The brightest source in the light field illuminates the surface from either above or behind the observer (i.e., frontal illumination). These poses, light fields, and geometries strongly modulate specular coverage.

Figure 5

 

Experiment 3 stimuli. They were created by smoothing the surface geometries from Experiment 2 and increasing the spread of the specular lobe of the reflectance function, which is identical between the surfaces. Relief stretch increases surface curvature, and strongly modulates the sharpness of the reflections for these surfaces.

Experiment 1

The goal of Experiment 1 was to test if variations in specular sharpness predict how perceived gloss varies as a function of 3-D shape. The goal was to generate a set of shapes that modulate specular sharpness while minimizing variations in specular contrast and coverage. The perturbed spheres shown in Figure 3 satisfy both of these requirements: The number of perturbations (i.e., dents and bumps) imposed on the spherical surfaces increases local surface curvature, which in turn increases specular sharpness; there are no large variations in coverage because the frequency of surface normals differs little between shapes. Note that the positive correlation between surface curvature and specular sharpness holds regardless of the particular illumination tested, which is either brightest to the left of the observer (top row), above the observer (middle row), or behind the observer (bottom row). Although all of the stimuli in Figure 3 have the same reflectance function, we predict that people will perceive gloss to increase with the number of shape perturbations due to the increase in specular sharpness. 

Methods

Observers

Observers were first year undergraduate students at the University of Sydney. They were recruited from the school of psychology participant pool and received a small amount of course credit. They were naïve as to the purposes of our experiment prior to their participation, but were subsequently debriefed in accordance with the human subject protocol of the university. The number of observers differed slightly between experiments according to their availability on the days of testing. 

Stimuli

Six shapes were constructed by denting a spherical mesh of 40,962 equispaced vertices. The dents were created by displacing all of the vertices along a Cartesian axis. The azimuth and elevation of this displacement axis was randomly selected for each successive perturbation applied to all of the vertices in the mesh. The distance that each vertex was shifted along the displacement axis was determined by a two-dimensional sinusoidal grating projected orthographically onto the vertex mesh. The projection axis was the same as the displacement axis. The grating had amplitude equal to 5% of the diameter of the mesh. The frequency, phase, and orientation of the grating were randomly selected for each successive perturbation applied to all of the vertices in the mesh. The frequency of the grating was constrained so that it repeated between one and four times across the diameter of the surface to avoid extremes of spatial frequency. The mesh was dented in this way 16 times successively, which progressively increased rates of surface curvature. We recorded the position of the vertices after the zeroth, first, second, fourth, eighth, and 16th dents to construct the six shapes tested (see Figure 3). The surfaces were linearly scaled to have a visual angle of 10.9° and height, width, and depth equal to 11.5 cm. 

The surfaces reflected light according to the Ward reflectance model (Ward, 1992, 1994).  

 

where: 

•  
  R is the reflected radiance;
•  
  I is the incident radiance;
•  
  ρ_d is the diffuse reflectance;
•  
  ρ_s is the specular reflectance;
•  
  α is the specular roughness;
•  
  θ_i is the angle of incidence;
•  
  θ_r is the angle of reflectance; and
•  
  δ is the angle between the surface normal and the half vector bisecting the incident vector and the reflected vector.

Gloss level (ρ_s in the Ward model) was 0.05 (5% of incident light reflected specularly) and gloss roughness (α in Ward model) was 0.032, which resembles high gloss plastic, similar to values used in other studies (e.g., Doerschner et al., 2010; Fleming et al., 2003). The diffuse reflectance component (ρ_d in Ward model) was relative RGB: 0.1, 0.2, 0.1, which has International Commission on Illumination (CIE) xyY color coordinates x = 0.33 and y = 0.65 for our LCD display screen (22 in. Samsung SyncMaster 2233). 

The surfaces were illuminated by patterned light from all directions, simulating real world light fields. The light probes are measurements of the light field at a single position in space that were photographed from real world scenes: the “kitchen,” “grove,” and “campus” light probes (obtained from the Debevec Light Probe Image Gallery; Debevec, 1998). The light probes were oriented to vary the main direction of illumination, which is left of the test surface for kitchen, above for grove, and frontal for campus. These illumination conditions were created by aligning the viewing direction of the observer with the viewing direction of the camera that photographed each field before rotating the light probes anticlockwise by 90° around its zenith. 

We rendered an image of each combination of the six shapes and the three illuminations using Radiance software (Ward, 1994) with two interreflections. The interreflections are an important source of illumination, particularly in the shaded regions of high reliefs. The use of two interreflections here is reasonable because for the albedo we use (0.2), only 0.04 or 4% of the luminance is left after two bounces. We presented the images using the Psychtoolbox extension of Matlab (Brainard, 1997; Pelli, 1997). The camera was 60 cm from the surface to match the viewing distance of the observer from the display screen. Observers viewed the screen binocularly without disparity. The luminance of the renderings exceeded the range of the monitor (0.2 cd/m² to 75), so raw values greater than 65 cd/m² were gradually compressed (tone mapped) and had a displayed luminance equal to:  

 

 

Procedure

Our methodology relies on observers to measure each of our proposed images cues. The logic of this approach rests on the belief that there exists an image measurement that would capture each cue, but is currently unknown. Our method relies on the assumption that whatever the appropriate image measurements are, they will have to accurately capture how each of these cues is perceived in a given context. For example, a perceptually relevant image-based measurement of specular contrast will have to capture how specular contrast is perceived in a given image. A similar assumption holds for both specular coverage and sharpness. The procedures described below explain how each cue to gloss was psychophysically measured. An advantage of this methodology is that it makes no assumptions about how the visual system measures and scales each of these cues, but instead, measures them directly. 

On each trial, observers compared two stimuli, which were presented side by side on the screen separated by a horizontal gap of 10.5°. Each of the 18 stimuli in the set was compared once with each of the other 17 stimuli in a block of 153 trials. The order of the trials was randomized. One group of observers selected the stimulus that appeared to be made from the glossiest material (n = 13). Each of the other groups judged either the stimulus with the higher specular contrast (n = 9), the higher specular sharpness (n = 9), or the higher amount of specular coverage (n = 9). Observers judging contrast were instructed that specular contrast is the difference between the brightness of the specular reflection and the brightness of its surround. Specular sharpness was defined as the perceived slope of the luminance gradient at the edge of the reflections. Coverage was defined as the proportion of a visible surface area that appears to be covered by bright specular highlights and the visible edges of dim specular reflections. Observers were instructed to fixate three points on each stimulus before responding: the left side of the object, the right side, and the top. The task was self-paced and its duration was typically 7 s per trial and 20 min per block. 

Results and discussion

Figure 6 depicts observer's judgments of perceived gloss, and the perceived contrast, sharpness, and coverage of specular reflections. The upper left graph plots the percentage of trials that observers judged a given surface to be glossier than the other 17 stimuli. These percentages are plotted as a function of the number of shape perturbations imposed on the spherical mesh. The three light fields tested are distinguished by the color of the data points. The judgments of the three image cues are plotted in the same manner as the gloss judgments and are shown in upper right graph (coverage), bottom left graph (sharpness), and bottom right graph (contrast). The gray arrows show the variance shared between different cues, and the black arrows show the variance shared between each cue and the gloss data. 

Figure 6

 

Results of Experiment 1. The upper left graph plots the percentage of trials that each stimulus appeared glossier than the comparison stimuli. The results for each light field are shown in black (kitchen), red (grove), and green (campus). The number of shape perturbations increases along the x axis. The cue judgments are plotted in the upper right graph (coverage), bottom right (contrast), and bottom left (sharpness). The R² correlation between the gloss judgments and each cue is shown on the black arrows. The R² →← correlation between each pair of cues is shown on the gray arrows. The gloss data was modeled as a weighted linear sum of the coverage (c_{i = 1}), contrast (c_{i = 2}), and sharpness (c_{i = 3}) data. The weights associated with the best fit were: 0.18 for coverage (w_{i = 1}), 0.13 for contrast (w_{i = 2}), and 0.69 for sharpness (w_{i = 3}). The best fit accounts for 97% of the variance and is depicted by the dashed lines in the upper left graph. Error bars are SEM.

Although the surfaces have identical reflectance functions, we predicted that they would appear to differ in gloss as a function of the three cues that covaried with their surface geometries and the illumination in which they were embedded. Specifically, we predicted that the shape perturbations imposed on the spheres would increase rates of local surface curvature, which should increase both specular sharpness and specular contrast, which in turn should increase perceived gloss. The results are consistent with this prediction. The shape perturbations increased both the perceived gloss of the surfaces and the perceived sharpness and contrast of the specular reflections. Moreover, there are high correlations between judgments of gloss, sharpness, and contrast, spanning R² values of 0.94 to 0.96. The effect of shape on specular contrast presumably arises from the proximity of specular highlights to the darkest regions of the surface, which correspond to regions of dark diffuse shading and/or the specular reflection of a dark surface in the world that is surrounded by lighter surfaces (i.e., lowlights, Kim et al., 2012). For example, the darkest regions of the unperturbed spheres (Figure 3, left column) appear at the edge of the surface, whereas the darkest regions of the maximally perturbed spheres (right column) are more widely distributed across the surface. The dark regions of the unperturbed spheres therefore have larger angular separations from specular highlights than do the dark regions of the maximally perturbed spheres. Thus the shape perturbation may have increased perceived specular contrast by increasing the proximity of the highlights to dark regions of the surface. The shapes we tested were designed to modulate specular sharpness and contrast more than coverage, which is consistent with observers' coverage judgments. Specular coverage has the lowest correlation with perceived gloss (R² = 0.74) and has the lowest intercue correlation (R² of 0.66 and 0.72). 

Another interesting feature of the data is that perceived gloss also depends on the particular light field in which the surfaces were embedded. All of the shapes appear glossier in the kitchen light field than either the grove or campus light fields. These differences in perceived gloss appear to be due to the differences in the sharpness of the reflections between these illuminations, which may depend in turn upon the sharpness of edges in the captured light fields. The perceived sharpness of the reflections is higher for the kitchen light field than either the grove and campus light fields, independently of the shape of the test surface. This suggests that the effects of illumination can be explained in the same way as the effects of shape: Both modulate perceived gloss by varying the same set of specular cues. 

The gloss data are well modeled as an additive mixture of the perceived sharpness, contrast, and coverage of specular reflections. The weights were found by varying each cue's weight in steps of 0.005 to find the best fit, and were constrained to sum to one (i.e., were treated as proportions). The goodness of fit was assessed by summing the least squares residual between the gloss judgments and the weighted average of the cues. The best fit is depicted by the dashed lines in the upper left graph of Figure 6 and accounts for 97% of the variance in the gloss judgments. The cue weightings associated with the best fit were 0.69 for sharpness, 0.18 for coverage, and 0.13 for contrast. It is important to note, however, that these cue weightings may not reflect the relative contribution of the cues to the gloss judgments. Specular sharpness and contrast correlate highly in this experiment, so if we eliminate sharpness from the model, the best fit still provides a good model of the gloss judgments (R² = 0.93 without sharpness, down from 0.97). Observation suggests that the changes in specular sharpness across our stimuli are perceptually larger than the correlated changes in specular contrast, and informs our view that observers probably relied more heavily on sharpness than contrast. Previous studies have disentangled specular sharpness and contrast and have shown that both cues can independently influence perceived gloss (Billmeyer & O'Donnell, 1987; Hunter & Harold, 1987; Pellacini, Ferwerda, & Greenberg, 2000). The contribution of the present work is to show that these cues are sufficient to understand the seemingly erratic way perceived gloss is modulated by variation in 3-D shape and/or the illumination (Doerschner et al., 2010; Fleming et al., 2003; Ho et al., 2008; Motoyoshi & Matoba, 2012; Nishida & Shinya, 1998; Obein et al., 2004; Olkkonen & Brainard, 2010, 2011; Pont & te Pas, 2006; te Pas & Pont, 2005; Vangorp et al., 2007; Wendt et al., 2010; Wijntjes & Pont, 2010). 

Experiment 2

The goal of Experiment 2 was to attempt to promote the importance of specular coverage in observers' gloss judgments by using surfaces that contain a more narrow distribution of surface normals. This was accomplished by using perturbed planes, rather than the perturbed spheres used in Experiment 1. We located the brightest illumination direction either above the observer or along the observer's viewing direction, and positioned the perturbed plane to have either a frontoparallel or oblique orientation relative to the viewing direction. Both of these manipulations will strongly modulate the proportion of a surface covered in specular reflections. Extremely high coverage will occur for a plane with a narrow range of surface normals that is inclined in depth by 45° when illuminated from above (i.e., left column, second row of Figure 4), or for frontoparallel oriented planes illuminated along the viewing direction (i.e., left column, third row). Switching these light fields while preserving the pose of the planes will eliminate coverage (top and bottom rows of Figure 4), because the restricted range of surface normals in the plane will no longer bisect the viewing direction and the brightest illumination directions. 

We reasoned that it should be possible to modulate coverage by introducing surface relief on a variety of spatial scales (moving from left to right in Figure 4). When the plane and light field are configured to generate high coverage (as in the middle rows of Figure 4), perturbing the plane will reduce coverage because it will broaden the range of surface normals present along the surface. When the unperturbed plane and the illumination are arranged to generate zero coverage (as in the top left and bottom left rows of Figure 4), perturbing the plane should increase coverage by increasing the range of surface normals that span the surface. 

Another issue influencing the design of the stimuli was the possibility that observers would judge gloss on the basis of specular sharpness instead of the large differences in coverage that the stimuli were designed to generate. Specular sharpness is difficult to equate between different surface geometries because small differences in surface curvature can have a large influence on the slope of specular gradients. We sought to minimize the variance of the sharpness cue by constructing a set of surfaces that all have very sharp specular reflections. For the high degree of mesostructure present in our surfaces, we reasoned that differences in the sharpness of specular gradients should be difficult to detect due to the spatial resolution of the visual system (i.e., the specular highlights will be extremely punctate due to the scale of the mesostructure). 

Method

The surface geometries were generated by introducing relief into a high resolution planar mesh of vertices (240,000 vertices) that was 8.5 cm square. Relief was generated by displacing each vertex in a direction perpendicular to the plane. The distance component of the displacement vector differed between vertices and was determined by a noise pattern that was mapped orthographically onto the plane. The noise had a power spectral density of 1/f² (i.e., Brownian noise) to simulate the relief of natural surfaces (Mandelbrot, 1983). The amplitude of the noise was linearly scaled to generate five surfaces differing in relief stretch. The depth between the nearest and farthest vertices was equal to 0.45 cm, 0.9 cm, 1.8 cm, 3.6 cm, or 7.2 cm. The surfaces were set either frontoparallel to the observer or inclined in depth by 45°. 

The illumination from above condition and the frontal illumination condition were both constructed by rotating the kitchen light probe around the cardinal axes in a specific sequence: around the vertical axis by 0° for the frontal illumination condition and by 180° for the above illumination condition; around the horizontal axis by 30° for the frontal illumination condition and by 60° for the above illumination condition (shifting the zenith of the light probe toward the observer); and around the viewing direction of the observer by 180° for the frontal illumination. Prior to this sequence of rotations, the light probe was set so to align the global z axis (i.e., the viewing direction of the observer) with the viewing direction of the camera that had photographed the light field. 

Gloss roughness (α in Ward model) was reduced to 0.01 from 0.32 in Experiment 1 in order to increase the sharpness of the reflections. Images of the surfaces were rendered, tone mapped, and displayed as in Experiment 1. Each combination of relief (five levels), pose (two levels), and illumination (two levels) was paired once with each other stimulus in a block of 190 trials. The gloss of the material was judged by a different group of observers (n = 13) to the groups judging the three dimensions of specular reflections: coverage (n = 8), sharpness (n = 8), and contrast (n = 9). Each group received the same instructions described for Experiment 1. 

Results and discussion

Figure 7 compares measurements of perceived gloss to measurements of specular contrast, sharpness, and coverage. The graphs plot the percentage of trials that a given surface was judged to have a greater gloss (upper left graph), specular coverage (upper right), specular sharpness (lower left), and specular contrast (lower right) in comparison to the other 19 surfaces. These percentages are plotted as a function of the relief height of the surfaces. The color of the data points depicts the orientation of the strongest directional component of the light probes, which illuminated the surfaces from above the observer (red series) or along the viewing direction (black series). The shape of the data points depicts the pose of the surfaces, which were either set frontoparallel to the observer (triangular data points) or were inclined in depth by 45° (square data points). 

Figure 7

 

Results of Experiment 2. The layout of the figure is described in the caption of Figure 6. The color of the data points distinguishes the frontal illumination condition (red) from the illumination from above condition (black). The shape of the data points distinguishes the frontal plane surfaces (triangular data points) from surfaces inclined in depth by 45° (square data points). The gloss data were modeled as a weighted linear sum of the coverage (c_{i = 1}), contrast (c_{i = 2}), and sharpness (c_{i = 3}) data. The weights →← associated with the best fit were: 0.92 for coverage (w_{i = 1}), 0.00 for contrast (w_{i = 2}), and 0.08 for sharpness (w_{i = 3}). The best fit accounts for 91% of the variance and is depicted by the dashed lines in the upper left graph. Error bars are SEM.

The data exhibit the predicted pattern of interaction between pose, illumination, and relief. Low reliefs were expected to yield high specular coverage in the appropriate illumination and low coverage for the other illumination. These extremes were expected to moderate as relief stretch increases the variance of surface normals in the planes. Both the gloss judgments and the cue data exhibit this interaction structure. The gloss data correlated most highly with the coverage cue (R² = 0.87) in the stimuli designed to modulate this cue, and correlated less strongly with the contrast (R² = 0.63) and sharpness (R² = 0.74) data. The shared variance of the cues was equal to 0.85 between coverage and contrast, 0.72 between coverage and sharpness, and 0.55 between contrast and sharpness. An important feature of the data that is not captured by these correlations is that specular sharpness has the lowest variability between conditions, and the highest variability between observers. These two components of the variance of the sharpness data suggest that we succeeded in engineering surfaces that did not appear to strongly differ in specular sharpness. The gloss judgments are therefore unlikely to have been based solely on the sharpness of the specular reflections or on the contrast of the reflections, which rose more steeply with relief stretch than did perceived gloss. Rather, the results suggest that these surfaces greatly increased the contribution of the coverage cue to gloss judgments compared to the surfaces tested in Experiment 1. 

A high proportion of the variance in the gloss data can be accounted for by a weighted average of the three specular reflectance cues. The best fit is depicted by the dashed and dotted lines in the top left graph of Figure 7 and accounts for 91% of the variance in the gloss data. This fit was obtained by assigning a large weight to coverage (0.92), a low weight to sharpness (0.08), and zero weight to contrast (0.00). Importantly, if coverage is eliminated from the model, sharpness and contrast together only account for 78% of the variance, down from 91% with coverage. These results provide evidence that observers' gloss judgments were primarily based on specular coverage in these stimuli. 

Experiment 3

The goal of Experiment 3 was to attempt to increase the relative contribution of specular sharpness and contrast to gloss judgments. We attempted to increase the variability of these cues by modifying the Experiment 2 stimuli. To accomplish this goal, we increased the spread of the specular lobe and smoothed the mesoscale structure of the surface geometries, producing the stimuli shown in Figure 5. The specular reflections from these stimuli generate blurrier gradients than did the stimuli in Experiment 2, so differences in specular sharpness should be easier to discriminate across the stimulus set. We attempted to increase both specular sharpness and contrast by increasing surface relief, which increases local surface curvature and the depth and extent of shadowing. 

Method

We smoothed each of the surface geometries tested in Experiment 2 by spatially filtering the noise pattern that was used to generate relief. The filter had a low pass profile that fully transmitted frequencies lower than 0.23 cycles/cm. The transmittance of the filter gradually fell off across higher spatial frequencies according to a Guassian profile with mean equal to 0.23 cycles/cm and a standard deviation equal to 0.35 cycles/cm. The roughness parameter (ρ_a in Ward model) of the specular lobe of the reflectance function was increased from 0.01 in Experiment 1 to 0.16 in this experiment to further blur the reflections. The surfaces had the same poses and light fields as in Experiment 2, and their images were rendered, tone mapped, and displayed using the same methods. The gloss judgments were collected from the same group of 13 observers that judged gloss in Experiment 2. Judgments of specular coverage, sharpness, and contrast were collected using the same observers that judged these cues in Experiment 2. We counterbalanced the order of Experiments 2 and 3 for these observers. 

Results and discussion

The results of Experiment 3 are shown in Figure 8 and follow the same format as the data for Experiment 2. The data show that our attempt to promote the significance of specular sharpness and contrast in gloss judgments was successful. Gloss judgments share 91% of variance with specular sharpness and 75% of variance with specular contrast, whereas specular coverage only accounted for 20% of the variance. The shared variance between cues is 60% for contrast and coverage, 54% for sharpness and contrast, and 7% between sharpness and coverage. These values provide further evidence that each of these specular cues provides some independent information that modulates perceived gloss. The judgments of sharpness, contrast, and gloss also vary as predicted. The increase in local rates of curvature generated by our manipulation of relief increased the steepness of specular gradients, increasing sharpness, and increased the amount of shadowing in the neighborhoods of specular highlights, which increased specular contrast. The effects on contrast were most pronounced when the direction of the primary light source were not directed along the axis of the surface relief. 

Figure 8

 

Results of Experiment 3. The layout of the figure is described in the caption of Figure 7. The gloss data were modeled as a weighted linear sum of the coverage (c_{i = 1}), contrast (c_{i = 2}), and sharpness (c_{i = 3}) data. The weights associated with the best fit were: 0.00 for coverage (w_{i = 1}), 0.53 for contrast (w_{i = 2}), and 0.47 for sharpness (w_{i = 3}). The best fit accounts for 97% of the variance and is depicted by the dashed lines in the upper left graph. Error bars are SEM.

The cue weightings of our model also imply that gloss judgments were informed by specular sharpness and contrast in these stimuli. The best fitting model accounts for 97% of the variance and assigns a high weight to sharpness (0.47) and contrast (0.53), with no weight attributed to coverage. If contrast is removed from the model, sharpness and coverage together account for 95% of the variance. However, if sharpness is removed from the model, the best fitting solution relies solely on contrast, which only accounts for 75% of the variance in gloss judgments. 

For the fractal surfaces used in Experiment 2, relief stretch decreased perceived contrast by shrinking the reflections to punctate points. The smoothed surfaces used in Experiment 3 do not shrink the reflections to the same extent, and contrast tends to increase rather than decrease as a function of relief stretch when the illumination direction is relatively oblique to the line of sight. The interdependencies between the geometric structure of the surfaces and the direction of the predominant sources of illumination are quite complex. Yet despite this complexity, their effects on our proposed specular image cues are quite predictable, and appear to provide a coherent account of the dependence of perceived gloss on ostensibly irrelevant scene parameters. 

Experiments 4(a–c)

One possible concern in measuring specular image cues psychophysically is that observers' experience of gloss contaminated their cue judgments. Experiments 4(a–c) control for this possibility by using stimuli that resemble photographic (black and white) negatives for the cue judgments. We remeasured each specular cue by inverting the polarity of the luminance gradients and averaging the different color channels to obtain a gray scale image. The reflections in the negatives appear as patches of dark pigment on gray surfaces. Observers judged the coverage, contrast, and sharpness of these patches of dark pigment, providing rough estimates of the variance of the cues in the original stimuli. We predict that the control judgments will exhibit similar trends to the original cue judgments: an increase in sharpness with surface curvature for Experiment 1 and 3 stimuli and an interaction in coverage judgments between the pose, light field, and relief height in Experiment 2 and 3 stimuli. 

Method

The three sets of stimuli in Experiment 4 were variants of the stimuli from Experiments 1, 2, and 3. The RGB color channels were averaged with equal weights to convert each stimulus to grayscale. The inversion of contrast polarity was achieved by subtracting the maximum luminance (75 cd/m²) and then discarding the negative sign (i.e., we inverted the displayed luminance, not the color look up table values). The background of the negative stimuli was also reversed from black to white (75 cd/m²). 

The stimuli were presented in pairs using the same procedure as each of the previous experiments. Variants of the stimuli from one experiment (e.g., Experiment 1) were not compared to variants of the stimuli from another experiment (e.g., Experiment 2). The specular reflections appear as patches of dark pigment in the control stimuli, so we instructed observers to judge the coverage, sharpness, and contrast of dark pigment. One group of observers (n = 9) judged each of the cues in the Experiment 1 control stimuli in separate blocks in a randomly selected order. Three independent groups of observers judged coverage (n = 7), contrast (n = 6), or sharpness (n = 8) for both the Experiment 2 and 3 control stimuli in a counterbalanced order. 

Results and discussion

Figures 9, 10, and 11 plot the cue measurements obtained from each set of control stimuli. The sharpness and contrast judgments in control stimuli correlate highly with the original cue judgments for each set of stimuli spanning R² values of 0.74 to 0.85. These correlations are impressive given that the control stimuli have a distorted contrast and sharpness due to the nonlinearity in luminance processing and the absence of color information. The control data therefore indicates that the original contrast and sharpness judgments were largely uncontaminated by percepts of gloss. The coverage data from the control stimuli correlates highly with the original judgments for the Experiment 3 stimuli (R = 0.92), but correlates poorly for the Experiment 2 stimuli (R = 0.51), and for the Experiment 1 stimuli (R = 0.17). It seems plausible that these two sets of control stimuli probably do not replicate the original judgments because small, dim reflections are imperceptible in the control stimuli. Additionally, the brightest specular reflections in Experiments 1 and 2 are significantly modulated by our tone-mapping procedure, which could introduce further differences between the normal and control stimuli. Nonetheless, we believe that the original coverage judgments are, on the whole, unlikely to be contaminated by perceived gloss. This conclusion is supported by three sources of evidence: the replication of coverage judgments in the Experiment 3 stimuli (which did not possess small, dim reflections); an agreement between the original coverage judgments and phenomenology; and an agreement between the predicted and the obtained variance of the coverage judgments in Experiments 2 and 3. 

Figure 9

 

Results of control Experiment 4a. The upper left graph replots the gloss data from Experiment 1 (see Figure 6). The cue judgments plotted in the other graphs were obtained from variants of the Experiment 1 stimuli that do not appear glossy. The gloss data was remodeled as a weighted linear sum of the coverage (c_{i = 1}), contrast (c_{i = 2}), and sharpness (c_{i = 3}) judgments obtained in the absence of a gloss percept. The weights associated with the best fit were: coverage 0.05 (w_{i = 1}), contrast 0.49 (w_{i = 2}), and sharpness 0.46 (w_{i = 3}). The best fit accounts for 73% of the variance and is depicted by the dashed lines in the upper left graph. Error bars are SEM.

Figure 10

 

Results of control Experiment 4b. The upper left graph replots the gloss data from Experiment 2 (see Figure 7). The cue judgments plotted in the other graphs were obtained from variants of the Experiment 2 stimuli that do not appear glossy. The gloss data was remodeled as a weighted linear sum of the coverage (c_{i = 1}), contrast (c_{i = 2}), and sharpness (c_{i = 3}) judgments obtained in the absence of a gloss percept. The weights associated with the best fit were: coverage 0.42 (w_{i = 1}), contrast 0.00 (w_{i = 2}), and sharpness 0.58 (w_{i = 3}). The best fit accounts for 85% of the variance and is depicted by the dashed lines in the upper left graph. Error bars are SEM.

Figure 11

 

Results of control Experiment 4c. The upper left graph replots the gloss data from Experiment 3 (see Figure 8). The cue judgments plotted in the other graphs were obtained from variants of the Experiment 3 stimuli that do not appear glossy. The gloss data was remodeled as a weighted linear sum of the coverage (c_{i = 1}), contrast (c_{i = 2}), and sharpness (c_{i = 3}) judgments obtained in the absence of a gloss percept. The weights associated with the best fit were: coverage 0.00 (w_{i = 1}), contrast 0.00 (w_{i = 2}), and sharpness 1.00 (w_{i = 3}). The best fit accounts for 93% of the variance and is depicted by the dashed lines in the upper left graph. Error bars are SEM.

Although the control data has reduced the overall fit of our model, it can still account for an impressive proportion of the variance in the gloss judgments. The top left panel of Figures 9, 10, and 11 shows the best fit to the gloss data provided by the control data. The best fit accounts for 73% of the variance in the gloss judgments for Experiment 1, 85% of the variance for Experiment 2, and 93% of the variance for Experiment 3. The cue weightings for the three sets of stimuli were equal to: 0.05 for coverage, 0.49 for contrast, and 0.46 for sharpness for the Experiment 1 stimuli; 0.42 for coverage, 0.00 for contrast, and 0.58 for sharpness for the Experiment 2 stimuli; and 0.00 for coverage, 0.00 for contrast, and 1.00 for sharpness for the Experiment 3 stimuli. The R² correlation between the cues and perceived gloss (shown on the black arrows in each figure) exhibits the same pattern as the cue weightings. The control data accounts for less variance than the original cue judgments because the contrast polarity inversion distorts the cue data due to the nonlinearity in luminance processing, and also eliminates all of the color information that can be used to distinguish specular structure from other sources of image variability. The correlations and the cue weightings nonetheless vary between the different sets of stimuli as predicted. Sharpness receives a high weight for the stimuli that were designed to strongly modulate sharpness (0.47 in Experiment 1 stimuli and 1.00 in Experiment 3 stimuli), and coverage receives a substantial weight for the Experiment 2 stimuli (0.42) that were designed to strongly modulate coverage. The rough estimates of the cue values obtained from the control stimuli therefore corroborate the prospective tests of Experiments 1, 2, and 3. 

General discussion

The goal of the experiments presented herein was to provide a prospective test of our previously proposed model (Marlow et al., 2012) that was designed to explain why materials with the same reflectance function can appear to vary in perceived gloss. We manipulated the three-dimensional shapes, poses, and light fields in ways that were designed to systematically alter the variation of our previously proposed cues to gloss: specular coverage, contrast, and sharpness. Consistent with previous studies (Doerschner et al., 2010; Fleming et al., 2003; Ho et al., 2008; Marlow et al., 2012; Nishida & Shinya, 1998; Obein et al., 2004; Olkkonen & Brainard, 2010, 2011; Pont & te Pas, 2006; te Pas & Pont, 2005; Vangorp et al., 2007; Wendt et al., 2010; Wijntjes & Pont, 2010), we found that the perception of gloss varied significantly as a function of 3-D surface shape and the structure of the illumination. The dependence of perceived gloss on scene variables that are ostensibly irrelevant (i.e., unrelated to the intrinsic gloss level of a surface) suggests that the visual system conflates intrinsic surface reflectance properties with extrinsic scene variables. The purpose of the present paper is to understand what causes this pattern of conflation. 

We have previously argued that the dependence of perceived gloss on 3-D shape and illumination can be understood as arising from the presence of specular image cues that the visual system uses to generate our percept of gloss. Our model assumes that the perceptual errors in perceived gloss—failures of gloss constancy—arise because the visual system relies on a set of image cues that are, in some contexts, only weakly correlated with the true specular reflectance of a surface. Specifically, we have previously argued that the visual system gauges specular reflectance from the contrast, sharpness, and coverage of visible specular highlights across a surface. Each of these dimensions of specular structure is correlated with the specular component of reflectance as defined physically (Figure 1), but they can also be modulated by the 3-D shape of a surface and the structure of the illumination. Thus, these cues may provide a possible explanation for both veridical percepts of gloss (to the extent that such percepts occur or have meaning), as well as errors in gloss constancy such as those described herein. 

Consistent with our previous findings, the results reported herein demonstrate that perceived gloss correlates with changes in specular coverage, contrast, and sharpness. We measured these changes by having different groups of observers order stimuli according to the strength of perceived gloss, or according to the strength of our three cues: specular coverage, contrast, and sharpness. We then modeled the structure of the gloss judgments as a weighted average of these cues. The model accounts for 97% of the variance in gloss judgments in Experiments 1 and 3, and 91% of the variance in Experiment 2. These fits are impressive considering that the gloss judgments and the cue judgments were obtained from different groups of observers, which was also controlled for in Experiment 4. 

One of the main goals of the present experiments was to provide a more prospective test of the proposed cues to gloss by designing stimuli that would directly modulate the salience of different specular image cues on the basis of how particular surface geometries interact with a given light field. We identified attributes of shape and illumination that modulate specular coverage and sharpness in a predictable manner, and found that perceived gloss is also modulated in a manner consistent with the variance of these cues. These successful prospective tests provide significantly stronger evidence of a causal relationship between the cues and perceived gloss than does our previous study. Nonetheless, it should be noted that it is always possible that our proposed cues are only correlated with some (unspecified) sources of information that modulate our experience of gloss. However, some strong evidence for the causal role of these proposed cues has been provided in studies manipulating each cue in isolation, holding other aspects of image structure constant (Beck & Prazdny, 1981; Berzhanskaya, Swaminathan, Beck, & Mingolla, 2005; Billmeyer & O'Donnell, 1987; Fleming et al., 2003; Hunter & Harold, 1987; Pellacini et al., 2000). The distribution of specular reflections has been manipulated independently of their contrast and sharpness, suggesting a causal role of coverage in the perception of gloss (Beck & Prazdny, 1981; Berzhanskaya et al., 2005). Evidence for a causal role of contrast and sharpness has been provided by studies that dissociate these two cues by varying both the diffuse and specular components of reflectance (Billmeyer & O'Donnell, 1987; Hunter & Harold, 1987; Pellacini et al., 2000). These studies provide important demonstrations of the independence of the contrast, sharpness, and coverage cues, which were often modulated similarly by the 3-D shapes that we tested. Moreover, these studies have shown that the contrast and sharpness cues increase monotonically with specular reflectance when 3-D shape and illumination are held constant, informing veridical as well as nonveridical judgments of gloss. Taken as a whole, these studies suggest that the visual system heuristically combines these dimensions of specular image structure to generate our experience of surface gloss. 

Our results demonstrate that our proposed specular image cues predict perceived gloss for an array of different 3-D shapes. We constructed 3-D surfaces that would generate markedly different patterns of specular reflections, ranging from sharp images of the surrounding environment, to localized, punctate highlights, and to broad, blurry highlights. The purpose of these manipulations was to see whether our proposed cues could accurately predict perceived gloss for arbitrary surface geometries and light fields for surfaces possessing fixed reflectance functions. Some of the most commonly used stimuli in studies of gloss perception have been smooth planes and spheres, which generate clear images of the surrounding environment inside the body of the reflecting surface (Doerschner et al., 2010; Fleming et al., 2003; Olkkonen & Brainard, 2010; Pont & te Pas, 2006; te Pas & Pont, 2005; Vangorp et al., 2007). In such contexts, the perception of gloss is essentially identical to a form of transparency perception. For planar and convex surface geometries, an image of the world appears inside or behind the depth of the reflecting surface. In such cases, there are (at least) two surfaces visible along the same line of sight: the body of the glossy surface, and the image of the world inside of (behind) the glossy surface. A second class of gloss stimuli include a variety of convex shapes that vary in their macro or global surface geometries, such as the perturbed spheres we used in Experiment 1 (Olkkonen & Brainard, 2011; Vangorp et al., 2007). The specular reflections of this class of shapes stretch along lines of minimal curvature and compress along lines of maximal curvature, such that environmental objects are not easily recognized in the specular reflections of the light field. A third class of surfaces includes surfaces that vary in both macro structure (such as the deformed spheres) and in mesostructure (surface texture or relief), such as the fractal surfaces that we used in Experiment 2. The smallest scale mesostructure contains very high rates of surface curvature that compress specular reflections into punctate highlights. In Experiment 3, we generated smoothed versions of the meso structure of the surfaces in Experiment 2 and increased the blur of the specular lobe by increasing the amount of simulated micro surface scatter. For all of these surface manipulations, we found that gloss judgments were well accounted for by observers' judgments of the coverage, sharpness, and contrast of specular reflections. These correlations generalize our earlier results obtained with a fixed shape and scale of 3-D structure (Marlow et al., 2012). Taken in conjunction with our previous results, the present work suggests that the proposed specular image cues provide a unified account of the effects of surface geometry and the light field on the perceived gloss of surfaces that are well approximated by the Ward model (i.e., dielectric materials such as plastic; Doerschner et al., 2010; Fleming et al., 2003; Ho et al., 2008; Nishida & Shinya, 1998; Obein et al., 2004; Olkkonen & Brainard, 2010; 2011; Pont & te Pas, 2006; te Pas & Pont, 2005; Vangorp et al., 2007; Wendt et al., 2010; Wijntjes & Pont, 2010). Our experiments do not indicate what role these cues play in the perception of gloss in surfaces with reflectance functions that are not well captured by this model. 

Limitations and future work

One limitation of the approach described herein is that we did not measure the proposed gloss cues directly from images. Rather, we relied on independent sets of observers to provide psychophysical estimates of each cue. There are a number of motivations for this approach, and a number of assumptions that must hold to justify its validity. First, we assume that observers who judge a particular specular cue are not merely judging gloss. The fact that the different cue judgments are not highly correlated with gloss judgments in all of our experiments suggests that observers were not simply judging gloss when asked to judge cues. The r² values ranged from 0.20 to 0.96 between different cue and gloss judgments, whereas the cue combination model only spanned the range from 0.91 to 0.97. We also found that the cue combination data from the inverted images provided a reasonably good match between gloss judgments and the cue combination model measured from the inverted images, generating r² values ranging from 0.73 to 0.93. Although this is not as impressive as our model fits for the original glossy images, they are still quite good considering that the inverted images contained no color information, and there is no obvious way to equate the contrast, coverage, and sharpness of specular structure in the inverted and uninverted images. 

A second potential concern that could be raised with our cue measurement approach is that it uses one perceptual output (which we label cues) to match another perceptual output (perceived gloss). In the cue tasks, observers are instructed to judge particular properties of specular image structure, which is itself a perceptual output: Specular structure refers to a perceptual classification of image structure, it does not identify a specific image property or specify how that property should be identified from images. There is nothing in our model (or any other model) that specifies precisely how the specular contribution to the image is derived from images. We view this as one of the fundamental problems in midlevel vision, and as an active area of research (Anderson & Kim, 2009; Beck & Prazdny, 1981; Blake & Bülthoff, 1990; Doerschner et al., 2011; Kim, Marlow, & Anderson, 2011, 2012; Marlow, Kim, & Anderson, 2011; Mury, Welchman, Blake, & Fleming, 2013). We have previously shown that the appearance of image structures as specular highlights depends upon their congruence with a surface's diffuse shading profile (Anderson & Kim, 2009; Kim et al., 2011; Marlow et al., 2011). The concept of congruence applies to both the local orientation flows in the neighborhood of the highlights, as well as the local brightness of the image regions in the neighborhood of a potential highlight. We are, however, quite far from being able to automatically segment the specular component of an image from other sources of image structure, such as reflectance, shading, and translucency. The goal of the present paper was to derive an understanding of the dimensions of specular reflections that modulate our experience of perceived gloss; our model remains mute on the question of how the specular component is derived (but see Marlow et al., 2012, supplementary information). 

One of the main motivations for using observers as measurement devices for our image cues relates to the inherent complexity in deriving a psychophysical model of each cue. Contrast, for example, is a term used extensively in vision, yet we still do not have a definition of contrast that captures perceived contrast for arbitrary images. In transparency perception, for example, Robilotto, Khang, and Zaidi (2002) found that the transmittance of a transparent filter is well predicted by the perceived contrast of the surface seen though the filter, but failed to find any measure of image contrast that adequately captured perceived contrast (see also Robilotto & Zaidi, 2004). Previous work has also shown that observers conflate the color of a transparent surface with its transmittance (Singh & Anderson, 2002), such that lighter transparent layers appear more opaque than darker surfaces that are equally transmissive. The computational difficulties in identifying a psychophysically meaningful measurement of contrast in conditions of transparency has also engendered debate as to the relevance of contrast as a cue in transparency perception (Anderson, Singh, & O'Vari, 2008a, 2008b). A similar dependence on contrast has been observed for gloss, where the perceived gloss of a surface decreases as its diffuse reflectance increases (Billmeyer & O'Donnell, 1987; Pellacini et al., 2000). These effects arise from a similar cause: a reduction in contrast caused by the diffuse reflectance of either the transparent layer or of the glossy surface, which raises the mean luminance in a region. Despite these commonalities, there is still no consistent metric of contrast that accurately captures perceived contrast in these two contexts. A related computational difficulty lies in the measurement of other spatial constructs such as specular sharpness. While simple image definitions of image sharpness can be developed, a psychophysically meaningful measure of sharpness will involve a multiscale representation of that image, with all of the attendant issues of contrast normalization (within and across scales), the number of scales involved, their sizes, etc. Lastly, coverage will only be directly computable from images following the development of a model that can segment the specular contribution to image structure in a psychophysically meaningful way. Given the complexity of each of these problems, we believe it is prudent to use psychophysics to determine some meaningful measure of these properties before attempting to compute them directly from images. 

Another unsolved problem involves explaining how the visual system weights the specular image cues in generating a percept of gloss. We found that the cue combination model relied most strongly on the particular cue that we intended to promote by varying 3-D shape and illumination. This is consistent with the possibility that the cue weightings are proportional to the relative reliability of the cues in discriminating the stimuli being compared. However, our experiments were not designed to measure the relative reliability of the cues to directly test this explanation. The explanation is moot on the question of how the visual system comes to know the relative reliability of the cues for a set of stimuli, or how it comes to learn that the proposed cues provide information about surface gloss. 

Although we have treated the perception of gloss as a single dimension in our experiments, a variety of work has shown that the perception of gloss can be decomposed into two or more dimensions (Billmeyer & O'Donnell, 1987; Hunter & Harold, 1987; Fleming et al. 2003; Pellacini et al. 2000). Using multidimensional scaling, Pellacini et al. (2000) identified two components in gloss judgments, which they labeled “contrast gloss” and “distinctness (sharpness) of image gloss.” Fleming et al. (2003) manipulated the roughness and specular contrast of the Ward model and found that observers could match both parameters, which also suggests that the perception of gloss has at least two dimensions. Our model can, in principle, readily accommodate the multidimensional structure of perceived gloss if we assume that observers match roughness by equating specular sharpness and match specular contrast when asked to equate specular reflectance. Further research is needed to determine how well our proposed appearance cues predict perceived gloss for forms of specular reflectance that are not well captured by the Ward reflectance model. 

Acknowledgments

This work was supported by a grant from the Australian Research Council (A7544) awarded to Barton L. Anderson. The authors thank Juno Kim for his comments on an earlier version of this manuscript and for contributing to the materials used to render the stimuli. 

Commercial relationships: none. 

Corresponding authors: Phillip J. Marlow; Barton L. Anderson. 

Email: [e-mail]; [e-mail]. 

Address: School of Psychology, University of Sydney, NSW, Australia. 

Footnotes

¹  We assume that this luminance difference is normalized in some way. However, for the purpose of the stimuli we used in these experiments, this term can be neglected without loss of generality.