Image statistics and the perception of surface gloss and lightness

Abstract

Despite previous data demonstrating the critical importance of 3D surface geometry in the perception of gloss and lightness, I. Motoyoshi, S. Nishida, L. Sharan, and E. H. Adelson (2007) recently proposed that a simple image statistic—histogram or sub-band skew—is computed by the visual system to infer the gloss and albedo of surfaces. One key source of evidence used to support this claim was an experiment in which adaptation to skewed image statistics resulted in opponent aftereffects in observers' judgments of gloss and lightness. We report a series of adaptation experiments that were designed to assess the cause of these aftereffects. We replicated their original aftereffects in gloss but found no consistent aftereffect in lightness. We report that adaptation to zero-skew adaptors produced similar aftereffects as positively skewed adaptors, and that negatively skewed adaptors induced no reliable aftereffects. We further find that the adaptation effect observed with positively skewed adaptors is not robust to changes in mean luminance that diminish the intensity of the luminance extrema. Finally, we show that adaptation to positive skew reduces (rather than increases) the apparent lightness of light pigmentation on non-uniform albedo surfaces. These results challenge the view that the adaptation results reported by Motoyoshi et al. (2007) provide evidence that skew is explicitly computed by the visual system.

Introduction

The structure in the light reflected to our eyes allows us to recover information about the physical reflectance properties of surfaces, such as their albedo, gloss, and 3D shape. Perception of surface gloss depends on the appearance of specular reflections, which generate intense luminance extrema in the image. A key task of the visual system is to distinguish specular highlights from other luminance maxima that arise from different environmental causes, such as inhomogeneous surface albedo. 

Historically, a large body of research has suggested that the visual system must perform a complex photo-geometric analysis that considers the spatial relationship between surface highlights and 3D surface shape in order to differentiate specular highlights from other surface highlights. Beck and Prazdny (1981) showed that strong luminance increments in an image only appear as specular reflections when their angular orientation is consistent with the apparent curvature of the underlying surface. When specular highlights were rotated by 90 degrees along the direction of maximal 3D surface curvature, the appearance of the highlights changed from specular reflections to surface pigmentation. Similar studies showed that displacing specular highlights away from their natural locations in an image caused a glossy surface to appear matte (Norman, Todd, & Orban, 2004; Todd, Norman, & Mingolla, 2004). They observed that repositioning specular highlights to regions of a surface that were inconsistent with their natural locations caused them to appear as diffuse reflectance or incident illumination, rather than as specular highlights. 

The steepest luminance gradients of a 3D, uniform albedo surface tend to arise along local directions of maximum surface curvature. The angular orientations of specular reflections are also coupled to this geometry, and hence align with the surrounding and underlying spatial gradient of diffuse surface shading (Koenderink & van Doorn, 1980, 2003). These gradients have been conceptualized as “orientation fields” (Ben-Shahar & Zucker, 2001; Breton & Zucker, 1996; Fleming, Torralba, & Adelson, 2004; Huggins, Chen, Belhumeur, & Zucker, 2001). Implicit knowledge of this natural photo-geometric constraint on the alignment of the spatial gradients created by specular highlights and the surrounding diffuse shading profile may be a critical source of information that allows the visual system to distinguish specular highlights from surface pigments in a 2D image (Anderson & Kim, 2009). 

Further evidence for the importance of geometry in the perception of specularity has been observed in stereoscopic experiments. When viewing convex and flat surfaces, the geometry of binocular vision causes surface highlights to have a disparity that biases them to appear behind a reflecting convex or flat surface (Wendt, Faul, & Mausfeld, 2008). Blake and Bülthoff (1990) showed that if specular highlights are artificially manipulated to appear at the same depth or in front of a surface, they appear as light textural smudges or pigmentation. Berzhanskaya, Swaminathan, Beck, and Mingolla (2005) reported that the appearance of gloss tends to propagate a finite distance from the region around a specular highlight. They observed an inverse relationship between distance from a specular highlight and the appearance of local surface gloss. Importantly, the relevant distance measure was the 3D distance on a surface, not the optical distances between the same points projected onto the 2D image. These observations demonstrate that the computation of surface gloss involves a complex analysis based on the position of image highlights in relation to a surface's 3D structure, not a simple 2D image analysis. 

There is further evidence to suggest that the computation of surface gloss is intrinsically coupled to computations of 3D surface geometry. The locations of specular highlights have been shown to affect perceived 3D surface relief. Repositioning the specular reflection of the light source by changing the direction of illumination (Caniard & Fleming, 2007) or rendering surfaces with a mirror reflection of the surrounding environment in the absence of diffuse shading (Fleming et al., 2004; see also Braje & Knill, 1994; Mingolla & Todd, 1986; Todd, Norman, Koenderink, & Kappers, 1997) has been shown to influence the perception of 3D surface structure. Thus, in addition to evidence showing that a specular highlight only appears as such when appropriately positioned relative to the information specifying a surface's 3D geometry, the existence of specular highlights can also play a critical role in specifying a surface's 3D shape. 

Collectively, these observations are consistent with the view that the visual system performs a complex image analysis wherein specular highlights are identified based on their spatial consistency with the photo-geometric constraints on surface optics. However, it was recently suggested that the visual system exploits simple 2D image statistics to determine properties such as surface gloss. Motoyoshi, Nishida, Sharan, and Adelson (2007) proposed that a simple image statistic—histogram or sub-band skew—is explicitly computed by the visual system to infer the physical reflectance properties of surfaces. They showed that pixel histograms for images of glossy uniform-albedo surfaces tend to be positively skewed (tail toward higher luminance values), whereas histograms for matte surfaces tend to be negatively skewed (tail toward lower luminance values). Coercing an image's histogram to have positive skew tends to produce percepts of increased surface gloss, whereas coercing an image's histogram to have negative skew tends to decrease the appearance of surface gloss. Based on these observations, Motoyoshi et al. (2007) proposed that the visual system explicitly computes image skew to infer surface gloss. This claim suggests a very different view of the computations underlying surface properties such as gloss and albedo, since the proposed computations largely (for sub-band skew) or completely (for histogram skew) disregards the spatial configuration of luminance information in the image that previous research has shown to be critical for the perception of gloss. 

In a previous paper, we showed that histogram skew of uniform-albedo surfaces is not diagnostic of surface gloss when the illumination field and 3D shape are also allowed to systematically vary (Anderson & Kim, 2009). Increasing the oblique angle of illumination of a matte surface significantly increases skew but does not increase perceived gloss. Accelerating the upper tail of a matte image's luminance histogram had a negligible effect on skew but significantly increases perceived gloss. Rotating specular highlights in an image relative to the underlying surface shape defined by diffuse shading destroys perceived surface gloss, even when image skew is held constant. In keeping with prior research demonstrating the importance of 3D shape on the perception of skew, our demonstrations and data strongly suggest that histogram skew statistics cannot provide a reliable index of surface gloss. 

Although our previous work raises strong doubts that skew is explicitly computed by the visual system, Motoyoshi et al. (2007) presented one source of evidence for the role of skew computations in gloss perception that has yet to be fully evaluated. They reasoned that if histogram or sub-band skew is explicitly computed, adaptation to an image with a given value of skew should modulate the perceived gloss of subsequently viewed surfaces. In support of this hypothesis, they reported that when adapting stimuli with opposite skews were presented simultaneously, subsequently viewed stimuli exhibited an opponent aftereffect: positively skewed adaptors putatively cause surfaces to appear more matte, and negatively skewed adaptors putatively cause surfaces to appear glossier. They obtained similar aftereffects using adaptors consisting of either stucco images or local difference of Gaussian (DoG) adaptors, suggesting that it was skew, and not the perception of gloss per se, that was responsible for the aftereffects they obtained. 

In this study, we conducted a series of adaptation experiments to determine whether the gloss aftereffects reported by Motoyoshi et al. were unambiguously attributable to the kind of skew adaptation model they proposed. We also performed simulations of their model to determine whether the kind of skew computation they propose could reliably distinguish specular highlights from local changes in surface pigmentation. 

Overview of the experiments

Attempts to replicate Motoyoshi et al.'s lightness aftereffects following skew adaptation were not successful (see 1), so Experiments 1 to 5 focus on assessing the effects of skew adaptation on perceived gloss. In Experiment 1, we performed a replication of their skew adaptation study and obtained similar gloss aftereffects following adaptation to image skew. Experiments 2 and 3 examined the extent to which these gloss aftereffects could be attributed to positively and negatively skewed adaptors separately and found that adaptation to luminance increments contained in positively skewed adaptors alone may account for the aftereffects that have been observed with these stimuli. Experiment 4 showed that similar gloss aftereffects could be obtained following adaptation to images with zero skew. Experiment 5 identified a dependence of these gloss aftereffects on the mean luminance of adaptation images. In Experiment 6, we further attempted to determine whether skew adaptation effects on perceived lightness, if any, could be generalized to non-uniform albedo (i.e., textured) surfaces. 

The results of these six experiments provide no evidence in support of skew being explicitly computed by the visual system. This is especially the case in Experiment 5, which shows that gloss aftereffects are abolished when the mean luminance of skew adaptors is altered. 

Experiment 1

We began by performing a replication of Motoyoshi et al.'s (2007) skew adaptation study, using DoG adaptors forced to have positive and negative values of skew. Adaptation to skewed images of stucco cement was not performed, as transforming the skew of these images also varies the appearance of high-level features (e.g., specular highlights) in addition to skew. Instead, we used two forms of non-glossy adapting stimuli that varied in skew: local DoG stimuli, as used by Motoyoshi et al.; and images of (matte) granite surfaces that were transformed to have different values of skew. Both the DoG and granite images do not appear as glossy surfaces, so any gloss aftereffects observed with these stimuli could not be attributed directly to high-level gloss adaptation. 

Methods

Observers

Five adult undergraduate/post-graduate students enrolled in the Faculty of Science at the University of Sydney participated in the experiment. All were pre-screened for normal or corrected-to-normal visual acuity and the absence of neurological pathology. The procedures of all experiments were approved by the Human Ethics Committee (HEC) at the University of Sydney. 

Skew adaptors

We used skew adaptors constructed from either light or dark difference of Gaussian (DoG) spots pooled within a single 512 × 512 image. The histogram skew of these images was controlled to either −2.0 or +2.0 by modulating the intensity gamma using the following equations: 

$I_{+}=I_0^{\gamma },\gamma >1,$

(1a)

 

$I_{-}=1-(1-I_0{)}^{\gamma },\gamma >1.$

(1b)

Positive skew was produced by applying an appropriate transform γ to the original pixel intensity I ₀ to obtain a new intensity I ₊. Negative skew of equivalent magnitude was produced by applying the same transform γ to the inverse of the original pixel intensity I ₀, where the inverse of the result provided the new intensity I ₋. Mean luminance of these adaptation images was subsequently controlled to 0.5 with contrast of the images multiplicatively controlled to 0.11 in RMS. Similar to Motoyoshi et al.'s (2007) study, these adaptors were presented on either side of a small white fixation point at the center of the CRT display. 

In addition to DoG adaptors, we also used adaptors constructed from a natural image of granite cropped to 512 × 512 pixels. The image's histogram was coerced to have positive or negative skew via the same gamma transformation described above. Values of skew, mean luminance, and RMS contrast for these images were identical to the comparable DoG adaptors. Sample images of DoG and granite adaptors used in the present study are shown in Figure 1. 

Figure 1

 

Negatively and positively skewed (A) difference of Gaussian (DoG) adaptors and (B) granite adaptors used in the present experiment. Distributions were mean matched to mid-gray and had histograms with a skew of ±2.0. The luminance values of these adaptors were multiplicatively adjusted to an RMS contrast value of 0.11.

Stimuli

Visual test stimuli consisted of the same 2D images of stucco presented to observers in tMotoyoshi et al.'s (2007) study, except that they were presented symmetrically on either side of the display (refer to t ₃ in Figure 2). This symmetry was achieved by horizontally reflecting the left or right image, which ensured that the same surface regions were presented at the same angular eccentricity on the left and right sides of the fixation point. The histogram skew of these images was transformed to specific values between ±1.5 (−1.5, −1.0, −0.5, −0.25, 0, +0.25, +0.5, +1.0, +1.5) using Equations 1a and 1b. Values of mean luminance and RMS contrast for these images were controlled to 0.5 mean and 0.17 RMS. 

Figure 2

 

Sequence followed for the presentation of top-up skew adaptation with skewed images of (A) DoGs or (B) granite for 5 s (t ₁), followed by a 0.8-s blank period consisting of just the uniform gray background (t ₂), followed by the 0.5-s test presentation of stucco image having opposing skew (t ₃), and the final acquisition of the response from the observer that concluded the current trial and immediately commenced the subsequent trial (t ₄).

Procedure

Subjects were seated approximately 50 cm from a 21″ CRT display used to present DoG adaptation images subtending 13 cm² (i.e., 15° viewing angle, approximately). Both the positive and negative skew adaptors were presented simultaneously on either side of the screen, with an inner edge distance of 6 mm. A small white fixation point (luminance ≈ 80 cd/m²) was situated between these adaptation patches. The experiment was controlled using custom Matlab software using Psychophysics toolbox, which ran under Windows XP on a MacPRO computer. 

Following an initial 100-s skew adaptation, each trial followed the sequence outlined in Figure 2. During top-up skew adaptation, the 2D texture mapping of the DoG or granite patches were offset randomly in the horizontal and vertical directions every 0.5 s. As intended in the original Motoyoshi et al. (2007) study, these regular offsets were imposed to minimize any effects of local contrast adaptation. Following top-up adaptation and the presentation of a 0.8-s blank period, where only the neutral gray background was visible, two stucco test stimuli with equal and opposite skew (level selected at random) were presented for 0.5 s, after which time the screen was cleared and the fixation point disappeared. This cued the participant to indicate which side of the display contained the image with the glossier stucco surface. After indicating their response using the left/right arrow key on the PC keyboard, the fixation point reappeared and the subsequent trial commenced. 

Observers performed up to 12 repeats for each stimulus level in a block of 2AFC trials. Positive and negative skew adaptors were counterbalanced on either side of the display between two repeat blocks of trials, in addition to a pre-adaptation control block using the neutral gray background in the absence of skew adaptors. 

Data analysis

Paired-comparisons data were transformed into the proportion of times a stimulus image (having a particular skew value) was selected as glossier, over the number of times it appeared during a block of counterbalanced 2AFC trials for each participant. The Weibull psychometric function was fit to the response probability data through numerical minimization of the error function (Equation 2), where E is the overall error, based on observed probability values (P) for specific values of skew (x). After estimating appropriate values for the shape coefficients λ and k, the point of subjective equality (PSE) was determined as the value of skew that equated to the probability of chance (i.e., 50%): 

$E=\Sigma [P-(1-{\mathrm{e}}^{-(x/\lambda {)}^k}{)}^2].$

(2)

 

Results and discussion

Probabilities of stucco images being selected as glossier are plotted as a function of their histogram skew in Figure 3A for DoG adaptors and Figure 3B for granite adaptors. These data are based on averages across observers in the two adaptation conditions. Positive shifts in the psychometric functions after adaptation to skewed DoG stimuli (red traces and points in Figure 3A) compared to the pre-adapt responses (black traces and points) indicate a reduction in perceived gloss on the side of the display adapted to positive skew. The mean PSE on the side of the display adapted to positively skewed DoGs (M = +0.26, SD = 0.10) was significantly larger than pre-adaptation control judgments (M ≈ 0.0, SD = 0.01), as indicated by a repeated-measures t-test (t ₄ = 5.92, p < 0.005). A similar positive shift in the psychometric function following granite adaptation (blue traces in Figure 5B) relative to the pre-adaptation responses shows a reduction in perceived gloss on the side of the display adapted to positive skew. The mean PSE on the side of the display adapted to positively skewed granite (M = +0.29, SD = 0.13) was significantly larger than pre-adaptation control judgments (M ≈ 0.0, SD = 0.01), as was indicated by another repeated-measures t-test (t ₄ = 4.96, p < 0.01). 

Figure 3

 

Mean psychometric functions across subjects showing probabilities of images differing in skew being selected as glossier in our 2AFC paired-comparisons scenario. (A) Aftereffect following simultaneous exposure to positively and negatively skewed DoG stimuli (red trace). (B) Aftereffect to simultaneous exposure to natural images of positively and negatively skewed granite (blue trace). Pre-adaptation control data are indicated by dark points and psychometric curves. Probability refers to the proportion of test stucco images being selected as glossier on the side of the display receiving adaptation to positive skew.

Perceived gloss of surfaces was lower on the side adapted to images of positively skewed DoGs, compared to the side adapted to negatively skewed DoGs. These results are consistent with Motoyoshi et al.'s report of gloss aftereffects following adaptation to images differing in histogram skew. Similar gloss aftereffects were observed following adaptation to images of granite forced to vary in skew. If adaptation to skew per se accounts for these gloss aftereffects, then adaptation to negative skew should facilitate gloss perception and adaptation to positive skew should mitigate gloss perception. This has not been tested previously, as the adaptation experiments performed thus far presented negative skew on one side of the display and positive skew on the other. The goal of Experiment 2 was to test each adaptor individually. 

Experiment 2

In this experiment, we presented positive and negative skew adaptors separately to one side of the display, rather than simultaneously presenting adaptors with opposite skew to both sides. This was done in order to evaluate separately the effectiveness of positive and negative skew adaptors on gloss aftereffects observed in Experiment 1 and previously by Motoyoshi et al. (2007). If adaptation to skew per se was responsible for these aftereffects, then skew thresholds for gloss perception should be increased following adaptation to positive skew and reduced following adaptation to negative skew. 

Methods

Observers

Fifteen adult undergraduate students enrolled in the Faculty of Science at the University of Sydney participated in the experiment. All had normal or corrected-to-normal visual acuity and the absence of neurological pathology. 

Procedure

Methods were identical to those of Experiment 1, except that separate presentations of positive or negative skew were accompanied by a uniform field on the opposite side of the display. The uniform field had the same mean luminance as the skew adaptors and the background of the display (approximately 40 cd/m²). Pre-adaptation control data were obtained by exposing observers to the neutral gray background on both sides of the display, as in Experiment 1. Blocks of adaptation trials were performed in a random order, counterbalanced across observers and the side of the display they were presented. As in Experiment 1, participants were instructed to indicate which side of the display presented the image of stucco that appeared glossier. 

Results and discussion

Figure 4 below plots the probabilities of an image with a particular skew value being selected as glossier after adaptation to positive skew (red) or negative skew (green). Data are based on averages across 15 participants. The strong rightward shift in the red psychometric function indicates the reduced appearance of gloss following adaptation to positive skew. A repeated-measures t-test indicated that the mean PSE obtained across subjects following adaptation to positive skew (M = +0.25, SD = 0.08) was significantly larger than that obtained during the pre-adaptation control condition (M ≈ 0.0, SD = 0.02; t ₁₄ = 12.14, p < 0.001). In comparison, there were no consistent changes in gloss perception after adaptation to negative skew. This was confirmed by the lack of significant difference in mean PSE observed following adaptation to negatively skewed DoG stimuli (M = +0.01, SD = 0.08) compared to pre-adaptation control responses (t ₁₄ = 0.56, p = 0.587). 

Figure 4

 

Means and standard errors showing probabilities of images differing in skew being selected as glossier in a randomized 2AFC scenario, following separate adaptation to DoG adaptors having either positive skew (red points and curve) or negative skew (green points and curve). Pre-adaptation control responses are shown in black. A shift in mean PSE was only observed following adaptation to positive skew.

The results of this experiment reveal that the gloss aftereffect observed following skew adaptation arises from exposure to the positive skew adaptor, which produced a decrease in subsequent sensitivity to perceiving gloss. Adaptation to negative skew should have increased subsequent sensitivity to gloss, but the lack of shift in PSE indicates that adaptation involving the image with negative skew produced no significant effect on gloss perception. Simultaneous presentation of negative skew in Experiment 1 should have facilitated the gloss aftereffects on the side adapted to positive skew. However, the magnitudes of aftereffects between Experiments 1 and 2 were statistically equivalent. This would appear to suggest that the presence of negative skew does not facilitate the percept of gloss, converse to the predictions of Motoyoshi et al.'s sub-band skew proposal. Gloss aftereffects observed in Experiment 1 may therefore relate to adaptation to features/properties contained in the positively skewed image and absent in the negatively skewed image. 

Experiment 3

The previous experiment showed that adaptation to luminance increments (positive skew), rather than luminance decrements (negative skew) influences gloss perception. Motoyoshi et al. presented oppositely skewed DoG adaptors to different sides of the display and matched adaptors in RMS contrast. However, the skew adaptors used here varied in contrast between opposite sides of the display (e.g., positive skew on one side and a uniform field on the other). In order to account for this potential role of contrast adaptation, we replicated Experiment 2, but this time substituting the uniform field with a zero-skew adaptor having equal RMS contrast and mean luminance as the skew adaptor. 

Methods

Observers

Eleven adult undergraduate students enrolled in the Faculty of Science or Education at the University of Sydney participated in the experiment. All had normal or corrected-to-normal visual acuity and the absence of neurological pathology. 

Adaptors

The positively and negatively skewed adaptors consisted of spatially distributed DoGs, with display parameters maintained identical to those used in Experiment 1. In order to equate RMS contrast and mean luminance adaptation on both sides of the display, we presented zero-skew images on the opposite side of the skewed adaptors. These zero-skew adaptors consisted of equal numbers of positively and negatively skewed DoG elements within the same adaptation image. This was achieved by randomly inverting the luminance profile of half the DoG adaptors used in Experiment 1, which ensured that both the mean luminance and RMS contrast of these new images were the same as used before, except that they now had zero skew (mean luminance ≈ 40 cd/m²; RMS contrast = 0.11; skew = 0.0). 

Procedure

The present experiment followed the same procedure as used in Experiment 2, except that a zero-skew adaptor replaced one of the skewed adaptors on one side of the display. Between blocks of skew adaptation trials, observers performed a block of control trials where the zero-skew adaptors were presented to both sides of the display. The temporal order and side of the display skew adaptors were presented and counterbalanced across observers. 

Results and discussion

Figure 6 plots the probability that a test stucco image with a particular skew (−1 to +1) was selected as glossier in our 2AFC scenario. The positive shift in the red psychometric function indicates a relative decrease in sensitivity to gloss following adaptation to positive skew. The negative shift in the green psychometric function indicates a relative increase in visual sensitivity to gloss on the side of the display exposed to negative skew. 

Repeated-measures t-tests indicated a significant positive shift in the average PSE following adaptation to positive skew (M = +0.09, SD = 0.10) compared to pre-adaptation with zero-skew adaptors (M = +0.01, SD = 0.05) on both sides of the display (t ₁₀ = 2.86, p < 0.05). There was also a significant negative shift in the average PSE on the side exposed to negative skew (M = −0.13, SD = 0.18) compared to the pre-adaptation control condition (t ₁₀ = 2.32, p < 0.05). However, there was no statistical difference found when we compared the absolute deviation in PSE following negative and positive skews (t ₁₀ = 0.79, p = 0.45). 

These results would seem to suggest that adaptation to skew of opposite signs produce opponent gloss aftereffects. Exposing one side of the visual field to positive skew reduced the subsequent sensitivity to perceiving gloss, compared to the side exposed to zero skew. In comparison, exposing one side of the visual field to negative skew increased the sensitivity to perceiving gloss relative to the side adapted to zero skew. These equal and opposite gloss aftereffects observed following skew adaptation of different signs would appear to be consistent with the claims provided by Motoyoshi et al. (2007). 

Simultaneous presentation of zero skew in the present experiment appeared to diminish the gloss aftereffect observed on the side adapted to positive skew. In the previous experiment, the size of the aftereffect was +0.25, but only +0.09 here when equal contrast zero skew was also presented. This would seem to suggest that much of the gloss aftereffect of previous experiments was attributed to adaptation to image features/properties other than skew per se, such as RMS contrast. 

Although the results of Experiment 3 appear to provide evidence that both negatively and positively skewed adaptors induce symmetric aftereffects when RMS contrast is equated, this is not the only interpretation possible. Both aftereffects are also consistent with the view that the side of the display containing the greater number of luminance increments was responsible for producing all of the aftereffects we have observed. In Experiment 2, only adaptation to positive skew containing these luminance increments produced gloss aftereffects. Adaptation to negative skew, lacking such highlights, produced no gloss aftereffects. For the present experiment, it can be seen in Figure 5A that the positively skewed adaptor (left) contains twice the number of luminance increments as the comparable zero-skew adaptor (right). The resulting decline in perceived gloss on the side adapted to positive skew is consistent with greater adaptation to luminance increments on this side. In Figure 5B, the zero-skew adaptor (right) contains 50% light DoGs, whereas the opposite side of the display adapting observers to negative skew has none (left). This resulted in a relative increase in perceived gloss on the side exposed to negative skew. This is consistent with a relative decrease in perceived gloss on the side exposed to zero skew, which contained the greater density of luminance increments. It is therefore possible that the aftereffects observed here and in the previous experiments occurred due to brightness adaptation to local luminance increments. The remaining experiments were designed to determine whether these adaptation effects can be attributed to adaptation of local increments that reduce sensitivity to subsequently viewed highlights, or whether there is any evidence to support the view that skew is explicitly computed for the purpose of inferring surface gloss. 

Figure 5

 

Mean luminance and RMS contrast of (A) positive skew and (B) negative skew DoG adaptors (left) was matched on the opposite side of the fixation point with the presentation of zero-skew adaptors (right).

Experiment 4

The difference in results between Experiments 2 and 3 suggests that the adaptation effects observed may have arisen from visual desensitization to local increments that are needed to sense highlights in glossy surfaces, rather than the computation of skew per se. In the present experiment, we examined the effect of adapting observers to zero-skew DoG patches. If adaptation to skew produced the gloss aftereffects in previous experiments, then adaptation to zero-skew adaptors should result in no gloss aftereffects. Any significant effects observed following adaptation to these zero-skew images would confirm that exposure to features/attributes other than skew in this image contribute to gloss aftereffects. 

Methods

Observers

Twenty-seven adult undergraduate/post-graduate students enrolled in the Faculty of Science at the University of Sydney participated in the experiment. All had normal or corrected-to-normal visual acuity and the absence of neurological pathology. 

Adaptors

Separate groups of nine subjects participated in four conditions, each with different variations of zero-skew adaptors we constructed. In the first series of these presentations, equal numbers of positively and negatively skewed DoG elements were pooled within the same adaptation image, as used in Experiment 2 (mean luminance ≈ 40 cd/m²; RMS contrast = 0.11; skew = 0.0). In another variation, we presented images consisting of a matrix of white dots (0.25°, visual angle) on an otherwise uniformly darker background, such that the mean luminance was equivalent to the subsequent test images. The images were constructed using equal numbers of light and dark pixels, each having one of two luminance values. This provided a greater dynamic range of increased image contrast (RMS = 0.4). The opposite was also performed using the inverse image with dark dots on a lighter background. Because equal numbers of light and dark pixels were used, these images had purely binomial luminance distributions, and therefore zero skew. Because the local density of dots varied in relation to the background in these images, we also used a checkerboard adaptor, which not only has zero skew, but equal spatial density of dark and light features. 

Procedure

Nine subjects were assigned to the DoG adaptation condition, and another nine to the checkerboard condition. A separate group of nine subjects performed both the light- and dark-dot adaptation conditions, counterbalanced in order of presentation. 

The same procedure as used in Experiment 2 was followed here, as only one type of adaptor was presented in a single experimental block of trials. A uniform field matched in mean luminance was presented to the oppose side of the display. Because the adaptors had zero skew, the only parameter that varied between the adaptors was RMS contrast. 

Observers performed one block of trials with one form of zero-skew adaptation and then performed the pre-adapt control condition, followed by another block of adaptation trials with the same form of zero skew presented on the opposite side of the display. Presentation order was counterbalanced across observers. 

Results and discussion

Figure 7 shows gloss judgments plotted as a function of histogram skew following exposure to the four different zero-skew adaptor types. Adaptation to zero-skew DoG images reduced the subsequent appearance of gloss (Figure 7A). This is indicated by the rightward shift in the psychometric function (red), relative to the pre-adaptation control (black). The associated mean PSE shift following adaptation to the zero-skew DoG pool (M = +0.11, SD = 0.06) was significantly greater than the mean PSE obtained during the pre-adaptation control condition (M = +0.01, SD = 0.01), as indicated by a repeated-measures t-test (t ₈ = 5.19, p < 0.001). 

Adaptation to the binomial zero-skew images also resulted in similar gloss aftereffects. Adaptation to the checkerboard pattern (Figure 7B) produced a rightward shift in the mean PSE (M = +0.20, SD = 0.09), which was significantly greater than the mean PSE obtained with the pre-adaptation condition (M = +0.01, SD = 0.01) as indicated by a repeated-measures t-test (t ₈ = 6.17, p < 0.0005). Likewise, adaptation to the white dots on the dark background (Figure 7C) resulted in a significant shift in mean PSE (M = +0.18, SD = 0.09) compared to the pre-adaptation control PSE (M = +0.01, SD = 0.01), as indicated by another repeated-measures t-test (t ₈ = 5.65, p < 0.0005). Adaptation to dark dots on the light background (Figure 7D) resulted in a significant positive shift in mean PSE (M = +0.23, SD = 0.16) compared to the pre-adaptation control condition (M = +0.01, SD = 0.01; t ₈ = 4.08, p < 0.005). 

Contrary to the predictions of Motoyoshi et al.'s sub-band skew model, all the zero-skew adaptors we used produced consistent gloss aftereffects, similar to aftereffects obtained with positive skew adaptation. As these adaptors had no skew, the effects observed can only be attributed to some image property other than skew. Adaptation to negative skew alone in the previous experiment (green curve, Figure 6) produced an equal and opposite gloss aftereffect when compared to adaptation to a zero-skew display with equivalent mean luminance and RMS contrast (red curve, Figure 7A). This could suggest that the failure to observe an adaptation effect in Experiment 2 with negatively skewed adaptors arose because the adaptation to RMS contrast and negative skew had approximately equal and opposite effects, resulting in no net adaptation. 

Figure 6

 

Aftereffects obtained following adaptation to positive or negative skew on one side of the display and zero skew on the other. The RMS contrast was matched between zero-skew and skew adaptors. Pre-adapt shows the response with adaptation to zero-skew adaptors on both sides of the display.

Figure 7

 

Positive aftereffects obtained using a family of zero-skew adaptors. In all cases, rightward shifts in the psychometric functions were observed, indicating reduced gloss perception in subsequent images following adaptation to zero skew. (A) Zero-skew DoG pool. (B) Uniform checkerboard pattern. (C) Light dots on dark background. (D) Inverse of (C). All adaptors had zero skew and mean luminance equal to the neutral gray background and subsequently viewed stucco test images.

Alternatively, these adaptation results are consistent with the notion that exposure to local light features in the image accounts for much (or all) of the gloss aftereffects observed in previous experiments. Overall RMS contrast and mean luminance of adaptors was matched between Experiments 2 and 4. Although these two adaptor types differed in skew, the proportion of light versus dark DoGs contained in these adaptors also differed. In the present experiment, the zero-skew image had 50% less positively skewed DoGs than used in Experiment 2. Accordingly, the overall PSE shift observed here following adaptation to the zero-skew DoG pool (+0.11) was roughly half the effect size obtained with positively skewed DoG adaptors in Experiment 3 (+0.25). The greater effect size obtained with adaptation to positive skew may relate to the greater exposure to locally bright features in that image. Adaptation to negative skew, lacking such luminance highlights, produced no gloss aftereffects in the previous experiment. This suggests that adaptation to bright image features, rather than image skew or contrast per se, may account for these gloss aftereffects. 

Experiment 5

Results from previous experiments suggest that adaptation to bright image features, and not skew, may account for the gloss aftereffects observed. Based on the stimuli we used, the side of the visual field exposed to adaptors having the greater number of light DoG elements showed a subsequent reduction in perceived gloss. According to the model proposed by Motoyoshi et al. (2007), sub-band skew is computed as the local difference between net responses of on-center and off-center early visual cells. Such computations should be robust against changes in mean luminance of the DoG adaptors used. In previous experiments, DoG adaptors only differed in terms of the numbers of local luminance features in the image, not their overall intensity. 

In this experiment, we examined the effect of altering mean luminance and skew on gloss perception together. If adaptation to skew accounts for the gloss aftereffects of previous experiments, then reducing the mean luminance of positive skew (constraining skew and contrast) should have little or no effect on the resulting gloss aftereffect. However, if the gloss aftereffects arise from adaptation to locally bright scene features, then reducing the mean luminance of the positively skewed adaptor should reduce its effectiveness in producing these aftereffects. 

Methods

Observers

Eleven adult undergraduate/post-graduate students enrolled in the Faculty of Science at the University of Sydney participated in the experiment. All were pre-screened as in the previous experiments. 

Adaptors

Two separate conditions of adaptation were performed as outlined in Figure 8. In the first condition, uniform field adaptors were constructed so that they varied only in mean luminance, with zero contrast and skew across the adapting fields (Figure 8A). A dark uniform field was presented on one side of the display (approximately 10 cd/m²) and a uniform field equal to the mean luminance of the background on the other side (40 cd/m²). The second set of adaptors was constructed to vary in skew in addition to mean luminance on either side of the display, keeping RMS equal to 0.11 (Figure 8B). Positively skewed DoG adaptors that were equated in mean luminance to the dark uniform field were presented on one side of the display, and zero-skew adaptors matched in RMS contrast were presented on the opposite side of the display. 

Figure 8

 

Observers were simultaneously adapted to images that either differed in (A) mean luminance or (B) both mean luminance and skew. Zero-skew adaptors on the left had a mean luminance equal to the pre-adapt control condition and RMS contrast matched to the parametrically manipulated adaptors on the right.

Procedure

Each participant performed 2AFC gloss ratings following adaptation to both conditions. Presentation order of adaptor pairs was counterbalanced across the display within subjects, and the order of conditional blocks of adaptation trials was counterbalanced across subjects. An additional pre-adaptation control condition identical to Experiment 1 was inserted between the two main test conditions. All other display parameters and experimental conditions were identical to the previous experiments. 

Results and discussion

We found that the sensitivity to gloss improved slightly on the side of the display exposed to a uniform field with a luminance that was darker than the mean luminance of the background and subsequently viewed surfaces. This was evident in the leftward shift of the blue psychometric function in Figure 9, compared to the pre-adapt control data shown in black. This mean difference in PSE (M = −0.056, SD = 0.04) was significantly smaller than the pre-adapt control (t ₁₀ = 4.77, p < 0.001). Importantly, however, adaptation to the positively skewed adaptors that had the same decrement in mean luminance (red psychometric function in Figure 9) produced essentially identical data to the dark adapting field alone. Relative to the PSE obtained with the dark uniform field alone, the added adaptation of positive skew exhibited no significant change in gloss perception that could not be attributed to the small effect produced by a change in the mean luminance (i.e., ΔM = +0.009) as indicated by another repeated-measures t-test (t ₁₀ = 0.32, p = 0.76). 

Figure 9

 

No consistent difference in gloss ratings was observed following adaptation to low-luminance positive skew (red) compared to the uniform field matched in mean luminance (blue). The shift in the psychometric function occurred in the wrong (leftward) direction, compared to the expected rightward shifts following adaptation to positive skew in previous experiments.

The data from this experiment demonstrate that positive skew produces no significant gloss aftereffects when the mean and peak luminance of DoG elements is lowered in comparison to DoGs contained in the zero-skew adaptor matched in RMS contrast. Although image contrast differed between the two sets of adaptors (the uniform fields versus the DoG adaptors), the RMS contrast was matched for the two adapting stimuli within each condition. Skew and mean luminance were the only two parameters allowed to vary across the display. After discounting the effect of adaptation to mean luminance, the residual gloss aftereffect following adaptation to skew was not significant. These results show that skew adaptation is not robust against changes in mean luminance. This could suggest that the gloss aftereffects observed in previous experiments were not due to adaptation to skew per se. 

Experiment 6

Motoyoshi et al. did also report a strong (inverse) relationship between lightness and histogram skew, suggesting that skew may have better predictive validity for lightness than for gloss. We attempted to replicate their lightness aftereffects, but found no consistency in the pattern of results across observers (1). In the present experiment, we assessed their lightness results in a more specific way to determine whether adaptation to skew or luminance extrema modulates their lightness aftereffects. We reasoned that if the aftereffects observed with gloss stimuli arose due to desensitization to local luminance increments, then any subsequent task that relied on sensitivity to local increments should exhibit a similar form of adaptation. To assess this possibility, we had observers view grayscale images of granite surfaces and judge which surface appeared to contain the locally lightest pigment. If adaptation to skew accounts for such aftereffects, then adaptation to positive skew should make lighter areas of the surface look lighter, as reported by Motoyoshi et al.'s lightness judgments. However, if adaptation to the luminance increments in these adaptors merely desensitizes observers to detecting lighter surface highlights, then lighter areas of the surface should look darker following adaptation to positive skew. 

Methods

Observers

Nine adult undergraduate/post-graduate students enrolled in the Faculty of Science at the University of Sydney participated in the experiment. All were pre-screened as in the previous experiments. 

Adaptors

We used the same simultaneous presentation of positively and negatively skewed DoG adaptors used in Experiment 1, closely matched to those used by Motoyoshi et al. 

Procedure

The 2AFC task was similar to that of Experiment 1, except that the test stimuli consisted of the granite texture that was previously used as a skew adaptor. The base image was transformed to have a range of skewness values between −1 and +1, with a mean luminance of 0.5 and image contrast of 0.17 RMS. Observers were given a standard briefing on the paired-comparisons task they needed to perform, as well as the following instructions specific to this experiment:

 

“Two images will appear for half a second, one on either side of the central fixation target. When they disappear, you need to press the arrow key on the keyboard that corresponds to the image containing the surface that appeared to have the lightest pigmentation.”

 

Results and discussion

Figure 10 shows mean probabilities of granite test images with different skew values being selected as containing the lightest pigmented material in our 2AFC scenario. There was a clear rightward shift in the psychometric curve (M = +0.25, SD = 0.07), relative to the pre-adapt control curve (M = +0.01, SD = 0.01). This difference was found to be significant by a repeated measures t-test (t ₈ = 9.48, p < 0.00005). 

Figure 10

 

Positive aftereffects in lightness judgments following simultaneous adaptation to positive histogram skew on one side of the display and negative histogram skew on the other. Pre-adapt shows the response with exposure to uniform gray fields on both sides of the display (i.e., no contrast and no skew). Probabilities are based on responses to test images presented on the side exposed to positive skew.

The aftereffect observed here shows that adaptation to skew adaptors affects our ability to detect local surface highlights. In contrary to the sub-band skew prediction that lightness will always increase following adaptation to positive skew, a significant decrease in the lightness of surface texture was found on the side of the display adapted to positive skew. We examine the implications of these findings in the General discussion section. 

General discussion

In Experiment 1, we repeated the procedure used by Motoyoshi et al. and replicated their gloss aftereffects. Results of Experiments 2 and 3 indicated that at least half of the gloss aftereffects could be attributed to image parameters other than sub-band skew. We found that adaptation to the negatively skewed adaptor alone had no effect on gloss perception (Experiment 2), and that adaptation to zero-skew adaptors produced similar gloss aftereffects (Experiment 4) as positive skew. Critically, when the mean luminance of the positively skewed image was reduced so that the local highlights were much lower in luminance, but with identical RMS contrast, there was no evidence that positively skewed images produce any gloss aftereffects (Experiment 5). This result demonstrates that any putative adaptation to skew is not robust against changes in luminance. As a whole, these findings support the interpretation that visual adaptation to globally bright features in the scene, rather than sub-band skew, mediates the observed gloss aftereffects. This is further supported by Experiment 6, where adaptation to skew modulated visual sensitivity for detecting the lightest features in a non-uniform albedo surface. 

The failure to find evidence in support of skew computations is consistent with the large body of previous research into the perception of surface gloss. This research has shown that the spatial relationship between image highlights and the information specifying 3D surface geometry is critical in eliciting the percept of surface gloss (Beck & Prazdny, 1981; Norman et al., 2004; Todd et al., 2004). Increasing the luminance of specular surface highlights increases the apparent strength of the surface's specular reflectance (Beck & Prazndy, 1981). This occurs because specular reflections of the light source will always have higher luminance values relative to the surface's diffuse reflectance. In the experiments presented herein, the strength of adaptation appears to be primarily modulated by the presence of local bright features in the adapting stimulus. The decrease in sensitivity to specular highlights following adaptation to stimuli containing a preponderance of local bright features suggests that it is largely these aspects of image structure that were responsible for the adaptation effects we observed. This is supported by our findings that positively skewed images (having many bright highlights) produced a larger gloss aftereffect than zero skew (having fewer bright highlights), and that negative skew (having no bright highlights) produced no detectable gloss aftereffects. 

Role of image contrast

The effect of contrast variation on gloss aftereffects suggests that image attributes other than skew contribute to these gloss aftereffects. Indeed, our failure to observe adaptation effects with negatively skewed images (compared to a uniform field) could have been due to the “nulling” of any adaptation to negative skew by adaptation to image contrast. Adaptation to negative skew should make a subsequently viewed surface look glossier, but the effect may have been diminished due to an equal and opposing adaptation to contrast, which reduces the percept of gloss. If contrast and negative skew produce opposite gloss aftereffects, complete cancellation can only occur if each component contributes equally to the overall effect size. This can only be achieved if 50% of the effect size is due to contrast adaptation and 50% is due to skew adaptation. Note that this argument implies that skew could only account for half the aftereffect observed in Experiments 1 and 2 and the experiments previously reported by Motoyoshi et al. (2007). 

However, the results from Experiment 5 suggest that skew adaptation does not even appear to account for the residual gloss aftereffect observed when RMS contrast is matched between adaptors. In this experiment, the contrast of the stimuli was equated, but the mean luminance of the positively skewed image was reduced. The purpose of this manipulation was to determine whether it was the skew, or the presence of locally bright image features, that induced the aftereffects obtained with positively skewed stimuli. Although we observed a small aftereffect of mean luminance, we found no evidence that positive skew contributed any effect beyond that induced by mean luminance. The skew and contrast of the positively skewed adapting stimulus were identical to the values used in our previous experiments, but no aftereffect was observed. This provides strong evidence against the claim that the aftereffects observed were the result of adaptation to a hypothetical skew computation. 

The above still holds true when a more robust computation of image contrast other than RMS contrast is considered. According to Weber's law, the perceived contrast of a feature in an image is inversely related to the image's background luminance. The resulting difference in sensitivity to contrast across images differing in background intensity (i.e., mean luminance) can be normalized by computing Weber contrast as 

$W=(I-I_{\mathrm{B}})/I_{\mathrm{B}},$

(3)

where I is the feature intensity, and I _B is the average background intensity. 

In Experiment 5, we reduced the mean luminance of positive skew but maintained RMS contrast. This effectively achieved the same result in the numerator but reduced the size of the denominator in Equation 3. The resulting Weber contrast was therefore greater for positive skew when mean luminance was reduced, compared to when it was higher. Assuming contrast adaptation played some role in the gloss aftereffects of the present study, this increase in image contrast should have enhanced the gloss aftereffect observed. However, we found that such manipulations reduced the effectiveness of positive skew to produce gloss aftereffects. These results suggest that adaptation to image contrast per se does not appear to account for the gloss aftereffects observed in the present study. 

Sub-band skew, surface gloss, and lightness

Additional simulation evidence suggests that Motoyoshi et al.'s sub-band skew model cannot provide any diagnostic leverage in distinguishing different sources of luminance variation in an image. Their model separately convolves an image with on-center and off-center DoG filters along different processing channels, followed by half-wave rectification of each channel's output. A squaring non-linearity is then applied to each signal, before unitary responses are pooled through local averaging. The resulting sub-band skew is determined as the difference between the net on-center and off-center channel responses. Sample output from a simulation with the uniform albedo image of St. Matthew is presented in Movie 1. 

 

Movie 1

 

Model output responses from the sub-band skewness computation proposed by Motoyoshi et al. applied to the glossy (left) and matte (right) versions of St. Matthew. Seven-level posterization of the results show increasing preponderance of on-center activity with increasing red saturation, while increasing preponderance of off-center activity is shown in increasing green saturation. The static image shows the response with a center to surround ratio of 1:2. The effects of increasing the filter size is shown in the first segment; then reducing the center size to increase the center–surround ratio up to 1:40 in the second segment; and finally reducing the size of the surround to reduce the center–surround ratio back to 1:2 in the final segment.

In Movie 1, we applied the sub-band skew model to the same input image used in Motoyoshi et al.'s study, but this time explored the effects of different filter sizes and center–surround ratios (C:S). The movie presents three segments where we (1) increase filter size; (2) increase C:S ratio; and (3) reduce filter size and restore the initial C:S ratio. Across all these manipulations, the sub-band skew model is able to distinguish between the two uniform albedo surfaces perceived to have different levels of gloss in the image. In the first segment of the animated model output, we increased the size of the filter surround from 2 pixels up to 40 pixels, while preserving a center to surround ratio of 1:2. In the second segment of the animated model output, we reduced the center size from 20 pixels back to 1 pixel, thus increasing the effective C:S ratio toward 1:40. Finally, in the last segment, we reduced the size of the surround back to 2 pixels, restoring the original state of the movie sequence. The region with the preponderance of positive sub-band skew (shown in red) corresponds to the glossy version of St. Matthew, whereas the region with the preponderance of negative sub-band skew (shown in green) corresponds to the matte version of St. Matthew. This shows that their sub-band skew model is able to distinguish reliably between the glossy and matte regions of a uniform-albedo surface image. 

However, similar results were obtained with an inhomogeneous granite surface, showing that sub-band skew statistics cannot determine the environmental cause underlying the difference in preponderance of light and dark image regions. The output shown in Movie 2 was obtained using identical simulation parameters to those used in the example with the St. Matthew surface, but this time using the non-uniform albedo granite surface used inExperiments 1 and 6. A similar distinction between the positively (left) and negatively (right) skewed regions is achieved, despite the fact that the base images of both granite surfaces appear matte and differ only in their proportions of light and dark surface pigments. This shows that Motoyoshi et al.'s sub-band skew model cannot effectively differentiate local regions of specular highlights from other types of surface reflectance in the image. 

 

Movie 2

 

Output response from the sub-band skew model applies to the positively skewed (left) and negatively skewed (right) versions of textured granite. Both modeling and display parameters are identical to those used to createMovie 1.

In conclusion, we have found no evidence to support the view that the gloss adaptation effects reported herein, or those reported previously by Motoyoshi et al., arise from the desensitization of neural elements that compute the skew of an image. We have also found that the model that Motoyoshi et al. proposed as a neural implementation of skew computations cannot distinguish between local luminance increments that arise from specular highlights and those that arise from surface pigmentation. Hence, their model cannot provide any diagnostic information about the source of local luminance highlights. These findings support the historical view that perception of surface properties, like specularity and albedo, requires a detailed analysis of images that preserves the geometric structure of luminance information provided in the image. 

Appendix A

We performed a replication of Experiment 1 where we instructed observers to select the stucco image that appeared darker (instead of glossier). All display parameters and procedures were identical to those used in Experiment 1. Observers performed two sessions of skew adaptation, one with DoG adaptors and another with granite adaptors. Presentation order was counterbalanced across subjects. Results from the four observers we tested on both conditions are shown in Figure A1. These data show that the percept of surface lightness is not consistent across observers. Despite this failure to achieve consistent baseline responses, no consistent rightward shifts in psychometric functions were observed following adaptation to skew adaptors. These data also show that adaptation to image skew does not result in clear, discernable lightness aftereffects. 

Figure A1

 

Psychometric functions for individual observers showing probabilities of images differing in skew being selected as darker in our 2AFC paired-comparisons scenario. Data obtained following adaptation to skewed DoG and granite images are shown in red and blue, respectively. Pre-adaptation control responses are shown in black. Note the lack of consistency in results across observers.

Acknowledgments

We kindly thank I. Motoyoshi and T. Adelson for their helpful exchange on some of the ideas presented in this paper. We would also like to thank R. Fleming and S. Solomon for their comments and suggestions on ideas presented in this paper. This project was funded by an Australian Research Council Grant awarded to B. Anderson. 

Commercial relationships: none. 

Corresponding author: Juno Kim. 

Email: [e-mail]. 

Address: School of Psychology, 304 Griffith Taylor Building (A19), The University of Sydney, NSW 2006, Australia.