The dominating impacts of Helmholtz-Kohlrausch effect on color-induced glossiness enhancement

Abstract

Glossiness can be increased by adding chromatic information to the object images. However, the mechanisms that create color-induced glossiness enhancement are unclear. In this study, we psychophysically measured the glossiness of object images to which various hue chromaticities were added to elucidate the perceptual and image factors that explain the color-induced glossiness enhancement effect. Two types of coloring conditions were tested: the both-colored (BC) condition, in which both specular and diffuse components were colored with the same chromaticity, and the diffuse-colored (DC) condition, in which only diffuse components were colored while specular components remained achromatic. The results showed that glossiness enhancement was more prominent in the BC than in the DC condition, and the dependency of glossiness enhancement on the stimulus color direction was similar to that of the Helmholtz-Kohlrausch (H-K) effect. Furthermore, we performed a regression analysis with a linear mixed model based on image features and an additional experiment in which an H-K effect-based increase in perceived brightness was imitated on achromatic stimuli by manipulating luminance. The results demonstrated that the H-K effect-based brightness enhancement in the highlight regions explains the glossiness enhancement effect well. These results suggest that the H-K effect, especially around the highlight region, is a dominant factor that creates the color-induced glossiness enhancement, although other color-related factors could also be partly involved.

Introduction

When looking at an object, we can easily perceive its surface qualities, such as glossiness and transparency, with glossiness being the most typical surface quality. Numerous psychophysical studies have investigated the mechanisms of the human visual system in perceiving glossiness from retinal images. In particular, the relationship between various features in object images and glossiness perception has been extensively investigated because such image features appear to be a type of heuristic for surface quality perception (e.g., Fleming, 2012). 

Most previous studies on glossiness perception have focused on luminance image features. For instance, Motoyoshi, Nishida, Sharan, and Adelson (2007) demonstrated that a simple luminance statistic, the skewness of the luminance histogram in an object image, is strongly correlated with perceived glossiness. They also showed that manipulating the luminance skewness of an image modulated perceived glossiness. Similarly, luminance contrast has been found to correlate more strongly with perceived glossiness than luminance skewness in photographic natural scene images (Wiebel, Toscani, & Gegenfurtner, 2015). Of course, simple luminance statistics alone are insufficient to explain perceived glossiness, but more complex features are crucial for glossiness perception. For instance, Anderson and colleagues (Anderson & Kim, 2009; Kim, Marlow, & Anderson, 2011) reported that disturbing the spatial alignment between the specular and diffuse reflection components dramatically attenuated the perceived glossiness, even if the luminance statistics were maintained. This suggests that higher-order image features are crucial for glossiness perception. In addition, object shapes and lighting environments are known to significantly alter glossiness perception, even when the surface reflectance is the same (e.g., Olkkonen & Brainard, 2011). Such glossiness variations can be reasonably well explained by image features related to specular highlights such as coverage, contrast, and sharpness (Marlow, Kim, & Anderson, 2012). Similarly, the luminance gradient and luminance order information in an object image can be cues for jointly estimating object shapes and reflectance properties, such as glossiness (Sawayama & Nishida, 2018). In many of these studies, luminance features in specular highlights have been demonstrated to play a significant role in glossiness, although some studies have suggested the roles of other features of dark regions on glossiness (Kim, Marlow, & Anderson, 2012; Kiyokawa, Tashiro, Yamauchi, & Nagai, 2019). 

In addition to luminance, chromaticity or color interacts with material perception. Several psychophysical studies have reported the effects of specular reflections on color perception. For instance, the perception of object colors or lightness on glossy object images is not determined by the mean color of the object image (Xiao & Brainard, 2008; Honson, Huynh-Thu, Arnison, Monaghan, Isherwood, & Kim, 2020; Toscani, Valsecchi, & Gegenfurtner, 2017). Rather, the perceived color is typically more saturated than the average color on the object's surface. In addition, perceived object color is more saturated on object images with clear highlights (Isherwood, Huynh-Thu, Arnison, Monaghan, Toscani, Perry, Honson, & Kim, 2021). These color perception properties raise the possibility that, when judging object colors, the visual system tends to ignore or discount specular highlights, which are typically near achromatic. Another intriguing property of specular highlights is that their colors on dielectric materials directly reflect spectral distributions of illuminations regardless of the object color (Thompson, Fleming, Creem-Regehr, & Stefanucci, 2011). Thus this property may help the human visual system estimate illumination colors for color constancy. Several previous studies have supported this idea (see, e.g., Yang, & Maloney, 2001; Wedge-Roberts, Aston, Beierholm, Kentridge, Hurlbert, Nardini, & Olkkonen, 2020; Xiao, Hurst, MacIntyre, & Brainard, 2012). 

In contrast, several studies have investigated whether chromaticity information on object images alters perceived glossiness. First, chromatic specular highlights may attenuate perceived glossiness (Nishida, Motoyoshi, Nakano, Li, Sharan, & Adelson, 2008; Nishida, Motoyoshi, & Maruya, 2011). Nishida et al. (2008, 2011) measured the glossiness of an object image in which either diffuse or specular reflectance components were colored and found that glossiness was attenuated when only specular components were colored. This glossiness attenuation may be relevant to a violation of the physical constraint that specular highlights must have a broader light-wave spectrum than the other regions. The colored highlights did not appear to be specular highlights perceptually but rather colored foils on an achromatic surface. In contrast, glossiness remained when only the diffuse reflection components were colored. However, these studies did not report quantitative comparisons of glossiness before and after coloring. 

Many other studies have reported positive effects of chromaticity on glossiness. Wendt and Faul (2018) suggested that the visual system uses colors to separate the specular highlights created by different light sources. They also claimed that color contributes to gloss constancy through the highlight separation. Similarly, Hanada (2012) reported that perceived glossiness was greater on object images with chromatic specular highlights on achromatic regions (i.e., the specular highlights had color contrast against the surroundings) than on object images with uniform chromaticities. These studies suggest that color may directly enhance glossiness by facilitating highlight extraction. On the other hand, other studies have suggested indirect effects of color on glossiness. For example, Okajima and Takase (2000) showed that perceived glossiness differed across stimulus chromaticities even if their luminance was fixed. In addition, they showed a correlation between the Helmholtz-Kohlrausch (H-K) effects and perceived glossiness. The H-K effect is a phenomenon in which the perceived brightness of colored light changes with its saturation and hue, even if the luminance is constant (Nayatani, 1998). It seems plausible that perceived brightness rather than luminance underlies gloss perception. Thus the perceived glossiness of chromatic surfaces may be naturally enhanced by the H-K effects. This concept can be regarded as an indirect color effect on glossiness. 

Each of the studies described above, however, focused on either of the two candidate factors affecting perceived glossiness: H-K effect and color contrast. In our daily life, both highlights and other regions can have different chromatic colors; that is, the H-K effect and chromatic contrast simultaneously occur on object surfaces. Nevertheless, the combined effects remain unclear. 

This study aims to elucidate the factors that cause glossiness enhancement by chromatic colors, focusing on the H-K effect and color contrast. First, we measured the glossiness of different types of achromatic and chromatic object images psychophysically. The color-induced glossiness enhancement was quantified by analyzing the results. We then quantitatively evaluated the contributions of the H-K effect and chromaticity contrast to the glossiness enhancement using a linear mixed model. These results suggest a predominant contribution of the H-K effect over a wide range of stimulus conditions. In Experiment 2, we further measured the glossiness of achromatic stimuli, where brightness enhancement caused by the H-K effect was simulated by luminance manipulation. This experiment aimed to measure the direct contributions of the H-K effect and the indirect contributions of other factors to glossiness enhancement. The results showed that highlight brightness increased by the H-K effect is a primary factor in glossiness enhancement. In addition, because the H-K effects alone cannot fully explain glossiness enhancement, it is suggested that color information itself, such as color contrast, may also partially contribute to glossiness enhancement. 

Experiment 1: Glossiness of chromatic object images

In Experiment 1, we psychophysically measured the perceived glossiness of object images under different chromaticity conditions. This study focused on two main aspects. The first was stimulus coloring. Two methods of object image coloring were used: the both-colored (BC) condition, in which all pixels were given the same chromaticity, and the diffuse-colored (DC) condition, in which only the diffuse reflection components were given chromaticity, whereas the specular components remained achromatic. The coloring conditions are helpful in dissociating the contributions of the H-K effect and color contrast to color-induced glossiness enhancement. For instance, specular highlights should appear brighter in the BC condition owing to the H-K effect, whereas the color contrast of specular highlights should be more prominent in the DC condition. Second, perceived glossiness was analyzed using a linear mixed model, with stimulus image features as explanatory variables, to identify the features contributing to color-induced glossiness enhancement. 

Methods

Observers

Six undergraduate and graduate students from the Tokyo Institute of Technology in their twenties (one female and five males) participated in this experiment as observers. All observers had normal or corrected-to-normal visual acuity and passed the Ishihara color vision test. The experiment was designed in accordance with the Declaration of Helsinki and was approved by the Ethical Review Committee of the Tokyo Institute of Technology. 

Apparatus

The experiment was controlled using a computer (Pavilion Desktop 595; Hewlett-Packard Japan, Tokyo, Japan) running Ubuntu 18.04 LTS. Self-built programs created with MathWorks MATLAB 2020a and Psychtoolbox 3 (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007) were used to create the stimuli, control experimental paradigms, and acquire observer responses. An organic light-emitting display (PVM-A250; Sony, Tokyo, Japan; resolution of 1920 × 1080 pixels, refresh rate of 60 Hz) placed in an otherwise dark room was used for stimulus presentation. To accurately present the desired luminance and chromaticity on the display, the gamma curves of the RGB primaries were measured using a color luminance meter (ColorCAL II; Cambridge Research Systems, Rochester, UK), and their spectral radiances were measured using a spectroradiometer (Specbos 1211-2; Jeti, Oberwesel, Germany). During the experiment, the observer's head was fixed on a chin rest at a viewing distance of 80 cm from the display. The participants binocularly and naturally observed the display. 

Stimulus: Overview

An example of a stimulus is shown in Figure 1. The stimulus was a pair of two computer-graphic (CG) images placed side-by-side on a dark uniform background. To create each image, two types of object images were first rendered using CG software: one with only specular reflection components (the specular image) and the other with only diffuse reflection components (the diffuse image). The chromaticities were then artificially and separately manipulated using the MATLAB software. Finally, each stimulus image was created by adding the XYZ tristimulus values of the diffuse and specular images. Details of each process are described in the following subsections. 

Figure 1.

 

Example of stimulus in a trial.

The size and spatial resolution of an image with the background were 8.25° × 11° in visual angle and 720 × 960 pixels, respectively. Although the object size in an image depended on its shape, it was approximately 4.6° × 4.0°. The images were separated by 0.16° (14 pixels). In addition, the dark area outside the image pairs was uniformly gray with a luminance of 2 cd/m² and a chromaticity of (u′, v′) = (0.188, 0.438). 

Stimulus: Computer graphics rendering

The geometries of the computer graphics images, such as object shapes and area light positions, were set using Blender 2.79. The ambient lighting and object reflection characteristics, such as surface roughness, were then set using Mitsuba 2 (Nimier-David, Vicini, Zeltner, & Jakob, 2019). Image rendering was also performed using Mitsuba 2. 

The bidirectional scattering distribution function of the objects was roughplastic, a preset in Mitsuba 2, with specular reflectance fixed at 1.0. In addition, several parameters for image rendering were manipulated to examine color-induced effects under various conditions. The object shapes were a bunny, dragon, and blob. The bunny and dragon were the Stanford Bunny and Stanford Dragon obtained from the Stanford 3D Scanning Repository (http://graphics.stanford.edu/data/3Dscanrep/). The blob was created by applying the displace modifier to a UV sphere in Blender 2.79 with a cloud texture, size of 0.70, depth of 0, and nabla of 0.03. Images of these three shapes are shown in Figure 2. The diffuse reflectances of the objects were 0.1, 0.3, and 0.5, and the surface roughness values were 0.05, 0.1, and 0.2, respectively. 

Figure 2.

 

Examples of object images of (a) bunny, (b) dragon, and (c) blob shapes.

There were two illumination conditions: area and ambient light conditions. In the area light condition, there were two lights: one on the upper right and the other on the upper left, in front of the object. The geometries of the illumination and objects are shown in Supplementary Figure S1. The object was placed on a board surrounded by walls on four sides: above, below, in front, and behind, as shown in Figure 2 and Supplementary Figure S1. The board and walls were matte surfaces with a diffuse reflectance of approximately 0.7 across the visible wavelengths. Under the ambient light condition, the lighting was set with an environment emitter in Mitsuba 2. For the environment emitter, we used a high dynamic range image, At the Window, from Bernhard Vogl's website (http://dativ.at/lightprobes/). The geometry was almost the same as that of the area light condition, except that no walls existed in the scene. 

Finally, images with only diffuse reflection components (diffuse images) and those with only specular reflection components (specular images) were rendered separately. We rendered a specular image by setting the diffuse reflectance to zero, whereas we rendered a diffuse image by setting the specular reflectance to zero. The exposure time of the camera was the same across all conditions. 

Stimulus: Coloring

There was some preprocessing of the coloring. As expected, the luminance of the rendered images was much higher in the highlight regions, leading to dark representations of image regions besides the specular highlights on the display. Thus nonlinear tone mapping was first applied to the original luminance according to the following equations:  

\begin{eqnarray*}k = \frac{{\log \frac{1}{{255}}}}{{{L_w}}}\end{eqnarray*}

 

\begin{eqnarray*}f\left( x \right) = \frac{{{L_{max}}}}{2}(1 - \exp \left( {kx} \right)),\end{eqnarray*}

where L_w is the luminance of white, x is the luminance of the original image, L_max is the maximum luminance of the display, and f(x) is the luminance after tone mapping. We used L_w = 1.5. By applying tone mapping, f(x) was found to have values ranging from 0 to L_max/2. The tone mapping function is illustrated in Figure 3. Supplementary Figure S2 shows the luminance histograms of the object regions in some resultant images. In addition, before the coloring procedure, the chromaticity of all image pixels was metameric to the white point of the display ((u′, v′) = (0.188, 0.438)). 

Figure 3.

 

Tone mapping function.

The chromaticities used for the coloring procedure were defined on the CIE1976 u′v′ chromaticity diagram. Nine chromaticities were used: an achromatic color and eight chromatic colors. The achromatic color had a u′v′ chromaticity corresponding to the white point of the display ((u′, v′) = (0.188, 0.438)). The eight chromatic colors had chromaticities centered on the achromatic color and in eight color directions at 45° intervals from 0° (u′ positive direction). The color directions on the u′v′ chromaticity diagram are shown in Figure 4. The saturations of chromatic colors (Euclidean distances from the achromatic color) were the same across all eight color directions. The saturation settings are described in the following subsection. 

Figure 4.

 

Color directions used for coloring procedure. The saturations depended on the coloring condition.

Under both BC and DC coloring conditions, coloring was applied only to the object regions. Subsequently, the colored object regions were embedded in the background image. A background image was created for each object shape and illumination condition by rendering an image (e.g., Figure 2 in the area light condition), and the chromaticity was then set at the display's achromatic point. In addition, the object area was made black after rendering. Consequently, the resultant background image contained only the achromatic board under the area light condition, and the floor and walls under the ambient light condition. 

In the BC condition, both diffuse and specular reflection components were colored. First, the XYZ values of the diffuse and specular achromatic images were added to create an achromatic image containing both reflection components. The chromaticity of the resultant achromatic image was then manipulated. Based on the luminance Y of the achromatic image, the maximum saturation within the display's color gamut was calculated for the eight color directions. The minimum value of the maximum saturation among the eight color directions was employed as the stimulus saturation and was applied for all color directions. Finally, the chromaticities in the object region were changed to have one of the eight chromaticities. 

In the DC condition, only the diffuse reflection component was colored. First, the u′v′ values of the diffuse image were changed to the chromaticity of the eight color directions, as in the BC condition. The colored diffuse and achromatic specular images were then summed in XYZ tristimulus values. The saturation used for coloring the diffuse components was the same across the color directions and was determined in a similar way as that in the BC condition; that is, we searched in a brute-force manner for the maximum saturation of the resultant image that fits within the display's color gamut among the eight color directions. 

The coloring differences between the BC and DC conditions appear mainly in the chromaticities of specular highlights. Figure 5 (a) shows examples of stimulus images of all color directions for the BC and DC conditions, and Figures 5(b) and (c) show enlarged images with the 180° chromaticity for visibility under the area and ambient light conditions, respectively. The highlights in the BC condition were chromatic, whereas those in the DC condition appeared closer to the achromatic color. This difference provides insight into the main factors contributing to color-induced glossiness enhancement. For instance, a candidate factor for glossiness enhancement is the increment of perceived brightness in the specular highlight regions owing to the H-K effect. If this hypothesis based on the H-K effect is correct, the color glossiness enhancement should be stronger in the BC condition with chromatic specular highlights than in the DC condition with somewhat achromatic highlights. Another candidate factor is the saliency of specular highlights induced by the color contrast between specular highlights and other regions. If this color contrast mainly induces glossiness enhancement, it should be stronger in the DC condition, where only the diffuse component is colored, than in the BC condition, where the diffuse and specular components have the same chromaticity. 

Figure 5.

 

Stimulus images for the BC and DC conditions. (a) Images of nine different color directions in area light condition. (b) Enlarged images of 180° color direction in area light condition. (c) Enlarged images of 180° color direction in ambient light condition.

Procedure

The experimental procedure was based on Thurston's pairwise comparison. The stimulus presentation sequence for each trial is shown in Figure 6. At the beginning of each trial, a pair of background images without object images was presented at a random position on the display for 500 ms, followed by a 1000 ms presentation of an image pair containing object images at the same position. Finally, the stimuli and the background were removed. After the stimulus presentation, the observer responded in a two-alternative forced-choice manner using a numeric keypad to indicate which object image appeared glossier. The subsequent trial started 1000 ms after the observer's response. Please note that the randomness of the stimulus presentation positions was used to suppress OLED display consumption, not because of any experimental requirements. 

Figure 6.

 

Stimulus presentation sequence in a trial.

In Experiment 1, two images in which only the color directions (including the achromatic color) were different and all other parameters were the same were selected as an image pair because we were interested in the effects of chromaticity on perceived glossiness. The total number of trials was 3888 \(\big(\)3 shapes × 2 illuminations × 3 diffuse reflectances × 3 surface roughness × 2 coloring conditions × \(36 \,\big( {=} \big( { {9}\atop {2}} \big)\big)\big)\) color direction combinations). Each session comprised 324 trials, resulting in 12 experimental sessions. Each session lasted approximately 15 minutes, and the observers took a short break between sessions. Image pairs were used in random order. At the beginning of each session, 20 practice trials were conducted using randomly chosen stimuli. Practice trials were conducted so that the observer could become accustomed to the experimental task and adapt to the experimental environment. Before starting the experiment, the observer was carefully instructed to observe the two object images regarding glossiness in each trial, rather than responding based on memory such as “the image with this color seemed glossier in previous trials.” In this instruction, we intended to avoid possible contamination of results by the intentional association between color and glossiness. 

Results

Preference scale values (perceived glossiness)

Preference scale values were calculated from the responses of all observers based on Thurston's case V model of pairwise comparisons. First, provisional preference scale values were calculated from the z-scores of the response probabilities. Then, the preference scale values were re-calculated with the maximum likelihood method using the provisional preference scale values as initial values to increase calculation accuracy. In each condition, the preference scale values were calculated such that those of the achromatic stimuli became zero. These preference scale values represent the magnitude of perceived glossiness. 

The preference scale values are shown in Figure 7. The rows and columns of the panels indicate the differences in roughness and diffuse reflectance, respectively. The results were averaged across object shape and illumination conditions because the general trends were similar across these conditions. In each panel, the horizontal axis represents the stimulus color direction, and the vertical axis represents the preference scale value. Chart colors represent the BC and DC coloring conditions. Error bars indicate 95% confidence intervals, calculated using the parametric bootstrap procedure with 10,000 repetitions. 

Figure 7.

 

Preference scale values. The panels correspond to different diffuse reflectance and roughness. The chart colors show the coloring conditions. The filled symbols for the chromatic conditions (0°-315°) show statistically significant differences from the achromatic condition.

The statistical significance of the scale value differences between the nine color directions was tested using a two-tailed parametric bootstrap test with 10,000 repetitions at a 5% significance level for each panel and plotted color (i.e., the testing was applied to the data averaged across object shapes and illumination conditions for each diffuse reflectance, roughness, and coloring condition). Because this test contained multiple comparisons, the significance level was corrected using the Holm method. One of the critical aspects of the results is the difference between the achromatic and chromatic conditions. The differences between the achromatic condition and each chromatic condition were statistically significant (p < 0.001) for all color directions in both the BC and DC conditions. Scale values were significantly higher for chromatic stimuli than for achromatic stimuli. This trend was widely observed for all the stimulus parameters tested, demonstrating that the coloring process increased perceptual glossiness. 

A comparison between color directions is also intriguing. Because the trend of scale values along the color direction seems similar across experimental conditions, the same statistical testing as the previous one was applied to the results averaged across experimental conditions, except for the coloring conditions and color directions. The statistical significance of the scale value differences between the color directions is summarized in Figure 8. The scale values differed significantly for many color direction pairs. The scale values tended to be remarkably higher at 0° and 315° and lower at 90° than in other color directions. These results suggest that the color-induced glossiness enhancement depends on the color direction used for coloring. 

Figure 8.

 

Statistical significance of differences in preference scale values between color directions for (a) BC condition and (b) DC condition. Before testing, the scale values were averaged across the experimental parameters, except for the coloring condition and color direction. The row and column indicate the color directions. The red elements indicate the color direction pairs with statistical significance under the significance level correction using the Holm method (p < 0.05). The values in the other white elements show p values.

Glossiness enhancement index

To further quantify the magnitude of color-induced glossiness enhancement, the difference in the preference scale values between each chromatic and the achromatic stimulus was calculated. The calculated value is referred to as the Glossiness Enhancement (GE) Index. 

Figure 9 shows the GE indexes averaged across the eight color directions for each coloring condition. The small circle plots represent GE indices for different stimulus conditions, and the bars represent the respective GE indices averaged across the stimulus conditions in the BC and DC conditions. The bootstrap test with 10,000 repetitions showed that the indexes were significantly positive in both conditions (p < 0.01), indicating that glossiness was larger for the chromatic stimuli than for the achromatic stimuli. In addition, the GE index was significantly greater under the BC condition than under the DC condition. 

Figure 9.

 

GE index averaged across color directions for BC and DC conditions. The small circles show GE indexes for different stimulus conditions, and the bars show mean GE indices across them. The error bars show 95% confidence intervals calculated through bootstrapping with 10,000 repetitions.

Discussion

Adding chromatic colors to the stimuli increased the perceived glossiness in both the DC and BC conditions. Our results reconfirmed the glossiness enhancement by chromatic colors reported in previous studies (Okajima & Takase, 2000; Hanada, 2012). 

To what extent did the H-K effect and color contrast contribute to glossiness enhancement? A comparison between BC and DC conditions is important to address this question. The GE indexes were significantly greater in the BC condition than in the DC condition. This result is consistent with the hypothesis that the brightness enhancement in highlight regions induced by the H-K effect contributed to the glossiness enhancement. However, the DC condition also exhibited an increase in glossiness with the addition of chromatic colors. Because the H-K effect was much stronger in the non-highlight regions in the DC condition, the highlight regions should be less salient in terms of perceived brightness than in the BC condition. Therefore, the glossiness enhancement in the DC condition raises the possibility that not only the H-K effect but also the color contrast contributes to the glossiness enhancement in the DC condition. Therefore both the H-K effect and color contrast were considered to have contributed to the glossiness enhancement in a combined manner. In the subsequent section, we analyze the preference scale values using a linear mixed model to evaluate the quantitative contributions of the H-K effect and color contrast. 

Linear mixed model analysis

Methods

We employed a linear mixed model to predict the GE index based on the H-K effect and color contrast. The model predicts the differences in preference scale values between achromatic and chromatic stimuli. We analyzed the DC and BC conditions separately because the magnitudes of the H-K effect and color contrast differed significantly between them. In addition, it was of interest to determine how the contributions of these factors differed between the DC and BC conditions. 

The explanatory and objective variables in this model were differences in some values (e.g., preference scale values and some image features) from achromatic to chromatic stimuli. Our analysis attempted to adopt three explanatory variables related to the psychological image features of specular highlights; they are the differences between achromatic and chromatic stimuli in (1) highlight brightness, (2) brightness contrast, and (3) color contrast. The first two features are related to the H-K effect, and the last to the color contrast. 

The three explanatory variables were measured in supplementary experiments (Supplementary Experiments A). Six observers, three of whom also participated in Experiment 1, participated in Supplementary Experiments A. All observers had normal or corrected-to-normal visual acuity and passed the Ishihara color vision test. The stimuli were the same as those used in Experiment 1. The procedures were also the same as those in Experiment 1, except for the experimental task; the observers judged highlight brightness, brightness contrast, or color contrast in separate sessions. They were carefully instructed not to confuse “brightness” and “glossiness.” However, we noticed that the judgment of brightness contrast in our chromatic stimuli was considerably difficult for the observers. Actually, the measured preference scale values for brightness contrast were not significantly different among the color directions (the results are shown in Supplementary Figure S6), indicating the possibility that the observers responded somewhat randomly. This response difficulty was verbally reported by the observers after the experiment. A possible cause of this difficulty is the contamination of brightness difference judgments by color differences in the chromatic stimuli. Therefore we decided not to use the brightness contrast results in the following analyses. The differences in the measured highlight brightness and color contrast between achromatic and chromatic stimuli were used as the explanatory variables. 

Finally, the objective variable was the difference in preference scale values (i.e., glossiness) between the achromatic and chromatic stimuli. This was calculated for each color direction as the difference in preference scale values between the achromatic and chromatic stimuli in Experiment 1. Hence, they were GE indexes calculated for each color direction, not for the mean across the color directions. 

A linear mixed model was used for regression. Because the effects of chromaticity on glossiness were of interest in this study, we focused on the differences between achromatic and chromatic stimuli and those between color directions. The two explanatory variables, highlight brightness and color contrast measured in Supplementary Experiments A, were fixed effects. The other experimental conditions (surface roughness, diffuse reflectance, object shape, and illumination) were the variable effects on the intercept and coefficient for each explanatory variable. All explanatory and objective variables were standardized before the analysis. 

Table 1 shows the correlation coefficients between the explanatory and objective variables under the BC and DC conditions, respectively. Similarly, Figures 10(a) and (b) show the change in the variables along the color direction. The variables were standardized and then averaged across experimental parameters, except for color direction and coloring conditions. In both the BC and DC conditions, the correlation coefficient with the GE index was higher for highlight brightness than for color contrast, although there were also strong correlations between the explanatory variables. 

Table 1.

 

Correlation coefficients between explanatory and objective variables in BC and DC conditions.

 

Figure 10.

 

(a)(b) Explanatory and objective variables as a function of color direction for (a) BC and (b) DC conditions. Each variable was z-scored and then averaged across conditions other than the color direction and coloring condition. The line colors show the variable types. (c, d) Relation with model prediction and z-scored GE index for (c) BC and (d) DC conditions. The circles correspond to individual experimental parameters. (e, f) GE index and model prediction as a function of color direction for (e) BC and (f) DC conditions. The plots and dotted lines show the z-scored GE index, and the solid line shows the model prediction. The differences between parameters other than the coloring conditions and color directions are averaged.

Results and discussion

Figures 10(c)–(f) show the regression results. First, Figures 10(c) and (d) indicate the relationship between the model prediction and z-scored GE index. The circle plots correspond to the individual experimental conditions. In both the DC and BC coloring conditions, the model predicted the GE index well, especially in the BC condition; the determination coefficient was 0.92 in the BC condition and 0.70 in the DC condition. Figures 10(e) and (f) show the model prediction and standardized GE index as functions of color direction. Although the trends seem similar, there are some discrepancies. In particular, the GE indexes at 90° and 225° seem different from the model prediction. 

Figure 11 shows the partial regression coefficients in the linear mixed model for the BC and DC conditions. The horizontal axis shows the explanatory variables, and the vertical axis shows the regression coefficients. Except for the color contrast in the DC condition, most coefficients were significantly different from zero, according to the t-test. The statistical values in the BC condition were as follows: highlight brightness, t(428) = 9.42, p < 0.001; and color contrast, t(428) = 2.64, p < 0.001. Those in the DC condition were as follows: highlight brightness, t(428) = 7.14, p < 0.001; and color contrast, t(428) = -0.89, p = 0.37. 

Figure 11.

 

Partial regression coefficient in regression analysis based on the linear mixed model for (a) BC condition and (b) DC condition. Statistically significant difference is indicated by asterisks.

In the BC condition, the coefficient of highlight brightness was the largest, although that of the color contrast was also significantly different from 0. These results seem plausible, considering that the stimuli in the BC condition should have strong H-K effects in highlight regions. Surprisingly, in the DC condition, the partial regression coefficient of highlight brightness was also the largest, whereas that of color contrast was not significantly different from zero. Before the experiment, we expected the color contrast to correlate more strongly with the GE index in the DC condition, since the color difference between the specular highlight and other regions was greater and the H-K effects in the highlight regions should be weaker in this condition. Therefore the regression analysis results suggest the profound role of the H-K effect in the highlight region on color-induced glossiness enhancement. 

However, there were some concerns regarding the regression analysis: (a) The two explanatory variables were highly correlated with each other. (b) Internal representations of brightness contrast may contribute to glossiness even if its perception is difficult. (c) From the high correlation between the GE index and highlight brightness, it is suspected that the observers confused glossiness and brightness in the highlight brightness task. Therefore we performed a similar regression analysis based on the explanatory variables (highlight brightness, brightness contrast, and color contrast) calculated from the stimulus images. The results of this additional analysis also support the idea of the substantial roles of highlight brightness on the color-induced glossiness enhancement effect. The details of the additional analysis are described in Supplementary Material 3. To estimate the relative contributions of the explanatory variables experimentally, we performed Experiment 2, in which only the effects of the H-K effect were psychophysically measured. 

Experiment 2: Glossiness enhancement by emulated H-K effect

The linear mixed model analysis in Experiment 1 suggested that the H-K effect predominantly contributed to color-induced glossiness enhancement under various conditions. However, this analysis did not sufficiently elucidate the contribution of highlight color contrast, possibly because of multicollinearity in the regression. Thus, in Experiment 2, we measured the glossiness of achromatic stimuli in which the increase in perceived brightness caused by the H-K effect in a chromatic object image was imitated by increasing luminance. This experiment directly examined the contribution of the H-K effect to color-induced glossiness enhancement. In addition, it can be indirectly examined by comparing the results of Experiments 1 and 2 to determine the extent to which factors other than the H-K effect, such as color contrast, contribute to color-induced glossiness enhancement. For instance, if only the H-K effect-based highlight brightness caused glossiness enhancement in Experiment 1, the glossiness enhancement in Experiments 1 and 2 should be comparable. Alternatively, if color contrast contributed even partially to glossiness, the glossiness enhancement in Experiment 2 should be inferior to that in Experiment 1. 

Methods

Observers and apparatus

The same observers as those in Experiment 1 participated in Experiment 2. We used the same apparatus as that used in Experiment 1. 

Stimulus

All stimuli in Experiment 2 were achromatic; however, their luminance was modulated to reproduce the brightness increase due to the H-K effect for the chromatic stimuli in Experiment 1. Thus, we measured the H-K effects in a supplementary experiment (Supplementary Experiment B) and simulated the H-K effect on the stimuli as follows: 

Measurement of H-K effect:

First, we measured the perceived brightness of achromatic and chromatic uniform patches at several luminance and saturation levels in Supplementary Experiment B. Based on the experimental results, the H-K effect was defined as the luminance ratio between achromatic and chromatic stimuli whose perceptual brightness matched. The Supplementary Experiment B is detailed in Supplementary Material 4. In the results, the trend of the H-K effects across the color directions was similar across luminance levels. Thus, after averaging the H-K effects between luminance levels, the averaged H-K effect was regressed with a linear model as a function of stimulus saturation. Another reason to employ single multiple regression based only on saturation was that the H-K effect estimated by multiple regression based on both luminance and saturation failed to predict the perceived highlight brightness measured for the linear mixed model (Figures 10(a), (b)). 

Creation of H-K luminance stimuli:

We then derived the achromatic luminance whose perceived brightness matched the chromatic images of Experiment 1 in a pixel-by-pixel manner as follows: (1) The H-K effect at every pixel was estimated using the regression model based on its saturation, and 2) the original luminance was multiplied by the estimated H-K effect. The resultant value is referred to as the H-K luminance. An achromatic image with H-K luminance is considered to exhibit perceived brightness similar to that of chromatic stimuli. In other words, they were images with H-K simulated luminance. Hereafter, we refer to these stimuli with luminance manipulation as H-K stimuli, whereas the stimuli without luminance modulation are referred to as original stimuli (i.e., they were the same as the achromatic stimuli in Experiment 1). 

Only some of the experimental conditions in Experiment 1 were employed. The object shape was the bunny, the illumination was the area light source, and the surface roughness was 0.05 and 0.2. However, the diffuse reflectance, coloring condition (DC and BC conditions), and color directions were the same as those in Experiment 1. Please note that “color direction” does not indicate the chromaticity condition but the luminance condition in Experiment 2. Because H-K luminance depends on the color direction in Experiment 1, the difference in H-K luminance is called “color direction” for convenience. 

Figures 12(a) and (b) show examples of H-K stimuli in the BC and DC conditions, respectively, in addition to the original stimulus in Figure 12 (c) for comparison. In the BC condition, the luminance is increased in the entire object region, including the highlight regions. In contrast, in the DC condition, the application of H-K luminance causes a larger luminance increase in non-highlight areas, resulting in a decrease in the highlight luminance (and brightness) contrast. 

Figure 12.

 

Examples of H-K (0 degrees) stimuli in the (a) BC and (b) DC conditions, and (c) the original stimulus. The original stimulus was the same as the achromatic stimulus in Experiment 1.

Before performing Experiment 2, perceived brightness was compared between the chromatic stimuli of Experiment 1 and the H-K stimuli of Experiment 2 in a supplementary experiment (Supplementary Experiment C). In the results, the perceived brightness and its trends across the color directions were approximately comparable between them. However, perceived brightness was rather higher in the H-K stimuli at 180° to 270°. This indicates that the luminance is slightly over-enhanced in the H-K stimuli as compared with brightness of the chromatic stimuli. This point is considered in Discussion section. The details of Supplementary Experiment C are described in Supplementary Material 5. 

Procedure

The experimental procedure was designed based on Thurston's pairwise comparison. The stimulus presentation sequence was the same as that used in Experiment 1. In each trial, the observer responded to which image appeared glossier. The stimulus images were paired between the nine stimuli (the original stimulus, and the eight H-K stimuli with different color directions), resulting in \(36 \,\big( {=} \big( { {9}\atop {2}}\big)\big)\) pairs for each stimulus condition. 

The number of trials was 432, comprising 12 conditions (three diffuse reflectance, two surface roughness, and two coloring conditions) and 36 color direction pairs. Each session consisted of 216 trials; therefore two sessions were performed for each observer. Both sessions were conducted on a single day, with the observer given a short break between sessions. 

Results

Preference scale values

As in Experiment 1, the preference scale values were calculated based on all observers’ responses and were used as the index of perceived glossiness. Figure 13 shows the preference scale values measured in Experiment 2. The filled and open symbols in the H-K stimuli show the conditions with and without statistically significant differences from the original stimuli, respectively, based on bootstrap testing with 10,000 repetitions. In addition, Figure 14 shows the statistical significance of differences between the color directions, including the original stimulus, assessed using the bootstrap procedure with 10,000 repetitions, after averaging the scale values across diffuse reflectance and roughness. The scale values were generally higher for the H-K stimuli than for the original stimulus in the BC condition. By contrast, in the DC condition, the scale values for the H-K stimuli were higher only in some color directions, whereas they were comparable or lower in approximately half of the color directions. Regarding the differences among the color directions, the scale values tended to be highest at 180° in the BC condition. Meanwhile, in the DC condition, the scale values tended to be highest at 135° and lowest at 225°. 

Figure 13.

 

Preference scale values in Experiment 2. The format is the same as in Figure 7.

Figure 14.

 

Statistical significance in differences among color directions. The format is the same as Figure 8.

Glossiness Enhancement (GE) index

We calculated the difference between the preference scale value for the original stimulus and the mean values across the eight H-K stimuli for each stimulus condition. The calculated value represents the magnitude of glossiness enhancement caused by the luminance manipulation. Thus, this resultant value was referred to as the GE index, as in Experiment 1. 

Figure 15 shows the GE index for each coloring condition. The index was larger in the BC condition than that in the DC condition, as in Experiment 1. The difference between the coloring conditions was statistically significant according to the parametric bootstrap test with 10,000 repetitions (p < 0.001). 

Figure 15.

 

GE index in Experiment 2. The format is the same as in Figure 9.

Here, we compared the glossiness enhancement between the coloring in Experiment 1 and the simulated H-K effect in Experiment 2 by calculating the ratio of GE indices in Experiments 1 and 2 for each coloring condition. Note that the GE index of Experiment 1 was re-calculated using only the results of the experimental conditions common to Experiment 2, because Experiment 2 used only part of the conditions. Figure 16 shows the ratio of the GE indexes between the coloring conditions. The mean ratios of the GE index were 1.07 in the BC condition and 0.54 in the DC condition. The mean ratio was not significantly different from one (i.e., the same magnitude of glossiness enhancement between Experiments 1 and 2) in the BC condition (p = 0.1396), whereas it was significantly lower than one in the DC condition (p < 0.05), according to the bootstrap test with the significance level adjusted based on the Holm method for multiple comparisons. 

Figure 16

 

The GE index ratio of Experiment 2 to that of Experiment 1.

Discussion

In the BC condition, there was no significant difference in the glossiness enhancement indices between chromatic stimuli (Experiment 1) and H-K stimuli (Experiment 2). This result is consistent with the interpretation that the increase in highlight brightness owing to the H-K effect is a dominant factor in color-induced glossiness enhancement. The dominance of highlight brightness induced by the H-K effect is reasonable because the color and brightness contrast of highlights are hardly modulated by the coloring operation in the BC condition. However, it should be noted that the perceived brightness of the H-K stimuli in the BC condition was higher than that of the chromatic stimuli in Experiment 1 (Supplementary Figure S15). This raises the possibility that the glossiness enhancement caused by the H-K effects inferred from the results of Experiment 2 may have been slightly overestimated. Therefore there may be slight room for additional contributions of color factors other than the H-K effects in the color-induced glossiness enhancement in the BC condition. 

In contrast, the results for the DC condition differed from our expectations. In the DC condition, applying the H-K effect-based luminance modulation depressed the brightness contrast of specular highlights, though it weakly increased the highlight brightness. The GE index was positive for the H-K stimuli, and the ratio of the GE index to that in Experiment 1 was 0.54. These results suggest that the H-K effect also contributes to the color-induced glossiness enhancement in the DC condition to some extent. In addition, considering the decrease in highlight contrast due to the H-K effect simulation, the increase in the GE index should have been induced by highlight brightness. These results appear consistent with those of the linear mixed model analysis, in which the coefficients of highlight brightness were the largest. Of course, it is unlikely that brightness contrast does not contribute to glossiness. In fact, in the DC conditions of Experiment 2, where the brightness contrast was lower than that of the original stimuli, the glossiness in the H-K stimuli was lower than that in the original stimuli in some color directions, such as at 225° as expected. This result suggests that the brightness contrast also contributes to glossiness, but only weakly. 

Regarding the effects of brightness contrast, another intriguing result is that the color direction dependency of glossiness largely differed between the BC and DC conditions. The general color direction dependency in the BC condition appears similar to the highlight brightness influenced by the H-K effect in Supplementary Figure S8 (a). In contrast, the trend in the DC condition appears to be similar to that of the brightness contrast in Supplementary Figure S8 (b). The color direction dependency in the DC condition also suggests that the highlight contrast contributed to glossiness enhancement in the DC condition of Experiment 2. However, the increase in glossiness in the DC condition cannot be fully explained by the brightness contrast because the brightness contrast decreased after luminance manipulation for all color directions. Thus glossiness in the DC condition is likely to be affected by both highlight brightness and brightness contrast; highlight brightness increased glossiness, and brightness contrast decreased glossiness. 

Finally, image features other than the H-K effect may have contributed to glossiness enhancement. The glossiness enhancement in the H-K stimuli was weaker than that in the chromatic stimuli under the DC condition, as described above. Furthermore, there were some differences in the glossiness variation trends along the color directions of chromatic colors in Experiments 1 and 2. In the BC condition in Experiment 1, glossiness was notably higher at 0° and 315°, whereas in the BC condition of Experiment 2, glossiness was the highest at 180°. This indicates that the glossiness variation along the color direction cannot be completely explained by the H-K effect alone. In other words, these results suggest the existence of factors other than the H-K effect that contribute to glossiness enhancement. The possible factors are considered color-specific, such as highlight color contrast. 

General discussion

The purpose of this study was to psychophysically elucidate the relative contributions of the H-K effect and color contrast to color-induced glossiness enhancement. In Experiment 1, we measured the glossiness of achromatic and chromatic object images and analyzed the contribution of the H-K effect and color contrast to color-induced glossiness enhancement using a linear mixed model. The results show that the partial regression coefficient of the H-K effect-based highlight brightness was the largest among the explanatory variables in both the BC and DC chromaticity conditions, suggesting a predominant contribution of the H-K effect to glossiness enhancement. In Experiment 2, we measured the glossiness of the luminance-modulated achromatic stimuli to directly measure the influence of the H-K effect on glossiness enhancement. In the stimuli, the brightness increase due to the H-K effect was emulated by manipulating the luminance of achromatic images. The results showed that H-K stimuli in the BC condition induced glossiness enhancement comparable to the chromatic stimuli in Experiment 1, whereas the increase in glossiness in the DC condition was about half of that in Experiment 1. These results indicate that the H-K effect is the dominant cause of color-induced glossiness enhancement, although other factors may also be involved. 

Contribution of the H-K effect to glossiness

Previous studies have suggested the contributions of the H-K effect and color contrast to color-induced glossiness enhancement separately (Okajima & Takase, 2000; Hanada, 2012). In this study, glossiness was measured on stimulus images that included both factors. In Experiment 1, a multiple regression analysis based on a linear mixed model demonstrated that the partial regression coefficient of the H-K effect-based highlight brightness was larger than that of the highlight color contrast in both the BC and DC conditions. In contrast, the partial regression coefficient for color contrast was significant in the BC condition but not in the DC condition. These results suggest that the H-K effect is a dominant factor in color-induced glossiness enhancement over a wide range of conditions. 

Other characteristics of Experiment 1 results also suggest a considerable contribution of the H-K effect to glossiness enhancement. In particular, the effects of diffuse reflectance are intriguing. The saturation in the highlight region of the chromatic stimuli increased with diffuse reflectance in the DC condition of Experiment 1 because the proportion of diffuse reflectance components increased with diffuse reflectance. This high saturation increased highlight brightness via the H-K effect (Supplementary Figure S3(a)). In contrast, it decreased the highlight brightness contrast and highlight color contrast to non-specular regions because the saturation difference between specular and non-specular regions became smaller (Supplementary Figure S3(b)). Therefore, the effects of diffuse reflectance on the GE index help us infer the factors influencing the color-induced glossiness enhancement. Figure 17 shows the GE index for different diffuse reflectances under the DC condition of Experiment 1. The GE index became larger with diffuse reflectance; there were significant differences between the diffuse reflectance of 0.1 and 0.3 and between 0.1 and 0.5 based on bootstrap testing with 10,000 repetitions, with an adjusted significance level using the Holm method (p < 0.05 after significance level correction). This result also suggests that the increase in highlight brightness due to the H-K effect contributed more significantly to color-induced glossiness enhancement than brightness and color contrast. 

Figure 17.

 

GE index for different diffuse reflectance in the DC condition.

The results of Experiment 2 also support the importance of highlight brightness for glossiness enhancement. The luminance in the highlight regions was higher in the H-K stimulus than in the original stimulus in the DC condition. However, the highlight luminance contrast of the H-K stimuli was weaker than that of the original stimuli because of the stronger H-K effects in the non-highlight regions, as shown in Supplementary Figure S4. Because previous studies have reported a positive correlation between highlight luminance contrast of and glossiness of object images (Marlow et al., 2012; Wiebel et al., 2015), we expected that under the DC condition, the H-K stimuli would exhibit lower glossiness than the original stimulus. Contrary to this expectation, however, the experimental results showed that the H-K stimuli exhibited greater glossiness. These results also support the idea that an increase in highlight brightness may be the predominant factor in the color-induced glossiness enhancement. 

Future works

The H-K effect explained the gloss variation across the color directions in Experiment 1 reasonably well. However, there are concerns that the brightness judgments in Supplementary Experiments A may have been confused with glossiness. If this was the case, the determination coefficients in the model might have been overestimated. There were also small discrepancies between the model prediction and GE index in some color directions, such as 90° and 180° (Figures 10(e) and (f)). Thus we should not dismiss the possibility of additional color-related factors for color-induced glossiness enhancement, other than those tested here. A candidate is an asymmetry in the bluish and yellowish directions; a bluish color on object surfaces tends to be perceived as achromatic than a yellowish color (Winkler, Spillman, Wener, & Webster, 2015). Such differences in perceived colors across hues might induce perceptual color differences in the specular highlight regions, resulting in glossiness differences. 

In addition, the magnitude of the color-induced glossiness enhancement may be affected by the luminance patterns on object images. In our analysis, the model based only on the H-K effect and color contrast could not explain the differences between the experimental parameters other than the coloring conditions and color directions because the effects of these parameters were absorbed as variable effects in the linear mixed model. For example, if highlight brightness is essential for glossiness perception, luminance patterns that induce perceptually large highlights may be responsible for the magnitude of color-induced glossiness enhancement. Therefore, the effects of these factors must be examined. 

Investigating color-induced glossiness enhancement for more naturalistic stimuli, such as photographs of real objects, is also essential. This study used object images of different shapes, lighting environments, and surface roughness as stimuli. However, the colors on the object surfaces differed from those on real objects because we controlled the stimulus colors artificially for parametric manipulation of the colors as experimental parameters. Therefore we must examine whether the same factors can explain glossiness enhancement in stimuli with physically accurate colors. Colored object images similar to our stimuli can be created using objects made of different materials such as metals and plastics. In future studies, we will examine color-induced glossiness enhancement on realistic stimuli and the contribution of the H-K effect and color contrast to the effects. 

Finally, it is worth noting that all analyses and suggestions in this study were based on the “luminance” definition of CIE 1931, following many studies on surface quality perception. This “luminance” should be close to the perceptual properties averaged across several observers which are measured by isoluminant measurement methods such as heterochromatic photometry. However, the additive properties of cone signals in luminance significantly depend on the psychophysical tasks to measure “equiluminance” (Koenderink, van Doorn & Gegenfurtner, 2018). Although the H-K effect is also an effect of chromaticity on CIE 1931 luminance, it is not clear how color contributes to other definitions of luminance. Thus further research is needed on the definition of “luminance” and its representation in the visual system in relation to surface quality perception. 

Conclusions

We measured the perceived glossiness of colored object images under two coloring conditions: the BC condition, in which chromatic color was assigned to both specular and diffuse components, and the DC condition, in which chromatic color was assigned only to diffuse components. The results indicated that coloring significantly increased the perceived glossiness. In addition, a linear mixed model analysis of the experimental results indicated that the H-K effect-based brightness enhancement in the highlighted region best explained the glossiness enhancement. To experimentally confirm the contribution of the highlight brightness enhancement due to the H-K effect, we measured the glossiness of achromatic stimuli in which the H-K effect-based brightness enhancement was imitated by luminance manipulation. The results demonstrated that glossiness enhancement was comparable to that of chromatic stimuli in the BC condition. In contrast, the H-K effect alone cannot thoroughly explain the glossiness enhancement of chromatic stimuli in the DC condition, such as its magnitude and the trend in glossiness enhancement along the color directions. In summary, our results suggest that the increase in highlight brightness caused by the H-K effect is a dominant factor in color-induced glossiness enhancement, although other factors such as color contrast may be partially involved. 

Acknowledgments

Supported by JSPS KAKENHI Grant Number JP19H04197 and 21H05810 to TN. 

Commercial relationships: none. 

Corresponding author: Takehiro Nagai. 

Email: [e-mail]. 

Address: Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, 4259-G2-1 Nagatsuta-cho, Midori-ku, Yokohama, Kanagawa 226-8502, Japan.