Looking against the light: How perception of translucency depends on lighting direction

Abstract

Abstract

Abstract:

Abstract  Translucency is an important aspect of material appearance. To some extent, humans are able to estimate translucency in a consistent way across different shapes and lighting conditions, i.e., to exhibit translucency constancy. However, Fleming and Bülthoff (2005) have shown that that there can be large failures of constancy, with lighting direction playing an important role. In this paper, we explore the interaction of shape, illumination, and degree of translucency constancy more deeply by including in our analysis the variations in translucent appearance that are induced by the shape of the scattering phase function. This is an aspect of translucency that has been largely neglected. We used appearance matching to measure how perceived translucency depends on both lighting and phase function. The stimuli were rendered scenes that contained a figurine and the lighting direction was represented by spherical harmonic basis function. Observers adjusted the density of a figurine under one lighting condition to match the material property of a target figurine under another lighting condition. Across the trials, we varied both the lighting direction and the phase function of the target. The phase functions were sampled from a 2D space proposed by Gkioulekas et al. (2013) to span an important range of translucent appearance. We find the degree of translucency constancy depends strongly on the phase function's location in the same 2D space, suggesting that the space captures useful information about different types of translucency. We also find that the geometry of an object is important. We compare the case of a torus, which has a simple smooth shape, with that of the figurine, which has more complex geometric features. The complex shape shows a greater range of apparent translucencies and a higher degree of constancy failure. In summary, humans show significant failures of translucency constancy across changes in lighting direction, but the effect depends both on the shape complexity and the translucency phase function.

Introduction

Many natural materials we encounter every day are translucent, including the food we eat, the liquids we drink, and our skin. The translucent appearance of these materials is caused by internal volumetric scattering. Figure 1A shows photographs of a cherub under two different lighting directions. One distinctive feature of the material is that it is permeable to light. We see part of the green background through the thin parts of the cherub, such as its wings. This is because some light penetrates into the cherub that allowed the background to be seen through the object. 

Figure 1

 

Translucent materials. (A) Studio photographs of a cherub figurine made with polyurethane lit from two lighting directions, from behind and from the side. The one on the left looks more translucent. (B) Photographs of two different translucent materials under the same natural illumination. The same “frog prince” shape was made of two different translucent materials, soap, and wax. Observers can discriminate the subtle differences between the two materials.

Humans are skilled at discriminating subtle differences in translucency, such as when they discriminate milk from cream, wax from soap, and human flesh from a doll. Figure 1B shows photographs under natural illumination of the same “frog prince” shape made of two different materials, soap and wax; the translucent appearance of the two materials is very different. Since in the natural world translucent materials are the norm, not the exception (Donner et al., 2009), it makes sense that the human visual system is well engineered to analyze them. However, very little is known about how the visual system recognizes and discriminates translucent materials from images. 

Creating controlled inputs to study translucency with real objects is a challenge as it is hard to manufacture objects with specific scattering parameters. Further, until very recently rendering realistic translucent objects was not feasible using computer simulation, thus preventing the use of such stimuli for controlled studies. Therefore, most previous work in translucency perception has focused on stimuli with simple shapes, unnatural illumination, and a limited set of materials. These works hypothesized that image cues such as specular highlights, blur, image contrast, and color variation could influence the perception of translucency. However, to understand translucency perception in the real world, we need to expand on each of these three parameters: We need to consider complex shapes, realistic lighting, and a richer set of materials. 

It is commonly observed that lighting has dramatic effects on translucent objects: They tend to appear more translucent under backlighting. For example, people often hold up a translucent object against a light while studying it. This change in lighting direction also causes many of the measureable image properties (such as image contrast, location of specular highlights, cast shadows, luminance histograms, etc.) to change. Studying the effect of lighting direction on translucency can be very informative of the images cues used by the visual system. In addition, the lighting effect varies depending on the 3D shapes and material properties of objects. For example, the translucent appearance of a jade bracelet might be more sensitive to a change of lighting direction than a bar of soap. Therefore, studying lighting on translucency perception with a controlled 3D shape and material properties is important. 

Changing lighting direction also influences the perception of 3D shape of an object, which indirectly influences the perception of material properties (Gerardin, Kourtzi, & Mamassian, 2010; Koenderink & van Doorn, 2001; Olkkonen & Brainard, 2011; Todd, 2004; Wijntjes & Pont, 2010). A previous study on the effect of lighting direction on translucency used a simple shape, a torus (Fleming & Bülthoff, 2005). By observing translucent objects in real life, we find lighting direction influences objects that have a variation of thin and thick geometry, such as the cherub shown in Figure 1A, in a different way from an object with a simple geometry, such as a torus (Xiao et al., 2012). Gkioulekas et al. (2013) showed that changing the shape of the spherical scattering distribution (which is called phase function; details are provided in the section “Background and related work”) affected translucent appearance. This effect is small for thick geometries because the appearance is dominated by high-order scattering. However, the phase function can impact appearance in a perceptually important way near thin geometric structures. 

The present study aims to systematically understand how lighting direction affects the perception of translucency of objects using natural shape, realistic lighting, and advanced rendering methods. Our result shows that lighting direction has a different effect for different phase functions, especially for 3D shapes that contain thin geometric structures. 

Background and related work

We begin by introducing the physical process of subsurface scattering that is responsible for the appearance of the translucent objects, the phase function models, and current rendering methods for translucency. We will then review previous studies on material perception and constancy, with an emphasis on the effects of lighting environment and 3D shape. 

Translucent materials

Subsurface scattering:

Figure 2A illustrates how light interacts with translucent objects and the parameters controlling subsurface scattering. When light hits the surface of an opaque object, it bounces back from the point that is illuminated. When light hits a translucent object, only some of it bounces off from the surface. The rest penetrates into the object, refracts, and scatters multiple times within the body of the medium, until it re-emerges outside from a different location. In computer graphics, this process can be modeled using the radiative transfer equation (RTE) (Ishimaru, 1978). This approach models the material using three wavelength dependent functions, the scattering coefficient σ_s, the absorption coefficient σ_a, and the phase function p. For light at a single wavelength, σ_s and σ_a describe how rapidly the light is scattered and absorbed in the medium. When light is scattered, the phase function, which is a probability distribution over directions, describes the angular distribution of scattered light. The sum of the scattering and absorption coefficients is called the extinction coefficient, σ_t = σ_s+ σ_a, also referred to as the density of the material, and the ratio σ_s/σ_t is called the albedo. The density controls the degree of translucency of the medium. A more translucent medium has a low density while a less translucent medium has a high density. When light interacts with the medium, the albedo controls what fraction of this light is scattered instead of being absorbed. The scattered light then continues in new directions controlled by the phase function. In this paper, we use the density, the albedo, and the phase function to describe the scattering behavior. 

Figure 2

 

Illustration of volumetric scattering and the phase function. (A) Illustration of subsurface scattering. Left: Translucent appearance is caused by scattering of light inside the volume of an object and described by the radiative transfer equation. Middle: A closer look of subsurface scattering: Light travels through a medium. The extinction coefficient controls the distance before a volume event occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered into some new direction within the volume. Right: The phase function describes the angular distribution of the scattered light. The green circle represents an isotropic phase function, meaning one that scatters equally in all directions. (B) The effect of the phase function on the final appearance of an object depends strongly on its geometry. The “Lucy” model is rendered with two different phase functions, represented by orange and purple frames. The shape of the phase function is displayed below the images. The green circle, which represents an isotropic phase function, is added to both. In the thick object parts, such as the body, the difference in the two images is small. However, in the thin object parts, such as the wing, the image differences help us distinguish between the two materials.

Phase function model:

In this work, we focus on the effect of phase function and study how the combination of phase function and lighting direction affect perceived translucency. We are motivated by the following reasons. First, the phase function can impact appearance in perceptually important ways near thin geometric structures, where light undergoes only a handful of scattering events before exiting towards the observer. Figure 2B illustrates how two different phase functions affect the appearance of translucent images, especially near thin edges. Second, since lighting direction often affects the appearance of thin edges (see Figure 2B), this leads us to ask whether there is an interaction between phase function and lighting direction on the effects of translucent appearance for complex-shaped objects that have both thin and thick parts. Therefore to thoroughly study the effects of lighting direction, we believe it is necessary to explore the perceptual space of the phase function. 

A common analytical form of the phase function used in graphics is due to Henyey and Greenstein (1941). It is controlled by a single parameter, and can represent both predominantly forward and predominantly backward scattering. To model more complex types of scattering, a linear combination of the two Henyey and Greenstein (HG) lobes, one forward and one backward, can be used, but there are still materials for which this is inadequate. It was shown in (Gkioulekas et al., 2013) that the appearance of materials such as microcrystalline wax and white jade cannot be reproduced using HG phase functions alone. 

Gkioulekas et al. (2013) extended the phase function family to consider lobes from another single-parameter family, the von Mises-Fisher (vMF) family. This extension allowed us to have all the linear combinations of two types of lobes from both HG and vMF, representing a much larger space of phase functions and hence, a much more diverse set of materials. This extension increased the physical parameter space, but very different phase functions could produce the same visual appearance. This suggested the perceptual space of translucency was much smaller than this expanded physical parameter space. That study established a lower-dimensional embedding of the phase function that captures most of the variance in perception. 

To achieve this goal, Gkioulekas et al. (2013) combined psychophysics with computational analysis. This involved three steps: First, we densely sampled the physical parameter space of phase functions to compute thousands of images, and employed a nonmetric multidimensional scaling methods (Wills, Agarwal, Kriegman, & Belongie, 2009) with various image distance measures to find a two-dimensional embedding of the space (Figure 3B). The two-dimensional embedding is consistent across different distance metrics, shapes, and lighting. Second, we ran a psychophysical study, where observers performed paired-comparisons between images. We found that these perceptual distances were consistent with a two-dimensional embedding that was similar to that found computationally, thereby affirming the perceptual relevance of that embedding. Thirdly, we performed statistical analysis of the computationally obtained and perceptually consistent two-dimensional embedding, to investigate how its dimensions related to phase function shape. We identified two linearly independent axes that were described as simple analytic expressions of generalized first and second moments of the phase functions. We also obtained a learned distance metric that could be used to compute an efficient approximation to the perceptual distance between any two tabulated phase functions. (For details of the parameterization of the 2D space, please see Gkioulekas et al., 2013.) The visualization of the 2D phase function was provided in Figure A1 in the Appendix. 

Figure 3

 

Phase functions and translucent appearance. (A) “Lucy” rendered with two different phase functions but with the same density and albedo. (B) Two-dimensional computational embedding of phase functions constructed using a multi-dimensional scaling method (Gkioulekas et al., 2013). The purple dot marks the phase function used to render the left “Lucy” and the orange dot marks the phase function that used to render the right “Lucy” in A. The gray dots represent full set of images used in the computational embeddings (for details, please see (Gkioulekas et al., 2013, figure 7).

Figure 3A shows how phase function can affect the appearance of translucent objects while the other scattering parameters are kept the same. The left image shows the rendered “Lucy” model (http://graphics.stanford.edu/data/3Dscanrep/) using a forward vMF and isotropic phase function, and the right image shows a “Lucy” rendered with a mixture of a forward Henyey and Greenstein and a backward von Mises-Fisher phase function. Both objects are side-lit. The “Lucy” on the left looks more diffusive (“marble-like”) and the “Lucy” on the right looks glassy (“jade-like”). Figure 3B shows a 2D computational embedding of phase functions and the positions of the two phase functions used to render images in Figure 3A in the 2D space. The possible differences in image brightness are caused by the physics of the scattering of the two-phase functions. Moving from left to right on the embedding results in phase functions with larger variance, implying that more light gets scattered towards side-directions, and therefore less light exits from the front for the camera to observe. This was discussed extensively in the discussion section in the paper by Gkioulekas et al., 2013. 

Rendering translucency:

Solving the radiative transfer equations is computationally expensive. In computer graphics, approximations of the RTE have been used to avoid the cost of simulation. Multiple scattering, defined as light that has scattered many times within the medium, often varies smoothly across the surface and is independent of the viewing direction. The pioneering work by Jensen (2001) proposed the dipole diffusion model, an approximate solution based on the diffusion equation. This model is fast and widely used, and it has created a large body of research in dipole solutions and multiple scattering. The dipole approximation, however, is based on a semi-infinite planar model and can be quite inaccurate in thin or highly curved regions. This approximation is also only suitable for thick objects whose appearance is dominated by high-order scattering. In this work, we instead used volumetric Monte Carlo path tracing to directly solve for the radiative transfer without the need for such approximations. While more expensive, this approach is guaranteed to produce the correct solution under all lighting, material, and shape conditions. 

Perception of translucent materials:

There has been a long history of research on transparent materials (Helmholtz, 1867; Koffka, 1935; Metelli, 1974). But traditional research on the perception of transparent materials has focused on scenes with overlapping thin transparent layers (Adelson, 1993; Anderson, 1997; Anderson, 2003; Anderson & Khang, 2010; Beck, 1978; Beck & Ivry, 1988; Beck, Prazdny, & Ivry, 1984; Chen & D'Zmura, 1998; Kersten, 1991; Kersten, Bülthoff, Schwartz, & Kurtz, 1992; Khang & Zaidi, 2002; Koenderink & van Doorn, 2001; Sayim & Cavanagh, 2011; Watanabe & Cavanagh, 1992; Watanabe & Cavanagh, 1993). Metelli developed an episcotister model to explain perception of transparent objects that are infinitely thin neutral density filters. However, this model cannot explain the effects of subsurface scattering, where light travels within the body of the object. Furthermore, the image cues in translucent layers are different than those observed in transparent layers. 

There has been little research on translucency perception of three-dimensional objects, with a few notable exceptions that we discuss now. Fleming and Bülthoff (2005) was the first study to use synthetic images of 3D objects to investigate the perception of translucency. Their study examined the effects of several low-level image cues, such as specular highlights, saturation in color, edge blurring, image contrast, and luminance histogram on the perception of translucency. Fleming and Bülthoff also studied the effect of direction of illumination direction on translucency perception. They used a simple torus-shaped object and varied the illumination direction by rotating a point light source. Their images were rendered using an isotropic phase function and the dipole approximation method (Jensen, 2001). They asked observers to adjust the scattering coefficient of a torus to match the translucency of a target torus under a different lighting direction. They found that observers were bad at discounting the effect of lighting source direction and objects tend to appear more translucent when illuminated from the back. In this paper, we use a similar matching paradigm and extend their work by using state-of-the-art rendering to capture the translucent appearances of complex-shaped objects under different lighting directions. 

Motoyoshi (2010) found that manipulating the contrast and blur of the nonspecular image component of translucent objects could alter the translucent appearance. Nagai et al. (2013) used a psychophysical reverse correlation method to extract spatial regions related to translucency perception from rendered images of objects. They show that the global root mean square contrast within an entire rendered image was not related to perceptual translucency, but the local mean luminance of specific image regions within the image correlates well with perceived translucency. 

Recent work by Fleming, Jakel, and Maloney (2011) proposes image cues for thick transparent objects that have irregular shapes. They discovered that image distortions that occur when a textured background is visible through a refractive object are a key cue that the visual system can use to estimate an object's intrinsic material properties. 

Material constancy under variation of lighting direction

Humans are good and fast at perceiving material properties of objects from images (Sharan, Rosenholtz, & Adelson, 2009). Similar to color constancy, the visual system must be able to discount variation of lighting and viewpoint to achieve a stable representation of material properties (“material constancy”). There have been many studies on material constancy under variations of illumination geometry, but most have focused on surface gloss and roughness (for reviews, see Anderson, 2011; Maloney & Brainard, 2010). In a pioneering study, Fleming, Dror, and Adelson (2003) illustrated the importance of real-world illumination on material perception. Obein, Knoblauch, & Viéot (2004) examined how gloss perception is affected by illumination direction and find that gloss difference scales obtained under two different illuminations are very similar, implying that humans can achieve “gloss constancy.” Ho, Landy, and Maloney (2006) and Ho, Maloney, and Landy (2007) studied how surface roughness perception was affected by scene illumination and viewpoints, and found that observers exhibited some constancy to variation of viewpoint. Doerschner, Boyaci, and Maloney (2010) studied how perceived gloss could be transferred from one light field to another. Despite all of the progress, very little is known about how illumination direction affects translucency perception. 

3D shape and material perception

It was previously found that the perception of 3D shape interacted with the perception of material properties, though most results pertain to the perception of surface gloss. Fleming, Torralba, and Adelson (2004); Norman, Todd, and Orban (2004); and Todd and Norman (2003) showed that specular reflectance aided shape estimation. Muryy, Welchman, Blake, and Fleming (2013) further demonstrated that disparity of specular highlights was important in shape estimation. Vangorp, Laurijssen, and Dutré (2007) showed the importance of 3D shape in the estimation of surface reflectance from images. However, a recent brain imaging study suggested that surface cues that contribute to object shape are processed separately from surface cues that are linked to an object's material properties (Cant, Large, McCall, & Goodale, 2008). 

In this paper, we expand the study of the perception of translucency to objects with natural and complex shapes, and with an expanded set of material parameters from those that have been previously studied. Using complex shapes, spherical harmonic basis functions to represent light, physically accurate rendering techniques, and an expanded space of phase functions to render rich materials, we find that lighting direction has a strong effect on translucent appearance for some phase functions but not others. This effect can be predicted from the position of the phase function on a 2D perceptual phase function space established in Gkioulekas et al. (2013). 

Method

We conduct an appearance-matching task to measure how perceived translucency depends on both lighting direction and phase function. We use the “Lucy” model from the Stanford 3D scanning repository as the 3D shape and spherical harmonic illumination of varying dominant direction. 

Stimuli

Rendering

Figure 4 shows an example trial. The scene contains a “Lucy” angel, which is roughly 7 cm tall standing on a dark gray floor. The interior of the angel is modeled as a volumetric homogeneous scattering medium with an index of refraction of 1.5. As mentioned before, these media can be characterized by three parameters: density, albedo, and phase function. In our experiments, observers adjust the density of the medium using a logarithmic (base = 2) scale. We used an albedo of 0.99 in our experiments and 12 different phase functions chosen from Gkioulekas et al. (2013). 

Figure 4

 

Experimental interface. We use a matching task to measure perceived translucency under different lighting conditions. During each trial, the observer adjusts the density of a simulated “Lucy” angel (match image on the right) using a slider to match the material properties of an angel of a particular density (target image on the left) simulated under another lighting direction. After each trial, the observer is asked to answer whether they are happy with the match or not. The observer is instructed that this is a subjective task. The image has the size of 481 × 421 pixels. The viewing distance is 50 cm. The visual angle of the image is 5.9° × 5.3°.

The surface of the “Lucy” is modeled as a rough refractive surface (Walter, Marschner, Li, & Torrance, 2007) with Beckmann roughness of 0.1. This choice affects the highlights and refraction. We use the rougher appearance of the surface so that the “Lucy” appears to be more natural. 

Spherical harmonic basis lighting

Our goal is to create lighting while controlling its direction: front, back, and side-lit. The illumination has its maximum value in one direction, and decreases monotonically to zero in the opposite direction. More precisely, its value is proportional to Display Formula where Display Formula is the angle to the maximal direction of the illumination pattern. This simulates a relatively large area of lighting while also providing a strong sense of lighting directionality (such as side- or front-lit). We can represent this pattern using 16 spherical harmonic coefficients, and this representation also allows arbitrarily rotating the illumination from a single volumetric simulation. In our experiments, we chose to use only a small set of preselected illumination directions. The lighting direction is controlled by the polar and the azimuthal angles so that the maximum brightness can occur at any direction we want (to get front-lit, side-lit, back-lit conditions, etc.). For more details about the spherical harmonic representation of lighting, please see Ramamoorthi and Hanrahan (2001). 

2D embedding of phase functions

In this experiment, we uniformly sample 12 phase functions from the perceptually uniform two-dimensional embedding established in our previous study (Gkioulekas et al., 2013), which we then use in all of our experiments. Figure 5 shows example images rendered using the 12 phase functions and their corresponding positions in the 2D embedding. Doing the sampling this way means that the set of phase functions we select is representative of all of the perceptually important appearance effects phase functions can produce. Previous studies used phase functions lying only on a one-dimensional slice concentrated to the leftmost part of this appearance space. Consequently, they neglected many perceptual effects that can be critical for accurately reproducing the appearance of different classes of translucent materials, such as jade. As our results demonstrate, it is necessary to study the full space of phase functions, as the magnitude of the effects we observe varies considerably depending on the phase function in the appearance space. The details of the full embedding of the phase functions can be seen in figures 7, 8, and 11 in Gkioulekas et al., 2013. The parameters of the 12 phase functions are provided in the Appendix at the end of the paper. 

Figure 5

 

Examples of experiment stimuli (polar angle = 90° and density = 4 (log) for three lighting directions, azimuthal angle = −30°, 90°, 270°. The phase functions numbers correspond to the numbers of the phase functions sampled from the 2D embedding shown in Figure 3B. (To see the original version of the figure, please go to the Supplementary Material website.)

Procedure

An asymmetric matching task is used to measure changes in perceived translucency caused by variations of lighting direction and phase function parameters. Figure 4 shows the interface of the experiment. The image on the left is the target image. The one on the right is the match image. The target image is rendered under a specific lighting direction and with a specific phase function at each trial. The match image is rendered with the same phase function as the target and under the “side” lighting direction (azimuthal angle = 0). The observer's task is to use the slider to adjust the translucent appearance of the “Lucy” on the right side (match image) so that it appears to be made of the same material as the “Lucy” on the left side (target image). Before the experiment, the observer confirms that changing from 0 to 10 in logarithmic density scale (left to right in Figure 4 on the sliding bar) makes the “Lucy” change from appearing transparent (“glassy”) to opaque (“marble”). During the trial, the observer can adjust the apparent translucency of the match “Lucy” through 40 values in its densities, in a range from 0 (log) to 10 (log) with a step of 0.25 (log). The observer has unlimited viewing time. After the observer finishes the adjustment, he or she is asked to press a radio button to indicate whether the match is satisfactory. Then the observer can move to the next trial in the sequence by pressing the “Next” button on the graphical user interface. 

Experimental conditions

Figure 5 shows target images of the “Lucy” rendered with 12 phase functions and three different lighting directions at density = 4(log). In the experiment, the density of the target “Lucy” can be one of the four values, 3, 4, 5, and 6 in the logarithmic density scales. 

The lighting directions of both target and match images are controlled by changing the polar and the azimuthal angles. To vary the lighting direction only in one dimension, at each trial, the polar angle is fixed at either 90° (horizon lighting) or 30° (overhead lighting). The azimuthal angle can be one of three conditions: front lighting (270°), side-back lighting (−30°), and back lighting (90°). The lighting conditions were chosen based on pilot studies. There are a total of 288 conditions (12 phase functions, six lighting conditions, and four densities). 

Figure 6 shows example images rendered with polar angle 90° and 30°. The match “Lucy” has the same polar angle as the target “Lucy.” But the lighting direction of the match is always at zero (side-lit). We chose the azimuthal angle to be zero to avoid having the same direction as one of the target lighting directions. The complete sets of target images and the parameter files used in the experiment are provided in the supplementary website.¹ 

Figure 6

 

“Lucy” with two different polar angle conditions, polar angle = 30°, and polar angle = 90°, for a particular phase function and density (Phase function = 7 and density = 4 (log), azimuthal angle = 90°.

Display

Observers perform the experiments in a dark room and the images are displayed on a 13-inch MacBook Pro laptop monitor (Apple, Inc., Cupertino, CA; dynamic range: 60:1; gamma: 2.2; maximum luminance: 340 cd/m²). The images are tone-mapped linearly with clipping, and gamma-corrected subsequently. We scale the values of an image by an exposure value. Then we convert the images to 8-bit integer scaled by 255. Values, which scale to >1.0 after exposure, will be mapped to an 8-bit value of 255, causing darkening and loss of contrast in the corresponding regions; for this reason, we chose exposures where this effect was minimal. Because images with polar angle 30° are significantly brighter than images with polar angle 90° (because much of the lighting is near or below the horizon or ground plane in the 90° condition), we use different exposure values (0.75 for polar = 30° and 0.85 for polar = 90°) for them so that the ground plane looks similar in brightness. 

Observers

Six naïve observers (four females and two males) with normal or corrected-to-normal visual acuity completed the task. The average age of the observers is 25 years. 

Results

The stimuli in our experiment vary along three dimensions: density, phase function, and lighting direction. We now present our results that answer the following three questions: (1) How well can observers discriminate material properties (defined by densities with fixed-phase functions) under different lighting conditions? (2) How does lighting direction affect translucent appearance? (3) How does the phase function interact with the lighting direction for translucent appearance? First, we look at material discrimination independent of lighting direction to see whether changing the density in the scattering parameters correlates with translucency perception. Second, we examine whether lighting direction affects translucency perception, for example, whether the stimuli rendered under front lighting look more opaque than the stimuli rendered under back lighting. Finally, we look at how the effect of lighting direction on translucent appearance depends on phase function shape. 

How well can observers discriminate translucent materials under different lighting conditions?

Figure 7A plots the mean data across observers for the 12 phase functions when the polar angle is set to be 90°. The x-axis represents the density of the target “Lucy,” varying from 1 to 10 in log scale (from high translucency to low translucency). The y-axis represents the mean density across the observers' matches. The diagonal line represents perfect matching. Since the matching experiment is asymmetric, meaning the match “Lucy” is always rendered with a different lighting direction (Side-lit, Azimuthal angle = 0°) than the target “Lucy” (Azimuthal angle = −30°, 90°, and 270°), it is not expected that the data would lie along the diagonal. If the data lie above the diagonal line, it means the perceived density is higher than the target. This is the case for most of the green line in the graphs (front lighting conditions). If the data lie below the diagonal line, it means the perceived density is lower than the target. This is the case for some of the red and blue lines in the graphs (side- and backlighting conditions, respectively). 

Figure 7

 

Results from the “Lucy” matching task of polar angle = 90°. (A) Mean subject matches plot against target densities for 12 phase functions (polar angle = 90°). The p values are computed using ANOVA with repeated measures, representing significance between the front lighting and the back lighting conditions. (B) Mapping the effect of lighting of each condition on the 2D embedding of phase functions. The purple circles represent phase functions for which lighting direction has no significant effect. The orange circles represent phase functions for which lighting direction has significant effect. (C) The histograms of the slopes of linear fit for each scatter plots shown in (A). The mean of the slopes is 1.01.

First, we ask whether changing the density, a physical parameter, alters the perception of translucent appearance. Figure 7A shows that for all conditions as the target density increases (this corresponds to less translucent appearance of the material), observers' matches increase in a monotonic and linear fashion (most data lies along a line, except a few cases for back-lit conditions). As the density increases, the “Lucy” is judged to appear less translucent. We confirmed this by fitting the data with linear functions and plot histograms of the slopes of the fitted lines (Figures 7C and 8C). We find the slopes are centered on 1.0, which suggests the monotonic relationship between the density parameter change and the changes in observers' matches. 

Figure 8

 

Results from the “Lucy” matching task polar angle = 30°. (A) Mean subject matches plotted against target densities for 12 phase functions. (B) Mapping the effect of lighting of each condition on the 2D embedding of phase functions. The symbols have the same meaning as those in Figure 7. (C) The histograms of the slopes of linear fit for each scatter plots shown in (A). The mean of the slope is 1.02.

How does lighting direction affect translucent appearance?

If lighting direction had no effect on perceived translucency, the colored lines representing matches for the three lighting conditions in Figure 7 would be overlapping. But Figure 7 shows that the green line is significantly above the red and the blue lines for some phase function conditions (9, 10, 11, 12, 7, 6, 5). This means that observers perceived the front-lit “Lucy” to appear less translucent than the side-lit and back-lit “Lucy,” which is consistent with previous findings for the torus stimuli (Fleming & Bülthoff, 2005). 

Also, the effect of lighting direction is not symmetric. The gap between the green lines and the diagonal lines is much bigger than the gap between the red or blue lines and the diagonal lines. This shows that the observers perceive the front-lit target stimuli as considerably less translucent than the match stimuli, but the back-lit stimuli only slightly more translucent than the match stimuli. Figure 7 also shows that lighting direction has little or no significant effect for phase function conditions 1, 2, 3, and 4. We used an ANOVA test with repeated measures (within observers) to examine the difference between front- and back-lighting conditions. The p value of the main effect (lighting direction) is displayed on the plot for each phase function condition. 

How does the phase function interact with the lighting direction for translucent appearance?

The data shows that lighting direction affects some phase function more than others. We now want to connect the results in the current experiment with the 2D embedding of phase functions we established in earlier work (Gkioulekas et al., 2013). We want to ask whether the effect of lighting direction on different phase functions can be predicted from their corresponding positions in the 2D embedding. For each phase function condition, we compute the p value with a within-observer ANOVA on the difference between the front- and the back-lighting conditions. 

In Figure 7B, the phase function that has a significant p value is marked with an orange dot on top of each panel and the phase function with nonsignificant p value is marked with a purple dot on each panel. We then label the corresponding test phase functions in the 2D embedding of phase functions. We can see in Figure 7B that the purple dots lie on the leftmost column of the phase function space (single-lobe Henyey-Greenstein or isotropic phase functions), whereas all the orange dots lie on the right-hand side of the phase function space (double lobe combining HG and von Mises-Fisher functions). This shows that the strength of the effect of the lighting direction on translucent materials can be predicted by the position of the corresponding phase function on the 2D space. The lighting direction has a strong effect on phase functions that have two lobes, with either HG + HG or HG + vMF phase functions (see figure 11 in Gkioulekas et al., 2013) for the locations of different types of phase function on the 2D embedding). In fact, the strength of lighting direction becomes stronger as we move from the left to the right on the 2D space. Note that phase functions in the right part of the embedding can yield relatively sharp (glassy) appearances, whereas the leftmost column phase functions yield diffusive appearances. 

Figure 8 shows the results from the conditions where the polar angle is set to be 30°. The data is plotted in the same fashion as in Figure 7 for the same group of observers. The results for polar angle 30° are qualitatively the same as for 90°, except that unlike before, the effect of lighting direction for phase function 8 is not significant (ANOVA test with repeated measure, p = 0.1). Note that the effect of lighting direction for this phase function is also relatively small in Figure 7. 

Observers' confidence

At the end of each trial during the experiment, the observer is asked to indicate whether they are “happy” with the match. This indicates observers' confidence on the matches. Figure 9 plots the percentage of trials in which all observers indicated they were “satisfied,” versus the total number of trials for each phase function condition. On average, the percentage is about 78%, suggesting observers are overall confident in their matches. The plot also shows that observers have slightly higher confidence for the conditions that have no significant effects of lighting direction on the matches than the conditions that have significant effects of lighting direction (mean confidence rating of the purple conditions is 0.84; mean confidence rating of the brown-yellow conditions is 0.73). 

Figure 9

 

Confidence ratings averaged across all observers. The bars represent the percentage of trials aggregated across all observers deemed “satisfactory” in their matches for each phase function condition of both azimuthally angles. The vertical bar represents each phase function with the same number scheme as in Figures 7 and 8. The purple colors represent phase functions that have no significant lighting effects while brown colors represent phase functions that have significant lighting effects.

Discussion

In this paper we explore the interaction of shape, illumination, and level of translucency. More important, we also consider the type of translucency, as specified by the scattering phase function. We use appearance matching to assess the degree of perceived translucency. We find that the degree of translucency constancy depends strongly on the phase function's location in the 2D perceptual space discovered in Gkioulekas et al., 2013, suggesting that the space captures genuinely useful information about different types of translucency. 

Translucency discrimination and lighting direction

From a material design point of view, it is interesting to ask whether lighting direction affects material discrimination. Our data show that observers can discriminate translucent appearance simulated with different densities under all lighting directions: All the lines in Figures 7 and 8 have similar slopes to the diagonal, indicating that the physical change in density can predict perceptual discrimination, except for a few cases in front-lighting conditions (Figures 7C and 8C). Fleming and Bülthoff (2005) show that when the torus is illuminated from the front, the image does not change very much as a function of the degree of translucency. Stated another way, the image information is not sufficiently discriminative using the image information alone; there is little basis for a visual system to distinguish between translucent and opaque objects. It is possible that this problem is idiosyncratic of the particular stimuli Fleming chose. Our data shows that even for the front-lit “Lucy,” the effect of density on translucent appearance is strong and mostly monotonically related to density changes (Figures 7 and 8, green line). It is possible that the “Lucy” has more features than the torus, providing enough image cues for the observers to distinguish between objects with different degrees of translucency in front-lit conditions. 

Which image regions are important?

Since the “Lucy” has a complex shape, which parts of the image are the most important for perception? Figure 10 compares the “Lucy” rendered with the average matched value from all observers and the target “Lucy.” The two stimuli are perceptually equivalent in terms of translucency based on the data. The top row in Figure 10 shows that on average observers matched a front-lit “Lucy” with a much lower density to a side-lit “Lucy” with much higher density (almost double the density). This confirms the results of Figure 7 that front lighting makes the object appear less translucent than the side and backlighting. The bottom row in Figure 10 shows the average observers' match of the object under the backlighting condition. The two “Lucy” images look very different in image contrast, but the “thin” parts of the object, such as the wings and hands of the “Lucy,” appear to be almost transparent in both of the images. This suggests that the thin geometric parts such as the edges contribute most to the overall translucent appearance of the “Lucy.” 

Figure 10

 

Examples of stimuli that subjects match to be perceptually equivalent in translucency. Top row: a front-lit target “Lucy” and the matched side-lit “Lucy” rendered with the averaged density selected by the observers. Bottom row: a back-lit target “Lucy” and the matched side-lit “Lucy” rendered with the averaged density set by the observer.

This observation is consistent with a recent study by Nagai et al. (2013), which shows that certain regions contribute more strongly to the translucent appearance of rendered images than other regions. Most of the regions extracted from the psychophysical reverse correlation method used by the authors are thin parts near the objects' edges or delicate structures, and regions that have high contrast shading patterns (see figure 7 in Nagai et al., 2013). Fleming and Bülthoff (2005) also show similar results. 

Comparison with the findings in Fleming and Bülthoff (2005)

The phase function in the top left corner of the 2D space (phase function 1) exhibits little effect of lighting direction in translucent appearance in the current study. However, Fleming and Bülthoff (2005) showed strong lighting effects on translucent appearance. Here, we speculate that the main causes of the difference in the results could be the different geometry of the stimuli and the rendering methods. Lucy provides far more density cues than the torus does especially for front-lit conditions. Also, Fleming and Bülthoff (2005) use Jensen's BSSRDF approximation, where we used a more accurate method in our study. Jensen uses similarity theory, which potentially generates errors from approximated refraction in single-scatter for side-lit conditions, which is what Fleming and Bülthoff (2005) used for the target images. The difference in the methods generates different translucent appearance. In addition, Fleming and Bülthoff (2005) use a point-light source and there is a strong cast shadow on the torus, which affects translucent appearance. We use area lights and there is not much shadow. Figure 11 demonstrates the effects of 3D shape and lighting on appearances. 

Figure 11

 

Both shape and lighting affects how phase function interacts with lighting direction in translucency perception. Comparison of the effect of lighting direction on object with thin geometric features “Lucy” and “Torus.” Top row: “Lucy” rendered with the same area light source as used in the experiment. Middle row: “Torus” (with similar setting as Fleming & Bulthoff, 2005) rendered with a point light source. Bottom row: “Torus” rendered with the same area light as “Lucy” in the top row. All the objects have the same scattering parameters. The chrome sphere included in the image shows the reflected image of the lighting environment.

Importance of the complexity of the lighting environment

Most previous studies used a simple lighting environment such as a point light source (Fleming & Bülthoff, 2005; Motoyoshi, 2010; Nagai et al., 2013). The bottom row in Figure 11 shows the same “torus” as in the middle row in Figure 11, but illuminated with the area lights we use in our “Lucy” experiment (generated by spherical harmonics up to the fourth order). We observe that the effect of lighting on translucent appearance is less dramatic in the images shown in the bottom row than those in the middle row in Figure 11. The back-lit “torus” appears less translucent in the area lights than the backlit torus under a point light source. This observation indicates that the complexity of the lighting environment plays a role in how lighting affects translucency. It is possible that more realistic lighting with a combination of both low and high (infinite) frequencies in the spherical harmonics could result in less dramatic effects of lighting direction on translucent appearance. 

Implications for material constancy

Translucent objects interact with light in a complex way. In this work, we find that for some translucent materials, lighting direction has a significant effect on appearance. Thus, for these conditions, the visual system cannot completely discount the lighting direction in material perception. On the other hand, there are also conditions where observers can indeed discount lighting direction changes. However, previous work has found humans are good at material discrimination and recognition (Sharan et al., 2009). Even for translucent objects, in order to have a stable percept of material properties, the visual system has to, at least to an extent, discount the changes caused by rotating the light source. To draw an analogy with studies of human color constancy, constancy is good in complex scenes where there are enough cues about the illuminant, such as nearby surfaces and specular highlights on shinny surfaces. But color constancy is poor when the scene context is reduced (Kraft, Maloney, & Brainard, 2002). For translucency, we hypothesize that complex lighting and shape make it possible for observers to discount lighting for simple materials (isotropic phase functions). But observers have a hard time discounting the effects of lighting when the object has both the complex 3D shape and is made of complex materials (for example, having a double-lobe phase functions with two types of distributions [e.g., vMF + HG]). However, this speculation of the relationship between complexity of shape and of phase function in translucency constancy needs more experimental support. In addition, most of the cues to the illuminant in our scenes are from the object itself. Though we believe there are sufficient cues to the illumination direction in the image (e.g., contrast between the wings and the body of the Lucy), the absence of cues to the illuminant could play a role in poor constancy. Examining how adding additional cues to the illuminant would affect the results is an interesting future direction. 

Conclusion and future work

In conclusion, using a natural scene with a complex 3D shape, we find that lighting direction has a strong effect on translucent appearance simulated with some phase functions but not others. This effect can be predicted by the position of the phase functions in a 2D perceptual embedding established in a previous publication (Gkioulekas et al., 2013). The phase functions that are associated with the strong lighting direction effect can simulate materials that appear glassy with sharp details. On the other hand, the phase functions that are associated with weak lighting direction effect can simulate materials with a diffusive appearance. Our results suggest that more consideration should be given to the richness of material properties, such as rich sets of phase function, when examining the effects of lighting direction on translucency perception. 

We also find that the geometry of an object is important. We compare the case of a torus, which has a simple smooth shape, with that of a figurine, which has more complex geometry with thick and thin sections and features at multiple scales. The complex shape shows a greater range of apparent translucencies, which allows observers to discriminate different degrees of translucency. But it also resulted in a higher degree of constancy failure. Although the complex figure offers more cues to drive translucent appearance, those cues do not necessarily increase the constancy. 

The focus of this paper has been on the phase function, but in order to fully understand how lighting affects translucency perception and constancy, we need to explore other dimensions of scattering parameters, such as scattering coefficient, albedo, and their spectral variations. Also, in this work, we used synthetic images. In the real world, there is higher dynamic range and observers have stereoscopic cues, which might also help with translucency constancy. In addition, we only take an initial step towards understanding the role of 3D shape in translucency perception. Future experiments should systematically study how varying 3D shape affects translucency perception using a rich set of material parameters. From a computational view, we would like to build models that quantify scene complexities and predict human translucency discrimination and constancy based on scene parameters such as shape, material, and lighting geometry. Such models would be useful in material design applications, guiding what scenes one should use to exhibit certain translucent appearance effects, and in user interfaces for rendering translucent materials. 

Acknowledgments

We want to thank Cathy Tingle for the photographs of the cherubs in Figure 1 and other photographs that inspired the experiment in this work. We wish to thank Shuang Zhao and Asher Dunn for their support of rendering and useful discussions, and Professor Roland Fleming for sharing with us his rendering parameters and very useful discussions. We also want to thank the Stanford 3D Scanning Repository for providing the Lucy. This research is supported by NSF awards 116173 (K. B., T. Z., E. A.) and 1161645 (K. B.) and Amazon Web Services in Education grant awards to I. G. and T. Z. 

Commercial relationships: none. 

Corresponding author: Bei Xiao. 

Email: [e-mail]. 

Address: Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. 

Footnotes

¹  Supplementary website: http://people.csail.mit.edu/beixiao/Supplementary_BX/.

Appendix

A. Phase function models and the parameters

The phase functions were defined mathematically either by a Henyey-Greenstein function (HG), or a von Mises Fisher function (vMF), or a combination of both. Henyey and Greenstein (1941) describe the phase function as the following:  

 

In Gkioulekas et al. (2013), we expanded the phase function by adding a lobe that is shaped according to the von Mises-Fisher distribution on the sphere of directions (Fisher, 1953). Its probability density function is given as:  

 

θ ∈ [0, π], and has a single shape parameter κ 

κ = 2C̄/(1 − M_C), where C̄ is the average cosine  

 

 

 

 

We represent the phase function as linear mixtures of the two distributions. We selected g from {0, ±0.3, ±0.5, ±0.8, ±0.9, ±0.95}; for the vMF lobes we found through experimentation that κ > 100 produced unrealistic results, and selected κ ∈ {± 1,±5, ±10, ±25, ±75, ±100}. We also chose values of the mixing weight of the first lobe from {0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99}, with the corresponding weight on the second lobe being one minus this value. Supplementary Table 1 shows the parameters of the 12 phase functions used in this study. 

Supplementary Table 1

 

Parameters of the 12 phase functions used in the study. Each phase function can have two lobes, the forward lobe and the backward lobe. For each lobe, it can be either defined by HG or vMF functions. The weight (w) is the weight of the linear sum of the two lobes that is always applied to the first lobe (which is the forward lobe).

Phase function	Forward lobe		Backward lobe		Weight
	κ	g	κ	g	w
1	1			−0.5	0.8
2	25			0	0.5
3		0.8	−5		0.9
4		0.8	−5		0.99
5		0.8		−0.9	0.6
6	10		−75		0.7
7	25		−25		0.8
8	25		−0.95		0.9
9	100		−0.95		0.6
10		0.95	−100		0.7
11	75		−100		0.8
12	100		−75		0.9

B. 2D embedding of phase function and the stimuli used in this study

Supplementary Figure 1 shows the computational embedding of the phase functions discovered in Gkioulekas et al. (2013) and the stimuli images used in this paper arranged as in the embedding. We see that moving from top to bottom results in more diffusion, an effect similar to that achieved by increasing the density of the scattering process. Moving from left to right results in greater surface detail and more “glassy” appearance. By selecting different points in the embedding, we can achieve different trade-offs between diffusion and sharpness. 

Figure A1

 

HG (shows in blue) and vMF (shows in red) phase functions for increasing value of average cosine C̄, shown as increase in saturation. vMF lobes are wider and not as strongly forward-scattering as HG lobe. This figure is copied from figure 3 in Gkioulekas et al. (2013).

Figure A2

 

Two-dimensional computational embedding and the experimental images. (A) Two-dimensional embedding of images rendered with phase functions sampled from the physical parameter space produced in cubic root metric. Gray dots represent the 753 rendered images (for details, see Gkioulekas et al., 2013). The numbers label the stimuli used. BA subset of the stimulus images, arranged in a grid according to their positions in the computational embedding. The numbers are in correspondence with those shown in (A). The “Lucys” were rendered side-lit and have the same parameters except for the phase functions [azimuthal angle = 0°. polar angle = 90° and density = 4 (log)].