Motion and texture shape cues modulate perceived material properties

Abstract

Specular and matte surfaces can project identical images if the surface geometry and light field are appropriately configured. Our previous work has shown that the visual system can exploit stereopsis and contour cues to 3D shape to disambiguate different surface reflectance interpretations. Here, we test whether material perception depends on information about surface geometry provided by structure from motion and shape from texture. Different surface textures were superimposed on a fixed pattern of luminance gradients to generate two different 3D shape interpretations. Each shape interpretation of the luminance gradients promoted a different experience of surface reflectance and illumination direction, which varied from a specular surface in frontal illumination to a comparatively matte surface in grazing illumination. The shape that appeared most specular exhibited the steepest derivatives of luminance with respect to surface orientation, consistent with physical differences between specular and diffuse reflectance. The effect of apparent shape on perceived reflectance occurred for a variety of surface textures that provided either structure from motion, shape from texture, or both optical sources of shape information. In conjunction with previous findings (Marlow, Todorović, & Anderson, 2015; Marlow & Anderson, 2015), these results suggest that any cue that provides sufficient information about 3D shape can also be used to derive material properties from the rate that luminance varies as a function of surface curvature.

Introduction

The computation of surface reflectance is theoretically challenging because there is no simple relation between the reflectance function of surfaces and the structure of the light that reaches the eyes. Surfaces with identical reflectance functions typically project different images for different surface geometries and light fields. However, surfaces with different reflectance functions can project identical images if the surface geometry and light field are appropriately configured. The difficulty of computing reflectance has inspired a large body of psychophysical work that examines how well human observers can judge reflectance properties. A recurring theme in this literature is that surface geometry and reflectance can affect the perception of each other (e.g., Ho, Landy, & Maloney, 2008; Todd & Mingolla, 1983). Physical differences in shape can induce apparent differences in specular reflectance (Marlow & Anderson, 2013; Marlow, Kim, & Anderson, 2012; Nishida & Shinya, 1998; Olkkonen & Brainard, 2011; Qi, Chantler, Siebert, & Dong, 2014; Vangorp, Laurijssen, & Dutré, 2007; Wendt, Faul, Ekroll, & Mausfeld, 2010; Wijntjes & Pont, 2010). Likewise, a specular surface can appear to have a different 3D structure compared to the same surface geometry rendered with a lower specular reflectance (Doerschner, Yilmaz, Kucukoglu, & Fleming, 2013; Ho et al., 2008; Mooney & Anderson, 2014; Todd & Mingolla, 1983). We have previously shown that these perceptual interactions correlate highly with image properties generated by specular luminance gradients (Marlow & Anderson, 2013; Marlow et al., 2012; Mooney & Anderson, 2014; Qi et al., 2014). Surfaces appear to increase in specular reflectance when there is an increase in the contrast, sharpness, or abundance of specular image structure, regardless of whether it is the surface geometry, reflectance function, or light field that actually generates that structure. These studies suggest that perceived specularity is driven by variations in image cues that are imperfectly corrected with the true specularity of a surface. 

These results suggested that the perception of gloss is driven by differences in image structure. However, we recently presented evidence that the visual system incorporates 3D shape constraints into the computation of reflectance in stimuli that contained identical luminance gradients (Marlow & Anderson, 2015; Marlow, Todorović, & Anderson, 2015). Specifically, information about surface orientation (and its derivatives) constrains the space of illumination directions and reflectance functions consistent with a given luminance gradient. Variations in surface orientation affect the diffuse and specular components of reflectance differently. Diffuse reflectance is brightest where the surface normal is directed at the primary light source, whereas specular reflectance is brightest where the surface normal bisects the observer's viewing direction and the illumination direction. Luminance decreases as surface orientation varies relative to the luminance maximum, and the rate of this decrease can theoretically provide information about surface reflectance. Lambertian reflectance decreases as the cosine of the angle between the surface normal and the illumination direction (e.g., Figure 1A, left). Specular reflectance decreases significantly steeper than this cosine fall off even if the specular reflectance lobe “scatters” or “spreads” significant amounts of light, such as occurs with roughened metal (e.g., Figure 1A, right). Thus, the visual system could theoretically compute reflectance from derivatives of luminance and surface orientation: A “matte” reflectance interpretation is plausible if luminance varies as cosine function of surface orientation (or more slowly), whereas a specular reflectance interpretation is more plausible if luminance varies more rapidly than a cosine function of surface orientation. If the visual system exploits these physical constraints to estimate the reflectance properties of surfaces, then it should be possible to generate identical image gradients that elicit different material properties if they are associated with appropriately chosen 3D shapes. 

Figure 1

 

(A) If the surface geometry is known, then the luminance of a surface can be represented as a function of 3D surface orientation. Material properties are derivable from the rate that luminance varies with 3D surface orientation, which is rapid for specular surfaces, and comparatively gradual for matte surfaces. (B) Identical luminance gratings can be perceived as either a matte or specular surface based on contour cues to surface geometry. The goal of the present study is to test whether motion parallax and texture gradient cues to shape can also influence perceived reflectance (from Marlow & Anderson, 2015).

Our previous work that assessed the influence of perceived shape on perceived reflectance relied on bounding contours and binocular disparity to modulate perceived shape (Marlow & Anderson, 2015; Marlow et al., 2015). The two surfaces in Figure 1B have identical luminance gratings cropped by different bounding contours along the left and right sides of the grating. The 3D shape appears to curve outwards towards the observer where the bounding contour is convex, and appears to curve inwards away from the observer where the bounding contour is concave (Koenderink, 1984; Todorović, 2014). The identical luminance gratings also elicit different experiences of illumination direction that depend on the apparent surface orientation of the luminance maxima (Ramachandran, 1988; Todorović, 2014). The light appears to come from above when the luminance maxima is associated with a (perceived) surface normal that is directed upward (left surface), whereas the light appears to come from along the line of sight when the surface normal of the brightest regions is directed towards the viewer (right surface). We showed that the perceived reflectance of the surfaces varies with the range of surface orientations in the neighborhood of the luminance maxima: The surface appears specular or metallic when surface orientation varies through a small range (right surface), whereas it appears comparatively matte when surface orientation varies through a large range (left surface). Subsequent work found the same correlation between perceived shape and reflectance when the geometry of the surface was specified by binocular disparity instead of bounding contours (Marlow & Anderson, 2015). These results provided strong evidence that the visual system can derive surface reflectance from the 3D shape information provided by bounding contours and stereopsis. 

The preceding suggests that it should be possible to modulate the perception of reflectance with any source of 3D shape information, such as motion parallax and texture gradients. Many surfaces exhibit spatial variations in albedo, and this surface texture can provide information about 3D shape in both moving and static displays (e.g., Gibson 1950; Wallach & O'Connell, 1953). Although it is theoretically possible to derive accurate judgments of local surface orientation across smoothly-curved surfaces from either motion (Ullman, 1979) or texture gradients (Gårding, 1992), human observers make significant errors when judging surface orientation and curvature (Norman & Todd, 1996; Norman, Todd, Norman, Clayton, & McBride, 2006; Norman, Todd, & Phillips, 1995; Perotti, Todd, Lappin, & Phillips, 1998; Reichel, Todd, & Yilmaz, 1995; Todd & Mingolla, 1983). The magnitude of these errors is similar for surfaces specified by binocular disparity and motion parallax (Norman et al., 1995; Norman et al., 2006), which suggests that motion parallax could potentially influence perceived reflectance with the same strength observed with our stereoscopic displays (Marlow & Anderson, 2015). Larger errors have been found for shape from texture than for structure from motion and stereopsis (Norman et al., 1995; Norman et al., 2006), so it is unclear whether the shape information provided by texture gradients is sufficiently rich to modulate perceived reflectance. 

Previous studies of specular motion have focused on the way specular reflections move relative to the underlying surface geometry (Doerschner et al., 2011; Doerschner, Kersten, & Schrater, 2009; Koenderink & van Doorn, 1980; Oren & Nayar, 1996; Sakano & Ando, 2010; Wendt et al., 2010). This motion can be used to distinguish specular image structure from optical texture that moves with the surface (Doerschner et al., 2011). Although several studies have found that motion improves the accuracy of specular reflectance judgments (Sakano & Ando, 2010; Wendt et al., 2010), it is unclear whether these effects arise because of the characteristic motion of specular reflections or because of the 3D shape information that motion parallax provides. 

The experiments reported as follows were designed to assess whether perceived reflectance depends on structure from motion and shape from texture. The experiments exploit motion and optical texture to induce different 3D shape percepts associated with otherwise identical luminance gradients. To anticipate the results that are reported herein, we find that both motion parallax and texture gradients can provide sufficiently rich information about 3D shape to modulate the perceived reflectance properties of surfaces. 

Experiment 1

Experiment 1 used both texture and motion to modulate perceived 3D shape. Two example surfaces are shown in Movie 1. The surfaces have identical luminance gradients; the only difference between the images is the optical texture superimposed on the luminance gradient. The size, density, aspect ratio, and motion of the texture vary systematically with the simulated 3D shape of the surfaces. The depicted 3D shape spreads across the entire luminance gradient; the luminance gradient on the upper surface appears to be generated by a convex cylinder, whereas the same luminance gradient on the lower surface appears concave above the luminance maxima and convex below it. The 3D shape of the luminance gradient constrains the material and illumination direction of the surfaces. Specifically, the upper surface is consistent with a specular surface illuminated from the front because the highlight (luminance maximum) lies on a small range of surface orientations that face the observer. The lower surface is consistent with a comparatively matte surface illuminated from above because the highlight spans a larger range of surface orientations that face upward. The goal of Experiment 1 was to determine whether these physical constraints impose different percepts of material properties. 

We varied the luminance gradient across the surfaces and the horizontal curvature of the surface geometries. The steepness of the highlight (luminance maximum) of the luminance gradient was parametrically varied to find the optimal luminance gradient eliciting the largest effect of 3D shape on perceived material. The two vertically-curved surfaces shown in Movie 1 were bent horizontally to form the multiply-curved surfaces shown in Movie 2. The purpose of including singly-curved and multiply-curved surfaces was to test whether the results generalize to both types of surface geometry. 

 

Movie 1.

 

Identical luminance gradients with different 3D shapes specified by motion parallax and texture gradients. The 3D shape suggests that the luminance gradient is either a shiny convex cylinder illuminated from the front (first half of movie) or an S-shaped matte surface illuminated from above (second half).

Methods

Observers

Eight observers participated in the experiment. They were recruited from the undergraduate psychology pool of the school of psychology and received a small amount of course credit for participating. They were debriefed about our aims at the end of the experiment in adherence with the Declaration of Helsinki. 

Display

The display screen was a 30-inch flat screen LCD monitor (DELL U3014t) and was calibrated using a SpectraScan 650 photometer (Photoresearch, Inc., Chatsworth, CA). The viewing distance was 70 cm. The Psychophysics Toolbox for Matlab (Brainard, 1997; Pelli, 1997) displayed the images and movies used in the experiment with a high temporal and spatial resolution (60 Hz; 750 × 1,000 pixels for each surface). 

Stimuli

The stimuli were constructed by superimposing images of two different surfaces: a planar surface with a smooth luminance gradient; and a curved surface with a sparse optical texture. The planar luminance gradient appeared to have the same, curved 3D shape as the optical texture that locally covered the luminance gradient. Identical luminance gradients were made to appear as different 3D shapes by manipulating the curved 3D shape generating the optical texture. 

The planar surface generating the luminance gradient was located in the plane of the display screen. The range of its horizontal X coordinates was ±5.75 cm. The range of its vertical Y coordinates depends on X and was given by  

 

where h is equal to 0 cm for the singly curved surfaces shown in the left column of Figure 2, and is equal 1 cm and 2 cm for the multiply curved surfaces in the middle and right column. The surface revolved around its vertical axis oscillating between left-side nearer and right-side nearer, and the rotation angle spanned ±7.5° varying at a constant rate of 19.5°/s. The luminance of the planar surface was modulated in the vertical direction with the peak of the gradient located in the middle of the surface. Specifically, the luminance of the plane was given by  

 

 

 

where M is the maximum luminance of the display screen (153 cd/m²); δ is the vertical separation from the upper edge of the planar surface (the range of δ was 0  < δ  <  7.5 cm); and S increases the steepness of the peak of the luminance profile. Six levels of steepness were tested; S = 2.4, 7, 12, 16, 21, and 26.  

Figure 2

 

Results of Experiment 1. The perceived specular reflectance of moving, textured surfaces plotted as a function of the steepness of the luminance profile. Vertically separated data points refer to different 3D shapes with identical luminance profiles. The three surface geometry pairs tested are shown below the results graphs, and example stimuli are shown in Movies 1 and 2. Horizontal curvature increases from the left graph to the right graph. Error bars show means and standard errors of all observers.

The curved surfaces generating the optical texture were created by combining two vertical curvatures with three horizontal curvatures (six distinct surface geometries). The vertical curvature had either a convex “C-shape” cross section (top row Figure 2) or an “S-shape” cross section that was concave for the top-half and convex for the bottom-half (bottom row Figure 2). The vertical-cross sections were shifted along a horizontal line to create the singly-curved surfaces, and were shifted along a sinusoidal path to create the doubly-curved surfaces. The amplitude of the sinusoid differed for the doubly-curved lower and doubly-curved higher conditions. The surface geometries were given by:  

 

where α is the diameter of the ellipses in the vertical cross-section and was equal to 7.5 cm for the C-shape cross section and 3.75 cm for the S-shape cross section; δ (already defined above) is vertical separation from the top edge of the surface, varying from 0  < δ  <  7.5 cm, and depends on the magnitude (h) of the sinusoidal modulation of the surface's X and Y coordinates. The surfaces were covered by a sparse distribution of solid black circles glued tangent to the surface. The diameter of each circle was 4 mm and the density of the texture was 1 circle per 2.36 cm². The distribution of circles was stochastically regular across the surface and was based on the probability of a black circle occurring at each image coordinate given the size and density of the surface texture and the distance and orientation of the 3D surface (see Todd & Mingolla, 1984). The center of both surface geometries and the peak of the luminance gradient projected to the same location in the image. Specifically, the planar surface generating the luminance gradient was tangent to the midline of the “C-shape” surface, and intercepted the inflexion point in the “S-shape” surface. The textured surfaces rotated in synchrony with the planar surface generating the luminance gradient. Images of the optical texture and the luminance gradient were derived using perspective projection with the image plane set to match the distance of the display screen.  

Procedure

We used a paired comparison method wherein observers viewed two surfaces on each trial and were instructed to indicate which surface appeared more specular. The surfaces were presented side-by-side separated by a horizontal gap of 3.5°. The three levels of horizontal curvature were not directly compared and were tested in different blocks of trials. In each block, there were 12 surfaces in total (six levels of image gradient steepness by two types of 3D shape, either “S” or “C” shaped). Each surface was compared once with the other 11 surfaces generating 66 unique trials. The order of the trials and blocks was a different random permutation for each observer. The intertrial delay was 1.5 s. There were no constraints on the length of a trial, and observers typically completed the three blocks in 20 to 30 min. 

Statistics

A multivariate analysis of variance was used to test for main effects of 3D shape and the steepness of the luminance gradient. The Decision-Wise error rate (α = 0.05) was controlled for each of these planned, within-subject contrasts (Bird, 2004). Separate statistical tests were performed for the singly-curved, doubly-curved lower, and doubly-curved higher conditions. 

Results and discussion

Figure 2 plots the proportion of trials that each surface appeared more specular than the other surfaces tested. Perceived specularity is plotted as a function of the steepness of the image gradients, and the white and black data points correspond to the two possible 3D shapes of the luminance gradient. The horizontal surface curvature is zero in the left graph, moderate in the middle graph, and highest in the right graph. 

The steepness of the image gradients had a nonmonotonic effect on perceived specularity. There was an overall increase in perceived specularity with the steepness of the image gradients, F(1, 7) = 54, p < 0.001, left graph; F(1, 7) = 129, p < 0.001, middle graph; and F(1, 7) = 115, p < 0.001, right graph. This increase is consistent with previous work showing that sharper specular reflections induce stronger percepts of surface gloss (Hunter & Harold, 1987; Kim, Marlow, & Anderson, 2012; Marlow & Anderson, 2013; Marlow et al., 2012; Pellacini, Ferwerda, & Greenberg, 2000). However, luminance gradients steeper than S = 20 slightly reduced perceived specularity. Similar nonmonotonicity for extremely steep gradients has also been found in other variants of these stimuli where the 3D shape is conveyed by bounding contours (Marlow et al., 2015) or binocular disparity (Marlow & Anderson, 2015). One plausible explanation is that the highest levels of steepness appear less realistic because essentially all of the luminance variation occurs in a narrow band around the highlight. 

The 3D shape of the surfaces had a significant effect on perceived specularity, F(1, 7) = 18, p = 0.004 for the left graph; F(1, 7) = 75, p < 0.001 for the middle graph; and F(1, 7) = 58, p < 0.001 for the right graph). The 3D shape that had low surface curvature across the highlight appeared more specular than the 3D shape that had comparatively high surface curvature. This effect of shape on perceived specularity occurs for a wide range of luminance gradients, and reveals that identical luminance gradients that appear as different 3D shapes can appear to be made of different materials. The results provide strong evidence that the visual system can exploit the shape information provided by the combined effects of texture gradients and motion parallax to derive surface reflectance properties. 

The 3D shape manipulations used in Experiment 1 were generated by the combined effects of motion parallax and texture gradients. One obvious question is whether the same effects can be observed if each source of depth information was presented separately. We therefore created variants of the Experiment 1 stimuli that contained only motion parallax or texture gradients. For the texture-only variants, observers viewed one static frame of the Experiment 1 stimuli. For the motion-only variants, the motion of the texture elements was correlated with the 3D shape of the surfaces but the size, density, and aspect ratio of the texture elements was consistent with a 2D planar surface (see Movie 3). Independent groups of observers judged the perceived specularity of the motion-only and texture-only variants (Figure 3), and there was no significant effect of 3D shape for either variant for any level of horizontal curvature [motion-only: F(1, 6) = 1, p = 0.35 for h = 0; F(1, 6) = 0.036, p = 0.86 for h = 1; and F(1, 7) = 0.24, p = 0.64 for h = 2; texture-only: F(1, 8) = 0.7, p = 0.43 for h = 0; F(1, 10) = 0.03, p = 0.42 for h = 1; and F(1, 11) = 0.76, p = 0.40 for h = 2]. The results suggest that the significant differences in perceived material that were observed in Experiment 1 required the combined information about shape provided by motion parallax and texture gradients. 

Figure 3

 

The effect of 3D shape on perceived specularity was eliminated when the 3D shape of the Experiment 1 surfaces was specified by only source: either optical texture (top row); or motion parallax (bottom row). Error bars show means and standard errors of all observers.

Experiment 2

The luminance profile used in Experiment 1 may have contributed to the ineffectiveness of the motion-only and texture-only variants. The luminance profile was not fully consistent with diffuse shading of the doubly-curved S-shaped surface geometries. Diffuse shading would vary from left to right across these surfaces due to their horizontal curvature, whereas the luminance profile that was texture-mapped onto the surfaces had constant brightness in this direction. In Experiment 2, the luminance profile texture-mapped onto the two surfaces varies with both directions of surface curvature. Example stimuli from Experiment 2 are shown in Figure 4. The upper and lower stimuli have identical luminance profiles, and only differ in the pattern of optical texture superimposed on the luminance profile. The goal of Experiment 2 was to test whether the 3D shape depicted by the optical texture can make the luminance profile appear as either a matte surface illuminated from above (bottom panel) or a specular surface illuminated from the front (upper panel). 

Figure 4

 

Two example surfaces from Experiment 2. Identical luminance profiles superimposed on optical textures consistent with different 3D shapes.

Methods

Observers

Forty-seven observers participated in Experiment 2. They were recruited from the undergraduate psychology pool of the school of psychology and received a small amount of course credit for participating. They were debriefed about our aims at the end of the experiment in adherence with the Declaration of Helsinki. We tested a large number of observers because texture gradients provide relatively weak 3D shape information compared to binocular disparity and motion parallax (Norman et al., 1995; Norman et al., 2006). 

Stimuli

The two 3D shapes used in Experiment 2 and shown in Figure 4 were multiply curved surfaces that curve most rapidly in the vertical direction. A vertical cross-section taken through the middle of the two surfaces is shown in the legend of Figure 5. The upper surface has a convex semicircular cross-section resembling a “C-shape,” whereas the lower surface is concave above the highlight and is convex below the highlight. The concave and convex semicircular components of the cross-section join in the center of the lower surface, forming an inflexion point resembling an “S-shape.” The surfaces were created by translating these cross-sections along a transverse sinusoidal path that began at the left side of the screen and ended at the right side. The orientation and scale of the cross-sections also varied along the sinusoidal path. The orientation of the cross-sections was perpendicular to the sinusoidal path, and the scale of the cross-sections increased towards the middle of the surface. The 3D coordinates of the surfaces (X, Y, Z) are given by,  

 

 

 

 

 

 

 

 

 

where u and v vary from −19.2 cm ≤ u ≤ 19.2 cm and 0 ≤ v ≤ 1. V and u are the vertical and horizontal screen coordinates of the transverse sinusoid through the middle of the surfaces. The constant k = 0.0833 sets the wavelength of the sinusoid to 24 cm. θ represents the direction perpendicular to the tangent plane of the sinusoid. α determines the radius of the semi-circular cross-sections that lie in the direction of θ, and was equal to 0.25 cm for the S-shaped cross-section and 0.5 cm for the C-shaped cross-section. Because the cross-sections are scaled by the height of the sinusoid (V), the radius of the semicircles that compose the cross-sections varies across the surfaces and ranges from 2 to 4 cm for the C-shape surface and 1 to 2 cm for the S-shape cross-section. The optical texture of the two 3D shapes simulated a stochastically regular distribution of black circles glued tangent to the surface. The radius of the circles was 0.1 cm and the density was 1 circle per 0.7 cm² of surface area. Images of the optical texture were derived using orthographic projection and both 3D shapes projected identical bounding contours.  

Figure 5

 

The perceived specularity of the Experiment 2 surfaces (see Figure 4) plotted as a function of the steepness of the luminance profile. Vertically separated data points refer to different 3D shapes with identical luminance profiles. Error bars refer to standard errors of the mean of all observers.

The optical texture of the two different 3D shapes locally covered an image of a smoothly varying luminance profile, which was identical for both 3D shapes. The luminance profile was constructed by texture-mapping an image of the surface normals of the S-shaped surface geometry. The highest luminance was assigned to the surface normal directed toward the zenith, and luminance declined monotonically with the angular separation of surface normals from the zenith. Specifically, the luminance profile was given by  

 

where N is the vertical component of the unit surface normal of the S-shaped surface geometry; M represents the maximum luminance of the display screen (153 cd/m²); and S increases the steepness of the peaks of the luminance profile. Six levels of steepness were tested; S = 1, 1.9, 2.8, 3.7, 5.6, and 7.4.  

Procedure

Observers performed a paired comparison task, where they viewed two surfaces on each trial vertically separated by 9.5 cm, and selected the surface that appeared more specular. There were 12 surfaces in total created by factorially combining the two possible 3D shapes with the six different levels of steepness of the luminance profile. Each surface was compared once with the other 11 surfaces in a block of 66 trials, and the order of the trials was a different random permutation for each observer. 

Results and discussion

The results show that texture gradients can modulate perceived specular reflectance even when all other cues to 3D shape are held fixed. Figure 5 plots the proportion of trials that each surface appeared more specular than the other surfaces tested. Perceived specularity is plotted as a function of the steepness of the highlight (luminance maximum), and the white and black data points correspond to the two possible 3D shapes of the luminance profile. There was a significant increase in perceived specularity with the steepness of the luminance profile, F(1, 46) = 17, p < 0.001, and a significant effect of 3D shape, F(1, 46) = 85, p < 0.001. The luminance profile again appeared more specular for the 3D shape with lower surface curvature than the 3D shape with comparatively high surface curvature. 

Experiment 3

The surfaces in Experiment 1 did not simulate the characteristic motion of specular reflections and diffuse shading across moving surfaces. The luminance maximum (highlight) was glued onto the rotating surfaces, which may have contributed to the ineffectiveness of the motion-only variants in Experiment 1. In Experiment 3, the two 3D shapes shown in Movie 4 were texture-mapped with identical, dynamic luminance profiles that depict the luminance maxima (highlight) to slide across the rotating surfaces. The 3D shape shown in the lower panel may appear as a matte surface illuminated from above because throughout the motion sequence the luminance maxima is associated with a broad range of surface normals directed upward. The 3D shape shown in the upper panel may appear as a specular surface illuminated along the line of sight because throughout the motion sequence the luminance maxima is associated with a narrow range of surface normals directed toward the viewer. 

Two different types of surface texture were superimposed on the luminance gradients to assess how the visual system exploits motion to derive surface reflectance. The 3D texture, shown in Movie 4, provides both motion parallax and texture gradient cues to 3D shape, whereas the planar texture, shown in Movie 5, provides only motion. The planar texture elements have a fixed size, density, and aspect ratio consistent with a frontal plane surface, so only the motion of the texture elements is correlated with the two 3D shapes. If the perception of surface reflectance differs between the two surfaces in Movie 5, then it must be computed from the 3D shape information that motion provides. 

Methods

Observers

Seven observers participated in Experiment 3. The undergraduate pool was not in session at the time of testing, so observers were recruited from the post-graduate pool of the school of psychology. One was the author (PM) who debriefed the other observers about our aims at the end of the experiment in adherence with the Declaration of Helsinki. 

Stimuli and procedure

The 3D shape of the upper surface shown in in Movie 4 and 5 used in Experiment 3 was identical to the specular surface from Experiment 2. The other 3D shape was a variant of the matte surface from Experiment 2 that was altered to slant backward in depth. The vertical cross-section of this surface (shown in the Figure 6 legend) had an inflexion of surface curvature in its center between a concave and convex circular segment. The depth of the surface was given by,  

 

where β = 0.55 influences the radius of the circular segments, which spanned the range 2.2 to 4.4 cm; α = 0.5 influences the arc length of the segments and aligns the two segments to form a smooth inflexion of surface curvature. The other parameters in the equation were given in Experiment 2 along with the horizontal (X) and vertical (Y) surface coordinates. The left and right sides of the surfaces had irregularly shaped edges so that the 2D shape of their bounding contours provided no 3D shape information.  

Figure 6

 

Results of Experiment 3. The perceived specular reflectance of the surfaces plotted as a function of the steepness of the luminance profile. Vertically separated data points refer to different 3D shapes with similar luminance profiles. The graphs refer to different viewing conditions that modulate the 3D shape cues present in the displays: motion with optical texture (left; e.g., Movie 4); motion with incongruent planar texture (middle; e.g., Movie 4); static incongruent texture (right). The dashed and dotted lines in the left graph show model judgments of specularity based on derivatives of luminance and 3D surface orientation. See text for details. Error bars show standard errors of the mean of all observers.

The rotation axis of the surfaces bisected the vertical screen axis and the observer's viewing direction. The rotation angle spanned ±25° varying at a constant rate of 15°/s. 

Two different types of surface texture were tested: 3D texture deformed by the geometry of the surface and planar texture (i.e., undeformed texture of a constant size). The 3D texture was composed of gray circles set in the tangent plane of the 3D surfaces. The radius of the circles (0.1 cm) and their density (1 circle per 0.7 cm²) were constant over the 3D surface geometry. The planar texture was a random dot texture composed of sparse grey dots set at the depth of the display screen. The radius of the dots (0.5 mm) and their density (3 dots per 1 cm²) were constant over the planar display screen. The motion of the dots tracked the perspective projection of corresponding points on the 3D surfaces. 

Both 3D shapes were texture-mapped with the same dynamic luminance profile. The luminance profile was constructed by assigning a luminance value to the S-shaped surface geometry based on its surface normals. The highest luminance was assigned to the surface normal directed toward the zenith, and luminance declined monotonically with the angular separation of surface normals from the zenith. Specifically, the luminance profile was given by  

 

where N is the vertical component of the unit surface normal; M represents the maximum luminance of the display screen (153 cd/m²); and the exponent S increases the steepness of the peak of the luminance profile. Six levels of steepness were tested; S = 1.9, 2.8, 3.7, 5.6, 6.5, and 7.4. The luminance profile was copied from the S-shaped surface geometry onto the C-shaped surface geometry using u, v texture mapping; the u, v parameters in the surface geometry equations define corresponding points between the two 3D shapes that share the same luminance. Images of the two 3D shapes were derived using perspective projection, which introduces slight differences between the bounding contours and luminance profiles of the two 3D shapes.  

There is a high correlation between the luminance of the pixels of the two surfaces, varying from R = 0.9 to 0.8 as a function of the rotation angle of the surfaces. Although there are only slight differences between the image gradients of the two shapes, observers could theoretically judge surface reflectance on the basis of these 2D image differences instead of the 3D shape information that motion provides. We therefore performed a control experiment in which observers viewed one static frame of the motion animation that provided no information about surface reflectance aside from the slight differences in the luminance profile of the two 3D shapes. Specifically, observers viewed the frame that contains the largest difference between the luminance profiles of the two 3D shapes (R = 0.8), which corresponded to the largest rotation angle in the motion sequence (±25° selected at random on each trial). The frames were taken from the planar texture animation (Movie 5) that contains no optical texture cues to 3D shape. 

Observers performed a paired comparison task in which they select the surface that appears more specular from two surfaces presented on each trial. Each combination of the two 3D shapes and the six luminance profiles were compared with the other surfaces in a block of 66 trials. The surfaces were viewed either rotating with 3D texture (left panel Figure 5), rotating with planar texture (middle panel), or static with planar texture (right panel). These three different viewing conditions were tested in separate blocks of trials and never compared to one another. The order of the three blocks and the order of the trials within a block was a different random permutation for each observer. 

Results and discussion

The results indicate that motion parallax can modulate perceived specularity even when there are conflicting texture gradient cues. The vertical axis of the graphs in Figure 6 depicts the proportion of trials that each surface appeared more specular than the other surfaces tested. The steepness of the luminance gradients increases from left to right across the x-axis, which again increases the perceived specularity of the surfaces, F(1, 7) = 88, p < 0.001 for the left graph; F(1, 7) = 91, p < 0.001 for the middle graph; and F(1, 6) = 21, p = 0.004 for the right graph. The vertically separated data points refer to similar luminance profiles on two different 3D shapes, which were moving in the left graphs or static in the right graph. The moving displays elicit a strong effect of 3D shape on perceived specularity, whereas the effect of 3D shape is lost for the static displays, F(1, 7) = 50, p < 0.001 left graph; F(1, 7) = 27, p = 0.002, middle; F(1, 6) = 2.3, p < 0.18 right. These results indicate that motion parallax is responsible for the effect of 3D shape observed in this experiment, not the slight differences in the luminance profile of the two 3D shapes. 

The perception of surface reflectance in these displays can be understood as arising from the 3D shape of luminance gradients. Although the luminance profile deforms relative to corresponding points on the rotating surfaces, a simple relation between luminance and surface orientation holds across the motion sequence. The highlight follows the surface normal directed upward for the surface that appears to be illuminated from above, whereas it follows the surface normal directed at the observer for the surface that appears to be illuminated from the front. This correlation between luminance and surface orientation is quite well preserved across the motion sequence for both 3D shapes (R > 0.99 for the matte surface and R = 0.8 for the surface judged more specular). The rate of change in luminance with respect to surface orientation provides information about surface reflectance. The dashed and dotted lines in the left panel of Figure 6 show that the steepest derivative of luminance with respect to surface orientation provides a good model of the psychophysical data (R² = 0.9). The steepest derivative was given by,  

 

where y is the luminance of the surface, and x is the angular separation of the surface normal from the illumination direction (i.e., the surface normal of the luminance maximum). The model performed the same paired-comparison experiment as observers, and judged surfaces with steeper derivatives (G) as being more specular. The model was computed on the first frame of the movie where the surfaces had a 0° rotation angle. The model demonstrates that the perception of surface reflectance in these displays is derivable from 3D shape attributes of luminance gradients.  

General discussion

Many studies of surface perception have found that physical manipulation of the surface geometry can induce illusory changes in surface reflectance (Ho et al., 2008; Marlow & Anderson, 2013; Marlow et al., 2012; Nishida & Shinya, 1998; Olkkonen & Brainard, 2011; Qi et al., 2014; Vangorp et al., 2007; Wendt et al., 2010; Wijntjes & Pont, 2010). Although the magnitude of such effects can assist in characterizing the accuracy of surface perception, a primary goal of this work has been to constrain thinking about issues of computation and representation. It is now well established that simple image statistics, such as the skew of the luminance histogram or sub-band skew (Motoyoshi & Matoba, 2012; Motoyoshi, Nishida, Sharan, & Adelson, 2007; Nishida & Shinya, 1998), cannot account for the complex effects of 3D shape and illumination on judgments of specular reflectance (Anderson & Kim, 2009; Kim & Anderson, 2010; Marlow et al., 2012; Olkkonen & Brainard, 2010; 2011; Wijntjes & Pont, 2010). We have previously shown that these interactions can be modeled using a set of appearance cues to gloss (Marlow & Anderson, 2013; Marlow et al., 2012). Perceived gloss is highly correlated with the apparent contrast, sharpness, and size of specular reflections irrespective of whether these appearance cues vary due to the physical surface geometry, reflectance, or light field (Marlow & Anderson, 2013; Marlow et al., 2012; Qi et al., 2014). These results have suggested that specular reflectance is computed from an imperfect set of image properties that do not invert the optical conflation of shape, material, and illumination. 

Our subsequent work has revealed that reflectance can also be derived from physical constraints that relate curvatures of 3D shape to derivatives of luminance (Marlow & Anderson, 2015; Marlow et al., 2015). If the surface geometry is known, then information about the reflectance function can be derived from the way luminance varies with respect to surface orientation. The present study is part of a sequence of experiments that have tested whether the visual system can derive reflectance from the distribution of surface orientations associated with a luminance gradient (Marlow & Anderson, 2015; Marlow et al., 2015). These experiments have manipulated cues to 3D shape to suggest two different distributions of surface orientation across a fixed luminance profile. The results show that each shape interpretation of the luminance profile promotes a different experience of surface reflectance and illumination direction (Marlow & Anderson, 2015; Marlow et al., 2015; Todorović, 2014). The shape that appeared most specular had the steepest derivatives of luminance respective to surface orientation, consistent with physical differences between specular and diffuse reflectance. The perceived illumination direction correlates with the apparent surface orientation of the luminance maximum, and is also consistent with the physics of specular reflection and diffuse shading. Although physical manipulation of shape can generate illusory variations in perceived reflectance, the results reported herein and previously (Knill & Kersten, 1993; Marlow & Anderson, 2015; Marlow et al., 2015) also show that perceived shape can be intelligently exploited to derive reflectance from physical constraints. 

Our previous work relied on bounding contours and stereopsis to modulate the perception of shape, material, and illumination (Marlow & Anderson, 2015; Marlow et al., 2015), and the present study provides the first evidence that motion and optical texture can elicit similar effects. Although some previous studies have shown that motion can improve the accuracy of gloss judgments (Sakano & Ando, 2010; Wendt et al., 2010), these studies did not test whether the benefit of motion was due to either motion parallax, the characteristic motion of specular reflections, or both. The results of the present study show that the visual system can derive specular reflectance from 3D surface geometry specified by motion parallax. In Experiment 1, the luminance profiles of the two 3D shapes were identical across the motion sequence, so the apparent differences in specular reflectance can only be derived from the deformations and motions of the overlaid texture that provides information about 3D shape. Experiment 2 and Experiment 3 isolated motion parallax and texture deformation cues to 3D shape and showed that surface reflectance can be derived from both forms of 3D shape information. 

The results presented here and elsewhere (Doerschner et al., 2011) suggest that the perception of specular reflectance also depends on the rigidity of optical motion. The strongest evidence of this comes from an experiment in which Doerschner et al. (2011) compared specular surfaces to surface textures in which the spatially varying albedo was engineered to have the same monocular image structure as the specular surface. Although these surfaces cannot be distinguished when static, they appear as strikingly different materials when rotating. The texture rotations are inherently coupled to the surface, whereas the specular structure slides and deforms over a surface when it or an observer moves. Doerschner et al. (2011) found that certain motion statistics can discriminate between the motion of surface texture and specular motion; however it is not yet known whether these cues can distinguish specular reflections from diffuse shading, which can both move and deform in ways unlike surface texture. Our stimuli demonstrate that specular reflection and diffuse shading can exhibit extremely similar patterns of motion if the 3D shape and illumination are free to vary (Movie 4). It is not yet known whether specular surfaces and diffusely shaded surfaces can be identified based on distinguishing features of their motion. 

Natural images contain many sources of 3D shape information that can assist in the recovery of surface reflectance and illumination. Although much has been learned about how to derive shape from binocular disparity, motion, and optical texture, comparatively little is understood about how shape might be derived from diffuse and specular image gradients. Our results suggest that insights into shape from shading and shape from specular reflections will also lead to insights into the perception of material properties and illumination. 

Acknowledgments

This work was supported by grants from the Australian Research Council awarded to Barton L. Anderson. 

Commercial relationships: none. 

Corresponding authors: Phillip James Marlow; Barton Anderson. 

Email: [e-mail]; [e-mail]. 

Address: Department of Psychology, University of Sydney, Sydney, New South Wales, Australia.