Influence of optical material properties on the perception of liquids

Abstract

In everyday life we encounter a wide range of liquids (e.g., water, custard, toothpaste) with distinctive optical appearances and viscosities. Optical properties (e.g., color, translucency) are physically independent of viscosity, but, based on experience with real liquids, we may associate specific appearances (e.g., water, caramel) with certain viscosities. Conversely, the visual system may discount optical properties, enabling “viscosity constancy” based primarily on the liquid's shape and motion. We investigated whether optical characteristics affect the perception of viscosity and other properties of liquids. We simulated pouring liquids with viscosities ranging from water to molten glass and rendered them with nine different optical characteristics. In Experiment 1, observers (a) adjusted a match stimulus until it had the same perceived viscosity as a test stimulus with different optical properties, and (b) rated six physical properties of the test stimuli (runniness, shininess, sliminess, stickiness, warmth, wetness). We tested moving and static stimuli. In Experiment 2, observers had to associate names with every liquid in the stimulus set. We find that observers' viscosity matches correlated strongly with the true viscosities and that optical properties had almost no effect. However, some ratings of liquid properties did show substantial interactions between viscosity and optical properties. Observers associate liquid names primarily with optical cues, although some materials are associated with a specific viscosity or combination of viscosity and optics. These results suggest viscosity is inferred primarily from shape and motion cues but that optical characteristics influence recognition of specific liquids and inference of other physical properties.

Introduction

In everyday life we continuously interact with our environment and the objects and materials it contains. To be able to do this effectively we need to be able to recognize familiar objects and materials, and infer their physical properties by sight. This is essential to our survival: It allows us to avoid eating rotting food, breaking our ankle on a slippery curb, or burning our hand on a hot pan. One highly challenging class of materials is liquids and gels. It is quite impressive that under typical conditions we can visually infer the properties of liquids and interact with them effectively despite their erratic nature and the large influence that external forces hold over their shape and flow. We are capable of distinguishing between water, toothpaste, caramel, shampoo, mercury, and numerous other liquids, and can even infer properties such as runniness, sliminess, and stickiness without physically touching them. This is important as it allows us to determine their affordances (i.e., whether it can be used for drinking, cleaning, gluing, etc.) and predict their likely behavior before interacting with them. 

Here, we sought to investigate the role of specific visual cues in the perception of liquids and their properties. In principle, there are several distinct sources of information that observers could draw on to recognize liquids and infer their physical characteristics by sight. Broadly, we can divide these into two classes: optical and mechanical. The main purpose of this study was to determine the relative contributions—and interactions between—these two broad classes of information. Some studies approach material perception by asking how the visual system estimates a single physical property of materials (e.g., glossiness, elasticity), and seeking specific visual cues to that property. In this study, by contrast, we look at a wide range of liquid properties to identify whether there are any stimulus or task conditions in which optical and mechanical cues interact to affect the perception of liquids. 

A liquid's optical material appearance can tell us many things about the liquid. For example, water is colorless and transparent, whereas milk is translucent; caramel and chocolate sauce have distinctive colors, whereas molten solder is lustrous. Because specific optical characteristics are associated with particular liquids, we could use the optical appearance—or low-level image correlates—to narrow down the range of expected behaviors of the liquid. In addition to the large amount of literature on the perception of surface color (see Foster, 2011, for a review), a growing body of research has investigated the estimation of optical properties such as gloss (Beck & Prazdny, 1981; Nishida & Shinya, 1998; Fleming, Dror, & Adelson, 2003; Motoyoshi, Nishida, Sharan, & Adelson, 2007; Ho, Landy, & Maloney, 2008; Kim, Marlow, & Anderson, 2012; see Chadwick & Kentridge, 2015, for a recent review), translucency (Fleming, Jensen, & Bülthoff, 2004; Fleming & Bülthoff, 2005; Xiao et al., 2014), transparency (Fleming, Jäkel, & Maloney, 2011; Faul & Ekroll, 2012; Schlüter & Faul, 2014) and surface texture (Landy & Graham, 2004; Dong & Chantler, 2005; Emrith, Chantler, Green, Maloney, & Clarke, 2010; Liu, Dong, Cai, Qi, & Chantler, 2015). These findings suggest that human observers are generally very good at inferring optical material properties under a wide range of conditions, and thus it is plausible that observers could base judgments about liquids on such cues. 

In contrast, it is the mechanical properties of liquids that determine the way they move and adopt particular shapes in response to external forces. Probably the most important mechanical parameter distinguishing different liquids and gels is viscosity. For example, water is very runny and therefore prone to splash and spread out in puddles, whereas toothpaste is thick and therefore tends to pile up into clumps when poured. Thus, the visual system could use the distinctive shape and motion caused by different viscosities to recognize liquids and predict their behaviors. Previous research has shown that we can infer viscosity from shape (Paulun, Kawabe, Nishida, & Fleming, 2015) and motion cues (Kawabe, Maruya, Fleming, & Nishida, 2015). Thus, again, it is plausible that human judgments about liquids could rely on their mechanical properties. 

In this study we sought to determine the relative contributions of optical and mechanical cues to the perception of liquids and their properties. We ask the following questions: Do observers recognize specific liquids based primarily on optical properties—like color, gloss, or translucency—or is viscosity also important for determining a liquid's identity? Are judgments of viscosity biased by a liquid's optical properties? What about the perception of other properties—like temperature, or stickiness—which cannot be so easily inferred from the motion or shape of the liquid? Such properties are potentially extremely important for determining the affordances of materials, but little is known about whether participants can infer them through visual information. 

A given material can change its optical and mechanical properties depending on the prevailing conditions: For example, the sugar concentration or temperature of syrup affects its viscosity, whereas small concentrations of dirt can make water cloudy without affecting the way it flows or splashes. Thus, both sources of information are imperfect cues to material identity. Although it is commonly argued that shape dominates other cues in object recognition (Biederman, 1987; Landau, Smith, & Jones, 1988), liquids are highly mutable, so it is plausible that color and other optical characteristics might be more diagnostic than shape. At the same time, if shape and motion can be computed accurately across a wide range of different optical conditions (Todd, Norman, Koenderink, & Kappers, 1997; Todd, 2004; Nefs, Koenderink, & Kappers, 2006; Khang, Koenderink, & Kappers, 2007; Vangorp, Laurijssen, & Dutré, 2007; Doerschner, Yilmaz, Kucukoglu, & Fleming, 2013; Dövencioğlu, Wijntjes, Ben-Shahar, & Doerschner, 2015), then viscosity could be estimated in a way that is unaffected by the surface material appearance, enabling “viscosity constancy.” Thus, there are grounds for believing that optical and mechanical properties may contribute to different extents depending on the specific judgments that observers are asked to make: whether it is estimating viscosity; rating other properties of liquids; or identifying (e.g., naming) specific materials, like paint, toothpaste or molasses. To test the contributions of optical and mechanical properties in the perception of liquids and their properties, we therefore asked participants to perform three tasks: (a) viscosity matching; (b) subjective rating of liquid properties; and (c) identifying which liquids correspond to verbal labels. 

For these experiments we used physically-based computer simulations of a wide range of liquids. The viscosities ranged from water to molten glass in six approximately perceptually uniform steps (established in an unpublished pilot experiment with the same stimuli using maximum likelihood difference scaling). Each liquid was rendered with nine different optical characteristics. Although the computer simulations are not absolutely perfect (careful observation reveals a few visible artifacts), they are accurate enough to elicit vivid and compelling impressions of distinct liquids, and were computed at higher resolutions than used in previous studies on the perception of liquids (Kawabe et al., 2015; Paulun et al., 2015). Moreover, only by using computer simulations is it possible to vary mechanical and optical properties independently in a parametric and perfectly controlled way. Only computer graphics allow us to render identical three-dimensional shapes with different optical properties, enabling us to perfectly isolate the relative contributions of the two classes of cue. 

In the experiments, observers were asked to adjust the viscosity of a match stimulus along a high-resolution viscosity scale (64 steps) until it appeared to have the same physical properties as a test stimulus that had different optical properties, in an asymmetric matching task. Observers also rated six different properties of the test stimuli, (runniness, shininess, sliminess, stickiness, warmth, wetness). These two tasks were performed with static and animated stimuli. Finally, observers participated in a liquid naming experiment to see how optical or mechanical properties interact to determine the identity of familiar liquids such as chocolate sauce, mouthwash, or milk. 

Methods

In Experiment 1, observers were asked to perform two tasks on each trial: an asymmetric viscosity-matching task, followed by a liquid property-rating task. The matching task showed a test stimulus with a specific viscosity and optical appearance, and observers could scroll through a standard set of liquids with fixed optical appearance, but finely varying viscosities, to select another stimulus that had the same apparent viscosity as the test. The test and match stimuli were sampled from different points in time in the animation sequence to encourage observers to base their responses on an internal representation of the physical properties of the liquid, rather than simply by identifying the stimulus with identical shape. Following the matching task, participants were asked to perform a series of ratings in which the same test stimulus was presented together with rating sliders for six different liquid properties: runniness, shininess, sliminess, stickiness, warmth, and wetness. 

Across participants, we varied (a) whether the stimuli were single static frames or 1-s animation sequences, and (b) whether the test or match stimuli were taken from the earlier time point. 

In Experiment 2, we measured how participants assigned names to the stimuli based on their mechanical and optical properties. First, one group of observers were presented with all 54 stimuli (6 viscosities × 9 optical appearances) and were asked to provide names for each material. Then, a second group of subjects filtered the word list to select the most descriptive and plausible liquid names corresponding to the stimuli. Finally, a third group of participants were provided with each name in the list and were asked to identify all of the stimuli from the 6 × 9 array that fit the description. The observers were allowed to select multiple liquids, allowing us to measure the extent to which each verbal term designated a mechanical or optical appearance (or both). 

Stimuli

All stimuli used in this study can be downloaded here: http://doi.org/10.5281/zenodo.154570. 

Simulation

The stimuli were generated using RealFlow 2014 (v. 8.1.2.0192; Next Limit Technologies, Madrid, Spain). This software enabled us to simulate and render liquids up to the standards used by the visual effects (VFX) industry. We used the “Hybrido” particle solver, which makes it possible to specify the dynamic viscosity of the liquids in real physical units (Pa·s). Hybrido is a FLIP (fluid-implicit particle) solver using a hybrid grid and particle technique to compute a numerical solution to the Navier–Stokes equations describing viscous fluid flow. All information for the fluid simulation is carried by discrete particles, but the solution to the equations is carried out on a grid. Once the grid solve is complete, the particles gather the information required from the grid to move forward in time to the next frame. The fluid boundary is then derived from the position of the particles by a meshing algorithm (when visible artifacts occur, it is primarily due this step of the algorithm, not the underling physics solver). For the match stimuli, a set of 64 different viscosities was simulated with logarithmically evenly placed steps from 0.001 Pa·s to 100 Pa·s (roughly corresponding to a range from water to molten glass in approximately perceptually uniform steps). The following equation can calculate the step number back to viscosity:  

 

where μ is the viscosity in Pa·s, b the starting value of the scale, in our case 0.001, d the range of the scale in decades, in our case 5 (10⁻³ to 10²), s the amount of steps of the scale (64), and n the step number of which we want to calculate the viscosity. Liquid density was held constant at one kilogram per liter. The number of particles used varied between 2 and 4.5 million particles depending on the viscosity, the only changing parameter in the simulation.  

The simulated scene (Figure 1) consisted of a 1 m² plane with a shallow wall around its perimeter and an irregularly shaped solid object (height = 17.5 cm, diameter = 19 cm) that was rigidly attached to the center of the plane. The liquid emerged from an “emitter,” located approximately 30 cm above the object (outside the frame of view). Gravity was the only external force acting on the liquid, which had no initial velocity on emerging from the emitter. The orifice of the emitter had a rounded cross shape, yielding distinctive ridges in the shape of the liquid, whose durability and distinctness varied with viscosity. 

Figure 1

 

Dimensions of the simulated scene.

The simulated animations had a total duration of 10 s (300 frames at 30 frames/s). For the experiments using static stimuli, the test and match images consisted of frames 90 and 150 from the animation (i.e., a 2-s time difference) in the first condition, and 150 and 90 in the second condition. The duration of the moving stimuli was one second: frames 80–110 or frames 140–170. 

For the target stimuli, six different viscosities were selected, which were evenly spaced on the existing 64-step scale. In this case steps (10, 19, 28, 37, 46, 55) corresponding to dynamic viscosity values of (0.005, 0.027, 0.139, 0.72, 3.73, 19.3) Pa·s. Figure 2 shows an overview of the static test stimuli. Video 1 shows the full 10-s animations of the six different viscosities with the same optical material. 

Figure 2

 

(A) An example trial with the physically correct match stimulus. (B) Another trial with inverted time points. (C) An overview of static stimuli with the nine different optical materials in the x-axis and the six different viscosities on the y-axis. The optical materials are approximations of the following materials: green goo, caramel, metallic car paint, chocolate, copper, a matte blue material, milk, water, wine.

Rendering

The render engine used to generate the final image frames was Maxwell (v. 3.0.1.3; Next Limit Technologies). Nine different optical materials were developed with diverse appearances, varying in their opaque, transparent, and translucent properties. The match stimulus set (consisting of 64 viscosities) was rendered with a translucent “green goo” appearance. The test stimuli consisted of approximations of the following materials: caramel, metallic car paint, chocolate, copper, a matte blue material, milk, water, and wine. These materials were selected to represent a wide range of different appearances that we could encounter in liquid form, including both common (e.g., colorless transparent) and unusual (e.g., matte blue) appearances. Video 2 shows a loop of the 1-s animations used during the experiment. It shows the nine different optical materials with the same viscosity. 

The images were rendered at an 800 × 600 resolution and the scene was lighted using a high dynamic range light probe depicting a beach scene (from the Maxwell Resource Library by Dosch Design, Marktheidenfeld, Germany). 

Observers

Matching and rating tasks

Forty-eight observers took part in the first experiment with static and animated stimuli and the two temporal orderings of test and match (i.e., four groups with 12 observers per condition). The average observer age was 25.3 (SD = 4.45). Thirty-three observers were female and 15 male. 

Naming experiments

Forty-two German speakers participated in the three experiments to match names with liquids. Ten observers took part in the free-naming (“brainstorming”) session with static stimuli, and a further 10 observers with animated stimuli. Six other observers took part in the “filtering” session to select a subset of terms from the brainstorming sessions. Finally, 16 observers participated in the main experiment, in which participants identified which stimuli corresponded with each verbal item. The average age was 24.9 (SD = 4.08), 27 were female, and 15 male. 

All observers gave written consent prior to the experiment and were paid for participating. All observers reported having normal or corrected-to-normal vision. 

Procedure

Matching and rating tasks

All experiments were performed in accordance with the Declaration of Helsinki, and prior approval was obtained from the local ethics committee of the University of Giessen. 

The experiments were performed on an Apple Mac Mini (Apple, Inc., Cupertino, CA) with a Dell U2412M 24-in. monitor (Dell, Inc., Round Rock, TX) using factory default settings, gamma of 2.2, and a resolution of 1920 × 1200 pixels. MATLAB 2015a (v. 8.5.0.197613) and the Psychtoolbox library (v. 3.0.11; Brainard, 1997; Pelli, 1997) were used to run the experiment, although Psychtoolbox was upgraded to version 3.0.12 over the period when different observers participated. 

Observers completed a short training session before starting the experiment. This consisted of a single trial to familiarize the participant with the interface for the matching and rating tasks and to ensure that the concepts on the six rating scales (liquidness, shininess, sliminess, stickiness, temperature, wetness) were clearly understood. Each trial consisted of the matching task followed by all six ratings for a given test stimulus. For the viscosity-matching task, the test stimulus was presented on the left-hand side of the screen, and the match stimulus was presented simultaneously on the right-hand side of the screen. Observers had to scroll through the viscosities of the match stimulus, with the left and right arrow key on the keyboard. A “page-turning” animation occurred with every button press, revealing the new match stimulus to avoid apparent motion between the different stimuli. Once the match stimulus on the right appeared to have the same physical properties as the target stimulus on the left, the observer could confirm by pressing the “spacebar” to proceed to the rating task for the same target stimulus. Here, the observer had to indicate a subjective rating for each of the six properties by using the mouse to move the randomly placed dots along the continuous rating bars (with seven tick marks). When the observer interacted with the dot on the rating bar, the dot would turn green. When all six dots were green, the observer could continue with the next trial by pressing the spacebar. The observer had to complete a total of 108 trials (two blocks, each consisting of 9 materials × 6 viscosities in random order). There were no time limits, and the experiment took observers 45 to 90 min to finish. 

Naming experiments

For the naming experiments the same Apple Mac Mini was used with the same Dell U2412M monitor as in the other experiments. The “brainstorming” experiment also used MATLAB and the Psychtoolbox library. On each trial, one of the 54 test stimuli (9 optical materials × 6 viscosities) was presented and observers were instructed to “name the liquid you see in the image.” There were four empty lines where observers could enter names for the liquids. Only one response per stimulus was required, although subjects were encouraged to provide multiple verbal terms if they applied. The brainstorming session resulted in a combined word list of 2,156 entries, 1,262 for the static stimuli and 894 for the moving stimuli. From this list, 10 names for each stimulus were selected, removing many duplicate and less descriptive entries. The resulting list of 540 words was used for the “filtering” experiment. 

Different software was used for the “filtering” and “name matching” experiments because of better interfacing possibilities. In this case a Flask (v. 0.10.1) based framework was used, compiled with Python 2.7.1. The front end was written using HTML5 technology displayed in Safari (v. 7.1.7). These browser-based experiments were displayed in “presentation mode” and therefore showed no interface of the browser itself. On each trial in the filtering experiment, an animated liquid stimulus was presented along with a randomized list of 10 names generated for that stimulus in the previous brainstorming session, 54 lists in total. Observers were asked to order the three most appropriate and descriptive names to the top of the list. This top three was weighted accordingly (three points for first choice, two points for second choice, and one point for third choice) during the selection process. All scores above 90% of the highest score were selected from the list. This means that if there was a close second, both words were selected, which happened 11 out of 54 times. Duplicate answers were filtered out, resulting in 49 words for the main name-matching experiment. 

The name-matching experiment used the same Flask and browser based presentation system as the filtering experiment. A new set of observers performed the task. On each trial, they were presented with a liquid name and a 6 × 9 grid containing static thumbnails of all stimuli. The viscosities were ordered vertically and the optical materials horizontally. When the observer dragged the mouse over a stimulus in the stimuli grid, a full-size animation for the corresponding stimulus would appear. If the observer thought that a given stimulus corresponded to the verbal item for the current trial, they could select it with a simple checkbox (subsequent unchecking was also possible but was rarely used in practice). Multiple stimuli could be selected for each name (i.e., each trial), but only one answer was required. Finally, the observers were asked to give a confidence rating for their response before continuing to the next trial. This experiment had 49 trials in which the names from the list were linked to the 54 different stimuli. There was no requirement for all of the stimuli to receive a name. 

All experiments were performed in German and have been translated to English for presentation here. 

Results

Raw data from all experiments can be downloaded here: http://doi.org/10.5281/zenodo.154570. 

For each of the matching and rating tasks, we tested four different versions: the static and animated stimuli with the test stimulus from an earlier or later time point in the animation sequence than the match. (See Methods for details.) 

Viscosity-matching task

Figure 3 shows the results from the viscosity-matching task for the four different conditions. The first notable observation is that observers are generally very good at matching viscosity: For all optical materials, the matching function is approximately linear with slope close to one. A linear regression can explain the data extremely well with a slope close to one for static stimuli: with the match from later than the test, β = 0.91, R² = 0.98, p < 0.001, and for the reversed time points, β = 1.02, R² = 0.97, p < 0.001. Especially for the moving stimuli, observers matched the liquids close to perfectly for the entire tested viscosity range, β = 1.002, R² = 0.99, p < 0.001, and for the reversed time points, β = 0.92, R² = 0.99, p < 0.001. 

Figure 3

 

Mean results of the matching task for the four different conditions, with static and moving stimuli, and reversed time points between match (M) and test (T) stimulus. Error envelopes represent standard error of the mean. Time points for the Moving conditions refer to a range of 30 frames, in this case frames 80–110 and 140–170.

There is, however, a systematic additive bias in the responses, which is most pronounced for the static stimuli. For the nonreversed condition (i.e., match stimulus from a later time point in the animation than the test stimulus) stimuli, there is a slight overestimation of viscosity. In other words, the liquids were perceived as having the same viscosity when the match stimulus was thicker than the test stimulus. This presumably reflects an imperfect compensation for the time offset between test and match, rather than a systematic overestimation of viscosity. This interpretation is supported by the observation that when the time points for test and match are swapped (i.e., test stimulus from a later time point than the match stimulus) the bias inverts. Evidently, in the absence of strong visual cues to indicate the precise point in time, it is difficult for observers to compensate for the difference in time point between test and match. Put differently, when asked to match viscosity in this task, there is a bias toward selecting similar shapes. This tends to lead to errors, because the shape of runnier liquids evolves more rapidly than for thicker liquids. Thus, the shape adopted by a given material at a particular point in time is often somewhat better approximated by a runnier fluid at an earlier point in time or a thicker fluid at a later point in time. 

To test more rigorously the hypothesis that participants simply selected the most similar shape, we (a) ran a control experiment and (b) developed a simple image similarity metric based on the Euclidean distance. The control experiment was exactly the same as the asymmetric matching task in the main experiment, except that instead of matching viscosity, 12 new observers were instructed to match shape. The match was only performed with static stimuli and all stimuli were of the green goo material. The Euclidean similarity metric used grayscale versions of the match and test stimuli with the same optical material, which were subtracted from each other. The mean pixel value of the resulting image is compared with other match/test stimuli combinations where the lowest mean value is the best Euclidean match. This allows us to derive a predicted match for each test stimulus, by identifying which of all the match images has the smallest Euclidean error (difference) to each test image. Figure 4 plots these predictions in comparison with observers' data for the static stimuli. Both results further support the interpretation that the additive bias is due to observers tending to match shape, while only partially compensating for differences in time point. The Euclidean predictions for runny liquids diverge more because of faster evolving shapes resulting in bigger differences between the two time points. Observers seem to partially compensate for this by not picking the most similar shape (in Euclidean terms). The hypothesis is further supported by the shape matches made in the control experiment. Performance was practically identical when observers were asked to match based on shape rather than viscosity, suggesting that viscosity judgements are very similar to shape similarity judgments. Our interpretation of this finding is that it is not very helpful to think of “viscosity perception” as a fixed process of creating a single, unified internal estimate of the physical parameter of the liquid, which can then be accessed psychophysically. Instead, depending on the specific task (e.g., matching viscosity, rating runniness) and stimulus context (i.e., other stimuli in the experiment), participants latch onto different cues in a highly flexible way (here focusing mainly on shape similarity between test and match stimuli). 

Figure 4

 

Mean viscosity match over all nine materials for time conditions with static stimuli in comparison with the shape match task (left) and predictions based on match stimulus with the most similar shape to each given test (using a simple Euclidean shape metric; right). Note that the solid data series depict identical data in both panels; only the predictions (dashed lines) differ.

Returning to the results of the main experiment, another notable aspect is the negligible differences between the nine optical materials, especially for the moving stimuli. This is confirmed by the linear regression analysis performed earlier where linear models based on viscosity explain 97% to 98% of the variance for the static stimuli. This means that at most 2% to 3% of the variance can be accounted for by the optical material differences and noise. With moving stimuli, this is even down to 1%. Thus, optical material appearance barely influences viscosity judgments, i.e., observers have very good viscosity invariance across changes in optical appearance, at least when reliable motion and or shape cues are present. 

Rating liquid properties

Observers were asked to rate runniness, shininess, sliminess, stickiness, wetness, and warmth. Figure 5 shows the scores observers gave for each material at the six different test viscosities. To save space, only graphs from the moving stimuli variation are shown. The graphs for the other variations, which are broadly similar, can be found in the Appendix. 

Figure 5

 

Mean rating scores for the six different liquid properties with moving stimuli. Error envelopes represent standard error of the mean. Point A (Shininess plot) indicates the point at which the liquid gathers into voluminous clumps, affecting the perceived shininess for the milk-like material.

There is a clear difference between properties that are driven mainly by mechanical cues (i.e., cues based on shape and motion), and optical cues, based on optical material appearance. As expected, runniness is clearly scored primarily on the viscosity of the stimulus and optical material appearance has almost no effect. A linear model based on the viscosity explains 98% of the variance, leaving 2% unexplained by the optical material appearance and noise. 

Conversely, shininess is driven primarily by optical cues. As expected, the matte blue material is seen as the least shiny, and the lustrous copper-metal is viewed as the most shiny. There is almost no effect of viscosity on perceived shininess: Most materials have a certain shininess independent of their viscosity, as indicated by the flat curves. The only exception seems to be the milk-like material. We believe this effect is caused by the high degree of subsurface scattering for this material. When the material's shape is thin, there is little scattering, so the body color appears darker, and the specular reflections have higher contrast. By contrast, when the material has more volume, scattering makes the body color whiter, reducing the contrast of highlights (Pellacini, Ferweda, & Greenberg, 2000). From the third viscosity step on, we see a notable decline in perceived shininess, shown at point “A” of Figure 5. From this viscosity on, the material gathers into thicker, more voluminous clumps, creating a more diffuse, matte appearance. Thus the interaction is probably not due to the perceived viscosity per se, but rather simply due to the shape. 

Sliminess is a property that depends on mechanical and optical cues. There are certain optical materials like green goo that appear slimier than others. At the same time, there is also a certain (intermediate) viscosity range that observers associate with sliminess. It is interesting to see that there appears to be little interaction between optical and mechanical cues. The different materials are shifted from each other vertically, but follow roughly the same curve. 

Stickiness is mainly driven by optical cues. For example, the matte blue material does not look sticky at all whereas a wine-like material appears to be stickiest. 

Wetness decreases with increasing viscosity. The matte blue material appears substantially less wet than all other materials. This is consistent with previous findings that specularity is associated with wetness (Sawayama & Nishida, 2015). 

Somewhat surprisingly, the warmth ratings do not show a substantial effect of viscosity. One might expect a runny metal or chocolate colored material to appear warmer than a more viscous variant. The instructions clearly stated that participants should rate the expected temperature as it would feel were the participant to put their finger in the liquid. However, participants did not seem to consider runniness as a cue to increasing temperature. It is possible that a forced-choice paradigm might reveal a tendency to associate runnier liquids with higher temperatures, but, if present, the association is not strong enough to show up in this experiment. Another notable result is that there is a clear bimodal distribution of warm and cold materials. This appears to be influenced by the “warmth” of the color of the liquid, where red, brown, and orange materials are warm and green, blue and transparent materials are cold. It is unclear whether this was simply a tacit association, or whether participants deliberately chose to base their warmth judgments on color, despite the explicit instructions to attend to the expected temperature. 

Model

As noted, most differences among the nine optical materials appear to be shifts in scores on the y-axis. This suggests that although optical and mechanical cues contribute to the perceived properties of liquids, the interactions between the two classes of information are generally relatively weak. We quantified this observation by fitting models to each of the nine materials for the six liquid properties shown in Figure 5. For each of the rated properties, we took the mean of all optical materials and fitted a linear and quadratic model to this. The best AIC (Akaike information criterion) score of the mean-based model defines the type of model for the individual materials. AIC is a statistical model fit measure based on the likelihood function and number of predictors. To test the hypothesis that most of the data can be explained by only shifting a fitted model on the y-axis, we took the slope of the mean-based model and fit only the intercept (“fixed slope model”). We compared this with a fit where each material had an independently fitted slope (“free slope model”). The results of the average AIC values from the nine different materials are shown in Figure 6. A lower value means a better fit of the model to the original data. Since AIC weighs in the complexity of the model and our fixed slope models are less complex, we can see if a decrease in complexity compensates for the decrease in goodness of fit. In cases where the orange bar is shorter in Figure 6, our fixed slope model outperformed the free slope model. This means that in these cases, the model without interaction between optical and mechanical cues explains the data better. Another measure, AIC_c, or the second-order corrected Aikaike information criterion assigns greater penalty for extra model parameters and is mostly applied in cases when the sample size (n) is small compared with the number of parameters (k) where n/k < 40 (Burnham & Anderson, 2002), which holds in this case. In all six cases, AIC_c prefers our fixed slope model. Overall, based on these results, it is safe to say that interactions between optical and mechanical cues are relatively limited. Shininess, with the outlier at point A of Figure 5, seems to be the reason why our fixed slope model does not perform well, where in the other negative two cases the differences between the two models are much smaller. 

Figure 6

 

Overview of average AIC and AIC_c values from the nine different materials for each liquid property. A lower value means that the model is a better fit. The free slope model fits the intercept and the slope. The fixed slope model only fits the intercept with a predetermined slope. The Q and L show if it is a linear or quadratic based model. The AIC criterion takes the amount of parameters used into account for an optimal tradeoff between goodness of fit and complexity of the model. AIC_c is similar to AIC but assigns greater penalty for extra model parameters.

Principle component analysis

Another way of representing the rating data, to gain insights into the relative contributions of mechanical and optical cues, is using a principal component analysis (PCA). Each stimulus can be represented as a point in a six-dimensional feature space, where each feature represents one of the six subjective rating scales. PCA allows us to summarize the relationships between the different stimuli as well as the relationships between the different liquid properties. Figure 7 plots the data from the experiment with moving stimuli with standard time ordering, in the space spanned by the first two principal components. 

Figure 7

 

(A) Samples in the PCA space (first two components), color-coded by optical material. Vectors represent projections of the different liquid property dimensions. (B) The same data points color-coded by viscosity instead of optical properties.

Caution is required in interpreting these plots, as the different ratings are not necessarily measured on a consistent scale. Although participants were asked to rate each property on a 0–10 range, they may have used very different internal scales for mapping the perceived differences between different liquids onto each scale. Thus, for example, a step of 0.1 on the Runniness scale is not commensurable with a step of 0.1 on the Warmth scale. This means that we cannot draw strong conclusions about the metric distances between different samples in the PCA space. Nevertheless, it is interesting to observe the orderly arrangement of the samples in the feature-space, which are systematically organized by optical and mechanical properties. 

The different dimensions plotted in Figure 7A reveal that runniness and shininess are approximately perpendicular to each other. As noted already, runniness is mainly driven by mechanical cues (viscosity), and shininess mainly by optical cues (material appearance). That runniness and shininess are perpendicular to one another in the PCA space confirms that we tend to separate optical and mechanical cues when judging liquid properties. In Figure 7A, the different optical materials are systematically organized along the shininess axis, where Figure 7B shows that the different viscosities clearly follow the runniness axis. It is also notable that for the range of viscosities and optical appearances we used here, and for the particular set of liquid properties we asked participants to rate, optical and mechanical cues play approximately equal roles. The spread of samples in terms of their optical properties is roughly the same as the spread in terms of the viscosities (although we cannot directly compare magnitudes across features, it is nevertheless interesting that across all features there is a roughly even spread of influence of optical and mechanical properties). 

Naming experiment

Figure 8 shows the results of the name-matching task in which observers were asked to select one or more stimuli for each of the 49 different liquid names that were generated in the “brainstorming” and “filtering” experiments. 

Figure 8

 

(A) Raw data of a sample of the 49 words. The 54 columns represent the 54 stimuli where the first nine optical materials of the runniest liquid are on the left. Periodic behavior (blue) suggests optical focus, sequential behavior (red) suggests mechanical focus, green is a combination of both, and gray are more noisy names. (B) Raw data of the “Molten gold” word. Six rows for six viscosities and nine columns for nine materials. The kurtosis of the sum of each row and column is used to calculate the optical focus (blue) and mechanical focus (red). (C) The distribution of votes per stimulus per trial with a maximum of 16 votes for the 16 participants. (D) The optical and mechanical focus for each word plotted in a single 2D space. The red area has high mechanical focus, the green area both high mechanical and optical focus, and the blue area high optical focus. The intensity of the dots represents the confidence interval given by observers.

For every participant and for each verbal item, we have a complete 6 × 9 binary array indicating whether the corresponding image was deemed to match the verbal item, along with a scalar confidence rating. Pooling across subjects gives us an integer array per verbal item, containing the number of votes each stimulus received across observers (Figure 8B). For display purposes, we can reorder the array into a 54-vector for each verbal item. Example response vectors for several liquid names are shown in Figure 8A. (A complete list is presented in the Appendix.) 

For most stimuli, the participants' responses were sparse: In other words, each name corresponded with only a small subset of the 54 candidate images (mean = 2.8 items, SD = 3.4). Moreover, there was a high degree of consistency between participants in the set of stimuli that were selected for each name. This can be measured by the kurtosis of the distribution of responses over all possible words and stimuli, where 16 votes is the maximum score (i.e., one vote per participant). The kurtosis is 17.74, making the distribution highly leptokurtic, meaning that in many cases multiple participants matched a stimulus with a word or none did (Figure 8C). If it is 16 it means that for one word all 16 observers chose a specific stimulus, which happened two times. Participants were very confident matching stimuli to words, with an average confidence interval of 7.3 on a 0–10 scale. Together, these findings suggest that observers associate liquid names with specific appearances, and thus that visual appearance is quite diagnostic of liquid identity for a wide range of common liquids. 

To gain a more thorough insight into the extent to which liquid identities are associated with specific ranges of optical and mechanical properties, we computed two indices to measure how selective participants were in terms of the optical and mechanical properties of the stimuli they chose. We define the “optical focus” of the responses as the extent to which the responses to a given verbal item were restricted to a particular optical material, specifically, the kurtosis of the sum votes for each optical material. Analogously, we define the “mechanical focus” as the extent to which the responses to a given item were restricted to a particular range of viscosity values, specifically the kurtosis of the sum votes for each viscosity (Figure 8B). Note that these two quantities are independent and not mutually exclusive, so that an item could have a low degree of focus for both properties (indicating that the verbal term is not very specific, e.g., “liquefied dough”); a high focus for one property but not the other (indicating that it specifies a particular optical appearance, but not a specific viscosity, or vice versa, e.g., “chocolate pudding” or “gum”); or a high focus for both properties (indicating that the name specifies a particular combination of optical appearance and viscosity, e.g., “grape juice”). In Figure 8A, example items with low focus are colored gray, items with high optical focus only are indicated in blue, items with high mechanical focus only are indicated in red, and items with high focus for both optical and mechanical properties are indicated in green. Note that due to the reordering of the array into a vector, periodic responses every nine steps indicate that observers selected based on the optical material (i.e., high optical focus), and an adjacent sequence of nine high values indicates that observers selected based on viscosity (i.e., high mechanical focus). 

These “focus” indices allow us to summarize the relative importance of optical and mechanical properties for all 49 liquid names in a single two-dimensional (2D) space, as shown in Figure 8D. (A complete overview is presented in the Appendix.) The x-axis shows optical focus and the y-axis mechanical focus. The intensity of the name dots represents the mean confidence ratings observers gave for each item. Note that items with lower focus values also tended to receive lower confidence ratings. Thus, it could be that items with low focus values could simply be liquids for which none of the images corresponded well with the name. Thus, we should be cautious about concluding that some liquid names do not specify very precise appearances: It could simply be that the stimulus set did not contain appropriate images. 

Most names are associated primarily with the liquid's optical appearance, as indicated by the blue region. Only four of the 49 names were associated with one specific viscosity but no specific optical appearance (red region). There are a few liquid names that specify a particular combination of optical and mechanical properties e.g., water that needs to be runny and transparent. These results suggest that although we are very well able to perceive different viscosities, optical material appearance seems to be more a more distinctive feature than viscosity, and is therefore assigned more linguistic value by observers. Alternatively, it could be that a given class of liquid is generally prone to vary more in viscosity than in optical appearance, relative to the range of values that we used (e.g., “chocolate sauce” comes in lots of different thicknesses, but they are all brown). 

Discussion

There are at least two routes by which optical properties could affect the perception of liquids: (a) via learned associations, or (b) by aiding (or hindering) the perception of shape and motion cues that are the basis for estimates of liquid properties. The former is specific to liquid perception, whereas the latter reflects general processes of midlevel vision. Our findings suggest that the extent to which observers rely on optical or mechanical information about liquids and their properties depends on the context and task. When asked to make visual matches of viscosity, shape and motion cues dominate, and optical material appearance barely influences perceived viscosity. This suggests that learned associations and effects of shading on shape and motion estimates only weakly affect viscosity matches, at least when motion and shape cues to viscosity are strong. Although it is surely possible to find combinations of lighting and reflectance that do adversely affect shape and motion cues to viscosity (as occurred to some extent with the “milk” material in Experiment 1), under typical viewing conditions, shape and motion processing is robust enough to derive viscosity-diagnostic information from the richly structured patterns that pouring liquids generate on the retina. In contrast to the viscosity-matching task, the rating task showed that subjective ratings of different liquid properties are based on mechanical cues, optical cues, or a combination of both, depending on the specific property. Moreover, the pattern of responses suggests that processing of mechanical and optical cues is independent because of very limited interactions between the two: Most rating patterns could be well explained by a simple linear combination of the two kinds of information. The liquid naming experiment suggests that in most cases, we tend to assign names to liquids based mainly on their optical material appearance. This could mean that the optical material appearance is more diagnostic of the liquid (or more invariant) than its mechanical properties, at least for the range of appearances that we considered. 

The finding that optical properties have only a weak effect on viscosity judgments makes intuitive sense because the physical processes determining viscosity are independent of those that affect the way the fluid scatters, reflects, and absorbs light. In principle, any given optical appearance could co-occur with any possible viscosity, and therefore optical characteristics do not provide a direct visual cue to viscosity. However, we reasoned that if a specific liquid with familiar viscosity properties is identified (via optical cues), this could bias or interact with viscosity estimates. Our findings suggest, however, that if this occurs, it is to a very small extent, at least when strong motion and or shape cues to viscosity are present. Especially with moving stimuli, observers show close-to-perfect performance at matching viscosity across variations in optical materials. This suggests that when observers are judging a mechanical intrinsic property of the liquid like viscosity, they rely primarily on shape and motion cues. As mentioned before with other scenes where mechanical cues are less dominant, the influence of optical cues might increase. We do think that with our stimuli, designed to study viscosity, mechanical cues will keep their dominant role, and therefore we will continue our studies investigating the perception of viscosity without taking potential influences of optical characteristics into account. 

However, this is not to say that there is no role of optical properties in the perception of liquids and their properties more generally. In the ratings and the naming task, some properties and liquids were associated with specific optical cues. However, our results provide an initial indication that optical and mechanical cues do not interact much with each other. This impression is amplified by the results of the second task in which observers had to rate six liquid properties: runniness, shininess, sliminess, stickiness, wetness, and warmth. In most cases the various properties were determined primarily by either optical or mechanical cues on their own, e.g., “runniness” decreases with increasing viscosity, but is unaffected by the optical properties of the liquids, whereas “shininess” varies as a function of the specular reflectance of the material, and is barely influenced by viscosity (apart from the translucent milk-like material discussed already). There are, however, some properties that are affected by optical and mechanical characteristics. For example, mechanical and optical cues play a role in the perception of “sliminess:” green goo looks significantly slimier than copper-like liquids, even when the shape and motion are identical, but there is also a certain viscosity range that is considered to be slimiest (neither too thick nor too runny—like Goldilocks' porridge). Nevertheless, even though both types of cue influence perceived sliminess, the interaction between the two is limited. All scores followed approximately the same curve, merely shifting additively up and down as a function of the optical characteristics (see Figure 5). This tends to suggest that the visual system treats the two kinds of information as distinct cues, which are then combined according to a simple “weak fusion” process (Landy, Maloney, Johnston, & Young, 1995) to arrive at a subjective rating of “sliminess.” Alternatively, it is possible that the influence of the optical and mechanical cues on the ratings proceeds via top-down associations. Specifically, it could be that the image cues serve to identify a specific liquid (e.g., green goo), whose cross-modal properties (e.g., sliminess) are recalled from memory. It is difficult to design experiments that tease apart the relative role of bottom-up and top-down contributions to ratings of high level properties of materials (Fleming, Wiebel, & Gegenfurtner, 2013). 

Some caution is required in generalizing the conclusions of the matching and rating experiments. Here, we used a somewhat restricted range of stimuli consisting of one single scene of pouring liquids. It is almost certainly the case that other stimuli—such as those shown in Figure 9—can yield more extreme percepts of many of the features we tested here. For example, none of the stimuli in our experiment appeared as “sticky” as the example shown in Figure 9A. Additional cues to stickiness presumably include the distinctive strands that span surfaces that have been stuck together and pulled apart, or in terms of motion, prolonged adhesion to other surfaces in the scene. Likewise, the molten metal in Figure 9B clearly conveys a stronger sense of high temperature than any of the stimuli in our experiments, presumably due to the visible glow, and other cues such as smoke or steam. If motion and shape cues to materials are extremely weak (e.g., in the limit a stationary liquid in a container), then optical cues will presumably carry a relatively stronger weight in determining perceived viscosity or other liquid properties. Nevertheless, we believe that the broader conclusion that different properties of liquids combine shape, motion and optical cues with different weights will withstand further scrutiny. This is for the simple reason that, although optical properties are almost always an ambiguous (i.e., unreliable) predictor of viscosity, shape and motion cues tend to be highly diagnostic of viscosity as soon as the liquid flows. 

Figure 9

 

(A) Example of a sticky material. (B) Example of a hot liquid. Images used under CC0 Public Domain license.

In our study, liquid names are mainly dominated by optical material appearance. The names considered descriptive of liquids in most cases span a range of possible viscosities. For example: the name “chocolate” is assigned to all viscosities as long as the optical material is chocolate. This presumably reflects the fact that different concentrations and temperatures of chocolate yield a wide range of viscosities, but changes to the surface color and optical appearance are less common. However, there are exceptions to the dominance of optical qualities. The term “water” specifies a specific colorless transparent appearance and a specific (runny) viscosity. “Plastic” needs to look viscous but can have a wide range of different optical materials. Thus, specific recognizable liquids can be associated with optical and mechanical properties. It seems that under many conditions, the optical material appearance (primarily color, gloss, and translucency parameters) are sufficiently distinct to specify many common liquids. Where the optical material appearance is not sufficiently specific for communicating a particular physical state, speakers may use additional terms that are specific to a liquid's mechanical aspects, such as sauce, paste, mouse, syrup, or cream. Our observers did not report any problem with using multiple terms to specify appearances—including materials in viscosity states that they have not personally experienced before. We suggest that this approach to linguistic labeling of fluids—with basic level terms for optical appearance and qualifiers for viscosity—may reflect how we prioritize the visual cues that are used to identify liquids in general (i.e., optical appearance may dominate mechanical under many circumstances). 

Acknowledgments

This work has been funded by the EU FP7 Marie Curie Initial Training Networks (ITN) project PRISM, Perceptual Representation of Illumination, Shape and Material (PITN-GA-2012-316746), the DFG (SFB-TRR-135: “Cardinal Mechanisms of Perception”), an ERC Consolidator Award (ERC-2015-CoG-682859: “SHAPE”) and a Google Faculty Research Award. Special thanks to the people of Next Limit Technologies, especially Gustavo Sanchez, Angel Tena, Alex Ribao, and Luis Miguel Mora of the RealFlow team for the support provided during the stimuli generation process. This publication represents a component of my doctoral (Dr. rer. nat.) thesis in the Faculty of Psychology at the Justus-Liebig-University Giessen, Germany. 

Commercial relationships: none. 

Corresponding author: Jan Jaap R. van Assen. 

Email: [e-mail]. 

Address: Department of Psychology, Justus-Liebig-University Giessen, Giessen, Germany. 

Appendix

Stimuli and experimental data

All stimuli and experimental data used in this study are available for download from: http://doi.org/10.5281/zenodo.154570. 

Remaining rating results for all four variations

Figure A1

 

Showing the liquid property rating results with static stimuli. Error envelopes represent standard error of the mean.

Figure A2

 

Showing the liquid property rating results with static stimuli of the reversed condition. Error envelopes represent standard error of the mean.

Figure A3

 

Showing the liquid property rating results with moving stimuli of the reversed condition. Error envelopes represent standard error of the mean.

Full data set of the naming experiment

Figure A4

 

Showing the raw data of the name matching experiment.

Figure A5

 

Showing the data of Figure A4 in 2D space. The intensity of the dots represents the confidence ratings. The names with high mechanical focus are in the red area, the names with high optical focus are in the blue area, and the names with high optical and mechanical focus are in the green area.