Shatter and splatter: The contribution of mechanical and optical properties to the perception of soft and hard breaking materials

Abstract

Research on the visual perception of materials has mostly focused on the surface qualities of rigid objects. The perception of substance like materials is less explored. Here, we investigated the contribution of, and interaction between, surface optics and mechanical properties to the perception of nonrigid, breaking materials. We created novel animations of materials ranging from soft to hard bodies that broke apart differently when dropped. In Experiment 1, animations were rendered as point-light movies varying in dot density, as well as “full-cue” optical versions ranging from translucent glossy to opaque matte under a natural illumination field. Observers used a scale to rate each substance on different attributes. In Experiment 2 we investigated how much shape contributed to ratings of the full-cue stimuli in Experiment 1, by comparing ratings when observers were shown movies versus one frame of the animation. The results showed that optical and mechanical properties had an interactive effect on ratings of several material attributes. We also found that motion and static cues each provided a lot of information about the material qualities; however, when combined, they influenced observers' ratings interactively. For example, in some conditions, motion dominated over optical information; in other conditions, it enhanced the effect of optics. Our results suggest that rating multiple attributes is an effective way to measure underlying perceptual differences between nonrigid breaking materials, and this study is the first to our knowledge to show interactions between optical and mechanical properties in a task involving judgments of perceptual qualities.

Introduction

The perception of materials is critical to our everyday interactions with objects in our environment. All objects and substances are made up of materials, and the visual identification of materials is a central part of object recognition (Adelson, 2001; Fleming, 2014). For example, we can rapidly distinguish whether objects such as flowers and fruit are real or fake based on their material properties (Sharan, Rosenholtz, & Adelson, 2014). Not only can humans visually distinguish between materials, we can also effortlessly infer different properties or attributes about those materials, for instance whether food is edible, paint is wet, or the floor is slippery. Furthermore, these attributes are closely associated with how we categorize materials into different classes—for example, plastic, wood, metal, glass, or fur (Fleming, Wiebel, & Gegenfurtner, 2013; Hiramatsu & Fujita, 2015; Nagai et al., 2015). 

Most research about the visual perception of materials has aimed at discovering the physical factors or image cues that influence how we perceive surface optical properties like gloss, translucency, roughness, and velvetiness in static scenes (i.e., scenes without motion; for a review, see Fleming, 2014). For example, gloss perception has been found to depend on the brightness, position, and orientation of highlights relative to diffuse shading (Anderson & Kim, 2009; Beck & Prazdny, 1981; Berzhanskaya, Swaminathan, Beck, & Mingolla, 2005; Fleming, Torralba, & Adelson, 2004; Kim, Marlow, & Anderson, 2011; Todd, Norman, & Mingolla, 2004), the illumination conditions (Fleming, Dror, & Adelson, 2003; Motoyoshi & Matoba, 2012), and shape properties such as relief height (“bumpiness”) of mesostructure (Marlow, Kim, & Anderson, 2012). For the perception of transparency, researchers have identified photometric conditions (luminance-polarity relationships) and geometric conditions (e.g., X-junctions) that are required for the perception of thin transparent surfaces (Adelson & Anandan, 1990; Anderson, 1997, 2003; Beck & Ivry, 1988; Beck, Prazdny, & Ivry, 1984; Metelli, 1970, 1974a, 1974b). The perception of thick transparent and translucent objects has been found to depend on factors such as the thickness of the material, the direction of illumination, and the relationship between shading and highlights (Fleming & Bülthoff, 2005; Fleming, Jäkel, & Maloney, 2011; Motoyoshi, 2010). These examples show that static image cues can be used by the visual system to infer surface optical properties, and that these cues are affected by both intrinsic object properties such as 3-D shape and reflectance properties and extrinsic factors such as illumination conditions. 

Recent studies have shown that image motion can also greatly affect the perception of surface optical properties (Doerschner, Fleming, et al., 2011; Doerschner, Kersten, & Schrater, 2011; Marlow & Anderson, 2016; Oren & Nayar, 1997; Sakano & Ando, 2010; Wendt, Faul, Ekroll, & Mausfeld, 2010; Yilmaz & Doerschner, 2014). For example, Doerschner, Fleming, et al. (2011) identified three motion cues (coverage, divergence, and 3-D shape reliability) that the visual system could potentially use to distinguish between moving shiny and textured matte surfaces that otherwise looked identical when presented as static images. Marlow and Anderson (2016) showed that motion parallax can modulate whether a moving luminance gradient is perceived as specular or diffuse reflectance, even when shape and texture cues are held constant. Thus, motion can independently provide the visual system with information about material properties. 

These studies focused on the surface appearance of rigid objects. Many materials, such as textiles and elastic objects, do not behave rigidly (i.e., they are nonrigid) or are not even distinct objects per se; rather, they are “stuff”-like, such as snow, water, and porridge (Adelson, 2001). We can infer mechanical properties of nonrigid materials from the way they interact with the environment, through shape and motion cues. For example, recent studies have suggested that different aspects of motion flow correlate with the perceived viscosity of liquids (Kawabe, Maruya, Fleming, & Nishida, 2015), the stiffness of cloth (Bi & Xiao, 2016), and the elasticity of soft bodies (Kawabe & Nishida, 2016), and can be used to differentiate between deformations caused by water and hot air (Kawabe & Kogovšek, 2017). Other studies have suggested that shape characteristics can be used to determine perceived liquid viscosity (Paulun, Kawabe, Nishida, & Fleming, 2015; van Assen, Barla, & Fleming, 2016) and the apparent stiffness of elastic soft bodies (Paulun, Schmidt, van Assen, & Fleming, 2017). 

One question that is beginning to be explored is how the perceptual qualities of nonrigid materials are influenced by manipulating the intrinsic mechanical properties of a material versus manipulating the intrinsic optical properties. Physically, mechanical properties are related to how materials react to an applied force (e.g., stiffness of solids or viscosity of liquids), whereas optical properties describe how light interacts with and is scattered by surfaces (sometimes called the bidirectional scattering distribution function; e.g. specular and diffuse reflectance, transmittance, index of refraction). Although there is a clear physical distinction, it is unclear how mechanical and optical properties influence perceived material qualities. Do mechanical properties, revealed through shape and motion information, dominate over surface optical appearance, or vice versa? Or do they contribute additively or interactively? For example, two substances could have almost identical optical properties but differ in their mechanical properties, like water and glass. Both materials are transparent, clear, and glossy, and refract light similarly (and indeed look optically similar), but they would behave very differently under force: Water would splash or slosh and glass would break or shatter. The mechanical information (via shape and motion cues) from the different ways these materials interact with the environment would provide the visual system with information about each substance's material attributes; water is wet and runny; glass is hard and fragile. 

It is also possible for two materials to behave in a similar way (i.e., they have similar mechanical properties) but differ in their optical appearance, as demonstrated in Supplementary Movie S1. Supplementary Movie S1 shows two computer-simulated substances that are dropped from a height and hit the floor with the same “splattering” motion. Although they have identical mechanical motion, the substances are rendered with different optical properties, which make them appear quite different: One looks like wet, gelatinous jelly and the other like an airy, fluffy marshmallow substance. Optical and mechanical properties might also interact to affect the perception of materials and their properties. Imagine the two materials in Supplementary Movie S1 were animated as runny liquids with equal viscosity. When poured, one substance might look like green cordial and the other like milk, but they would have very similar material attributes (runny, wet, sticky). Similarly, if they were animated as solid fragile objects that shattered when dropped, one might look like green glass and the other like porcelain, but again they would have very similar material properties (smooth, hard, fragile). In these examples, the optical appearance affects the perceptual qualities of soft bodies, but not in the liquid and solid case. 

These examples illustrate that, although they are physically independent, the influence of optical and mechanical properties on perceived material qualities is not independent. However, in the literature it remains unclear how manipulating these intrinsic properties affects the perception of nonrigid materials, as the results are mixed. When subjects are asked about single attributes such as the viscosity of liquid or the softness of elastic and plastic soft bodies, mechanical and optical properties have been found to either dominate one over the other (Paulun et al., 2017; Schmidt, Paulun, van Assen, & Fleming, 2017; van Assen & Fleming, 2016) or contribute additively (Schmidt et al., 2017; van Assen & Fleming, 2016) to the perception of these qualities (this can also depend on whether the materials are shown as static images or dynamically—i.e., in motion; Schmidt et al., 2017). However, van Assen and Fleming (2016) showed that interactions arise in category naming of liquids, and Aliaga, O'Sullivan, Gutierrez, & Tamstorf (2015) found interactions during a fabric similarity-matching task. What are the possible reasons for these differences in results? For the measurement of single attributes like softness or viscosity, Schmidt et al. (2017) suggest that the visual system performs reliability-weighted cue combination—that is, it uses the cues that are most reliable. In some cases, such as judging the viscosity of liquids (van Assen & Fleming, 2016) or the softness of deformed cubes (Paulun et al., 2017), shape and motion cues are more reliable than the surface's optical appearance and therefore dominate. However, when shape cues are ambiguous, such as with the irregular stimuli of Schmidt et al. (2017), the visual system has to rely on surface optical appearance and learned associations with material properties such as softness. On the other hand, the interactions found by van Assen and Fleming and Aliaga et al. could have arisen because asking about categories or matching materials in a general way takes into account multiple perceptual qualities. To support this view, Fleming et al. (2013) showed that categories are closely linked to multiple attribute ratings of materials. Thus, another possible reason for the mixed results in the literature is the limited perceptual qualities tested (e.g., viscosity, stiffness). 

Perceptual space and research goals

The present study has two aims. First, we aim to explore the relative influence of and possible interaction between optical and mechanical properties in the perception of nonrigid materials for a range of material attributes and a range of substances. Our approach differs from previous studies in three ways: First, we assess multiple perceptual attributes—for example, not just softness; second, we include more than one class of stimuli (i.e., both hard and soft substances); third, the substances break apart in different ways (see Experiment 1, Methods) rather than transforming smoothly under force (Paulun et al., 2017; Schmidt et al., 2017). 

A second aim of this study is to assess the contribution of motion versus shape cues as mechanical information. To test this, we included conditions that separate motion (Experiments 1 and 2) and static cues (Experiment 2). Point-light versions of the stimuli were rendered to isolate motion cues. Previous research has shown that people can infer a lot from image motion, from recognizing and deriving the 3-D structure of objects (Vuong & Tarr, 2004; Wallach & O'Connell, 1953) to perceiving biological motion (Johansson, 1973, 1976; Troje, 2013) and animacy or intention (Heider & Simmel, 1944). 

The experiments are exploratory in nature, meaning that our aim is not to decisively determine the influence of manipulating intrinsic optical and mechanical properties, nor to test the whole space of nonrigid breaking materials (indeed, the space of nonrigid breakable materials is large, and extrinsic properties like lighting and the type of force applied to a material would all contribute to perceived material qualities, which we do not test here). Rather, our aim is to explore a large enough space, in terms of both stimuli and material attributes, to see whether interactions between mechanical and optical properties arise for perceptual judgments. 

To anticipate our results, we found interactions between optical and mechanical properties for several attribute ratings. Furthermore, attributes that were initially hypothesized to be predominantly optics or motion driven were influenced by the other factor. Finally, motion cues and static cues alone provided substantial information for observers to rate material properties; however, observers' ratings differed considerably compared to when all cues were present. 

Experiment 1

In Experiment 1, observers rated 30 material attributes for a range of soft and hard nonrigid substances with different mechanical properties that caused them to break apart differently when dropped onto the ground (see Methods). The substances were computer animated and rendered with various optical properties (Experiment 1a) and as point-light versions (Experiment 1b). 

Thirty adjectives were chosen that could be used to describe the substances in our experiments (see Table 1). Adjectives were collected mostly from the word lists of Fleming et al. (2013) and in Guest et al. (2011), which list candidate words for describing materials both visually and haptically. From these extensive lists and a few additions from our own brainstorming session, we narrowed the choice to a subset of attributes that we thought would apply to the stimuli. We conceptually divided attributes into three categories: optical, motion, and inferred attributes (see Table 1). Attributes were categorized as optical if we thought they could not be judged for the point-light stimuli (these attributes were excluded in Experiment 1b). For example, glossiness and transparency are surface qualities that would be impossible to rate for point-light displays, which do not have surfaces. Note that some attributes like smooth may be not only optical but tactile as well; we categorize them as optical here because to judge them visually, one would need surface information. Motion attributes specifically asked about the motion of the substances—for example, “How much does this substance look like it is crumbling?” The remaining attributes were categorized as inferred. These attributes were not necessarily directly perceivable like other qualities such as glossiness, but could nevertheless be derived from shape, motion, or surface optical information and probe perceptual differences in material qualities between the substances. 

Table 1

 

Material attributes that were rated in the experiments, organized by optical, motion, and inferred attributes. Thirty attributes were rated in Experiment 1 (all except for wobbling, which replaced jiggling/wiggling in Experiment 2). The 15 bolded adjectives were used in Experiment 2.

 

Methods

Apparatus and stimuli

Stimuli were presented on a Sony OLED monitor running at a refresh rate of 120 Hz with a resolution of 1,024 × 768 pixels controlled by a Dell computer running Windows 7. Stimulus presentation and data collection were controlled by a MATLAB script (release 2015a; MathWorks, Natick, MA) using the Psychophysics Toolbox (Brainard, 1997). Stimuli were viewed in a dark room at a viewing distance of approximately 60 cm. The only source of light was the monitor that displayed the stimuli. 

The stimuli were novel computer animations of cubes made of substances that broke apart in various ways when dropped onto the ground (see Supplementary Movie S2). The stimuli were modeled and rendered in the open-source software Blender (v. 2.7), using the Molecular Script¹ and Cube Surfer² add-ons made by Jean-Francois Gallant (2013). The Molecular Script allows particles to be generated that are linked together with “springs” that can stretch and break. Various linking parameters can be set, which affect factors like how stiff, dampened, and breakable the linking springs are (for more details, see http://pyroevil.com/molecular-script-docs/). Three substances were animated to resemble different levels of stiffness: (a) an elastic soft body that broke with relatively low stiffness (Supplementary Movie S2, top row); (b) an elastic semisoft body that broke and behaved more stiffly than the soft body (Supplementary Movie S2, middle row); and (c) a stiff hard body that broke and did not wobble (Supplementary Movie S2, bottom row). Each substance has certain mechanical properties that could be revealed through shape or motion cues. Note that there is no correspondence between the particle linking parameters and physics (see General discussion). Appendix A provides more details about how the stimuli were animated. 

The Cube Surfer add-on allows the particles to be surrounded by a mesh, which creates a surface that can be rendered with various optical properties (for more details, see http://pyroevil.com/cubesurfer-documentation/). Each animation was rendered with three surfaces: an opaque matte surface (“matte”; Figure 1, third column), a transparent glossy surface (“glassy”; Figure 1, first column), and a mixed-optics surface that was semitransparent and semiglossy (Figure 1, second column). Each animation was also rendered as point-light displays (see Supplementary Movie S3 and the last column of Figure 1). In these displays, the particles were not covered by a surface; rather, random particles were lit up using the emission material in Blender. Different numbers of particles were lit up depending on the condition (low, mid, and high density; see Table A2). Each animation was rendered from right camera angles. Each camera angle had equal elevation but a different azimuth (0°, 45°, 90°, 135°, 180°, 225°, 270°, or 315°). Appendix A provides details about the optical and point-light rendering parameters. 

Figure 1

 

A frame of each animation after the substance impacts the ground. Each row shows a different substance type (soft, semisoft, and hard body). The first three columns show the different optical properties in Experiments 1a and 2. The last column shows point-light stimuli in Experiments 1b and 2 (mid dot density).

The animations were rendered with the Cycles render engine in Blender, which is an unbiased, physically based path-tracing engine designed for animations. All scenes were illuminated by the “campus” light field from the Debevec Light Probe Image Gallery (Debevec, 1998), and frames were rendered twice as large as required and antialiased, resulting in movies with dimensions of 512 × 512 pixels. 

Procedure

Task

In each trial, observers watched a movie of a cube falling and breaking apart on the ground. The ground was not visible, and the movie was presented against a black background. The task was to rate the substance on a particular attribute, such as hardness, softness, glossiness, etc. (see Table 1), by moving the mouse vertically to adjust the level of a bar on the right side of the screen (see Figure 2). Trials were blocked by attribute, meaning that observers rated a particular attribute for all substances before moving onto the next attribute. 

Figure 2

 

An example of a trial. Note that this figure is not to scale. See text for details.

Instructions

Before each block, observers saw an instruction screen for the specific attribute they would be rating. For example, for ratings of hardness the instructions were, “Rate how HARD each substance looks. A setting of zero means not at all hard. A setting of Max means extremely hard.” Instructions for most ratings followed this format, except for the motion attributes, where the wording was changed slightly to emphasize movement. For example, for ratings of crumbling the instructions were, “Rate how much each substance looks like it is CRUMBLING. A setting of zero means it does not look like crumbling. A setting of Max means it looks extremely like crumbling.” 

Trial layout

Each block started with the adjective (e.g., HARD) displayed at the top of the screen and the rating bar (2.01° × 15.33°) displayed on the right side of the screen (see Figure 2). The adjective and rating bar remained on the screen for the duration of the block (i.e., they did not disappear between trials). The height of the rating bar directly corresponded to the vertical position of the (invisible) mouse curser, so each trial began with the rating bar at the height of the previous setting. The first trial started after 2 s, with a fixation square (36 arcmin) appearing in the center of the screen and lasting 750 ms, followed by the movie (25.36°) of the falling cube (approximately 8.5°–17°), which played once and lasted 1.7 s (39 frames, 24 frames/s) before disappearing. The trial ended when the observer pressed the space bar to set their rating. As soon as the space bar was pressed, a new fixation square appeared and the next trial commenced. 

Conditions

There were two stimulus-type conditions: full-cue (shape, motion, and optical information all present; Experiment 1a), and point-light (no shape or optical information present; Experiment 1b). Experiment 1a (full-cue stimuli) had three optical conditions (glassy, mixed-optics, and matte) and three substance-type conditions (soft body, semisoft body, and hard body). There were 30 blocks (one for each attribute in Table 1), resulting in a total of 270 trials for each observer in Experiment 1a. Experiment 1b (point-light stimuli) had three dot-density conditions (low, mid, and high density) and the same three substance-type conditions as Experiment 1a (soft body, semisoft body, and hard body). There were 23 blocks (we excluded the seven optical attributes), resulting in a total of 207 trials for each observer in Experiment 1b. The order of the blocks was randomized in each experiment, as were the trials within each block. Each animation was rendered from eight camera angles (azimuth 0°, 45°, 90°, 135°, 180° 225° 270° and 315°). The camera angle in each trial was randomly determined. 

Observers

Ten observers participated in Experiment 1a (full-cue stimuli) and 10 participated in Experiment 1b (point-light stimuli). Participants were research staff and PhD students in the psychology faculty at Justus-Liebig-University Giessen in Germany. All observers participated on a voluntary basis without compensation and had at least some experience with psychophysical experiments. The experiment was translated for German speakers. All participants were unaware of the aims of the study and had normal or corrected-to-normal vision. The experiment followed the guidelines set forth by the Declaration of Helsinki. 

Analyses

For Experiment 1a (full-cue stimuli), intersubject correlations and correlations between attributes were calculated to discover to what extent the attributes we chose were (a) sensible descriptions of the stimuli and (b) independent or related to one another. An exploratory factor analysis was performed based on the assumption that many material attributes seemed to tap into a few underlying common percepts. The factor analysis was performed on individual subjects' ratings (i.e., no averaging was done). The maximum-likelihood extraction method was used with orthogonal (varimax) rotation.³ Three factors were extracted based on the eigenvalues in the scree plot (Figure 5D). Next, mean ratings of each attribute were subject to a 3 (substance type: soft, semisoft, hard) × 3 (optical condition: glassy, mixed-optics, matte) within-subject ANOVA, to see how precisely ratings varied as a function of our manipulation of mechanical and optical properties. Within-subject ANOVAs were also performed on ratings in Experiment 1b (point-light stimuli), with dot density (low, mid, high) replacing the optical conditions. 

Experiment 1a results and discussion

Consistency across subjects

First, we examined the consistency across observers' ratings to see whether the adjectives were sensible descriptions of the stimuli. Figure 3 shows Pearson correlations between subjects in Experiment 1a (full-cue stimuli). Figure 3A shows intersubject correlations, with low correlations plotted as light blue (r = 0 would be white) and high correlations as dark blue. Figure 3B shows the distribution of the 45 correlation coefficients in Figure 3A. We also looked at the mean intersubject correlations separately for each attribute (Figure 3C). It is possible that observers interpreted the meaning of each attribute differently, which would lead to inconsistencies in the ratings between the different observers. However, if observers agreed on attribute ratings for the materials, then it suggests that the material attributes we chose to include appropriately describe the perceptual differences between the stimuli. 

Figure 3

 

Pearson correlations between observers in Experiment 1a (full-cue stimuli). (A) Inter-subject correlations (i.e., correlations between each subject and all other subjects). Correlation coefficient values are stated in each cell. All correlations are positive, ranging from 0.34 to 0.72, and significant at the level of p < 0.001. (B) Histogram of the correlation coefficients between all 10 observers across all full-cue stimuli and attributes tested. The histogram consists of the 45 correlation coefficients in (A). (C) Mean intersubject correlation coefficients for the 30 attributes.

All observers' ratings are substantially positively correlated, ranging from 0.34 to 0.72, and significant at the level of p < 0.0001 (Figure 3A and 3B). This is on par with previous studies (e.g., Fleming et al., 2013), and indicates that observers were overall rather consistent with each other in their rating of material attributes. Figure 3C shows that many attributes are rated highly consistently (e.g., glossy, matte, transparent, runny, hard, soft, liquidy/fluid, solid, gelatinous, mushy). However, some attributes, namely rubbery, fragile/brittle, bouncy/springy, and shattering, show hardly any consistency. These results suggest that most of our attributes tapped into perceived differences between the stimuli, while only a few did not apply to our stimuli or were highly subjective. Note that these are also the attributes that showed no main effects or interactions between mechanical and optical properties (see ANOVAs). 

Correlations between attributes

Correlations between attributes were calculated to discover to what extent the attributes we chose were independent or related to one another. These correlations were performed across ratings for all subjects and stimuli (i.e., no averaging was done prior to calculating each correlation). Figure 4 shows correlations between attributes for the full-cue stimuli in Experiment 1a. Many attributes were highly positively or negatively correlated, ranging from almost 0.9 to −0.9, which indicates that they were not independent. In the following we discuss the relationships between and among optical, motion, and inferred attributes. 

Figure 4

 

Correlations between attributes in the full-cue condition of Experiment 1a. Only significant correlations are shown. Blue and red squares indicate positive and negative correlations, respectively. The more saturated the color, the stronger the correlation. These correlations complement the results of the factor analysis: Attributes that load similarly or oppositely onto a factor (see Figure 5) are significantly positively or negatively correlated with one another, respectively.

Figure 5

 

Factor analysis of attribute ratings in Experiment 1a: Factor loadings of attributes onto the first three factors (filled blue circles in A–C), and initial eigenvalues before extraction of factors (D). The initial eigenvalues show that three principal components would account for 62.27% of the total variance, and eight principal components would account for 82.16%. Based on these eigenvalues, three factors were extracted. The three-factor solution (A–C) is responsible for the common variance constituting 58.16% of the total variance: Factor 1 explains 30.49% of the total variance, Factor 2 explains 15.72%, and Factor 3 explains 11.95%.

Correlations with optical attributes

Ratings of most optical attributes correlated positively or negatively with one another (Figure 4, top left). Ratings of glossy, transparent, and smooth positively correlated with one another, as did ratings of matte, opaque, and frosted. The former and latter attributes were negatively correlated. Optical attribute ratings were not completely independent from motion or inferred attributes (Figure 4, middle and bottom left). For example, stimuli that looked glossy, transparent, and smooth also tended to appear gelatinous and wet. Opaque, frosted stimuli tended to look spongy, airy, and fluffy. Interestingly, ratings of gritty correlated mostly with inferred and motion attributes: Substances that looked gritty tended to also look crumbly, solid, dry, hard, heavy, dense, and unbreakable, and not jiggling/wiggling, fluffy, lightweight, mushy, soft, gelatinous, and wet. 

Correlations with motion attributes

Motion attributes only somewhat correlated with one another (Figure 4, center), and correlated more with inferred attributes (Figure 4, bottom middle plot). For example, ratings of runny and jiggling/wiggling correlated positively with liquidy/fluid, wet, gelatinous, soft, mushy, lightweight, fluffy, airy, and spongy, and negatively with dense, heavy, hard, dry, and solid. In contrast, shattering and crumbling stimuli tended to also look dense, heavy, hard, and solid, and less fluffy, lightweight, mushy, soft, gelatinous, wet, and liquidy/fluid. 

Correlations with inferred attributes

Many inferred attributes were highly positively or negatively correlated with one another. For example, ratings of solid, dry, hard, heavy, dense, and unbreakable were all positively correlated, as were ratings of liquidy/fluid, wet, gelatinous, soft, mushy, lightweight, and fluffy, and the former attributes were negatively correlated with the latter attributes. 

Factor analysis

The high number of correlations between attributes indicates that they were not independent, and suggests the existence of underlying common dimensions (i.e., it suggests that observers were using the same underlying criteria to judge groups of adjectives). An exploratory factor analysis was performed on the attribute ratings, which allowed us to explain some of this covariation and helps to visualize how the stimuli are distributed in the perceptual feature space of material attributes. Figure 5 shows the results of the factor analysis. Note that we excluded four attributes from the analysis—rubbery, fragile/brittle, breaking, and shattering—because they showed such low intersubject correlations. However, the solution would be almost identical if we did include these attributes (see Supplementary Figure S1). The initial eigenvalues in Figure 5D show that, before factor extraction, three principal components would account for 62.27% of the total variance, and eight principal components would account for 82.16%. Based on these eigenvalues, three factors were extracted (the amount of extra variance explained by each factor after the third factor would be small). The three-factor solution (shown in Figure 5A–5C) is responsible for the common variance constituting 58.16% of the total variance: Factor 1 explains 30.49% of the total variance, Factor 2 explains 15.72%, and Factor 3 explains 11.95%. This is comparable to previous studies. For example, Fleming et al. (2013) conducted a principal-components analysis on 42 attribute ratings for five material classes using individual subjects' data (like in the present study), and found that seven components explained 50% of the total variance; in our study seven components explain nearly 80% of the total variance.  

Figure 5A–5C plots loadings on the first three factors against each other. This exploratory factor analysis allows for inferences about potential underlying (latent) dimensions that are captured by our attributes (i.e., the common percepts that the adjectives are tapping into). Note that our stimulus set was small (as was our sample size), and in the following we do not presume to make any claims about material dimensions in general (confirming this would be the role of future studies); our purpose here is to simplify interpretation of the perceptual differences between our stimuli, and to determine which of the properties we manipulated (i.e., mechanical and optical properties) affected these perceptual differences. 

The factor loadings of attributes on the first two factors (blue circles in Figure 5A) show that Factor 1 is strongly positively correlated with soft, wet, mushy, and gelatinous, and negatively with hard, dry, crumbling, gritty, and solid. Factor 2 is strongly positively correlated with matte and opaque and negatively with glossy and transparent. We suggest that Factor 1 might represent variations in perceived softness and hydration, contrasting soft wet materials with hard dry materials, and that Factor 2 reflects changes in optical appearance, contrasting matte opaque surfaces with glossy transparent ones. 

The mean factor scores for each experimental condition are shown in Figure 6. Figure 6A shows that substances with different mechanical properties are organized along Factor 1 (circles = soft, triangles = semisoft, and squares = hard substances), and that the stimuli with different optical properties are organized along Factor 2 (dark symbols = matte, medium = mixed-optics, light = glassy). Thus, it appears that ratings of the attributes were tapping into a few underlying perceptual dimensions, the first two of which reflect the manipulated mechanical and optical material properties. Figure 7 emphasizes this, showing the mean ratings of each attribute for each stimulus, organized by loadings onto Factor 1 (Figure 7A) and Factor 2 (Figure 7B). The vertical light-to-dark (or dark-to-light) gradients show how attributes that load strongly onto Factor 1 and Factor 2 have ratings that vary systematically with mechanical and optical properties, respectively. 

Figure 6

 

Mean factor scores for each substance type and optical condition plotted in the factor space (A–B), and separately for each factor (C). Light-colored symbols/bars are glossy transparent stimuli, dark-colored symbols are matte opaque stimuli, and the medium shade represents mixed-optics stimuli; circles in (A–B) are soft substances, triangles are semisoft substances, and squares are hard substances. Black lines connect soft, semisoft- and hard stimuli, and glassy, mixed-optics, and opaque stimuli. Error bars in (C) are standard error of the mean.

Figure 7

 

Mean ratings of each attribute for each stimulus, organized by loadings onto Factor 1 (A) and Factor 2 (B). Darker squares indicate higher ratings. The vertical light-to-dark (or dark-to-light) gradients show how attributes that load strongly onto Factor 1 and Factor 2 have ratings that vary systematically with mechanical (A) and optical (B) properties, respectively.

It is also interesting to look at loadings onto Factor 3 (Figure 6B and 6C). Factor 3 contrasts positive loadings on runny, liquidy/fluid, spongy, mushy, and airy against negative loadings on solid, dense, and hard. We suggest that Factor 3 may reflect changes in how the substances break apart, specifically the fluidity of motion, ranging from runny, more fluid motion of softer materials through the jiggling/wiggling motion (still somewhat fluid) of firmer gelatinous materials to the solid motion of harder, denser materials. The different substance types are organized along Factor 3, similar to Factor 1, suggesting that fluidity of motion was predominantly affected by mechanical properties. However, there does appear to be an influence of optics: The glassy soft and semisoft substances (light circle and triangle in Figure 6B) are positioned close to one other on Factor 3 (i.e., they look similar in fluidity), whereas the same substances with mixed optics (orange circle and triangle) are distributed differently on Factor 3 (i.e., they differ in perceived fluidity). It is interesting to note that this reflects the interaction between optical and mechanical properties for ratings of runny and liquidy/fluid (see next section; Figure 8), which have the highest positive loading on Factor 3. 

Figure 8

 

Mean ratings for attributes in Experiment 1a (full-cue stimuli). Each plot shows the mean ratings of a particular attribute (glossy, transparent, etc.), for each of the nine stimuli (3 substance types × 3 optical conditions). Darker squares represent higher ratings. Within each plot, substance type varies from top to bottom (top = soft, middle = semisoft, bottom = hard). Optical condition varies from left to right (left = glassy, middle = mixed-optics, right = matte). The red arrows show the main effects of substance type (vertical) and optical condition (horizontal). Yellow circles indicate an interaction between substance type and optical condition. Plots are ordered by their main effects and interactions. The first row shows attribute ratings with main effects of optics; the second and third rows show main effects of substance type; the fourth row shows both main effects; the fifth row shows interactions between substance type and optical condition; the last row shows no effects. Attributes we categorized as optical are colored orange, motion attributes are green, and inferred attributes are blue.

ANOVAs

Two-factor within-subject ANOVAs were performed on ratings of each attribute to see how precisely ratings varied as a function of our manipulation of mechanical and optical properties. The results of the ANOVAs are presented in Figure 8, where each plot shows the mean ratings of a particular attribute (glossy, hard, heavy, etc.), for each of the nine stimuli (3 substance types × 3 optical conditions in the full-cue condition). Darker squares represent higher ratings. Within each plot, substance type varies from top to bottom (top = soft, middle = semisoft, bottom = hard) and optical condition varies from left to right (left = glassy, middle = mixed-optics, right = matte). The ANOVA results are plotted over the data in Figure 8. Main effects within each plot are represented by vertical and horizontal red arrows (or lines in some cases), and interactions are represented by yellow circles in the top left of the plot. F values, significance levels, and degrees of freedom for each ANOVA are shown in Table B1 in Appendix B. 

ANOVA results: Optical and motion attributes

First, we consider main effects for attributes that we labeled optical and motion attributes (orange and green attributes in Figure 8, respectively). There were main effects of optical condition (horizontal red arrows) for all except one of the optical attributes. That is, the attributes we thought would predominantly be affected by varying optical properties were indeed affected by this manipulation. Glassier stimuli were rated as glossier, smoother, and more transparent, averaging across substance type. More matte stimuli were rated as more matte, opaque, and frosted, averaging across substance type. These results act as a sanity check for our method, as glossiness and transparency were the optical properties that we experimentally manipulated, and these manipulations are reflected in the data. 

There were main effects of substance type (vertical red arrows/lines) for four out of six of the motion attributes (colored green in Figure 8). That is, ratings of attributes that emphasized the motion of the substances were mostly affected by our manipulation of substance type. Softer substances looked runnier and less like they were crumbling compared to harder substances, averaging across optical condition. The semisoft substance looked more like it was jiggling/wiggling and bouncy/springy compared to the soft and hard substances, averaging across optical condition (these effects are represented by a red line with no arrowhead). Two motion attributes were not affected by any of our manipulations: All stimuli looked equally like they were breaking and shattering (there is a slight trend suggesting that harder substances looked more like they were shattering, though this effect was not significant). These null effects are not entirely surprising, given that all substances broke apart on impact with the ground and had similar amounts of breakage. 

The main effects reported here fit with our categorization of optical and motion attributes. There were also some surprising main effects and interactions that went against our expectations. Smooth and gritty were optical attributes that were affected by substance type (colored orange in Figure 8). Ratings of gritty were higher for harder substances and were not affected by optical condition at all. It is possible that the optical properties we chose did not affect perceived grittiness, or that the motion cues were very strong and dominated any optical cues (this is discussed more in Experiment 2). There was a significant interaction between substance type and optical condition for ratings of smooth, and an unexpected (but subtle) interaction between substance type and optical condition for ratings of glossy. We elaborate on these interactions in Appendix B. 

Ratings for some of the motion attributes were affected by optical condition, which was also surprising (colored green in Figure 8). Matte stimuli looked more like they were crumbling compared to glassier stimuli, averaging across substance type. This could be because crumbling is associated with substances that are dry and matte like dirt (wet surfaces are almost never matte), whereas soft glossy surfaces are usually wet and therefore will not crumble. There was an interaction between substance type and optical condition for ratings of runny. Again, this is discussed further in Appendix B. 

ANOVA results: Inferred attributes

The results for the 17 inferred attributes are colored blue in Figure 8. Just like the optical and motion attributes, ratings for inferred attributes were affected by both mechanical and optical properties, and sometimes an interaction between the two. Seven attributes were affected only by mechanical properties: Harder substances were rated as harder, denser, and more solid; softer substances were rated as fluffier, mushier, softer, and more lightweight (main effect of substance type). Spongy was the only inferred attribute affected by optical condition and not substance type: Matte stimuli looked spongier than glassier stimuli (main effect of optics). Ratings of rubbery and fragile/brittle were affected by neither mechanical nor optical properties (all stimuli were rated the same). The remaining seven attributes were affected by both mechanical and optical properties, or an interaction between the two. Softer, glassier substances were rated as wetter and more gelatinous; harder, more matte surfaces were rated as dryer; softer, more matte substances were rated as airier; and harder, mixed-optics surfaces were rated as more unbreakable. Results for liquidy/fluid were very similar to ratings of runny and wet. It appears that mechanical information is a powerful cue to a substance having liquid properties, but in some—perhaps ambiguous—cases, the visual system also has to rely on optical cues. 

Experiment 1b results and discussion

Consistency across subjects

Figure 9 shows Pearson correlations between subjects in Experiment 1b (point-light stimuli). Figure 9A shows intersubject correlations, with low correlations plotted as light blue (r = 0 would be white) and high correlations as dark blue. Figure 9B shows the distribution of the 45 correlation coefficients in Figure 9A. Figure 9C shows the mean intersubject correlations separately for each attribute, comparing the point-light stimuli (light-gray bars) with the full-cue stimuli (dark-gray bars). 

Figure 9

 

Pearson correlations between observers in Experiment 1b (point-light stimuli). (A) Intersubject correlations (i.e., correlations between each subject and all other subjects). Correlation coefficient values are stated in each cell. All correlations are positive, and 43 out of 45 are significant at the level of p < 0.05 (42 are significant at the level of p < 0.01, and 39 at the level of p < 0.001). (B) Histogram of the correlation coefficients between all 10 observers across all point-light stimuli and attributes tested. The histogram consists of the 45 correlation coefficients in (A). (C) Mean intersubject correlation coefficients for the 23 attributes tested in the point-light condition. The full-cue condition is included for comparison: Mean intersubject correlations are higher in the full-cue versus the point-light condition for 18 out of 23 attributes (dark-gray bars vs. light-gray bars, respectively).

Similar to Experment 1a, all correlations are positive, and 43 out of 45 are significant at the level of p < 0.05 (42 are significant at the level of p < 0.01, and 39 at the level of p < 0.001; Figure 9A and 9B). Figure 9C shows that mean intersubject correlations are higher in the full-cue versus the point-light condition for 18 out of 23 attributes. However, observers are still overall rather consistent with each other in their rating of material attributes for the point-light stimuli, despite inherent ambiguity. 

Effects of mechanical properties

The ANOVA results of Experiment 1b (point-light stimuli) are presented in Figure 10. In these plots, substance type varies from top to bottom (top = soft, middle = semisoft, bottom = hard) and dot density varies from left to right (left = high, middle = mid, right = low). For the 14 main effects of substance type for inferred attributes in the full-cue condition (colored blue in Figure 8), all but lightweight were also present for the point-light stimuli in Experiment 1b (colored blue in Figure 10). For both full-cue and point-light stimuli, softer substances looked wetter, fluffier, mushier, softer, airier, and more liquidy/fluid and gelatinous. Harder substances looked heavier, harder, denser, dryer, and more solid and unbreakable. It is striking that such impoverished stimuli can portray a large amount of information about material properties from motion cues alone. However, note that for most attributes these effects of substance type were larger for full-cue stimuli. This is investigated further in Experiment 2. 

Figure 10

 

Mean ratings for attributes in Experiment 1b (point-light stimuli). Each plot shows the mean ratings of a particular attribute (liquidy, gelatinous, etc.), for each of the nine stimuli (3 substance types × 3 dot densities). Darker squares represent higher ratings. Within each plot, substance type varies from top to bottom (top = soft, middle = semisoft, bottom = hard). Dot density varies from left to right (left = high, middle = mid, right = low). The red arrows show the main effects of substance type (vertical) and dot density (horizontal). Yellow circles indicate an interaction between substance type and dot density. Plots are ordered by their main effects and interactions. The first two rows show main effects of substance type; the third row shows main effects of density (or both main effects); the fourth row shows interactions between substance type and dot density; the last row shows no effects. Attributes we categorized as motion attributes are coloured green, and inferred attributes are blue.

This result is very interesting considering that for the motion attributes, only three out of six had comparable results in the full-cue and point-light conditions (compare green-colored attributes in Figures 8 and 10). For motion attributes, ratings of runny and jiggling/wiggling were similar between the full-cue and point-light stimuli in that softer substances looked runnier than harder substances, and semisoft substances looked most like they were jiggling/wiggling, averaging across optical condition and dot density. Interestingly, main effects of substance type for ratings of crumbling and bouncy/springy were eliminated for the point-light stimuli. It seems that the presence of a surface was required to see differences in these attributes. Although there were no differences in how much the full-cue stimuli looked like they were breaking, there were differences for the point-light stimuli; softer substances looked more like they were breaking than harder substances. This might be because only a few dots revealed the breaking parts of the hard substance, even for the high-density dot stimulus (see Supplementary Movie S3). 

Effects of dot density

For motion attributes (colored green in Figure 10), there were interactions between substance type and dot density for ratings of runny and jiggling/wiggling. Follow-up tests revealed that dot density did not affect how runny the hard and semisoft substances looked, but did affect how runny soft substances looked; the soft substance looked runnier at mid compared to low dot density, t(36) = 3.116, p = 0.0036. High and mid dot densities made the semisoft substance look more like it was jiggling/wiggling compared to low dot density. Thus, it appears that high and mid dot densities provided more information than low dot density. This is interesting because jiggling/wiggling is an attribute that is likely to rely on finer scale motion information (e.g., differences between adjacent parts of the substance), which seems to be revealed only when more dots are present. Similarly, ratings of shattering, breaking, and crumbling depended on dot density, with higher ratings for high and mid dot densities compared to low density. These types of motion also seemed be amplified with more dots and masked with fewer dots. 

Some inferred attributes were affected by dot density (colored blue in Figure 10). Substances looked wetter, fluffier, and mushier when presented at mid dot density compared to high and low dot densities, averaged across substance type. There were interactions between substance type and dot density for ratings of wet and lightweight. Follow-up tests revealed that for the two softer substances, mid dot-density stimuli looked wetter than low dot-density stimuli, p < 0.0179,⁴ but hard substances looked equally not wet, regardless of dot density. In contrast, the two softer substances looked equally lightweight, regardless of dot density, whereas the hard substance looked lighter for low versus high dot density (i.e., when there were fewer dots, the substance looked lighter), t(36) = 3.274, p = 0.0023. 

These results suggest that a finer level of motion detail is required to rate attributes that focus on motion, through either the presence of a surface or higher dot density (indeed, point-light stimuli that did reveal effects of substance type depended on dot density). In contrast, judgments of inferred properties that were affected by substance type may have relied on larger scale motion information that was present in the point-light stimuli. Taken together, the results from Experiment 1b suggest that point-light stimuli can provide a lot of information about many material attributes, but surface optics do provide additional information for many attributes. Mid and high dot-density stimuli tended to provide more information than low dot-density stimuli, with perhaps the mid density being superior to high density. Experiment 2 directly compares how mechanical properties influence attribute ratings in the full-cue versus mid-density point-light stimuli. 

Experiment 2

Experiment 1a showed that manipulating both mechanical and optical properties influenced judgments of material attributes, and that there was an interaction between the two for some attribute ratings. Experiment 1b (point-light stimuli) showed that motion cues alone provided a lot of information about some material attributes (particularly the inferred attributes), but the presence of a surface in the full-cue condition seemed to provide more information (the main effects of substance type were more pronounced). In Experiment 2 we formally compare ratings of different substance types between full-cue and point-light stimuli (we chose mid dot density based on the results of Experiment 1). 

It is possible that observers in the full-cue condition of Experiment 1 could have obtained a lot of information about the material from shape cues. Therefore, we also investigate how much shape contributed to ratings of the full-cue stimuli in Experiment 1 by comparing ratings when observers were shown one frame of the animation after the point of impact versus a short movie clip around the point of impact. 

Observers rated the 15 bolded attributes in Table 1. We reduced the number of attributes based on the results of Experiment 1, which showed that some attributes were redundant (e.g., matte, soft, and dry showed the opposite pattern of data to glossy, hard, and wet, respectively) and others were not affected by our mechanical and optical manipulations (e.g., rubbery and fragile/brittle). The attributes that remained were the ones we found the most interesting to explore further (note that we added wobbling as an attribute because we thought it described the motion of the softer substances well). 

Methods

Stimuli and procedure

Trial layout was the same as in Experiment 1, except that the movies were shortened to 23 frames around the point of impact (Frames 5–27, 958 ms). Static images were Frame 16 of the movies (see Supplementary Figure S2). 

Stimuli were the same (albeit shortened) animations used in Experiment 1. There were three stimulus-type conditions (full-cue, point-light, and static frame). All conditions had the same three substance-type conditions as Experiment 1 (soft body, semisoft body, and hard body). The full-cue and static images conditions had the same three optical conditions as Experiment 1 (glassy, mixed-optics, and matte). There were 15 blocks in these conditions (one for each bolded attribute in Table 1), resulting in a total of 135 trials for each observer. The point-light condition contained only the mid dot-density condition from Experiment 1. There were 13 blocks in this condition (we excluded the two optical attributes), resulting in a total of 117 trials for each observer. We also decided to include a practice block before the experimental blocks, where observers rated how much each substance looked like it was breaking. This was to familiarize them with the range of stimuli before the real trials. However, this block was not analyzed. 

Observers

We increased the sample size to 60 observers in Experiment 2. The same 30 observers participated in the full-cue and point-light conditions (the order of conditions was counterbalanced), and a different set of 30 observers participated in the static-frame condition. Participants were students at Justus-Liebig-University Giessen in Germany, and were compensated 4 euros per half hour, which was the approximate length of the experiment. In Experiment 1 we noticed some problems for second-language English and German speakers understanding some of the adjectives in Table 1, even for fluent speakers. To avoid this problem, we recruited only native-level speakers of English or German for Experiment 2. 

Results and discussion

Analyses

First, we compared the results of the full-cue (movie) and static-frame conditions. We compared the consistency of observers in each condition, and performed a factor analysis including ratings from both full-cue and static conditions. This way, average factor scores of each stimulus can be directly compared between the movie and static conditions. To simplify analyses, we used the factor scores to compare the movie and static conditions, and not the individual attribute ratings. Experiment 1a showed that the main effects and interactions of individual attribute ratings were reflected in how the factor scores were arranged in the factor space (we also separately verified this for Experiment 2, but for the purpose of simplicity do not report it here). To this end, mean factor scores of each factor were subjected to a 2 (stimulus condition: full-cue, static) × 3 (substance type: soft, semisoft, hard) × 3 (optical condition: glassy, mixed-optics, matte) ANOVA (i.e., three factors, with stimulus condition as a between-subjects factor and substance type and optical condition as within-subject factors). 

Next we compared the results of the full-cue and point-light conditions. Mean ratings of each attribute were subjected to a 2 (stimulus condition: full-cue, point-light) × 3 (substance type: soft, semisoft, hard) ANOVA (which averaged across optical condition for the full-cue stimuli). 

Consistency across subjects

Figure 11 shows the mean intersubject correlations separately for each attribute, comparing the full-cue movie stimuli (dark-gray bars) and the static stimuli (striped bars). Intersubject correlations were higher in the full-cue (movie) versus the static condition for 10 out of 15 attributes. This suggests that some attributes were more ambiguous to rate when observers were shown only a static image—for example, liquidy/fluid and wobbling are attributes that might rely heavily on motion. Interestingly, some attributes, like lightweight, heavy, and crumbling have lower intersubject correlations compared to Experiment 1 even in the full-cue movie condition. A potential reason for this is that observers actually saw a shorter version of the animation in Experiment 2 compared to Experiment 1 (see Methods). The last 12 frames of the animation were not shown in the full-cue condition of Experiment 2, and perhaps seeing these end frames is important for observers to get a reliable impression of weight and crumbliness. 

Figure 11

 

Mean intersubject correlation coefficients for the 15 attributes tested in Experiment 2. Mean coefficients are higher in the full-cue versus the static condition for 10 out of 15 attributes (dark-gray solid bars vs. light-gray stripy bars, respectively).

Factor analysis

The results of the factor analysis are shown in Figures 12 and 13. The factor loadings of attributes on the first two factors (blue circles in Figure 12A) show that Factor 1 is strongly positively correlated with wobbling, mushy, wet, gelatinous, and liquidy/fluid, and negatively with hard. This is similar to Experiment 1a, where we suggested that Factor 1 represents variations in perceived softness and hydration. However, the fluidness of motion (liquidy/fluid vs. crumbling) is also captured by this first dimension in Experiment 2. Factor 2 is positively correlated with airy, fluffy, spongy, lightweight, and crumbling, and negatively with dense and heavy. Factor 3 contrasts smooth, glossy, and hard things against mushy things. Based on this, we refer to Factor 1 as a “hydration and fluidness” dimension that contrasts wet and fluid stuff against dry and crumbling stuff; Factor 2 is an “airiness/density” dimension; and Factor 3 is a “smoothness and hardness” dimension that contrasts smooth and hard things against rougher and softer substances. 

Figure 12

 

Factor analysis of attribute ratings in the full-cue and static-frame conditions of Experiment 2. The attribute loadings onto the first three factors are shown in (A–B). The three-factor solution is responsible for the common variance constituting 41.99% of the total variance: Factor 1 explains 20.69% of the total variance, Factor 2 explains 11.48%, and Factor 3 explains 9.82%. (C–D): Mean factor scores for each substance type and optical condition plotted in the factor space. Light-colored symbols are glossy transparent stimuli, dark-colored symbols are matte opaque stimuli, and the medium shade represents mixed-optics stimuli. The circles are soft substances, triangles are semisoft substances, and squares are hard substances. Black lines connect soft, semisoft, and hard stimuli, and glassy, mixed-optics, and matte stimuli.

Figure 13

 

Mean factor scores for each stimulus condition (columns), plotted separately for each factor (rows). The bars within each plot show the scores for each stimulus (substance type and optical conditions). Error bars are standard error of the mean.

The attribute loadings differ somewhat between Experiments 1 and 2. We propose two possible reasons for this, which are elaborated on further in the General discussion. First, we removed many of the attributes in Experiment 2, including most of the ones describing the optical properties of the stimuli. This means that differences purely in perceived optical properties (which was a factor of its own in Experiment 1) do not get as much weight in Experiment 2. Second, the stimuli were different in Experiment 2 and included both static and dynamic stimuli. Recall that the movies had a shorter duration compared to Experiment 1 (observers in Experiment 2 did not see the end of the movie). As noted in the previous section, attributes like liquidy/fluid might rely heavily on motion, and will therefore load differently onto the factors if motion information is changed. 

Figure 12C and 12D plots the average factor scores for the full-cue (movie) and static conditions, showing how the stimuli are arranged in the factor space for the first two factors. Figure 13 plots these scores separately for each factor. The first thing to notice is that dynamic and static stimuli are arranged differently in the factor space, especially for the first two factors. For example, differences between the glossy stimuli (light-colored points) are more pronounced for the dynamic stimuli (Figure 12C) compared to the static stimuli (Figure 12D), which are clustered together in the space. In the next section, we examine the relationship between the stimuli in the factor space in more detail. 

ANOVAs on factor scores

Below we report the results of the 2 (stimulus condition: movie, static) × 3 (substance type: soft, semisoft, hard) × 3 (optical condition: glassy, mixed-optics, matte) ANOVAs for each factor. 

Factor 1: Hydration and fluidness

Figure 13A and 13B shows that for both static and movie stimuli, glossier substances looked wetter and more fluid than matte stimuli, which looked dryer and crumblier—main effect of optical condition: F(2, 116) = 247.8, p < 0.001. Mechanically softer stimuli also looked wetter and more fluid—main effect of substance type: F(2, 116) = 55.4, p < 0.001. Importantly, this difference in hydration/fluidness between mechanically soft and hard stimuli was enhanced with motion—interaction between substance type and stimulus condition: F(2, 116) = 31.3, p < 0.001. There was also an interaction between substance type and optical condition, F(4, 232) = 18.7, p < 0.001, but this interaction differed for movies and static stimuli—three-way interaction between substance type, optical condition, and stimulus condition: F(4, 232) = 4.45, p = 0.002. This three-way interaction will be discussed later. 

Factor 2: Airiness/density

Figure 13C and 13D shows that glossier stimuli looked denser and matte stimuli looked airier—main effect of optics: F(2, 116) = 35.44, p < 0.001. Mechanically softer substances also looked airier—main effect of substance type: F(2, 116) = 11.98, p < 0.001. There was a three-way interaction between substance type, optical condition, and stimulus condition, F(4, 232) = 3.231, p = 0.013, which is discussed in the following. 

Three-way interactions

The three-way interactions between substance type, optical condition, and stimulus condition that were found for Factor 1 (hydration and fluidness) and Factor 2 (airiness/density) are nicely visualized in Figure 12C and 12D. These plots show a number of things. First, softer and more matte stimuli were perceived as airier when shown dynamically versus statically (seen by the rightward shift of points along Factor 2 in Figure 12C vs. 12D). Second, glossy stimuli differed on both factors when shown dynamically (Figure 12C, light-orange points); softer substances looked wetter and more fluid, whereas the hard substance looked denser and not at all fluid. In contrast, there were no such differences for the static stimuli (Figure 12D, light-orange points); all glossy stimuli looked dense and not very wet or fluid. The third thing to notice is that in dynamic scenes (Figure 12C), semisoft substances (triangles) differed more on both factors compared to the soft and hard substances (circles and squares, respectively). In static scenes (Figure 12D), differences between optical conditions are more noticeable: The mixed-optics stimuli (middle orange points) showed larger differences in hydration and fluidness compared to the glossy and matte stimuli (light- and dark-colored points, respectively). 

Factor 3: Smoothness and hardness

Figure 13E and 13F shows that for both static and moving stimuli, mechanically harder substances looked smoother and harder than mechanically soft substances—main effect of substance type: F(2, 116) = 57.83, p < 0.001. Glossier stimuli also looked smoother and harder than matte stimuli—main effect of optics: F(2, 116) = 53.10, p < 0.001. This difference in perceived hardness/smoothness between glossy and matte substances is enhanced for the static stimuli—interaction between optics and stimulus condition: F(2, 116) = 5.831, p = 0.004. In other words, adding motion attenuated perceptual differences caused by optics. 

Summary

Motion, shape, and surface optical cues all contributed to the perception of our nonrigid breaking materials. Furthermore, they contributed interactively. Motion provided unique mechanical information above static shape cues about how hydrated and fluid the materials looked (Factor 1; the differences between mechanically soft and hard substances were amplified for dynamic stimuli). Differences in hydration/fluidness between glossy and matte stimuli were also enhanced by motion. Motion also provided unique information about the airiness of the softer, more matte substances that was not revealed by optics and shape cues alone (Factor 2). Finally, motion dominated over information provided by static optical cues about the density and smoothness/hardness of glossy materials (Factor 3). 

Full-cue versus point-light stimuli

Figure 14 shows the mean attribute ratings for the point-light stimuli plotted against the mean ratings for the full-cue stimuli. Note that there were no optical conditions for the point-light stimuli, which is why ratings for the different optical conditions are identical on the x-axis. Ratings for each attribute in the point-light conditions were subjected to a one-way ANOVA comparing substance types (Table B2, rightmost column), which revealed main effects of substance type for all attributes except spongy. This again strikingly shows that observers can use motion information alone to infer differences in material attributes between substances. To compare full-cue versus point-light stimuli, mean ratings of each attribute were subjected to a 2 (stimulus condition: full-cue, point-light) × 3 (substance type: soft, semisoft, hard) ANOVA (which averaged across optical condition for the full-cue stimuli). F values, significance levels, and degrees of freedom are shown in Table B2 in Appendix B. The blue stars next to the attribute labels in Figure 14 indicate an interaction such that the difference between soft and hard substances is larger for full-cue versus point-light stimuli. This was the case for ratings of fluffy, mushy, wobbling, wet, and gelatinous. This suggests that motion alone provides a lot of information about these attributes, but the presence of a surface in the full-cue condition provided additional information that affected these material judgments. The red star for ratings of dense indicates an interaction such that the difference between soft and hard substances is larger for point-light versus full-cue stimuli. The red star next to crumbling indicates an interaction such that the difference between soft and hard substances is opposite for full-cue and point-light stimuli. For full-cue stimuli, harder substances looked more like they were crumbling, whereas for point-light stimuli, softer substances looked more like they were crumbling. Motion and surface optical cues clearly provide different information about the density and crumbliness of a material. 

Figure 14

 

Mean attribute ratings for the point-light stimuli plotted against mean ratings for the full-cue stimuli. Symbols are the same as in Figures 6 and 12: Light-colored symbols are glossy transparent stimuli, dark-colored symbols are matte opaque stimuli, and the medium shade represents mixed-optics stimuli; circles are soft substances, triangles are semisoft substances, and squares are hard substances. Points that fall on the diagonal line are stimuli that were rated the same for both full-cue and point-light conditions. Points above the line mean that point-light stimuli were underestimated (i.e., ratings were higher in the full-cue condition), and points below the line mean that point-light stimuli were overestimated (i.e., ratings were higher in the point-light condition).

General discussion

Summary of results

We presented exploratory experiments that sought to determine how manipulating intrinsic optical and mechanical properties influences the perception of nonrigid breaking materials when observers rate multiple material attributes. We found that manipulating optical and mechanical properties had an interactive influence on ratings of several material attributes. Interestingly, we found this for some attributes that we thought would be driven solely by mechanical motion (e.g., runny) or surface properties (e.g., smooth). One question that cannot be fully answered here is what these interactions mean. It is possible that judging some attributes like runny and wobbling relies predominantly on shape and motion cues (which varied with the mechanical properties manipulation), and that surface optical qualities like gloss affect 3-D shape (Marlow et al., 2012; Todd et al., 2004; Vangorp, Laurijssen, & Dutré, 2007) and motion (Doerschner, Yilmaz, Kucukoglu, & Fleming, 2013) in a bottom-up fashion. Alternatively, interactions could arise from learned associations between mechanical and optical properties, where certain combinations of these properties resemble familiar materials, and relational knowledge about those materials (either explicit or implicit) influences the rating task. Our experiments were not designed to tease apart these two options, though the interactions we observed could be a combination of both. For example, soft and semisoft substances looked more fluid when the material was glossy (compared to matte) in the movie condition of Experiment 2 (see Figure 12C, circles and triangles, respectively). This influence of optics was small for the soft substances, suggesting that perceived fluidity was enhanced in a bottom-up fashion by the additional motion of the specular highlights. A larger influence of optics was observed for the semisoft materials; it is likely that glossiness and translucency together with these shape and motion cues resembles jelly, which affects the associated qualities (hydration and fluidity) in a top-down fashion, whereas matteness is typically not associated with jelly and wet materials. 

We also investigated how the different sources of mechanical information—motion and shape cues—contributed to material appearance. We created point-light displays to isolate motion cues, and a static-frame condition to separate 2-D shape cues from motion cues (note that optical properties were still present). Our results suggest that motion and shape information have an interactive influence on material perception. Motion and static cues separately provided substantial information about material properties. Combining these cues (in the full-cue condition) affected material qualities in different ways. For some qualities, motion dominated over optical information—for example, motion attenuated differences between glossy and matte materials for qualities related to smoothness and density. Other times motion enhanced differences between glossy and matte materials, as was the case for qualities related to hydration and fluidness. Particularly interesting were cases where greater perceptual differences were observed for intermediate mechanical and optical conditions. For example, perceptual differences between matte and glossy substances were greater for the semisoft substances compared to the hard and soft substances in the movie condition. For static stimuli, larger differences in hydration and fluidity were observed for mixed-optics stimuli compared to the glossy and matte stimuli. One explanation is that the intermediate stimuli are potentially more ambiguous. Thus, observers might rely more on optical appearance and associations with material qualities for the semisoft moving stimuli (e.g., jellylike vs. crumbling) and more on shape cues for the static mixed-optics stimuli (wetness is often associated with glossy surfaces, and the mixed-optics stimuli are possibly quite ambiguous). This is similar to the idea that the visual system performs reliability-weighted cue combination—that is, it uses the cues that are most reliable (Schmidt et al., 2017). 

One caveat to these results is that we presented only one frame of the animation in the static condition, which limited the available shape information. It should be investigated further whether showing multiple static frames provides the same information as showing the full movie. 

The influence of optical versus mechanical properties

The interactions we found between optical and mechanical properties seem at odds with recent findings in the literature. Fleming and colleagues have investigated how judgments of material attributes are affected by optical and mechanical properties in liquids and soft bodies (Paulun et al., 2015; Paulun et al., 2017; Schmidt et al., 2017; van Assen & Fleming, 2016). Van Assen and Fleming (2016) showed observers movie clips of simulated pouring liquids that varied in viscosity and optical characteristics, and had them rate six physical attributes (runniness, shininess, sliminess, stickiness, warmth, and wetness). They found that some of these attributes, such as sliminess, depended on both mechanical (viscosity) and optical properties; for example, optical materials like green goo looked slimier than other materials, and intermediate viscosity ranges were associated with sliminess. However, although both optical and mechanical properties affected ratings of attributes, little interaction was found. Additionally, optical properties had no effect on observers' judgments of viscosity in a matching task; perceived viscosity was solely driven by mechanical cues. 

Liquids may be a special class of materials because they can behave in many ways and take on a wide range of shapes depending on the forces applied to them. Van Assen et al. (2016) found that cues related to curvature, periodic movements, and spread could predict viscosity judgments for pouring liquids but not for other types of liquid motion like stirring, raining, pushing, and smearing. Mechanical cues involved in the perception of other nonrigid materials, such as soft bodies, may be more reliable. Paulun et al. (2017) simulated soft elastic cubes that varied in both stiffness and optical appearance, and subjected them to forces that deformed them in different ways. Observers witnessed these deformations and rated perceived softness in each condition. The results suggested that perceived softness depends on how an object's shape changes (perturbation depth) in response to forces. They also found that perceived softness in static, unperturbed cubes was affected by surface appearance. However, when the cubes were animated (i.e., when the forces were applied), mechanical shape cues completely overrode optical appearance in the perception of softness. 

Neither van Assen and Fleming (2016) nor Paulun et al. (2017) found interactions between mechanical and optical properties for their stimuli and for the attributes that were rated. Although the stimulus set in the present study is small, it spans a range greater than those used in previous studies; van Assen and Fleming looked only at liquids, and Paulun et al. and Schmidt et al. (2017) looked only at a small range of smoothly deforming elastic and plastic soft bodies, respectively. This could contribute to the differences in results. Note, however, that van Assen and Fleming did find interactions in a liquid category-naming task. In the introduction we suggested that asking about categories taps into multiple qualities about stimuli (see Fleming et al., 2013). An advantage of using a multiattribute approach is that it helps to overcome semantic issues with looking at any single attribute. For example, there could be subtle differences in interpretation of the words, but by asking about multiple attributes we can filter out any variations in semantic interpretations of any single attribute. Furthermore, single attributes miss out on perceptual qualities not captured by that adjective. For example, Paulun et al. (2017) found that the same cube rendered with a metal-looking material and a rubbery material was rated equally soft when deformed by a rod. However, the judgments of softness missed out on the strange, hollow quality that some people perceived with the metal-looking material. We suggest that interactions might have emerged in previous ratings-based experiments if a larger number of attributes had been rated (and perhaps for a greater range of stimulus classes). For example, we found interactions for ratings of liquidy/fluid and runniness (similar to viscosity) for stimuli that ranged from soft bodies to hard bodies. 

Scope of the study: Choice of stimuli and attributes

A note on realism

Our stimuli were designed to be ambiguous and do not correspond to any particular materials in the real world (there is no correspondence between the particle linking parameters and physics). Nevertheless, the optical properties were compelling enough for observers to make extremely consistent judgments about the properties we manipulated (glossiness/matteness and transparency/opacity; see Results and Figure 3). The fact that observers could successfully and consistently rate different material attributes for these substances supports the idea that, for many observers, the stimuli could have resembled familiar materials. For example, the softer transparent substances could resemble wobbling jelly, and the hard substance cracking glass. Nevertheless, future studies should compare the results of computer-animated nonrigid breaking stimuli with real breaking materials in the world (see, e.g., Aliaga et al., 2015). 

What if different adjectives were used?

The specific set of attributes in this study is obviously not exhaustive; our aim was to include enough adjectives to explore how general perceptual differences between nonrigid materials were affected by optical properties, mechanical properties, or an interaction between the two. The results of the factor analyses suggested that observers used the same underlying criteria to judge groups of adjectives. In other words, the adjectives seemed to be tapping into a few underlying common percepts. Thus, our choice of adjectives was sufficient for our purposes. However, an important question is how the factor space would be affected by adding or subtracting attributes. After eliminating 15 attributes in Experiment 2, we did see differences in the arrangement of the factors. For example, removing most of the optical attributes eliminated the optics dimension as its own emerging factor. However, we do not see this as a problem. We emphasize that our aim was not to determine the cardinal axes of nonrigid material space, if such a thing even exists. To do so would require an exhaustive list of attributes for an extremely large stimulus set. The particular choice of stimuli and attributes would affect the exact loadings of adjectives onto the factors (as it did in Experiment 2). However, the important thing is how the stimuli relate to one another in the space. The attributes included in Experiment 2 were the ones that captured interactions between optical and mechanical properties. Note that in both experiments, despite the change in stimuli and adjectives, the factors still suggested the same underlying perceptual qualities, namely hydration, fluidness of motion, airiness/density, hardness/softness, and smoothness or optical qualities. 

How do these factors relate to previous studies that have looked at multiple material attributes?

Our factors are similar to those found in previous material perception studies that have run principal components analyses on ratings of multiple attributes (Baumgartner, Wiebel, & Gegenfurtner, 2013; Fleming et al., 2013; Nagai et al., 2015; Schmidt et al., 2017; van Assen & Fleming, 2016). Three of these studies (Baumgartner et al., 2013; Fleming et al., 2013; Nagai et al., 2015) had participants rate photographs or real stimuli of materials from a wide range of classes, including wood, metal, stone, fabric, and glass, and found that the first two components reflected differences in roughness and hardness between materials. Schmidt et al. (2017) and van Assen and Fleming (2016) used a smaller class of rendered stimuli. Van Assen and Fleming found that for ratings of animated liquids, runniness ratings were perpendicular to shininess ratings in the feature space. Schmidt et al. found that for ratings of plastic deformed soft bodies, the first two components reflected differences in softness versus heaviness, and crumbliness versus slipperiness/stickiness. These results are similar to what we found in our experiments. Interestingly, the latter is very similar to our hydration dimension, which contrasted wet and fluid materials against dry and crumbly materials. 

Visual versus semantic representations

The approach that we adopted in this study was to ask observers to directly judge or rate different attributes such as glossiness and transparency, which are usually considered to be directly perceivable, or fluffiness and airiness, which are more inferred (i.e., derived from associations with shape, motion, and/or optical appearance). This approach has been criticized by some researchers for relying on observers' ability to use language to describe material properties (e.g., Xiao, Bi, Jia, Wei, & Adelson, 2016). We do not disagree with this view; however, we believe that in the present context such an approach was both appropriate and very informative. This is because we did not focus on a particular attribute per se; rather, we measured ratings of many attributes as an exploratory way to probe general perceptual differences between materials. Furthermore, Fleming et al. (2013) found a high consistency between semantic representations of classes of material (e.g., stone, metal, wood, fabric) and ratings of attributes (e.g., glossiness, roughness) of individual members of a class. This consistency suggests that visual and verbal (semantic) tasks access similar stored knowledge about material properties. Another example of the congruency between representational spaces is the study by Baumgartner et al. (2013), which found that participants were highly consistent between solely visual and solely haptic judgments of the same material attributes for the same stimuli, suggesting that they relied on similar underlying representations of material properties. This raises the interesting question of whether asking about material attributes (particularly the inferred ones) taps into a perceptual space of material representations or rather semantic or haptic representations. This might be a challenging question for further investigations. 

What information is available?

Our finding that motion in the full-cue condition provided additional information over and above shape information is in line with findings in the object-recognition literature, which suggest that in learning about rigid (Balas & Sinha, 2009) and nonrigid (Chuang, Vuong, & Bülthoff, 2012) objects, motion information makes a distinct contribution to object recognition “more than the sum of their views” (Balas & Sinha, 2009, p. 1). This idea is also supported by structure-from-motion studies, which have shown that observers can derive information about the 3-D structure of objects from nonrigid motion (Jain & Zaidi, 2011), or semirigid motion in the case of biological motion (Johansson, 1973, 1976; Troje, 2013). Our work has extended this research and shown that it is possible to extract information about the material properties of nonrigid substances from point-light displays. 

What motion information might observers have used in our experiments? Jain and Zaidi (2011) showed that motion perspective models, which are usually applied to rigid motion, can also be applied to nonrigid motion. That is, if an object translates in front of a stationary observer, retinal velocities are inversely proportional to the distances of different parts of the object. Jain and Zaidi suggest that this principle is used by the visual system to extract depth from relative velocities. Bingham, Schmidt, and Rosenblum (1995) studied event recognition with patch light displays, and found that observers could distinguish between rigid-body dynamics, hydrodynamics, aerodynamics, and biodynamics from motion information alone. In that study, observers were asked to identify events and describe the motions, both freely and from a list of descriptors. The authors found that observers were quite good at recognizing the events and also whether the events contained animate or inanimate motions. For example, they could distinguish between a compression spring being moved by hand versus a free-falling bouncing spring. A cluster analysis revealed that the perceived similarities and differences between events reflected the similarities and differences in the underlying dynamics. That is, rigid events looked more similar to each other than to hydrodynamic or aerodynamic events. The authors analyzed the form of projected trajectories, comparing position, time, and velocity of the trajectories. They found that animate events were distinguished from inanimate events based on whether energy increased along portions of the trajectories. Energy lost through friction, collision, or damping was injected back when motion was animate. 

An interesting topic that we are concurrently investigating is identifying important events in our movie clips to determine what motion cues are available around those events. That is, we aim to further investigate what information observers use when in order to make judgments of material attributes. For stimuli used in Experiments 1 and 2 we have direct access to individual particle trajectories which could be analyzed further in a number of ways. One could, for example, obtain the three dimensional XYZ coordinates of the particles in each substance for every frame and calculate velocity magnitude (change in position over consecutive frames—i.e., speed), acceleration (change in velocity magnitude over consecutive frames), and change in acceleration over consecutive frames (called jerk). Appendix C shows the calculations used to derive these values from the 3-D particle coordinates at each frame, and Figure 15 shows histograms of particle acceleration (m/s²) for each substance between consecutive frames. The color indicates the proportion of particles accelerating (decelerating) at a particular rate (y-axis) for each frame (x-axis). Frame 12 is the point of impact with the ground. 

Figure 15

 

Acceleration histograms showing proportion of particles (indicated by color) accelerating within a particular range (y-axis; m/s²) for each frame (x-axis). Acceleration values are grouped into 1.5-unit bins ranging from −84 to 51 m/s². Negative numbers indicate deceleration.

Even this simple analysis reveals that there are clear differences in motion information between the soft, semisoft, and hard substances used in our experiments. For example, from Figure 15 (see also Figure C1) it is evident that the particles in the softer bodies exhibit smoother motion transitions than the hard body—that is, smoother changes in speed over time. Figure 16 further illustrates this, showing jerk, which is change in acceleration or force between consecutive frames. Note that positive values of jerk indicate increasing change in force (increasing acceleration/deceleration), and negative values indicate decreasing change in force (decreasing acceleration/deceleration). The plots indicate that the change in force is both more sudden and more uniform (i.e., most particles change force similarly) for the hard-body particles. 

Figure 16

 

Jerk histograms showing proportion of particles (indicated by color) with values of jerk within a particular range (y-axis; m/s³) for each frame (x-axis). Jerk values are grouped into 50-unit bins ranging from −1,900 to 2,600 m/s³. Positive values of jerk indicate increasing acceleration (deceleration), and negative values of jerk indicate decreasing acceleration (deceleration).

Supplementary Movie S4 shows the acceleration and jerk values mapped onto each particle as colors, and Figure 17 shows this information for Frames 12–16. Yellower particles have higher positive values of acceleration and jerk, and bluer particles have more negative values of acceleration and jerk, with green indicating a value of zero. Figure 17 illustrates that the hard-body particles clump together and tend to form chunks that break off from the main body of the substance, moving uniformly. The semisoft substance also forms clumps or chunks as it breaks, but particles within each clump (and within the main body) do not move uniformly; their motion is more varied. 

Figure 17

 

Acceleration (A) and jerk (B) values mapped onto each particle as colors, for Frames 12–16. Yellower particles have higher positive values of acceleration and jerk, and bluer particles have more negative values of acceleration and jerk, with green indicating a value of zero.

These are just a few illustrations highlighting potential differences in motion information between the three substances that could be used by the visual system as cues to infer differences in mechanical properties. Finding precise descriptors of the physical differences in motion events, and relating them back to a particular perceptual attribute, is the goal of future experiments. Moreover, the perception of some material attributes may be relatively stable (or vary) when different forces are applied to the same substances, such as being thrown against a wall, being hit by an external object, or colliding with another substance. Future work should aim to identify motion cues underlying perceptual stability or change across such events. 

Conclusions

We found that both mechanical and optical properties affected the material perception of soft and hard breaking substances. The contribution of each property was interactive when multiple perceptual attributes were taken into account. The present study is the first to our knowledge to show interactions between optical and mechanical properties in a task involving judgments of perceptual qualities. Our results suggest that rating multiple attributes is an effective way to get at underlying perceived differences between nonrigid breaking materials. Furthermore, unlike category or similarity tasks, it may help us to determine what those perceptual differences are, while overcoming semantic issues with looking at any single attribute in isolation. Our results also suggest that motion and shape have an interactive influence on material perception. Motion and static shape cues were separately able to provide substantial information about many material properties. However, when combined they influenced observers' ratings substantially differently depending on the perceptual qualities. For some qualities, motion dominated over optical information—for example, motion attenuated differences caused by optical properties in qualities related to smoothness and density. Other times motion enhanced the effect of optics, as was the case for qualities related to hydration and fluidness. 

Acknowledgments

This research was supported by a Sofja Kovalevskaja Award from the Alexander von Humboldt Foundation, sponsored by the German Federal Ministry for Education and Research. 

Commercial relationships: none. 

Corresponding author: Alexandra C. Schmid. 

Email: . 

Address: Department of Psychology, Justus-Liebig-University, Giessen, Germany. 

Footnotes

¹  http://pyroevil.com/category/scripts-addons/molecular-script/.

Footnotes

²  http://pyroevil.com/category/scripts-addons/cubesurfer/.

Footnotes

³  Factor scores were calculated using the Regression method in SPSS, though there was little difference between these scores and Bartlett or Anderson–Rubin scores.

Footnotes

⁴  t(36) = 4.187, p = 0.0002, for soft; t(36) = 2.563, p = 0.0147, for semisoft.

Footnotes

⁵  Analysis of simple effects using a Sidak-corrected α value of 0.0179 per test. This value was calculated as α_PC = 1 − (1 − α_FW)^1/k = 1 − (1 − 0.15)^1/9 = 0.0179, where α_PC is the paired-comparisons α value, α_FW is the family-wise error rate for the ANOVA, and k is the number of comparisons.

Footnotes

⁶  t(36) = 4.498, p < 0.0001, for glassy; t(36) = 6.16, p < 0.0001, for matte.

Footnotes

⁷  t(36) = 3.462, p = 0.0014, for glassy; t(36) = 5.279, p < 0.0001, for matte.

Footnotes

⁸  The soft substance looked less heavy than both the hard substance, t(36) = 6.185, p < 0.0001, and the semisoft substance, t(36) = 4.81, p < 0.0001.

Appendix A: Animation and rendering details

Table A1 shows the key differences in the particle physics and Molecular Script parameter setting between the soft-, semisoft-, and hard-body cubes. There are too many parameters to report here, so we have made the original Blender animation files available for download from http://doi.org/10.5281/zenodo.400257. For details about what each parameter does, see http://pyroevil.com/molecular-script-docs/. 

Table A1

 

Particle physics and Molecular Script parameters that varied between substances.

 

We chose a terra-cotta color for the surfaces. The node editor in Blender was used to set the material of the meshes for the full-cue stimuli. The Glass bidirectional scattering distribution function (BSDF; r = 0.89, g = 0.612, b = 0.5) and Diffuse BSDF (r = 0.16, g = 0.033, b = 0.008) nodes were connected to a Mix Shader node, which was in turn connected to the Material Output node. For the Glass BSDF, the Beckmann distribution function was used; however, this only controls the appearance of rough reflections and refractions, and in our case roughness was always set to 0. The factor (Fac) of the Mix Shader node determined the material: It was set to 0 for the transparent glossy material, 0.343 for the mixed-optics material, and 1 for the opaque matte material. 

The point-light stimuli were made by rendering randomly chosen particles with the Emission BSDF. Blender allows you to change the material every X number of particles in a random order (see Table A2). The unlit particles were rendered as invisible empty objects. The high-density stimuli contained 1,024–2,048 lit particles, the mid-density stimuli contained 128–256 lit particles, and the low-density stimuli contained 16–32 lit particles. 

Table A2

 

Number of empty particles for every lit particle for the point-light stimuli.

 

Appendix B: Results of ANOVAs for interactions in Experiment 1a

Optical attributes (colored orange in Figure 8)

There was a significant interaction between substance type and optical condition for ratings of smooth: There was a general trend for glassier stimuli to look smoother than more matte stimuli (main effect of optical condition). However, follow-up tests⁵ revealed that for the mixed-optics stimuli, the hard substance looked less smooth than both the soft substance, t(36) = 4.13, p = 0.0002, and the semisoft substance, t(36) = 2.779, p = 0.0086. This could be explained as a “frosted glass is rougher” effect; the hard mixed-optics stimulus looks like frosted glass, which often has a rough exterior, whereas the softer bodied mixed-optics stimuli look like they are made from gelatin, which is smooth on the surface. There was also an unexpected interaction between substance type and optical condition for ratings of glossy, though this interaction is very subtle. Follow-up tests revealed that for the mixed-optics material, the hard substance looked less glossy than both the soft substance, t(36) = 3.364, p = 0.0018, and the semisoft substance, t(36) = 4.664, p < 0.0001. We have no principled explanation for this result, although perhaps the way softer substances break apart causes more specular reflections compared to hard substances (shape has been shown to affect perceived glossiness; Marlow et al., 2012; Vangorp et al., 2007). 

Motion attributes (colored green in Figure 8)

There was an interaction between substance type and optical condition for ratings of runny. Follow-up tests revealed that hard substances did not look runny (regardless of surface optics), that semisoft substances looked runnier when they were glassy compared to matte, t(36) = 4.24, p = 0.0001, and that soft substances looked runnier with the mixed-optics surface versus the glassy surface, t(36) = 2.71 p = 0.0102, or matte surface, t(36) = 2.552, p = 0.0151. It makes sense that glassier stimuli would look runnier than more matte stimuli, because liquid substances are usually transparent and glossy. The effect for soft substances (mixed-optics stimuli looked runnier than glassy stimuli) is subtle, but somewhat surprising. Perhaps because the glassy surface appeared more gelatinous, it looked less runny than the mixed-optics material. 

Inferred attributes (colored blue in Figure 8)

Follow-up tests for the interactions for inferred attributes revealed that for both glassy and matte stimuli, hard substances looked heavier than soft substances, p < 0.0001,⁶ and semisoft substances, p < 0.01,⁷ but for mixed-optics stimuli, the semisoft substance looked just as heavy as the hard substance, p = 0.19.⁸ We are unsure why the mixed-optics stimulus would look heavier than the other two semisoft substances, especially since this effect was not replicated in Experiment 2. Finally, soft substances looked equally liquidy/fluid, hard substances looked equally not liquidy/fluid, and semisoft substances looked more liquidy/fluid when they had a glassy surface versus both a matte surface, t(36) = 5.141, p < 0.0001, and a mixed-optics surface, t(36) = 3.197, p = 0.0029. 

Table B1

 

F values for main effects and interactions in Experiment 1. Degrees of freedom are shown in the first row and were the same for all attributes. Boldface indicates significant effects. *p < 0.05, **p < 0.01, ***p < 0.001.

 

Table B2

 

F values for the main effects and interactions for the two-factor ANOVA in Experiment 2—Stimulus type (full-cue, point-light) × Substance type (soft, semisoft, hard)—and from the one-way ANOVAs for the point-light stimuli in Experiment 2 (rightmost column). Degrees of freedom are shown in the first row and were the same for all attributes. *p < 0.05, **p < 0.01, ***p < 0.001.

 

Appendix C: Calculating velocity magnitude, acceleration, and jerk

The following calculations show how we tracked the velocity magnitude (i.e., speed), acceleration, and jerk of the particles in each substance over time. We represent the X, Y, and Z positions for each particle i as a vector P at time t:  

\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\unicode[Times]{x1D6C2}}\)\(\def\bupbeta{\unicode[Times]{x1D6C3}}\)\(\def\bupgamma{\unicode[Times]{x1D6C4}}\)\(\def\bupdelta{\unicode[Times]{x1D6C5}}\)\(\def\bupepsilon{\unicode[Times]{x1D6C6}}\)\(\def\bupvarepsilon{\unicode[Times]{x1D6DC}}\)\(\def\bupzeta{\unicode[Times]{x1D6C7}}\)\(\def\bupeta{\unicode[Times]{x1D6C8}}\)\(\def\buptheta{\unicode[Times]{x1D6C9}}\)\(\def\bupiota{\unicode[Times]{x1D6CA}}\)\(\def\bupkappa{\unicode[Times]{x1D6CB}}\)\(\def\buplambda{\unicode[Times]{x1D6CC}}\)\(\def\bupmu{\unicode[Times]{x1D6CD}}\)\(\def\bupnu{\unicode[Times]{x1D6CE}}\)\(\def\bupxi{\unicode[Times]{x1D6CF}}\)\(\def\bupomicron{\unicode[Times]{x1D6D0}}\)\(\def\buppi{\unicode[Times]{x1D6D1}}\)\(\def\buprho{\unicode[Times]{x1D6D2}}\)\(\def\bupsigma{\unicode[Times]{x1D6D4}}\)\(\def\buptau{\unicode[Times]{x1D6D5}}\)\(\def\bupupsilon{\unicode[Times]{x1D6D6}}\)\(\def\bupphi{\unicode[Times]{x1D6D7}}\)\(\def\bupchi{\unicode[Times]{x1D6D8}}\)\(\def\buppsy{\unicode[Times]{x1D6D9}}\)\(\def\bupomega{\unicode[Times]{x1D6DA}}\)\(\def\bupvartheta{\unicode[Times]{x1D6DD}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bUpsilon{\bf{\Upsilon}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(\def\iGamma{\unicode[Times]{x1D6E4}}\)\(\def\iDelta{\unicode[Times]{x1D6E5}}\)\(\def\iTheta{\unicode[Times]{x1D6E9}}\)\(\def\iLambda{\unicode[Times]{x1D6EC}}\)\(\def\iXi{\unicode[Times]{x1D6EF}}\)\(\def\iPi{\unicode[Times]{x1D6F1}}\)\(\def\iSigma{\unicode[Times]{x1D6F4}}\)\(\def\iUpsilon{\unicode[Times]{x1D6F6}}\)\(\def\iPhi{\unicode[Times]{x1D6F7}}\)\(\def\iPsi{\unicode[Times]{x1D6F9}}\)\(\def\iOmega{\unicode[Times]{x1D6FA}}\)\(\def\biGamma{\unicode[Times]{x1D71E}}\)\(\def\biDelta{\unicode[Times]{x1D71F}}\)\(\def\biTheta{\unicode[Times]{x1D723}}\)\(\def\biLambda{\unicode[Times]{x1D726}}\)\(\def\biXi{\unicode[Times]{x1D729}}\)\(\def\biPi{\unicode[Times]{x1D72B}}\)\(\def\biSigma{\unicode[Times]{x1D72E}}\)\(\def\biUpsilon{\unicode[Times]{x1D730}}\)\(\def\biPhi{\unicode[Times]{x1D731}}\)\(\def\biPsi{\unicode[Times]{x1D733}}\)\(\def\biOmega{\unicode[Times]{x1D734}}\)\begin{equation}\tag{C1}{P_i}(t) = ({X_i}(t),{Y_i}(t),{Z_i}(t)){\rm ,}\end{equation}

from which we can derive velocity V_i(t), the change in position over consecutive frames:  

\begin{equation}\tag{C2}{V_i}\left( t \right) = \left( {{{d{X_i}} \over {dt}},{{d{Y_i}} \over {dt}},{{d{Z_i}} \over {dt}}} \right) = \left( {{V_{{X_i}}}\left( t \right),{V_{{Y_i}}}\left( t \right),{V_{{Z_i}}}\left( t \right)} \right){\rm ,}\end{equation}

and velocity magnitude M:  

\begin{equation}\tag{C3}{M_i}(t) = \sqrt {V_{{X_i}}^2(t) + V_{{Y_i}}^2(t) + V_{{Z_i}}^2(t)} {\rm .}\end{equation}

Acceleration A was calculated as the change in velocity magnitude over consecutive frames:  

\begin{equation}\tag{C4}{A_i}\left( t \right) = {{d{M_i}} \over {dt}}{\rm ,}\end{equation}

and jerk J was calculated as the change in acceleration over consecutive frames:  

\begin{equation}\tag{C5}{J_i}\left( t \right) = {{d{A_i}} \over {dt}}{\rm .}\end{equation}

Figure C1

 

Velocity-magnitude histograms showing proportion of particles (indicated by color) traveling within a particular velocity-magnitude range (y-axis; m/s) for each frame (x-axis). Velocities are grouped into 0.05-unit bins ranging from 0 to 4.5 m/s.