Deformation-induced transparency resolves color scission

Abstract

Dynamic image deformation produces the perception of a transparent material that appears to deform the background image by light refraction. Since past studies on this phenomenon have mainly used subjective judgment about the presence of a transparent layer, it remains unsolved whether this is a real perceptual transparency effect in the sense that it forms surface representations, as do conventional transparency effects. Visual computation for color and luminance transparency, induced mainly by surface-contour information, can be decomposed into two components: surface formation to determine foreground and background layers, and scission to assign color and luminance to each layer. Here we show that deformation-induced perceptual transparency aids surface formation by color transparency and consequently resolves color scission. We asked observers to report the color of the front layer in a spatial region with a neutral physical color. The layer color could be seen as either reddish or greenish depending on the spatial context producing the color transparency, which was, however, ambiguous about the order of layers. We found that adding to the display a deformation-induced transparency that could specify the front layer significantly biased color scission in the predicted way if and only if the deformation-induced transparency was spatially coincident with the interpretation of color transparency. The results indicate that deformation-induced transparency is indeed a novel type of perceptual transparency that plays a role in surface formation in cooperation with color transparency.

Introduction

Vision researchers have proposed many phenomena and mechanisms relating to surface perception (Gibson, 1950; Kanizsa, Legrenzi, & Bozzi, 1979; Komatsu, 2006; Nakayama & Shimojo, 1991; Paradiso & Nakayama, 1992). In particular, perceptual transparency has fascinated many researchers for decades (Adelson & Anandan, 1990; Anderson, 1997; Beck & Ivry, 1988; Metelli, 1974). Perceptual transparency has two main components. One is surface formation, through which the visual system determines foreground and background layers. The other is color scission (Gerbino, Stultiens, Troost, & de Weert, 1990; Khang & Zaidi, 2002), wherein the visual system decomposes a single color input into foreground and background colors. In conventional transparency effects the two components are hard to separate, as they are like two sides of the same coin because they are based on tightly connected image information. Surface-layer formation is based on the spatiotemporal properties of surface boundary contours, such as x-junctions (Adelson & Anandan, 1990; Anderson, 1997; Beck & Ivry, 1988), occluding motion (Cicerone, Hoffman, Gowdy, & Kim, 1995; Miyahara & Cicerone, 1997), and binocular disparity (Kingdom, Blakeslee, & McCourt, 1997; Tse, 2005); while color scission is based on the image changes (e.g., luminance or color contrast) across the surface boundary contours (Adelson, 1993, 2000; Shevell & Kingdom, 2008). In conventional perceptual transparency effects, the simulated transparent materials are not perfectly transparent. They are visible due to such optical factors as partial light absorption and scattering within the layer, and weak light reflectance at the surface. In natural image formation, these factors produce visible image changes in luminance, color, or blurring at the outer boundary of a transparent layer. These image changes provide cues for the visual system to infer the presence and properties of transparent layers that are then converted into what is actually perceived. 

This scenario does not work for purely transparent materials such as clear water; but even for such materials, there are potential visual cues to the presence of a transparent layer, including image deformation of the background texture produced by light refraction at the surface of the transparent layer. This insight brought us to the discovery of a novel type of perceptual transparency effect wherein human observers perceive a transparent layer when a texture pattern is dynamically deformed (Kawabe & Kogovšek, 2017; Kawabe, Maruya, & Nishida, 2015). The layer looks like a liquid, glass, or hot air depending on the spatiotemporal parameters of image deformation. Although the deformation-induced transparency is enhanced by the presence of deformation-defined contour junctions at the surface's outer boundary (Kawabe & Nishida, 2017), a transparent layer remains visible regardless of the layer boundary condition, provided that the deformation parameters are optimized. This is a critical difference from conventional perceptual transparency. 

The outstanding question, however, is whether deformation-induced transparency is indeed a perceptual transparency effect or merely a cognitive inference of the presence of a transparent material on the basis of observer's knowledge about the relation between the pattern of dynamic image deformations and material types. Because previous studies have investigated the phenomenal appearance of transparent materials by using subjective rating tasks, it is unclear whether deformation-induced transparency was represented at some perceptual levels that were also involved with conventional perceptual transparency. To find out whether image deformations are able to produce a perceptual representation of multilayered surfaces as does conventional perceptual transparency, we have been studying the interactions of deformation-induced transparency and conventional transparency effects. We recently reported the effect of luminance-induced transparency on deformation-induced transparency (Kawabe & Nishida, 2017). The magnitude of deformation-induced transparency was enhanced when the transparency layer was additionally supported by luminance-defined x-junctions at the surface's outer boundary, whereas it was suppressed when the luminance changes across the surface's outer boundary were inconsistent with luminance transparency. This suggests an influence of luminance-induced transparency on perceptual surface formation in deformation-induced transparency (Figure 1, rightward arrow), but it is also possible that the perceptual evidence for a transparent surface from the two independent sources is simply integrated at the decision level. 

Figure 1

 

A schematic explanation of the relationship between surface formation and color/luminance scission in color/luminance-induced transparency and deformation-induced transparency. Here we theoretically decompose color/luminance-induced transparency into two components: surface formation and color/luminance scission. Kawabe and Nishida (2017) have demonstrated that the surface formation in luminance-induced transparency strongly regulates the surface formation in deformation-induced transparency. In this study, we oppositely investigate the effect of surface formation in deformation-induced transparency on the color/luminance scission on the basis of the interaction of surface formation between color/luminance-induced transparency and deformation-induced transparency.

To obtain more convincing evidence for the formation of a perceptual surface representation by image deformations, we examined whether deformation-induced perceptual transparency regulates color scission in color/luminance-induced transparency. We assumed that deformation-induced transparency could affect color scission only through modulating surface formation in color/luminance-induced transparency (Figure 1, leftward arrow), which most probably indicates the formation of a common surface representation. We report that the deformation-induced perceptual transparency does disambiguate color scission in color transparency when the edges of chromaticity modulation are spatially coincident with those of dynamic image deformation. Based on the results, we discuss the role of deformation-induced perceptual transparency in the determination of surface formation and color scission. 

Experiment 1

Purpose

We explored whether deformation-induced transparency affected color scission in an ambiguous transparent display. We employed an ambiguous color transparency display wherein two stripe-shaped areas having chromatic modulation were orthogonally overlapped with each other. The chromatic modulations of the areas were made in opposite directions to each other in color space. At the location where the two stripes overlapped, no chromatic modulation occurred, because the chromatic modulations in the two stripes canceled each other out. We added dynamic image deformations to one of the stripes so that the deformations overlapped with one of the stripes undergoing chromaticity modulation. With static presentation, the central region was perceptually ambiguous in three ways: two layers with the vertical stripe in front, two layers with the horizontal stripe in front, and a single layer with no color modulation. If the deformation-induced transparency affected surface formation in color transparency and thus influenced color scission, materials induced by dynamic image deformations would be tinged with color consistent with the chromatic modulation in the spatially coincident stripe even at the location where the two stripes overlapped, wherein no chromatic modulation existed. In addition, we checked a control condition in which dynamic image deformations subtended across two types of chromatic modulation. It was expected that under the control condition a colorless transparent material would be reported more often, because deformation-induced transparency was not consistent with any interpretation of surface formation in color transparency. 

Figure 2

 

Background images used in this study. The images were sampled from the McGill Calibrated Colour Image Database (Olmos & Kingdom, 2004).

Figure 3

 

(a) Example snapshots of stimulus clips as used in Experiment 1. Left and right panels show the snapshots of the consistent and inconsistent conditions, respectively. The static images are not suitable for comprehending our effect; please watch Supplementary Movie S1 to understand what is happening in our stimuli. (b) The proportion of reports of the expected transparent color in color-response trials for the consistent condition. (c) The proportion of reports for colorless transparent material under the consistent and inconsistent conditions.

Method

Observers

Observers in this experiment (nine women, one man; mean age = 35.3 years, SD = 5.08) were unaware of the specific purpose of the experiments. They reported having normal or corrected-to-normal visual acuity. They were recruited from outside the laboratory and were paid for their participation. Ethical approval for this study was obtained from the ethical committee at Nippon Telegraph and Telephone Corporation (NTT Communication Science Laboratories Ethical Committee approval number H28-008). The experiments were conducted according to principles that have their origin in the Helsinki Declaration. Written informed consent was obtained from all participants. 

Apparatus

Stimuli were presented on a 21-in. CRT monitor (Sony GDM-F500R) with a resolution of 1,024 × 768 pixels and a refresh rate of 60 Hz. The gun outputs of the CRT monitor were gamma corrected. The CIE coordinates of the phosphors of the monitor were R(x = 0.5824, y = 0.3612), G(x = 0.2931, y = 0.6111), and B(x = 0.1561, y = 0.0889), which were measured using a colorimeter (Bm-5A, Topcon, Tokyo, Japan). A computer (Mac Pro, Apple Inc.) controlled stimulus presentation, and data were collected with PsychoPy (ver. 1.83; Peirce, 2007, 2009). 

Stimuli

Five natural texture images, which were sampled from the McGill Calibrated Colour Image Database (Olmos & Kingdom, 2004), were used as background images in stimuli (Figure 2). The images were clipped and resized to 256 × 256 pixels (7.68° × 7.68° of visual angle). A stimulus clip consisting of 120 frames (16.6 ms/frame) was created in the following way (Supplementary Movie S1). First, one of the texture images was randomly chosen as the background. Second, the image was deformed within a rectangular window that subtended 1.44° in width and 7.68° in height (Figure 3a). The center coordinate was shared between the rectangular window and the background image. Dynamic image deformations were applied to the image using a pixel-warp method that was based on the deformation maps defining the magnitude of the pixel shift for deformation. A single snapshot of the deformation maps consisted of low-pass-filtered two-dimensional white noise. The cutoff frequency of the filter applied to the noise was 32 cycles/image (4.16 cycles/°). This was high enough to cause transparent material perceptions from dynamic image deformation (Kawabe et al., 2014). The filtered white noise was shifted in a single direction (upward or downward) by 0.06°/frame (and thus 3.6°/s). The shift created image deformation flows (i.e., dynamic image deformations) and perceptually produced a transparent material. 

Third, different chromatic modulations were applied to each of two orthogonally striped areas. The chromaticity manipulation was conducted in the DKL color space (Derrington, Krauskopf, & Lennie, 1984; Krauskopf, Williams, & Heeley, 1982; MacLeod & Boynton, 1979). For a vertical stripe, an area 1.44° wide and 7.68° tall underwent chromatic modulation with a positive or negative shift in the chromaticity of 0.125 in the L–M axis. For a horizontal stripe, an area 7.68° tall and 1.44° wide underwent chromatic modulation with a positive or negative shift in the chromaticity of 0.125 in the L–M axis. When the horizontal stripe had a positive shift, the vertical stripe had a negative shift, and vice versa. The center coordinates of both stripes were shared with the center coordinate of the background image. In the area where the two stripes overlapped with each other, no chromatic modulation was expected to occur because the directions of the shifts in the horizontal and vertical stripes were opposite to each other, and hence the two chromatic modulations were canceled out. We pasted the corresponding part of the original background image (before any chromatic modulation) to the location where the two stripes overlapped because we wanted to avoid any calculation artifact caused by chromatic modulation. The size of the area onto which the original image was pasted was 1.44° × 1.44°. We refer to this condition as the consistent condition because the spatial structure of dynamic image deformations was coincident with one of two chromatic modulations in the vertical stripe. 

As the control condition (we refer to this as the inconsistent condition; Figure 2a), we tested another configuration of an ambiguous color transparency display wherein different chromatic modulations with a single color shift were applied to each of an L-shaped and an inverted L-shaped area. The corner portions of the L-shaped and inverted L-shaped areas were overlapped so that the size of the overlapped area was 1.44° × 1.44°. The L-shaped and inverted L-shaped areas underwent chromatic shifts in opposite directions to each other. Under this condition, image deformation subtended across an area consisting of the vertical arms of the L-shaped chromatic modulation, central overlapped areas, and inverted L-shaped chromatic modulation. The upper and lower sides of the image deformation coincided with the positive and negative (or negative and positive) chromatic modulations, respectively. In actual image generation, dynamic image deformations were first applied to an image, and chromatic modulation was then applied to the deformed image. Otherwise, the spatial structure of chromatic modulation would have been influenced by dynamic image deformations. 

Procedure

The experiment was conducted in a darkened room. Each observer was tested individually. The observer sat at a distance of 60 cm from the surface of the CRT display. Each session started by clicking a green button on the PsychoPy interface. After 10 s, the first trial started with the presentation of a stimulus video clip. The clip was repeated without a break until the observer made a response. The task of the observer was to judge whether transparent materials were tinged with red or green, or whether the central square location (the area where the two chromatic modulations overlapped) was colorless. Each observer completed a single session of 64 trials consisting of 2 (stimulus configuration: consistent vs. inconsistent) × 2 (chromatic modulations) × 16 repetitions. The order of trials was pseudorandomized. 

Results and discussion

For each configuration and chromatic-modulation condition, the proportion of reports for red-tinged, green-tinged, and colorless transparent materials was calculated. We first checked how the color of transparent materials was perceived under the consistent condition. In slightly less than 90% of trials, the observers reported colored materials. Figure 3b shows the proportion of trials reporting the expected color in the color-reporting trials. The expected color was the chromatic shift applied to the top and bottom of the vertical stripe to which dynamic deformations were applied. Using a one-sample t test, we confirmed that the proportion of reports for the expected color deviated significantly from the level of chance (0.5), t(9) = 8.81, p = 1.001 × 10⁻⁵. Next, with the proportion of reports for a colorless transparent material, we conducted a paired t test to check the difference in mean proportions between the consistent and inconsistent conditions (Figure 3c). The proportion of reports for a colorless transparent material was significantly lower under the consistent condition than under the inconsistent condition, t(9) = 1.1235, p = 0.003. 

The results indicate that deformation-induced transparency influences the surface formation of color transparency and consequently leads to the disambiguation of scission in an ambiguous color transparency display. When the deformation-induced transparency was consistent with one of possible surface formations in ambiguous color transparency, the transparent material induced by dynamic image deformation was perceptually tinged with color that was consistent with the chromatic modulation even at the location where no chromatic modulation occurred. On the other hand, when deformation-induced transparency was inconsistent with any possible surface formations (i.e., the inconsistent condition), the transparent material was more frequently perceived to be colorless than when deformation-induced transparency was consistent with some surface formations. 

Experiment 2

Purpose

In the stimuli used in the previous experiment, the display contained chromaticity-defined x-junctions that are reportedly a strong cue to conventional perceptual transparency (Shevell & Kingdom, 2008). In the next experiment, we examined whether color scission was produced by the addition of dynamic image deformations to an image with chromatic modulation but without strong static transparency cues. Specifically, the two vertical stripes of color modulations were horizontally placed side by side with a small spatial gap. In this stimulus configuration, no strong cues for color transparency existed and transparent surface formation was physically possible (i.e., a stripe overlapped with another stripe by an amount identical to the spatial gap). We assume that deformation-induced transparency influences color scission when dynamic image deformations are consistent with the possible interpretation of surface formation that is driven by the chromatic modulation. Thus, it would be expected that when dynamic image deformations were added to one of the stripes and the spatial gap simultaneously, the color of a transparent material at the spatial gap of chromatic modulation would be tinged with color consistent with the chromatic modulation under dynamic image deformations even in the absence of strong cues for static transparency. For the control condition, we added dynamic image deformations to both stripes and the spatial gap. Because the transparent material was not consistent with any interpretation of surface formation in color transparency, the colorless material would be seen under the control condition. 

Method

Observers

Eight observers who had participated in Experiment 1 participated in this experiment. They were still unaware of the purpose of the experiment. The observers participated in this experiment on the same day as the previous experiment. 

Apparatus

The apparatus was identical to that used in Experiment 1. 

Stimuli

Stimuli were again 120-frame video clips consisting of the background image, image deformation, and chromatic modulation (Supplementary Movie S2). The method used to generate stimuli was identical to the one used in Experiment 1 except for the following. The chromatic modulation was applied to two vertical stripes that were placed side by side with a spatial gap (Figure 4a). Each stripe subtended 1.44° in width and 7.68° in height. The central coordinates of the stripes were horizontally shifted by 0.96° to the left and right of the center of the background image. Thus, a spatial gap of width 0.48° was generated between the two stripes with chromatic modulations. When the positive chromatic shift was applied to the left stripe, the negative chromatic shift was applied to the right stripe, and vice versa. Dynamic image deformations were applied to part of the image so that under one condition (left condition) the image deformation area subtended across the left stripe and the spatial gap, and under the other condition (right condition) it subtended across the spatial gap and the right stripe. Under the control condition (center condition), the image deformation area subtended across part of the left stripe, the spatial gap, and part of the right stripe; the center coordinate of the image deformation area was then shared with one of the background images. Under all conditions, the size of the image deformation area was 1.92° in width and 7.68° in height. In the actual image generation, dynamic image deformations were first applied to the image, and chromatic modulation was then applied to the deformed image. Two thin black bars, of width and height 0.48° and 0.12°, were added to upper and lower edges of clips to make observers aware of the horizontal location of the spatial gap between the chromatic modulations. 

Figure 4

 

(a) Example snapshots of stimulus clips used in Experiment 2. The left, center, and right panels show the snapshots of the left, center, and right conditions, respectively. The static images are not suitable for comprehending our effect; please watch Supplementary Movie S2 to understand what is happening in our stimuli. (b) The proportion of reports for the expected color under the right and left conditions. (c) The proportion of reports for colorless transparent material under the left, center, and right conditions in Experiment 3.

Procedure

The procedure was identical to that used in Experiment 1 except for the following. The task of the observers was to judge whether transparent materials were tinged with red or green, or whether the central spatial gap between the two chromatic modulations was colorless. The observers were instructed to report the color of the area of the transparent material that was in between the black bars presented at the upper and lower edges of the video clip. Each observer completed a single session of 96 trials consisting of 3 (image-deformation location: left, center, and right) × 2 (chromatic modulations) × 16 repetitions. The order of trials was pseudorandomized. 

Results and discussion

For each configuration and chromatic-modulation condition, we calculated the proportion of reports for red-tinged, green-tinged, and colorless transparent materials. We first checked how the color of transparent materials was perceived under the left and right conditions. In about half the trials, the observers reported colored materials. Figure 4b shows the proportion of observers reporting the expected color in the color-reporting trials. The expected color was the chromatic shift applied to most of the area to which dynamic deformations were applied. Using a one-sample t test, we confirmed that the proportion of reports for the expected color was significantly higher than the level of chance (0.5), t(7) = 4.20, p < 0.005. Next, we compared the proportion of reports for a colorless transparent material among the left, center, and right conditions (Figure 4c). A one-way repeated-measures ANOVA with the location of image deformation as a within-subject factor showed a significant main effect, F(2, 14) = 8.186, p < 0.005, η_p² = 0.539. Multiple comparison tests using Ryan's method (Ryan, 1959) showed that the proportion was significantly higher under the center condition than under the left and right conditions (p < 0.015). 

Compared to the results of Experiment 1, the proportions of reports of a colorless transparent material were higher in Experiment 2. For the proportion of reports of a colorless transparent material, we calculated effect sizes (Cohen's d) between the consistent condition of Experiment 1 and the right and left conditions of this experiment and found that the effect size was sufficiently large (Cohen's d = 1.38 vs. the left condition and 1.03 vs. the right). The large effect sizes indicate that a colorless material impression was weaker in Experiment 1 than in Experiment 2. The results can be attributed to the lack of a strong color transparency cue—x-junctions—in Experiment 2. Apart from this, the results of the two experiments were qualitatively similar. 

The results demonstrate that deformation-induced transparency affects color scission even when the stimulus does not contain a strong cue to color transparency. Since the structure of transparent layers remains ambiguous when based only on the color-configuration information of the stimulus, the visual system may use deformation information together with color-configuration information to form unambiguous surface representations, and then assign a color to each surface representation based on the chromatic information around the surface boundary. 

Experiment 3

Purpose

The previous experiments succeeded in showing that deformation-induced transparency affects color scission through the disambiguation of surface formation in color transparency. Deformation-induced transparency caused scission when the edge of the dynamic image deformations was spatially coincident with the edge of chromatic modulation in the background. Previous studies have shown that spatial alignments are critical to integration between different visual attributes (Kingdom, 2008; Landy & Kojima, 2001; Rivest & Cavanagh, 1996). If the color scission by dynamic image deformations comes from the integration of these visual attributes, the misalignment of these attributes would hamper color scission. To address this issue, we used novel stimulus displays wherein chromaticity modulation was applied to a part of the background image while dynamic image deformations were applied to the part of the image with a spatial offset from the chromatic modulation. Using the displays, we systematically investigated how the spatial offset between dynamic image deformations and chromaticity modulation impacted the perception of color of a transparent material. 

Method

Observers

The ten observers who had participated in Experiment 1 participated in this experiment. They were still unaware of the purpose of the experiment. They joined in this experiment on the same day as the previous experiment. 

Apparatus

The apparatus was identical to that used in Experiments 1 and 2. 

Stimuli

A stimulus clip (Supplementary Movie S3) again consisted of 120 frames. The method to generate stimuli was identical to that used in Experiment 1 except for the following. The image was deformed within a rectangular window that subtended 1.92° in width and 7.68° in height. The horizontal position of the center of the rectangular window was offset from the center of the image in the following six levels: 0°, 0.06°, 0.12°, 0.24°, 0.48°, and 0.96° (see Figure 5a for some example snapshots of the stimulus clips). Chromatic modulation was applied to the deformed images. The width and height of the window for chromatic modulation were 1.92° and 3.84°, respectively. The horizontal and vertical centers of the window coincided with the horizontal and vertical centers of the deformed image. Within the window, overall chromaticity was shifted by 0.125 or −0.125 on the L–M axis, which produced red-tinged and green-tinged images, respectively. 

Figure 5

 

(a) Some example snapshots of stimulus clips used in Experiment 3. Left, center, and right panels are the snapshots for the conditions with 0°, 0.24°, and 0.96° spatial offset. The horizontal spans of image-deformation regions are shown as black bars because in static pictures it is hard to find the region undergoing image deformations. The static images are not suitable for comprehending our effect; please watch Supplementary Movie S1 to understand what is happening in our stimuli. (b) Experiment 3 results. The proportion of reports for a colorless transparent material is plotted as a function of spatial offset between the modulations of dynamic image deformations and chromaticity.

Procedure

The procedure was identical to that used in Experiment 1 except for the following. The task of the observer was to judge whether the transparent material was tinged with red or green, or whether it was colorless. Each observer completed a total of 192 trials, which were divided into two sessions each of 96 trials. Each session consisted of 6 (spatial offset of the window for image deformation) × 2 (chromatic modulation: positive vs. negative) × 8 repetitions. The order of trials was pseudorandomized across the observers. 

Results and discussion

For each condition of spatial offset and chromatic modulation, we calculated the proportion of reports for red-tinged, green-tinged, and colorless transparent materials. Under all conditions, when the observers reported color-tinged transparent materials, positive and negative shifts of chromaticity, respectively, resulted in the reports for red-tinged and green-tinged transparent materials. Figure 5b shows the proportion of reports for a colorless transparent material as a function of spatial offset of the image deformation window from the center of the background image. For the proportion, we conducted a one-way repeated-measures ANOVA with the spatial offset as a within-subject factor. The main effect was significant, F(5, 45) = 61.972, p < 0.001, η_p² = 0.873. Multiple comparison tests showed that the proportions in the 0° and 0.06° offset conditions were significantly different from the proportions in the conditions with 0.12° or greater offset (p < 0.001). Moreover, the proportions in the 0.12° offset condition were significantly different from the proportions in the conditions with 0.24° or greater offset conditions (p < 0.001). We also checked the difference in proportions from a chance level (0.5) by using multiple comparison tests with the Holm–Bonferroni method. The proportions were significantly lower than 0.5 when the offset was 0° (p = 3.00 × 10⁻⁸) or 0.06° (p = 7.50 × 10⁻⁶), and significantly higher than 0.5 when the offset was 0.24° (p = 0.047), 0.48° (p = 0.023), or 0.96° (p = 0.0002). 

The results showed that deformation-induced transparent materials were perceptually tinged with color that was consistent with the chromatic modulation in the background when the edge of dynamic image deformations was spatially coincident with the edge of chromaticity modulation. However, as the spatial offset between the two attributes increased, the transparent material came to be perceived as colorless. Taken together, the results indicate that color scission by deformation-induced transparency occurs due to the integration between dynamic image deformation and chromaticity. 

General discussion

To summarize, we report that deformation-induced transparency resolves color scission when dynamic image deformations are spatially consistent with one of the interpretations of surface formation that arise from chromaticity modulation. In addition, we found that the coincidence of edges between dynamic image deformations and chromaticity is an important factor contributing to color scission. 

Past studies on deformation-induced transparency have been based on subjective reports on the presence of a transparent layer (Kawabe, 2017; Kawabe et al., 2015; Kawabe & Kogovšek, 2017; Kawabe & Nishida, 2017). It has remained poorly understood whether the effect produces a perceptual representation of transparent surfaces as conventional perceptual transparency effects do. The present findings provide convincing evidence that deformation-induced transparency contributes to the formation of surface representations to which color information is assigned. 

The present results suggest that deformation-induced transparency alters color scission by influencing surface formation that arises from the spatial context of chromaticity modulation. While conventional perceptual transparency usually occurs due to spatial contexts of luminance and color (Adelson & Anandan, 1990; Anderson, 1997; Beck & Ivry, 1988), deformation-induced transparency does not require such spatial context (Kawabe et al., 2015; Kawabe & Kogovšek, 2017; Kawabe & Nishida, 2017). For these reasons, it is possible that surface formation by conventional perceptual transparency and by deformation cues are executed independently during visual processing, and then the surface representations are integrated into a single common whole if it is likely that they come from an identical source based on spatial coincidence. Alternatively, a common surface representation may be directly computed from two types of cues. In either case, the color-assignment computation should be based on a common surface representation. Otherwise, deformation-induced transparency cannot affect color scission. 

Given the physical properties of transparent materials, it is reasonable to assume that the visual system integrates dynamic image deformations and chromaticity modulations so that a coherent transparent material is seen. In the external world, transparent materials may be related to several physical factors—such as transmittance, reflectance, scatter, and refraction—at the same time. Thus, it is plausible to assume that the visual system integrates visual features stemming from several physical factors to calculate the appearance of a single transparent material. A previous study (Kawabe & Nishida, 2017) has reported that the impression of a transparent material from dynamic image deformations was hampered when the luminance configuration at deformation-defined junctions was inconsistent with conventional perceptual transparency. Consistent with that study, the present study shows that the visual system is acutely sensitive to the spatial coincidence between dynamic image deformations and chromaticity modulations. On the basis of the spatial consistency, surface formations as well as color scission are determined. 

It may be useful to mention the generic-view principle (Albert, 2001; Freeman, 1994) in interpreting our results. It would have been possible for the observers to interpret the stimuli without spatial offset in Experiment 3 as involving a chromatic modulation in the background image spatially coinciding with a colorless transparent material. However, it seems that the visual system interprets such coincidence as unlikely. According to the generic-view principle, “the visual system generally ignores the possibility that salient image features are the result of coincidences between the viewpoint of the observer and the internal geometry of an object or scene” (Albert, 2001, p. 198). Rather than choosing such an accidental interpretation, the visual system may interpret the coincidence between the modulations of dynamic image deformations and chromaticity as coming from a color-tinged transparent material that refracts light. 

The phenomenon reported in this study is possibly related to the capture of background features by a forefront surface (Häkkinen, Liinasuo, Kojo, & Nyman, 1998; Ramachandran, 1986; Ramachandran & Cavanagh, 1985). Subjective surfaces that are placed on a textured image apparently pull the texture into the surface. The phenomenon indicates that the visual system uses the information derived from surface formation to interpret disparity signals in the background features. In our stimuli, a deformation-induced transparent material apparently pulled the chromatic modulation in the background into the material, and might as a result disambiguate the scission in color transparency. Further study is necessary to elucidate the relationship among surface formation, color scission, and surface capture. 

Finally, perceptual transparency can be produced by a difference in motion properties between multiple superimposing pattern layers (motion transparency; Snowden, Treue, Erickson, & Andersen, 1991; Snowden & Verstraten, 1999). The visual system is able to assign different colors to each layer, though sometimes in an incorrect way (Wu, Kanai, & Shimojo, 2004). How deformation-induced transparency interacts with motion transparency in surface formation and scission remains an open question that warrants further intensive investigation. 

Acknowledgments

This research was supported by MEXT/JSPS KAKENHI JP15H05915 (Innovative “SHITSUKAN” Science and Technology). 

Commercial relationships: none. 

Corresponding author: Takahiro Kawabe. 

Email: . 

Address: NTT Communication Science Laboratories, Atsugi, Japan.