Local motion signals silence the perceptual solution of global apparent motion

Abstract

Stimuli for apparent motion can have ambiguity in frame-to-frame correspondences among visual elements. This occurs when visual inputs cause a correspondence problem that allows multiple alternatives of perceptual solutions. Herein we examined the influence of local visual motions on a perceptual solution under such a multistable situation. We repeatedly alternated two frames of stimuli in a circular configuration in which discrete elements in two different colors alternated in space and switched their colors frame by frame. These stimuli were compatible with three perceptual solutions: globally consistent clockwise and counterclockwise rotations and color flickers at the same locations without such global apparent motion. We added a sinusoidal grating continuously drifting within each element to examine whether the perceptual solution for the global apparent motion was affected by the local continuous motions. We found that the local motions suppressed global apparent motion and promoted another perceptual solution that the local elements were only flickering between the two colors and drifting within static windows. It was concluded that local continuous motions as counterevidence against global apparent motion contributed to individuating visual objects and integrating visual features for maintaining object identity at the same location

Introduction

Visual inputs from the outer world in our daily life contain lots of dynamic changes in signals originating from various outer events, such as walking pedestrians and flashing traffic lights. Owing to the elaborate functioning of our visual system in perceptual construction on object individuation and identification out of these dynamically changing visual inputs, we can enjoy a visual world filled with objects in spatial and temporal continuity. 

To make a good estimation of the spatiotemporal unfolding of various objects in the environment from band-limited visual inputs, individuation and integration must take place as important processes involving incessant computational decisions. When an observation is made at time t and another observation at time t + 1, the visual system must determine whether these inputs come from the same or distinct objects. When the inputs are more likely to come from a change in relative position over time, spatiotemporal integration is voted through the decision process; otherwise, object individuation is favored as these inputs are presumed to originate from distinct objects. These decisions are usually made in parallel fashion without selective attention, and the most parsimonious solution is somehow reached, allowing us to experience an orderly visual world. 

Individuation and integration must be made out of a bundle of spatiotemporal input signals with varying sampling frequencies. Various psychophysical investigations have been reported to elucidate the computational characteristics of these processes dealing with this challenging problem, often using visual illusions such as apparent motion (Ullman, 1979), streaming/bouncing motion (Bertenthal, Banton, & Bradbury, 1993; Goldberg & Pomerantz, 1982; Moore, Stephens, & Hein, 2020), and illusory conjunction (Humphreys, Cinel, Wolfe, Olson, & Klempen, 2000; Treisman & Schmidt, 1982; Vul, Rieth, Lew, & Rich, 2020). Among these, apparent motion is especially interesting because the perception dramatically reflects the outcome of object individuation and integration from artificially discrete spatiotemporal samples, and the visual system is more or less faced with a problem of deciding a correspondence between a certain element at time t and a certain element at time t + 1. This correspondence problem is eminent in a situation of multistable perception from visual inputs having considerable ambiguity concerning possible perceptual solutions. For example, an apparent-motion stimulus termed the “motion quartet” comprises two consecutive frames (Ramachandran & Anstis, 1983), one containing two horizontally aligned dots and the other containing two vertically aligned dots, and has perceptual ambiguity in matching dots in the first frame with those in the second frame. In this stimulus, a perceptual solution of matching one of the dots in the first frame with two dots in the second frame does not typically occur. Globally consistent spatiotemporal correspondence, which we will henceforth refer to global apparent motion, is usually preferred, such as the top and bottom dots matched with the leftward and rightward dots, respectively, both achieving a one-to-one correspondence (Ramachandran & Anstis, 1983). Prolonged observation may occasionally lead to intermittent perceptual reversals from one solution to another. 

When object correspondence is perceptually ambiguous as such, visual motion information is one of the crucial cues to achieving a perception solution. On the one hand, several previous studies have demonstrated the importance of spatiotemporal continuity as one of the determining factors for object continuity and identity (Kahneman, Treisman, & Gibbs, 1992; Pylyshyn & Storm, 1988). However, what makes the situation more complicated is that motion information is not always a cue embedded in the visual inputs but is often a perceptual construct inferred from poorly sampled sensory evidence, and one type of motion can interfere with another motion, disrupting a specific perceptual solution. The literature on the correspondence problem of bistable apparent motion has demonstrated that, in addition to the features and properties of visual elements, their spatiotemporal configuration exerts a critical effect on the solution of motion correspondence, such as the one observed in the above-mentioned motion quartet and its variants (Dawson, 1991; Ramachandran & Anstis, 1986). When the first and second frames of these bistable stimuli are alternated in a spatiotemporal range within which Korte's law (Korte, 1915) fails to hold, another option of just perceptual flicker emerges. Therefore effective perceptual solutions for these stimuli may be sufficiently described as tristable apparent motion/flickers. This study starts from such a multistable apparent motion, as an interesting case in which the visual system may attempt to derive one of the possible solutions. We aimed to elucidate the influence of local motions contained within the visual elements for global apparent motion on their perceptual correspondence. More specifically, we examined what would happen if locally continuous drifting motions we deliberately inserted into an apparent-motion display had a temporal resolution exceeding, and thus violating, the sampling rate of apparent motion that would be presumed by the visual system. Does apparent motion cease to move, giving its way to another solution, that is, flicker? 

Thus, this study focused on a stimulus that was open to three major options of a perceptual solution, namely clockwise (CW) and counterclockwise (CCW) global apparent motions, and color flicker at the same location without apparent motion (Figure 1). We used a “motion duodecet” stimulus, in which 12 elements in two different colors alternated in space in every odd frame and switched their colors in every even frame, with these two frames repeatedly alternated in time, where “frame” refers to a period of 200 ms during which the elements maintained their colors. Theoretically, these elements could be randomly jumping between different locations from frame to frame, and one element could be matched with multiple elements. Therefore this stimulus was technically open to an almost infinite number of element-to-element correspondences. However, pilot observations confirmed that realistic percepts were quite categorical and limited to these three aforementioned options, just as predicted from the global consistency heuristic for apparent motion (Ramachandran & Anstis, 1983). 

Figure 1.

 

Schematic of a “motion duodecet” stimulus (top-left), three alternative perceptual solutions (top-right), and screenshots of stimuli used in the first (bottom left) and second (bottom right) experiments. Twelve elements in two different colors (red/green or bright/dark) alternate in space in every odd frame, and they switch their colors in every even frame. These frames alternate in time. Three major options of perceptual solution, namely CW rotation, CCW rotation, and color flicker at the same location without apparent motion, can occur. Gabor-shaped elements are circularly and regularly arranged at an eccentricity of 8 deg around a central fixation point.

Because each frame lasted for 200 ms, continuous animations could be inserted within this period at the time resolution of the monitor (100 Hz). We added a sinusoidal grating moving continuously within each element and drifted each grating in a direction that was inconsistent with the directions of global rotations to examine whether the perceptual solution for the multistable apparent motion was affected by local motions when local and global motions were pitted against each other. We measured the length of time each percept persisted within a total observation period of 10 seconds. As mentioned above, spatiotemporal continuity involving motion is deemed to determine object identity. Because each element's position did not continuously shift over time, the continuously drifting grating at the same location, if it functioned as a spatiotemporal continuity signal, would also function as a dominant signal that contradicted the presumed sampling rate of apparent motion. This contradiction might suggest the continuity of the element at this location, increasing the length of time during which no apparent motion but a flicker was observed. Alternatively, if a pair of drifting gratings within two neighboring elements functioned as features to suggest object identity between the elements, the addition of the gratings would rather promote an apparent jump between the elements, increasing the length of time during which one of the apparent motion percepts dominates. Finally, if global jumps and grating motions were processed in parallel, perhaps through independent channels, as qualitatively distinct events, the addition of the gratings would have no effect on the length of time each percept dominated. 

We started our experiment with directionally ambiguous/unambiguous stimuli with colors. Then, by using achromatic stimuli, we examined whether chromaticity was a necessary condition. Finally, we examined what would happen if each drifting grating moved back and forth in sync with the frame alternation. 

Methods

Observers

This study comprised four experiments, each of which was planned to acquire data from 10 observers. In total, 29 adults (17 males and 12 females; aged 18–29) with trichromatic normal or corrected-to-normal vision participated in this study. All but the first author were naïve to the experimental purpose. This study was conducted per the Declaration of Helsinki guidelines. The experimental protocol was approved by the Ethics Committee of the Graduate School of Humanities and Sociology at the University of Tokyo and was conducted in accordance with the approved guidelines. Each participant gave written informed consent before participation. 

Apparatus

The experiment was controlled with a computer (Apple Mac Pro) with the MATLAB (MathWorks) programming environment and the Psychophysics Toolbox extensions (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). All stimuli were presented on a 22-inch color CRT monitor (Mitsubishi Electric RDF223H; 1280 × 960 pixels; refresh rate 100 Hz; Mitsubishi Electric, Tokyo, Japan) in a dark room. A color-lookup table was used to correct luminance linearity for the gamma profile. The viewing distance was fixed at 57 cm using a chin–forehead rest. 

Stimuli

Colorful bistable stimulus

The stimuli were displayed on a gray (CIE [x, y] = [0.290, 0.331], 50.6 cd/m²) background on the monitor. In a stimulus referred to as the “colorful bistable stimulus,” the motion correspondence of global apparent motion based on color information was bistable, and two mutually exclusive perceptual solutions existed for apparent motion, namely CW and CCW rotations (Figure 1, Supplementary Movie S1). The stimulus was composed of 12 elements staying at the same locations and changing their colors from frame to frame. Thus, two different elements were graphically generated: a “red element” that appeared as a grating confined within a reddish static region and a “green element” that appeared as a grating confined within a greenish static region. Each element was created by applying different spatial modulations to the R, G, and B channels of the video output. A Gabor patch was applied to a single channel and a Gaussian blob to the remaining two channels. The Gabor patch was a sinusoidal modulation (spatial frequency 1 c/deg, speed 6°/s or 0°/s) tapered by a static isotropic two-dimensional Gaussian contrast envelope (σ = 0.75°, peak contrast 0.99), and the resulting spatial modulation was applied only to the R (CIE [x, y] = [0.623, 0.335]) channel for the “red element” and only to the G (CIE [x, y] = [0.283, 0.610]) channel for the “green element.” The Gaussian blob was a descending Gaussian modulation relative to the background level with a peak contrast of 0.99 and was created by using a spatial function identical to the negative of the above-mentioned Gaussian contrast envelope in position and shape and applying it to the G and B channels for the “red element” and the R and B channels for the “green element.” Because the mean luminance differed between the R (9.8 cd/m²) and G (35.9 cd/m²) channel modulations, the red and green elements differed in luminance, as well as chromaticity, both of which were informative with respect to the motion correspondence of global apparent motion. 

These Gabor-shaped elements were circularly and regularly arranged at an 8° eccentricity around a central fixation point (Figure 1). In each odd frame, six red and six green elements were placed alternatingly. In each even frame, each red element was replaced by a green, and each green element was replaced by a red, such that two perceptual solutions could be achieved (i.e., CW and CCW rotations) for global apparent motion based on chromaticity and luminance (Figure 2). Each frame lasted 200 ms with no interstimulus interval. In contrast to such transient changes in color every 200 ms, each sinusoidal grating was seamlessly drifting (6°/s) or static (0°/s) with no transient phase reset between frames. 

Figure 2.

 

Schematic of the stimulus configurations, with the global solutions for apparent motion indicated by the purple arrows. In the “colorful bistable stimulus” (column 1), two mutually exclusive perceptual solutions exist for apparent motion, namely CW and CCW rotations. In the “colorful unidirectional stimulus,” the red, green, and yellow elements are circularly and regularly arranged, and in each consecutive frame, the color arrangement is shifted in the CCW (column 2) or CW (column 3) direction. Global apparent motion based on chromaticity and luminance is uniquely determined in the direction of the successive shifts, given the proximity heuristic. In the “colorless bistable stimulus” (column 4) and “colorless unidirectional stimulus” (columns 5 and 6), the chromaticity information is removed from the above-described stimuli.

Colorless bistable stimulus

In a stimulus referred to as the “colorless bistable stimulus,” the chromaticity information was removed from the colorful bistable stimulus (Figure 2). The Gabor patch was defined more conventionally as luminance modulation (with a peak contrast of 0.49) around the mean luminance, and the Gaussian blob was either an ascending or descending function (with a peak contrast of 0.25) relative to the background luminance. Bright and dark elements were created by linearly adding the Gabor patch to the ascending and descending Gaussian blobs, respectively. Other spatiotemporal parameters were identical to those of the colorful bistable stimulus. 

Table 1.

 

Mean ± SE (sec) of the key press in Experiment 1.

 

Table 2.

 

Mean ± SE (sec) of the key press in Experiment 2.

 

Table 3.

 

Mean ± SE (sec) of the key press in Experiment 4.

 

Colorful unidirectional stimulus

A stimulus referred to as the “colorful unidirectional stimulus” contained the red and green elements and also average yellow elements, each of which was created by the spatial dithering of a red and a green element with an identical sinusoidal modulation (Figure 2). 

In the first frame, the red, green, and yellow elements were circularly and regularly arranged, and the color arrangement was shifted in the CCW or CW direction consistently in each successive frame. Because each shift was one-third as large as a cycle of the color series, global apparent motion based on chromaticity and luminance was uniquely determined in the direction of the successive shifts, given the proximity heuristic (Ullman, 1979). 

Colorless unidirectional stimulus

A stimulus referred to as the “colorless unidirectional stimulus” contained dark and bright elements and average gray elements, each of which was created by the spatial dithering of a dark and a bright element with an identical sinusoidal modulation (Figure 2). The animation was identical to that of the colorful unidirectional stimulus. 

Elemental gratings

In each trial, the elemental gratings were either all drifting (referred to as the “drifting condition”) or all static (referred to as the “static condition”). In the drifting condition, 12 directions (0°, 30°, 60°, ..., 330°) were assigned to the 12 gratings in random order. The static condition was identical to the drifting condition except that the elemental gratings were static. 

Procedure

In each trial, a motion stimulus chosen in random order from the possible combinations of stimulus conditions was presented for 10 seconds, followed by a dynamic dense random-dot pattern (with each square dot subtending 0.125°) that was refreshed every 0.02 second for 0.4 second for backward masking. During each observation period, the observers were requested to maintain fixation at the central fixation point and report the way the global rotation or flicker subjectively arose by pressing a key on the computer keyboard as long as a particular percept persisted. Three different keys were assigned to three alternative percepts: (1) CW rotation, (2) CCW rotation, and (3) no global motion but only flicker. For each key, the accumulated length of time during which the key was pressed was calculated as an index of the dominance of each percept. 

Each observer performed 20 repetitive trials for each stimulus condition. Four experiments were conducted over several days in the order of 3, 1, 2, and 4. Each experiment consisted of several sessions (five in Experiment 3, four in Experiments 1 and 2, and two in Experiment 4), each of which lasted for roughly six to nine minutes, with ample breaks between sessions. 

Results

As described in the Stimuli section, two factors were manipulated as stimulus conditions: (1) “grating speeds” signifies whether sinusoidal gratings were drifting or static, and (2) “solvabilities” signifies whether the stimulus sequence was inherently bistable or compatible with a unidirectional perceptual solution as global apparent motion (either CCW or CW). The accumulated lengths of time during which the flicker was reported are shown with the results of statistical tests, whereas the others, namely the lengths of time for the CW and CCW rotations, are plotted only sparingly without performing redundant statistical tests. 

Colorful bistable/unidirectional stimuli

Figure 3 and Table 1 show the results of an experiment (Experiment 1) with the colorful stimuli, in which six conditions are considered (2 [drifting and static] grating speeds × 3 [bistable, CCW, and CW] solvabilities). Notably, when the elemental gratings were static, the bistable condition produced the perceived rotations in the CW and CCW directions in roughly equal amounts (three to four seconds). This is naturally expected because the stimulus is inherently bistable with respect to the perceived direction. Moreover, any biases can be canceled by averaging data across repetitive trials and across observers. The unidirectional CW and CCW conditions resulted in accumulated lengths of time of approximately eight and seven seconds, respectively, during which a global rotation was indeed observed in the CW and CCW directions, respectively. Accordingly, the perceived motion frequently led to a uniquely solvable direction. 

Figure 3.

 

Results of Experiment 1 for the colorful bistable and unidirectional (CCW and CW) stimuli. The mean time of the key press is plotted against the six conditions (2 [drifting and static] grating speeds × 3 [bistable, CCW, and CW] solvabilities).

Importantly, a significant difference was found between elemental grating speeds. When the elemental gratings were static (Supplementary Movie S1, Supplementary Movie S3), global rotation perception based on color occurred in most of the observation period of 10 seconds, as above-described. However, the flicker-seen time became dominant, at least under the bistable condition, when the elemental gratings were moving in directions that contradicted the global rotations (Supplementary Movie S2). Perceptual dominance was statistically tested by examining whether the observers reported the flicker or global rotation (either CW or CCW) by chance, where the chance level was half the total key press time (9.55 s, i.e., the total observation period minus the sum of the periods during which no button was being pressed). A one-sample t-test confirmed that the flicker-seen time was significantly longer than half the key press time (4.77 seconds) in the bistable condition (t(9) = 3.78, p < 0.01). 

The ambiguity in motion correspondence was not a required condition because the elemental motions increased the flicker-seen time in the other two, unidirectional CCW and CW, conditions as well, if not up to a dominant level (Supplementary Movie S4). A repeated-measures analysis of variance (ANOVA) with speed and solvability as two factors revealed a significant main effect of speed (F(1, 9) = 35.6, p < 0.001, η² = 0.42), a significant main effect of solvability (F(1.37, 12.32) = 10.5, p = 0.004, η² = 0.13), and their interaction (F(2, 18) = 16.54, p < 0.001, η² = 0.03). Under significant interaction, the simple main effects of speed were significant in the bistable (F(1, 9) = 75.6, p < 0.001, η² = 0.65), unidirectional CCW (F(1, 9) = 18.4, p = 0.002, η² = 0.37), and unidirectional CW (F(1, 9) = 19.63, p = 0.002, η² = 0.41) conditions, indicating that the presence of elemental grating motions biased the solution against global rotation irrespective of solvability. Nevertheless, the degree of the bias differed across conditions. When the elemental gratings were drifting, the simple main effect of solvability was significant (F(2, 18) = 17.0, p < 0.001, η² = 0.30), and multiple comparisons with Shaffer's method confirmed that the differences were significant between bistable and CW (t(9) = 4.4, p < 0.01) and between bistable and CCW (t(9) = 4.8, p < 0.01) but not between CW and CCW (p > 0.10). In addition, the simple main effect was not significant when the elemental gratings were static (F(1.1, 9.94) = 3.41, p = 0.09, η² = 0.19). These results suggest that when the elemental gratings were drifting, the suppression of global rotation was stronger in the bistable condition than in the unidirectional conditions. Thus, this pattern of results could be related to solvability, although the difference in the flicker-seen time might have simply reflected the difference in the number of colors available (i.e., two colors versus three colors) to achieve motion correspondence. In any event, the interim conclusion is that local motions defined by the drifts of sinusoidal modulations help individuate the elements at their locations whether the global apparent motion based on the color is bistable or not. 

Colorless bistable/unidirectional stimuli

A question arose as to whether chromaticity variations are required to produce this effect. To address this issue, we removed chromaticity information and conducted a replicative experiment (Experiment 2) using a grayscale version of the stimuli (Figure 2). Otherwise, the methods and procedures were identical to Experiment 1. Although Experiments 1 and 2 were conducted separately, the same observers participated in both. 

When the elemental gratings were static (Supplementary Movies S5, S7), the unidirectional CW and CCW conditions resulted in accumulated lengths of time within the range of 8–9 s, during which each predicted global rotation was indeed observed (Figure 4, Table 2). The bistable stimulus equally produced perceived rotation in the CW and CCW directions (approximately two seconds), just as predicted, and the length of the flicker-seen time was longer (approximately 5.5 seconds) than that observed in Experiment 1 (approximately two seconds). 

Figure 4.

 

Results of Experiment 2, in which the colorless bistable and unidirectional stimuli were used. Conventions are identical to those of Figure 3.

When the elemental gratings were drifting (Supplementary Movies S6, S8), the flicker-seen time was longer than that when the gratings were static. As shown in Experiment 1, a one-sample t-test confirmed that the flicker-seen time was significantly longer than half the key press time (4.91 seconds) in the bistable condition (t(9) = 5.29, p < 0.001). The flicker-seen time also differed across solvability. A repeated-measures ANOVA with speed and solvability as two factors revealed a significant main effect of speed (F(1, 9) = 14.8, p = 0.004, η² = 0.11) and a significant main effect of solvability (F(1.02, 9.18) = 45.8, p < 0.001, η² = 0.58), whereas their interaction was not significant (F(1.11, 9.97) = 3.23, p = 0.101, η² = 0.005). Overall, the pattern of results was essentially similar to that of Experiment 1; that is, the drifting elemental gratings tended to suppress global motion perception, although more modestly in the unidirectional CW and CCW conditions. 

Various elemental motion patterns

In another experiment (Experiment 3), we used the colorful bistable configuration to test various elemental motion patterns. Specifically, we created two new patterns of elemental motions to examine whether the phenomenon was specific to random elemental motions. In the “laminar” pattern, all the gratings were oriented horizontally and either remained static or consistently drifted all together in the same direction (upward in half of all the trials and downward in the remaining half). In the “radial” pattern, all the gratings were oriented tangentially with respect to the circular configuration and either remained static or consistently drifted all together in the same radial direction (expansion in half of all the trials and contraction in the remaining half). These two patterns and the original pattern, namely the “random” elemental motions, were presented in random order from trial to trial. We did not test another possible elemental motion pattern, that is, all the gratings drifting in the same direction of rotation, because it was impossible to discern whether the observer's response indicating rotation perception might arise from local or global rotation. We confirmed that the phenomenon was robustly replicated. The flicker-seen time in the drifting condition was significantly longer than that in the static condition for the “laminar” (mean ± SE, 5.38 ± 0.92 seconds for drifting and 1.69 ± 0.57 seconds for static), “radial” (5.88 ± 0.76 seconds for drifting and 0.82 ± 0.31 seconds for static), and “random” (6.70 ± 0.60 seconds for drifting and 2.56 ± 0.60 seconds for static). 

Elemental motions flipped in sync with frame changes

Throughout Experiments 1–3, elemental motions maintained their directions irrespective of frame changes. A question arises about whether this constant velocity information biases the perceptual solution toward the object individuation. To examine this issue, we reversed the direction of the elemental motions in synchrony with each color change in the colorful bistable stimulus (Experiment 4). To maintain the directional identity between neighboring elements across frames, we used the laminar pattern used in Experiment 3, that is, the downward motion always corresponded to one color (e.g., red) and the upward motion to the other color (e.g., green) (Figure 5, “Flip” condition) within each trial. If the directional identity in the elemental motions promoted motion correspondence, the above-mentioned stimuli would bias the perceptual solution toward global rotation (i.e., each element in red and downward motion would be more likely to jump to a neighboring element in red and downward motion in the next frame), rather than the flicker at stabilized locations. The static and drifting conditions in Experiment 1 were also conducted in Experiment 4 under the term “static” and “constant” conditions, respectively. 

Figure 5.

 

Schematic of the stimulus configurations in Experiment 4. All the gratings are oriented horizontally. In the “static” condition, all the gratings are static. In the “constant” condition, all were constantly drifting upward (or downward) all together at 6°/s. In the “flip” condition, the downward and upward motions were placed alternatingly; the downward motion corresponds to red and the upward motion to green in half of all the trials (as shown in this example), and the upward motion corresponds to red and the downward motion to green in the remaining half.

When the elemental gratings were static, the CW- and CCW-seen key presses resulted in the accumulated length of time of approximately nine seconds in total (Figure 6, Table 3). When the elemental gratings were drifting, the flicker-seen time was longer than that when the gratings were static. This was consistent with Experiment 3. Moreover, the flicker-seen time when the motion directions were flipped was longer than that when the motion directions were constant (Supplementary Movie S9). One-way repeated-measures ANOVA revealed a significant main effect (F(2, 18) = 50.57, p < 0.001, η² = 0.70). Moreover, multiple comparisons with Shaffer's method exhibited significant differences between the constant and static conditions (t(9) = 4.7, p < 0.01), flip and static conditions (t(9) = 11.5, p < 0.001), and flip and constant conditions (t(9) = 4.9, p < 0.001). 

Figure 6.

 

Results of Experiment 4. The mean time of the key press is plotted against the three conditions.

If the identity of motion as a visual feature facilitated object correspondence, the global rotation would be more preferred in the flip condition. However, the result was contradictory; the flicker-seen time was rather enhanced in the flip condition than in the constant condition. This may be because the simultaneity between the color and direction changes strengthened the sensory evidence that the features of each object are changing over time at the same location. Alternatively, the system that is responsible for global motion correspondence may be intelligent enough to reject the unnatural scenario that vertically drifting local motions awkwardly change their locations from frame to frame to maintain global rotation. 

Discussion

This study demonstrated that in the presence of local continuous motions, the perceptual impression of global apparent motion in the form of a rapid consistent rotation was replaced by periodical alternations of colors at the same location. This phenomenon suggests that the local continuous motion within each object somehow individuates the objects and thereby stabilizes them at their current location and overrides another possible perceptual solution of the same-colored object jumping to a neighboring location frame by frame. The individuation is somehow prioritized at the expense of color consistency, causing the observer to assume that the same object is oddly alternating its color at a fixed location. 

The important role of object location in perceptual formation has been emphasized in various fields of vision research. For example, the discrimination of objects defined by the integration of multiple visual features requires scrutiny within feature dimensions using location-based attention (Treisman & Gelade, 1980). Similarly, location continuity has been demonstrated as an important condition for multiple features to become integrated into a unitary object (Kahneman et al., 1992). Several studies have recently reported the spatial congruency bias, supporting our tendency that different objects may be assumed to have the same feature value when presented at the same location (Golomb, Kupitz, & Thiemann, 2014). However, the location, which plays an important role in the perceptual formation, belongs to the perceptual representation in its right. It is neither given a priori nor is based only on the stimulated position on the retina but must be determined out of an unavoidable ambiguity by using natural constraints imposed by the visual system. The elements in this study could be flickering at the same locations or randomly jumping between different locations from frame to frame. Among the possible ways of achieving such jumping behaviors, the constraint of global consistency discourages chaotic random jumps and promotes synchronized jumps between nearest locations and in the same rotation direction everywhere to maintain global coherence (Ramachandran & Anstis, 1983). Our findings indicate that the presence of local motion signals can disrupt the formation of such orderly unfolding of positions and position-based object identities. 

Local continuous motion signals could be argued to simply operate as one of the features suggesting object continuity at the original location. Ullman (1979) proposed a model of motion correspondence determination, capturing the matching between elementary constituents in reference to a built-in similarity metric termed affinity. Lots of studies have examined potential dimensions that may be effective in such a matching, including retinal proximity (Burt & Sperling, 1981; Kolers, 1972; Ullman, 1979), orientation (Green, 1986; Ullman, 1980), spatial frequency (Green, 1986; Nishida, Ohtani, & Ejima, 1992; Watson, 1986), phase (Green, 1986), luminance (Anstis & Mather, 1985; Green, 1989; Nishida & Takeuchi, 1990; Ullman, 1979), and color (Green, 1989). Although the sinusoidal modulations used in this study could be viewed as one of such elementary constituents, this view cannot explain our results. In particular, each element in Experiment 4 flipped its motion direction and its color simultaneously. Consequently, the elementary constituents in terms of color and motion direction predicted that a red and upward-drifting element in one frame should have a perfect affinity with a red and upward-drifting element 30° away in the next frame but not with a green and downward-drifting element at the same location. Nevertheless, motion correspondence between neighboring elements did not occur more frequently. In most cases, the stimulus was still perceptually solved as the same object staying at the same location, only simultaneously alternating its color and motion direction. Therefore the matching in reference to directional similarity fails to explain the results. This discrepancy is not very puzzling, however, when differences in stimulus configuration are taken into account. In the afore-mentioned classical investigations, apparent motion was maximized in a conventional stimulus configuration in which a single item was turned off when two items at its neighborhood were turned on, and relative frequencies of jumps were compared. By contrast, in this study, two feature values (e.g., red and green) were alternated at the same location, and every element was constantly visible throughout each trial. This configuration was an inevitable requirement because otherwise a competing perceptual solution of flicker would not have been applicable. Apparent-motion displays defined by changes in feature values relative to the background level, as used in this study, thoroughly have been investigated in previous studies (Hock, Gilroy, & Harnett 2002; Hock & Gilroy, 2005; Hock, Schöner & Gilroy, 2009), in which first-order change (luminance), second-order change (contrast), and their crossed correspondence are reported and also the generalizability to conventional apparent motion is argued. 

What process is responsible for the disruption of global apparent motion? We propose that this stems from the dominance of visual signals within a high temporal frequency range in resolving motion correspondence. There exist a number of perceptual characteristics suggesting discrete temporal sampling despite our conscious narrative of a phenomenally continuous visual world. An example is a “wagon wheel illusion” in the natural daylight that is deemed to occur because the visual system's temporally discrete subsampling of a rotating object produces apparent rotation in the reversed direction (e.g., Purves, Paydarfar, & Andrews, 1996; VanRullen & Koch, 2003). This illusion may also involve attention (e.g., Macdonald, Cavanagh, & VanRullen, 2014; VanRullen, Reddy & Koch, 2005) or other properties of low-level vision (e.g., Holcombe, Clifford, Eagleman, & Pakarian, 2005). In any case, the visual system is endowed with the functionality of manufacturing from temporally discrete samples to smooth continuity in the narrative of the visual world, and apparent motion processing is an important subsystem for it. The display for global apparent motion in this study is an artificial situation that we imposed on the observer's visual system as if the sampling rate was diminished to 5 Hz. With no other temporal information available, the visual system must make do with the presumption that the spatiotemporally continuous outer world is viewed from a device with a sampling rate of 5 Hz. However, once local continuous motions are introduced, they are received as surprising news violating the presumption of 5-Hz sampling—the visual system realizes that the apparent motion display, which seemed to be updated at 5 Hz, is actually updated at a higher rate, with each element actually staying still at an identical place across several temporal samples. Therefore, temporal smoothing between discrete frames at 5 Hz turns out a wrong strategy, and newly available information within a higher temporal frequency range is only compatible with a new motion correspondence, that is, the rejection of global apparent motion and the acceptance of flickers. In other words, the process responsible for apparent motion is not a stand-alone process but rather has access to a range of temporal frequency bands in deciding whether some event should be connected to another event over time, thereby accomplishing the most parsimonious and usually the most appropriate method of perceptual continuity. 

A question arises regarding whether any effect of motion-induced position shift or an illusory displacement of a moving stimulus was observed. The contour of a stationary envelope that contains a moving carrier can appear to be shifted in the direction of the carrier motion (Anstis, 1989; De Valois & De Valois, 1991; Ramachandran & Anstis, 1990). If this had occurred in our stimuli, especially in the “random” pattern, the overall apparent shape would have deviated from a perfect circle because each element had a randomly determined distinct direction, possibly discouraging the perceptual solution of a circular global motion. Therefore, in an additional observation, we mimicked such a deviation from circularity by adding random positional jitter to each element of the laminar pattern used in Experiment 4 (Figure 5, “Constant” condition). Thus, the circularity was physically imperfect similar to what would be observed in motion-induced position shifts in random directions. The positional jitter was realized by randomly shifting the elements by 19.2 arcmin, according to data from De Valois and De Valois's study (i.e., the maximum amplitude of the motion-induced position shift at a spatial frequency of 2 c/deg, temporal frequency of 4 Hz, and eccentricity of 8°) (De Valois & De Valois, 1991). This manipulation had no observable effect on the flicker dominance in the drifting condition. Although the overall circularity was physically distorted, the flicker-seen time (mean ± SE, 5.31 ± 0.86 seconds, N = 10) was comparable to that of the laminar pattern with no jitter (5.38 ± 0.92 s). Therefore illusory displacements of the local elements, if any, could not have inflated our principal results. 

It is also possible to discuss the present study from an attentional point of view. According to Cavanagh, Holcombe, and Chou (2008), who used stimuli similar to that under the flip condition in our study, feature integration occurs within an attentional window. In their study, observers failed to integrate color and motion direction if they focused their attentional window on a single element that alternated its color and motion direction. However, these features were successfully integrated if the observers moved their attentional window so that they were able to continue watching the same combination of color and motion within the window. Contrarily, our study did not constrain any aspects of the attentional window when we requested our observers to indicate their percepts. Thus it is interesting to examine whether similar attentional manipulation along the path of global apparent motion could alter the pattern of results about the perceptual solution for global apparent motion, which has not been tested in any of these studies. 

Possible interactions between local motion processing and feature tracking are also worth mentioning. When elemental gratings were static, the colorful bistable stimulus yielded a higher proportion of global rotations than the colorless bistable stimulus. In other words, providing chromaticity information in addition to luminance information helped make the realization of global motion correspondence. Both an energy-based motion mechanism and a feature tracking-based mechanism may be involved in this advantage in the colorful apparent motion over the colorless one (Cavanagh, 1992; Lu & Sperling, 2001). If feature tracking is critical in this advantage, the presence of local motions hinders the benefit of feature tracking because the global apparent motion was suppressed in both colorful and colorless bistable stimuli in roughly equal amounts. The relative contributions of the energy- and feature-based motion mechanisms to the global apparent motion examined in this study and the relative suppressions thereof will be elucidated in future studies. 

A potential factor that may be pertinent in suppressing feature tracking is the competition for attentional resources. In our stimuli, each sinusoidal grating with a spatial frequency of 1 c/deg was drifting at 6°/s, counting a temporal frequency of 6 Hz. This spatiotemporal modulation may occur as one of the most conspicuous modulations, given our sensitivity with respect to spatial and temporal frequencies at the tested eccentricity of 8° (Kelly, 1984), possibly efficiently directing attention to the inner grating of one of the elements. If such an attentional deployment competes with a cognitive load of realizing frame-by-frame correspondence between neighboring features, the global apparent motion would be attenuated by an inattentional blindness (Simons & Chabris, 1999). 

For the neural substrate, several neuroimaging studies have demonstrated that the perception of apparent motion is related to the activation of the area V1 (Chemla et al., 2019; Larsen, Madsen, Lund, & Bundesen, 2006; Muckli, Kohler, Kriegeskorte, & Singer, 2005), which receives feedback signals from hMT+/V5 (Sterzer, Haynes, & Rees, 2006; Wibral, Bledowski, Kohler, Singer, & Muckli, 2009). Therefore the inhibition of apparent motion is possible because the local motion interferes with the generation of apparent motion involving hMT+/V5 or with the feedback transmission from there to V1. Notably, feedback from the ventral stream such as lateral occipital complex can be required to resolve the correspondence problem of apparent motion in reference to feature identity (Liu, Slotnick, & Yantis, 2004; Zhuo et al., 2003). The present study unequivocally shows that local motion signals influence global motion processing. However, their influence on these global computational networks that would determine object identity and correspondence constitutes the slope for future investigations. 

If the effect we found is viewed as an example of the dominance of local motions in our perceptual awareness, phenomenologically similar effects are also observed in other visual illusions, such as motion silencing (Suchow & Alvarez, 2011) and motion-induced blindness (Bonneh, Cooperman, & Sagi, 2001; Grossberg, 2017). Motion silencing is a phenomenon in which observers fail to identify feature changes within objects when several objects with various colors or shapes are moving smoothly. It provides a line of evidence that motion signals and object appearance are tightly connected. Some studies have reported that this effect depends on motion velocity (Suchow & Alvarez, 2011), dot spacing/crowding (Turi & Burr, 2013), flicker frequency (Choi, Bovik, & Cormack, 2014), and eccentricity (Choi, Bovik, & Cormack, 2016). However, the underlying mechanisms are still under debate. Motion-induced blindness is the illusion in which the superposition of moving stimuli causes the complete disappearance of salient stationary objects behind. Various possibilities, such as attention and surface completion, have been proposed for the underlying mechanism of this phenomenon (e.g., New & Scholl, 2008). Our findings may be related to these phenomena as all these observations depict the tendency of local motions to silence other perceptual solutions. Moreover, the question of whether similar aspects of visual stimuli constitute the determining factors of the local motion dominance observed in these phenomena should be clarified, although, currently, no study has reported whether a shared underlying mechanism exists among them. 

Conclusions

In this study, we found that when a stimulus has ambiguity in spatiotemporal correspondence among local elements between successive frames, the presence of local continuous motions within the elements suppresses perceptual solutions as global apparent motion and promotes another perceptual solution that local elements are only flickering in color and drifting within static windows and never jumping globally between frames. This phenomenon does not accord with the best solution of a correspondence problem based on feature similarity but suggests that the presence of motion information helps individuate each object, getting rid of any other interpretations about their transition over time, and contributes to object identification. The promotion of object individuation also has the other side of the coin, namely the rejection of apparent motion. We suggest that the underlying process opts for the solution of apparent motion only when the visual circumstance is consistent with the presumption that a continuous world is viewed from a sampling device at a frame rate of the apparent-motion display. Once the information of continuous motion as counterevidence against this frame rate is available, apparent motion cannot coexist and therefore becomes silenced. 

Acknowledgments

The authors thank Katsuhito Yamamoto for cooperating in conducting preliminary experiments. 

Supported by JSPS KAKENHI Grant Numbers JP18H01099 and JP18H05523. 

Data are available at https://osf.io/bhwnf/. 

Commercial relationships: none. 

Corresponding author: Hoko Nakada. 

Email: [e-mail]. 

Address: Department of Psychology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan. 

Supplementary material

Supplementary Movie S1. The static condition with the colorful bistable stimulus (Experiment 1). 

Supplementary Movie S2. The drifting condition with the colorful bistable stimulus (Experiment 1). 

Supplementary Movie S3. The static condition with the colorful unidirectional stimulus (Experiment 1). 

Supplementary Movie S4. The drifting condition with the colorful unidirectional stimulus (Experiment 1). 

Supplementary Movie S5. The static condition with the colorless bistable stimulus (Experiment 2). 

Supplementary Movie S6. The drifting condition with the colorless bistable stimulus (Experiment 2). 

Supplementary Movie S7. The static condition with the colorless unidirectional stimulus (Experiment 2). 

Supplementary Movie S8. The drifting condition with the colorless unidirectional stimulus (Experiment 2). 

Supplementary Movie S9. The flip condition with the colorful bistable stimulus (Experiment 4).