Intermittent occlusion enhances the smoothness of sampled motion

Abstract

Discrete sequences of sampled motion often appear flickering and jerky. We present evidence showing that a target in sampled motion is perceived as smoother when structure in the background appears and disappears synchronously with the target. Specifically, we found that target flicker is turned into permanent target visibility at short interstimulus intervals and jerkiness is replaced by smoothly accelerating and decelerating motion at longer ones. We argue that this “smoothening” effect is essentially a form of amodal completion in space–time being evoked by spatiotemporal cues to occlusion. The effect highlights the importance of amodal representations in perception.

Introduction

Apparent motion is one of the examples in which our perception clearly differs from the given sensory input. An appropriate sequence of static images can produce an impression of an object in motion that is indistinguishable from that produced by a real object in continuous motion. This is trivial as long as the differences between the two kinds of stimulations are below the spatiotemporal resolution of the sensory system (Watson, Ahumada, & Farrell, 1986). As is well known, though, a sampled version of a continuous motion stimulus may evoke an impression of motion even though it deviates detectably from a presumably “optimal” continuous motion stimulus. A natural explanation of this would be that the sampled stimulus activates motion detection mechanisms in spite of the deviations from a presumably “optimal” continuous motion stimulus. Therefore, studying how the quality of the motion impression depends on the amount and kind of sampling artifacts might provide insight into the characteristics of the underlying mechanisms of motion detection. Obviously, increasing the mere amount of sampling error by reducing the sampling rate can be expected to impair the quality or compellingness of the motion impression. Such effects are well known to occur. In motion pictures with low frame rates, for instance, motion often appears jerky instead of smooth. 

Panels a and d in Figure 1 show the space–time diagrams of sampled motion stimuli in front of a textured background. Both can be thought of as discretely sampled versions of a continuously moving stimulus as illustrated in panels b and e, but the spatiotemporal sampling rate in panel a is twice of that in panel d. As pointed out by Adelson and Bergen (1985), the difference between the space–time diagram of the sampled motion stimulus and that of a corresponding continuous motion stimulus may be responsible for a number of motion artifacts such as jerkiness and flicker—provided, of course, that the difference is large enough to be detected. In panel c, the portions of the motion stimuli in panels a and b that are equal are shown in gray. The portions that are present in panel a but absent in panel b are shown in white and those absent in panel a but present in panel b are shown in black. If the visual system tries to “explain” the sensory input shown in panel a by attributing it to a distal object in real continuous motion as in panel b, then the white regions in panel c would constitute unexplained presence of the target stimulus (in the space–time diagram), and the black ones would constitute unexplained absence. Panel f analogously shows the differences between panels d and e. Comparing panels c and f, it is obvious that the amount of “noise,” i.e., the unexplained presence and absence of the target, increases with decreasing sampling rate. 

Figure 1

 

Space–time diagrams, adapted from Adelson and Bergen (1985, Figure 4). (a) A target stimulus in sampled motion. A stationary target is presented for a duration D, then disappears during an interstimulus interval ISI before it reappears with a horizontal displacement Δx relative to its previous position, and so on. The stimulus onset asynchrony SOA equals D + ISI. (b) A target stimulus in continuous motion. (c) The difference between the sampled and the continuous version, i.e., the sampling artifacts: White areas indicate positive differences (“unexplained presence”), black areas indicate negative differences (“unexplained absence”), and gray areas indicate equality. (d–f) Diagrams analogous to panels a–c with doubled spatial (Δx) and temporal (D, ISI, SOA) parameter values. The halved spatiotemporal sampling rate results in more sampling artifacts.

If we assume that apparent motion artifacts such as jerkiness depend on the amount of unexplained stimulation, it would of course depend on the spatiotemporal sampling rate. Interestingly, though, it could also depend on other factors. Consider, for instance, the two apparent motion sequences shown in Figure 2. In both of the sequences, the target stimulus is absent in every other frame. The interruptions of target visibility in the left sequence are singular local events, while, in the right sequence, the entire background disappears and reappears along with the target. Thus, the absence of the target could be attributed to the sudden appearance of a mask occluding the entire scene, such that the net amount of unexplained absence of the target would be considerably reduced. In accordance with this idea, we observed that apparent motion sequences sometimes look smoother when structure in the background appears and disappears synchronously with the target. This effect can be seen in Auxiliary Demo 1 

Figure 2

 

Sample sequences (a) with unchanged and (b) with masked background during the ISIs.

In Experiment 1, we investigated how this enhancement in the smoothness of the motion percept depends on the spatiotemporal sampling rate. The main results were that (i) target flicker and (ii) motion jerkiness are reduced when the interruptions of target visibility are synchronous with background masking. The former effect occurred primarily at high sampling rates, while the latter one occurred mainly at low sampling rates. In Experiment 2, the latter result was further substantiated using an objective measure of motion interpolation performance. In Experiment 3, we tested hypotheses regarding the critical variables responsible for the smoothening effect. The results suggest that the improved spatiotemporal interpolation is essentially due to processes of amodal completion in space–time. 

Experiment 1

The aim of this experiment was to determine for which spatiotemporal sampling rates the effect of the global ISI mask on the visibility and jerkiness of the motion target occurs. 

Methods

We presented apparent motion sequences in which a squared green target (side length = 0.3° visual angle, luminance = 53 cd/m ²) was displaced in discrete steps along a horizontal path with a length of 10.7°. The target was shown at position i for a duration D, then disappeared during the interstimulus interval (ISI), before it reappeared at position i + 1, as illustrated in Figures 2 and 3. The stimuli were presented in the central region of a 21-in. CRT monitor running at 85 Hz with a resolution of 1280 × 1024 pixels and were viewed from a distance of ≈100 cm. The background was textured (mean luminance ≈13 cd/m ²) and either remained unchanged during the entire sequence (“unchanged background” condition) or was replaced by a uniform black field (≤1 cd/m ²) during the ISIs (“masked ISI background” condition). The rest of the screen was black. D and ISI were varied independently, each in the following five steps: 47, 141, 235, 329, and 424 ms. The target displacements Δ x were chosen such that the target velocity v was held constant at ≈1.5°/sec in all trials, according to the equation v = Δ x / ( D + ISI). Thus, the displacement steps ranged from 0.15° to 1.3°. Correspondingly, the number of discrete steps in the motion sequences ranged from 9 to 75. The textured background was square with a side length equal to the length of the motion path (10.7°). 

Figure 3

 

Space–time diagrams of sequences (a) with unchanged and (b) with masked background during the ISIs. The target was green and the default background was textured (cf. Figure 2) in the experiment.

The combination of five stimulus durations ( D), five interstimulus intervals (ISIs), and two ISI conditions (unchanged vs. masked background) yields 50 different motion sequences. With six repetitions of each sequence (three times leftward and three times rightward target motion), each subject rated a total of 300 stimuli, presented in random order, over three sessions. 

The task of the subjects was to describe the perceived visibility and motion characteristics of the target. The target could be judged to be (a) permanently visible, (b) flickering, or (c) intermittently invisible. In addition to this “visibility dimension,” subjects were also asked to state whether the target was (1) moving with constant velocity, (2) accelerating and decelerating continuously, or (3) jumping in abrupt jerks. This classification scheme was adopted based on informal observations made in pilot experiments. 

The visibility categories were defined as follows: 

1.  
   Permanently visible: The target is perceived as being continuously visible and constant in color and brightness during the entire sequence.
2.  
   Flickering: The target is perceived as being continuously visible but rapidly changing in color or brightness.
3.  
   Intermittently invisible: The visibility of the target is intermittently interrupted.

The definitions of the motion categories were as follows: 

1.  
   Constant velocity: The target appears to move smoothly with constant speed at all times.
2.  
   Continuous acceleration and deceleration: The target is perceived to move smoothly, but its speed increases and decreases repeatedly. Brief stops are allowed when they are naturally embedded in a deceleration–acceleration cycle and do not appear abrupt.
3.  
   Abrupt jerks: Instead of moving smoothly along a continuous path, the target is displaced in abrupt jerks.

In pilot experiments, we observed that it is quite possible to perceive the target moving smoothly although it clearly disappears intermittently. Conceptually, this percept resembles a light bulb being switched on and off as it moves with constant or gradually changing velocity in the dark. This light bulb analogy was used to sensitize the subjects to the fact that smooth motion of an object is not impossible even when it is only intermittently visible. 

After each trial, subjects entered their judgment by choosing a cell in a 3 × 3 matrix showing the possible combinations of the above visibility and motion dimensions (cf. Table 1). They were also given the opportunity to choose a “neither” response in the event that none of the aforementioned response options were experienced as appropriate. 

Table 1

 

Response matrix with 3 × 3 elements (plus “neither” response).

		(a) Permanent	(b) Flickering	(c) Intermittent
		Visibility
(1) Constant velocity	Motion	1a	1b	1c
(2) Continuous acceleration/deceleration		2a	2b	2c
(3) Abrupt jerks		3a	3b	3c
				Neither

Before the first session started, subjects were verbally instructed and performed a couple of test trials until they felt familiar with the keyboard controls and the categorization scheme. Besides the two authors, five students participated in the experiment. The latter were naive with respect to our hypothesis and received course credits or monetary compensation for their participation. All subjects had normal or corrected-to-normal visual acuity. 

Results

Each row in Figure 4 shows the percentages of responses matching the target visibility categories “permanently visible” (top), “flickering” (middle), and “intermittently invisible” (bottom), pooled across motion categories. In each plot, the response percentages for the unchanged background condition (red) and the masked ISI background condition (black) are plotted against ISI for a fixed target duration D. 

Figure 4

 

Results by visibility category (response frequencies pooled across all motion categories) for conditions with unchanged and masked background during the ISIs. Error bars indicate ±1 SEM across observers.

Permanent target visibility is generally more often reported in the condition with masked background than in the condition with unchanged background, while target flicker is less often reported. These two effects of the ISI background type can be regarded as complementary since they are similar in magnitude throughout (cf. also Figure 5). This complementarity is logically equivalent to the fact that the “intermittently interrupted” percentages are practically identical in both conditions. This follows because the different responses must add to a value of 100% and there were virtually no “neither” responses at all (<0.1% of all trials). Thus, with respect to the target visibility dimension, the main effect of the masked ISI background is to turn target flicker into permanent target visibility. 

Figure 5

 

Dashed curves: Increase in permanent target visibility due to the masked background during the ISIs, i.e., the differences between the two curves in the upper row in Figure 4. Solid curves: The corresponding reduction of flicker, i.e., the differences between the two curves in the middle row in Figure 4. Since both curves are similar throughout, these effects of the ISI background on the target visibility can be regarded as complementary.

Figure 6 shows the percentages of responses by target motion categories “constant target velocity” (top), “continuous acceleration/deceleration” (middle), and “abrupt jerks” (bottom), irrespective of the chosen visibility category. 

Figure 6

 

Results by motion category (response frequencies pooled across all visibility categories) for conditions with unchanged and masked background during the ISIs. Error bars indicate ±1 SEM across observers.

The bottom row shows that the target motion is more often perceived as being jerky when the ISI background remains unchanged, i.e., a masked ISI enhances the perceived smoothness of target motion. This effect is generally more pronounced at longer ISIs. As evident in the top row, there are only minor differences between the percentages of the “constant velocity” category in the two background conditions. Following the same reasoning as above, the effect of background condition on the amount of continuous target acceleration/deceleration (middle row) is complementary to the effect on perceived jerkiness, except for at very short display durations. Analogously to Figure 5, Figure 7 shows how well this complementarity holds. In other words, with respect to the target motion dimension, the main effect of the masked ISI background is to reduce jerkiness in favor of continuous target acceleration/deceleration. 

Figure 7

 

Dashed curves: Increase in continuously accelerated/decelerated target motion due to the masked background during the ISIs, i.e., the differences between the two curves in the middle row in Figure 6. Solid curves: The corresponding reduction of jerkiness, i.e., the differences between the two curves in the bottom row in Figure 6. Since both curves are similar except for brief display durations D, these effects of the ISI background on the target motion can be regarded as complementary.

In summary, using a masked ISI background has two consequences. First, target flicker is reduced in favor of “permanent target visibility,” and second, jerkiness is reduced in favor of continuous target acceleration/deceleration. As illustrated in Figure 8, the former effect tends to be more pronounced at brief ISIs (dashed curves), while the latter tends to be more pronounced at longer ISIs (solid curves). 

Figure 8

 

Dashed curves: The effect of the background mask during the ISIs on target visibility, defined as the mean of the increase in permanent visibility and the reduction of flicker (cf. Figure 5). Solid curves: The effect on perceived target motion, defined as the mean of the increase in continuous acceleration/deceleration and the reduction of jerkiness (cf. Figure 7).

Discussion

This experiment shows that the perception of a target stimulus in apparent motion depends strongly on the nature of the ISI. When the background changes synchronously with the onsets and offsets of the target, the perceived flicker and jerkiness of the target is reduced. More specifically, target flicker is turned into permanent target visibility and the motion appears smoother. 

Experiment 2

The subjective ratings in Experiment 1 suggest that motion is perceived as smoother when the ISI involves global changes. In this experiment, we asked whether this effect also occurs when more objective, indirect measurements are used. By asking subjects to judge the position of a probe flash relative to the perceived position of the target at different points in time, we obtained data allowing us to reconstruct the interpolated motion path. 

Methods

The stimuli were similar to those used in Experiment 1, but the target duration and the interstimulus interval were fixed at the values D = 235 ms and ISI = 329 ms, respectively. The target was displaced about 0.9° during each ISI. These parameter values were chosen because they were found to produce a good smoothening effect in Experiment 1. For expedience, only the 13 first of the original 18 displacement steps from Experiment 1 were presented here. At a specific point of time between the 9th and 10th target onset, a thin white probe flash (luminance ≈ 78 cd/m ²) consisting of two vertically aligned disjoined lines (see Figure 9) was presented for 12 ms. The temporal phase lag of the probe onset relative to the 9th target onset was varied in 12 equally spaced steps covering the entire time interval between the 9th and 10th target onset. 

Figure 9

 

True-to-scale illustration of the stimuli used in Experiment 2.

The task of the subjects was to judge whether the probe was located to the left or to the right of the target's center. An adaptive double-staircase procedure was used to determine the horizontal probe position corresponding to the perceived location of the target. The procedure terminated as soon as six consecutive judgments were within ≈0.1° (≈10% of displacement width), which was typically achieved within about 30 iterations. 

As depicted in Figure 9, the target was presented in a central horizontal textured stripe (10.7° × 1.8°) and the probe was presented on uniform gray regions (luminance ≈ 13 cd/m ²) above and below. The target always “moved” rightwards. 

As in Experiment 1, there were two ISI conditions (unchanged vs. masked background). Combined with the 12 temporal phase lags of the probe flash, this yielded 24 different conditions which were presented in random order. Eleven students, all naive with respect to our hypothesis, participated in the experiment in exchange for course credits. All subjects had normal or corrected-to-normal visual acuity and none of them had any prior experience with similar experiments. 

Results

The mean results are shown in Figure 10. The spatial position at which the target was perceived according to the subjects' adjustments is plotted against the temporal phase lag of the probe flash relative to the target offset. 

Figure 10

 

Mean results across subjects. The spatial position at which the staircases converged are plotted against the temporal phase lag of the probe flash relative to the target offset. Data for the unchanged background condition (red) and masked ISI background condition (black) are plotted separately. The actual positions of the target are shown as gray horizontal line segments, a perfect linear motion interpolation during the ISI is indicated by the dashed gray line. The error bars represent ±1 SEM. Due to the cyclic nature of the display, the position data for the nth cycle should be identical to the position data for the ( n + 1)th cycle except for a shift corresponding to the spatial target displacement. For purposes of illustration, the data have been transposed according to this logic, so that the two first data points are actually plotted last.

As can be seen, both data curves are grossly similar to linear interpolation. However, the data curve for the unchanged background condition is slightly more jagged, in particular at the beginning of the ISI. More interestingly, the variation between subjects was larger in this condition, as indicated by the error bars. 

The fact that the individual variation is larger in the unchanged background condition is more easily seen in Figure 11, where the individual data are plotted separately for both conditions. 

Figure 11

 

Individual data from Experiment 2 for the unchanged background condition (a) and for the masked ISI background condition (b). Conventions as in Figure 10.

Visual inspection of Figures 10 and 11 reveals that the data curves for the unchanged background condition are slightly less smooth than those for the masked ISI background condition. This can be more clearly seen in Figure 12a, where the (discrete) derivates of the curves in Figure 10 are shown, i.e., the “instantaneous” target velocity is plotted against the temporal phase lag. During the ISI, the velocity fluctuates more strongly in the unchanged background condition. Figure 12b plots the absolute change of velocity, which is clearly larger in the unchanged background condition during the ISI. In order to assess the statistical significance of this finding, we computed the difference between the absolute changes of velocity in the two conditions across all temporal phase lags and subjects. The mean of these differences was significantly larger than zero according to a one-tailed t-test ( p = 0.014). 

Figure 12

 

Perceived velocity (a) and perceived absolute change of velocity (b) in the unchanged background condition (red) and in the masked ISI background condition (black). The velocity and the absolute change of velocity were derived from the position data in Figure 10. In panel a, the actual velocity of the target is shown as solid gray horizontal line segments, and the velocity corresponding to a perfect linear motion interpolation during the ISI is indicated by the dashed gray horizontal line segment. Error bars represent ±1 SEM.

Figure 13a explicitly shows that—as already hinted to above—the variations between subjects were generally larger in the unchanged background condition than in the masked ISI background condition. The mean of the differences between the standard errors in the two conditions at the 12 temporal phase lags was significantly different from zero according to a two-tailed t-test ( p = 0.010). This may be taken to suggest that the interpolation task is more difficult to perform when the background remains unchanged. If so, one would also expect larger within-observer variability in the unchanged condition. Since a staircase procedure was used, however, we do not have such data, but the number of adjustments needed for the staircase to converge may serve the same function. As shown in Figure 13b, the staircases did indeed converge later in the unchanged background condition. The mean of the differences with respect to this parameter between the two conditions, pooled across all temporal phase lags and subjects, was significantly different from zero (two-tailed t-test, p = 0.008). 

Figure 13

 

(a) SEM of the position data in Figure 10 for the unchanged background condition (red) and for the masked ISI background condition (black). (b) Mean number of adjustment steps until the staircase procedure converged (colors as in panel a). Due to the staircase design, at least 18 adjustment steps were necessary. Error bars represent ±1 SEM.

The results of this experiment are summarized in Figure 14. First, the absolute change of velocity was larger in the unchanged background condition both (a) in terms of the mean and (b) in terms of the maximum, suggesting that motion was jerkier. Second, two measures which may be taken to reflect the difficulty of the interpolation task are larger in the unchanged background condition, namely, (c) the variation between subjects and (d) the number of adjustment steps necessary for the staircase procedures to converge. 

Figure 14

 

Summarized results of Experiment 2 for the unchanged background condition (red) and for the masked ISI background condition (black). (a) Absolute change of velocity, averaged first across all temporal phase lags and then across subjects. (b) Maximum absolute change of velocity, determined for each subject and then averaged. (c) Standard errors of the mean position data in Figure 13 (a), averaged across all temporal phase lags. (d) Number of steps in the staircases, averaged first across all temporal phase lags and then across subjects. Error bars in panels a, b, and d represent ±1 SEM across subjects.

Discussion

On a coarse level, the interpolated motion paths reconstructed from the data averaged across all observers were fairly similar in the unchanged and masked background conditions. Finer analysis, though, revealed that the absolute changes of velocity were generally larger in the unchanged background condition. This may be interpreted to mean that the visually interpolated motion is jerkier in that case. It cannot be ruled out, however, that this may be an artifact related to the second major finding of this experiment, namely, that the interpolation task seems to be performed with a higher degree of uncertainty in the unchanged background condition. 

Position judgments for an apparent motion stimulus during the ISI, that is, at moments in time when the target is actually absent, are only possible by way of interpolation. Therefore, one would expect the precision of position judgments to be better under conditions where the visual system generates a stable perceptual representation of the motion path (Burr, 1979). By this logic, the results of the present experiment would suggest that the visual interpolation is facilitated in the masked ISI background condition. 

Considering that the unchanged background condition is the presumably simpler motion stimulus, this may seem slightly surprising. One possible explanation for this finding appeals to the notion that the onsets and offsets of the target may be taken to signal that the target is “materializing” and “dematerializing” rather than being continually present. In the masked ISI condition, where the onsets of the target are synchronous with the offsets of the mask and vice versa, the appearance and disappearance of the target can be attributed to the mask rather than to the target itself so that it is no more suggesting that the target repeatedly comes into existence and ceases to exist. When the target is interpreted as being continuously present behind the mask, an amodal motion interpolation is called for (Burke, 1952; Michotte & Burke, 1951). 

Experiment 3

The results of Experiments 1 and 2 agree in suggesting that the perceptual motion interpolation is superior in the masked ISI background condition. A possible explanation of this would be that the global mask provides cues to occlusion which in turn leads to amodal motion interpolation. Framed this way, the critical difference between the two conditions is the presence or absence of occlusion cues. The two conditions do, however, differ in other regards so that alternative explanations cannot be ruled out. For instance, only the masked ISI condition contains global transients. These may draw attention away from the target, possibly making the deviations between the actual sampled stimulus and an ideal one less conspicuous. Such an explanation would, for instance, be consistent with Yeshurun and Levy's (2003) finding that apparent motion looked better under conditions with reduced attention. A further confounding factor in our experiments was that the ISIs had different mean luminances and contrasts in the two conditions. In the present experiment, we sought to determine which of these and other potential stimulus variables are responsible for the smoothening effect. 

Methods

The stimuli were similar to those in Experiment 1. In addition to the textured and the uniform black backgrounds used there ( Figures 15a and 15b), three further background types were used ( Figures 15c– 15e). The height of the central horizontal stripe in panels d and e was about 0.4° and the height of the flanking regions above and below was about 5.15°. Together, the stripe and the flankers formed a square with a side length of 10.7° and were presented in the center of the screen. The rest of the monitor was black. Every background type could be used both when the target was present and when it was not (i.e., as a mask during the ISI). Thus, 5 × 5 = 25 background/mask combinations were possible. We investigated all of these except c/d, d/c, c/e, and e/c. As in Experiment 2, we held both the display duration D and the length of the interstimulus interval (ISI) constant at 235 and 329 ms, respectively. The target always “moved” rightwards. Every condition was repeated 10 times and the resulting 210 trials were presented in random order. Ten of the 11 participants of Experiment 2 took part in this experiment, too. Their task was the same as in Experiment 1, that is, the subjects judged each stimulus according to the 3 × 3 response matrix ( Table 1). 

Figure 15

 

Stimuli used in Experiment 3. (a–b) Textured and uniform black backgrounds as used in Experiment 1. (c) A uniform gray background with a luminance equal to the mean luminance of background a. (d–e) “Stripe versions” of backgrounds a and b, respectively. (f) Note that the panels (a–e) only show the central portion of the stimuli corresponding to the dashed inset of this true-to-scale sketch of the stimuli. Note also that the green target was not present during the ISI frames.

Results and discussion

As in Experiment 1, there were virtually no “neither” responses (<1% of all trials). All except 1 of the 10 participants almost always reported that the target was intermittently invisible in all conditions. Thus, differences in the stimuli in this experiment effectively only influenced the “motion” dimension of the response matrix ( Table 1), where the three possible response categories were (1) “constant velocity,” (2) “continuous acceleration/deceleration,” and (3) “abrupt jerks.” 

The differences between the stimuli mainly influenced the frequency f ₃ of “abrupt jerks” responses. The sum of the frequencies of the two remaining categories f ₁ + f ₂, which both indicate smooth motion, did of course vary accordingly, since f ₁ + f ₂ + f ₃ ≈ 1, but the ratio f ₁/( f ₁ + f ₂) varied only moderately (between 0.3 and 0.61). The frequencies f ₃ of “abrupt jerks” responses are shown in Figure 16. The stimulus conditions, indicated by the icons on the left, are sorted in order of increasing jerkiness. 

Figure 16

 

The frequency of “abrupt jerks” responses in Experiment 3 for each background condition, indicated by the icons on the left. Conditions are sorted in order of increasing jerkiness. Error bars represent ±1 SEM. The columns on the right indicate whether a stimulus condition fulfills the structural criteria described in the text.

Conditions 2 and 18 were replications of the masked ISI background condition and the unchanged background condition of Experiment 1, respectively, both for the temporal parameters D = 235 ms and ISI = 329 ms (cf. Figure 6, bottom panel). In Experiment 1, the frequency of “abrupt jerks” responses decreased from 43% to 7% when the ISI background was masked. A similar decrease from 61% to 13% was observed in the present experiment. 

Distributed attention and synchronous changes in the remote flanker regions

Because the masked ISI condition in Experiment 1 contains background transients that are absent in the unchanged background condition, it is not unlikely that attention is drawn away from the target. This could make the differences between the sampled motion stimulus and an ideal continuous one less conspicuous and the jerkiness less noticeable. Based on this hypothesis, one would expect that the presence and absence of changes in the flanker regions should have a significant influence on the effect. This is not the case as can be seen by comparing Conditions 2 and 4, which are identical except that only the former contains transients in the flanker regions. Nevertheless, they were both perceived as very smooth. The same argument applies to Conditions 18 and 19. Only the latter contains transients in the flanker regions, but in both cases the motion was judged to be very jerky. Therefore, global transients are neither a necessary nor a sufficient condition for the effect. 

Synchronous changes in the central stripe region

Synchronous background changes in the stripe region are also neither necessary nor sufficient. To see this, consider that the smoothness effect is very strong both in Conditions 1 and 2, although the latter but not the former contains transients in the central stripe region. Consider also that Condition 20 produced a very jerky percept although it contains transients in the central region. 

Textured vs. uniform ISIs

One possible explanation of jerkiness appeals to the notion that visual motion is based on two sources of information, namely, background-relative position signals and pure motion signals. Sampled motion stimuli could be jerkier when position signals dominate over pure motion signals and smoother when the converse is the case. When a uniform surround is used instead of a textured one, background-relative position signals will be largely absent. This reasoning may account for the smoothening effect observed in Experiment 1. The results of Experiment 3, however, suggest that uniformity in the central region of the ISI background alone is not in itself a sufficient condition for the smoothening effect, since Condition 21 yielded very strong jerkiness. Global uniformity of the ISI background does not seem to be a sufficient condition either, since Condition 13 yielded rather strong jerkiness. It is interesting to note, however, that in the six conditions yielding the smoothest motion percepts (1–6), the ISI background is uniform at least in the central region. Thus, although competition between position signals and motion signals can not account for all features of our findings, it may at least influence the perceived smoothness of sampled motion stimuli. (We are indebted to one of the anonymous reviewers for drawing our attention to this possibility.) 

Luminance and contrast differences between target and ISI background

A further difference between the two conditions in Experiment 1 was that the luminance difference between target and the ISI background was larger in the masked ISI background condition. The present results, though, show that changes in this luminance difference are not critical. The difference between the luminance of the target and the mean luminance of the ISI background is the same in Conditions 6 and 18, but the jerkiness ratings are three times as frequent in the latter case. A similar argument can be made by comparing Conditions 2 and 21. Here, the frequencies differ by a factor of five. 

Since the target was always uniform, second-order differences (“contrast–contrast”) between target and ISI background were present whenever the ISI background was textured. The results of Experiment 1 could be taken to suggest that absence of “contrast–contrast” facilitates the effect. Since this was only the case for uniform ISI backgrounds, the same conclusions can be drawn as in the above section. That is, the absence of second-order differences between target and ISI background is not sufficient for the smoothening effect but potentially necessary. 

Background scene and disruptions of visibility

When two different frames are alternating cyclically, this may be open to several different interpretations. It may be “literally” interpreted as two different scenes which continuously disappear and reappear in counterphase. Alternatively, one of the two frames may be interpreted as a background scene, the visibility of which is repeatedly disrupted by an extraneous factor. Whether such a segmentation occurs and which of the frames is categorized as scene is likely to depend on the structural properties of the two frames. For instance, if one frame contains structure that is absent in the other, the former may be more likely to be categorized as scene, while the other is categorized as disruption. 

If a target such as the one used in our experiments is presented in the frame categorized as background scene, its onsets and offsets can be attributed to the offsets and onsets of a more global disruption, given that the disruption encompasses both the target and adjacent regions. This would make the alternative interpretation that the target itself suddenly “materializes” and “dematerializes” unnecessary; the target object would continue to exist during the disruptions and perceptual interpolation would be called for. Thus, the smoothening effect should occur whenever the ISI frame—rather than the target frame—is characterized as a disruptor. One may expect this to occur whenever the following criteria are fulfilled: 

1.  
   No structural gain: The ISI frame does not contain structure which is absent in the target frame (Criterion 1).
2.  
   Structural loss: The target frame contains structure which is absent in the ISI frame (Criterion 2).
3.  
   Target enclosure: The structural loss occurs in the region immediately surrounding the target (Criterion 3).

As can be seen in Figure 16, these three criteria are simultaneously met for the conditions in which the smoothness effect was most pronounced (1–6) and not for the remaining conditions which all had higher jerkiness ratings. It may, at first sight, not be obvious why we have categorized Condition 3 as fulfilling Criteria 2 and 3. This becomes clear when one considers that the regions of the monitor outside the central nominal background were black. Therefore, the entire monitor surface contains more structure when the central canvas is gray than when it is black. 

Conclusions

Many of the factors we have considered seem to be loosely associated with the smoothening effect, but most of them fail to be sufficient in themselves. An explanation in terms of background and disruptions of visibility does however seem to capture the major aspects of the data fairly well. According to this hypothesis, the structural relations between target frame and ISI frame determine how the visual system categorizes the two frames as scene and disruption. 

Discussion

Our experiments agree in indicating that the perceptual interpolation of a target in sampled motion can be improved by introducing transients in the background which are synchronous with the onsets and offsets of the target. In Experiment 1, we found that target flicker is turned into permanent target visibility and that motion appears smoother when the structured background is replaced by a uniform black mask during the ISI. The reduction of flicker was most noticeable at lower ISI levels, while the smoothening effect was more predominant at longer ISIs. In Experiment 3, we investigated how the smoothening effect depends on the spatial characteristics of the backgrounds presented during the presence and absence of the target. An explanation in terms of distributed attention due to global transients was not supported by the data. Neither could a number of simple stimulus variables such as uniformity of the ISI background, luminance contrast, and second-order differences between target and ISI mask account for the smoothening effect. The most promising and parsimonious hypothesis seems to be that the smoothening effect occurs whenever the ISI frame is interpreted as an extraneous disruption interrupting the visibility of the target and the scene background simultaneously. Whenever this is the case, the target onsets and offsets can be attributed to the intermittent global disruptions of visibility, which would indicate that the target continues to exist rather than to “materialize” and “dematerialize” repeatedly. This would make the activation of perceptual interpolation mechanisms reasonable (Michotte & Burke, 1951). 

In Experiments 1 and 3, the quality of the perceptual interpolation was measured using a rating task. In Experiment 2, we asked whether the smoothening effect could also be measured by comparing the position of a probe flash with the perceived position of the motion target at different points of time. The global shape of the perceived motion path measured with this method did not depend very strongly on the nature of the ISI. Finer analysis of the data, however, revealed that the absolute changes of velocity were generally larger in the unchanged background condition. This may be taken to suggest that the perceived motion is indeed jerkier in this condition. It may, however, also be an artifact related to the second major finding of Experiment 2, namely, that the interpolation task seems to be performed with more uncertainty in that condition. This was indicated both by the variance across observers and the speed with which the staircases converged. A weakness of the method employed in this experiment is that the interpolation task may not only be based on the immediate perceptual impression but also on conscious cognitive inference. A priori, it is quite conceivable that interpolation based on cognitive inference will yield essentially the same results as interpolation based on immediate perceptual impression with respect to the shape of the interpolation path. One would, however, expect interpolation based on a dedicated perceptual mechanism to be more precise and more consistent across observers. Therefore, our results are quite consistent with the hypothesis that the output of perceptual interpolation mechanisms was available to observers in the masked ISI background condition, but only to a lesser degree in the unchanged background condition. 

Possible explanations

Many models of motion detection (e.g., Hock & Gilroy, 2005) are primarily concerned with predicting whether motion is perceived or not and do not address the issue of motion smoothness directly. The Reichardt model and its variants (Reichardt, 1957; van Santen & Sperling, 1985; van Wezel, Lankheet, Fredericksen, Verstraten, & van de Grind, 1997; Zanker, 1994), though, can be used to derive predictions concerning motion smoothness. In our experiments, however, the smoothening effect primarily occurred at rather long ISIs which are probably beyond the temporal range across which Reichardt detectors integrate (van Doorn & Koenderink, 1982). Nevertheless, we performed simulations using Adelson and Bergen's (1985) spatiotemporal energy model to see whether the smoothening effect can be explained using this type of model. We found that the output of the energy model did not differ for the unchanged background and the masked ISI background conditions. As an example, the results of the simulation for a particular set of parameter values are shown in Figure 17. It is therefore not obvious to us how the smoothening effect could be an artifact related to the specific workings of early motion detectors. 

Figure 17

 

Space–time diagrams of a stimulus in sampled rightward motion (a) without and (b) with an ISI mask. The corresponding outputs of Adelson and Bergen's (1985) energy model are shown in panels d and e. The gray of the background represents zero output and white represents rightward motion energy. Panel c represents the pixel difference between the stimuli in panels a and b. Panel f represents the pixel difference between the detector outputs in panels d and e, which was negligibly small.

Perhaps, then, it is more helpful to think of the smoothening effect as a result of the rules according to which the onsets and offsets of the target are interpreted by the visual system. Target onsets and offsets can be thought of as object borders in time. Drawing an analogy with object borders in space, for which well-developed theoretical concepts are available in the literature (Kanizsa, 1979; Nakayama, Shimojo, & Silverman, 1989; Rubin, 1915/1958), may provide some insight here. Border ownership, which has proved to be of central importance for our understanding of the spatial interpolation phenomenon of amodal completion, is a case in point. The basic observation is that a border between two regions in the visual field can be attributed to, or “owned” by, an object corresponding to either of the regions. As soon as the border is attributed to one of these objects, there is no reason to infer that the other object ends there, too. Instead, it is natural to assume that it extends further behind the first object. Thus, the assignment of border ownership determines which of the objects is amodally completed behind the other. This standard reasoning can be applied analogously to our stimuli if one thinks of the target onsets and offsets as borders in time. If a target onset (offset) is “owned” by the target itself, it would indicate that the target comes into existence (ceases to exist). In contrast, if the target onset (offset) is “owned” by the ISI mask, the indication is that the target was already in existence (continues to exist). Therefore, amodal interpolation in space–time is called for (Michotte & Burke, 1951; Michotte, Thinès, & Crabbé, 1991). 

From this perspective, the critical variable that determines whether the smoothening effect occurs or not is whether the target transients are “owned” by an ISI mask. Our data suggest that the border ownership assignment depends on the structural relations between the two alternating backgrounds. The target transients tend to be “owned” by the ISI background whenever it contains less structure than the target background in the region immediately surrounding the target. 

Why should this be so? That a given spatial pattern or structure disappears and reappears completely unchanged after a brief period of time would be a highly unlikely event if the structures do not represent the same visual scene. Therefore, the intermittent absence of the structure would most plausibly be attributed to an external factor interrupting the visibility of the visual scene. 

When a structured pattern is replaced by a uniform mask, spatiotemporal T-junctions emerge. Thus, spatiotemporal T-junctions may function in much the same way as spatial T-junctions (Anderson, Singh, & Fleming, 2002; McDermott & Adelson, 2004a, 2004b), which are thought to serve as local cues resolving border ownership ambiguities and indicating occlusion relationships. This has previously been suggested by Holcombe (2002). 

Conclusions

Our results suggest that flicker and jerkiness of sampled motion stimuli are reduced in conditions where amodal completion is supported by occlusion cues. Thus, occlusion cues can not only suppress apparent motion as previously shown by Sigman and Rock (1974) but also enhance the perceptual quality of sampled motion. This underscores the importance of amodal representations and internal structure, which has previously been demonstrated in several domains of perception (e.g., Bregman, 1990; Mausfeld, 2002; Miller, Dibble, & Hauser, 2001; Miller & Licklider, 1950; Noë, 2005; Öǧmen, 2007; Shimojo & Nakayama, 1990). It also agrees with a large body of previous research demonstrating an intimate link between occlusion cues and motion percepts (Duncan, Albright, & Stoner, 2000; McDermott, Weiss, & Adelson, 2001; Shimojo, Silverman, & Nakayama, 1989; van der Smagt & Stoner, 2008; Wallach, 1935). 

Supplementary Materials

Movie 1 - Movie 1 

Acknowledgments

We are indebted to two anonymous reviewers and Franz Faul for helpful suggestions. We also thank Laura Kornmayer and Leif Trampenau for technical assistance. Supported by grant EK 71/1-1 from Deutsche Forschungsgemeinschaft to V. E. 

Commercial relationships: none. 

Corresponding author: Tom R. Scherzer. 

Email: [e-mail]. 

Address: Olshausenstr. 62, D-24118 Kiel, Germany.