The lawful perception of apparent motion

Abstract

Visual apparent motion is the experience of motion from the successive stimulation of separate spatial locations. How spatial and temporal distances interact to determine the strength of apparent motion has been controversial. Some studies report space–time coupling: If we increase spatial or temporal distance between successive stimuli, we must also increase the other distance between them to maintain a constant strength of apparent motion (Korte's third law of motion). Other studies report space–time tradeoff: If we increase one of these distances, we must decrease the other to maintain a constant strength of apparent motion. In this article, we resolve the controversy. Starting from a normative theory of motion measurement and data on human spatiotemporal sensitivity, we conjecture that both coupling and tradeoff should occur, but at different speeds. We confirm the prediction in two experiments, using suprathreshold multistable apparent-motion displays called motion lattices. Our results show a smooth transition between the tradeoff and coupling as a function of speed: Tradeoff occurs at low speeds and coupling occurs at high speeds. From our data, we reconstruct the suprathreshold equivalence contours that are analogous to isosensitivity contours obtained at the threshold of visibility.

Introduction

In this article, we propose a solution to a long-standing controversy about the perception of apparent motion. Apparent motion is produced by a sequence of frames portraying a stimulus at different locations (Nakayama, 1985; Ullman, 1979; Wertheimer, 1912). To describe the problem, consider a display in which short-lived dots appear sequentially at three loci: o, a, and b (Figure 1A). Suppose the spatial distance between a and b is very long, and motion from a to b is unlikely. Then, the dot at o has two potential matches: a and b. Because the dot at o has two potential matches, the display is ambiguous; one can perceive motion either from o to a or from o to b. 

Figure 1

 

Perceptual equilibrium in apparent motion. (A) A stimulus for ambiguous apparent motion is depicted in two spatial coordinates (x, y) and one temporal coordinate. The spatial projection of the stimulus is shown in the inset. Element o has two potential matches: a (giving rise to motion path ) and b (giving rise to motion path ). Each motion path has two parameters: temporal distance (T_a or T_b) and spatial distance (S_a or S_b). (B) In the distance plot, each motion path is represented by a point {T_i, S_i}. We constrain the parameters such that T_a and S_a are fixed and T_b = 2T_a. The spatial distance S_b is allowed to vary to determine the point of perceptual equilibrium where the competing motions are equally likely to be seen (Figure 2). Note that the distance plot does not represent information about the direction (and, therefore, about velocity) of motion.

We can find combinations of spatial and temporal components of the competing motion paths such that they are seen equally often. Under these conditions, we say that the two motion paths are in “perceptual equilibrium”. According to some measurements (Koffka, 1935/1963; Korte, 1915), equilibrium can be observed only when the spatial and temporal components of one motion path are longer than the spatial and temporal components of the other motion path. This has been called Korte's third law of apparent motion. We call this result “space–time coupling”. According to other measurements (Burt & Sperling, 1981), equilibrium can be observed only when the spatial component of one path is longer than the spatial component of the other, whereas the temporal component of the first is shorter than the other. We call this result “space–time tradeoff”. In this article, we show that by changing the conditions of stimulation, we can cause the pattern of results to change smoothly from space–time coupling to space–time tradeoff. 

We develop our argument in three steps. (a) We examine how the inconsistent results in the motion literature are related. (b) We show that both coupling and tradeoff are consistent with a normative theory of motion measurement and with human sensitivity to continuous motion, which suggests that both tradeoff and coupling may occur, but under different conditions. (c) We confirm this in two experiments and show that coupling and tradeoff are special cases of a simple law. 

Regimes of apparent motion

In the motion display illustrated in Figure 1A, let us denote the motion path from o to a by m _a and from o to b by m _b. Let us denote the (potential) percepts of motion along these paths by μ _a and μ _b and the strength of the apparent motion experienced when μ _a or μ _b is seen in isolation by

${\mathcal{A}}_{\mathrm{a}}$

and

${\mathcal{A}}_{\mathrm{b}}$

, respectively. Each path has a temporal ( T _a and T _b) and a spatial ( S _a and S _b) component ( Figure 1A). The two motion paths are in perceptual equilibrium for such combination of conditions ( S _a, T _a) and ( S _b, T _b) that the two motions are seen equally often and their strengths are equal:

${\mathcal{A}}_{\mathrm{a}}$

=

${\mathcal{A}}_{\mathrm{b}}$

. The controversy concerns the relationship between the pair ( S _a, T _a) and the pair ( S _b, T _b) when the two motions are in equilibrium:

•  
  Coupling: Equilibrium is obtained when S _a and T _a are both longer or both shorter than S _b and T _b. This is Korte's THIRD LAW OF APPARENT MOTION (Koffka, 1935/1963; Korte, 1915).
•  
  Tradeoff: Equilibrium is obtained when S _b > S _a and T _b < T _a or S _b < S _a and T _b > T _a. This result was obtained by Burt and Sperling (1981).

Let us represent each motion path by a point in a plot of spatial and temporal distances: a distance plot ( Figure 1B). Suppose we hold the spatial and temporal components of m _a constant, so it is represented in the distance plot by a fixed point. Suppose also that we hold the temporal component of m _b constant, twice as long as the temporal component of m _a: T _b = 2 T _a. Then, we can vary the spatial component S _b and find the value of r _ba = S _b/ S _a for which μ _a and μ _b are in perceptual equilibrium: p( μ _a) = p( μ _b) = 0.5. This manipulation is represented in the distance plot by moving the point for m _b along the line T _b = 2 T _a ( Figure 1B). 

This manipulation can give rise to the three outcomes illustrated in Figure 2: coupling ( r _ba > 1), tradeoff ( r _ba < 1), and an intermediate condition that we call “time independence” ( r _ba = 1). Each of these outcomes corresponds to a different slope of the line connecting the representation of the competing motion paths in the distance plot: positive slope for coupling, negative slope for tradeoff, and zero slope for time independence. We will use the notion of slope between equivalent conditions in the distance plot to relate results on apparent motion to predictions of a normative theory of motion measurement and to data on human spatiotemporal sensitivity. Each of the three regimes has played a role in the literature on motion perception, as we show next. 

Figure 2

 

Hypothetical conditions of perceptual equilibrium between two paths of apparent motion, m _a and m _b, in Figure 1. The temporal and spatial distances { T _a, S _a } between successive elements in m _a are represented by Point 1, which is fixed. The temporal distance of m _b is also held constant at T _b = 2 T _a. We vary the spatial distance S _b to determine its value when m _b is in equilibrium with m _a, that is, p( μ _a) = p( μ _b) = 0.5. Space–time tradeoff occurs whenever S _b < S _a, in the dark gray region (e.g., at Point 2). Space–time coupling occurs whenever S _b > S _a, in the light gray region (e.g., at Point 4). Time independence—or “shortest spatial path”—occurs at S _b = S _a (Point 3).

Space–time coupling

Space–time coupling means that the equilibrium between m _a and m _b occurs when m _b is longer than m _a both in time ( T _b > T _a) and in space ( S _b > S _a). It is illustrated in the distance plot ( Figure 2) by the positive-slope line between 1 and 4. 

An example of coupling is Korte's third law of motion (henceforth “Korte's law”; Koffka, 1935/1963; Korte, 1915; Lakatos & Shepard, 1997; Neuhaus, 1930). Using a tachistoscope, Korte presented his observers with two brief visual stimuli separated by variable spatial and temporal distances. First, he found a space–time pair that gave rise to a compelling experience of motion (“good motion”). Then, he varied the spatial and temporal distances between the two stimuli and recorded the observers' descriptions of the motion. From these descriptions, he derived a rating of motion strength (Figure 3). He found that when conditions for good motion held, he could not change just the spatial distance or just the temporal distance without reducing the strength of motion. To restore the experience of good motion, he had to increase or decrease both. 

Figure 3

 

Data reconstructed from Korte (1915). The rows contain data for different observers; and the columns, for different interstimulus intervals (ISI). The highest rating corresponds to good motion. To maintain the experience of good motion, the temporal distance between successive stimuli and their spatial distance should be both increased or both decreased.

Sometimes, Korte's law is portrayed as evidence of speed invariance (e.g., Koffka, 1935/1963; Kolers, 1972). This would be the case only if the space–time pairs that correspond to the same speed of motion had also corresponded to the same strength of apparent motion. In our terms, spatial and temporal distances in the conditions of equilibrium would then be related directly: S = vT, where speed v is a positive constant. 

Note that if v denotes physical speed, Korte's data do not provide support for speed invariance. However, if perceived speed is meant, and if perceived speed is related to physical speed nonlinearly, then Korte's data could be consistent with perceived speed invariance. In that case, the physical spatial and temporal distances in the conditions of equilibrium would be related by S = η( v) T, where v is physical speed and η( v) is a nonlinear function. 

Korte's law has been surprising researchers ever since it was proposed. Koffka (1935/1963, p. 293)—Korte's dissertation supervisor—wrote many years later:

 

The law has puzzled psychologists … at the time of Korte's work one was still inclined to think as follows: if one separates the two successively exposed objects more and more, either spatially or temporally, one makes their unification more and more difficult. Therefore increase of distance should be compensated by decrease of time interval and vice versa.

 

They expected tradeoff but found coupling. 

The low-speed assumption

A special case of coupling is the “low-speed assumption”, motivated by a phenomenon first reported by Wallach (1935), known today as the “aperture problem” (Adelson & Movshon, 1982; Ullman, 1979). When you look at a moving line through an aperture that occludes its terminators, you are likely to see it moving orthogonally to its orientation, even when it is moving in other directions, and because a line moving along the orthogonal path is moving at the lowest speed consistent with its displacement, Wallach (1976) attributed the phenomenon to the visual system's preference for slower motion. This assumption has been used to specify the prior distribution of motion estimates in a Bayesian model of motion perception (Hürlimann, Kiper, & Carandini, 2002; Stocker & Simoncelli, 2006; Weiss & Adelson, 1998; Weiss, Simoncelli, & Adelson, 2002; see also Heeger & Simoncelli, 1994, and van Hateren, 1993). 

A preference for slow physical speeds implies that perceptually equivalent space–time pairs should fall on the line of constant speed (1 and 5 in Figure 2): If m _b had corresponded to locations above 5 on the 2 T _a line, the speed of m _a would be slower than the speed of m _b, and m _b would lose the competition with m _a. But if m _b had corresponded to locations below 5 on the 2 T _a line, then the speed of m _b would be slower than that of m _a, and m _b would win the competition. Thus, a preference for slow physical speeds implies coupling. 

Space–time tradeoff

Space–time tradeoff means that the equilibrium between m _a and m _b occurs when m _b is longer than m _a in time ( T _b > T _a) but is shorter than m _a in space ( S _b < S _a). It is illustrated in the distance plot ( Figure 2) by the negative-slope line between 1 and 2. 

As we mentioned earlier, Burt and Sperling (1981) obtained evidence for tradeoff. They used ambiguous apparent-motion displays: Observers could see motion along one of several directions. The display consisted of a succession of brief flashes of a horizontal row of evenly spaced dots. Between the flashes, the row was displaced horizontally and downward, so that observers could see the row move downward and to the right or downward and to the left, parallel to one of several orientations, three of which are shown in Figure 4A (paths p₁, p₂, and p₃). The interstimulus interval, T, was constant within a display. To measure strength of motion, Burt and Sperling derived new stimuli from the one shown in Figure 4A by deleting subsets of dots. For example, when every other dot in the row was deleted, the spatial and temporal separations along path p₂ doubled without affecting path p₁ (Figure 4B). Now, the dominant path becomes p₁. Burt and Sperling used two methods to measure strength of apparent motion: rating and forced choice:

•  
  Rating: On each trial, observers saw two displays (such as those shown in Figure 4) in alternation and used a seven-point scale to rate the strength of the motion along p ₁ in one display compared with the other, R( p ₁): 0 = I can't see p ₁ in the first display, 5 = p ₁ is equally strong in both displays, and 6 = p ₁ is stronger in the second display. They also recorded ratings for the strength of motion along p ₂, R( p ₂). They plotted R( p ₁) and R( p ₂) as a function of T. As T increased, R( p ₁) increased and R( p ₂) decreased. They called the value of T at which these two functions crossed, T ^* _1,2, the transition time between the two paths.
•  
  Forced choice: They presented each stimulus at several values of T. They designed the stimuli so that one motion was to the left of vertical and the other to the right. On each presentation, observers reported the direction of motion. Using the results of the rating experiment, they selected spatial parameters so that for each stimulus, the selected values of T were smaller and larger than a transition time, to find the transition time by interpolating the proportions of responses across the tested magnitudes of T.

Figure 4

 

Stimulus of Burt and Sperling (1981). (A) A horizontal row of dots is flashed sequentially while it is displaced horizontally and downward. The rows of unfilled dots are occupied by the row of dots across time: Every row is occupied once, at time T_i, designated on the right. Under appropriate conditions, one perceives a flow of motion along a path p_i. The three paths marked by arrows are most likely. (B) Removing every second dot in the row affects the spatial and temporal parameters of paths p₂ and p₃, but not of path p₁.

They used the rating method to study the effect of T and the forced-choice method to study the interaction of T with spatial parameters. Across all conditions, they found tradeoff. The authors concluded that their data were incompatible with Korte's law and conjectured that Korte's methodology had been faulty. 

Time independence

Between coupling and tradeoff lies time independence, or the preference for the “shortest [spatial] path” (Wallach, 1935; see also Wallach, 1976). Wallach (1935) proposed that the visual system prefers the shortest spatial path in perception of apparent motion before he offered the low-speed assumption. Wallach thought that orthogonal motion is preferred because it allows the line to travel the shortest spatial distance. Such a preference for the shortest spatial path has been assumed in many other studies that used ambiguous motion displays (Hock, Schöner, & Hochstein, 1996; Kolers, 1972; Ramachandran & Anstis, 1983; Ullman, 1979; von Schiller, 1933). The shortest path hypothesis is a straightforward generalization of the spatial proximity principle to space–time: It simply ignores temporal parameters. 

Prospect of resolving the controversy

Data on human spatiotemporal sensitivity and a normative theory on motion measurement suggest that the regimes of tradeoff and coupling might both exist, but for different parameters of stimulation. 

Many studies of human and animal vision have explored spatiotemporal contrast sensitivity (Burr & Ross, 1982; Nakayama, 1985; Newsome, Mikami, & Wurtz, 1986; van de Grind, Koenderink, & van Doorn, 1986; van Doorn & Koenderink, 1982a, 1982b). A comprehensive summary of human contrast sensitivity was obtained by Kelly (1979). In Figure 5A, we plot Kelly's estimates of human sensitivity to drifting sinusoidal gratings converted into the format of our distance plot. (See Gepshtein, Tyukin, & Kubovy, 2007, for the details of conversion.) The colored contours are the isosensitivity contours: Each contour connects the conditions for which human observers are equally sensitive to drifting sinusoidal gratings. Kelly and others found that for every speed, there exists a condition for which contrast sensitivity is maximal. The data about maximal sensitivity for motion from many studies are consistent with Kelly's data, as summarized by Nakayama (1985). 

Note that Kelly (1979) obtained his estimates of spatiotemporal sensitivity using an image stabilization technique that allowed him to precisely control motion on the retina. As Kelly and Burbeck (1984) observed, those data had the same form as the data obtained with no image stabilization (Kelly, 1969, 1972; Kulikowski, 1971; Robson, 1966; van Nes, Koenderink, Nas, & Bouman, 1967). Thus, the data obtained by Kelly (1979) can be used to interpret results from studies that did not use image stabilization. 

Figure 5

 

Empirical and theoretical equivalence classes. The format of both panels is similar to that of Figure 2, except that the two axes are logarithmic. (A) Human isosensitivity contours. Each contour is a spatiotemporal isosensitivity curve. The color of each curve corresponds to the normalized magnitude of sensitivity, as explained in the color bar on the right. (B) Theoretical equivalence contours predicted by the normative theory of Gepshtein et al. (2007). The roughly hyperbolic curve connects conditions expected to be optimal for speed estimation across speeds. The colored curves connect conditions equally suboptimal for motion measurement and hence expected to be equally preferable: The warmer the color, the more resources should be allocated to the corresponding contour and the higher sensitivity is expected. The pairs of connected circles in Panels A and B demonstrate that the regimes of coupling (at a high speed) and tradeoff (at a lower speed) are expected from normative considerations (in Panel B) and are consistent with data on human spatiotemporal sensitivity (in Panel A). The vertical arrows correspond to the arrow in Figure 1B; they illustrate the procedure we use to reveal the regimes of coupling and tradeoff.

Gepshtein et al. (2007) proposed an equilibrium theory of motion perception. They investigated how limited neural resources (a limited number of neurons that can be tuned to speed) should be allocated to different conditions of stimulation and found that human spatiotemporal sensitivity approximately follows the optimal prescription. In this theory of Gepshtein et al., resources should be allocated according to the degree of balance between measurement uncertainties and stimulus uncertainties. The authors estimated measurement uncertainties using the uncertainty principle of measurement; they estimated stimulus uncertainty from measurements of speed distribution in the natural ecology (Dong & Atick, 1995). Using these estimates, Gepshtein et al. derived optimal conditions and to equally suboptimal conditions for speed measurement. 

According to equilibrium theory, at every speed more resources should be allocated to the optimal condition than to other conditions yielding a normative prediction of the maximal sensitivity for that speed (the grey curve in Figure 5B). Similarly, equal amount of resources should be allocated to equally suboptimal conditions, yielding normative predictions for the isosensitivity sets (the colored contours in Figure 5B). According to the theory, the maximal sensitivity set has a roughly hyperbolic shape, and the isosensitivity sets form closed contours whose shapes conform to the shape of the maximal sensitivity set. The theory further predicts that the slope of every isosensitivity contour should change smoothly from negative to positive: negative slopes should dominate at low speeds (the right part of the distance plot) and positive slopes should dominate at higher speeds (the left top part). The similarity between the theoretical and empirical equivalence contours in Figure 5 suggests that human motion sensitivity varies as it does because the visual system distributes its spatiotemporal sensitivity according to a balance of uncertainties. 

From the data on human spatiotemporal sensitivity and from the equilibrium theory of Gepshtein et al. (2007), it follows that some conditions of stimulation are more favorable for motion measurement than others. Suppose that equally favorable conditions for motion measurement correspond to the conditions equally preferred by the visual system in competition between alternative apparent-motion paths, as in Figure 1. In other words, suppose that the conditions of perceptual equilibrium in apparent motion lie on the same equivalence contour: empirical (Figure 5A) or theoretical (Figure 5B). Then, the shapes of equivalence contours predict different regimes of motion perception under different conditions of stimulation: coupling when the slopes of the isosensitivity contours are positive, and tradeoff when the slopes are negative, as we illustrate in both panels of Figure 5. 

Take the pair of points labeled “coupling” in Figure 5A or 5B. The two points belong to the same equivalence contour and therefore mark two conditions equally sensitive to motion (in Panel A) or equally suitable (equally suboptimal) for speed measurement (in Panel B). Let us vary the spatial coordinate of the stimulus represented by the right point in the pair, as we did in the thought experiment in Figure 1B. If we increase the spatial coordinate (arrowhead up), the expected system's sensitivity should decrease, manifested by cooler colors of contours that indicate less favorable conditions for motion measurement. Thus, the stimulus should be perceived less often than the stimulus on the left side of the pair. But if we decrease the spatial coordinate (arrowhead down), then the system's sensitivity should increase (warmer colors of contours), and this stimulus should be perceived more often than the stimulus on the left side. Therefore, in this region of the distance plot, we expect to find space–time coupling. 

By applying this reasoning to other regions of the distance plot, we expect to find space–time tradeoff in some regions, where the slopes of equivalence contours are negative. By this argument, the regime of tradeoff should smoothly change to the regime of coupling as a function of stimulus parameters, for example, by increasing the spatial distance of stimuli from the bottom left region labeled “tradeoff” to the top left region labeled “coupling” in the distance plot in Figure 5. This manipulation also leads to change in speed of motion; thus, we expect the smooth transition between regimes also as a function of speed. Specifically, as speed grows from the bottom right corner of the distance plot to its top left corner, we expect tradeoff to be found at low speeds and coupling to be found at high speeds. 

Note that by this argument, the regime of time independence (not shown in Figure 5) should be found only accidentally: It is obtained only when the line connecting equivalent conditions in the distance plot happens to be parallel to the abscissa. 

To summarize, predictions of the equilibrium theory and the data on human spatiotemporal sensitivity suggest that both regimes could hold: coupling (Korte's law) at high speeds and tradeoff at low speeds, and the controversy would be resolved. These considerations motivate the following two experiments on apparent motion, in which we ask whether tradeoff and coupling are obtained under the different conditions of stimulation. 

Methods

Experiment 1

Stimuli

To create competing paths of motion with independently manipulable spatial and temporal parameters, we use alternating dot patterns called motion lattices (Gepshtein & Kubovy, 2000; Kubovy & Gepshtein, 2003), which are a generalization of the stimuli used by Burt and Sperling (1981). To create a motion lattice (Figure 6), we take a lattice of spatial locations, whose columns are called baselines, and split it into six frames f_i, i ∈ {1,…, 6}, so that each frame contains every sixth baseline. In Figure 6A, dot locations in six successive frames are distinguished by six levels of gray, as explained in Figure 6B (screen shots of the stimuli are shown in Figure 7 and the animated demonstrations in the 1). 

Figure 6

 

The design of a six-stroke motion lattice M6. (A) The six successive frames of M6 are shown superimposed in space. Gray levels indicate time. (B) The time course of M6. To differentiate dots from different frames, in this illustration (but not in the actual stimulus), they are shown in different shades of gray. The three most likely motions—along m₁, m₂, and m₃—can occur because dots in frame f_i can match dots in either frame f_i+1 or frame f_i+2. (C–D) Conditions in which different motion paths dominate: m₁ in Panel C and m₃ in Panel D. (In all our experiments, we chose conditions in which m₂ would never dominate.)

Figure 7

 

Single frames of motion lattices (not to scale). Only the filled dots appear in the actual frames; the open dots are shown in the figure to indicate the locations of dots in other frames. The frames in Panels A and B correspond to conditions in Panels C and D in Figure 6, respectively. Four animated demonstrations of motion lattices are available in the 1.

Observers viewed the frames in rapid succession ( f ₁, f ₂,…), separated by time τ, the temporal scale of each motion lattice. (All other temporal parameters of the stimulus are integer multiples of τ.) When spatial and temporal distances between the frames are chosen appropriately, the motion lattice is perceived as a continuous flow of motion. (Six-stroke lattices are notated

${\mathcal{M}}^{\mathcal{6}}$

, to distinguish them from two-stroke ones,

${\mathcal{M}}^{\mathcal{2}}$

, used by Gepshtein & Kubovy, 2000.) 

A dot that appears in frame f _i can match a dot that appears in frame f _i+1 (after a time interval τ) or in frame f _i+2 (after a time interval 2 τ). As a result, motions parallel to three paths can be perceived: m ₁, m ₂, and m ₃. We chose conditions such that m ₂ would never dominate, so that observers saw motion only along m ₁ or m ₃ (red arrows in Figure 6A). 

Other motions can be seen in motion lattices. For example, when temporal scales are short, motions along paths with t > 2 τ become perceptible, in which case the matching process can skip more than one frame (Burt & Sperling, 1981). In this study, we used temporal scales for which neither motions along paths with t > 2τ nor zigzag motions were perceived. We imposed another constraint on the design of our stimuli: We chose spatial parameters to prevent observers from seeing the motion of spatial groupings of dots (Gepshtein & Kubovy, 2000). We did this by making the baseline distance b (Figure 6A) much longer than the spatial distances of m₁, m₂, and m₃. 

We denote by S _k and T _k, respectively, the spatial and temporal distances of m _k. The temporal components of m ₁ and m ₂ are equal ( T ₁ = T ₂ = τ); the temporal component of m ₃ is twice as long ( T ₃ = 2 T ₁ = 2 τ). Although the ratio of the temporal components of m ₁ and m ₃ is fixed at 2, we can vary the relative magnitudes of their spatial components, as we did in Figure 2B. When m ₃ is much longer than m ₁ both in space and in time ( S ₃ ≫ S ₁ and T ₃ = 2 T ₁; Figure 6C), μ ₁ is seen more often than μ ₃. (Note that S ₂ ≫ S ₁; thus, μ ₁ is also more likely than μ ₂.) But when S ₃ ≪ S ₁ ( Figure 6D), μ ₃ is often seen, even though T ₃ = 2 T ₁. (Here, S ₂ ≫ S ₁ ≫ S ₃; thus, μ ₃ is also more likely than μ ₂.) For μ ₃ to be seen, the visual system must have matched elements separated by interval 2 τ even though other elements appeared in the display during this interval at time τ. 

We defined all spatial parameters of the motion lattice ( S ₂, S ₃, b) relative to S ₁ its spatial scale. The radii of the dots were 0.3 S ₁. To minimize edge effects, we modulated the luminance of dots according to a Gaussian distribution, with the maximal luminance of 88 cd/m ². The spatial constant of the Gaussian luminance envelope of the lattices was σ = 1.5 S ₁. 

In Figures 6 and 7, we show two extreme configurations of the motion lattice: In Figure 6C, S ₃ ≫ S ₁ and m ₁ prevails (see also Figure 7A). In Figure 6D, S ₃ ≪ S ₁ and m ₃ prevails (see also Figure 7B). As we vary the spatial ratio r ₃₁ = S ₃/ S ₁ between these extremes, we find the equilibrium point: a ratio r ₃₁* = S ₃*/ S ₁, at which μ ₃ is as likely as μ ₁. 

Procedure

We presented the stimuli on a computer monitor (1,280 × 1,024 pixels, refresh rate = 75 Hz) in a dark room. All stimuli were viewed binocularly (dioptically). The observer's head was stabilized using a chin-and-head rest. 

Each trial began with a 498-ms fixation point, followed by 12 frames of a motion lattice at a random orientation, and ended with a response screen consisting of pairs of circles connected by radial lines with orientations parallel to m ₁, m ₂, and m ₃ (Gepshtein & Kubovy, 2000). Observers clicked on one of the circles to indicate the direction of motion they perceived. This triggered a mask (an array of randomly moving dots) and initiated the next trial. 

Five naive observers and one of the authors each contributed 100 trials per condition. We obtained equilibrium points in 25 motion lattices at five spatial scales, S ₁ ∈ {0.38°, 0.65°, 1.10°, 1.90°, 3.00°}, at a viewing distance of 0.39 m. The smallest spatial scale ( S ₁ = 0.38°) was the smallest scale at which observers could reach perceptual equality between the competing motion paths. The temporal scale τ was 40 ms. 

Experiment 2

In this experiment, we measured equilibrium points at four temporal scales, τ ∈ {27, 40, 53, 67} ms, using the same five spatial scales S ₁ as in Experiment 1. Our apparatus did not allow us to present motion lattices at a temporal scale smaller than 27 ms. The upper limit on the temporal scales was perceptual: At the temporal scale of above 67 ms, observers started to experience fluctuations between motion along m ₁ and other motion paths within trials (i.e., they saw a zigzag motion). 

For each of the 20 combinations of spatial and temporal scales, we tested five magnitudes of r ₃₁ (as in Experiment 1) to obtain 100 lattices. Nine naive observers and one of the authors each contributed 24 trials per condition. Otherwise, this experiment was identical to Experiment 1. 

Results

Experiment 1

For each spatial scale S ₁, we tested five magnitudes of r ₃₁. In Figure 8A, we plot the log-odds of the probabilities of μ ₃ and μ ₁,  

$L=\log \frac{p\left({\mu }_3\right)}{p\left({\mu }_1\right)}$

(1)

at one spatial scale (3°) for observer C.C. We found the equilibrium points r ₃₁* as explained in Figure 8A. Perceptual equilibrium holds when the competing percepts μ ₃ and μ ₁ are equiprobable, that is, when the log-odds of their probabilities is zero. We found r ₃₁* by a linear interpolation (the oblique solid line in Figure 8A) between the data points that straddle L = 0 (the filled circles). In Figure 8A, the equilibrium point is indicated by the vertical red line. Here, r ₃₁* > 0, indicating the regime of tradeoff, in support of Korte's law. 

Figure 8

 

Results of Experiment 1. (A) Computation of a single equilibrium point r ₃₁*. Perceptual equilibrium holds when the competing percepts μ ₃ and μ ₁ are equiprobable, that is, when the log-odds of corresponding probabilities (the ordinate L) is zero. To find r ₃₁*, we performed a linear interpolation (oblique solid line) between the data points that straddle L = 0 (filled circles). The equilibrium point is marked by the vertical red line. (B) Equilibrium points plotted as a function of spatial scale for six observers. We observe equilibrium points in the tradeoff region ( r ₃₁* < 1) and the coupling region ( r ₃₁* > 1). The vertical bars (visible only where they are larger than the data symbols) correspond to ±1 SE. The plots for observers S.G., T.K., and C.C. contain only four equilibrium points because at the smallest spatial scale, they always saw μ ₁ more often than μ ₃ (i.e., L < 0).

The results for all spatial scales are shown in Figure 8B, a plot of the equilibrium points as a function of spatial scale in six observers. For each, we found equilibrium points that imply both tradeoff ( r ₃₁* < 1) and coupling ( r ₃₁* > 1). 

The right-hand y-axis in each panel of Figure 8B shows the speed ratios:  

$\frac{v_3}{v_1}=\frac{S_3}{2T_1}\frac{T_1}{S_1}=\frac{r_{31}}{2}.$

(2)

Because the ratios are always less than 1, the speed of motion in m ₁ is always greater than in m ₃ when m ₁ and m ₃ are in equilibrium. Despite this, sometimes m ₁ is seen and sometimes m ₃ is seen. This finding is inconsistent with the low-speed assumption (see Discussion). 

We noted above that both tradeoff and coupling regimes of motion perception should be obtained in perception of apparent motion if predictions of the equilibrium theory hold and if data on human spatiotemporal sensitivity can predict perception of suprathreshold apparent motion. Now, we found in each observer that, indeed, both tradeoff and coupling regimes hold. Tradeoff holds at small spatial scales and coupling holds at larger scales. 

We also noted that, from the predictions of the equilibrium theory and from data on human sensitivity, we expect tradeoff to hold at low speeds and coupling to hold at high speeds. In Experiment 2, we ask how the regime of motion perception depends on the speed of motion. 

Experiment 2

In this experiment, we varied the temporal scale of our displays (which we held constant in Experiment 1). 

Figure 9A is a plot of the 20 equilibrium points, averaged across observers, as a function of spatial scale, S ₁. The equilibrium points obtained under different temporal scales follow different functions. However, if we plot the equilibrium points as a function of speed, S ₁/ T ₁ ( Figure 9B), they fall on a single function. Its value varies from tradeoff to coupling, passing through time independence at about 12°/s. Thus, speed (rather spatial scale) determines the regime of motion (tradeoff or coupling). 

Figure 9

 

Results of Experiment 2. (A) Equilibrium points r ₃₁* (averaged across observers) plotted against spatial scale. The four curves were derived from the fitted linear models in Figure 10A. (B) When equilibrium points are plotted against speed, they follow one function. The solid curve is a fit by the statistical model in Figure 10B. The model accounts for 98% of the variability in the data. The dashed curve corresponds to the dashed line in Figure 10B.

The function relating equilibrium points to speed is nonlinear. However, if we plot our data against slowness (i.e., reciprocal speed, T ₁/ S ₁; Johnston, McOwan, & Benton, 1999), we obtain linear functions (Figures 10A and 10B). 

Figure 10

 

Results of Experiment 2. (A) When equilibrium points are plotted against reciprocal spatial scale, they follow linear functions. (B) When equilibrium points are plotted against reciprocal speed (slowness), the data fall on a single linear function. The dashed line is a fit to all the data; the solid line excludes two outliers (the two leftmost black dots) at the two largest spatial scales for τ = 27 ms.

As in Experiment 1, the results are inconsistent with the low-speed assumption. We found that sometimes m ₁ is seen and sometimes m ₃ is seen at equilibrium, despite the fact that the speed of motion in m ₁ is always greater than in m ₃, as indicated on the right ordinates in Figures 9 and 10. 

From the isosensitivity contours in Figure 5, we expected that tradeoff is obtained at low speeds and coupling is obtained at high speeds. The results of Experiment 2 confirmed this prediction. We found a gradual transition between the regimes of tradeoff and coupling. The transition could follow a variety of functions. The fact that the observed function is linear on the slowness scale is remarkable. 

Discussion

Summary

We have reconciled allegedly inconsistent data on apparent motion: Korte's law and later results. In agreement with prediction of the equilibrium theory and data on continuous motion, our results indicate that previous findings on apparent motion were special cases. The allegedly inconsistent results are embraced by a simple law in which a smooth transition from tradeoff to coupling occurs as a function of speed: Tradeoff holds at low speeds of motion (below 12°/s on average), whereas coupling (Korte's law) holds at high speeds. 

Equivalence contours of apparent motion

Because the speed (or slowness) of motion determines the regime of motion perception, and because our data are linear on the slowness scale, we can summarize our results in terms of the speeds of the competing motions:  

$\frac{S_3}{S_1}=k\frac{T_1}{S_1}+l,$

(3)

where k is the (negative) slope and l is the intercept. By multiplying both sides by speed v ₁ (i.e., by S ₁/ T ₁), and noting that v ₃ = S ₃/2 T ₁, we have  

$v_3=\frac{l}{2}{v}_1+\frac{k}{2}.$

(4)

If we substitute the fitted slope and intercept, we obtain a very simple summary of our data:  

$v_3=0.58v_1-1.$

(5)

To plot this result in the format of the distance plot ( Figure 2), we rewrite Equation 5 as  

$S_3=1.16S_1-2T_1,$

(6)

which leads us to a functional equation:  

$f(2T)=1.16f\left(T\right)-2T.$

(7)

 

Figure 11 shows our data (in red) superimposed on a family of numerical solutions of Equation 7 (thin black lines on the background). To obtain each solution, we chose an arbitrary value of f( T) at a very small value of T on the left edge of the figure. Using these coordinates—[ T, f( T)]—we obtained f(2 T) from Equation 7. We used the coordinates [2 T, f(2 T)] to obtain f(4 T) and thus iteratively propagated each solution to the maximal temporal distance in the figure. We repeated this procedure several times, starting at the same value of T but with a linearly incremented value of f( T), to obtain all the solutions plotted in the figure. 

Figure 11

 

Results of Experiment 2 and the empirical equivalence sets in the space–time plot. The thin lines on the background are the empirical equivalence sets we reconstructed from the results of Experiment 2 using the linear model in Figure 10. The pairs of red connected circles represent the measured equilibrium points; each pair corresponds to a data point in Figure 10. The slopes of both the empirical equivalence sets and the lines connecting the circles gradually change across the plot, indicating a gradual change from tradeoff to coupling, as do the slopes of isosensitivity contours measured for continuous motion at the threshold of visibility ( Figure 5A). The oblique thick line indicates the boundary speed of 12°/s.

The equivalence contours of apparent motion ( Figure 11) are similar to the isosensitivity contours derived from measurements at the threshold of visibility ( Figure 5A): In both cases, contour slopes change gradually from tradeoff to coupling as a function of speed. This result implies that there exists a monotonic relationship between human isosensitivity contours ( Figure 5) and the equivalence conditions for apparent motion. 

We do not wish to suggest that the similarity between the equivalence conditions for apparent motion and human isosensitivity contours constitutes a reduction of the mechanisms of apparent motion to the mechanisms responsible for the detection of motion at the threshold. Rather, the similarity implies that the perception of motion is governed by the same constraints at the threshold of visibility and above the threshold. 

Why different regimes?

As we mentioned earlier, the equilibrium theory of motion perception (Gepshtein et al., 2007) predicts that space–time coupling and tradeoff should hold at different speeds. The theory prescribes how a visual system should distribute resources to optimize the estimation of motion speed. The theory predicts uniformly suboptimal sets for speed estimation, which are shown in Figure 5B as the colored contours. The theory prescribes that equal amount of resources should be allocated to the uniformly suboptimal conditions, yielding normative predictions for the equivalence sets. The slopes of theoretical equivalence sets (Figure 5B) change smoothly, taking negative and positive values, which correspond to the tradeoff and coupling regimes of apparent motion, as it is the case in the equivalence contours of apparent motion (Figure 11). This similarity in the shape of the equivalence contours of apparent motion and the shape of theoretical equivalence sets suggests that the different regimes of motion perception emerge as an outcome of an optimization process that balances the measurement and stimulus uncertainties associated with speed estimation. 

Note that one might also predict the different regimes of apparent-motion perception at different speeds from the empirical isosensitivity contours ( Figure 5A). To do so, one had to assume that equal motion sensitivities correspond to equal motion strengths. The equilibrium theory obviates that assumption because the theory tells which conditions are equivalent for motion measurement, regardless of whether it involves stimuli at the threshold or above the threshold. 

Shapes of equivalence contours

In their quantitative details, the shapes of the empirical equivalence contours of apparent motion are different from the shapes of the empirical contours we derived from the isosensitivity data (Kelly, 1979), as one can see by comparing Figures 5 and 11. The empirical contours of apparent motion are shallower in the region where we could measure the equilibria of apparent motion. There are two reasons to expect such a discrepancy:

1.  
   Kelly (1979) obtained the estimates of spatiotemporal sensitivity (Figure 5A) with narrowband stimuli (drifting sinusoidal gratings), whereas our stimuli are spatially broadband. Also, he used image stabilization to limit motion on the retina, whereas our observers were free to move their eyes during stimulus presentation; as a result, our stimuli cover a broader temporal-frequency band. Such increases in the width of the spatial and temporal frequency bands flatten the equivalence contours. We illustrate this in Figure 12, which shows the results of simulating the effects of widening of the frequency bands. We obtained the three panels by averaging Kelly's estimates of sensitivity across an increasing range of spatial and temporal frequencies and plotted the resulting equivalence contours in the distance plot.
2.  
   As Gepshtein et al. (2007) showed, a normative theory predicts that estimates of the sensitivity of the visual system depend on the task and the stimuli used in obtaining the estimates. Consider a task that uses an ambiguous stimulus for which motion matching is difficult. Such a task depends more on estimating stimulus frequency content than stimulus location. (See also Banks, Gepshtein, & Landy, 2004, who emphasized the role of stimulus spatial-frequency content for solving the binocular matching problem.) According to Gepshtein et al., an optimal visual system should change the distribution of its sensitivity across the parameters of stimulation so that its estimate of stimulus frequency content becomes more reliable than its estimate of stimulus location. On this view, there would be little reason to expect a quantitative agreement between the equivalence contours obtained using different tasks and different stimuli.

Figure 12

 

An illustration of how the size of spatiotemporal frequency band of visual stimulus affects the shapes of equivalence contours. We simulated the effect of frequency band by averaging empirical estimates of spatiotemporal sensitivity ( Figure 5A; Kelly, 1979) across a range of spatial and temporal frequencies; this range grows across the panels, from left to right. The equivalence contours grow shallower as the averaging range increases.

In order to determine how the shape of equivalence contours depends on measurement conditions, future research should estimate contour shapes within observers who perform a variety of tasks (e.g., stimulus detection and motion-direction discrimination using unambiguous stimuli, such as drifting gratings, and ambiguous stimuli, such as random-dot kinematograms and motion lattices). 

The low-speed assumption

Our results contradict the low-speed assumption as it was formulated by Wallach (1935, 1976). The assumption was motivated by observations that shorter motion paths are preferred to longer ones in apparent motion and by the aperture problem, as we mentioned the Regimes of apparent motion section. On this view, a low speed should always prevail in competition with a faster speed. This is not the case in our results. Under perceptual equilibrium, the competing motion paths were seen equally often, but the ratio of speeds in the two paths was always less than unity (Figures 8B, 9, and 10). That is, motion was not always seen along the slower path. 

Although our results are inconsistent with Wallach's formulation of low-speed assumption, they are consistent with the evidence that low speeds prevail in the perceptual ecology. The equilibrium theory of motion perception (Gepshtein et al., 2007) takes into account the distribution of speeds in the perceptual ecology (Dong & Atick, 1995), which implies that low speeds prevail in the natural stimulation. However, the theory does not predict that competition between alternative apparent-motion paths must be resolved in favor of slower motion. It predicts that the outcome of competition depends on the degree of balance between measurement uncertainties and stimulus uncertainties. The degree of balance is reflected by the slope of the equivalence contours in the distance plot (Figure 5B). It is the sign of the slope that determines the outcome of competition. Thus, the equilibrium theory is consistent both with the fact that low speeds prevail in perceptual ecology and with our data showing that one motion path can dominate another, independent of which motion is slower. 

Conclusions

Korte's counterintuitive law does hold under some conditions, but its claim to being a general law of motion perception is incorrect. The apparently inconsistent results on apparent motion using suprathreshold stimuli are special cases of a lawful pattern consistent with predictions of a normative theory of motion perception and data on continuous motion at the threshold of visibility. 

Supplementary Materials

Demonstrations of Motion Lattices (M⁶) 

Sergei Gepshtein 

Brain Science Institute, RIKEN, Japan 

Michael Kubovy 

Department of Psychology, University of Virginia  

Motion across short temporal separation, i.e., in the direction of vector m1:     MOVIE 1: All lattice locations visible.   MOVIE 2: Lattice Locations hidden.

Motion across long temporal separation, i.e., in the direction of vector m3:     MOVIE 3: All lattice locations visible.   MOVIE 4: Lattice Locations hidden.

 

The design of motion lattices is explained in the accompanying manuscript The lawful perception of apparent motion.  

Appendix A

Demonstrations of motion lattices

Four demonstrations of motion lattices

${\mathcal{M}}^{\mathcal{6}}$

are available online at: Supplementary Materials. 

In Animation Movies 1 and 3, all lattice locations are made visible for illustration, as in Figure 7. Only the filled dots appeared in the actual stimuli, as in Animation Movies 2 and 4. 

Acknowledgments

We thank A. van Doorn, M. Landy, and C. van Leeuwen for their comments on an earlier version of this article; M. Rudd and I. Tyukin for stimulating discussions; H. Hecht for help in translation from the German; and E. Doherty for assistance in running the experiments. This work was supported by NEI Grant 9 R01 EY12926. 

Commercial relationships: none. 

Corresponding author: Sergei Gepshtein. 

Email: [e-mail]. 

Address: Perceptual Dynamics Laboratory, Brain Science Institute, RIKEN, Wakoshi, Saitama, Japan.