The influence of spatial orientation on the perceived path of visual saltatory motion

Abstract

Visual saltation is the illusory mislocalization that occurs when multiple elements are rapidly presented to two peripheral locations; mislocalized elements appear to fill in the intermediate space. We investigated the influence of element orientation on the path of illusory saltatory motion. Experiment 1 showed that congruence in element orientation at the two locations (horizontal–horizontal or vertical–vertical) produced rectilinear saltation, while incongruent orientations (vertical–horizontal or horizontal–vertical) elicited curvilinear saltation consistent with rigid rotation around a common point. In curvilinear saltation, mislocalized elements were perceived with an intermediate orientation. Experiment 2 showed that the perceived shape of the motion path was directly dependent on the salience of orientation information. In Experiment 3, we showed that the circular path of curvilinear saltation (induced by orientation incongruence) is altered by background motion (wedge-shaped regions of inward and outward moving dots) that overlaps only with the inter-element space. An ellipsoid path, where the major axis corresponds to the mislocalized element overlapping with outward motion and the minor axis corresponds to the mislocalized element overlapping with inward motion, is produced. These findings reveal that the interpretation of visual saltation arises from high-level computations in which the percept is derived through an interaction of form and motion.

Introduction

The visual percept of motion can arise from the perceptual grouping of stationary stimuli briefly presented in sequence at different locations separated over a large spatial distance. Long-range “Stroboscopic” or “Apparent” Motion (AM; see Braddick, 1974, 1980; Larsen, Farrell, & Bundesen, 1983) typifies this phenomenon: when two spatially separated visual stimuli are viewed in rapidly alternating succession, illusory movement resembling a single stimulus smoothly traversing the shortest path between two locations is produced (e.g., Shepard, 1984; Shepard & Zare, 1983). AM arises through perceptual grouping and filling-in such that visual motion is inferred from the spatiotemporal characteristics of inducing elements (Cavanagh & Mather, 1989; Khuu, Phu, & Khambiye, 2010; Kolers, 1963, 1972; Wertheimer, 1912), not from retinal stimulation. AM likely reflects “top-down” processing in higher cortical areas where an internal representation of motion is initially generated, with activation then fed back to the lower cortical areas that directly code the retinopy. In this manner, the visual system is able to derive a motion percept using an alternative strategy to motion energy analysis (e.g., Adelson & Bergen, 1985; Anstis, 1980; Braddick, 1980; Smith, 1994). Neural imaging studies have confirmed a functional arrangement in which the AM percept is initially generated in motion-processing areas, such as area MT (e.g., Liu, Slotnick, & Yantis, 2004; Muckli, Kohler, Kriegeskorte, & Singer, 2005; Pascual-Leone & Walsh, 2001), and then fed back to lower cortical areas, such as V1, where neural receptors mapping the area between the first and second element locations are activated to generate the percept of smooth motion. The operation of feedback projections from MT to V1 in the coding of high-level motion is well supported by single-cell recording studies (e.g., Sillito, Cudeiro, & Jones, 2006). 

Because the perception of high-level motion reflects an interpretative neural process, it is highly susceptible to influence from other visual attributes that may additionally define the elements inducing AM (see, e.g., He & Nakayama, 1994a, 1994b). For example, where inducing elements are different colors, the object appears to change color as it moves perceptually from one location to the next (Kolers & Von Grunau, 1976). Analogously, if elements differ in shape, the AM is accompanied by a systematic morphing from one element type to the other. This “transformational apparent motion” is thought to reflect an integrative interpretation of form and motion constrained by geometrical properties (reflecting Gestalt principles) such as continuity and smoothness (Tse, Cavanagh, & Nakayama, 1998). It is postulated that these perceptual changes arise because the visual system, in constructing an internal representation of motion, must also consider the possibility that other featural changes may accompany this movement. Consequently, if it were assumed that an element has undergone motion, any potential differences in the form of elements at two spatial points constrain the percept and the generated solution resembles a single object undergoing simultaneous transformational change in motion and form (Tse et al., 1998). While transformations in form have been noted in the perception of motion, this process is not well understood; the nature of the interaction between form and high-level motion arising from “top-down” processing and the conditions under which interaction arises, in particular, is unclear. The goal of this study is to contribute to the understanding of this process specifically by examining the nature of the mutual influence of spatial orientation and high-level motion. 

It has been reported that, in representing AM, the visual system considers the influence of local orientation (e.g., Werkoven, Snippe, & Koenderink, 1990), which (in addition to a transformational shape change as noted above) may skew the path of AM. For example, Foster (1975, 1978) noted that if line elements inducing AM differed in spatial orientation and location, the percept resembled a moving line changing orientation. Crucially, rather than follow the shortest path between the two stimulus locations, the percept traverses a curved path. McBeath and Shepard (1989) suggested that illusory curved motion paths are consistent with the visual system following “real-world” geometrical principles. They (see also Foster, 1978) propose Euler's theorem as one possible principle. This theorem suggests that, for an object imaged in two-dimensional (2D) space, there exists a unique center point around which that object can rigidly rotate (maintaining a constant distance from the center point) back and forth following a curvilinear path (see Figure 2D). Thus, in the perception of AM, it is implied that successive presentations of incongruent orientations/shapes at the two locations provide cues to this rotational point. These cues then guide AM along a curvilinear trajectory. 

Although it is clear from previous studies that shape/orientation incongruence between inducing elements can lead to a curved AM path, methodological issues inherent to the tasks and stimuli employed prevent clear understanding of the mechanisms. For example, previous studies have required observers to make qualitative judgments such as verbally reporting whether they saw a curved path or to judge the path of motion relative to a probe flashed between the two inducing elements. These techniques may not be the most appropriate, as verbal reports do not allow for careful and precise measurement of the extent of curvature. Additionally, where judgments of path curvature are made with reference to a probe flashed between the two locations, the mere presence of an object in that position may itself cause AM to move on a curved path. Berbaum and Lenel (1983), Deatheridge and Bitterman (1952), and Kolers (1972) noted that the AM path appears “deflected” by, and curved around, a stationary object presented between identical inducing elements. McBeath and Shepard (1989) quantified the curvature of an AM sequence induced by asymmetric shapes by requiring observers to judge whether the sequence moved through a “window”—a gap between two vertical lines—that was placed between the two inducing elements. McBeath and Shepard reported that observers placed the window at a position consistent with an AM trajectory following a curved path. Despite the promise of this technique, it cannot be determined whether this effect is a result of a shape/orientation difference between elements or whether the window probe itself caused the observed deflection in motion. Their observation that the extent of curvature did not follow a perfectly circular path, as would be expected if an object was perceived to actually undergo geometric rotation, suggests the latter explanation. 

Fundamentally, examination of the interaction between high-level motion and spatial orientation is limited by the characteristics of the AM phenomenon itself. An object undergoing rotation would be expected to undergo smooth transformation in form as the moving stimulus morphs from one shape to the other. However, because AM resembles smooth motion between the two locations, elements are not individuated, limiting the stimulus to simple judgments about the quality (e.g., smoothness) of the motion percept. Given this, the precise role of element orientation in the perceived path of high-level motion remains difficult to assess. What is required is an alternative high-level motion stimulus that overcomes the methodological issues inherent to the quantification of AM and allows for individuation of elements along the motion path, the orientation and location of which can be objectively quantified. Here, we employ “visual saltation” as an alternative to AM that fulfills these criteria. 

Visual saltation is a compelling motion illusion characterized by substantial and systematic mislocalization of a number of elements along a path between two locations. It is elicited when a number of elements are briefly presented in the periphery first to one location, and then to another, in rapid and regular succession (see Figure 1A and the movie clip associated with this figure—looping this movie sequence elicits compelling saltation when viewed in peripheral vision). While the first and last elements of the sequence are perceived at their physical positions (anchoring the sequence), intermediate elements are sequentially mislocalized to intermediate positions between the two locations, resulting in the appearance of an object saltating across the space between the two locations. The conditions optimizing the saltation percept have been discussed extensively in previous publications (e.g., Khuu, Kidd, & Errington, 2010; Khuu, Kidd, Phu, & Khambiye, 2010), but it is important for the present purposes to note that saltation is dependent on the inter-stimulus interval (ISI) and is most compelling at short intervals, with elements perceived veridically at sufficiently long ISIs. Visual saltation cannot be accounted for by low-level spatiotemporal motion mechanisms that compute motion energy (see Khuu et al., 2010; Moradi & Shimojo, 2004) and most likely reflects high-level perceptual grouping and filling-in (see Khuu et al., 2010 and Khuu, Kidd, Phu et al., 2010 for detailed discussion). This process is likely one in which elements made ambiguous through their brief presentation in the periphery are grouped and interpreted as a single object jumping across the non-stimulated space. Khuu, Kidd, Phu et al. (2010) suggested attentive tracking as a mechanism for this illusion: As attention is shifted from one location to the next, elements are associated together leading to the interpretation of a single object jumping from one location to the next. Visual saltation may, therefore, share a common mechanism with previously documented visual phenomena that report spatial mislocalization associated with the perception of high-level motion (e.g., Brigner, 1984; Shim & Cavanagh, 2004). The saltation and AM percepts are phenomenologically different though: unlike AM, saltation is only apparent in the periphery and is produced by briefly presenting at least three elements. In addition, as noted by Moradi and Shimojo (2004), visual saltation and AM are easily distinguishable regardless of ISI used to generate the percepts: AM resembles smooth continuous motion, while visual saltation resembles an object jumping in equidistant steps between the two points. Visual saltation generates a different motion percept to AM: The rapid onset and offset of presented elements produces flicker that acts as a marker to temporally partition the motion sequence. This cue allows individuation of elements in the saltation sequence, which is not possible with AM. 

Figure 1

 

Schematics of the illusory saltation sequence and hypothetical percepts. (A) The physical placement of elements in the sequence as a function of time (irrespective of orientation). The first and second elements are presented at the same location, while the third element is displaced to a different location. The ISI is kept constant between all elements. (B) Illusory saltation induced by peripheral viewing of elements at short ISIs. (C, D) The first and third elements differ in orientation. (C) If the orientation difference does not affect the motion trajectory, the path of illusory motion will be rectilinear. (D) If the orientation difference does affect the motion trajectory, the path will be curvilinear, consistent with rotation (σ) around a common point.

As mentioned, the advantage of saltation over traditionally used AM stimuli is that elements are spatially mislocalized in sequence with their position and form clearly individuated. The perceived orientation and position of individual elements can be easily and effectively quantified, giving an accurate measure of the perceived motion path. With this in mind, the present study explored the hypothesis that the orientation of inducing elements would bias the perceived path of motion in saltation. Figure 1A depicts the functional arrangement of the stimulus used. Three oriented elements were presented in the periphery, two at the first location and one at the second (note that in Figure 1A, the X position of elements is plotted as a function of time). In Figure 1B, the presentation of elements depicted in Figure 1A produces saltation such that the second element is mislocalized to an intermediate position, diagonally separated (in X and Y Cartesian space) with the first element at bottom left and the last element at top right. The identical orientations of the three elements entail that visual saltation follows a straight (rectilinear) path between the two locations (as shown by Khuu et al., 2010). In Figures 1C and 1D, elements differ in orientation, with horizontal oriented elements presented to the first location and a vertical element presented to the second. We raise two predictions. First, as shown in Figure 1C, if the orientation of elements does not influence the interpretation of motion, the path of saltation will be rectilinear as in Figure 1B, with elements perceived at their actual orientation. Second, in Figure 1D, an orientation difference between the two locations may be assumed to be the endpoints of a single object undergoing rigid rotation around a common point. In this case, the perceived path of motion will be curvilinear with the second element mislocalized along the curved path. Though, the extent of curvature will be dependent on the distance and relative orientation of elements. Accompanying this motion ought to be a perceptual change in the perceived orientation of elements, with the orientation change parallel to radii originating from the point of rotation. A goal of the present study was to determine, through measurement of the perceived position of elements, which of these outcomes characterizes the influence of spatial orientation on the perceived path of visual saltation. 

We report three experiments. In the first, we determine whether an orientation difference between elements at the first and second locations (as shown in Figure 1) influences the shape of the saltation path and/or the perceived orientation of mislocalized elements. In the second, we quantify the effect of orientation salience on the motion path by systematically manipulating element luminance contrast. Finally, we examine whether the perceived curvilinear path of saltation generated by elements of incongruent orientation can be distorted by background motion overlapping the non-stimulated space between the two elements. We reasoned that the neural representation of saltation generated in higher cortical areas would be altered by background motion such that elements perceptually overlapping with the background motion would be shifted in the motion direction (see De Valois & De Valois, 1991; Dickinson, Han, Bell, & Badcock, 2010), leading to a global distortion in the path of motion. 

Experiment 1: The effect of element orientation on the perceived path of illusory saltatory motion

Experiment 1 aimed to examine the impact of spatial orientation on the perceived path of visual saltation. We quantified percepts relating to two characteristics of the stimulus. First, we wished to establish which of the predictions raised in Figures 1C and 1D characterize the path of saltation (i.e., is the path rectilinear or curvilinear?) when the illusion is induced by incongruently oriented elements. Second, we examined the perceived orientation of elements. We reasoned that, if the saltation follows a curvilinear path, a change in orientation might be perceived as the element “jumps” from one location to the next. Specifically, the mislocalized second element may appear at an intermediate orientation despite its physical orientation being the same as the first. As noted previously, ISI is fundamental to eliciting visual saltation. Therefore, we also examined the impact of ISI on the perceived location and orientation of elements. 

Methods

Observers

Six observers (aged 20–33 years) participated. One was an author (SKK), while the others were experienced observers who were naive to the goals of the study. All had normal or corrected-to-normal visual acuity and no history of visual deficits. 

Stimuli

The stimuli were movie sequences showing three “hard-edged” circular elements (of a diameter of 1.25 deg of visual angle), presented briefly (for 50 ms) on a gray background at a luminance of 40 cd/m². Each element had a luminance pedestal of 30 cd/m² above the background luminance and windowed a sinusoidal grating (periodicity: 1.5 c/deg, Michelson contrast: 0.57 of a particular spatial orientation). To generate the percept of saltation, two elements were presented to the first location, while one element was presented to the second. Each element of the saltation sequence was interleaved with a period in which a blank background was seen. The duration of this period corresponded to the ISI and was changed depending on the condition (see Procedures section). As depicted in Figure 1D, these elements were located in a 5° × 5° square with elements diagonally separated such that the first location at which elements were presented was at the bottom-left corner, while the second location was positioned in the top-right corner of the stimulus area. The entire stimulus was placed at a retinal eccentricity of 10° to the left of central fixation (which was indicated by a black spot on the screen). To aid the judgment of the position of elements, two vertical black (0.1° × 6°) lines were presented to act as landmarks indicating the left and right edges of the stimulus area (note that these lines did not overlap with the motion path). These lines were continuously displayed throughout the experiment. Observers viewed this stimulus in a dark room at a viewing distance of 60 cm. Stimuli were generated using MATLAB version 7 and displayed on a linearized 24-inch Mitsubishi Diamond Pro monitor driven at a frame rate of 100 Hz. 

Procedures

Stimulus sequences were presented to the observers twice in quick succession (with an inter-sequence interval fixed at 0.25 s). After the second sequence, the stimulus disappeared from the screen and observers used a computer-mouse-controlled probe (a circular Gabor patch: spatial frequency 2 c/deg, standard deviation: 0.5°, Michelson contrast: 0.5) to indicate the perceived position and orientation of either the first, second, or last element of the sequence, all the while under instruction to maintain fixation. Observers did not judge the position of all elements in one trial but only one element; the to-be-judged element was randomized between trials. The luminance pedestal of the to-be-judged element was doubled (making it appear “brighter” than the others in the sequence) to aid these judgments. 

On each trial, observers were required to first indicate the two-dimensional (2D) position of the cued element (the “brighter” element) by placing the probe over the perceived 2D position and then judge its spatial orientation by adjusting the orientation of the Gabor probe (by pressing two buttons on a keyboard, which tilted the Gabor probe either to the left or to the right in steps of 1°). The probe was not visible on the screen during the stimulus presentation rather it appeared at a random location, and at a random orientation, on the screen at the offset of the second sequence and disappeared immediately after the observer had pressed the mouse button. Observers responded as quickly and accurately as possible. Prior to the experiment proper, observers were given practice trials sufficient to become familiarized with the task, minimizing judgment errors. 

In randomized trials, these procedures were adopted for stimuli in four different element orientation configurations (in which the orientations at the first and second locations were the same—either horizontal or vertical—or the elements at the two locations were different such that element at the first location was vertical and the element at the second location was horizontal, or vice versa). Each orientation configuration was presented at six different ISIs (0.1, 0.2, 0.4, 0.6, 0.8, and 1 s), and judgments were made of the position and orientation of the first, second, and last elements of each sequence. Each permutation was repeated 5 times. Therefore, a block comprised 360 trials. Observers each completed 5 blocks such that each stimulus condition had 25 trials. Results were averaged across the 25 trials for each condition. 

Results

The judged positions and orientations of elements are shown in Figures 2 and 3, respectively. Results for all observers were similar and therefore averaged for each condition. Error bars represent one standard error of the mean and signify the variance between observers. In Figure 2, the perceived Cartesian X and Y positions (relative to the physical position of the first element) of the first (triangles), second (circles), and last (squares) elements of the saltation sequence are plotted for different ISIs (6 different gray levels—see legend). Separate panels represent different orientation configurations between elements presented at the first and last locations as denoted by the schematic illustrations accompanying each graph. Figures 2A and 2B represent conditions in which there was no orientation difference at the two locations, with elements being either both horizontal or both vertical. Figures 2C and 2D represent conditions in which element orientation at the two locations differed (either horizontal–vertical or vertical–horizontal). Dashed lines represent a rectilinear diagonal (black), or a curvilinear (gray), motion path (see below). 

Figure 2

 

The average perceived X and Y positions (relative to the physical position of the first element) of the first (triangles), second (circles), and last (squares) elements of the saltation sequence are plotted for different ISIs (different gray levels) and for orientation configurations in which elements at the first and second locations were the same [(A) horizontal; (B) vertical] or different [(C) horizontal–vertical; (D) vertical–horizontal]. Dashed lines indicate motion paths following either a diagonal rectilinear (black dashed line) or curvilinear (gray dashed line) trajectory. Solid black lines indicate the physical position of the first and second locations. Error bars signify one standard error of the mean.

Figure 3

 

The perceived orientation of the first (triangles), second (circles), and last (squares) elements of the saltation sequence plotted as a function of the ISI used to generate visual saltation. As in Figure 2, different panels reflect different orientation configurations.

Figure 2 reveals a number of interesting findings about judged element position. First, the perceived positions of the first (triangles) and last (squares) elements of the saltation sequence corresponded to their physical locations; regardless of ISI and orientation configuration, judgments for these two elements were clustered around the corresponding solid lines. This indicated that observers were accurate in performing this task despite the fact that stimuli were presented to the periphery. Second, these results contrasted with the judged position of the second element (circles), which varied according to both the ISI and the orientation configuration of elements. With regard to ISI, as shown in Figure 2A, when ISI is brief (e.g., 0.1 s), the second element (physically presented to the first location) is spatially mislocalized to the intermediate location, consistent with the percept of saltation. For longer ISIs (e.g., 1 s), the judged position of the second element corresponded to its physical position, indicating no saltation. These results replicate previous findings (e.g., Khuu et al., 2010) that saltation is dependent on ISI, with compelling saltation observed at ISIs of approximately 0.1 to 0.3 s but not at longer ISIs. With regard to element orientation, where elements presented to the first and second locations had the same orientation, the second element was mislocalized (as a function of ISI) on a rectilinear diagonal path (black dashed line) between the two locations (see Figures 2A and 2B that are consistent with Figure 1C). Where the first and last elements were incongruently oriented, saltation was equally vivid but instead followed a curvilinear trajectory (gray dashed line) conforming to an arc of a circle (of radius of 2.5°, spanning an angular range of 0° to 90°). The curvilinear motion trajectory was dependent on the orientation configuration of elements: A horizontal–vertical configuration produced saltation that traveled up and to the right (Figure 2C), while a vertical–horizontal configuration gave saltation that traveled to the right and up (Figure 2D). We suggest that this is because the orientation configurations imply different rotational points. 

Judgments of element orientation are depicted in Figure 3. The perceived orientations of the first, second, and last elements of the saltation sequence are plotted as a function of ISI. As in Figure 2, Figures 3A and 3B represent data for conditions in which elements had congruent orientations, while Figures 3C and 3D give data for conditions presenting incongruently oriented elements. Note first that, regardless of orientation configuration and ISI, the perceived orientations of the first (triangles) and last elements (squares) are accurately judged. Where the stimulus configuration presented congruently oriented elements, the orientation of the middle element (circles) was judged accurately, regardless of ISI (Figures 3A and 3B). However, where orientation was incongruent, perceived orientation of the second element was dependent on ISI (Figures 3C and 3D). Where saltation was not observed (long ISIs), the perceived orientation of the second element corresponded with its actual orientation (which was the same as the first element—Figure 3C: 90° and Figure 3D: 0°). Where saltation was observed (short ISIs), the second element was judged to be at an angle intermediate to that of the first and last elements. To exemplify, at the shortest ISIs employed (eliciting the most compelling saltation), the perceived orientation of the second element was approximately 40–45°. 

In summary, the results of Experiment 1 reveal an interaction between form information and the interpretation of high-level motion. At appropriately short ISIs, inducing elements of congruent orientation produced saltation that followed a rectilinear path. However, an orientation difference between the inducing elements produced the interpretation of an object undergoing rotation as it moves between the two locations. This was manifest as both a curvilinear motion path and a change in the perceived orientation of the middle element. In relation to Figure 1, where saltation was reported, orientation information plays a critical role in determining the perceived path of saltation with the trajectory of motion following a circular path as indicated by Figure 1D and not a rectilinear one as illustrated in Figure 1C. 

Importantly, where elements differ in orientation, the perceived motion paths that arise are consistent with an orientation change parallel to radii from the center of a circle, not with the alternative possible solution in which the motion path might have been orthogonal to radii. Both solutions might have been possible given that the orientation configurations used in Experiment 1 could imply either a parallel or orthogonal change in orientation about a common point. Note though that, if the latter outcome did apply, the second element would be mislocalized to the inverse position of that actually seen in Figures 2C and 2D, and additionally, the mislocalized element would be perceived as having an orientation orthogonal to that seen in Figures 3C and 3D. Were this alternative result to have materialized, a possible explanation would have presented itself. The solution might have been consistent with the visual system solely relying on the orientation of elements as a cue to motion perhaps in the manner of “speed lines” or motion streaks that are used to imply motion in static pictures (e.g., see Burr & Ross, 2006; Geisler, 1999; Or, Khuu, & Hayes, 2007, 2010; Ross, Badcock, & Hayes, 2000). A “motion streak” argument is consistent with low-level processing, reflecting the activation of local mechanisms that jointly code spatial orientation and motion. This was not observed. Instead, the findings depicted in both Figures 2 and 3 are consistent with an interpretation of an object that is moving in a direction that is orthogonal to the perceived orientation of the element and the transformational change in orientation (as noted in Figure 3) guides the motion path. 

Experiment 2: The effect of orientation salience on the perceived path of visual saltation

The results of Experiment 1 strongly suggest that orientation information is important in the interpretation of saltation and that the path of motion is constrained by element orientation; spatial orientation influences the perceptual filling-in that gives rise to saltation. In this case, one would expect a direct relationship between orientation salience and the shape of the perceived motion path. Specifically, if the visibility of the orientation conveyed by each element were sufficiently reduced, with orientation information no longer able to constrain the motion path and the visual system unable to judge whether elements have undergone rotation, the saltation path ought to be rectilinear. Experiment 2 examines this possibility, systematically measuring the perceived position of elements in saltation as a function of the contrast of the sinusoidal grating of each element (the luminance pedestal remained unchanged at 30 cd/m²). 

Methods

The observers were the same as in Experiment 1. The methods and procedures were also similar, though only two element orientation configurations were presented (elements at the two locations were either the same, in a horizontal–horizontal configuration, or different, in a horizontal–vertical configuration), at only one ISI (0.2 s, sufficient to induce compelling saltation). The task of the observer was to use a mouse control probe to indicate only the perceived position of the first, second, and last elements of the saltation sequence. These judgments were repeated for 6 contrast levels produced by changing the amplitude of the sinusoidal grating: 0, 0.0675, 0.125, 0.25, 0.5, and 1. An amplitude of 0 gave elements that were uniformly illuminated and 30 cd/m² (corresponding to the element pedestal) above the background luminance; these elements had no associated spatial orientation. Given that Michelson contrast changes when a pedestal is added (see Badcock & Sevdalis, 1987), the contrast of elements for the different sinewave amplitudes was: 0, 0.038, 0.071, 0.14, 0.28, and 0.57. Obviously, the orientation of each element becomes more apparent with increasing contrast. A block comprised 360 trials: judgments of the position of each of the 3 elements in the sequence for the 2 orientation configurations at each of 6 contrast levels, repeated 10 times. Stimulus conditions were randomized within and between each block. Observers each completed 5 blocks such that each condition had 50 trials. Results were averaged across the 50 trials for each condition. 

Results

The results of Experiment 2 are shown in Figure 4. Results for all observers were averaged and the perceived Cartesian X and Y positions of the first (triangles), second (circles), and last (squares) elements of the sequence are plotted for the 6 different contrast levels (different gray levels). Figure 4A depicts results for the condition in which elements have congruent orientation (horizontal–horizontal), and Figure 4B gives results for that condition in which the orientations were incongruent (horizontal–vertical). As in Figure 2, dashed lines indicate the possible motion paths, either curvilinear (gray dashed line) or rectilinear (black dashed line). 

Figure 4

 

The perceived X and Y positions of the elements of the saltation sequence are plotted for different grating contrasts (different gray symbols, see legend) in the same format as Figure 2.

Figure 4 points to a number of findings. First, the perceived positions of the first and last elements are unaffected by changing grating contrast; judged X and Y positions for these elements are tightly clustered around their actual positions (solid lines). Second, while compelling saltation was evident at this ISI (with the second element perceived midway between the two locations), the path of motion was dependent on the orientation configuration made available through contrast information. As in Experiment 1, for the congruent orientation condition (Figure 4A), the second element was mislocalized along a rectilinear path. Consistent with this, changing contrast did not affect the perceived path of motion where elements had the same orientation; judged positions are clustered midway between the two physical locations regardless of contrast. Likewise, as in Experiment 1, where saltation was induced by a horizontal–vertical element orientation configuration, the motion followed a curvilinear path. However, the extent of path curvature was dependent on the contrast of the grating. For comparatively high contrasts of 1 and 0.5 (dark symbols), the perceived path of motion closely followed a circular trajectory resembling observations made in the previous experiment (gray dashed line). Where elements had a lower contrast, a systematic return to a rectilinear trajectory was observed such that contrast levels of 0 and 0.0675 (light symbols) elicited a straight-line motion path. Therefore, the results of this experiment confirm the findings of Experiment 1 and additionally show that systematic manipulation of orientation information via contrast is proportional to systematic change in the motion path. Clearly then, the visual system reserves its interpretation of saltation induced by incongruently oriented elements as rotation only for circumstances in which this incongruence in orientation is salient. 

Experiment 3: The effect of background motion on the perceived path of visual saltation

As with AM, it is thought that visual saltation is the product of processing in high-level cortical areas, whereby a neural representation of motion (a single object traversing the space between two locations) is generated in higher cortical areas and then fed back to lower cortical areas (that which codes the non-stimulated space between the two physical locations) to produce the percept (see Khuu et al., 2010 and Khuu, Kidd, Phu et al., 2010 for discussion). Incongruence in the orientation of elements results in the illusory percept of a curvilinear path between two spatial locations. In Experiment 3, we examined whether curvilinear saltation is distorted if the response of mechanisms coding position and motion at lower levels of processing was biased in a particular direction. 

Recently, Li, Khuu, and Hayes (2009) demonstrated that a circular path defined by motion is distorted when superimposed on a background containing dots undergoing radial contracting and expanding motion in alternating sectors. This background produced the percept of an ellipsoid path, with the major axis of the path coinciding with expanding sectors and the minor axis with contracting sectors. These findings were interpreted in line with the observations of De Valois and De Valois (1991) who reported that the perceived position of an object is shifted in the motion direction (though see Dickinson et al., 2010), possibly due to a shift in the receptive field of neurons that code position in the motion direction (see Fu et al., 2002). With the study by Li et al. in mind, we questioned whether the curvilinear saltation could be distorted by background motion confined to sectors discretely overlapping with the non-stimulated space between the locations at which elements are presented. In biasing the response of cells that locally code position in the non-stimulated space, it may be possible to shift the perceived position of the second element in the direction of background motion. Such a local position shift might well lead to a perceived distortion in the global path of visual saltation. However, it is important to note that the second element has its physical origin at the first location, despite being mislocalized to an intermediate position, and therefore would not physically overlap with the background motion. If the origins of visual saltation were low level and feed-forward, rather than high level and feedback as we have suggested, we would expect the visual system to be sensitive to the physical (rather than perceived) location of elements, a situation likely resulting in no path distortion with this type of background motion. The purpose of Experiment 3 was to test these possibilities. In so doing, we aimed to contribute to the understanding of how the visual system constructs a neural representation of high-level motion, particularly the extent to which perceptual change in form is indicative of the response of neurons coding local motion. 

Methods

Observers

Four of the original 6 observers in Experiment 1 participated in Experiment 3. One (SKK) was one of the authors. 

Stimuli

Stimuli were similar to those used in Experiment 1 except that elements were presented at four locations (rather than two) equally separated (5°) and arranged in a “cross” configuration (see Figure 5 for a schematic diagram of the stimulus). Pairs of elements were presented at each of the 4 locations (with an ISI of 0.2 s), in a clockwise order, to elicit saltation. The orientation of elements was parallel to the center of the “cross” such that elements that were vertically separated were vertically oriented, while horizontally separated elements were horizontally oriented. When viewed in the periphery, this saltation stimulus resembles a circular motion path as reported in Experiment 1 (see movie clip 2 associated with Figure 5—when the movie is looped, the path of saltation appears to rotate around the midpoint of the stimulus). 

Figure 5

 

Schematic diagrams of the stimulus used in Experiment 3. (A) The test (left image) and reference (right image) stimuli presented to the observers. Illusory saltation was generated in the test stimulus by presenting two elements at 4 locations (corresponding to the 0°, 90°, 180°, and 270° positions of a circle), while the test stimulus underwent non-illusory saltation along a circular trajectory (i.e., elements were actually presented at intermediate positions and orientations). Observers matched the shape of the motion path in these stimuli. The background motion moved coherently for the test stimulus and randomly for the reference stimulus. (B, C) Different background motion configurations. In (B), the background consisted of alternating sectors undergoing outward and inward motion, while (C) has the opposite configuration. In both figures, white circles indicate a circular motion path around a common central point, while a black ellipsoid represents the motion path distorted by the background motion.

This “circular” saltation stimulus was presented on a background of moving dots confined to 4 wedge-shaped sectors placed along the 45/225° and 135/315° axes (relative to the vertical axis). Sectors did not, therefore, overlap with the physical positions of elements but with the non-stimulated space between elements. Each sector subtended an arc of 45° from the center of the stimulus (radius of 4°) and contained 30 circular black and white dots (radius of 0.08°) set to a luminance of 4 cd/m² and 76 cd/m², respectively. To prevent density artifacts, dots were excluded from being placed within a circle of a radius of 1° around the center of the stimulus, and when they left the stimulus, they were randomly replotted to a random position in the stimulus. The dot density in each wedge sector was approximately 5 dots/deg². All dots within each sector moved in the same direction and at the same speed (which was changed across conditions, see Procedures section) and were generated asynchronously with a lifetime of 0.25 s. 

Procedures

The task of the observers was to judge the shape of the saltation stimulus path using Method of Adjustment. The previously described stimulus was presented 10° to the left of central fixation. This was the test stimulus. At the same peripheral location, but to the right, a reference stimulus was presented. The reference stimulus was a saltation sequence but different from the test stimulus in that the saltation was not illusory: Each element was physically and sequentially presented at regular intervals corresponding to polar angles of 0, 45, 90, 125, 180, 225, 270 and 315° of a circle (with a radius of 2.5°). Figure 5A illustrates this stimulus presentation. Moradi and Shimojo (2004) showed analogous illusory and non-illusory saltation percepts to be indistinguishable when presented in the periphery at short ISIs, indicating that the percepts are comparable. The comparability of these percepts was exploited in the present study as we quantified distortion of the test stimulus due to influence of the background motion. This required observers to physically change the motion path of the reference stimulus to match the test stimulus, as described below. 

Both test and reference stimuli were presented on a background of dots. In the reference stimulus, there was no coherent motion that could potentially distort the perceived position of overlapping elements. However, in the test stimulus, dots in each segment moved coherently along a radial trajectory (either inward or outward motion, see Figure 5A). Dot speed was the same for both stimuli. In the case of the test stimulus, we manipulated the configuration of the background dots between trials. There were two possible background configurations, both of which had alternating sectors containing inward followed by outward motion (clockwise): (1) sectors 1 and 3 containing outward motion and sectors 2 and 4 containing inward motion or (2) sectors 1 and 3 containing inward motion and sectors 2 and 4 containing outward motion. In both cases, we expected that elements that do not physically overlap with the sectors would be unaffected by the background motion. However, elements that are mislocalized to a position that overlaps with the background motion (when saltation occurs) would appear shifted in the background motion direction, causing the saltation path to become ellipsoid rather than circular. This is because elements overlapping with sectors containing outward motion would be displaced away from the center, elongating the motion path at these locations, while elements overlapping with sectors containing inward motion would be simultaneously shifted toward the center, shortening the path of motion. Additionally, a background motion configuration in which alternating sectors contained outward and inward motion would produce an ellipsoid path that is tilted to the left (Figure 5B), while the opposite configuration would produce a ellipsoid motion path tilted to the right (Figure 5C). 

Observers were presented with a continuous movie of both reference and test stimuli. The task was to adjust the shape of the reference stimulus (non-illusory saltation, presented on the right) to match the perceived shape of the illusory saltation (presented to the left) while maintaining fixation. The path of motion of the reference stimulus conformed to a sinusoidal modulation of a circular path according to the equation: R′ = R + Rαsin(2θ), where R′ is the modified distance of the element from the center, R is the radius of the base circle (which was 2.5°), α is the amplitude of the sinusoidal modulation, and θ is the angular difference between the element and the horizontal axis. According to this equation, α = 0 produces pure rotational saltation—a circular path; α > 0 gives an ellipsoid path with the major axis tilted 45° to the right; and α < 0 gives an ellipsoid path with the major axis tilted 45° to the left. Observers increased or decreased the α of the reference stimulus (in steps of 0.025) to match the test stimulus. The α value of the reference stimulus was randomized between −1 and 1 from trial to trial. After the observer was satisfied with the perceptual match in shape, he/she pressed a button to end the trial. Before the next presentation, random dynamic noise (5° × 5°; black (6 cd/m²) and white (76 cd/m²) pixels (0.021° × 0.021°) changing at a temporal frequency of 15 Hz) was presented overlapping with the position of the reference and test stimuli for 10 s. This prevented any buildup of an aftereffect, which may affect subsequent shape judgments. 

The adjusted α value of the reference stimulus provides a direct measure of whether the background motion distorts the path of saltatory motion (the sign of α), as well as the extent of distortion (the magnitude of α). Previous studies have reported a larger position distortion with increasing background speed (e.g., De Valois & De Valois, 1991; Tsui, Khuu, & Hayes, 2007). We therefore also examined whether any distortion in the shape of the path of visual saltation is similarly speed dependent. In separate trials, judgments of the path shape were repeated for 5 different background dot speeds (0, 1, 2, 4, and 8°/s). There were 50 trials in a block: shape judgments for 2 different background configurations, at the 5 different background speeds, repeated 5 times each. Conditions were randomized within and between blocks. Observers each completed 10 blocks such that each condition had 50 trials in total. Results were averaged across the 50 trials for each condition. 

We repeated these procedures for conditions in which both the test and reference stimuli consisted of elements undergoing non-illusory saltation (with ISI of 0.2 s). This gave an indication of whether background motion distorts the position of elements that do actually overlap with the background sectors and the extent of this distortion as a function of dot speed. Moreover, this allowed direct comparison with the illusory saltation conditions. 

Results

Results of Experiment 3 are shown in Figure 6, which plots the sinusoidal amplitude modulation (α) at which the reference stimulus was judged to be similar in shape to the test stimulus, as a function of the background dot speed, averaged across observers. Black circles represent data for conditions in which the background of the test stimulus consisted of alternating inward/outward sectors (as in Figure 5B), while gray symbols represent data for conditions in which the background consisted of alternating outward/inward sectors (as in Figure 5C). Data in Figure 6A are for the non-illusory saltation test stimulus, while data in Figure 6B are for the illusory saltation test stimulus. 

Figure 6

 

The ellipsoid shape of the reference stimulus (α) plotted as a function of background dot speed. Average observer data are shown for conditions in which the background motion configuration was either alternating outward and inward motion (black symbols and lines) or alternating inward and outward motion (gray symbols and lines). In (A), the test stimulus was non-illusory saltation, while in (B) the test stimulus was illusory saltation.

Note first from Figure 6A that the apparent shape of the reference stimulus changed as a function of the dot speed for both background configurations. When the background dots were stationary, there was no shape distortion and the judged shape of the reference stimulus matching the test stimulus was a circle (α = 0). As background dot speed increased, the reference stimulus took on a path of motion that appeared ellipsoid with the extent of distortion (as indicated by the absolute magnitude of α) increasing monotonically with speed. For the fastest speed of 8°/s, the distortion was substantial at approximately 20–30% from a circular trajectory. Additionally, the configuration of the background motion produced ellipsoid paths tilted in different directions. Where alternating sectors contained outward and inward moving dots (black symbols and lines), the ellipsoid path was tilted to the right (positive α values), while the opposite background motion configuration (gray symbols and lines) produced a path tilted to the left (negative α values). 

The results depicted in Figure 6A are consistent with the observation that motion distorts the perceived position of overlapping elements. Elements not overlapping with the background motion (i.e., those located on the 0/180° and 90/270° axes) were unaffected and appeared at their actual positions. However, elements at 45/225° and 135/315° axes (thus elements 2, 4, 6, and 8 in the sequence) did actually overlap with the background motion, and their perceived location appeared shifted in the background motion direction as a consequence. These local position shifts account for the perception of ellipsoid forms (tilted either to the left or right) reported by the observers. 

A second finding evident from Figure 6 is that illusory and non-illusory saltation test stimuli provide for similar data trends: As background dot speed increases, there is a distortion of the motion path, with the direction of distortion dependent on the configuration of the background motion. Critically, these observations are evident from Figure 6B despite the fact that, for the illusory saltation test stimulus, elements were only perceived as overlapping with the background motion—there was no physical overlap. We account for this as follows: Because the visual system interprets this stimulus as a single object undergoing circular saltation (due to the orientation configuration of elements), elements 2, 4, 6, and 8 in the sequence are mislocalized to intermediate positions that overlap with the background motion. This leads to a perceived shift in their positions in the motion direction. These local distortions are perceived on a global level as conforming to an ellipsoid path, akin to that observed when non-illusory saltation is presented on identical backgrounds. 

While the effect of background motion on the perception of illusory and non-illusory saltation is comparable, different processes must underlie their percepts. For non-illusory saltation, the percept can be accounted for by neural mechanisms that are capable of detecting the background motion and coding the position of elements that are physically presented in their receptive field. However, for illusory saltation, this cannot occur because there is no element physically placed in the receptive field. The percept most likely arises from high-level processing in which a neural representation of motion is formed in higher cortical areas and is then fed back to activate the non-stimulated space to produce the percept. However, this internal representation is distorted if the response of low-level neurons coding the non-stimulated space is biased in a particular direction by the background motion (e.g., Khuu et al., 2010). This conclusion is supported by single-cell recording studies that show reciprocal activation and projection to and from MT to lower cortical areas in response to high-level motion (e.g., McGraw, Walsh, & Barret, 2004; Sillito et al., 2006). 

General discussion

This study was concerned with the influence of featural transformation between inducing elements—in this case, spatial orientation—on the illusory motion path of visual saltation. It is clear from previous studies that differences in the spatial orientation of inducing elements gives rise to the percept of a motion path that deviates from rectilinear when a standard AM stimulus is used. However, shortfalls with the AM stimulus (as raised in the Introduction section) have meant that the exact shape of the motion path, and the orientation of elements along that path, cannot be comprehensively quantified, making the underlying mechanisms difficult to assess. This issue was, therefore, revisited here with the use of the visual saltation illusion, which overcomes many methodological difficulties inherent to AM. Experiment 1 confirmed that inducing illusory saltation with elements of incongruent spatial orientation produces the perception of a curvilinear path consistent with geometric rotation around a common point. It also showed that mislocalized elements were perceived with an orientation intermediate to the two inducing elements, indicating that the visual system is highly sensitive to the constraints that govern the movement of objects in the physical world. Experiment 2 showed a consistent relationship between the salience of orientation information and the shape of the perceived motion path. As the contrast within elements is reduced, percepts of the trajectory change from curvilinear (at high contrast) to rectilinear (at low contrast), suggesting that the visual system reserves its judgment of curvilinearity for situations in which orientation information is salient and unequivocal, otherwise defaulting to an interpretation of rectilinearity. Finally, Experiment 3 showed that background motion overlapping the non-stimulated space between elements distorts the perceived curvilinear path of illusory saltation. Moreover, distortion caused to the path of illusory saltation is equivalent to that which occurs when elements do actually overlap with the background motion, as with non-illusory saltation. This finding shows that the neural representation of curvilinear motion may occur in two steps: first, a high-level interpretation of the stimulus as an object undergoing rotation is formed, and then the percept is achieved through activation at lower cortical areas that code local position. Together, these findings give a clear indication that, even at the high level, form and motion information are not independent but rather are highly interactive. 

The results of this study strongly accord with those of previous studies that have shown that constraints are placed on the computation of high-level motion by spatial orientation information. Like Foster (1978) and McBeath and Shepard (1998), we showed that the orientation configuration of elements is important to the perceived motion path; in using the saltation stimulus rather than AM, we showed that this distortion is not an artifact of the quantification technique used in those studies. Our results, therefore, agree with the conclusion that the visual system is dependent on geometric constraints to guide its interpretation of high-level motion. Our findings are also in agreement with previous observations using AM, in particular the perception of “transformational apparent motion” where an object is perceived to change form while undergoing motion. In that case, the percept of form change obeys geometric constraints such as continuity and smoothness, much like the orientation change noted in Experiment 1 (Figure 3). Our findings add to this by showing that this interpretation of transformational change in form guides the path of motion. That illusory motion perception, as suggested by Foster, is perhaps constrained by geometric principles reflects the computation by which the visual system attempts to resolve ambiguity in the visual signal. 

The circular rotation induced by elements of incongruent orientation evident for illusory saltation in the present study is somewhat at odds with McBeath and Shepard's (1989) report of non-circular curvature in their AM path. It is likely that this difference is due to the types of stimuli and viewing positions used in these studies. McBeath and Shepard used a centrally viewed AM stimulus, and because of the fine acuity afforded by foveal viewing, the position of elements in the AM sequence is unambiguous. This likely attenuates the requirement for an interpretive process and, therefore, the influence of spatial orientation on the perceived motion path, leading to a non-circular AM path. On the other hand, the saltation stimulus used in the present study can only be perceived in the periphery where visual localization is much coarser (due to poorer visual acuity). This poorer visual localization and the ambiguity it creates likely necessitates more of an interpretative process that then gives rise to a more compelling path curvature. Accordingly, we believe that the percept of curvature in both AM and saltation is the result of the same high-level grouping mechanism, but that the influence of this mechanism depends on the amount of ambiguity in localization of elements. It is possible that greater curvature would be observed from an AM stimulus with incongruently oriented elements presented to the periphery rather than central vision. Recall though that the saltation stimulus, rather than AM, was used in the present study because of inherent difficulties in the measurement of such an effect. 

As mentioned, the mechanism(s) underlying the perception of illusory saltation is (are) likely to involve visual attention and attentive tracking. Consistent with Cavanagh (1992), Khuu, Kidd, Phu et al. (2010) argued that the illusory motion percept arises from attentive tracking whereby elements are perceptually grouped, with intermediately presented elements mislocalized as visual attention is shifted from one physical location to the next. The present findings show that this process is highly influenced by spatial orientation, which constrains the perceived path of motion if sufficiently salient (Figure 4). This attentive tracking explanation can also be applied to AM. For example, Shepard and Zare (1983) reported that the percept of curved AM is induced if a low-contrast curved line is flashed in between sequentially presented inducing elements. The path of motion appears to follow this curved path and not the straightest path between the two elements. The brief presentation of this path would guide attentive tracking along the curved path, leading to curvilinear AM. 

To our knowledge, the neural mechanisms underlying the perception of visual saltation remain unexamined. However, some guidance in this regard can be sought from previous investigations of the neural mechanisms of AM. It is widely believed that cortical areas high in the motion-processing stream, particularly the middle temporal area (MT), are responsive in the perception of high-level motion (Liu et al., 2004; Muckli et al., 2005). It is believed that a neural representation of AM is generated in MT and then fed back to lower cortical areas that code the retinotopy on a finer scale (Muckli et al., 2005). Because AM and saltation are both spatiotemporal illusions arising from grouping, it is likely that neural representation of saltation follows a similar computation. The present study suggests that these percepts may involve reciprocal interactions with form areas that code global form and shape, such as the lateral occipital cortex (LOC). Joint activation of MT and LOC has been noted in the perception of high-level motion, making this interaction a candidate system in which form and motion might interact in the perception of visual saltation. It is also possible that successive presentations of incongruently oriented elements (representing segments of the circumference of a circle) to different spatial locations activates curvature detectors, which would in turn guide the path of motion. This is speculative and requires future study directly investigating the cortical origins of this interaction. 

While we have characterized the influence of form and motion on the perception of visual saltation in terms of the assumption of geometric rotation, an alternative explanation should be considered. At first glance, the perceived path of visual saltation appears to reflect a vector combination of the direction of motion signaled by the orientation and motion of elements at the local level of analysis. Geisler (1999) noted that spatial orientation is important in the perception of motion, and a number of psychophysical studies show that the perception of motion direction is influenced by local orientation signals (Badcock & Dickinson, 2009; Or et al., 2010; Ross, 2004). For example, Or et al. (2010) demonstrated that the perceived direction of a drifting Glass pattern comprising local dot pairs oriented in a common direction is deflected in the form direction, consistent with vector averaging. While a curvilinear saltation path implies an analogous operation, we suggest that this cannot be the case. If the path of motion were a combination of form and motion at the local level, the perceived direction of motion at each spatial position within the saltation sequence would be a simple vector average of the orientation and the local motion (which is orthogonal to the orientation of the grating). This would indeed predict a curved path but one in which the curvature is (two times) greater than that noted in the present study. More importantly, it predicts the endpoint as mislocalized to a point vertically in line with the location of the first element. Neither prediction is supported by our results: the first and last elements in the saltation sequence are consistently judged at their veridical positions, grounding the percept, constraining the illusory motion to the space between these two locations. Thus, the only solution consistent with our observations is one in which the motion of elements follows a direction orthogonal to the perceived orientation of elements, and that this solution is derived from a high-level interpretation of motion. 

To conclude, the present study observed that incongruence in spatial orientation information between inducing elements distorts the perceived path of visual saltation. The perception of a curved visual saltation path implies a computation in which the orientation configuration of elements provides a cue for spatial rotation around a common point. This would suggest that perception of visual saltation arises from a high-level interpretation that is highly reliant on the form of the elements in the temporal sequence. 

Acknowledgments

We thank the observers who participated in the study. This research was supported by an Australian Research Council (ARC) Discovery Project Grant (DP110104713) to S. Khuu and ARC Grants (DP0666206 and DP1097003) to D. Badcock. 

Commercial relationships: none. 

Corresponding author: Sieu K. Khuu. 

Email: [email protected]. 

Address: School of Optometry and Vision Science, The University of New South Wales, Sydney, NSW 2052, Australia.