The shape of motion perception: Global pooling of transformational apparent motion

Abstract

Abstract

Abstract:

Abstract  Transformational apparent motion (TAM) is a visual phenomenon highlighting the utility of form information in motion processing. In TAM, smooth apparent motion is perceived when shapes in certain spatiotemporal arrangements change. It has been argued that TAM relies on a separate high-level form-motion system. Few studies have, however, systematically examined how TAM relates to conventional low-level motion-energy systems. To this end, we report a series of experiments showing that, like conventional motion stimuli, multiple TAM signals can combine into a global motion percept. We show that, contrary to previous claims, TAM does not require selective attention, and instead, multiple TAM signals can be simultaneously combined with coherence thresholds reflecting integration across the entire stimulus area. This system is relatively weak, less tolerant to noise, and easily overridden when motion energy cues are sufficiently strong. We conclude that TAM arises from high-level form-motion information that enters the motion system by, at least, the stage of global motion pooling.

Introduction

Considerable psychophysical and neurophysiological evidence indicates that different neural pathways and cortical regions process form and motion (Goodale & Milner, 1992; Livingstone & Hubel, 1987; Ungerleider & Mishkin, 1982). There is, however, a growing body of research showing that form and motion can interact. In biological motion, for example, the relative motion of a small number of dots gives the vivid impression of a moving figure (Johansson, 1973, 1976). Evidence suggests that the same global form mechanism is sensitive to shape information, and shape information derived from illusory displacement in perceived position (Dickinson, Han, Bell, & Badcock, 2010). Motion streaks, arising from the extended temporal integration period of neurons in early visual cortex, improve global motion discrimination (Edwards & Crane, 2007) and provides a form cue in the direction parallel to the motion signal, which can refine direction estimates (Apthorp et al., 2013; Badcock & Dickinson, 2009; Barlow & Olshausen, 2004; Burr & Ross, 2002; Francis & Kim, 2001; Geisler, 1999; Ross, 2004; Ross, Badcock, & Hayes, 2000). Providing form cues indicating a closed contour also enhances recovery of the global motion direction compared to an open contour (Lorenceau & Alais, 2001; Lorenceau & Lalanne, 2008). In addition, form information provided by the aperture edge of an ambiguous motion signal dramatically changes the direction of perceived motion (Badcock, McKendrick, & Ma-Wyatt, 2003; Beutter, Mulligan, & Stone, 1996; Kooi, 1993), and orientation of the first-order carrier texture alters the perceived direction of motion of a second-order envelope (Cropper & Badcock, 2008). Moreover, an object's shape and orientation influences its perceived speed as objects appear to be moving faster when aligned to the motion direction (McCarthy, Cordeiro, & Caplovitz, 2012; Seriès, Georges, Lorenceau, & Frégnac, 2002). Finally, and most surprisingly, adapting to still images that depict movement purportedly generates the motion aftereffect when tested with real motion stimuli (Winawer, Huk, & Boroditsky, 2008). 

Apparent motion arises when two spatiotemporally separated objects are presented in succession, leading to the percept of a single moving object (Exner, 1888). Ambiguity in apparent motion displays can arise when more than two objects are present, forcing the visual system to decide which objects are matched (Anstis, 1980; Ullman, 1979). Most studies showed that shape and color have little effect on this matching, supporting the traditional separation between form and motion processes (Kolers & Pomerantz, 1971; Kolers & von Grünau, 1976; Navon, 1976). The dominance of spatiotemporal factors in matching is commonly referred to as the “nearest neighbor” principle, which is usually consistent with a motion-energy account of motion perception (Adelson & Bergen, 1985; Chubb & Sperling, 1988). There have, however, been some demonstrations showing that form can influence matching but only after controlling for the more salient factor of spatiotemporal proximity (Green, 1986a, 1986b, 1989; Hein & Cavanagh, 2012; Hein & Moore, 2012). Form can also affect the perceived path of apparent motion after controlling for matching (Khuu, Kidd, & Badcock, 2011; Kim, Feldman, & Singh, 2011; Shepard & Zare, 1983). For example, Khuu et al. (2011) showed that orthogonally rotating the element's orientation in a manner consistent with rigid rotation around a distant point caused the path of apparent motion to also appear curved. 

Transformational apparent motion (TAM) is argued to be a unique type of apparent motion stimulus that arises when an instantaneous shape change causes the perception of motion in the direction explaining the shape change. Importantly, TAM seems to suggest that form information alone can drive the motion system. In initial demonstrations of TAM, originally called “illusory line motion” (ILM) (Hikosaka, Miyauchi, & Shimojo, 1993a, 1993b; Kanizsa, 1951), a small square instantaneously changes into an overlapping long bar, leading the square to appear to smoothly morph in the direction of the elongation (Movie 1). The effect was initially thought to arise through a high-level attentional gradient with the small square acting as an attentional cue, resulting in faster processing of the surrounding space and the consequent appearance of a growing line. Critically, however, these initial demonstrations of TAM do not require high-level processes in order to be explained and can be entirely accounted for by low-level motion energy as the perceived direction of motion is consistent with the shift of the luminance centroid (Downing & Treisman, 1997; Fuller & Yu, 2009). 

 

Movie 1.

 

Shows an example of ILM, the original description of TAM. This example consists of discrete overlapping form cues with a small square alternating with a bar. The perceived direction of motion explains the shape change.

Following this initial description of form-induced motion, Tse and colleagues (Tse & Caplovitz, 2006; Tse, Cavanagh, & Nakayama, 1998; Tse & Logothetis, 2002) developed the now stereotypic TAM stimulus in which motion is perceived in the direction opposite to motion energy (Movie 2). They also showed that certain versions of TAM can violate the nearest-neighbor principle with matching, instead following gestalt-like grouping rules rather than spatiotemporal proximity (Tse et al., 1998). In these displays, multiple shapes are simultaneously perceived to morph in directions that maintain contour continuation rather than following spatiotemporal proximity. The high-level nature of the system was further confirmed by showing the form information driving the percept of TAM occurs after the stage of 3-D shape processing (Tse & Logothetis, 2002). On the basis of these results, Tse and colleagues concluded that TAM is processed in a separate high-level form-motion stream, which is not dependent on low-level motion energy (Tse & Caplovitz, 2006; Tse et al., 1998; Tse & Logothetis, 2002). Relatively little experimental work has, however, examined how this high-level system interacts with the low-level and well-described energy motion system (Lu & Sperling, 1995). 

 

Movie 2.

 

The stereotypical TAM stimulus that Tse and colleagues (1998) claim cannot be explained by low-level motion-energy detection. The small block on the left appears to grow into the large blob on the right following shape change. The location of the luminance centroid of the shape was found in each frame (indicated by the black dot), which moves 6 pixels left (∼10% of frame length) from the first to the second frame. Motion is thus perceived in the direction opposite low-level motion-energy predictions.

One way to investigate this issue is by examining how multiple local TAM motion signals integrate into a global percept of motion, a phenomenon that has been well investigated with conventional motion stimuli (e.g., Baker, Hess, & Zihl, 1991; Newsome & Paré, 1988). Spatiotemporally-oriented neurons in early visual cortex initially detect local motion in the environment, but the small size of their receptive fields (∼1° of visual angle) leads to the well-known aperture problem in which individual neurons cannot signal the veridical direction of 1-D stimuli (Adelson & Movshon, 1982; McCool & Britten, 2008). Pooling multiple local motion signals in a higher stage, referred to as global motion pooling, is thought to solve this problem (Adelson & Movshon, 1982; Amano, Edwards, Badcock, & Nishida, 2009). Area V5/MT, in which receptive fields are approximately 10 times larger than in V1, is important for global motion integration, responding strongly to both global motion signals (Born & Yu, 2005; Kohn & Movshon, 2004; Smith, Snowden, & Milne, 1994) and apparent motion sequences (Sterzer, Haynes, & Rees, 2006; Wibral, Bledowski, Kohler, Singer, & Muckli, 2009). More recently, global motion stimuli have also been shown to strongly activate area V3a (Braddick, O'Brien, Wattam-Bell, Atkinson, & Turner, 2000; Tootell et al., 1997). The involvement of V5/MT and V3a has been confirmed as lesions and trains of magnetic pulses to these areas decreases sensitivity to global motion (Baker et al., 1991; Beckers & Zeki, 1995; ffytche, Guy, & Zeki, 1996; Harvey, Braddick, & Cowey, 2010; McKeefry, Burton, Vakrou, Barrett, & Morland, 2008; Newsome & Paré, 1988). 

Given that the first- and second-order motion systems globally integrate (Badcock & Khuu, 2001; Edwards & Badcock, 1995), it would seem sensible to predict the same phenomenon would occur with TAM. The precise nature of pooling, however, may differ from that arising from first- and second-motion signals as form cues in conjunction with motion energy give the precise axis of motion (Geisler, 1999). Previous studies have shown that, although motion signals can be pooled, the pooling process depends on whether the signals are 1-D or 2-D (Amano et al., 2009). Form cues act to convert local motion signals to 2-D, for example, when motion streaks, additional lines, or hard-edged apertures are employed (Badcock et al., 2003; Edwards, Cassanello, Badcock, & Nishida, 2013), and suggest grouping (using form inputs that imply a 2-D direction) and will be pooled using vector averaging rather than intersection of constraints. Dynamic Glass patterns (a series of independently generated static Glass patterns shown in a fast temporal sequence) do not present a consistent motion energy signal but do give the impression of coherent motion consistent with the pattern type, showing the strong effect form cues can have on the perceived direction of motion (Ross et al., 2000). The perception of coherent motion in the absence of coherent motion energy suggests the form information in the Glass patterns causes the perceived axis of motion (Apthorp et al., 2013; Badcock & Dickinson, 2009; Dickinson & Badcock, 2009; Nankoo, Madan, Spetch, & Wylie, 2012). 

It should also be noted that there are both theoretical and empirical reasons to believe that TAM might not act in the same manner as other motion systems. For example, Tse and colleagues (Tse & Caplovitz, 2006; Tse et al., 1998) have argued that selective attention is uniquely necessary to track the shape changes in TAM. If that is the case, global integration may be impossible as numerous studies have shown that attention limits the visual system to simultaneous tracking of a maximum of four objects (Intriligator, 2001; Pylyshyn & Storm, 1988; Yantis, 1992). Alternatively, Grossberg and colleagues (Berzhanskaya, Grossberg, & Mingolla, 2007; Francis & Grossberg, 1996) proposed that TAM results from low-level processing with form information that is extracted in V2 being passed to V5/MT, in which the percept of apparent motion is generated. It would be expected from Grossberg's model that multiple TAM signals could be integrated as form information enters the motion system by the global pooling areas of V5/MT or V3a. Tse (2006) conducted a functional imaging study comparing activation between versions of illusory line motion shapes to forms with similar spatial properties but not resulting in TAM. He showed that both form and motion processing areas (V2, V3, V3a, V5/MT, LOC) show greater activation for TAM compared to the static form stimuli. More recently, a study using intracranial electrode recordings that, unlike Tse (2006), carefully controlled for motion energy showed that TAM produces activation in LOC and V4 but only a weak response in V5/MT (Bertrand et al., 2012). On the other hand, if we do find that TAM globally integrates, then it would suggest the form information needed for TAM feeds into the motion system by at least this level. 

In a series of experiments, we examined global pooling of multiple TAM signals. We first established that multiple TAM stimuli would yield a coherent global motion percept. To do this, we developed the “global TAM” array, which consists of multiple TAM stimuli that are aligned with a common global direction. We initially used modified versions (referred to as “Plunger stimuli”) of the stereotypic TAM elements shown in Movie 2. When these were placed in the global array, the perceived global motion direction was in the opposite direction to TAM motion but consistent with motion energy. A follow-up experiment confirmed these stimuli were pooled using motion energy. We then used TAM versions of Kanizsa (1976) stimuli that created a motion percept in the direction of local TAM motion, indicating they pooled using high-level TAM-related mechanisms rather than low-level motion energy. This system appears to have a much lower noise tolerance than the conventional motion energy system. A control experiment showed that the reported coherence thresholds for motion direction cannot be explained by inferences from form cues alone, suggesting that observers were indeed using the illusory motion. Finally, it has previously been argued that selective attention is needed for the high-level matching process needed in single TAM displays. We show that attention does not limit pooling in global TAM, and instead, observers appear to simultaneously integrate multiple signals across the entire array. 

General methods

Apparatus and procedure

The stimuli were generated in MATLAB 7.2 on a Pentium PC (2.4 GHz) for all experiments. Stimuli were presented from the frame buffer of a VSG2/5 (Cambridge Research Systems, Cambridge, UK) on a Sony Trinitron G420 monitor with a refresh rate of 100 Hz at a resolution of 1024 × 768 pixels (17° × 12.8° of visual angle). Observers viewed the stimuli from a distance of 102 cm, making each pixel extend 1′ of visual angle with viewing distance maintained with a chin rest. Luminance was gamma-corrected using an Optical OP-200-E (head model no. 265) photometer and associated CRS software. Observing occurred in a darkened room with luminance <1 cd/m². The background luminance of the monitor was set at 45 cd/m², and the elements were presented with a constant luminance of 90 cd/m². 

All experiments employed versions of the global TAM array with the same presentation periods. Every trial began with a centrally-presented fixation dot with luminance of ∼0 cd/m² displayed for 250 ms. This was followed by the two-frame global TAM array with each frame lasting 250 ms and no interstimulus interval. The presentation time was chosen to be well within the 100 ms integration time of single TAM elements (Tse & Logothetis, 2002). Observers made responses with a computer mouse. 

Observers

Four experienced psychophysical observers (all males), ranging in age from 21 to 48 years (median = 26.5 years), participated in the study with three observers included in each experiment. MT and ED are authors, and all other observers were naïve to the aims of the experiments. All observers had normal or corrected-to-normal visual acuity as assessed using a Snellen chart. The procedure was in accordance with the Declaration of Helsinki and approved by the Human Research Ethics Committee at the University of Western Australia with all observers providing written informed consent. 

Experiment 1: TAM “Plungers” stimuli pool in motion energy direction

Stimuli

Multiple spatially-distributed stationary Plunger TAM elements were placed in a circular array to assess global pooling of TAM (Movie 3). The arrangement of the elements was similar to the global Gabor array (Amano et al., 2009; Cassanello, Edwards, Badcock, & Nishida, 2011) except Gabors were replaced with TAM stimuli. Forty-eight local Plunger elements, each enveloped in a 64′ × 64′ window, were arranged in an annulus with an 8.53° diameter. No TAM elements were displayed in the central 128′ of the array because the limited space in this region constrained the range of possible orientations. Analogously to global dot motion and Glass patterns (i.e., Edwards & Badcock, 1995; Wilson & Wilkinson, 1998), a proportion of the elements were assigned a common global direction (clockwise or anticlockwise rotational motion) with the remaining proportion assigned random directions. Two frames of each array were presented with all elements changing into the completed TAM shape in the second frame, giving the perception of apparent motion. Under conditions of high signal and low noise, observers reported the array appeared to rapidly rotate following shape transformation. 

 

Movie 3.

 

An example of the novel TAM global array stimuli (with a 100% coherence level) that effectively pools for a perception of global motion. In this example, the global motion direction is clockwise, which is opposite the perceived direction of motion for a single element but is consistent with motion energy predictions.

Procedure

We conducted an experiment measuring the proportion of coherently aligned local TAM elements necessary for global motion perception, using a single-interval forced-choice (SIFC) task to evaluate global integration. The proportion of elements coherently aligned with a common global direction instead of random directions (noise) was varied to determine the minimum signal needed to detect the direction of global motion. The method of constant stimuli (MOCS) with nine linearly-spaced levels was used to vary the signal-to-noise ratio with steps individually varied for each observer to sample the entire psychometric function. The signal elements were made to produce either clockwise or anticlockwise rotational motion. Observers were required to indicate the direction of motion, and auditory feedback was given following each response. Observation showed that auditory feedback did not alter the perceived direction. Blocks of 180 trials were completed with each observer making 100 responses for each of the nine MOCS steps. Cumulative Gaussian functions were fitted to each observer's responses using the PAL_CumulativeNormal function in the MATLAB Palmeades toolbox (Prins & Kingdom, 2009), which yielded the 75% threshold. The 95% confidence interval for threshold was computed using a bootstrap procedure from Palmeades. 

Results and discussion

Unexpectedly, the perceived global motion of the array was in the direction opposite to the perceived individual element motion. The direction of global motion was, therefore, consistent with the low-level motion energy signals defined by centroid shifts, suggesting the global percept arises from motion energy rather than the separate high-level, form-driven motion system. Figure 1 shows mean coherence thresholds for global TAM for each observer. Observers required 21% to 35% of elements moving coherently for reliable global motion discrimination. These thresholds are considerably higher than those reported for global dot motion, which are typically between 5% and 15% (Baker et al., 1991; Edwards & Badcock, 1995; Newsome & Paré, 1988), but similar to those for static Glass patterns (Badcock, Clifford, & Khuu, 2005; Dickinson, Broderick, & Badcock, 2009; Wilson & Wilkinson, 1998). It is likely that some of this disparity arises because global dot motion thresholds are derived from a series (typically eight) of stimulus frames. In fact, our two-frame thresholds align well with other experiments using only two stimulus frames (Edwards & Badcock, 1995). 

Figure 1

 

Mean coherence thresholds for all observers. Error bars indicate bootstrapped 95% confidence intervals.

Experiment 2: Plunger TAM stimuli are pooled using motion energy

The results of Experiment 1 suggest the Plunger stimuli are pooled using motion energy rather than TAM. To determine whether the motion-energy signals evoked by the Plunger stimuli were being pooled, we exploited the “motion drag” effect, in which the perceived position of peripherally-presented objects is spatially displaced in the direction of motion (Scarfe & Johnston, 2010; Whitney, 2006; Whitney & Cavanagh, 2000; Whitney et al., 2003). To do this, we first established that the global TAM array induced motion drag. We then measured the effects of placing a brief flash between the TAM frames, which masks conventional apparent motion by providing substantial nondirectional motion energy (Braddick, 1973). The masking occurs because the flash provides a uniform field that activates low-level motion detectors, thereby disrupting motion energy–based matching between successively presented shapes. On this reasoning, if the Plungers are being pooled using motion energy, then the presentation of a flash should reduce motion drag. 

Method and procedure

This experiment used the same two-frame global array employed in Experiment 1 with the addition of two horizontally-aligned Gabors briefly presented on either side of the array coincident with the shape change (Figure 2). The procedure was based on that used by Scarfe and Johnston (2010), who showed that a global Gabor array can induce motion drag. Each peripheral Gabor was 0.83° of visual angle with a spatial frequency of 4.9 c/° and a peak contrast of 84%. The phase of the Gabors was randomized for each trial. The Gabors were presented for the first three frames (30 ms) simultaneously with the TAM elements changing shape. The task required the observers to judge the relative spatial offset of the Gabors, which was controlled for using the MOCS procedure. There were nine linearly-spaced levels with the distance between each level varied between observers to allow for the entire psychometric function to be sampled. Three interleaved conditions were employed with the local TAM elements producing either clockwise, anticlockwise, or random local (unidirectional) global motion. In the clockwise and anticlockwise conditions, all elements were aligned to a common direction that resulted in a strong motion percept. The no-motion condition was a control condition with the direction of all elements randomized. The perceived position of the Gabors will be offset in the direction of motion if the TAM array causes motion drag. For example, in the clockwise condition, the right Gabor will generally be perceived to be higher and will, therefore, need to be physically lower to be perceived as aligned with the left Gabor. 

Figure 2

 

Stimuli from Experiment 2 measuring motion drag induced by the Plunger stimuli. (a) A schematic diagram of a trial. (b) An enlarged version of the test frame of the sequence, which was displayed for 30 ms. In this example, the left Gabor is physically higher than the right Gabor but will often be perceived to be lower because of the anticlockwise motion of the global TAM array. In the flash condition of this experiment, a disk covering the area of the annulus was displayed for 10 ms immediately preceding the frame with the Gabors.

To determine whether the drag induced was due to motion energy, the same procedure was used with the addition of a white flash presented in the middle of the TAM sequence between the first and second frames. The Gabors were presented synchronously for 30 ms with the onset of the second frame of the TAM sequence in which the elements changed shape, which was immediately following the bright flash. The flash, presented for one frame (10 ms) immediately preceding the shape change, had a luminance of 90 cd/m² and covered the same circular area as the global TAM array. Each observer completed 100 trials in a randomized order for each MOCS step in all conditions. Cumulative Gaussian functions were fitted to each observer's responses for the three motion conditions in both flash and no-flash conditions using GraphPad Prism (version 5.0d, GraphPad Software, San Diego, CA). This yielded the point of subjective equality (PSE) with 95% confidence intervals, indicating the degree of motion drag induced by each type of TAM array. 

Results and discussion

The top panel of Figure 3 shows that the global TAM array produced significant drag in the direction of global motion. For all observers, the Gabors were displaced in the direction of motion energy and opposite TAM for both coherent motion conditions, and there was no displacement for the randomized-direction condition. There was a slight response bias for observers MT and TM with the right Gabor being perceived as higher than the left Gabor in the no-motion condition. This was consistent in the other two conditions with the psychometric functions shifting to the left. Motion drag differed significantly between the three conditions: one-way repeated-measures ANOVA of the PSEs, F(2, 8) = 21.59, p < 0.01. The slope of the functions also significantly increased in the condition with the flash (Mean slope = 0.21, SD = 0.01) compared to the condition without the flash (Mean slope = 0.11, SD = 0.02), suggesting that when there is motion within the arrays it significantly reduces the ability to judge misalignment. 

Figure 3

 

Psychometric functions for observers MT and RG in Experiment 2 for the clockwise (blue lines), anticlockwise (red lines), and no-motion (maroon lines) conditions. The top panel shows the condition without the flash, and the bottom panel shows the flash conditions. Note that step size for the flash condition is half that of the no-flash condition. Error bars indicate 95% confidence intervals.

The bottom panel of Figure 3 shows psychometric functions when a flash was inserted between the frames. There was significantly less drag here compared to the no-flash condition. To quantify this effect, the difference between the PSE of the clockwise and anticlockwise conditions was found for flash and no flash conditions (Figure 4). The PSE was significantly lower when there was a flash inserted between the frames, t(2) = 4.80, p = 0.04, R² = .92. As the flash disrupts short-range motion energy by providing a strong omnidirectional motion energy signal, this result is consistent with the conclusion that the global Plunger stimuli are pooled using motion energy rather than high-level TAM-related motion systems. 

Figure 4

 

The difference in PSE between clockwise and anticlockwise motion in both flash (red bars) and no-flash (blue bars) conditions. For all three observers, the difference between these was significantly reduced when a flash was inserted between frames.

Experiment 3: TAM is pooled by a separate system

Taken together, the results of Experiments 1 and 2 indicate that Plunger stimuli are globally pooled but using motion-energy signals rather than the proposed high-level TAM system. This suggests that when multiple elements are presented, the high-level TAM system can be relatively easily overridden by the low-level motion energy system. However, our initial question concerning whether multiple TAM signals can be pooled into a global motion percept has yet to be answered. We used two additional TAM stimuli to answer this question. The first was derived from Baloch and Grossberg (1997), in which TAM is created by morphing a bar with an illusory middle to an illusory square defined by four “Pac-man” inducers (Movie 4). As with the Plunger stimuli, the shape change of these stimuli—hereafter referred to as the Kanizsa (1976) stimuli—displaces the centroid of the motion implied by low-level motion detectors opposite to the direction of perceived motion. But, unlike the Plunger stimuli, when the Kanizsa stimuli were placed in the global array, they were found to pool in the direction of TAM, i.e., opposite the direction signaled by motion energy. The second novel stimulus was a variation of the ILM stimuli shown in Movie 1. This stimulus offers an interesting comparison as both motion energy predictions and TAM are in consistent directions. 

 

Movie 4.

 

A single element of a Kanizsa stimulus. Like the Plunger stimulus, the centroid of the object (indicated by the black dot) moves in the opposite direction to the perceived direction of TAM, but this shift is four times smaller than in the Plunger stimulus. The centroid for each frame was found using MATLAB, which showed the centroid moved ∼2 pixels left from the first to the second frame.

The purpose of this experiment was to compare global pooling between three different stimuli, pooled using TAM and/or motion energy. We first examined global motion thresholds for TAM pooling opposite to motion-energy predictions (Kanizsa stimuli). This was compared to a condition in which TAM globally pools in the motion-energy direction, but motion is perceived locally in the opposite direction (Plunger stimuli). Finally, Plunger stimuli were compared to ILM in which TAM and motion-energy predictions are the same. If thresholds were higher for the Plunger stimuli than the ILM stimuli, it would suggest that systems underlying global TAM pooling interact with the low-level motion energy system. 

Method and procedure

Global thresholds were measured separately for each of the three stimulus types: Plungers, Kanizsa, and ILM. The Plungers were the same as those used in Experiments 1 and 2. The Kanizsa stimuli consisted of four nonoverlapping circles with radii of 7′. The stimuli were slightly rectangular with the centers of the top and bottom row of circles separated by 21′, and the centers of the left and right circles were separated by 25′. To induce TAM motion, a line with a width of 4 pixels and the same luminance as the background extended from the center of the lower circle to the center of the upper left circle in the first frame. For the second frame, the area between the circles was converted to an illusory square by removing a quarter of each circle, which produced the percept of the line growing into a rectangle (see Movie 4). For the ILM stimuli, a 5′ × 9′ rectangular block was changed to a 5′ × 25′ rectangular block. 

To measure global TAM thresholds for the three stimulus types, we presented an array with X% of signal elements and 100% – X% of noise elements (Movie 5). A SIFC adaptive staircase procedure with a three-down, one-up rule was used to estimate the 79.4% point of the psychometric function (Levitt, 1971). Observers were required to identify whether the array appeared to rotate clockwise or anticlockwise. Following previous studies of global motion (Cassanello et al., 2011; Edwards & Badcock, 1995), all staircases started with high signal levels, in this case, 100% coherence threshold (48 signal elements). Until the first mistake was made, the coherence level was reduced by eight elements following each correct response. After the first incorrect response, the step size was reduced to four, two, and one for subsequent reversals and maintained at one for the final four reversals. Staircases terminated after eight reversals with the threshold being taken as the mean coherence level of the final four reversals. Auditory feedback was given following each response before the next trial was initiated. Observers completed 10 staircases for each stimulus type in a counterbalanced order. 

 

Movie 5.

 

Two versions of TAM stimuli placed in the global array with both stimuli having 100% coherence. Left panel: A TAM global array containing the Kanizsa stimuli, which pool in the direction of TAM but in the opposite direction of motion-energy predictions.

 

Movie 5.

 

Right panel: A global TAM array containing ILM stimuli that pool in both the direction of TAM and motion energy. The third stimulus type (Plungers) for this experiment is shown in Movie 3.

Results and discussion

Figure 5 shows mean coherence thresholds for the three types of stimuli. The results clearly show that the Kanizsa stimuli integrated into a global motion percept in the direction opposite to motion-energy predictions. Motion thresholds were, however, significantly higher than for either the Plunger or ILM stimuli. For two observers (MT and RG), both Plunger and ILM stimuli have similar thresholds. Observer ED shows a slightly different pattern of results with thresholds for Plunger stimuli being between those for ILM and Kanizsa. The results for the Kanizsa stimuli suggest that multiple TAM signals can be integrated into a global motion percept, but noise tolerance is reduced compared to motion energy–based pooling as indicated by the higher thresholds. Another possible explanation for this threshold increase is the greater stimulus complexity of the Kanisza figures relative to Plunger or ILM may have led to increased crowding. ILM appears to be pooled using motion energy as thresholds are similar to the Plunger stimuli and motion is in the opposite direction to the single-element motion. 

Figure 5

 

Coherence thresholds for the three types of stimuli in Experiment 3 for the three observers. Error bars represent 95% confidence intervals.

Experiment 4: Global TAM is not due to form pooling

Although observers were asked to report the direction of the “twist” that followed shape changes, it is possible the observers were instead determining the direction of the global structure from the orientation information provided by the form cues from the TAM stimuli. To test this possibility, a control experiment was conducted to examine whether the coherence thresholds for global TAM were due to observers using the global pattern rather than motion information in the arrays. This was done by presenting only one frame of the TAM arrays in order to retain form information without generating illusory motion. 

Method and procedure

The same global arrays with Kanizsa and Plunger stimuli were used as in Experiment 3 but with only one frame of the stimulus shown. Only the second frame of the Plunger stimuli was shown, and the first frame of the Kanizsa was shown for 250 ms because these frames gave the strongest directional cue. The second frame of the Kanizsa was rectangular and centrally placed in the inducers and therefore did not indicate whether the element was oriented clockwise or anticlockwise. In the first frame, however, the line in the two left “Pac-men” could be used to signal direction. Observers were required to indicate whether elements had a global clockwise or anticlockwise orientation. Ten thresholds were taken for each stimulus type using adaptive staircase procedures in a counterbalanced order. 

Results and discussion

Figure 6 shows mean coherence thresholds for the two stimulus types. All observers could identify the global orientation direction if the signal was sufficiently high. Critically, however, thresholds for the Kanizsa stimuli were approximately 20% higher (71% vs. 49%) than in Experiment 3 when TAM motion signals were present. Thresholds for the Plunger stimuli were also higher (45% vs. 40%) than in Experiment 3 although the difference is less dramatic because the Plungers provide a much clearer orientation. This could suggest that the observers were using the orientation information from the form cues within the Plungers to determine the global motion direction rather than the illusory motion itself although observers reported using the illusory motion for direction judgment rather than the form cues. Overall, the results clearly show that thresholds increased for static Kanizsa stimuli compared to the motion-inducing displays used in Experiment 3, suggesting that observers in Experiment 3 integrated local TAM signals to infer global motion rather than relying on static form information only. 

Figure 6

 

Mean coherence thresholds for Experiment 4 when a single frame of the Kanizsa and Plunger stimuli was presented to measure coherence thresholds when TAM was not present. Error bars represent 95% confidence intervals.

Experiment 5: Global TAM pooling cannot be explained by selective attention

It has previously been suggested that selective attention is needed for the spatiotemporal matching underlying perception of TAM (Tse et al., 1998; Tse & Logothetis, 2002). Spatial attention has a finite capacity as only a small number of items (possibly four) can be simultaneously tracked (Intriligator, 2001; Pylyshyn & Storm, 1988; Yantis, 1992), and this would therefore imply that multiple TAM signals should not be globally integrated. On this account, the relatively high thresholds found for the global Kanizsa TAM stimuli may reflect the fact that observers derived the motion direction by selectively monitoring small regions of space rather than integrating motion signals across the display. One way to assess this possibility is to measure thresholds while increasing the number of individual elements in an array. Thresholds for both conventional global form and motion pooling show a dependence on the number of local elements in an array, increasing by increasing the total number of elements when the area is held constant (Badcock et al., 2005; Dickinson et al., 2009; Edwards & Badcock, 1995; Williams & Sekuler, 1984). With TAM signals, however, it is difficult to increase the number of elements without increasing the area of the annulus. Therefore, we instead used two complementary sets of conditions: one in which the number of elements remained constant but the area of the annulus increased and a second in which both the number of elements and the annulus area increased (see Figure 7). If observers are determining the global motion direction by monitoring a small region of space, thresholds should remain constant as long as the density of the array is constant. On the other hand, if observers are monitoring the entire array, thresholds will increase with increases in the size of the annulus and the total number of elements. 

Figure 7

 

Single-frame examples of the global TAM arrays for the two levels of conditions in Experiment 5. Examples from the constant-density (outlined in red) and the constant-number (outlined in blue) conditions with increasing annulus area. Note the same array for the constant-number and constant-density condition for the 89° square condition is the same.

Method and procedure

In all the conditions, the size of the local Kanizsa elements was halved from the previous experiments so that each was contained within a 32′ × 32′ window. These were placed in annuli that varied in area (89°, 175°, 389°, 430° squared). In the first level of conditions, there were 100 local TAM elements for all annulus sizes; thus the element density decreased with increasing annulus size. In the second level of conditions, the number of elements increased (100, 204, 312, 447) with increasing annulus size to maintain a constant element density. Coherence thresholds were estimated for seven conditions (the 89° square annulus condition was the same for both levels) using the same SIFC adaptive staircase procedure used in Experiments 3 and 4 with 10 staircases conducted for all conditions in a counterbalanced order. 

Results and discussion

There was still clear pooling of TAM using the smaller Kanizsa elements with the annulus appearing to twist following shape change when coherence was high. Figure 8 shows coherence thresholds across the two conditions as a function of increasing annulus size. The pattern of results is the same across all three observers. When the number of elements was kept constant, thresholds increased with increasing annulus size. However, thresholds increased much more rapidly when the density of the array remained constant with increasing annulus size. To quantify this relationship, we fitted √N (with N being square degrees of visual angle) curves to the constant-number level to determine how area affects coherence thresholds. Figure 8 shows that these curves predict the thresholds for the constant-number condition, suggesting that the area over which the signals are integrated itself acts as noise, interfering with detection of global TAM. On this reasoning, we then fitted √N² curves for the constant-density condition, assuming that when both area and number of elements was increasing, it was likely the independent noise from each would summate, causing a quadratic increase in threshold. Consistent with this logic, these curves provided a good fit for the coherence thresholds in the constant-density conditions. 

Figure 8

 

Thresholds for the three observers in Experiment 5 for increasing the size of the annulus in two conditions; one with 100 TAM elements always presented (constant number) or with the number of elements increased so that density was constant in all conditions. The continuous lines indicate the fitted √N and √N² curves for the constant-number and constant-density levels of conditions, respectively.

The additive effect of area and number of elements with increasing annulus size strongly indicates that observers were using signals across the entire annulus to determine motion direction. Our results, therefore, show the global TAM thresholds do not reflect merely monitoring a small region of the array but instead reflect integration across the entire annulus. In turn, to use signals across the entire annulus area, observers would necessarily need to perceive multiple simultaneous instances of TAM (up to at least the 447-element arrays used here). These results, therefore, demonstrate that TAM occurs without needing to attentively track each element; contrary to previous claims (Tse & Caplovitz, 2006; Tse et al., 1998), based on observations using only a single TAM element, that selective attention is needed in the parsing and matching steps that drive TAM. Our demonstration that multiple TAM elements can be simultaneously integrated provides empirical evidence that selective attentional engagement is not always necessary for the perception of TAM. It is also highly unlikely that crowding explains these results as thresholds increased in the constant-number condition with increasing eccentricities, and there was decreasing density, meaning that crowding was consistent or even decreasing (Nandy & Tjan, 2012). Considering the strong relationship we found between the results of the two conditions, it seems highly likely thresholds in the constant-density condition increased because noise came from, as in the constant-number condition, area combining with the additional elements presented in this condition. This explanation is, therefore, consistent with crowding not causing the increase in thresholds. 

The result that both the number of local elements and the area itself that the elements are presented over act as noise is novel. Thresholds increased by √N with increasing area size in the constant-number condition and by √N² in the constant-density condition when both number of elements and area increased. Previous studies in motion examined thresholds when increasing the number of local elements in the same size annulus, thus increasing density, finding a ∼√N increase (Badcock et al., 2005; Dickinson et al., 2009; Edwards & Badcock, 1995). But no studies using conventional motion stimuli have yet systematically determined the relationship between both annulus area and number of elements on coherence thresholds. Our results are different from global form pooling as thresholds are constant when area increases with constant density, showing area does not act as noise (Dickinson et al., 2009). This suggests that global TAM pooling is not being treated as form information. The effect of both area and number of elements for TAM, therefore, present a novel finding for the global pooling of a motion stimulus. In future studies, it would be beneficial to establish the effect of area on conventional global motion pooling in order to determine whether TAM is being treated as a motion stimulus. 

General discussion

Summary of the experiments

The present study demonstrates that multiple TAM elements can integrate into a coherent global percept. However, this global pooling was only observed after carefully controlling for motion energy as such signals easily override the high-level TAM pooling. This suggests that TAM is pooled by a separate system with considerably less tolerance to noise than the standard motion-energy system. Control experiments showed that coherence thresholds cannot be explained by observers discriminating motion direction on the basis of static form cues that produced much higher thresholds. Our work also shows that global TAM pooling is not limited by observers selectively monitoring a small region of the annulus and instead reflects integration across a larger region. This is in contradiction to previous claims that selective attention is necessary for the perception of TAM with simpler stimulus displays (Tse et al., 1998). 

Global form-motion interactions

The thresholds reported for all three types of global TAM motion were significantly higher than those for global dot motion, which are typically 5% to 15% (Baker et al., 1991; Edwards & Badcock, 1995; Newsome & Paré, 1988). Notably, however, global dot-motion coherence tasks need to consist of at least eight frames for asymptotic performance, suggesting that optimal global motion integration occurs over an extended period whereas thresholds are ∼20% when only two frames are presented (Edwards & Badcock, 1995; Snowden & Braddick, 1989). This could explain the higher thresholds obtained for the Plunger and ILM arrays, which are pooled using motion energy and consisted of only two frames. However, thresholds for the Kanizsa stimuli, which are pooled in a direction consistent with TAM, were also considerably higher than Plunger and ILM stimuli. One possible explanation is that global form-induced motion pooling integrates more weakly than that using motion-energy inputs. Another option is suggested by Tse and Logothetis (2002) who argued that TAM objects are matched and parsed in each frame. It is possible that this process has less noise tolerance than simple spatiotemporal matching, thereby yielding higher thresholds. However, we think this explanation is unlikely as Experiment 5 showed thresholds decreased (as a proportion of total elements) when the number of elements in the same size annulus increased. A final possibility is that increased thresholds may have resulted from crowding as the Kanizsa stimuli were more complex than the simpler Plunger and ILM stimuli. 

Our finding that multiple Kanizsa TAM signals can be integrated provides additional insights into where form information can enter the motion system. There is considerable evidence showing V5/MT is necessary for global motion perception as this area shows greater responses to coherent than incoherent motion (Born & Yu, 2005; Kohn & Movshon, 2004; Smith et al., 1994), and disruptions to activity lead to reduced global motion sensitivity (Beckers & Hömberg, 1992; Beckers & Zeki, 1995; Newsome & Paré, 1988). Similarly, V3a also strongly responds to global motion stimuli (Braddick et al., 2000). Considered together with evidence that versions of TAM stimuli that do not control for motion energy seem to activate V5/MT (Tse, 2006; but see Bertrand et al., 2012), our results provide strong evidence that form information necessary for the TAM percept can enter the motion system by the level of V5/MT and/or V3a. This result is consistent with several different propositions about how form motion might reach V5/MT. For example, Grossberg and colleagues (Baloch & Grossberg, 1997; Francis & Grossberg, 1996) have argued for a form-motion “boundary completion wave” emerging from activity in V2 that feeds into V5/MT. This model can account for the perception of TAM, including the Kanizsa stimuli used in the current study. 

There is also substantial literature suggesting that V4 is central to global form perception as it integrates shape information across disparate orientations (Felleman & Van Essen, 1991; Gallant, Connor, Rakshit, Lewis, & Van Essen, 1996; Ungerleider, Galkin, Desimone, & Gattass, 2008). Furthermore, Kanizsa stimuli strongly activate both V2 and V4 while only weakly activating V1 (Lee & Nguyen, 2001; Pan et al., 2012). Joint consideration of these points opens up the possibility that extensive lateral cortical connections between V4 and V5/MT allow form information to provide an input for global processing in the motion system (Felleman & Van Essen, 1991; Maunsell & van Essen, 1983; Ungerleider et al., 2008). Alternatively, there is growing evidence that V4 itself is sensitive to motion with a large number of direction-sensitive cells within this area (Roe et al., 2012). This opens up the possibility that the TAM percept could also result directly from activation within V4 although, when considered in conjunction with the aforementioned results, it is likely that the global motion areas are also involved. As for V3a, globally processed form information in TAM could have reached this area through its bilateral projections with V4 (Ungerleider et al., 2008). These results are also consistent with the finding that long-range apparent motion causes activation in the ventral visual stream (Zhuo et al., 2003). V3a is responsive to moving shapes with contour curvature, showing the region is sensitive to form-motion interactions (Caplovitz & Tse, 2006). Tse and Logothetis (2002) have previously shown that 3-D processing of form information in TAM precedes the motion percept. Again, this suggests a strong role of both V5 and V3a in global TAM pooling as these areas are strongly responsive to disparity cues necessary for 3-D form perception (Adams & Zeki, 2001; Anzai, Chowdhury, & DeAngelis, 2011; Backus, Fleet, Parker, & Heeger, 2001). 

It is also worthwhile to comment on likely differences between brain areas subserving TAM and biological motion. The later areas of the dorsal stream, superior temporal sulcus, and superior temporal polysensory areas have all been found to be central in the separate form-motion interaction of biological motion (Grossman & Blake, 2002; Vaina, Solomon, Chowdhury, Sinha, & Belliveau, 2001). Additionally, biological motion is still found in observers with bilateral lesions to the temporal lobes, who are unable to globally process form (Gilaie-Dotan, Bentin, Harel, Rees, & Saygin, 2011). This highlights what is likely a central difference between biological motion and TAM; TAM could not be perceived without global form processing because the form changes appear to drive the illusory percept. 

Crossover between TAM and motion energy pooling

One of the most notable aspects of our results was the difference between global motion percepts generated by the Plunger and Kanisza TAM stimuli. Whereas the former showed global motion in the direction of motion energy, the latter did not. This dissociation suggests that the high-level filling-in process necessary for TAM breaks down when the Plunger stimuli are placed in the global array, and the visual system reverts to low-level centroid tracking. One possible explanation for this lies in the fact that the Plunger stimuli contain a stronger motion energy signal than the Kanizsa stimuli as the centroid shift is approximately twice as large. To test this hypothesis, we constructed a modified version of the Kanizsa global array in which the luminance of the elements was altered to produce a larger centroid shift in the direction opposite to TAM (Movie 6). This modification diminished the perceived global rotation in the direction of TAM following the shape changes although, unlike the Plungers, the motion direction was not reversed. We attempted to collect global motion thresholds using an adaptive staircase procedure for this stimulus. However, observers could no longer reliably judge the motion direction even with very high coherence levels. This suggests that increasing low-level motion energy simply swamped the processes underlying the global motion percept. When taken in conjunction with our previous results, it provides additional evidence that the motion-energy system overrides the TAM system when motion energy cues are sufficiently strong. 

 

Movie 6.

 

The modified version of the Kanizsa global TAM array with a greater centroid shift. The luminance of the two “Pac-men” in the direction opposite TAM motion increases in the second frame of the sequence. This causes the stimuli to have a similar motion energy signature as the Plunger stimulus. Observers could no longer complete a coherence task when this stimulus was placed in a staircase procedure, suggesting that the illusory motion had been greatly diminished.

The relationship between global TAM pooling and motion energy suggests a possible functional role for form changes during optic flow. If an observer is continually fixating forward in an optic-flow situation, then the appearance of shapes on the retina will morph with the relative motion of the observer. The ability for multiple TAM signals to integrate suggests that the visual system could use the information from form changes during optic flow to help ascertain motion direction. However, as we found that the motion energy relatively easily overwhelms global TAM pooling, it appears that the visual system has a preference for motion-energy signals over motion information derived from form changes. Furthermore, the results from Experiment 5 show TAM thresholds dramatically increase with an increasing number of elements in the arrays—far more so than conventional motion stimuli. This suggests that the motion system may not be optimized for integrating information derived from shape changes. 

Conclusions

Overall, the current results show that multiple TAM signals can integrate to create a global perception of motion although this can only be seen after carefully controlling for motion energy. A separate global system appears to be involved in global TAM pooling that shows a much lower tolerance to noise than the conventional motion energy–driven system. This is possibly due to the complexity of the stimuli or because the motion information is form-driven, which requires a greater signal level. Furthermore, we showed that observers were integrating local TAM signals across the entire array and, therefore, simultaneously perceived multiple TAM signals, indicating that selective attention is not necessary for TAM motion to be perceived. 

Acknowledgments

This work was supported by Australian Research Council grants DP130102580 to DRB, DP110104553 to DRB and ME, and DP120102313 to TAWV. The University of Western Australia also provided a scholarship to MFT. 

Commercial relationships: none. 

Corresponding author: Matthew F. Tang. 

Email: [e-mail]. 

Address: School of Psychology, The University of Western Australia, Crawley, WA, Australia.