Multiple mechanisms of illusory contour perception

Abstract

A variety of mechanisms have been proposed to explain illusory contour formation. However, since prior studies have focused on a single mechanism, there is no clear consensus regarding contour formation mechanisms. Here we developed a novel vivid dynamic display we call the chomping pacman that allows measurement of minute differences in contour clarity. This illusion is shown to be more vivid than traditional static displays. Using the method of adjustment, we are able to identify three mechanisms of contour formation: extrapolation, interpolation, and figural. We posit that these three mechanisms combine to form the general illusory contour percept in traditional static Kanizsa displays and likely contribute to real-world contour perception.

Introduction

Contours and surfaces form the basic building blocks of visual perception. Ideally, the object surfaces represented in an image are each delineated by a closed set of contours signaled by a contiguous path of luminance, texture, and/or color changes. Simple and complex cells in the primary visual cortex can then easily extract local contour information and pass it through a hierarchy of visual processing stages to permit the perception of surfaces and objects (Felleman & Van Essen, 1991). However, the human visual system is also very effective in detecting visual contours even when local image information fails to provide complete luminance, texture, and/or color cues (Ehrenstein, 1941; Kanizsa, 1955; Schumann, 1900; Varin, 1971). Such contours are known as ‘illusory contours.’ Contrary to the name, ‘illusory contours’ in natural images nearly always reflect real-world surface boundaries rather than illusions. 

Illusory contours have been studied for more than a century (Kanizsa, 1955; Schumann, 1900; Varin, 1971), since illusory contour displays serve as useful tools for investigating of contour and surface perception. Some investigations have framed their findings in terms of computational mechanisms (e.g., Lesher & Mingolla, 1993; Shipley & Kellman, 1992b), while others have discussed their results in terms of perceptual cues (e.g., Albert, 2001; Anderson, 1994). Here, the focus is on mechanisms. There is no clear consensus on the mechanisms of illusory contour formation (Albert, 2007; Anderson, 2007; Kellman, Garrigan, Shipley, & Keane, 2007; Lesher, 1995; Petry & Meyer, 1987; Seghier & Vuilleumier, 2006; Spillmann & Dresp, 1995). 

Existing mechanistic theories of illusory contour processing can be divided into three broad classes: contour interpolation (Dresp & Grossberg, 1997; Grossberg & Mingolla, 1985; Grossberg, Mingolla, & Ross, 1997; Grossberg & Yazdanbakhsh, 2005; Guttman & Kellman, 2004; Kellman & Shipley, 1991; Raizada & Grossberg, 2003; Singh & Hoffman, 1999), contour extrapolation (Field, Hayes, & Hess, 1993; Li, 1998; Pillow & Rubin, 2002; Shipley & Kellman, 2003; Singh & Fulvio, 2005), and figural feedback (Albert & Hoffman, 2000; Anderson, 1997; Gregory, 1972; Grossberg et al., 1997; Lee, 2002; Nakayama & Shimojo, 1990, 1992; Raizada & Grossberg, 2003; Rock & Anson, 1979). Contour interpolation mechanisms inwardly connect pairs of aligned inducer fragments to close contours across intervening gaps, for example a bipole uniting two edges in the Boundary Contour System (Grossberg & Mingolla, 1985). Contour extrapolation mechanisms extend outward from a single fragment, for example by taking a random walk outward from inducers (Williams & Jacobs, 1997). In contrast to these contour-based computations, figural feedback mechanisms suggest that the contour fragments support the perception of a surface or figure and that this figural representation feeds back to fill in contour gaps, for example by combining visual cues to a surface (Carman & Welch, 1992). Contour interpolation and contour extrapolation are often observed neurophysiologically and frequently described as either feed-forward or local cooperative processing mechanisms (Ffytche & Zeki, 1996; Grosof, Shapley, & Hawken, 1993; Hirsch et al., 1995; Lee & Nguyen, 2001; Ramsden, Hung, & Roe, 2001; Seghier et al., 2000; Sheth, Sharma, Rao, & Sur, 1996; von der Heydt, Peterhans, & Baumgartner, 1984), and evidence for figural feedback is often observed in later visual areas (Mendola, Dale, Fischl, Liu, & Tootell, 1999; Stanley & Rubin, 2003). 

When establishing any classification of existing theories, some theories may not fall directly within category boundaries. With this caveat in mind, some existing theories of illusory contour formation contain components consistent with extrapolation, interpolation, and surface/figural processing, as well as some components that are orthogonal to these categories. For example, Anderson (1997) has proposed a theory of illusory contour formation governed by occlusion relationships, where occlusion cues such as junctions are utilized to establish depth relationships of surfaces. These junction and/or disparity cues must provide evidence of an occluding surface; otherwise, illusory contours are not formed. In this vein, this work is included as indicative of a figural or surface processing mechanism, although these mechanisms are fundamentally different than other theories within this category. Other works contain components of multiple categories, such as the work of Grossberg and colleagues (Grossberg & Mingolla, 1985; Grossberg & Yazdanbakhsh, 2005; Raizada & Grossberg, 2003), which initially (Grossberg & Mingolla, 1985) contained an interpolation mechanism in the boundary contour system (BCS) and a surface-based mechanism within the feature contour system (FCS). Later models refined these computations to an interpolation mechanism in layer 2/3 (Raizada & Grossberg, 2003; Ross, Grossberg, & Mingolla, 2000) and a monocular surface filling-in process in V2 and a binocular process in V4 (Grossberg & Yazdanbakhsh, 2005). Extrapolation is permitted in these models when supported by interpolation (Grossberg et al., 1997; Raizada & Grossberg, 2003; Ross et al., 2000). 

Although interpolation, extrapolation, and figural mechanisms give different theoretical accounts of illusory contours, these mechanisms work on different representational levels and are in no way mutually exclusive. In order to investigate the hypothesis that multiple mechanisms may contribute to illusory contour perception two tools are required: a robust illusory contour display and a metric for measuring the strength of illusory contours. The present studies employ both. We have developed a novel illusory contour display, the ‘chomping pacman’ (CP) shown in Movie 1, in which the ‘mouth openings’ of pacman inducers in a Kanizsa square display oscillate between acute and obtuse angles. This display produces a very robust illusory contour percept that allows the dissection of multiple mechanisms of contour formation. 

 

Movie 1

 

The chomping pacman illusion.

Here we present evidence that the three classes of contour mechanisms contribute to the perception of one illusion and show that the relative contribution of each mechanism can be directly observed by manipulating features of the same illusion. 

Overview of experiments

There are three primary sets of goals to the present study. The first set of goals is to introduce and investigate a new illusory contour display, the chomping pacman (CP) display, and to produce an extremely vivid illusory contour percept. These goals are pursued in Experiments 1 and 2. The second set of goals is to utilize this robust illusory display to investigate and distinguish extrapolation, interpolation, and figural mechanisms contributing to illusory contour perception. These goals are pursued in Experiments 3 and 4. The final set of goals is to investigate the competitive interactions between these illusory contour mechanisms and mechanisms that support luminance-defined contour perception. Experiments 5 and 6 investigate these questions by introducing luminance-defined occlusion cues into variants of the chomping pacman display. 

These investigations are predicated on the ability to measure the vividness of illusory contour percepts as a continuous value. Although illusory contours are frequently discussed in binary terms (present or absent), there is strong evidence that the strength of illusory contours varies continuously and that subjects can perceive and report differences in illusory contour strength ( Figure 1). In the present study, we developed a method of adjustment paradigm for quantifying vividness in which subjects adjust the support ratio of a static illusory contour display to match the vividness of the illusory contour in the experimental display. This method was chosen because it is well established that the clarity of illusory contours is directly tied to the support ratio (Shipley & Kellman, 1992b). Using the support ratio to quantify contour clarity allows subjects to perform a direct comparison to an existing illusory contour, rather than to make subjective ratings (Lesher & Mingolla, 1993; Shipley & Kellman, 1992b), or to make a difficult comparison to another type of contour (Banton & Levi, 1992). Manipulating support ratio of a standard that is also an illusory contour avoids making assumptions about the precision or type of contour that is generated (Banton & Levi, 1992; Guttman & Kellman, 2004; Stanley & Rubin, 2005) and allows for a direct and continuous comparison of cue contributions. We found this method to be both sensitive and reliable and subjects reported little difficulty in employing this method. Additional details of this method are described below. 

Figure 1

 

The support ratio is defined as the ratio between the real (solid orange) and the real and illusory (dotted blue) components of the illusory contour figure. As the support ratio increases, illusory contour clarity increases (Shipley & Kellman, 1992b).

Experiment 1

The spatio-temporal properties of the new chomping pacman (CP) display were investigated in order introduce and explore the illusion produced by this display format. Experiment 1 parametrically varies angular sweep frequency (chomp frequency), support ratio, inducer separation, and color components. Investigation of these tuning properties will help illuminate the mechanisms that support contour perception for this display. The primary goal of these experiments was to find the spatio-temporal parameters that produce the most vivid illusion. Maximizing the illusion strength serves to support the investigation of multiple mechanisms in two ways. First, from a signal-to-noise standpoint, a large amplitude effect should be much easier to dissect than a small effect. Second, large amplitude effects may result from recruitment of multiple mechanisms, while weaker effects might be supported primarily by a single mechanism. The spatial properties of this kinetic illusion were compared to known properties of static illusory contour displays. Motion is expected to improve the clarity of illusory contours (Anderson & Barth, 1999; Anderson & Sinha, 1997; Bradley, 1987; Bruno & Gerbino, 1991; Kellman & Cohen, 1984; Ni, Wang, Wu, Wang, & Li, 2003); however, speed dependence has not been previously investigated with non-rigid motion of the inducers. Speed-dependent effects also might help to reveal a signature of the perceptual grouping mechanisms that support enhanced contour clarity in the CP display. Alternatively, motion cues might have an “all or none” effect on clarity enhancement. The CP display allows for a parametric variation of motion speed, by altering the chomp frequency of pacmen inducers. Finally, these effects might vary by stimulus size, so we tested the effects at 3 different support ratios and 3 different figure sizes. 

Methods

Four experienced psychophysical observers (2 female, all right-handed, 23–28 years old) participated in the study. All subjects had normal or corrected-to-normal vision. One author (MH) served as a subject. All subjects were previously familiar with the display and the measurement technique. Subjects viewed displays presented by either an Apple Powerbook G4 or PowerMac G4 (Apple Computer, Cupertino, CA) generated by VisionEgg software ( http://www.visionegg.com). Subjects freely viewed a 19-in. monitor (LaCie, Hillsburo, OR) from a distance of roughly 57 cm. 

Subjects viewed a solid white CP Kanizsa on a black background. The inducers of the test stimulus were continuously swept between openings of 60 degrees and 120 degrees (see Movie 1). Subjects viewed stimuli at three different inducer center separations of 2.64, 5.27, and 7.89 degrees and three different support ratios of 0.25, 0.5, and 0.75. Subjects viewed angular motion at 6 different frequencies (0.25, 0.5, 1.0, 2.0, 4.0, and 8.0 Hz; corresponding to π/24, π/12, π/6, π/3, 2 π/3, and 4 π/3 radians/sec). 

Subjects were instructed to adjust the size of inducers of a static match stimulus with a mouse to match the perceived clarity of a test stimulus. The match stimulus had the same inducer separation as the test stimulus and thus varied between the three inducer separations. The test stimulus was presented for 250 ms, and subjects could review the presentation at will. The match stimulus was always visible. Subjects performed one trial for each support ratio, inducer separation and speed, resulting in 54 trials. Trials were presented in a pseudorandom sequence. This method has the advantage that subjects used one (static) illusory contour to report the clarity of another (moving) illusory contour. The method of adjustment technique allows for subjects to make finer grain comparisons of contour clarity. Typically, such measurements are made on a discrete 10-point rating scale (Albert, 2001; Lesher & Mingolla, 1993; Poom, 2001; Shipley & Kellman, 1992b), which can be improved by using a continuous measure. The use of an illusory contour as a matching stimulus avoids making assumptions about the nature of illusory contours or difficulties in matching an illusory contour to a real contour (Banton & Levi, 1992). Contour clarity is known to vary with changes in support ratio (Shipley & Kellman, 1992b). It is important to note that contour clarity may vary in a number of other dimensions, especially in asymmetric and complex figures; however, the manipulation of inducers of symmetrical Kanizsa diamond only serves as metric of clarity, not a complete description of all illusory contour phenomena. Subjects did not report any difficulty in comparing static and moving illusory contours for any of the experiments in the present study. 

For each trial, the support ratios of the test and match stimuli were calculated. The support ratio of the match stimulus was divided by the support ratio of the test stimulus to obtain a measurement of relative illusion strength. Thus, relative illusion strengths of 100% match in clarity a static Kanizsa of equal support ratio, relative illusion strengths greater than 100% reflect an illusion stronger than a static Kanizsa diamond, and relative illusion strengths less than 100% reflect an illusion weaker than a static Kanizsa diamond. Note that this measure does not allow for a measurement of “no illusion.” Subjects were asked to reduce the size of the inducers to 0 if they did not perceive an illusory contour. Subjects never reported that an illusion was too strong to be rated by the static Kanizsa diamond. 

Results and discussion

Figure 2 shows a plot of relative illusion strengths over the speeds tested. At lower speeds, the illusion strength of the test stimulus is close to the strength of a static illusion, as the speed increases, the illusion reaches a peak, and the illusion becomes weaker past 2 Hz. The data exhibited an inverted U shape. For simplicity, we choose to fit the data with a quadratic function; other reasonable models produce qualitatively similar estimates of the peak frequencies ( Equation 1):  

$S\left(\widehat{f}\right)=A+B{\log }_{10}(f+1)+C{\log }_{10}(f+1{)}^2$

(1)

 

Figure 2

 

CP Illusion strength as a function of angular speed. Error bars represent standard error. The solid line represents a quadratic fit to the data.

This model indicates that the peak illusion strength occurs at approximately 2 Hz (1.94 Hz; A = 90, B = 164, C = −177). The quadratic term was significant ( F(1,21) = 9.31, p < 0.01), which suggests the relationship between chomp frequency and illusion strength is non-linear. Thus, because the quadratic term is significant and negative, there exists a peak illusion strength. 

The frequency tuning of the illusion strength likely reflects an intrinsic property of the mechanisms of illusory contour completion. The peak frequency response is slow compared to the range of speeds that the visual system, especially area MT, can process (Newsome, Mikami, & Wurtz, 1986). The slow speed may be indicative of either a figural or interpolation mechanism, which operates at a lower frequency because it integrates information non-locally. The peak frequency of 2 Hz lies somewhat below preferred motion local processing frequencies in V1 and V2 (Levitt, Kiper, & Movshon, 1994; Priebe, Lisberger, & Movshon, 2006), the likely location of an interpolation mechanism (Grossberg & Mingolla, 1985; Kellman & Shipley, 1991; Lee & Nguyen, 2001; von der Heydt et al., 1984). Alternatively, the slow peaks may reflect the temporal limitations of the ventral visual stream in human V4 and ventral temporal lobe, regions likely to be associated with a figural mechanism. Recent neuroimaging work has shown that peak temporal frequency responses in VO-1 (Liu & Wandell, 2005) and V4, FFA, and PPA (McKeeff, Remus, & Tong, 2007) occur at slower frequencies than occur in lower-order visual cortical areas. Thus, a figural mechanism may have a slow temporal limit reflected in our results. 

The finding of a slow ideal motion frequency can also be thought of in terms of binding motion information. Investigations in motion processing across space (Verghese & Stone, 1996), attentional tracking of rotation (Verstraten, Cavanagh, & Labianca, 2000), and apparent depth motion displays (Beverley & Regan, 1974; Norcia & Tyler, 1984) find similar performance decrements at high speed and/or frequency. This suggests that while the visual system can accurately perceive motion at high rates, the integration of motion into a coherent percept may require slower mechanisms, such as a slow interpolation or figural mechanism. 

Figure 3 shows the relative illusion strength by speed across a range of support ratios. Although the peak remains relatively constant around 2 Hz, the enhancement from the moving stimulus is more pronounced at lower support ratios. There was no significant effect of support ratio on illusion strength measurements at either 2 or 4 Hz (2 Hz: F(2,9) = 3.28, p = 0.09; 4 Hz: F(2,9) = 2.52, p = 0.14). Despite the lack of an effect of support ratio, there may be a bias in the data for lower support ratios to have stronger illusion strengths. For high support ratios, subjects have a smaller dynamic range to adjust the support ratio of the illusion, which may result in a smaller illusion strength ratio for large support ratios. 

Figure 3

 

Illusion strength as a function of support ratio. Quadratic fit values, see Equation 1 (0.25: A = 57.9, B = 370.1, C = −344.2, peak = 2.4 Hz; 0.50: A = 112.7, B = 84.7, C = −121.6, peak = 1.2 Hz; 0.75: A = 99.5, B = 42.8, C = −65.1, peak = 1.1 Hz).

Figure 4 shows the relative illusion strength by speed across a range of figure sizes. There was no significant effect of figure size at 2 or 4 Hz on illusion strength (2 Hz: F(2,9) = 0.17, p = 0.84; 4 Hz: F(2,9) = 0.07, p = 0.93). As in Figure 3, the peak remains constant around 2 Hz. This suggests the enhancement of the contours is not dependent on figure size but rather on the speed of motion and the support ratio of the inducers. This lack of change in illusion strength across figure sizes is in agreement with previous studies who find illusion strength to be dependent on support ratio (Banton & Levi, 1992; Shipley & Kellman, 1992b) and not the retinal size (but see Dumais & Bradley, 1976). The use of a comparison method to like-sized figures has the potential to introduce an artifact if clarity varies by retinal size (Dumais & Bradley, 1976) because the ratio technique will not reflect this absolute difference. The size-independence findings can be explained by multi-scale contour mechanisms and/or figural mechanisms, and are inconsistent with fixed-scale contour mechanisms. 

Figure 4

 

Illusion strength as a function of figure size, quadratic fit values, see Equation 1 (2.6: A = 90.8, B = 123.9, C = −124.4, peak = 2.1 Hz; 5.3: A = 92.5, B = 138.7, C = −134.8, peak = 2.3 Hz; 7.9: A = 86.8, B = 235.1, C = −271.6, peak = 1.7 Hz).

Experiment 2

The CP display creates a very vivid illusory contour, as demonstrated by Experiment 1. One goal in designing the CP display is to create an illusory contour that is vivid enough to allow dissection of multiple components of contour completion. Static displays, which produce relatively weak illusory contours, do not provide sufficient dynamic range in perceptual salience to resolve fine changes in contour clarity. Thus, we sought to increase to clarity of the CP display by incorporating other cues known to enhance the clarity of static displays. We also sought to investigate the interactions between different cues to illusory contour perception. 

Motion is not the only cue that enhances the clarity of illusory contours. Replacing a solid pacman with a pacman composed of several lines, each with distinct “line ends,” enhances static contour clarity (Lesher & Mingolla, 1993). Additionally, contours are more vivid when neon color spreading variants are employed (Varin, 1971). Although each cue form enhances illusion strength, the interaction between the different cue types is unclear. It is possible that as more cues are added together, the contour clarity could saturate. To investigate the role of spatial cues, we examined 3 variants of inducers: Kanizsa, Varin rings, and neon Varin rings. 

Methods

Subjects were five experienced psychophysical observers (2 female, all right-handed, 25–28 years old). All subjects had normal or corrected-to-normal vision. One author (MH) served as a subject. All five subjects were previously familiar with the display and the measurement technique. The test stimulus varied between moving and static Kanizsa, Varin ring, and neon Varin ring configurations. Stimulus configurations are shown in Figure 5. The Varin ring inducers consisted of 6 concentric rings of 25-arcmin width, with an inner 37-arcmin radius pacman. The neon Varin ring figure consisted of the same concentric rings as the Varin experiment; however, in the open region the lines were magenta. The chomp speed of the test stimulus in moving conditions was held constant at 2 Hz. In all conditions the test stimulus support ratio was held constant at 0.45 (an inducer radius of slightly larger than 1 degree), and the center-to-center horizontal inducer separation was held constant at 6.8 degrees. 

Figure 5

 

Experiment 2 stimulus configurations. From left to right, Kanizsa, Varin rings, and neon Varin rings.

Results and discussion

Figure 6 shows the results of Experiment 2. A within-subjects ANOVA revealed main effects of motion ( F(1,4) = 103.11, p < 0.05) and inducer type ( F(2,8) = 5.86, p < 0.05). No significant interaction effect between inducer type and motion was found ( F(2,8) = 2.51, p = 0.14). Paired t-tests revealed significant differences between moving and non-moving Kanizsa trials, Varin ring trials, and neon Varin ring trials ( t(4) = 11.14, p < 0.05; t(4) = 5.63, p < 0.05; t(4) = 6.87, p < 0.05), showing that static cues that enhance the illusion are each enhanced with motion. Paired t-tests failed to reveal a significant difference between moving Kanizsa and Varin figures ( t(4) = 0.32, p = 0.76) or between Varin ring and neon Varin ring ( t(4) = 1.99, p = 0.11). A significant difference was observed between Kanizsa and neon Varin ring ( t(4) = 3.90, p < 0.05). This shows that the strongest illusion comes from a moving neon color-spreading figure. 

Figure 6

 

Illusion strength as a function of inducer type.

The addition of each cue to the display enhances the strength of the illusory contour percept. In static conditions, changing the figure to concentric rings makes a stronger illusion, as does making the figure a complete neon color spreading illusion. Adding motion as a cue enhances each one of these conditions; thus, the moving neon color spreading illusion is most vivid illusory contour display. Although we do not quantify the nature of the relative contribution of each type of cue, motion appears to have an additive effect on contour clarity across the three display types as shown by the lack of an interaction effect between inducer type and motion. 

Experiment 3

The goal of this experiment is to investigate the hypothesis that extrapolation mechanisms may contribute to illusory contour perception. There is much experimental and theoretical work to support the role of bottom-up processes in illusory contour formation (Grossberg & Mingolla, 1985; von der Heydt et al., 1984). However, both extrapolation and interpolation effects could explain experimental results (Lee & Nguyen, 2001; von der Heydt et al., 1984). Interpolation requires a long-distance linking mechanism to join two aligned contour segments. Alternatively, early visual areas could use local connections to extrapolate a contour from already existing local contours (Pillow & Rubin, 2002; Williams & Jacobs, 1997). Extrapolation and interpolation mechanisms make very different predictions about the percept that should result from a single inducer. Extrapolation mechanisms predict that an illusory contour should extend out from even a single inducer, while interpolation mechanisms predict no illusory contour for a single inducer. While there is little prior evidence to suggest that a single static inducer supports an illusory contour percept in a non-stereo display, we have observed that a single “chomping” inducer, regardless of inducer type, can result in the perception of illusory contours that “shoot out” from the inducer. These illusory contours are relatively weak and fade in clarity with the distance from the inducer, however, they can be enhanced by the presence of a large white object opposing the mouth of the contours. These effects were measured in Experiment 3. 

Methods

Five experienced psychophysical observers (2 female, all right-handed, 21–28 years old) participated in the study. All subjects had normal or corrected-to-normal vision. One author (MH) served as a subject. Four subjects had previously participated in earlier experiments in the study. The stimulus was a single pacman figure with a radius of 1 degree, equivalent to the size of a single inducer in the test stimuli of Experiment 1. The open mouth portion of the pacman was solidly filled with magenta, appearing as a disk with a magenta wedge. In the moving condition, the wedge varied in size from 60 to 120 degrees, at a chomp frequency of 2 Hz. In the static condition, the inducer opening was 90 degrees. A large white arc of 3.41 degrees radius was simultaneously presented at various distances from the single pacman inducer. The arc was clipped by a window of 13 × 10 degrees (see Movie 2). Subjects viewed static or moving single inducers for 20 trials at each of 10 different arc distances for a total of 400 trials. Trials were presented in a random order. Subjects were asked to respond if an illusory contour was present or absent in the figure, completely between the inducer and the arc. If subjects saw an illusory contour near the pacman inducer, but not near the large arc, they were instructed to respond “illusory contour absent.” 

 

Movie 2

 

Single inducer stimulus configuration. A large white arc is placed near the single inducer. The size of the inducer is enlarged relative to experimental display in order to enhance contour formation.

Results and discussion

Individual subject data (illusory contour ‘present’ percentage) were fitted to separate standard logistic sigmoid psychometric functions for moving and static conditions. Figure 7 shows the mean subject thresholds (50% illusory contour ‘present’) of Experiment 3. A paired t-test revealed a statistical difference between static and moving 50% thresholds, t((4) = 3.15, p < 0.05). To identify a contour, the arc must be placed beyond 0 degrees from the inducer. In other words, a significant difference from 0 for the 50% threshold would indicate that observers identified an illusory contour. A paired t-test of thresholds greater than 0 was not significant for the static condition ( t(4) = 2.27, p = 0.08). The moving condition was significantly different from 0 ( t(4) = 11.25, p < 0.05). 

Figure 7

 

Single inducer illusion strength (arc distance thresholds; 50% “contour present” responses) for moving and static illusion forms.

These results support an extrapolation mechanism of illusory contour formation. Without another colinear inducer or edge to interpolate between, a contour is still generated. Previous work has suggested that contour completion can be redirected by other non-inducing cues such as a dot (Shipley & Kellman, 2003). Other reports have generated illusory contours from single inducing elements, utilizing stereo (Anderson, 1994; Anderson & Julesz, 1995) or motion (Anderson & Barth, 1999; Anderson & Sinha, 1997). The illusory contour generated by our display differs fundamentally from the two prior displays created by motion: this illusory contour is perpendicular to the direction of motion and thus is not traced out in space-time by the contour terminators (Anderson & Sinha, 1997); and the contour does not imply that an occluding surface is present (Anderson & Barth, 1999). Importantly, with the single inducer display, an illusory contour is observed to be ‘shooting out’ from the inducer, whereas previous demonstrations have been of interpolation between inducing components (Anderson & Barth, 1999; Shipley & Kellman, 2003). Perhaps the most striking effect of this display is the transfer of the percept to the static case. After participating in the study, some subjects reported that initially before viewing the moving contours, they did not see a contour in the static condition, however, after repeated viewing during measurement, subjects occasionally did see a contour in the static condition. This suggests that the mechanism for illusory contour formation utilized by the single pacman display is enhanced by motion, but not formed by motion. 

Extrapolation effects have been described in previous models (Grossberg & Raizada, 2000; Ross et al., 2000); however, extrapolation was “pruned” away with competitive interactions (Ross et al., 2000). The contour generated in this manner seems to be perceptually weaker than the other contours in this paper; however, direct comparisons between this weak contour and other illusions were difficult to make. A weak illusory contour suggests that an extrapolation mechanism of completion is also weak and may require support from other mechanisms such as interpolation or figural/surface interactions. 

Experiment 4

Experiment 3 suggests that extrapolation could be one mechanism of illusory contour formation. The purpose of this experiment is to investigate interpolation and figural mechanisms. If illusory contour formation is governed solely by extrapolation or interpolation mechanisms, altering the relationship of inducers on one side of the figure should not affect clarity of the illusion on the other side. Alternatively, if a figural mechanism exists, contour clarity could be enhanced beyond the sum of interpolation and extrapolation effects. In this experiment the phases of the chomping inducers are manipulated to support and disrupt putative figural mechanisms. 

Methods

Seven experienced psychophysical observers (3 female, all right-handed, 21–28 years old) served as subjects. All subjects had normal or corrected-to-normal vision. One author (MH) was a subject. All seven subjects were previously familiar with the display and the measurement technique. 

Stimuli were similar to the Kanizsa condition of Experiment 2, except the test stimulus had inducers completely in phase (all-in-phase), three inducers in phase (3-in-phase), two inducers in phase (2-in-phase), or no inducers in phase (out-of-phase). In the static trials, out-of-phase inducer mouth openings were placed at 120 and 60 degrees. Thus, for 3-in-phase, the top, left, and right inducers were placed at 120 degrees, and the bottom at 60; for 2-in-phase, the top and left inducers were placed at 120 degrees and the bottom and right at 60 degrees; and for out-of-phase, the top and bottom were placed at 120 degrees and left and right at 60 degrees. For motion trials, the out-of-phase inducers were chomping one-half cycle out of phase. Thus, for motion trials, for 3-in-phase, the bottom inducer was one-half cycle out of phase with the top, left, and right inducer; for 2-in-phase, the right and bottom inducer was one-half cycle out of phase relative to the left and top inducer; and for out-of-phase, the top and bottom inducers were one-half cycle out of phase with the left and right inducers (see Movie 3). 

 

Movie 3

 

Temporal phase stimuli of Experiment 4. From left to right: out-of-phase, 2-in-phase, 3-in-phase, and all-in-phase. Subjects were asked to rate the upper left illusory contour in all conditions. Note that for 2-in-phase, 3-in-phase, and all-in-phase, the upper left illusory contour inducers are locally and temporally identical.

The procedure was the same as Experiment 2. Subjects performed seven repetitions of four phase types (all-in-phase, 3-in-phase, 2-in-phase, or none-in-phase) and two motion types (moving or static) for a total of 56 trials. Subjects were asked to only compare the upper left edge of the test and adjust stimuli, which contained an in-phase edge for all-in-phase, 3-in-phase, and 2-in-phase conditions. These comparisons permit the analysis of possible figural contributions in excess of possible interpolation and extrapolation mechanisms. 

Results and discussion

Figure 8 shows the results of Experiment 4. A within-subjects ANOVA reveals significant main effects of motion ( F(1,6) = 43.71, p < 0.05) and phase ( F(3,16) = 27.05, p < 0.05). Paired t-tests of the motion illusion show that the difference between out-of-phase and 2-in-phase is significant ( t(6) = 4.30, p < 0.05), as is between 2-in-phase and all-in-phase ( t(6) = 3.58, p < 0.05), and the difference between 3-in-phase and all-in-phase was not significant ( t(6) = 2.01, p = 0.09). However, there is no statistical difference between 2-in-phase and 3-in-phase ( t(6) = 0.73, p = 0.49). This shows that having a pair of inducers aligned enhances an illusion, thus providing support for interpolation mechanisms. There is also an additional enhancement of the illusion when the whole figure is aligned, thus providing support for figural mechanisms. Three subjects reported seeing no contour in the static all-in-phase condition, on one or two trials. These reports may be a result of adaptation to the illusory contours and are included in the data. 

Figure 8

 

Effects of relative inducer phase on illusion strength.

These results suggest that there is a figural component of the formation of illusory contours in the chomping pacman illusion. Some subjects reported seeing a waving figure when all inducers are out-of-phase, with much weaker contours. This effect has been observed previously by generating stereoscopic contours (Carman & Welch, 1992; Liinasuo, Kojo, Hakkinen, & Rovamo, 2000). When the illusion does not form a complete figure, the clarity of the remaining contours is reduced. However, there is no difference between 2 and 3 inducers in alignment. 

The finding of a figural completion mechanism is in agreement with previous results. Liu, Jacobs, and Basri (1999) found that the overall shape of an amodally completed figure influenced perceptual completion; when the shape of the completing curve is of equal energy, figures with more relatable figure interiors are completed more readily. Similarly, Fulvio and Singh (2006) found that when the edge geometry was the same, convex completions were less angular than concave completions, showing the overall shape of the completed figure influences the curvature of completion. 

Recent neuroimaging results have also suggested complete figures influence the perception of illusory contours. Lateral occipital cortex (LOC), a human brain region responsible for shape processing (Kourtzi & Kanwisher, 2001), has been found to be responsive to illusory contours (Mendola et al., 1999). A number of studies have proposed that LOC sends completion information back to primary cortical visual areas (Halgren, Mendola, Chong, & Dale, 2003; Murray, Foxe, Javitt, & Foxe, 2004; Stanley & Rubin, 2003). LOC responds similarly regardless of boundary completion information (Stanley & Rubin, 2003), which suggests that LOC is responding a figural percept (Stanley & Rubin use the term ‘salient region’). This activity in LOC seems to be temporally distinct from activity in early visual cortex (Halgren et al., 2003; Murray, Imber, Javitt, & Foxe, 2006), although there still exists considerable debate as to the timing relationship of cortical visual areas in the completion process (Halgren et al., 2003; Kaiser, Bühler, & Lutzenberger, 2004; Lee & Nguyen, 2001; Murray et al., 2002, 2004, 2006). Nonetheless, an independent figure completion mechanism is well supported by current neuroimaging of LOC. 

Experiment 5

Experiments 5 and 6 investigate competitive interactions between illusory contour mechanisms and luminance-defined contour mechanisms. These studies employ two different forms of occlusion cues. In Experiment 5, occlusion cues are employed to disrupt local junction structure. Typically, occlusion cues such as a ring around the inducers lead to a percept of no contours (Davis & Driver, 1994). Many theories of illusory contours propose that a requirement for the formation of illusory contours from a Kanizsa figure is a complete L-junction or a ‘degenerate’ T-junction (Anderson, 1997; Rubin, 2001). These results suggest that disruption of local junction structures should abolish modal illusory contour percepts. Alternatively, the multiple mechanism hypothesis suggests that occlusion may reduce the contribution of one mechanism but that remaining mechanisms could still support an illusory contour. To test this hypothesis, a varying thickness ring was inserted into the gap of Kanizsa diamond. 

Method

Four experienced psychophysical observers (1 female, all right-handed, 21–28 years old) participated in the experiment. All subjects had normal or corrected-to-normal vision. One author (MH) served as a subject. All four subjects had previously participated in the study. Stimuli were the same as the Kanizsa condition of Experiment 2, except a ring of varying thickness was placed in the opening of the inducers. The three thickness values of the ring were 12 arcmin thin, 25%, and 50% of the inducer radius. A fourth condition had no ring. Movie 4 shows the three different ring thicknesses. The procedure was the same as Experiment 2. Subjects performed seven repetitions of four ring types (no ring, thin, 25% or 50%) and two motion types (moving or static) for a total of 56 trials. 

 

Movie 4

 

Three different ring sizes. From left to right: thin ring, 25% ring, 50% ring.

Results and discussion

Figure 9 shows the results of Experiment 5. A within-subject ANOVA revealed significant main effects of both motion ( F(1,3) = 26.32, p < 0.05) and ring thickness ( F(3,9) = 62.99, p < 0.05). All ring types except the 50% ring showed significant differences between static and moving conditions in paired t-tests (no ring t(3) = 6.33, p < 0.05; thin ring t(3) = −4.02, p < 0.05; 25% ring t(3) = 4.25, p < 0.05; 50% ring t(3) = 2.1, p = 0.12). The moving thin ring illusion strength was not different than a static Kanizsa diamond with no ring ( t(3) = 2.01, p = 0.13), showing that the motion cue can override the occlusion cue to create a modal contour. The modal contours observed with the 25% rings are particularly noteworthy; junctional structure is disrupted on multiple spatial scales, yet the illusory contour is almost as vivid as the static Kanizsa display without occlusion of the junctions. 

Figure 9

 

Effect of occluding ring thickness on illusion strength.

These results suggest that a motion as a cue can cause modal completion, even when occluding lines disrupt local junction information. This result is contrary to previous reports that removing local junction information disrupts contour formation (Davis & Driver, 1994; Rubin, 2001), even in cases including motion (Rubin, 2001). Similar static displays are often described as amodally-completing (Albert, 2007; Lee & Nguyen, 2001; Ringach & Shapley, 1996). As in Experiment 3, observers report that modal completion transfers to the static conditions after viewing motion trials. Although the occluding ring provides contradictory information to modal completion of an illusory contour, a modal contour is still formed. 

Some authors have suggested that the processes of amodal completion are distinct from the processes of modal completion (Anderson, 2007; Anderson, Singh, & Fleming, 2002; Ringach & Shapley, 1996), while others have proposed the two share the same underlying mechanism, also called the “identity hypothesis” (Albert, 2007; Kellman et al., 2007; Shipley & Kellman, 1992a). Our results suggest that there may be underlying commonality in some displays thought to be amodal. The degradation of contour clarity by the ring in our display indicates that in some cases the only difference between amodal and modal completion is salience. This salience difference may be reflected in neural responses to modal and amodal contours. For example, Lee and Nguyen (2001) found decreased responses to amodal contours, however, response time courses remained similar. However, the finding of similarity between amodal and modal completion does not mean they are the same process (Anderson, 2007; Anderson et al., 2002). 

Experiment 6

Experiment 5 suggested that modal completion still occurs in the presence of a ring occluder. However, other types of occlusion, such as the addition of lines across the illusory contours, have been shown to reduce and/or prevent the formation of illusory contours (Gurnsey, Poirier, & Gascon, 1996). One theory of illusory contour formation suggests that perceptual scission separates the illusory figure from the inducing objects (Anderson, 1997; Singh & Anderson, 2002; Watanabe & Cavanagh, 1993). To test this hypothesis, we added a single cross centered across the figure, which should disrupt the formation of contours (Gurnsey et al., 1996). However, just adding a static occluder separate from the CP display inducers will result in depth stratification, where the occluder is perceived as ‘above’ the illusory contours (Bressan, Ganis, & Vallortigara, 1993; McDermott & Adelson, 2004). In order to overcome this perceptual depth stratification, a stereo display was used to force the cross to different depths, including behind the illusory surface. 

Methods

Five experienced psychophysical observers (3 female, all right-handed, 25–28 years old) participated in the experiment. All subjects had normal or corrected-to-normal vision. One author (MH) was a subject. All five subjects were previously familiar with the display and the measurement technique. 

Apparatus was the same as Experiment 1, except subjects viewed the screen through LCD shutter goggles (Stereographics Corp, San Rafael, CA). 

Stimuli were the same as the Kanizsa condition of Experiment 2, except a blue ring of 33% thickness, was added to the inside of figure, and a cross was added to the center of the figure. At 120 Hz refresh rate of the monitor (60 Hz to each eye), green and blue phosphors were still visible to the opposite eye, so the occluding cross was drawn in red. To enhance the contour percept, blue rings of a width 33% of the inducer size were added to the Kanizsa figure (magenta could not be used due to possible interactions between the red cross and magenta neon color spreading). Movie 5 demonstrates a cross in the same depth plane as the inducers. Subjects adjusted a Kanizsa figure with a blue ring 33% of the inducer size. 

 

Movie 5

 

Cross occlusion stimulus.

The procedure was the same as Experiment 2. Because the cross occluded the location of the illusory contour, subjects were instructed to match the illusory contours near to the cross rather than next to the inducers. Subjects performed seven repetitions of four cross conditions (no cross, in front, in plane, behind) and two motion types (moving or static) for a total of 56 trials. 

Results and discussion

Figure 10 shows the results of Experiment 6. A within-subjects ANOVA revealed significant main effects of motion ( F(1,4) = 13.62, p < 0.05) and depth of cross ( F(3,12) = 9.48, p < 0.05). Overall, it seems that a cross in front of or in the same plane as the illusory contour display has little effect on contour strength. A paired t-test between moving inducers without a cross and moving inducers with a cross in front reveals no significant difference ( t(9) = 1.99, p = 0.076). However, a cross behind the inducers causes a significant reduction of the effect, compared to a cross in front ( t(9) = 3.27, p < 0.05) or a cross in the same plane as the inducers ( t(9) = 2.97, p < 0.05). When the cross is behind the inducers the strength of the moving illusion is not statistically different than a static Kanizsa diamond in a paired t-test ( t(4) = 0.08, p = 0.93). Thus, the most disruptive case, the occluding cross behind the inducers still does not effectively kill the illusion, as it does in static form. This result is inconsistent with a strict scission account of illusory contour perception. 

Figure 10

 

Effect of occluding cross at different depths.

Just as observed in the occlusion conditions of Experiment 5, adding a cross occluder does not stop modal illusory contour completion in a moving illusion. However, the illusion is dramatically weaker when the “occluding” cross lies behind the illusory surface than when it is in front of the illusory surface, suggesting a surface/depth mechanism is partially responsible for the formation of modal illusory contours in the CP illusion. Surface mechanisms appear to play an important role, since the addition of a cross that is perceived to lie in front of the illusory surface does not change the clarity of illusory contours, despite local occlusion cues. The surface process that is impaired must operate at a level that has access to binocular information, as the monocular stimulation between cross in front and cross behind is the same. 

General discussion

There are a plethora of displays that produce illusory contours, and almost as many possible explanations of their formation (Lesher, 1995; Petry & Meyer, 1987; Seghier & Vuilleumier, 2006; Spillmann & Dresp, 1995). However, by using one extremely vivid display, it is possible to reveal multiple mechanisms of illusory contour formation within the same figure. These three mechanisms work in concert to form illusory contours, even if one mechanism is unable to create a contour. We performed 3 sets of experiments to demonstrate these multiple mechanisms. By varying the angular speed, size of inducers, and size of the chomping pacman illusion, it is possible to observe a similar support ratio effect that has been observed in static displays (Banton & Levi, 1992; Shipley & Kellman, 1992b). Motion appears to have an additive effect, as varying the inducer type yields similar increases in contour clarity between conditions. A single static inducer does not typically induce an illusory contour percept; however, a single moving inducer is sufficient to support the percept of a contour shooting outward from the inducer. This single inducer reveals the formation of a weak illusory contour that cannot be explained by interpolation, or linking mechanisms. Changing the temporal phase relationship of inducers suggests evidence for a figural mechanism that enhances the illusory contours, on top of extrapolation or interpolation effects. Adding occlusion cues such as a ring or cross to the illusion reduces contour clarity, possibly by impairing one or more mechanisms of contour formation. 

Our data are consistent with the hypothesis that three mechanisms, extrapolation, interpolation, and figural processing, work in concert to form illusory contours. While prior results indicate that a single mechanism alone can provide an account of how some contours are formed (Albert & Hoffman, 2000; Field et al., 1993; Gregory, 1972; Grossberg & Mingolla, 1985; Kellman & Shipley, 1991; Pillow & Rubin, 2002; Rock & Anson, 1979; Shipley & Kellman, 2003; Singh & Fulvio, 2005), we argue that all three mechanisms have the potential to contribute to the perception of illusory and real contours. 

A key contribution of the present experiments is their exploration of interpolation, extrapolation and figural mechanisms in closely related display variations. Our results make evident that, at a phenomenological level, each of these factors can play an important and even determinative role in the percepts. We suggest that each of these factors is involved to some extent in every illusory contour percept and perhaps most natural contour percepts as well. Published theoretical models of illusory contour formation have, on the other hand, have tended to focus their explanatory emphasis on mechanisms that primarily address one or two of these factors, rather than the interaction of all three. 

Our results are consistent with some aspects of previously presented models. Interpolation is a key component of the BCS/FCS model that implements a long range completion mechanism (Grossberg & Mingolla, 1985). Later models by Grossberg and colleagues include binocular matching mechanisms, possibly residing in macaque V4, that can perform figural feedback to early visual areas to enhance contours already formed (Kelly & Grossberg, 2000; Raizada & Grossberg, 2003). Some models have implemented features consistent with extrapolation mechanisms (Field et al., 1993; Li, 1998; Singh & Fulvio, 2005; Williams & Jacobs, 1997). Other models incorporate structures that permit extrapolation that persists when supported by interpolation (Grossberg et al., 1997; Raizada & Grossberg, 2003; Ross et al., 2000). Figural and surface mechanisms are a common feature to models that produce visual segmentation (Fantoni, Bertamini, & Gerbino, 2005; Grossberg et al., 1997; Grossberg & Swaminathan, 2004; Grossberg & Yazdanbakhsh, 2005; Kelly & Grossberg, 2000; Neumann & Sepp, 1999; Raizada & Grossberg, 2003) and such models can also support illusory contour formation. 

It is important to point out that some theories are largely orthogonal to this three mechanism approach. For example, a Bayesian approach that focused on cue combination might also account for many of the present results; however, such an approach does little to inform us about the underlying neural mechanisms. A different view comes from Anderson who suggests that illusory contours arise as a result of surface stratification (Anderson, 1997). Notably, Anderson and colleagues (Anderson & Barth, 1999; Anderson & Sinha, 1997) have reported illusory contours produced by moving inducers and have interpreted the results in terms of accretion and deletion of surface regions. This general idea has application in possible explanations of the CP display; however, the present results are distinguished in several ways. A primary finding of Anderson and Barth (1999) was that the perceived shape of illusory contours is determined by the velocity of contour terminations and the direction of motion of a partially occluded figure. These findings do not hold for the CP display (see Movie 1). While the perceived shape of the illusory contour changes with the motion of the contour terminations, shape at any moment in time appears independent of contour termination velocity and independent of the direction of motion of the chomping pacmen. Anderson and Sinha (1997) reported that a critical property for illusory contour perception in their displays was the motion of contour terminations ‘along’ the orientation of the illusory contour. That is, the contour terminations trace out the illusory contours in space-time. In the CP display, the contour terminations (the outer corner points of the pacman) move “orthogonal” to the illusory contour. Moreover, in Experiment 5, we found that the CP display produced a modal contour even when the circumference contour of the pacman did not terminate. A purely junctional account would have predicted either an amodal contour or no contour. Anderson has also argued that perceptual scission process play a central role in illusory contour formation (Anderson, 1997). In Experiment 6, modal illusory contours are perceived to lie in front of occluding contours. This is an apparently impossible scission configuration. We interpret our findings as supporting the notion that accretion, deletion, scission, and junctional accounts are important but are insufficient to fully explain the present and other illusory contour phenomena. 

These results help to resolve a discrepancy in the neurophysiology literature between bottom-up processing (Ffytche & Zeki, 1996; Grosof et al., 1993; Hirsch et al., 1995; Lee & Nguyen, 2001; Ramsden et al., 2001; Seghier et al., 2000; Sheth et al., 1996; von der Heydt et al., 1984) and top-down processing (De Weerd, Desimone, & Ungerleider, 1996; Mendola et al., 1999; Stanley & Rubin, 2003). Interpolation and extrapolation could be performed by neurons in early visual areas V1 and V2 (Grossberg & Mingolla, 1985; Pillow & Rubin, 2002), and figural processing could be performed by later stages by regions such as LOC and macaque V4 (Mendola et al., 1999; Stanley & Rubin, 2003), or other extrastriate areas (Seghier et al., 2000). Psychophysically, these mechanisms seem to interact to produce stronger illusions; this is consistent with observations of feed-forward and feedback activity within and between V1, V2, and LOC (Halgren et al., 2003; Lee & Nguyen, 2001; Murray et al., 2002, 2004). 

In summary, we have developed an especially robust illusory contour display that utilizes moving inducers. We have employed this display to probe the mechanisms of illusory contour formation. Our results provide support for three mechanisms of illusory contour formation: contour extrapolation, contour interpolation, and figural feedback. We suggest that these three mechanisms work in concert to support most traditional illusory contour percepts and that they also support the percept of real contours in complex natural scenes. 

Acknowledgments

We thank Bart Anderson, Arash Yazdanbakhsh, Jascha Swisher, Stephanie McMains, Summer Sheremata, Katherine Bettencourt, and Susan Mosher for helpful discussions and comments. This research was supported by National Science Foundation BCS-0236737 and BCS-0726061 to DCS. 

Commercial relationships: none. 

Corresponding author: Mark A. Halko. 

Email: [e-mail]. 

Address: Center for Memory and Brain, Room 109, 2 Cummington Street, Boston, MA 02215.