The flight path of the phoenix—The visible trace of invisible elements in human vision

Abstract

How features are attributed to objects is one of the most puzzling issues in the neurosciences. A deeply entrenched view is that features are perceived at the locations where they are presented. Here, we show that features in motion displays can be systematically attributed from one location to another although the elements possessing the features are invisible. Furthermore, features can be integrated across locations. Feature mislocalizations are usually treated as errors and limits of the visual system. On the contrary, we show that the nonretinotopic feature attributions, reported herein, follow rules of grouping precisely suggesting that they reflect a fundamental computational strategy and not errors of visual processing.

Introduction

Objects consist of features such as shape, color, and texture. The visual mechanisms that analyze features have been extensively studied. It is well known that the early visual system contains neurons that are tuned to analyze specific stimulus features such as shape or color (Livingstone & Hubel, 1988). According to a deeply entrenched view, this analysis is spatially localized and consequently features are perceived at the locations they are presented. Although there have been sporadic cases of illusions where features appear spatially mislocalized or misattributed, these cases were interpreted to reflect limitations or errors of the visual system (Arnold, Clifford, & Wenderoth, 2001; Baldassi & Burr, 2000; Bedell, Chung, Ogmen, & Patel, 2003; Butler, Mewhort, & Browse, 1991; Enns, 2002; Herzog & Koch, 2001; Nijhawan, 1997; Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Treisman & Schmidt, 1982; Werner, 1935; Wilson & Johnson, 1985; Zeki, 2001). 

For example, when observers are presented with a display containing a red “X” and a green “O,” in a small number of trials, observers may report seeing a green “X.” Typically, such illusory feature conjunctions occur when observers' attention is diverted (Treisman & Schmidt, 1982). Consequently, this illusion has been interpreted to reflect an error resulting from the limited attentional resources of the observer. 

In contrast, we present a new visual illusion showing a systematic relationship between feature mislocalizations in the focus of attention and grouping. A typical metacontrast display (Bachman, 1994; Breitmeyer & Ogmen, 2006; Stigler, 1910) is shown in Figure 1A where a central target line is followed by two flanking lines. With an appropriate choice of spatiotemporal parameters, the central line can be rendered invisible. We used an extension of the classical metacontrast paradigm by presenting a central target line followed by a sequence of flanking lines (Figure 1B; Piéron, 1935). Two streams of lines were perceived as expanding from the center to the left and right whereas the central line did not register in perceptual awareness (for an animation, see Movie 1). If we insert, as a feature, a spatial vernier offset to the central line, this offset can be perceived in the stream of straight lines. Our analysis of whether and how different features (vernier offsets) in motion streams are integrated indicates that the feature “mislocalizations,” we report herein, reflect not erroneous but systematic processing. Furthermore, our results indicate that grouping operations can access and process individual features prior to a feature integration stage. 

Figure 1

 

(A) Classical metacontrast. A central line is followed by two, nonoverlapping flanking lines. The central line is rendered largely invisible if the flanks appear 50 ms later. (B) Sequential metacontrast. The central line is followed by four successive pairs of flanking lines. A percept of two streams of lines expanding to the left and right is elicited with the central line being invisible. (C) Identical sequence as in panel B with the only exception that the central line is not presented (for an animation of panels B and C, see Movie 1).

 

Movie 1

 

The animation shows the stimuli of Figures 1B and C. Note that the animation strongly depends on the resolution and timing of your monitor and hence may differ from the actual stimuli presented on the X–Y displays with high spatiotemporal resolution that we are using in our laboratory. Please seat yourself approximately two meters away from your monitor. In some trials, there is a central line presented ( Figure 1B) whereas in others it is not ( Figure 1C). Your task is to determine whether the central line is present or not. In this demonstration, all lines are aligned, that is, no vernier offset is introduced. To see the animated sequence frame by frame, press “pause” and use the arrow keys.

Methods

Observers

Data were obtained from one of the authors (T.O.) and paid, naive observers. The general purpose of the experiment and possible consequences of the studies were explained to each observer. Moreover, subjects were told that they could quit the experiment at any time they wished. After observers signed a consent form, acuity was determined by means of the Freiburg visual acuity test (Bach, 1996). To participate in the experiments, subjects had to reach a value of 1.0 (corresponding to 20/20) at least for one eye. The experiments were undertaken with the permission of the local ethics committee. 

Apparatus

Stimuli appeared on an X–Y display (HP-1332A, Tektronix 608) controlled by a PC via fast 16-bit D/A converters. Stimuli were composed of dots drawn with a dot pitch of 250–350 μm at a dot rate of 1 MHz. The dot pitch was selected so that dots slightly overlapped; that is, the dot size (or line width) was of the same magnitude as the dot pitch. Stimuli were refreshed at 200 Hz; that is, a stimulus duration of 20 ms was realized by four refreshes. Luminance of the stimuli, measured by means of a dot grid (same dot pitch and refresh rate as above) with a Minolta LS-100 luminance meter, was 80 cd/m ². The room was dimly illuminated (0.5 lx) and background luminance on the screen was below 1 cd/m ². Viewing distance was 2 m. 

Procedure

A central line was followed by four pairs of flanking lines (see Figure 1B). The central line consisted of two segments of 600″ (arcsec) length separated by a vertical gap of 60″. The length of the flanking lines was 700″ for the lines next to the central line and increased progressively by 100″ for the following flanking lines. The distance between two subsequent lines was 200″. All lines were presented for a duration of 20 ms each. The stimulus onset asynchrony (SOA) between the central line and the first pair of flanking lines was 50 ms and the SOA between subsequent flanking lines was 40 ms. Hence, the duration of the whole sequence was 190 ms. 

In a detection task, we presented two stimulus sequences in two consecutive temporal intervals separated by half a second in random order. One sequence contained a central target line ( Figure 1B) whereas the other sequence did not ( Figure 1C). We asked eight observers to indicate which interval contained the central line by pressing one of two buttons (two interval forced-choice task). 

In the next experiment, we inserted a spatial (vernier) offset to the central line; that is, the two line segments were slightly displaced in the horizontal direction randomly either to the left or right (“target-offset,” see for example Figure 2A). Only one temporal interval with one sequence was presented. At the beginning of a block of trials, we asked observers to attend to one motion stream, for example, the lines shifting to the left, and to indicate the perceived offset direction (binary forced-choice task). To achieve comparable performance levels across observers, offset sizes of the central line were determined individually for each observer before the experiments took place. Using an adaptive method, we determined the offset size yielding a performance level around 75–80% correct responses. Individual offset sizes ranged from 45″ to 100″ (for details, see Table 1). In some conditions, we inserted an additional offset to one of the flanking lines. The direction of this additional offset was always chosen to be opposite to the target-offset (“anti-offset,” see for example Figure 2B). The sizes for the anti-offset were determined individually, as well, and ranged from 30″ to 70″ ( Table 1). We recorded the percentages of responses in accordance with the target-offset. 

Table 1

 

Individual offset sizes (arcsec) as used in the experiments. Note that offset discrimination thresholds for an unmasked vernier are by a factor of 5–10 smaller.

Experiment	Mean offset	Individual offset
Figure 2		AH	FF	GT	ID	JU	NI	OR	TE
Target-offset	77.5	80	70	80	80	80	60	90	80
Anti-offset	53.8	50	30	60	60	70	50	50	60
Figure 3		AH	FF	ID	MM	TO
Target-offset	76.0	90	70	80	70	70
Figure 4		AD	GR	ID	JU	MC	TC	TF	TO
Target-offset	80.0	90	60	90	80	70	100	70	80
Anti-offset	53.8	60	40	50	70	50	70	50	40
Figure 5		AC	AL	GR	GT	ID	JU	TO	TS
Target-offset	70.6	60	45	60	80	90	80	80	70
Anti-offset	56.3	50	45	40	60	60	70	60	65

Figure 2

 

Invisible element, visible offset. (A) The central line was randomly offset to the left or right followed by non-offset, flanking lines. Observers were asked to attend to one stream of lines, for example, the leftward stream (indicated by the arrow). The task was to discriminate the offset direction perceived in the stream. Responses were assessed with respect to their accordance with the target-offset. Although the central line was rendered invisible by sequential metacontrast, observers could discriminate the target-offsets very well. (B) Performance strongly changes by inserting an anti-offset to the penultimate line in the attended motion stream indicating a combination of the two offsets. The difference between panels A and B is significant (two-tailed, paired t test: p = 0.0006). (C) Performance, compared to panel A, is almost not affected by an anti-offset presented in the nonattended motion stream. In the schematics, offset elements are highlighted in black (in the experimental display, all elements had the same luminance).

In each experiment, each trial was initiated with four markers at the corners of the screen and a fixation dot in the center presented for 500 ms followed by a blank screen for 200 ms. Then, the stimulus sequence was presented. After stimulus presentation, a blank screen appeared and observers responded by pressing one of two buttons. No feedback was given. A new trial was initiated 500 ms after the observer gave a response. A block consisted of 80 trials. The order of conditions was randomized across observers to reduce the influence of hysteresis, learning, or fatigue effects in the averaged data. For each observer, each condition was measured twice (i.e., 160 trials per observer). After each condition had been measured once, the order of conditions was reversed for the second set of measurements. We computed means and standard errors of the mean ( SEM) across subjects. For statistical analysis, we computed two-tailed, paired t tests with α = .05. 

Results

Sequential metacontrast

In sequential metacontrast (see Figure 1B), observers fail to perceive the central line. To quantify this invisibility, we presented two stimulus sequences in random order in two consecutive temporal intervals. The only difference between the two sequences was that one sequence contained the central line ( Figure 1B) whereas the other sequence did not ( Figure 1C). We asked eight observers to indicate which interval contained the central line. Mean performance was around chance level (55.2%, SEM: 3.3; d′ = 0.27, SEM: 0.18); that is, observers can hardly, if at all, detect the interval containing the central line. 

Feature attribution in sequential metacontrast

If feature attributions were carried out in a spatially localized manner, rendering the central line invisible should also make all of its features invisible. To test this prediction, we inserted a spatial (vernier) offset to the central line; that is, the two line segments were slightly displaced in the horizontal direction randomly either to the left or right in each trial ( Figure 2A). Surprisingly, this offset was perceived in the stream of straight lines. To quantify this observation, we asked eight observers to attend to one stream of lines, for example, the leftward stream, and to indicate the perceived offset direction. We recorded the percentage of responses in accordance with the offset direction of the central line (“target-offset”). Results in Figure 2A show that the target-offset was well discriminated. Hence, the target-offset can be recreated in the stream after the extinction of the central line's visibility like a phoenix arising from its pyre. 

Next, we show that the offset of the central line is not only perceived in the motion stream but can also be integrated with features of other lines. For this, we inserted a second offset to the line displayed penultimate either in the attended ( Figure 2B) or unattended stream ( Figure 2C). The direction of this second offset, which we call an “anti-offset,” was always chosen to be opposite to the target-offset. If, for example, the central line was offset to the right, the penultimate flanking line was offset to the left and vice versa. 

When the anti-offset was in the attended stream, a strong change in performance occurred ( Figure 2B). Hence, features can be integrated in a fairly broad spatiotemporal window because target-offset and anti-offset were separated by 10 arcmin and 130 ms. 

The invisibility of the central line rules out the possibility that observers perceive the target-offset and the anti-offset separately. Hence, a cognitive decision strategy basing decisions on one of the elements is not at work here. Indeed, no observer reported seeing multiple offsets within one stream of lines. We also conducted a control experiment where we presented an offset either at the central line or at the penultimate flanking line randomly interleaved. In both cases, offset discrimination was around 75–80% (i.e., comparable to Figure 2A). However, observers can hardly, if at all, determine whether the central line or the penultimate flanking line was offset (mean of 5 observers: 53.6%, SEM: 1.1; d′ = 0.19, SEM: 0.06). 

Feature integration in sequential metacontrast was not confined to two offsets of lines but occurred when multiple lines in the attended stream were offset. Performance gradually changed with the number of offset lines as well as with the sizes of offsets ( Figure 3). These graded changes of performance show that feature integration occurs along the entire motion trajectory in a predictable manner. 

Figure 3

 

Integration of multiple offsets. (A) Identical condition as in Figure 2A. The offset central line was followed by non-offset, flanking lines. Observers were asked to attend to one stream of lines, for example, the leftward stream (indicated by the arrow). Performance is comparable to Figure 2A. (B–D) The influence of the anti-offset on performance increases with increases in the offset-sizes and the number of anti-offset elements in the attended stream. We tested conditions of 1, 2, and 3 anti-offsets with offset sizes of 10″, 15″, and 20″, respectively (note that sizes of anti-offsets in Figure 2B were larger than in this experiment).

When the anti-offset was presented in the unattended stream of lines, it had no significant effect on performance ( Figure 2C). This attention-specific effect shows that feature integration does not include indiscriminately all features presented in the display but is specific to those features that belong to the attended motion stream. As the next two experiments will show, feature integration is governed by rules of perceptual grouping. 

Grouping-based feature integration

After the display of the central line, we presented two “parallel streams of lines” shifting in the same direction ( Figure 4). Eight observers were instructed to attend either to the left or to the right stream and to indicate the perceived offset direction. As in the previous experiment, observers' judgments agreed strongly with the target-offset ( Figures 4A and B). We inserted an anti-offset to the second line of the right stream, that is, at the same spatial location as the central line but 90 ms later. A significant change in performance occurred when the right stream was attended ( Figure 4C) but not when the left stream was attended ( Figure 4D), although the physical stimuli were identical in both conditions. Hence, two offsets at the same spatial location are integrated only if they belong to the same stream. Eye movements are not likely to play a role in this effect because target-offset and anti-offset were separated by 90 ms and stimuli were confined within a narrow foveal region of 13.3 arcmin extent (Kalesnykas & Hallett, 1994; Wyman & Steinman, 1973). 

Figure 4

 

Grouping-based feature integration I. The central line was followed by two streams of lines shifting in parallel. Lines in both streams were shifted by 200″ per step to the left. (A) Observers were asked to attend to the left stream of lines. The offset of the central target line is discriminated correctly in about 80% of trials. (B) Similarly, the target-offset is well discriminated if subjects attend to the right stream. (C) Performance is changed by an anti-offset presented at the second line if observers attend to the right stream. It is important to highlight that the anti-offset line is presented at the same spatial position as the central line. (D) Performance, compared to panel A, is not significantly affected by this anti-offset when observers attend to the left stream. The difference between panels C and D is significant (two-tailed, paired t test: p = 0.0008). Stimuli in panels C and D are the same, only the focus of attention differs.

The previous experiment demonstrated that the target-offset can be perceived in the left as well as in the right stream depending on attention. We suggest that this result occurs because the central line is equidistant to the two subsequent flanking lines, and hence its grouping to either stream is ambiguous (Neuhaus, 1930; Ramachandran & Anstis, 1983). To provide additional evidence that feature integration follows rules of grouping, we added an additional line either to the right or left of the central line (Figures 5C and D, respectively). By adding the right line, we expected the target-offset to be attributed to the left stream only (Figure 5C). Hence, no integration of the target-offset, which is grouped to the left stream, and the anti-offset, which is grouped in the right stream, should occur. Consequently, if observers attend to the right stream, responses should be determined by the anti-offset predominantly. Results in Figure 5C confirmed this prediction because performance is clearly below 50% (the percentage of responses in accordance with the target-offset is slightly higher compared to a condition where the target-offset was replaced by a straight line, 25.9%, SEM: 2.8; this indicates that a small leakage occurs from the target-offset to the right, attended stream). On the other hand, by adding a line to the left of the central line rather than to the right, we expected no change in the grouping of the central line to the right stream (Figure 5D). Indeed, in this case, performance was comparable to Figure 5B, indicating an integration of target-offset and anti-offset. 

Figure 5

 

Grouping-based feature integration II. (A and B) Identical conditions and comparable results as in Figures 4B and C, respectively. (C) The central line is flanked by an additional line on the right side, 400″ apart. Performance is dominated by the anti-offset (performance is below 50% because we determine responses in accordance with the target-offset). We suggest that the additional line disambiguates the motion grouping, present in panels A and B, by assigning the “central” line with the target-offset to the left motion stream. Because only the anti-offset is present in the attended right stream, this anti-offset determines performance. The difference between panels B and C is significant (two-tailed, paired t test: p = 0.0003). (D) The additional line is presented to the left. Subjectively, a percept of two bending motion streams is elicited. Performance is comparable to panel B. We suggest that the additional line changes the motion percept (bend motion) but not the grouping of the “central” line to the right motion stream. Hence, target-offset and anti-offset are integrated. Note change of ordinate.

Discussion

Our results show grouping-based feature integration but they do not reveal the mechanisms underlying the grouping process. Although in Figure 5 a single line is sufficient to alter the grouping relations, more complex stimulus configurations will be used in future research to determine the spatial and temporal extents over which grouping relations operate. 

Under normal viewing conditions, motion and figural features of an object are perceived jointly, indicating interactions between the systems that process motion and form (Anderson & Sinha, 1997; Baloch & Grossberg, 1997; Kolers, 1972; Lorenceau & Alais, 2001; Wallach, 1935; Watanabe, 1997). The analysis of motion and features is particularly challenging when multiple objects are in motion because features of different objects can blend; that is, they can overlap in space and time (Figures 4 and 5). Errors in such conditions may be explained in terms of a limited processing capacity—as proposed in other paradigms of illusory misbindings and mislocalizations—such as a lack of attention (Treisman & Schmidt, 1982), erroneous feature migration (Butler et al., 1991; Herzog & Koch, 2001), feature misbinding in object substitution (Enns, 2002), crowding (Parkes et al., 2001), pooling (Baldassi & Burr, 2000; Parkes et al., 2001), unpredictability in motion extrapolation (Nijhawan, 1997), asynchrony in distributed microconsciousness (Zeki, 2001), and asynchronies in feature processing (Arnold et al., 2001; Bedell et al., 2003). Moreover, feature misattributions occur in classical metacontrast masking (Hofer, Walder, & Groner, 1989; Hogben & Di Lollo, 1984; Stewart & Purcell, 1970; Stoper & Banffy, 1977; Werner, 1935; Wilson & Johnson, 1985; for pattern masking, see also Herzog & Koch, 2001). 

In accordance with these interpretations, the integration of the target-offset and the anti-offset may be viewed as an error due to the fast rate of line presentations (Cai & Schlag, 2001; Nishida, Watanabe, & Kuriki, 2005). However, as the results in Figures 4D and 5C show, offsets are not integrated if they belong to different motion streams although these streams overlap in space. On the other hand, integration takes place for those offsets that belong to the same motion stream (Figure 4C and Figures 5B and D). Due to this selectivity of integration, we suggest that “feature misattributions” should not be viewed as an error of the visual system caused, for example, by the fast line presentations and a limited spatiotemporal resolution. On the contrary, we suggest that the human brain can analyze the individual offsets in one motion stream separately as it is the case when the corresponding lines do not belong to the same motion stream. Our findings indicate that grouping operations can access and process these individual features prior to an integration stage. Hence, the feature attribution that we show in this contribution may point, not to an error, but rather to a fundamental computational processing strategy of how the human brain attributes features to objects. 

These conclusions are supported by another recent finding using the Ternus–Pikler display in which three lines are presented in two successive frames (Ogmen, Otto, & Herzog, 2006). We could show that the vernier offsets in the first frame can be perceived at a line in the second frame although the line positions do not overlap (nonretinotopic feature attribution). We interpret our findings of nonretinotopic feature attribution not as an error but as a computational strategy of the brain to cope with the vast information present in motion streams. 

Previous research showed that interpolation in sampled motion streams can make a temporal offset appear as a spatial offset (Burr, 1979; Fahle & Poggio, 1981; Morgan, 1976). In contrast, in sequential metacontrast, the offset perceived in the stream of lines is caused by a physical spatial offset of the first line only, which itself is invisible. Moreover, the temporal integration windows seem to differ in the two paradigms. 

Interestingly, the binding of features in our paradigm occurs in the focus of attention rather than in its absence as in illusory conjunctions (Treisman & Schmidt, 1982; see also Herzog & Koch, 2001; Sharikadze, Fahle, & Herzog, 2005). Additional experiments are required to study the explicit role of attention and the level at which it operates. Answers to these questions will contribute to the discussion about the differences between attention and consciousness (Kiefer & Brendel, 2006; Lamme, 2003; Naccache, Blandin, & Dehaene, 2002). 

Acknowledgments

We thank M. Repnow for technical support and L. Overney, B. Breitmeyer, G. Francis, and I. Molnar-Szakacs for comments on the manuscript. This work is supported by the Swiss National Science Foundation, the National Institutes of Health (R01-MH49892), and the Hanse-Wissenschaftskolleg. 

Commercial relationships: none. 

Corresponding author: Thomas U. Otto. 

Email: [e-mail]. 

Address: EPFL SV INS LPSY, Station 15, 1015 Lausanne, Switzerland.