Crowding, feature integration, and two kinds of “attention”

Abstract

In a recent article, Pelli, Palomares, and Majaj (2004) suggested that feature binding is mediated by hard-wired integration fields instead of a spotlight of spatial attention (as assumed by Treisman & Gelade, 1980). Consequently, the correct conjoining of visual features can be guaranteed only when there are no other competing features within a circle with a radius of approximately 0.5E (E = eccentricity of the target object). This claim seems contradicted by an observation that we can easily see—for example, the orientation of a single blue bar within a dense array of randomly oriented red bars. In the present study, possible determinants of the extent of crowding (or feature integration) zones were analyzed with feature (color) singletons as targets. It was found that the number of distractors has a dramatic effect on crowding. With a few distractors, a normal crowding effect was observed. However, by increasing the number of distractors, the crowding effect was remarkably reduced. Similar results were observed when the target and distractors were of the same color and when only a differently colored circle indicated the target location. The results can be explained by bottom-up “attention” that facilitates the processing of information from salient locations in the visual field.

Introduction

The present study is concerned with the relations between crowding and feature integration mechanisms in vision. 

When a target stimulus is surrounded by other (irrelevant) stimuli, its perception is impaired. This is known as a crowding effect.¹ Often, this effect has been studied with letters or numerals as objects (Bouma, 1970; Chung, Levi, & Legge, 2001; Strasburger, Harvey, & Rentschler, 1991); however, it also occurs with more simple objects such as line segments or grating patches (Andriessen & Bouma, 1976; Wilkinson, Wilson, & Ellemberg, 1997). The spatial extent of crowding is approximately proportional to the eccentricity of the target stimulus and reaches 0.5E (E = eccentricity of the target object; Bouma, 1970; Toet & Levi, 1992). There have been two proposed explanations for crowding: inhibitory interactions between spatially adjacent mechanisms sensitive to similar visual features (e.g., Bjork & Murray, 1977) and spatial pooling of the responses of feature detectors at some higher level of processing (e.g., Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Wilkinson et al., 1997). Recent studies (Levi, Hariharan, & Klein, 2002; Pelli, Palomares, & Majaj, 2004) reported that crowding does not inhibit detection of simple visual features, which is consistent with pooling theories. 

According to the widely accepted view, spatial attention plays an important role in recognition of visual objects. It is assumed that attention is necessary for the integration of simple visual features to perceive coherent objects (Treisman & Gelade, 1980; Wolfe & Bennett, 1997). If there are many objects in a visual field, then it is necessary to switch attention serially from one to another to recognize them. 

Intriligator and Cavanagh (2001) indicated that there may be a close relationship between theories of attention and crowding. They argued that crowding is an effect of insufficient spatial resolution of attention that is limited by the large size of receptive fields at some level of visual processing after feature detection. 

Pelli et al. (2004) went a step further. They suggested that the binding of simple visual features into coherent objects is not mediated by a spotlight of attention (as assumed by Teisman & Gelade, 1980) but by fixed-size hard-wired integration fields. The same fields determine the extent of crowding. Although Pelli et al. cited studies about the effect of target–distractor similarity on the extent of crowding and admitted some selectivity of feature integrators, they seem to favor the simple idea that the extent of crowding is approximately a constant fraction of target eccentricity, regardless of the nature of visual objects. Consequently, visual features can be effectively conjoined only when there are no other competing features within an integration field that has a radius of approximately 0.5E. Otherwise, when several objects are located within an integration field, their features are compulsorily mixed together and crowding or illusory conjunctions are observed. 

This simple account was shown to be consistent with many crowding and illusory conjunctions studies. However, one can easily design stimuli that appear to contradict this model (see also Popple & Levi, 2005). While fixating the cross in Figure 1, you can probably observe that the blue bar is horizontally oriented. Consequently, the blue color and horizontal orientation are correctly conjoined, regardless of the many vertical bars within the radius of 0.5E. 

Figure 1

 

Fixate the cross at the right and try to see the orientation of the blue bar among red distractors. This is not very difficult regardless of the large number of randomly oriented bars within the hypothetical integration field (indicated by the black circle).

Why is this condition different from usual crowding studies? Are the stimuli too simple? Is it that the distractors located in a dense array inhibit each other? The following experiments try to answer these and other related questions. 

Experiment 1: Effect of the number of distractors

In this experiment, the number of distractors varied from a usual crowding configuration up to a large matrix like that shown in Figure 1. 

Methods

Stimuli consisted of horizontal and vertical bars with a width of 1.6 mm and a length of 2.5 mm (observers EP and LP) or 2.2 mm (observer KH). The target bar was always blue, whereas distractors were red. Orientations of all bars were random (either horizontal or vertical, with equal probability) and independent of each other. A target was presented at a fixed eccentricity, 47 mm either to the left or to the right of the fixation point. From a viewing distance of approximately 50 cm, this eccentricity corresponds to 5.3 deg of visual angle. Distractors (when present) formed a square matrix comprising 3 × 3, 5 × 5, 7 × 7, 9 × 9, or 13 × 13 objects with the target in the center. Distances (center to center) between rows and columns of objects were 4.7 mm. Actual positions of objects were jittered randomly within a range of ±0.6 mm, both horizontally and vertically. Stimuli were presented on a background of light gray (40 cd/m²); a dark cross indicating the fixation point was permanently present. Examples of stimuli for Experiment 1 are given in Figure 2A and B. 

Figure 2

 

Examples of stimuli used in the present study. Horizontal–vertical bars used in Experiment 1: with (a) 8 and (b) 80 distractors. (c) Letter-like stimuli used in Experiment 2. (d) Bar stimuli with colored circles used in Experiment 3. The stimuli were presented unpredictably either to the left or to the right of the fixation point (eccentricity = ∼5.3 deg).

A matrix of objects was exposed briefly (60 ms) either to the left or to the right of the fixation point. The observer's task was to identify the orientation (horizontal or vertical) of the target (a blue bar at the center of the matrix). Matrix size was held constant within blocks of 50 trials. A control condition with a target alone was run as well. 

Three observers with normal or corrected-to-normal vision took part in the experiment. Each of them completed 600 trials (100 trials per matrix size). 

Results and discussion

Percentage correct as a function of the number of distractors is given in Figure 3A. Without distractors, performance was nearly perfect. A single ring of distractors causes a normal crowding effect—percentage correct drops to near-chance level. However, by increasing the number of distractors, the crowding effect is dramatically reduced. With three to four rings of distractors, performance is recovered to 80–95% correct. These results suggest that additional distractors must inhibit the crowding effect of the distractors adjacent to the target. 

Figure 3

 

Results of the experiments of the present study: (a) Experiment 1, horizontal–vertical bars of different colors; (b) Experiment 2, letter-like shapes; and (c) Experiment 3, horizontal–vertical bars of the same color. Different lines represent the results for different observers. Error bars indicate the standard error of the observed percentages.

The stimuli in Experiment 1 were very simple and different from those in typical crowding studies, where mostly letters were used. It has been argued that actual crowding cannot be observed with simple (horizontal vs. vertical) orientation discrimination (Pelli et al., 2004; although color cuing here turns the task into conjunction detection that complicates the interpretation). Also, it is possible that color-orientation conjunctions are special because they can be detected by low-level mechanisms selective along both of these dimensions (e.g., Holcombe & Cavanagh, 2001; VanRullen, Reddy, & Koch, 2004). To check whether the adverse interactions studied here follow the usual regularities of crowding, one observer (EP) ran a supplementary experiment. The stimulus display consisted of a target and eight distractors. Percentage correct as dependent on the target–distractor distance was measured for three target eccentricities. The data were fitted with cumulative Gaussian curves, and thresholds of target–distractor distance corresponding to 75% correct responses were estimated for each eccentricity. The thresholds (with 95% confidence intervals in parentheses) were 0.52 (0.34–0.64), 1.2 (0.7–1.5), and 2.4 (1.7–3.0) deg for eccentricities 2.7, 5.3, and 8.0 deg, respectively. These values are reasonably close to being proportional with eccentricity, and 75% thresholds of approximately 0.2–0.3E are consistent with the extent of interaction up to 0.5E. 

Experiment 2: More complex stimuli

Experiment 2 explored the generality of the finding from the first experiment using more complex stimuli, similar to those in usual crowding studies. In this experiment, observers had to discriminate between the letters X and O. 

Methods

The methods in Experiment 2 were identical to those of the first experiment except for the stimuli. The stimulus objects were letter-like shapes, either X or O (randomly selected). Both letters consisted of diagonally oriented strokes with a width of 1 mm. Their size was 3 × 3 mm, and they had an approximately equal number of pixels with foreground color (see Figure 2C for a stimulus display). The observer's task was to decide whether the target (central and blue object) was an X or an O. Two observers with normal or corrected-to-normal vision participated in the experiment, each completing 600 trials. 

Results and discussion

The results are given in Figure 3B. Regardless of different levels of performance, the curves for both observers are similar to each other and to those of the first experiment. Consequently, the reduction of the crowding effect with an increasing number of distractors is not a peculiarity of the very simple stimuli used in Experiment 1. Instead, it seems to be observable with different objects and tasks. (In pilot experiments, similar results were observed with other letter-like shapes and with the roles of color and orientation reversed relative to Experiment 1.) 

Experiment 3: Possible mechanisms

Many studies have shown that responses of the neurons in the visual cortex can be suppressed by the presence of similar stimuli in adjacent locations outside the classical receptive field (e.g., Blakemore & Tobin, 1972; Cavanaugh, Bair, & Movshon, 2002; Li & Li, 1993). This surround suppression has been considered a neural basis for visual segmentation and pop-out (e.g., Knierim & van Essen, 1992; Li, 2000). However, the idea that additional distractors suppress the responses to the distractors adjacent to the target and thereby unmask the target in the present study may have been realized in several ways. 

One line of explanation can be based on direct inhibitory interactions between mechanisms that are located in adjacent positions and sensitive to similar visual features. This model may also include mechanisms sensitive to conjunctions of simple features. Assume that there are mechanisms tuned to vertical blue, horizontal blue, vertical red, and horizontal red bars. Further, assume that the responses of the mechanisms are inhibited by the activity of adjacent mechanisms with similar tuning in either dimension. As a consequence, red distractors are more heavily suppressed than the blue target by additional red items; actually, the response to the target can be disinhibited. Because the same (conjunction) units carry orientation signals, the orientation of the target may be better identified. 

Another model, which does not need conjunction detectors, assumes that a general measure of salience is calculated for each location, summing up all feature differences between a given location and its neighborhood (Itti & Koch, 2000; Koch & Ullman, 1985; Nothdurft, 2000). In these models, exogenously controlled attention is directed to locations of salience maxima. However, one may suggest that “attention” is redundant here, as salience can be used directly to modify (e.g., multiply) signals from each location. 

A reduced crowding effect with feature singletons can also be explained by a nonspatial feature-based selection (as suggested by Intriligator & Cavanagh, 2001). For example, prior knowledge that the target stimulus is blue and distractors are red can be used to facilitate signals from blue objects and/or inhibit those from red objects. However, there seems to be no simple explanation why such a feature-based selection should be more effective with a larger number of distractors as observed in the present study. 

The third experiment was designed to probe possible mechanisms of the effect observed in the previous experiments. It was attempted to exclude the effect of interaction between hypothetical mechanisms tuned to both color and orientation. In this experiment, the colored and oriented parts of the stimuli were separated. Both the target and distractors were black bars; only a circle with a unique color indicated the target location. 

Methods

The methods of Experiment 3 were similar to those of the first experiment, with the following exceptions: In the present experiment, both the target and distractor bars were black. There were circles with a diameter of 3 mm around the bars: blue around the target and red around distractors. The aspect ratio of the bars was increased (the size of bars was 3 × 1 mm in this experiment) to compensate the masking effect of the circles, and the positional jitter of objects was decreased from 0.6 to 0.3 mm (to avoid accidental overlapping of the circles). An example of the stimulus display is shown in Figure 2D. Two observers participated in the experiment, each completing 600 trials. 

Results and discussion

The results are given in Figure 3C. It seems that a differently colored circle did approximately the same job as a different color of the target bar itself. It would be hard to explain these results with selective inhibition between simple mechanisms tuned to both color and orientation (although we cannot exclude the possibility of some more complex conjunction mechanisms). The results are consistent with a salience-based mechanism that facilitates the processing of all visual features at locations that are clearly different from the surrounding background by any feature. 

Also, the results provide further evidence against direct feature-based selection (no simple color-based mechanism could select a black bar surrounded by a blue circle among identical black bars surrounded by red circles). 

General discussion

The experiments reported here demonstrate that with feature singleton targets, the crowding effect decreases dramatically with an increasing number of distractors for both simple orientation and more complex letter identification tasks. A singleton feature can be easily conjoined with other features present in the location of the singleton, regardless of competing features in proximity. (Note that with a target that is not a salient feature singleton, the increasing number of distractors worsens rather than improves the perception of the target; Felisberti, Solomon, & Morgan, 2005.) Experiment 3 showed that the observed effect is based on bottom-up salience of target location or exogenous attention rather than a direct interaction between mechanisms tuned to simple features and feature conjunctions. 

At first glance, the present results seem to contradict the view that the crowding effect is a consequence of (anatomically determined) large sizes of integrative receptive fields because the size of the integration field appears to be modulated dramatically by the number of distractors. However, it is not difficult to reconcile the integration field model with the present results. The salience-based facilitation of the target (or inhibition of distractors) may well occur at earlier levels of visual processing (e.g., V1), and the levels with large receptive fields (e.g., V4) may see already a modulated image. Thus, the psychophysically observed feature integration spotlight may be a combination (e.g., multiplication) of two spatial profiles of receptive fields at different levels of visual processing. This scheme is supported by some evidence that bottom-up (exogenously controlled) attention has a higher spatial resolution as compared with top-down (endogenously controlled) attention (Gobell, Tseng, & Sperling, 2004) and by the finding of Briand and Klein (1987) that only exogenous attention is effective in feature binding when there are distracting objects adjacent to the target. Recently, Popple, Petrov, and Levi (2006) proposed quite a similar two-mechanism account for the distribution of pop-out localization errors. Furthermore, in a recent article, Strasburger (2005) discussed a combination of different attentional mechanisms for the explanation of some properties of crowding. 

According to the present hypothesis, crowding occurs when there are salient distractors (or, perhaps, salient edges of an array of distractors) within an integration field centered on the target. When the salience of distractors close to the target is suppressed by additional distractors (or edges of the array of objects are moved out of the integration field), the target becomes a single salient object within the integration field and crowding effect disappears. 

Target pop-out can reduce crowding effect also in normal crowding displays with a few distractors (Felisberti et al., 2005; Kooi, Toet, Tripathy, & Levi, 1994), although large differences across visual features and observers have been found. There is some evidence that pop-out according to the parallel search criterion (no effect of the set size on performance) is not sufficient for the reduction of the crowding effect (Felisberti et al., 2005). This can be explained by the idea that there is a sort of competition between salient stimuli, and, to reduce the crowding effect, the salience of the target must outperform that of the distractors. 

If something like exogenously controlled attention plays an important role in determining the extent of crowding, then directing of attention to the target location by a peripheral precue also should reduce crowding. Indeed, some studies have reported precuing effects on crowding (Huckauf & Heller, 2002; Morgan, Ward, & Castet, 1998; Strasburger, 2005), whereas others (Nazir, 1992; Wilkinson et al., 1997) have found crowding to be relatively immune to this type of manipulation of attention. It is possible that spatial precuing can sufficiently raise the salience of a target in limited conditions only. However, these controversial results deserve further analysis. 

Nonmonotonic effects of the number of distractors on pop-out detection have been found earlier in several studies, but their mechanisms look far from being clear. Sagi and Julesz (1987) observed a nonmonotonic effect of number of objects on the detection of orientation singletons among line segments. This finding was interpreted as evidence that orientation differences can be efficiently calculated over limited retinal distances only. Bacon and Egeth (1991), however, found no effect of the distance between the target and the nearest distractors and concluded that it is the grouping of distractors that matters. Meinecke and Donk (2002) replicated the results of Sagi and Julesz and suggested several mechanisms that must be combined to explain their results. Also, they showed that an irrelevant empty space in the otherwise regular matrix of distractors deteriorates target detection, which is consistent with the salience account. 

The experiment most similar to Experiment 1 of the present study was run by Li, Their, and Wehrhahn (2000). These authors found a surprising improvement in orientation discrimination thresholds when the number of irrelevant context lines around the target line was increased. In their experiment, the luminance of the target was 16 times higher than that of the distractors. Probably, the luminance difference played the same role as did the color in the present study (although the authors argued that the luminance difference cannot explain their results). 

Similarity of the present results with those from the feature detection experiments and no difference between simple and complex shapes in Experiments 1 and 2 suggest that the effect of number of distractors is determined at a relatively low-level or detection stage of visual processing. The present study affirms that this early stage plays an important role in crowding and feature integration through modifying the input for these processes. 

A related effect has also been reported in backward-masking studies. Herzog, Fahle, and Koch (2001; see also Herzog & Fahle, 2002) showed that presenting five line segments after a briefly presented Vernier target (two abutting line segments) greatly reduces the visibility of the target. However, the masking effect of a grating consisting of 10 or more regularly spaced lines was much less. The authors explained the effect in terms of grouping and object segmentation and suggested that an underlying neural mechanism could be some sort of lateral interaction. The same seems to be valid for the present results. 

In conclusion, the present study demonstrates the important role of bottom-up salience in the binding of visual features. It suggests that the effective spatial window of feature integration is the result of a combination of (at least) two mechanisms at different levels of the visual system, with different sizes of receptive fields. This suggestion is in accordance with other recent ideas about attention-like selection occurring at many levels of visual processing (e.g., Tsotsos et al., 1995; VanRullen, 2003) and may be useful for developing new models of the visual system to replace the old attentive–preattentive dichotomy. 

Acknowledgments

I thank Talis Bachmann, Greg Francis, and two anonymous reviewers for their useful comments and suggestions. 

The main results of this study were presented at the European Conference on Visual Perception 2005, August 22-26, 2005, A Coruña, Spain. 

Commercial relationships: none. 

Corresponding author: Endel Põder. 

Email: [e-mail]. 

Address: Department of Psychology, University of Tartu, 78 Tiigi St., Tartu 50410, Estonia. 

Footnote

Footnotes

 ¹Consistent with several recent articles, the author used crowding to designate adverse interactions between adjacent visual objects. In earlier studies, lateral masking has been used for roughly the same phenomenon. Some authors have ascribed more specific meanings to these terms, assuming particular mechanisms of interaction.