How do flankers' relations affect crowding?

Abstract

The visual system can integrate discrete units of information to construct a coherent description of the input it receives. Little is known about the processes of grouping and their implementation in the visual system. Previously we described a configural effect in which the global arrangement (degree of co-circularity) of Gabor flankers affected the degree of crowding with a Gabor target that they surrounded. Here we tested possible mechanisms by which the configural effect might operate in crowding. We ruled out simple explanations based on the effect of basic units constructing these configurations (pairs of opposing Gabors). Our results support an explanation for crowding that is based on grouping processes between flankers. They also suggest that not all flankers necessarily directly affect the target. Flankers might group together and interact with the target as a single element. Finally, using a computational model of crowding based on compulsory grouping (following Gestalt principles) and segmentation, we define the relative contribution of different pair relations of grouped elements to crowding.

Introduction

Crowding is mostly thought of as interference from independent flankers to the target in an additive way (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Pelli, Palomares, & Majaj, 2004). We have previously demonstrated a limitation of such an account (Livne & Sagi, 2007). A perceptually continuous (smooth co-circular) arrangement of Gabor flankers was found to produce considerably less interference than a pseudo-random (henceforth called interrupted) arrangement in an orientation discrimination task. This suggests that spatial relations between flankers are an important factor in crowding. In that article we proposed two possible explanations for this result. According to the first, the difference can be accounted for by the larger orientation contrasts found between the adjacent flankers in the interrupted arrangements. The presence of these contrasts may reduce the efficiency of target detection and localization, as previously described by Rubenstein and Sagi (1990) for a texture segmentation task. The second explanation assumes the grouping of flankers to be responsible for the differences. More specifically, it assumes that the smooth co-circular arrangement of the flankers leads them to group among themselves much more than with the target. This could lead to the reduced amount of crowding found with this arrangement. In the interrupted arrangement, grouping between the target and flankers was as strong as the grouping between the flankers themselves, and therefore we observed a strong effect. Differences in the grouping of flankers and targets has been shown to affect acuity performance in both the fovea and the periphery (Malania, Herzog, & Westheimer, 2007; Saarela, Sayim, Westheimer, & Herzog, 2009). Similarly to our grouping account Saarela and his colleagues suggested that whenever the target stands out from the flankers, crowding is weakened. However, unlike our original grouping manipulation in their study they manipulated grouping by changing the similarity between the target and the flankers. Therefore, it is unclear whether this explanation applies to our case as well. The work presented here describes three experiments that further study the configural effect. In Experiment 1, we examined the relationship between crowding effects produced by circular configurations (smooth, interrupted, & sun) and the effects that are produced by pairs of Gabor flankers situated along the contour of these configurations. The results showed that crowding is not additive. In Experiment 2 the local contrasts account was tested. The results indicated that the degree of local orientation contrasts and their amount in the display fail to explain the configural effect. In the third experiment the grouping account was tested. The results of these experiments are discussed in the context of early visual analysis and representation of simple objects. We conclude that crowding is preceded by a processing stage where local features are grouped into simple objects. Finally, using a computational grouping model, based on the elements' distance and continuity, we estimate the relative interference caused by different flankers' relations. 

General methods

Apparatus and procedures

Experiments 2 and 3 were constructed using the Psychophysics Toolbox extension to Matlab (Brainard, 1997; Pelli, 1997); all experiments were performed on a PC, with a gamma corrected display luminance. 

Subjects performed an orientation discrimination task in which they were asked to report only the tilt of the central target (left or right relative to horizontal baseline), appearing randomly at 2.5° eccentricity ( Experiments 1 and 2) or 3° eccentricity ( Experiment 3) to the left or right of the fixation cross. The stimulus presentation time was 100 ms. Trials were self paced and were initiated by pressing a mouse button; the interval between trial initiation and stimulus presentation was 1000 ms in Experiment 1, 150–210 ms (randomly set) in Experiment 2, and 800 ms in Experiment 3. 

Data collection

A staircase method was used in which the orientation deviation from the horizontal increased by 20% after a wrong response and decreased by the same factor after three consecutive right responses. Eight such reversals were allowed, only the last six were taken into account. This method was shown to converge to 79% correct response (Levitt, 1971). A 45° orientation deviation from the horizontal was set as an upper bound and crossing it three times resulted in terminating the block. 

Results are averaged across sessions. They are log transformed and are presented as threshold elevation, calculated by subtracting the threshold of the target-only condition (treated as the baseline threshold) from each condition separately. Error bars in all figures represent the standard error ( SE) of the mean log data. 

Subjects

Eleven subjects with normal or corrected to normal vision participated (six in Experiment 1, four in Experiment 2, and three in Experiment 3). Ten were naïve to the purpose of the experiment. TL is the first author. 

Experiment 1

We previously found (Livne & Sagi, 2007) that an additive explanation fails to account for the configural effect, but there was a possibility that our reduced stimuli were too complex, since they were composed of four flankers (two pairs from each of the original configurations). Examination of the effects of individual pairs might reveal a clearer picture, and provide a simple explanation for our results. 

Stimuli

Stimuli consisted of Gabor patches, which are defined by wavelength λ and an SD of the Gaussian envelope σ, in this case λ = σ = 0.16°, of 80% contrast. Eight Gabor pairs were tested; those pairs were chosen from the three original configurations (smooth, interrupted, & sun— Figure 1—bottom right). The Gabors in each pair were either collinear or parallel to each other. They had one of three locations relative to the target-fixation plane: either above and below the target, left and right of it, or in a diagonal arrangement (there were two sets of diagonal pairs tested independently, but since the results were similar, we report them together). In addition, three of the subjects were tested with the three original configurations for comparison (see Figure 1). 

Figure 1

 

Experiment 1 stimuli and results. We tested the degree of crowding caused by the six pairs of opposing flankers composing the three configurations from Livne and Sagi (2007) shown on the right (smooth, interrupted, & sun). Whereas the interrupted configuration (2nd from the right) caused more crowding than the smooth one (3rd from the right), the effect measured with the pairs they differ by: collinear-diagonal (interrupted) vs. parallel-diagonal (smooth), showed a reverse pattern. Average results of 6 subjects. One asterisk represents p < 0.05, two asterisks: p < 0.01.

Results

We found large differences in the degree of crowding the different pairs of flankers produced; the results are presented in Figure 1. Three pairs produced significant threshold elevation (single sample t-tests with a Bonferroni correction to the significance level): the two diagonal pairs: the collinear pair ( t(5) = 5.26, p = 0.003), and the parallel pair ( t(5) = 10.71, p = 0.0001); and the horizontal collinear pair ( t(5) = 5.96, p = 0.002) (uncorrected alpha). The other three pairs did not produce a significant threshold elevation. The results of the three original configurations were in agreement with our previous study, with the smooth configuration producing the least amount of crowding. 

Discussion

Among the pair configurations, the highest degree of threshold elevation was measured with the diagonal-parallel and horizontal-collinear pairs. The smooth and interrupted configurations differ by the diagonal pairs, and the higher threshold elevation was measured with the pairs of the smooth configuration (diagonal-parallel pair). This confirms our previous assumption that the configural effect is not accounted for by an additive explanation. Additionally, these results rule out a maximal interference explanation. According to such an account, the configuration's threshold elevation would be determined by the pair producing the highest threshold elevation by itself. Here we would expect the smooth and sun configurations to produce more elevation than the interrupted one, which was not the case. Also, the smooth configuration produced less elevation than the diagonal-parallel pair, which again contradicts such an explanation. 

The present results show a strong anisotropy in crowding, a large reduction in crowding on the vertical axis (to practically non-existent), relative to the effects observed on the horizontal and diagonal (45°) axes. This anisotropy is partially consistent with that reported by Toet and Levi (1992), using different stimuli (Ts vs. Gabors), who reported stronger crowding with a radial arrangement of flankers than with a tangential arrangement. However, in our case, all subjects exhibited an equally strong crowding effect with a diagonal-flankers arrangement on the horizontal meridian, as with a horizontal arrangement, whereas in Toet and Levi's study, only about half of the subjects exhibited that pattern. For the other half, the diagonal effect was considerably smaller than the radial one. Preliminary results suggest that with Gabor stimuli, the strong effect of the diagonal arrangements persists even when different flankers' local orientations are used. 

Experiment 2

In this experiment we tested several factors that might contribute to the configural effect. We were especially interested to see whether a local contrast explanation could account for the effect. To this end, we tested whether the degree of local orientation contrasts between neighboring flankers, and the number of such pairs in a display could be used to predict crowding in multi flankers' displays. 

Stimuli

Eleven Gabor flankers' configurations (plus a baseline measure) were used ( Figure 2). In all configurations the diagonal-parallel pair from Experiment 1 was present. This was done in order to have a fixed baseline in all stimuli against which we could evaluate differences based on spatial and geometrical relations. Four of the configurations were based on the smooth configuration: one was arranged around the target (a), in another (b), one diagonal pair (two flankers) was removed; in the third configuration (c), two smooth circular contours were placed either in an upper-right lower-left relation with the target or in an upper-left lower-right relation. In the fourth arrangement (d), we removed five flankers from the two circular ones, ending up with a mirror flipped version of each trio in (b). Four configurations were based on the interrupted arrangement (now rotated by 45° to contain the diagonal-parallel pair) with the same alternation as described above (e–h). The other three configurations were: the diagonal-parallel pair itself (i); in (j) two parallel Gabors were added away from the target to each flanker of (i), positioned on the same axis. And, finally, in (k) we added 2 collinear Gabors to each one from (i). Thus, arrangement (i) was practically present in all other conditions. In each arrangement the minimal target flanker separation was 1°, which was the separation of the diagonal-parallel pair; the inter-flanker separation (from a flanker to its nearest neighbor) was 0.76° (except for (i)). The configurations were designed so that we could test the following effects: (1) the effect of local contrasts between flankers on crowding, using for instance, a condition with no local contrasts as in (k); (2) the effect of increasing the number of local contrasts while removing them farther from the target; (3) the effect of a number of flankers, having a variety of number of flankers in either a similar or different configuration; and (4) testing grouping factors such as grouping by continuity (configurations a & b for instance), and by proximity (g). 

Figure 2

 

Configurations used in Experiment 2. In this experiment we were interested to determine what conditions lead to crowding in general and to the configural effect in particular. Configurations are referred to in the text as follows: (a) smooth; (b) two smooth halves; (c) two smooth ones; (d) smooth flipped halves; (e) interrupted; (f) two interrupted halves; (g) two interrupted ones; (h) interrupted flipped halves; (i) pair; (j) parallel trios; (k) collinear trios.

Results

As expected, the results indicated different degrees of threshold elevation under various conditions. Again, the smooth configuration produced the least amount of interference. Although the interrupted configuration still produced a large threshold elevation, three configurations were found to be more disruptive. Results are presented in Figure 3. The smallest threshold elevation (representing crowding) was measured with (a)—the smooth co-circular arrangement (0.13 ± 0.07). Only three configurations produced stronger crowding than the pair configuration (i) (0.5 ± 0.06): (f)—two interrupted halves (0.73 ± 0.15), (j)—parallel trios (0.71 ± 0.07), and (k)—collinear trios (0.75 ± 0.08). Condition (e)—the interrupted configuration (0.51 ± 0.06) was similar to (i). Other results were (b)—two smooth halves (0.24 ± 0.08); (c)—two smooth ones (0.3 ± 0.1); (g)—two interrupted ones (0.39 ± 0.15); (h)—two interrupted flipped halves (0.41 ± 0.1); and (d)—smooth flipped halves (0.43 ± 0.09). A two-way ANOVA was conducted testing the effects of the degree of local orientations contrasts (3 levels: no local contrasts—configurations i–k; low local contrasts—configurations a–d; and high local contrasts—configurations e–h) and the number of flankers (4 levels) in crowding. The results show a significant effect for the degree of local contrasts ( F(2,36) = 17.89, p < 0.0001) and a marginal effect for the number of flankers ( F(3,36) = 2.84, p = 0.052) and the interaction of the two factors ( F(2,36) = 3.2, p = 0.053). Although the effect of local contrasts was significant, a post-hoc test indicated that the pattern of the results is at odds with the local contrast explanation we described above. Low local contrasts (found in the smooth configurations) were found to produce less interference than both the no-contrast conditions ( p < 0.0001) and the high-contrast conditions ( p = 0.004). The other two conditions were not significantly different. This means that there is a non-monotonic relation between the degree of local contrasts and the degree of crowding, in contrast to the prediction of the local contrasts explanation. Although the other two effects were only marginally significant we decided to analyze them as well. Post-hoc tests for the number of flankers' effect found no significant differences between any of the conditions. We tested the interaction by analyzing the effect of number of flankers separately for each local contrast group. The only significant effect was found in the no-contrasts group ( F(1,10) = 7.68, p = 0.02), indicating more crowding in the collinear and parallel trios (configurations j & k, respectively) than in the pair configuration (i). For the other two local contrasts groups there was no such effect. This indicates that the total number of local contrasts is not predictive of the crowding in the display. 

Figure 3

 

Upper graph: average results over four subjects, error bars are SE. Configurations are ordered by threshold elevation values in ascending order. The pair condition (i) resulted in a high degree of crowding, but not the highest. The strongest effect was obtained with the parallel (k), and collinear (j) trios, and the two interrupted halves (f). The smallest degree of crowding was measured with the smooth configuration (a), followed by the two smooth halves (b), and the two smooth ones (c). All other configurations' effects fell in between; see text for details. Lower graph: orientation contrast levels between adjacent elements in each configuration.

Discussion

The results indicate the failure of the local contrasts' explanation. Actually, the configuration with the least amount of local contrasts (collinear trios—k) was found to induce the strongest interference (significantly higher than most configurations). A similar conclusion can be derived from the results of the parallel trios' configuration (j). The results, however, do not support a simple grouping explanation either. The highest magnitude of crowding produced by the collinear arrangement (k) poses problems for this account as well. The results further show that the number of flankers in a stimulus is uninformative as a predictor of the magnitude of crowding. We found that stimuli with the same number of flankers produced different degrees of crowding, regardless of whether all the flankers were at the same or at different distances from the target (e.g., configurations b, d, f, h, & k, or configurations a & e); configurations with more flankers produced less interference than others with less flankers (e.g., configurations a & b compared with i); whereas other configurations showed the reverse pattern (e.g., configuration i vs. k). 

Experiment 3

Having ruled out alternative explanations for the configural effect in crowding (additive, max effect, and local contrasts), in this experiment we tested a grouping explanation. Specifically, we tested whether flankers directly affect the target or alternatively, whether flankers group together and affect the target as a single group. To this end, we arranged two layers of flankers around the target; we kept the target–flanker separation of the closer layer fixed while varying the distance of the remote layer. In a previous study we had found that with a single layer of flankers, increasing the separation reduced crowding (Livne & Sagi, 2007). We reasoned that if the distant flankers directly affect the target, crowding should be reduced with increasing separation; however, if, on the other hand, their effect was mediated by the closer flankers, the effect of separation would be reduced. 

Stimuli

The stimuli consisted of several 8 cpd Gabor patches, ( λ = σ = 0.12°), of 90% contrast. Eight flankers were arranged around the target in either a smooth co-circular arrangement ( Figure 4a) or in a perpendicular arrangement relative to the imaginary circle (sun configuration— Figure 4d) at a fixed target–flanker separation of 0.61° (equal to 0.2 of the used eccentricity—E). Eight additional flankers were added farther away from the target arranged at one of the above arrangements ( Figures 4b, 4c, 4e, and 4f). Four different target–flanker separations were used for this second layer of flankers: 1.05°, 1.23°, 1.41°, & 1.6° (0.35E, 0.41E, 0.47E, & 0.53E), respectively. Each arrangement and separation was tested separately. 

Figure 4

 

Configurations used in Experiment 3. We had eight flankers at 0.61° from the target (0.2E) arranged in either a smooth or sun arrangement (a & d, respectively). Additional eight flankers were arranged in one of these configurations farther away from the target at 1.05°, 1.23°, 1.41°, and 1.6° (0.35E, 0.41E, 0.47E, 0.53E), respectively. (b) Smooth–smooth; (c) smooth–sun; (e) sun–sun; and (f) sun–smooth.

Results

First, we verified that each condition produced crowding using the regular threshold elevation method ( Figure 5, upper graph). For this purpose, we ran four single-sample t-tests with a Bonferroni correction for multiple testing; the results were significant for all the conditions: smooth–smooth ( t(11) = 9.36, p < 0.0001); smooth–sun ( t(11) = 11.7, p < 0.0001); sun–sun ( t(11) = 13.76, p < 0.0001); sun–smooth ( t(11) = 11.72, p < 0.0001) (uncorrected alpha). Next, we calculated the additional threshold elevation produced by the outer eight flankers. This was done by subtracting from the measured threshold the threshold measured using only the inner eight flankers (either the smooth or the sun configuration). Results are presented in Figure 5 (lower graph). A 2-way ANOVA (configuration × separation) indicated a significant effect for configuration ( F(3,16) = 24.59, p < 0.0001), but not for an outer layer separation from the target ( F(3,16) = 0.18, p > 0.1). Finally, we performed another four single-sample t-tests with a Bonferroni correction on these results and found that even after the subtraction, all the configurations showed significant crowding: smooth–smooth ( t(11) = 5.34, p < 0.0001); smooth–sun ( t(11) = 13.85, p < 0.0001); sun–sun ( t(11) = 4.95, p < 0.0001); and sun–smooth ( t(11) = 4.10, p = 0.002) (uncorrected alpha), although in some cases this elevation was rather small. 

Figure 5

 

Results of Experiment 3: upper graph—threshold elevation of four configurations (three subjects' mean results); we subtracted a no-flankers' threshold from the measured threshold for the respective condition. Lower graph—outer flankers' effect; here we subtracted the threshold obtained by the inner configuration (smooth in the smooth–smooth and smooth–sun conditions, and sun in the other two conditions; see inset in the lower graph). The values of the x-axis represent the separation of the outer flankers from the target in multiplication of Eccentricity units (E = 3°). The results show that the outer flankers' effect (lower graph) does not diminish with increasing separation.

Discussion

As can be seen in the lower graph of Figure 5, the effect of the eight outer flankers does not diminish when their separation from the target is increased. This raises the possibility that these flankers do not affect the target directly (crowd it). If they did, we would expect their effect to decrease as they are removed farther away from the target, as was demonstrated in Livne and Sagi (2007) when only one set of flankers was used. The fact that we found no such clear reduction with increasing separation suggests that their contribution to crowding was mediated by the eight closer flankers. This finding is consistent with the finding of Strasburger, Harvey, and Rentschler (1991). They found that additional numeral flankers positioned farther away from the two original numeral flankers increased the spatial extent of the effect. However, unlike Strasburger et al. (1991), who kept their flanker–flanker distance fixed and manipulated the target–flankers' distance, we kept the minimal target–flanker distance constant and manipulated the flanker–flanker distance. Thus, we were able compare target–flanker and flanker–flanker proximity-based grouping, showing that the latter factor had no effect on crowding. 

General discussion

We tested which factors contribute to the configural crowding effect. Several possible explanations were tested and rejected; first, an effect based on the strongest crowding pair in each configuration was ruled out. We found that the pairs composing the low-crowding smooth configuration produced more crowding than those of the high-crowding interrupted configuration. These results also helped reject an additive explanation based on the effects of these pairs. Of particular interest are cases where the removal of flankers actually increases crowding. For instance, the smooth configuration as a whole produced less interference than only one of the pairs composing it did (the parallel-diagonal pair). 

We also found that large orientation contrasts between flankers do not necessarily increase crowding relative to low orientation contrasts. Flanker configurations with no local contrasts (collinear or parallel) caused higher levels of crowding than arrangements with varying degrees of contrast, or even single flankers. Additionally, we found that placing flankers farther away from the target can either increase or decrease crowding, depending on the flankers' relations to one another. Also, apparently not all flankers that produce crowding necessarily do so by directly interacting with the target. In certain cases (depending on the target–flanker and flanker–flanker relations) the effect of the more distant flankers might be in modulating the effect from the closer ones. 

Based on our results, we suggest grouping as a major factor contributing to the configural crowding effect. According to such an explanation, the global properties of groups of flankers determine their interference with the target, overriding the effects that the individual flankers may have when used in isolation, such as eccentricity-separation scaling. In 1 we describe a computational model that was constructed along these lines. The model works in two stages. It starts by grouping all the elements in the display based on Gestalt principles of proximity and continuity (distance and degree of collinearity). In 2 we relate this stage to contour integration processes (see below). In the second stage our model computes an interference factor (representing crowding) based on the relations between the grouped flankers. By fitting the model behavior (crowding prediction) to our psychophysical results (of Experiments 2 and 3), we derived a ranking of the relative interference five of these relation types might cause in crowding. Our results suggest that the strongest interference results from parallel flankers pairs, followed by collinear, T-junction, and contrasting (all other cases) respectively, with co-circular relations causing the least amount of interference. Future work would examine these relations in detail. A complete description of the model is provided in 1. In 2 we describe how the grouping algorithm was optimized using an independent measure. We had the grouping algorithm perform a contour integration task and used the performance in this task to derive constraints for the crowding model ( 1). The fact that we were able to model both tasks (crowding and contour-integration) by a single mechanism (the grouping algorithm) supports the possibility of a common neural mechanism for both, as has been previously suggested (May & Hess, 2007). 

In our previous article (Livne & Sagi, 2007) describing the configural effect, we discussed how this effect poses a problem regarding the plausibility of explanations based on a fixed receptive-field size (integration fields—Pelli et al., 2004, attentional resolution limitation—He, Cavanagh, & Intriligator, 1996; Intriligator & Cavanagh, 2001). Our present results show a further limitation of separation-based explanations. In Experiment 3 crowding did not diminish with increasing separation of some of the flankers from the target, as predicted by these explanations. Additionally, our grouping effect suggests no loss of flankers' positional information. While, previously it has been shown that the loss of positional information is one of the factors contributing to crowding (Strasburger, 2005), it is possible that configural information prevents this positional information loss. This interpretation seems consistent with the fact that the loss of positional information occurred only in about one third of the cases in Strasburger's study. In contrast to these previous explanations to crowding we suggest that flankers cause interference to the target based on their relations to other elements in the display. 

Recently, Põder (2008) suggested that the complexity of the stimulus (target and flankers) might determine the amount of crowding. Although in a sense this suggestion is analogous to our grouping account, it lacks essential definitions of “stimulus complexity”. Furthermore, it also seems that in its simplistic form it cannot account for some of our results. According to Põder, the smooth configuration might be perceived as a single “simple” object that leaves out more attentive resources for identifying the target. According to the same reasoning, we would expect the collinear trios of Experiment 2 (configuration k) to cause much less crowding than they did (say an effect similar to that of configuration c). This should be the case since collinearity is efficiently and quickly detected and represented in vision (Bonneh & Sagi, 1998; Field, Hayes, & Hess, 1993), and a collinear string is probably simpler than a co-circular one (single vs. multiple orientations). Since our results show that the collinear trios were the most interfering ones, a simple grouping explanation is insufficient. It is not simply that bounding flankers together reduces crowding. A minimal additional requirement for such an explanation is for it to include a measure of the group's properties, such as flankers' spatial relations for instance, other than simplicity or detectability (see 1 for one possible solution). 

Taken together with results showing that categorical or semantical similarity levels of target and flankers affect crowding (Huckauf, Heller, & Nazir, 1999), it seems that crowded elements are not simply degraded. They are encoded at least to some degree and acted upon, grouped to construct simple or complex objects, and only when reporting is required, do we find them difficult to describe. 

Appendix A

A computational account of flankers' relations effects in crowding

We tested whether a computational model of grouping, based on spatial and geometrical relations, could account for our psychophysical results. The model itself is not designed to describe biological processes nor do we assume it to be a full and accurate model of crowding. Our intention is to demonstrate the possibility of such an explanation to account for the configural effect. We describe two processing stages: first grouping of all the elements of the stimuli, and then segmenting the target and evaluating the interference of the flankers. Running the model using the stimuli from Experiments 2 and 3, we were able to estimate the relative interference caused by different pair relations. 

Grouping stage

The model starts by identifying each of the elements in the stimuli; since we are using Gabor patches, we encode at this stage their location (on a 2D grid), and their orientation. We then compute geometrical relations between each pair of patches based on their 2D Euclidean distance and their continuity (collinearity or co-circularity). This is computed using Equation A1:  

$D_s=D_e(1+f_{ij}),$

(A1)

where D _e is the two-dimensional Euclidean distance between the two patches ( i, j) and f _ij is a measure of their continuity defined as:  

$f_{ij}=a({\theta }_i+{\theta }_j)+b·{\varphi }_{ij}.$

(A2)

Where θ _i and θ _j are the acute angles between Gabor patches g _i and g _j (correspondingly) and the line connecting their centers, and ϕ _ij is the acute angle between the two virtual lines aligned to the two patches (all angles measured in radians). Variables a & b are constants (2.4 & 0.8, respectively). Their values are derived to optimize performance on a contour integration task independently of the crowding task as described in 2. Function f emphasizes continuity by taxing (assigning high values to) deviation from continuity (see 2 for more details and Figure A1 for its output for different orientations). Although D _s values may be best described as representing the strength of relations between the two elements in each pair, it is more convenient to consider them as distances between elements. We compute D _s for each pair and connect each element to its closest neighbor(s). The connection is performed by a variation of Kruskal's minimum spanning tree algorithm (Kruskal, 1956). In our version, at each step, one distance value is considered (out of all the computed D_ss—starting with the smallest one and increasing it in consecutive steps). Each pair of elements that have the considered distance is connected (unless an indirect path connecting them has been established in previous steps). In the next step, the distance value is increased and the connection process is repeated. As in the original algorithm, our process stops when all the elements are connected by either a direct or an indirect connection. There is one main difference between our version and the original algorithm, leading to differences in their outputs. In Kruskal's original algorithm, at each step only one pair of elements is connected. In the next step, the same distance value is considered. If there are unconnected pairs that have this value, one of them is connected unless it is connected by an indirect path. If no such pair exists, the distance value is increased. The first difference in the outputs is that whereas in the original method we end up with a unique path between all elements, in our version we might end up with multiple alternative paths. The second difference is that since we connect pairs having the same distance value in parallel, there is only one grouping solution. In the original version, the connection is performed serially and therefore the solution may not be unique: it may depend on the order in which we chose to consider the pairs. Figure A2 displays the grouping pattern of several configurations. When the grouping stage is complete, we define for each Gabor a set (Ce) of other Gabors directly connected to it. 

Figure A1

 

A color map of the output of function f ( Equation A2) for two Gabor patches separated by a fixed horizontal displacement. The two axes represent the orientation of the two Gabors. The origin (0, 0) corresponds to two horizontal Gabors which form a collinear configuration, having the lowest f value. The four corners (± π/2, ± π/2) correspond to two vertical Gabors which form a parallel configuration, having the highest f value.

Figure A2

 

Several grouping examples. The lighter and thinner the connecting line is, the weaker the connection (the higher is the Ds value). (a)—Smooth; (b)—smooth halves; (c)—interrupted; (d)—smooth flipped halves.

Calculating crowding strength

After the grouping stage, we proceed to calculate a crowding factor based on the created pattern. We define a crowding zone around the target as the Euclidean distance of the most distant flanker directly connected to it. In calculating crowding, we consider only flankers that are within this zone. For each of these flankers ( i), we compute an interference value ( I _i) based on its orientation relations with the flankers directly connected to it ( j, belonging to the set Ce—defined by the grouping process not including the target itself), using Equation A3:  

$I_i=c+\sum _{j\in (\mathrm{C}e\subset /‘\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}’)}G({\theta }_i,{\theta }_j),$

(A3)

where c is a constant representing the baseline interference of a flanker. As a first approximation, we assume that regardless of its orientation each flanker produces the same baseline effect. This is in line with the observation that the location and local orientation anisotropy of flankers observed in Experiment 1 might not extend to the more complex configurations used in that experiment. G( θ _i, θ _j) is the contribution to crowding of a connected flankers' pair, based on their orientation relations. These relations can be divided into five types: collinear, parallel, co-circular, T-junction, and all other relations). We assume that each of the five relation types contributes a different degree of interference. For each configuration, we sum all the Is (influence of the flankers in the crowding zone). We treat this sum as the stimulus crowding factor ( Equation A4).  

$\mathrm{C}\mathrm{r}\mathrm{o}\mathrm{w}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{g}=\sum _{I\in ‘\mathrm{C}\mathrm{r}\mathrm{o}\mathrm{w}\mathrm{d}-\mathrm{z}\mathrm{o}\mathrm{n}\mathrm{e}’}{I}_i$

(A4)

 

Estimating pair relations' effects

Our computational model describes a possible realization of grouping and individuation mechanism leading to the interference observed in crowding. In this section we show that it is also possible to derive a quantitative prediction based on it. By fitting the output of the model with our measured results of Experiments 2 and 3, we can rank the relative interference of these five relation types of function G ( Equation A3). This is done by optimizing the values of constant ( c) and function G ( Equation A3) to produce crowding predictions for the relative interference of these configurations, which are in agreement with the measured results ( r ² = [0.75, 0.76] for Experiments 2 and 3, respectively, the explained variance based on Pearson's correlation), see Figure A3. The values obtained by this method suggest that the strongest interference results from parallel pairs, followed by collinear, T-junction, and contrasting; with co-circular relations causing the least amount of interference ( c = 0.1, G( θ _i, θ _j) = [1.2, 5.1, −0.3, 0.7, 0.4], for relation types: collinear, parallel, co-circular, T-junction, and other, respectively). 

Figure A3

 

Model's fit to average performances in Experiments 2 and 3.

Appendix B

Crowding and contour integration

Our model performs geometrical and spatially based grouping of oriented Gabor patches. This process is similar to the one performed in contour integration (CI) and detection tasks (Field et al., 1993; Li, 1998; May & Hess, 2007; Pettet, McKee, & Grzywacz, 1998; Watt, Ledgeway, & Dakin, 2008; Yen & Finkel, 1998). Unlike the present experiment where a-priori estimation of the eventual grouping pattern is unavailable, in CI tasks the target, a predefined Gabor chain, is known in advance. The straightness and smoothness of Gabor chains has been found to affect their detection. Straight lines and smooth co-circular arrangements are known to be most efficiently detected (Field et al., 1993). We therefore defined our continuity estimation (Equations A1 and A2) to promote grouping of pairs that conform to one of these relation types. We ran our grouping algorithm on three types of stimuli: 1) line targets (straight or broken lines), 2) circle targets (smooth or jagged circles), and 3) smooth ellipses; all embedded within random arrays of oriented Gabors (noise). We varied the values of a & b (Equation A2) and tested how many iterations are needed with every set of values for efficient integration of the elements of the target configurations. We define efficient integration by three criteria: 1) a direct connection between adjacent elements along the configuration's path must be achieved; 2) a complete connection of all path elements should be achieved prior to the construction of accidental noise chains longer than the target chain; and 3) the target group should include a minimal number of appendage noise elements linked to the target patches (see Figure B1 for several examples of successful and unsuccessful outputs). The results of this procedure indicated that the model was able to complete the task successfully using several pairs of a & b values. The difference in the results between these values mostly demonstrated a tradeoff between integration of straight lines and integration of circles (with ellipses being a mixture of both). To determine the most efficient pair values, we concentrated on the number of iterations it took the model to integrate the target group's elements. The number of iterations was transformed to z-scores across a & b pairs (for each display independently). Then for each of the target types, lines, circles, or ellipses, an average z-score was computed for each pair of a & b values. This gave us three vectors representing the relative efficiency of each pair for each target type. These vectors were sorted by efficiency (from high to low), and their first point of intersection (representing best overall efficient pair of values) was chosen for our crowding simulation (described in 1). In addition, we ran the simulation with some of the other (efficient) pairs and got results comparable with those reported in 1, although some differences in the grouping pattern were observed. This might allow for individual customization of the model for different subjects. 

Figure B1

 

Three contour integration outputs obtained by three sets of a & b values ( Equation A2). a) Presents the stimuli; in the rest of the figure the pattern obtained by the grouping algorithm when all the target elements are linked to a single group is shown. b) A successful integration; a few accidental noise pairs were integrated prior to the integration of the target contour. c) An acceptable solution, more accidental noise elements integrated, but they are still relatively short chains. In addition, several noise elements are linked to the contour (appendages), but the target contour is still clearly detectable. d) Unsuccessful integration; targets' elements are linked to long noise chains prior to linking with one another. In addition, there is no direct link between adjacent elements of the target contour.

Although we have shown that our grouping algorithm can integrate contour elements in noisy displays, it still lacks several properties needed to make it a contour integration model; mainly, we assume that all elements are connected by the end of the grouping stage. This connectivity means that no isolated contours are present when the grouping process is complete. However, as explained above, grouping is realized in the model as a serial process: in each iteration we group Gabor pairs with larger D _s values than those grouped in the previous iteration. And as a result of our chosen a & b values, nearly collinear or co-circular flankers will group at earlier iteration stages than other pairs. Therefore, at the end of each iteration, we can check whether any group might be our target configuration. The signal on which a decision would be based could be group saliency. Grouped elements might facilitate or synchronize each other, making their combined signal distinguishable from that of other elements in the stimuli. Since each of the separate groups created in the intermediate stages is defined in the model as independent from the rest (until grouped in a later iteration), localization information may be obtained as well for any group passing the detection threshold (based on retinotopic information). Figure B2 demonstrates several stages of grouping on a large Gabor array. A co-circular contour composed of twelve Gabor patches is grouped together after 28 iterations. To group together all the elements, the model completed 135 iterations. There are a few aspects that we have not included which should be considered in future work: 1) Control of the grouping pattern: It might be desirable to bias grouping toward a predefined pattern. Some serial models of CI suggest that later iterations should try to preserve the relation type of earlier connections (e.g., depending on previous connections, the next connection is preferred to be either the snake or ladder type, May & Hess, 2007). Similarly, in certain cases the algorithm might be biased toward circles rather than lines (if the target is known in advance or its type is primed). Furthermore, for higher-level stimuli, such as letters, this biasing might be even more complex and driven by stored templates (see below). 2) Our model does not describe enhancement of grouped elements, which might be a desired feature for certain tasks, as well as for group saliency estimation (see above). 3) Our model does not enhance detection based on closure, as some researchers suggest human subjects do (Kovács & Julesz, 1993; Mathes & Fahle, 2007). Although these properties were not included in the present version of the grouping algorithm, it is not difficult to see how they might be added without affecting our model's crowding estimation. 

Figure B2

 

Detection of a circle composed of twelve Gabor patches in a field of 144 Gabors. (a) The full stimulus set is shown. Each Gabor in the rest of the figure is connected to one or more Gabors. (b)—Connection established at the 1st iteration; (c)—Connections established at the 20th iteration; (d)—Connections established at the 28th iteration; (e)—Connections established at the 60th iteration. (f) The complete processing of the whole stimulus set (all elements connected) took 135 iterations.

A possible role for top-down effects

Our model does not use global template matching, such as straight-lines detectors or circle detectors. This was done for simplicity and in light of the fact that we used simple stimuli. It is possible, however, that such a property would come in handy in future models, especially when trying to tackle more complex stimuli such as letters and numbers. We might find bottom-up and top-down competition on the grouping pattern useful. This might be in line with results reported by Huckauf et al. (1999) showing that semantically well-defined flankers (letters) produced less crowding than pseudo or rotated letters. In terms of our model, this might indicate that letter representations are constructed before they are grouped (and interfere) with the target. This is unlike the case where (for pseudo-letter flankers) the flankers' sub-elements group with the target before completely grouping with each other, when no such higher level representations may be constructed from these flankers. This interpretation is in line with the suggestions made by Strasburger (2005) and Strasburger and Rentschler (2007). 

Acknowledgments

We thank Hans Strasburger for helpful comments on the manuscript. 

Supported by the Basic Research Foundation administered by the Israeli Academy of Science & Humanities. 

Commercial relationships: none. 

Corresponding author: Dov Sagi. 

Email: [e-mail]. 

Address: Department of Neurobiology, The Weizmann Institute of Science, Rehovot 76100, Israel.