Ladder contours are undetectable in the periphery: A crowding effect?

Abstract

We studied the perceptual integration of contours consisting of Gabor elements positioned along a smooth path, embedded among distractor elements. Contour elements either formed tangents to the path (“snakes”) or were perpendicular to it (“ladders”). Perfectly straight snakes and ladders were easily detected in the fovea but, at an eccentricity of 6°, only the snakes were detectable. The disproportionate impairment of peripheral ladder detection remained when we brought foveal performance away from ceiling by jittering the orientations of the elements. We propose that the failure to detect peripheral ladders is a form of crowding, the phenomenon observed when identification of peripherally located letters is disrupted by flanking letters. D. G. Pelli, M. Palomares, and N. J. Majaj (2004) outlined a model in which simple feature detectors are followed by integration fields, which are involved in tasks, such as letter identification, that require the outputs of several detectors. They proposed that crowding occurs because small integration fields are absent from the periphery, leading to inappropriate feature integration by large peripheral integration fields. We argue that the “association field,” which has been proposed to mediate contour integration (D. J. Field, A. Hayes, & R. F. Hess, 1993), is a type of integration field. Our data are explained by an elaboration of Pelli et al.'s model, in which weak ladder integration competes with strong snake integration. In the fovea, the association fields were small, and the model integrated snakes and ladders with little interference. In the periphery, the association fields were large, and integration of ladders was severely disrupted by interference from spurious snake contours. In contrast, the model easily detected snake contours in the periphery. In a further demonstration of the possible link between contour integration and crowding, we ran our contour integration model on groups of three-letter stimuli made from short line segments. Our model showed several key properties of crowding: The critical spacing for crowding to occur was independent of the size of the target letter, scaled with eccentricity, and was greater on the peripheral side of the target.

Introduction

The means by which low-level features in the visual environment are grouped to create high-level structure has been studied for more than a century (e.g., Ehrenfels, 1890/1988; Wertheimer, 1923/1938) but remains largely mysterious. In this paper, we attempt to shed some light on the grouping processes that underlie the perceptual integration of contours. 

Pelli, Palomares, and Majaj (2004) outlined a two-stage model of feature integration, in which the outputs of simple feature detectors are combined using integration fields. They proposed that the integration fields are involved in any task which depends on the outputs of more than one simple feature detector. One such task is letter identification. Pelli et al. argued that the phenomenon of crowding (whereby identification of a peripherally located target letter is impaired by the presence of flanking letters) is caused by inappropriate integration of features from target and flankers due to the absence of small integration fields in the periphery. 

Pelli et al. (2004) argued that, at any point in the visual field, the minimum integration field size is proportional to eccentricity. The evidence for this is that the critical target-mask spacing for crowding to occur is about half the eccentricity for several quite different tasks: letter identification (Bouma, 1970; Pelli et al., 2004; Toet & Levi, 1992), fine discrimination of line orientation (Andriessen & Bouma, 1976), and fine discrimination of contrast, orientation, and spatial frequency of gratings (Wilkinson, Wilson, & Ellemberg, 1997). 

Contour detection is another task that requires the outputs of several simple feature detectors to be integrated (Field, Hayes, & Hess, 1993). Field et al. (1993) proposed that the elements along a smooth contour are integrated by an association field. We consider that the association field is an example of Pelli et al.'s (2004) integration field. 

In most studies of contour integration, the contour elements have formed tangents to the path of the contour. Performance is very poor when the elements are oriented at 45° to the path (Ledgeway, Hess, & Geisler, 2005) but can be reasonably high when the elements are orthogonal to it (Bex, Simmers, & Dakin, 2001; Field et al., 1993; Hess, Ledgeway, & Dakin, 2000; Ledgeway et al., 2005). The contour types with elements tangential and orthogonal to the path were termed “snakes” and “ladders,” respectively, by Bex et al. (2001). Having found that ladders were harder to detect than snakes, Field et al. (1993) concluded that “the association between elements is stronger along the axis of the element than orthogonal to the axis” (p. 185). 

We therefore have reasonable grounds for inferring two characteristics of the association fields: (1) the minimum field size at any point in the visual field scales with eccentricity; and (2) the associations are stronger for snake than ladder configurations. In this paper, we consider how these two characteristics might combine to determine the detectability of snake and ladder contours in the periphery. 

Let us consider two broad classes of association field model. In one class of model, snakes and ladders are integrated by different types of association field, so that integration proceeds independently for the two types of contour. If the minimum field size scales with eccentricity in the same way for both types of association field (as appears to be the case for other types of integration field), then we might expect that peripheral viewing would have a roughly similar effect on each type of contour. In the other class of model, snakes and ladders are integrated by the same association field, and “snake” and “ladder” associations compete directly against each other for ownership of the elements. In this second class of model, if ladder associations were weaker than snake associations (as concluded by Field et al., 1993), then we would expect peripheral viewing to cause a much greater impairment of ladder detection than snake detection. This is because, as the association field size gets larger, there is an increased probability that a ladder element will be approximately collinear with a distractor within the same field; if snake associations are stronger than ladder associations, then the ladder element is likely to form a snake with the collinear distractor element, preventing it from forming part of the ladder contour. Straight snake contours, on the other hand, would be relatively undisturbed by increases in association field size because, even with large association fields, the snake elements would be most strongly linked to their neighbors along the contour. Later, we present modeling results that confirm the intuitive argument presented here. 

Our argument that snake detection would be relatively unimpaired in the periphery only applies to snakes that are fairly straight. We would expect detection of snakes with substantial curvature to be impaired in the periphery, for the same reason as ladders: If the snake elements are not completely collinear then, as the field size increases, there is an increased probability that an element of a curved snake will be more collinear with a distractor element within the field than with its neighbors along the contour. The snake element may then form an association with the distractor element, interrupting the integration of the target contour. These predictions relating to straight and curved peripheral snakes have already been confirmed: In the periphery, performance on curved snakes tends to drop more substantially than on straight snakes (Hess & Dakin, 1997, 1999; Nugent, Keswani, Woods, & Peli, 2003). This finding would be predicted by both classes of association field model outlined above. The two classes of model differ in their predicted effects of peripheral viewing on fairly straight ladders: Peripheral viewing should have a severe impact only in the class of models in which snake and ladder associations are in competition with each other. In this study, we compared peripheral performance on snakes and ladders. 

An unresolved issue relating to peripheral contour detection is the extent to which alternating-phase snakes are detectable in the periphery. In this type of contour, the phase of the elements differs by 180° between neighboring elements along the contour. Detection of alternating-phase contours in the periphery can be good when the paths form smooth, closed, recognizable shapes, such as circles or ellipses (Kuai & Yu, 2006). In contrast, using contours that randomly changed direction from element to element, Hess and Dakin (1997, 1999) found a complete failure to detect alternating-phase snakes at an eccentricity of 20° visual angle. However, Nugent et al. (2003) attempted to replicate Hess and Dakin's experiment as closely as possible and found that performance on alternating-phase snakes at 20° was around 70–80% correct and did not differ substantially from performance on same-phase snakes. The reasons for this discrepancy are not clear. One possible explanation is that there were individual differences between the subjects in the two studies. The small number of subjects in these studies makes it likely that the results could have been affected by individual differences (Hess & Dakin, 1997, 1999, had two subjects, and Nugent et al., 2003, had three subjects). Further support for this idea is that one of Hess and Dakin's (1999) subjects (RH) has subsequently shown chance performance in another experiment, when a different subject in the same experiment was substantially above chance: Ledgeway et al. (2005) found that, for straight contours with elements oriented at 45° to the path, subject RH (labeled RFH in Ledgeway et al.'s paper) was at chance, whereas subject TL obtained a score of around 70% correct. This discrepancy between RH and TL in Ledgeway et al.'s experiment closely mirrors the discrepancy between Hess and Dakin's results and those of Nugent et al. In this study, we have restricted ourselves to same-phase snakes and ladders. 

Methods

Subjects

Experiment 1 used four male subjects, BCH, BT, DHB, and KAM, who all had normal or corrected-to-normal vision. One subject, KAM, was aware of the purposes of the study. All subjects were experienced psychophysical observers, but prior experience with contour integration experiments ranged from virtually none to considerable experience. All subjects had had approximately equal experience with snake and ladder contours, making them ideal for this study, in which performance on the two contour types was being compared. BCH and KAM had had the most experience with contour integration experiments, and these two subjects participated in Experiment 2. 

Apparatus

The experiments were run on a PC fitted with a graphics card with a DVI digital output (GeForce 6800DDR 256 MB 256 bit PCI-E MS-8984-12S). The DVI output was connected to a Cambridge Research Systems Bits++ processor, which combined the red and the green channels, to produce a 14-bit digital grayscale image. The Bits++ processor then converted this 14-bit digital image into an analogue signal, which was displayed on a Sony CPD-520GS Monitor with a resolution of 1024 × 768 pixels, and frame rate 85 Hz. The background luminance remained at 50.6 cd/m ² throughout the experiment. Images were calibrated by using a Cambridge Research Systems OptiCAL luminance meter to find the function that mapped 14-bit digital values onto luminance levels; the inverse of this function was used to convert luminance levels to 14-bit digital values, which were then distributed across the red and the green channels appropriately. Experiments were controlled using MATLAB (The MathWorks, Inc.), which was also used to create the images. Images were displayed using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). Subjects viewed the screen binocularly from a distance of 100 cm. 

Procedure

The procedure was very similar in the two experiments. There were two independent variables: contour type (snake or ladder) and eccentricity of fixation ( Experiment 1: 0°, 4°, or 6° visual angle; Experiment 2: 0°, 4°, 6°, or 8° visual angle). Snakes and ladders were randomly interleaved within sessions. Fixation eccentricity was blocked into separate sessions. Subjects performed four sessions at each eccentricity, fixating left, right, up, or down in separate sessions, except for the foveal condition (0° eccentricity), in which subjects fixated at the center of the screen for all four sessions. The order of sessions was randomized differently for each subject, except that the first session in each experiment for each subject was a foveal condition, so the subjects could get a good look at the two types of contour before performing any peripheral condition. 

Within each session, there were 50 trials for each contour type. Each trial consisted of a 1000 ms fixation period, followed by two 247 ms intervals separated in time by 1000 ms: One randomly selected interval contained a contour stimulus, and the other contained a stimulus with only distractor elements. The stimuli were always centered on the center of the screen. The fixation period and interstimulus interval contained a small fixation dot at the fixation location. The fixation dot was a dark Gaussian blob with standard deviation 0.04° visual angle and contrast 0.5. For the foveal condition, the fixation dot disappeared during presentation of the stimuli, so that it did not interfere with them; for the other conditions, the fixation dot stayed on for the duration of the trial to help the subject to maintain correct fixation. On each trial, the subject indicated with a mouse which interval he thought contained the contour. After each trial, the subject received auditory feedback to indicate whether the response was correct or incorrect. 

The main difference between Experiments 1 and 2 was that, in Experiment 1, the contour elements were perfectly straight and jitter-free, whereas, in Experiment 2, the contours had a small amount of curvature (a 10° path angle between adjacent contour segments), and the path angle, the element separation, and the orientation of the elements relative to the path were randomly jittered to bring foveal performance away from ceiling. The algorithm for generating the stimuli is described in the next section. Examples of stimuli from both experiments are shown in Figure 1. 

Figure 1

 

(A and B) Examples of the snake (A) and the ladder (B) stimuli used in Experiment 1. (C and D) Examples of the snake (C) and the ladder (D) stimuli used in Experiment 2. To make these examples of snakes and ladders as comparable as possible, the elements in panels B and D have the same positions as those in panels A and C, respectively. Readers who have difficulty seeing the contours can view Supplementary Figure 1, in which the contour elements have a higher contrast than the distractor elements.

Stimuli

There were four separate aspects of stimulus construction: generating the Gabor elements, positioning the contour elements, positioning the distractor elements, and choosing the orientations of the elements. These are described in detail in the next four subsections. 

The Gabor elements

The Gabor elements were generated using Equation 1:  

$L(x,y)=L_0(1+cw),$

(1)

where c is the carrier, and w is the envelope, as defined in Equations 2 and 3, respectively:  

$c=C\sin [2\pi f(x\cos \theta +y\sin \theta \left)\right],$

(2)

 

$w=\exp (-\frac{-(x^2+y^2)}{2{\sigma }^2}),$

(3)

where L is the luminance at position ( x, y), measured from the center of the Gabor patch; L ₀ is the mean (background) luminance (50.6 cd/m ²); C is the Michelson contrast, which was set to 0.5; f is the spatial frequency of the Gabor carrier, which was 4.65 cycles/deg; θ is the orientation of the element; and σ is the standard deviation of the Gaussian window and was set to 0.108°, that is, half the carrier wavelength. 

Positioning the contour elements

The construction of the contour stimuli is illustrated in Figure 2. The parameters (shown in Table 1) differed between Experiments 1 and 2: In particular, the contours in Experiment 1 were straight and jitter-free. The centers of the Gabor elements were positioned as follows. First, a path was generated, which formed the invisible “backbone” of the contour. This consisted of a set of eight invisible lines, of length d, called path segments, which were joined end-to-end. The orientation of each segment differed from that of the adjoining one(s) by a randomly selected angle of ( α + Δ α) or (− α + Δ α), where α is a fixed path angle and Δ α is a small random value (the path angle jitter). An element was placed on each path segment, at a distance Δ d from the center of the segment, where Δ d is a small random value (the separation jitter). The element separation is the distance between the centers of adjacent path segments (assuming no path angle jitter). The segment length, d, was set to achieve an element separation, s, of 0.645° visual angle (three times the wavelength of the Gabor Carrier), using the equation, d = s/cos( α/2). Once the contour had been generated, it was shifted so that its midpoint (the junction between the segments containing the fourth and the fifth elements) lay exactly at the center of the display. In Experiment 1 only, the contour was then rotated so that its global orientation was vertical (for left or right fixations) or horizontal (for up or down fixations), plus a small random value between ±10°. The purpose of this constraint was to give a tighter control on the distance of the contour elements from the fixation point. In Experiment 2, there was no constraint on the orientation of the contour as a whole: Because these contours were curved, it is not so straightforward to define what is meant by the “orientation” of the contour. 

Table 1

 

Stimulus parameters. Each jitter value was selected independently for each element or path segment in a stimulus.

Parameter	Value
Stimulus duration	247 ms
Interstimulus interval duration	1000 ms
Number of contour elements	8
Total number of elements	144 (12 × 12 grid)
Michelson contrast	0.5
Carrier spatial frequency (cycles/deg)	4.65
Carrier wavelength, λ (degrees of visual angle)	0.215
σ (degrees of visual angle)	0.108
λ/ σ	2
Separation, s (degrees of visual angle)	0.645
s/ λ	3
s/ σ	6
Path segment length, d (degrees of visual angle)	Experiment 1: 0.645
	Experiment 2: 0.648
Path angle, α	Experiment 1: 0°
	Experiment 2: 10°
Path angle jitter, Δ α	Experiment 1: none
	Experiment 2: uniform probability between ±5°
Separation jitter, Δ d	Experiment 1: none
	Experiment 2: uniform probability between ±0.05 d
Orientation jitter, Δ θ	Experiment 1: none
	Experiment 2: uniform probability between ±20°
Global contour orientation	Experiment 1: random value between ±10° from vertical for left and right fixations, and ±10° from horizontal for up and down fixations.
	Experiment 2: completely random

Figure 2

 

A schematic representation of part of a snake contour, illustrating the different parameters. For illustration purposes, the snake shown here has a higher curvature than used in the experiments, and some of the jitter values are exaggerated. The thick solid lines represent the invisible segments (of length d) that form the backbone of the contour. A Gabor element was positioned at the midpoint of each segment and then shifted along the segment by an amount Δ d _i (the separation jitter for element i). The angle between each segment, i, and the next was equal to ± α (the path angle), plus a random jitter value, Δ α _i. The sign of the path angle was randomly determined for each junction between segments. s is the element separation, that is, the distance between the centers of adjacent path segments, assuming no path angle jitter. For snakes, each element, i, was aligned with its segment before a jitter value, Δ θ _i, was added to its orientation. For ladders, the element was orthogonal to its segment before adding the jitter value. In Experiment 1, the path angle was 0°, and there was no jitter in any of the parameters; in Experiment 2, the path angle was 10°; the jitter values are given in Table 1.

Positioning the distractor elements

An invisible grid (12 rows × 12 columns) was generated, centered on the center of the display. For stimuli without a contour, a distractor element was placed in each grid square; for stimuli with a contour, a distractor element was placed in any grid square not occupied by a contour element. The position of each distractor was determined randomly such that the horizontal and the vertical positions were not more than a quarter of a grid square width from the center of the square. An additional constraint was that the minimum allowed distance between the centers of any two elements was 4 σ, where σ is the standard deviation of the Gabor envelope. Because the elements were quite closely packed, the algorithm would often reach a state in which it was impossible to place the next element in such a way that it was sufficiently far away from other elements that had already been placed. If this situation arose, the algorithm would reject the stimulus and start a new one. 

To avoid a density cue, we calculated the width of the grid squares so that the mean separation between adjacent distractor elements was equal to the element separation, s. If each distractor had been located exactly on the center of the grid square, the mean separation between adjacent distractors would have been (1 + √2) W/2 ≈ 1.207 W, where W is the width of the grid square (Beaudot & Mullen, 2003). However, the jitter in distractor position was large, so we took account of the effect that this would have on the mean distance. It proved difficult to derive a closed-form expression for the mean separation, taking distractor jitter into account, so we generated a million 3 × 3 grids of distractor elements, with positions randomized as described above, and found the mean distance of the central element from the eight surrounding elements. It was found to be 1.225W, so the grid square width was set to s/1.225. Because the diagonal distance across a grid square is 1.414W, there was a small possibility that two contour elements could fall within the same grid square. If this happened, the stimulus was rejected. Thus, in every stimulus, there were exactly 144 elements—one in each grid square. 

Element orientations

For snakes, the orientation, θ, of each contour element was set to ( ϕ + Δ θ), where ϕ was the orientation of that element's path segment, and Δ θ was a random value (the orientation jitter—see Table 1). The orientation of ladder elements was ( ϕ + 90° + Δ θ). The orientation of each distractor element was completely random. 

Modeling

The simulations consisted of three stages. Firstly, all the stimuli used in the experiments were processed with a simple association field algorithm that formed contours by linking pairs of elements. Secondly, each contour in each stimulus was rated according to three criteria: number of elements, straightness, and proximity to the center of the stimulus. ¹ Thirdly, the experiments were simulated, separately for each subject, by taking the two stimuli on each trial and selecting as the target the stimulus that gave rise to the highest-rated contour. 

The association field algorithm

We implemented an association field algorithm in which snake and ladder associations competed directly for ownership of the elements. We do not claim that our algorithm is a realistic model of the cortical processes underlying the association/integration field: As Pelli et al. (2004) made clear, “findings to date provide only hints as to the nature of this computation” (p. 1138). Instead, our simulations should be seen as a concrete confirmation of our earlier intuitive argument that, when weak ladder associations compete with strong snake associations, performance will be good on both types of contour when the association field is small (i.e., in the fovea) but will be much more greatly impaired for ladders than snakes when the association field is large (i.e., in the periphery). 

The inputs to the algorithm were the positional coordinates and the orientations of the Gabor elements, rather than the images themselves. This is because, in Pelli et al.'s (2004) model, the initial processing of the retinal image is performed, not by the integration fields, but by the earlier “feature detection” stage, which we do not address here. It seems that the representations of features in the integration field are somewhat symbolic, on/off representations, rather than the quasi-linear representations found in early vision. One piece of evidence for this is that introducing very large variability in the contrasts of the stimulus elements has a minimal effect on contour detection (Hess, Dakin, & Field, 1998). Additional evidence is that the function relating mask contrast to magnitude of crowding is similar to the psychometric function for letter identification performance as a function of contrast (Pelli et al., 2004, Figure 11); this suggests that whatever the target-mask spacing, as long as the mask is high-contrast enough to be identified, it produces the same crowding effect as a mask of maximum contrast. 

For each element, the association field algorithm assessed the strength of the snake and the ladder associations between that element and every other element. The strength of the association between any two elements was calculated by multiplying together three scores, M, P, and C. M is a scaling factor that causes snake and ladder associations to have different strengths. For snake associations, M was set to 1, and for ladder associations, M was less than 1, the exact value being a free parameter fitted to the data of each individual subject. P is a measure of the spatial proximity of the elements. C scores the configuration of the pair of elements and measures the extent to which the pair of elements forms part of a perfectly straight snake or ladder contour. 

The spatial proximity score, P, varies as a cosine function of the separation between the elements, S. If the elements in a pair are separated by a distance, S, and the spatial radius of the association field is r, then P for that pair of elements is given by  

$P=\{\begin{array}{cc}\cos (\pi S/(2r\left)\right)\hfill & S\le r\\ 0\hfill & S>r\end{array}$

(4)

This function is plotted in Figure 3. 

Figure 3

 

The function relating the proximity score, P, to the separation between the elements, S. r is the spatial radius of the association field.

It is clear from Equation 4 and Figure 3 that the proximity score for a pair of elements depends on the size of the association field that is used to associate that pair of elements. To explain how the association field radius, r, was determined for each pair of elements, we need to look at Pelli et al.'s (2004) model in more detail. They proposed that, at each point in the visual field, there is a range of available integration field sizes, and the observer will attempt to choose an integration field size that is appropriate for the task at hand. In our model, there is a free parameter (fitted separately to each individual subject's data) called the “preferred association field radius”; this parameter specifies the association field radius that the model will choose, if available (we would expect this parameter to be task and stimulus dependent but, because all our stimuli had the same element separation, we were able to fit this parameter across all conditions of both experiments). Pelli et al.'s critical proposal was that the smallest available association field radius at each retinal location is proportional to the eccentricity so that, in the periphery, the preferred association field radius may not be available; in this case, the model will choose the smallest available field radius. The model therefore requires a further parameter, k, which is the rate at which the smallest available field radius increases with eccentricity: k is multiplied by the eccentricity to give the smallest available field radius. Rather than fitting k to our data, we gave it a value of 0.5 because the critical spacing for crowding to occur has already been found to be about half the eccentricity (Bouma, 1970; Pelli et al., 2004; Toet & Levi, 1992). At locations for which the preferred radius was greater than half the eccentricity, the model used the preferred association field radius; at all other locations, the association field radius was set to half the eccentricity. This meant that, for a given pair of elements, the association fields centered on the two elements would usually have different sizes. When calculating the association strength between two elements, the model always used the larger of the two association fields. 

The configuration score, C, was a function of the extent to which the pair of elements corresponded to a perfect snake or ladder configuration. This was evaluated as follows. First, the algorithm found the orientation difference between each of the two elements and the line joining them (see Figure 4); these two angles, β ₁ and β ₂, were expressed as positive values between 0° and 90°, and their average,

$\overline{\beta }$

, was found. The ideal value of

$\overline{\beta }$

(labeled β ₀) is 0° for snake associations and 90° for ladder associations. The configuration score, C, varied as a cosine function of the deviation of

$\overline{\beta }$

from the ideal value:  

$C=\{\begin{array}{cc}[\cos \left(8|\overline{\beta }-{\beta }_0|\right)+1]/2\hfill & |\overline{\beta }-{\beta }_0|\le {22.5}^{\circ }\\ 0\hfill & |\overline{\beta }-{\beta }_0|>{22.5}^{\circ }\end{array}$

(5)

This function is plotted in Figure 5. 

Figure 4

 

(A) A perfect snake configuration: β ₁ and β ₂ (not shown) are both 0°. (B) A configuration that is neither a perfect snake nor a perfect ladder. (C) A perfect ladder configuration: β ₁ and β ₂ are both 90°.

Figure 5

 

The configuration score ( C) for snake associations (red line) and ladder associations (blue line) as a function of β ¯.

For each possible pair of elements, the algorithm found the strength of the snake and the ladder associations between them, by multiplying together the scores M, P, and C. All the associations were then sorted into decreasing order of association strength (regardless of whether they were snake or ladder associations). Starting with the strongest association, the algorithm worked its way down the list; for each association, a link of the appropriate type (snake or ladder) was inserted between the associated elements as long as none of the following rules were broken:

1.  
   an element can be linked to a maximum of two other elements;
2.  
   if an element is linked to two others, the change of direction at the junction between the two links should be no more than 70° (this follows from Field et al.'s, 1993, finding that performance is always at or very close to chance for path angles beyond this value);
3.  
   if an element is linked to two others, both links must be of the same type (snake or ladder).

The algorithm continued to insert links according to these rules until the association strength was less than 1% of the maximum for that stimulus. Because the M score was higher for snakes, the snake-type links tended to be inserted first, preventing the linked elements from forming ladder-type links.

After the links between elements had been inserted, contours were formed by taking elements with only one link, and tracing along the links from element to element until reaching another element with just one link. 

Rating the contours

Each contour was given three scores (between 0 and approximately 1) reflecting its length, straightness, and proximity to the center of the stimulus. These scores were then multiplied together to give a single rating. Thus, a contour would have to have a high value on all three scores to get a good rating. 

The score for length was given by n/8, where n is the number of elements in the contour. Thus, the shortest possible contour (two elements) had a score of 0.25, and one with eight elements had a score of 1. Because the target contours all had eight elements, a perfectly integrated target would achieve a score of 1, but the score could be higher if a distractor element was linked to the contour. 

The score for straightness was given by (70 − γ)/70, where γ was the mean orientation difference (in degrees) between adjacent links along the contour. Seventy degrees was the maximum possible value of γ because the association field stage would not allow an element to have links that differed in orientation by more than 70°. 

The score for proximity to the center of the stimulus was given by ( D − R)/ D. R is the distance between the center of the stimulus and the element in the contour that was closest to the center. D is half the diagonal distance across the invisible 12 × 12 grid into which the Gabor elements were inserted; this value (4.47° visual angle) was the maximum possible value of R. 

Simulating the experiments

For each subject, the model was fitted to that subject's full set of percent-correct scores from Experiments 1 and 2 (BCH and KAM) or Experiment 1 only (BT and DHB). The experiments were simulated separately for each subject, using the stimuli that that subject had seen during the experiments (for modeling purposes, rather than using the actual stimulus images, we used the sets of element positions and orientations from which those images had been generated). For each trial, we ran the model on the two stimuli that the subject had seen on that trial and selected as the target the stimulus with the highest-rated contour. The model's responses were classified as correct or incorrect to obtain a percent-correct score for each condition. The two free parameters (the preferred association field radius and the M score for ladders, which sets the strength of ladder associations relative to snake associations) were varied to find the best fit to each subject's data set. 

Results

Psychophysical data

In both experiments, the data from the four sessions (up, down, left, and right fixation) on each eccentricity and contour type were pooled to give 200 trials per condition. The results of Experiments 1 and 2 are shown in Tables 2 and 3, respectively, and Figures 6 and 7. 

Table 2

 

Results of Experiment 1. The snake and ladder columns give the percent correct for each condition. The p values in these columns are the results of one-tailed binomial tests of the null hypothesis that the subject was equally likely to be correct or incorrect on each trial. Asterisks indicate significant p values. The criterion of significant for each binomial test was .0085, giving a per-experiment criterion of .05. The chi-square test column gives the results of the chi-square tests comparing performance on snakes and ladders at each eccentricity. The criterion of significance on each of these tests was .0170. Again, asterisks indicate significant p values.

Subject	Eccentricity	Snake	Ladder	Chi-square test
BCH	0	99.5 ( p = 1.3 × 10 −58)*	99.5 ( p = 1.3 × 10 −58)*	χ 2 = 0 ( p = 1)
	4	99.5 ( p = 1.3 × 10 −58)*	81 ( p = 9.9 × 10 −20)*	χ 2 = 38.9 ( p = 4.5 × 10 −10)*
	6	99.5 ( p = 1.3 × 10 −58)*	52 ( p = .31)	χ 2 = 123 ( p < 2 × 10 −16)*
BT	0	99 ( p = 1.3 × 10 −56)*	98 ( p = 4.1 × 10 −53)*	χ 2 = 0.677 ( p = .41)
	4	98 ( p = 4.1 × 10 −53)*	59 ( p = .0066)*	χ 2 = 90.1 ( p < 2 × 10 −16)*
	6	97.5 ( p = 1.6 × 10 −51)*	45.5 ( p = .11)	χ 2 = 133 ( p < 2 × 10 −16)*
DHB	0	100 ( p = 6.2 × 10 −61)*	100 ( p = 6.2 × 10 −61)*	-
	4	100 ( p = 6.2 × 10 −61)*	57.5 ( p = .02)	χ 2 = 108 ( p < 2 × 10 −16)*
	6	99.5 ( p = 1.3 × 10 −58)*	52.5 ( p = .26)	χ 2 = 121 ( p < 2 × 10 −16)*
KAM	0	99.5 ( p = 1.3 × 10 −58)*	100 ( p = 6.2 × 10 −61)*	χ 2 = 1.00 ( p = .32)
	4	100 ( p = 6.2 × 10 −61)*	50.5 ( p = .47)	χ 2 = 132 ( p < 2 × 10 −16)*
	6	100 ( p = 6.2 × 10 −61)*	53 ( p = .22)	χ 2 = 123 ( p < 2 × 10 −16)*

Table 3

 

Results of Experiment 2, presented in the same way as for Experiment 1 ( Table 2). The criterion of significance was .0064 for each binomial test and .0127 for each chi-square test.

Subject	Eccentricity	Snake	Ladder	Chi-square test
BCH	0	82.5 ( p = 1.1 × 10 −21)*	81.5 ( p = 2.3 × 10 −20)*	χ 2 = 0.0678 ( p = .79)
	4	70 ( p = 7.5 × 10 −9)*	56 ( p = .052)	χ 2 = 8.41 ( p = .0037)*
	6	65.5 ( p = 6.9 × 10 −6)*	55.5 ( p = .069)	χ 2 = 4.18 ( p = .041)
	8	63 ( p = 1.4 × 10 −4)*	53.5 ( p = .18)	χ 2 = 3.71 ( p = .054)
KAM	0	92 ( p = 1.2 × 10 −37)*	93 ( p = 7.9 × 10 −40)*	χ 2 = 0.144 ( p = .70)
	4	71.5 ( p = 5.1 × 10 −10)*	55 ( p = .089)	χ 2 = 11.7 ( p = 6.2 × 10 −4)*
	6	73 ( p = 2.8 × 10 −11)*	51 ( p = .42)	χ 2 = 20.5 ( p = 5.8 × 10 −6)*
	8	75 ( p = 4.2 × 10 −13)*	51.5 ( p = .36)	χ 2 = 23.8 ( p = 1.1 × 10 −6)*

Figure 6

 

Results of Experiment 1. The red and blue symbols show the psychophysical data (given in Table 2) from snake and ladder conditions, respectively; filled symbols represent scores that are significantly different from chance, and open symbols represent chance-level performance. The red and the blue lines show the performance of the best-fitting models.

Figure 7

 

Results of Experiment 2, plotted in the same way as for Experiment 1 ( Figure 6).

Two kinds of statistical analysis were performed on the data. One analysis was a one-tailed binomial test of the null hypothesis that the subjects were equally likely to be correct or incorrect on each trial. A one-tailed test was used because we would not expect performance to be below chance. This test was carried out for each individual condition, and the p values are shown in Tables 2 and 3. There were multiple tests for each subject, and the criterion of significance for each test, α _t, was chosen such that the type I error probability for the whole experiment, α _e, was equal to .05. If there are n tests, then  

${\alpha }_t=1-(1-{\alpha }_e{)}^{1/n}.$

(6)

There were six tests per subject in Experiment 1 and eight tests per subject in Experiment 2, which gave a criterion of significance of .0085 for each individual test in Experiment 1 and .0064 in Experiment 2. 

In the other analysis, we compared performance on snakes and ladders at each eccentricity, using a method suggested by Lovell (2002). For each eccentricity, each trial was classified according to two variables: snake/ladder and correct/incorrect. This yielded a 2 × 2 contingency table, on which a chi-square test could be performed to see if the contour type had had a significant effect on the proportion of correct responses. Separate chi-square tests were carried out at each eccentricity for each subject. There were three chi-square tests per subject in Experiment 1, and four in Experiment 2; using Equation 6, this give criteria of significance per test of .0170 and .0127 in Experiments 1 and 2, respectively. 

Experiment 1

In Experiment 1 ( Table 2, Figure 6), the contours were completely straight. Performance on snakes was at or close to ceiling at all eccentricities. Performance on ladders was at ceiling in the foveal condition but was not significantly different from chance at the 6° eccentricity for any of the subjects. At the 4° eccentricity, only two subjects showed ladder detection performance that was significantly better than chance. The chi-square tests showed that, on all nonfoveal conditions, performance on snakes was significantly better than ladders for all subjects. 

Although the effects of peripheral viewing were very different for snakes and ladders, Experiment 1 does not provide unequivocal support for the view that detection of ladders is disproportionately impaired in the periphery because performance on both contour types was at ceiling in the fovea. There might instead have been a general impairment for ladders, which only showed up in the periphery, where the performance on ladders fell away from ceiling. To rule out this possibility, subjects BCH and KAM also performed Experiment 2, in which the contours were jittered to bring foveal performance away from ceiling. 

Experiment 2

For both subjects in Experiment 2, the performance levels on snakes and ladders were very closely matched away from ceiling in the foveal condition but diverged in the eccentric conditions (see Table 3 and Figure 7). These results suggest that detection of ladder contours is indeed disproportionately impaired in the periphery, compared with snakes. Performance on ladders dropped to chance level at 4° eccentricity, whereas performance on snakes remained significantly above chance at all eccentricities. The chi-square tests showed that, for KAM, snake detection was significantly better than ladder detection at all eccentricities except the foveal condition. For BCH, only the 4° eccentricity showed a significant difference: Although snake detection performance at the 6° and 8° eccentricities was significantly above chance (unlike ladder detection), it was not quite high enough to be statistically different from the ladder detection performance on these conditions. 

Modeling

The best-fitting parameter values for each subject are given in Table 4. The model's performance is shown in Figures 6 and 7; the numerical values are given in the auxiliary file, SimulationResults.txt. The model fitted very well to the data from Experiment 1. Performance on snakes was 100% correct at each eccentricity; performance on ladders was close to 100% correct in the fovea but dropped quickly to chance in the periphery. The model fitted less well to the data from Experiment 2 but still showed the right kind of pattern: Performance was quite closely matched for snakes and ladders in the fovea (although not as closely matched as in the psychophysical data); in the periphery, performance on ladders dropped to chance at 4° eccentricity, whereas performance on snakes dropped more gradually. For snakes, the model's performance fell with increasing eccentricity at a very similar rate to BCH's data, although BCH consistently performed worse than the model. 

Table 4

 

Best-fitting parameters and root mean square (RMS) error across Experiments 1 and 2 (BCH and KAM) or Experiment 1 only (BT and DHB). The M score for ladders sets the strength of ladder associations relative to snake associations.

Subject	M score for ladders	Preferred association field radius (°)	RMS error in % correct scores
BCH	0.61	0.87	7.90
BT	0.475	0.775	1.79
DHB	0.44	0.74	1.66
KAM	0.57	0.83	6.79

To give an insight into the workings of the model, Figure 8 shows the results of processing the example stimuli shown in Figure 1. In the fovea, where the association fields are small, the target contours all get higher ratings than the spurious background contours. In the periphery, where the association fields are larger, the ladder associations cannot compete with spurious snake associations, and the target ladder contours are not detected. 

Figure 8

 

The results of processing stimuli A, B, C, and D of Figure 1 with our association field algorithm, with parameters that fitted best to the data from subject K. A. M. Snake contours are shown in red, and ladder contours are shown in blue. The width of each contour's line is proportional to that contour's rating. The left half of this figure shows the results when the image is centered on the fovea: The snake and the ladder target contours are all detected. The right half of the figure shows the results when the image is centered 6° to the right of fixation. Because of the larger association fields, many more spurious contours are formed. The target snake contours still have higher ratings than the spurious contours, whereas the target ladder contours are not detected.

A demonstration of crowding

The central argument in this paper is that the same kind of mechanism underlies both contour integration and crowding. To demonstrate that our contour integration model shows crowding-like behavior, we applied our model to letter stimuli, composed from short line segments (shown in Figure 9). The results are shown in Figure 10. We assumed that crowding had occurred if contours had formed between the letters, joining them together. The model correctly predicts three key characteristics of crowding, identified by Pelli et al. (2004): The critical spacing for crowding to occur is independent of the size of the target letter, scales with eccentricity, and is greater on the peripheral side of the target. 

Figure 9

 

The top row (in black) shows the letter stimuli used in our crowing demonstration; the bottom row (in red) shows the contours that the model detected in these letters when they were presented in the fovea. Only snake contours were detected.

Figure 10

 

Contours formed when we applied our association field algorithm to letter stimuli, composed of short line segments. The letters had a height of 1° visual angle (panels A and C) or 0.5° (panels B and D). They were separated by 2°, 4°, or 8°. Each row of letters E, F, and A was presented separately to the model. The center of the letter “F” was positioned 4° to the right of fixation (panels A and B) or 8° to the right (panels C and D). The model was the same as in Figure 8, except that the preferred association field size was set to 0.1° to be more appropriate for perception of letters of these sizes. The critical spacing for crowding to occur is independent of the size of the target letter, scales with eccentricity, and is greater on the peripheral side of the target. Panels E–H are the same as panels A–D, except that the identities of the two flanking letters have been swapped. These simulations confirm that it is the positions of the flankers, not their identities, that gave rise to the patterns of crowding shown in panels A–D.

The critical spacing has been found to be about half the eccentricity (Bouma, 1970; Pelli et al., 2004; Toet & Levi, 1992). Our model does show some crowding at larger spacings than this, but in these cases, interference is only caused by the more peripheral flanker. A single flanker is much less effective at producing crowding than a pair of flankers (Pelli et al., 2004), presumably because the letter identification mechanisms are reasonably tolerant to interference from just one side. Thus, we might predict that the crowding effect from the peripheral flanker alone would be relatively small: Crowding would be more likely to be detected when both flankers interfered with the target and, in our model, this occurs when the spacing is half the eccentricity, as found psychophysically. 

Discussion

Experiment 1 showed that straight ladders are easy to detect in the fovea but are undetectable at eccentricities of around 4° to 6° visual angle. In contrast, performance on straight snakes was at or close to 100% correct at all the eccentricities that we tested. Experiment 2 showed that this difference between snakes and ladders did not result simply from a general deficit in detection of ladders: When snakes and ladders were matched below ceiling in the fovea, ladder detection fell to chance at an eccentricity of 4°, whereas snake detection remained significantly above chance up to the highest eccentricity that we tested (8°). The most striking comparison is perhaps between straight ladders ( Experiment 1) and jittered snakes ( Experiment 2). Performance on jittered snakes was above chance at each eccentricity; straight ladders were much easier to detect than jittered snakes in the fovea but were undetectable in the periphery. 

Our results may partly explain the relative difficulty in detecting ladders that has been reported in previous studies (Bex et al., 2001; Field et al., 1993; Hess et al., 2000; Ledgeway et al., 2005): In all of these studies, the position of the contour was randomized to some extent. The difference in the effect of eccentric viewing on snakes and ladders means that the positional randomization would have caused a greater disruption to detection of ladders. Experiment 2 shows that performance on snakes and ladders can be very similar if the contours are fairly straight and positioned exactly at the fixation point. 

We were surprised to find such a severe deficit of ladder detection at such small eccentricities. Ladders would be expected to give rise to extended regions of response in a “second-order” or “non-Fourier” channel, of the kind proposed in many models of texture segregation (Graham, 1991; Graham, Beck, & Sutter, 1992; Graham & Sutter, 1998; Graham, Sutter, & Venkatesan, 1993; Lin & Wilson, 1996; Sutter, Beck, & Graham, 1989; Wilson, 1993). In these models, a small-scale linear filter is followed by a nonlinearity (usually squaring or full-wave rectification), followed by a large-scale linear filter with orientation orthogonal to that of the first filter. Such a mechanism gives a strong response at a texture border (such as a border between two areas of different orientation) but a weak response elsewhere. This kind of mechanism could be used to detect fairly straight ladders: A small-scale filter aligned with the elements will give strong positive and negative responses along the path of the contour; if these responses are rectified, then a large-scale filter orthogonal to the small-scale filter will give a positive response along the whole of the contour. The periphery is known to contain many neurons that respond to second-order signals (Zhou & Baker, 1994, 1996), and second-order signals are certainly detectable in the periphery (Hess, Baker, May, & Wang, 2007), so we might have expected subjects to be able to detect peripheral ladder contours. 

Our preferred explanation is that the failure to detect ladders in the periphery is a form of crowding. Pelli et al. (2004) proposed that crowding is caused by inappropriate feature integration by large integration fields in the periphery. We have shown that the same kind of mechanism can account for our results. We simulated our experiments with an association field model in which the minimum association field size at each retinal location was half the eccentricity, as suggested by the crowding literature. In our model, the only difference between the processing of snakes and ladders was that ladder associations were about half as strong as snake associations (see Table 4). It may be that future work will reveal further differences between the processing of snakes and ladders, but our simulations show that, if the association field size scales with eccentricity, a difference in strength of snake and ladder associations is sufficient to explain the very different effects of peripheral viewing on these two types of contour. 

Our current lack of knowledge about the mechanisms of crowding makes it difficult to provide direct evidence that crowding causes the poor performance on ladders in the periphery. However, we feel that the indirect evidence that we have presented here is reasonably strong. Our model explained the results of our contour integration experiments and also showed three key characteristics of crowding with letter stimuli: Critical spacing was independent of the size of the target letter, scaled with eccentricity, and was greater on the peripheral side of the target. Our proposal also fits with the subjective experience of viewing contour stimuli in the periphery: As with crowding, it is easy to detect the presence of the elements within a peripheral ladder stimulus but difficult to interpret them. 

Our model fits very well to the data from Experiment 1 but deviates a little from those of Experiment 2. The main problem is that the model tended to overestimate the performance level on snakes in Experiment 2. This may have arisen for two reasons: (1) In the experiment, the stimuli were presented for a short time (247 ms), to minimize eye-movements, and this may have given the subjects insufficient time to process the stimuli fully; in contrast, the computer model had as much time as it needed. (2) No noise was implemented in the model; the orientation and position of each Gabor element were assumed to be represented with perfect accuracy, whereas a more realistic model would add noise to these representations, reducing the model's performance. 

In the Introduction, we outlined two possible types of association field model. In one type, snake and ladder contours are integrated using completely independent mechanisms; in the other, snake and ladder associations compete directly for ownership of the elements. In our model, it is the competition between snakes and ladders that leads to the very poor performance on ladders in the periphery. If we removed this competitive aspect so that snakes and ladders were processed by independent mechanisms, then the model would detect snakes and ladders equally well. This is because the model relies on relative strength of associations rather than the absolute values: Scaling all the ladder association strengths by the same factor would have no effect if ladders were processed by an independent mechanism. Our data therefore favor association field models in which snake and ladder associations are in direct competition. 

Our model of peripheral contour integration is somewhat different from the model proposed by Hess and Dakin (1997, 1999). Hess and Dakin proposed that there is an absence of explicit contour linking in the periphery, and that contour integration occurs due to the spatial overlap of filter responses within individual orientation channels. This model was rejected by Lovell (2002, 2005): He showed that the performance of Hess and Dakin's model increases with the number of points of inflection in a contour, whereas human observers generally show the opposite effect in both fovea and periphery. Our current proposal is that explicit contour linking is present in the periphery but occurs over a wider area, allowing more distractors to interfere with the process. In these circumstances, contour integration would be consistently successful only in the most favorable conditions, that is, when the contour was fairly straight and the elements were collinear. 

Our current model would not show an effect of the number of points of inflection. This behavior could easily be added to the model, but we did not do so here because this would have added extra parameters which would not have been constrained by our data: The contours in Experiment 1 were perfectly straight, so they had no points of inflection, and those in Experiment 2 had a path angle that was too small to give rise to a large effect of the number of points of inflection. 

In this paper, we have not addressed the issue of the effect of phase alternation on the detection of peripheral contours. Given the very different results obtained by Hess and Dakin (1997, 1999) and Nugent et al. (2003), we feel that this issue can only be resolved by a large study focusing on this issue. Our model is agnostic about the effects of phase because phase is not represented in the model: Only the position and orientation of each Gabor element are represented. It would probably be possible to accommodate the apparent individual differences in the effect of phase alternation by introducing a parameter that controlled the relative strengths of associations between same-phase and opposite-phase elements. 

Our results shed some light on the computations performed by the integration/association fields. Firstly, our results suggest that the associations between collinear elements are stronger than those between parallel elements. Secondly, these two types of association appear to be in direct competition with each other, suggesting that they are processed by the same mechanism, rather than independent mechanisms (the view that snakes and ladders are processed by the same mechanism is also supported by May & Hess's, 2007, finding that snakes and ladders are integrated at similar speeds). Thirdly, our results support Pelli et al.'s (2004) proposal that, at each point in the visual field, the minimum integration/association field size is proportional to eccentricity. Because of the large size of contour stimuli, future models of contour integration should take into account the scaling of association field size with eccentricity, even when modeling the detection of foveally presented contours. It is hoped that future work delineating the successes and failures of feature integration in the periphery will reveal more details about the integration/association fields, which appear to play a crucial role in any visual task more complex than detection of basic features. 

Conclusions

Straight ladder contours were easily detectable in the fovea but were undetectable at eccentricities of around 4° to 6°. Performance on straight snakes was at ceiling at all eccentricities that we tested. An association field model can account for these results if it has three key properties: (1) Snake associations are stronger than ladder associations; (2) snake and ladder associations compete against each other for ownership of the elements, rather than being processed by independent mechanisms; (3) the minimum association field size is proportional to the eccentricity. The last of these properties has been proposed to account for the phenomenon of crowding: Large integration fields in the periphery cause inappropriate integration of features from different parts of the stimulus (Pelli et al., 2004). We have shown that the same kind of process could underlie the failure to detect ladders in the periphery. We conclude that the failure to detect peripheral ladders is a form of crowding. 

Supplementary Materials

Supplementary Figure - Supplementary Figure 

Figure S1. 

Acknowledgments

This work was supported by NSERC Grant no. RGPIN 46528-06. 

Commercial relationships: none. 

Corresponding author: Keith A. May. 

Email: [e-mail]. 

Address: Vision Sciences Research Group, School of Life Sciences, University of Bradford, Richmond Road, Bradford, West Yorkshire, BD7 1DP, UK. 

Footnote

Footnotes

¹  The first two criteria were based on the empirical finding that that performance improves with increasing contour length (Dakin & Hess, 1998) and straightness (Field et al. (1993); the third criterion was included because the subjects were told that the contours passed through the center, so they could use this knowledge to perform the task.