Context influences contour integration

Abstract

Much research over the last decade has examined how the brain links local activity within primary visual cortex to signal the presence of extended global structure. Here we bring together two themes within this area by addressing how the immediate context that features arise in influences how they are integrated into contours. Specifically, observers were required to detect and discriminate the shape of contours that were surrounded by elements with a fixed orientation offset compared to contour elements. By comparing performance with contours made of elements oriented either near parallel (“snakes”) or near perpendicular (“ladders”) to the contour orientation, we were able to isolate the effect of orientation contrast on observers' ability to perform our task with near-collinear contour structure. We report both substantial facilitation of contour integration in the presence of near-perpendicular surrounds and inhibition in the presence of near-parallel surrounds. These results are consistent with known orientation dependence of suppressive surround interactions in the primary visual cortex and suggest that the “rules of association” for contour integration must incorporate the influence of local orientation context. Specifically we show that our results are consistent with contour integration relying on an opponent-orientation energy response from a bank of first-stage oriented filters.

Introduction

Artists have long known that much of the important information within complex natural visual scenes is contained within visual contours. Inasmuch as contours convey the shape and identity of objects, our ability to detect them must underpin object recognition and, by extension, support much of our visually guided behavior. Because neurons in early visual areas primarily signal information within a limited area of visual space known as their classical receptive field (CRF), it is becoming clear that the process by which the visual system links these local responses to signal the presence of spatially extensive structure (e.g., contours) is central to understanding spatial vision (e.g., Hess & Field, 1999). Over the last decade, two areas of research have been largely responsible for illuminating how the visual system solves this problem. The first is concerned with how a visual neuron's response can be influenced by structure falling outside of its classical receptive field; this is known as contextual modulation (Gilbert & Wiesel, 1990; Kapadia, Ito, Gilbert, & Westheimer, 1995; Knierim & van Essen, 1992; Nelson & Frost, 1978; Nothdurft, Gallant, & Van Essen, 1999; Sillito, Grieve, Jones, Cudeiro, & Davis, 1995; Zipser, Lamme, & Schiller, 1996). The second is concerned with grouping or binding of features and, in particular, how features are linked together across space into elongated contours: this is contour integration (for a review see Hess & Field, 1999). This paper is concerned with how these two processes interact. 

The visual context a feature arises in can profoundly affect the response of neurons. More than 90% of neurons in primary visual cortex are inhibited by the activation of their neighbors (Jones, Grieve, Wang, & Sillito, 2001). This phenomenon—termed surround inhibition—is generally thought to enhance salient visual structure (supporting phenomena such as pop-out, etc.). For example Nothdurft et al. ( 1999) report that the response of many neurons to a line falling in their classical receptive fields (CRF) is suppressed when extra identical lines are introduced outside the CRF, but that activity returns to normal when those flanking lines are rotated through 90°. Several lines of psychophysical inquiry bear on this issue. For example Sagi ( 1990) has shown that the detection of an orientation singularity in a field of identical elements shows a non-monotonic dependence on distracter density (with the addition of a modest number of distracters improving performance) and a dependence on the distance from distracters to the target. Sagi interprets these results as evidence for specialized mechanisms that signal salient orientation discontinuities (“second-stage filters”) that could be supported by the type of contextual modulation reported earlier. A second relevant line of psychophysical inquiry is concerned with masking and facilitation of low-contrast features. Polat and Sagi ( 1993) report that the detection of a Gabor target can be facilitated when it is flanked by two similar Gabors so that their overall configuration is consistent with the presence of a contour. Extending this work to electrophysiology, a similar mixture of surround suppression and facilitation has been reported when features are consistent with isolated contours or with fields of texture, respectively (Kapadia et al., 1995; Polat, Mizobe, Pettet, Kasamatsu, & Norcia, 1998). 

Modern research on integration has exploited the development of effective psychophysical paradigms for examining contour linking. In particular, Field, Hayes, and Hess ( 1993) developed the ‘path’ paradigm where subjects are required to detect contours consisting of locally co-aligned Gabor elements, embedded in a background made up of identical, randomly oriented elements ( Figures 1a and 1b). In order to distinguish the contour from the background, the individual contour elements must be integrated into a global whole; large receptive fields will not suffice (Hess & Dakin, 1997). Various studies have used this paradigm in order to characterize contour integration. Contour integration performance is best for straight contours and gets worse as the curvature of the contour increases (Dakin & Hess, 1998; Field et al., 1993; Geisler, Perry, Super, & Gallogly, 2001; Hess & Dakin, 1997). Subjects are best at detecting contours containing smooth changes in orientation (Dakin & Hess, 1998; Pettet, 1999; Pettet, McKee, & Grzywacz, 1998). They are relatively insensitive to the contrast (Hess, Dakin, & Field, 1998) and contrast polarity of contour elements (Field et al., 1993; Hess & Dakin, 1997) but are sensitive to their spatial frequency (Dakin & Hess, 1998), degree of phase alignment (Dakin & Hess, 1999), and depth (Hess & Field, 1995). Performance is also affected by the orientation of the contour elements (independent of the contour orientation). Contour integration is best with elements that match the local contour orientation (Field et al., 1993); elements oriented perpendicular to the contour elicit relatively poor performance (Field et al., 1993), although surprisingly poorest performance is obtained with orientation differences of 45° (Ledgeway, Hess, & Geisler, 2005). This dependence on element orientation is important; a contour composed of, e.g., locally orthogonally oriented elements is statistically identical to a co-aligned contour. The devastating effect of this manipulation on performance confirms the psychological validity of the paradigm. 

Figure 1

 

The path paradigm (Field et al., 1993). (a) An isolated contour with 10° orientation difference between successive elements. (b) The same stimulus embedded in a background of randomly oriented elements. The first and last elements of the contour are shaded to assist the reader in finding the contour. (c) As (b) with contour elements rotated 90° to the contour orientation; notice how much harder the contour is to detect. (d) The “association field”. Contours are thought to be signaled by the presence of long-range facilitatory connections (thick red lines) between neurons signaling contour-consistent orientations while neurons with inconsistent orientations are inhibited (dashed blue lines). The contribution of orientation structure in the immediate vicinity (dotted green lines) is unknown.

Field et al. ( 1993) have interpreted these findings in terms of an “association field” model ( Figure 1d). Connections between neurons with locations and orientations that are mutually consistent with the presence of a contour have facilitatory connections, while those with inconsistent orientations will tend to inhibit one another. The neural substrate for this scheme is thought to be long-range horizontal connections in primary visual cortex (Gilbert & Wiesel, 1989) that allow neurons to exert influence well beyond the confines of their classical receptive field. Consistent with the association-field model, such connections are thought to be strongest between cells selective for like orientations, with spatially offset receptive fields (Bosking, Zhang, Schofield, & Fitzpatrick, 1997; Malach, Amir, Harel, & Grinvald, 1993). Furthermore, these connections extend further along the axis of the cell's preferred orientation (Bosking et al., 1997). However it is worth noting that elegant combined anatomical and physiological studies have shown while the scale of horizontal connectivity is sufficient to account for contextual modulation of V1 receptive fields they are insufficient to account for the spatial range of local–global integration as observed, e.g., in contour integration (Angelucci et al., 2002). Rather these authors conclude “The spatial scale of feedback circuits from extrastriate cortex to V1 … (and)… could represent an anatomical substrate for contextual modulation and global-to-local integration of visual signals.” 

From a computational standpoint, this approach has inspired more detailed models of contour completion in complex images (Yen & Finkel, 1998). Furthermore, if the visual system has evolved to process visual information with maximum efficiency then one might expect that the factors determining the strength of association should to be reflected in the statistical properties of contours in natural scenes. This has found to be the case. Pairs of local-orientation estimates are more likely to originate from the same contour if they are co-circular (tangent to the same circle; Geisler et al., 2001). Thus, the principles of contour integration as encapsulated in the association field model are broadly in line with the statistical properties of natural scenes. 

To summarize so far, the response of neurons in primary visual cortex is determined not just by features falling in their classical receptive field but also by the response of other cells in their vicinity. Their response is frequently suppressed by activity of nearby neurons with similar orientations but can be enhanced by the activity of neurons whose orientations are consistent with the presence of an underlying contour. The latter findings are consistent with a body of behavioral evidence examining contour integration, but nothing is known whether the former local contextual effects interact with integration. Figure 1d summarizes the issue; the contribution of neurons connected to the central element with green dotted lines is unknown. Why might this structure be important? Previous studies have investigated contour integration by manipulating the contour itself, with the background being usually kept constant. However, day-to-day the visual system must segregate contours from a variety of backgrounds so that the context surrounding a contour may have a significant influence on our ability to detect it. In previous studies of the statistics of contour structure in natural scenes (Geisler et al., 2001) contours were isolated from the background so that the contribution of local contextual cues was effectively minimized. In this paper, we quantify the importance of the background on contour integration by modifying the traditional path paradigm to allow systematic manipulation of the orientation context surrounding the contour. 

One potential problem with this approach is that the difference between the orientation of the surround and the orientation of the contour may provide ‘pop-out’ cues (Sagi, 1990; Treisman & Gelade, 1980). If the contour elements have a different orientation to their surround, this may automatically capture the subjects' attention. In order to quantify the effect of the background on contour integration it is necessary to isolate the influence of pop-out. To this end, we use two types of contours: ‘snakes,’ where contour elements are aligned parallel to local contour direction; and ‘ladders,’ where contour elements are orthogonal to local contour direction. Because pop-out cues can be exactly matched between snakes and ladders, by comparing performance between both conditions, it is possible to factor out any general advantage conferred by pop-out. 

Methods

Stimuli

Stimuli consisted of sequences of seven spatial-frequency band-pass Gabor micro-patterns whose orientations were set according to an underlying contour “spine,” embedded in a field of distracter Gabors ( Figure 2b; Field et al., 1993). All elements were in sine phase (positive contrast-polarity centre), had a peak spatial frequency of 3.75 pixels c/deg with an envelope SD of 5.7 arc min, and were presented at maximum contrast. Inter-element separation was on average 40 arc min and stimuli subtended 12.8 deg. square containing on average 220 elements ( SD 3.9 elements). 

Figure 2

 

(a) The two possible path configurations. Starting from a randomly oriented element, the orientation of subsequent elements in contours fell either 15° clockwise (+) or anticlockwise (−) from the orientation of the last element in one of the two sequences indicated. (b) Details of the procedure for positioning distracter elements. Every time an element is dropped, a circular region (the drop zone; radius 2 r _stim + d _min and colored red) is cut from an image whose pixels (colored green) indicate possible drop locations. In the example shown (which had the minimum inter-element distance for the given drop-zone size, i.e., a d _min of 0) note that as long as a new patch's center (blue crosses) falls within the green zone it can never overlap an existing element. Increases in spacing are achieved by increasing d _min. Paths generated as in (a) were randomly positioned in an image and a field of distracter elements dropped around it using the procedure shown in (b), to generate (c) the stimulus. (d) Subjects were then presented with the two possible path configurations (with the same orientation used in the stimulus) and indicated which contour they had just seen.

We modified the standard contour integration paradigm to ensure that subjects had to determine the shape of the contour to successfully perform the task. We did this for two reasons. First, this ensures that the subject saw a long sequence of contour elements (rather than just, say a dual-element sequence). Second, we could not use a standard detection paradigm (i.e., presence-versus-absence of the contour) since we intended to manipulate the statistical properties of the surrounding distracter elements and this manipulation could have served as a cue to the presence of the target contour in its own right. So that we could pose the shape-discrimination task as a 2AFC, we constrained the contours to be one of two types (illustrated in Figure 2a). Orientation differences between successive elements fell into one of two possible sequences of clockwise (+) or anti-clockwise (−) rotations: either ++−+−− or −−+−++. This procedure generates jagged broadly ‘S’ shaped contours. We refer to stimuli whose elements have orientations co-aligned with local contour orientation as “snakes.” In a second condition we generated contours in exactly this manner but then rotated each element 90° relative to the contour direction so they were mutually perpendicular (“ladders”). 

Contours were presented within a background of randomly oriented distracter elements. To do this we first dropped the path elements into the image and then placed distracter elements using an iterative procedure to ensure a minimum mutual distance between all elements in the display. Note that this is a departure from conventional distracter-element positioning used by, e.g., Field et al. ( 1993) that places distracter elements on a jittered grid. We find that our iterative procedure produces more uniform element density near the contour (MatLab code for this procedure is available on request from SCD). We refer to conditions when distracter elements are randomly oriented as the “baseline” condition; the stimuli so generated are similar to those used conventionally ( Figure 2c). In some conditions we manipulated the orientation of background elements to change the immediate orientation context the contour fell in. We did this by first computing the distance of all background elements to their nearest contour element. We used this distance to determine the probability density function used to set the distracter element orientation. Specifically we Gaussian-weighted the distance (so that the degree of contextual manipulation fell away smoothly with distance from the contour)—with a standard deviation of 1.0 deg.—and used the inverse of this function to set the variance of the probability density function that determined the orientation of surround elements. 

Examples of snakes and ladders embedded in near-parallel or near-perpendicular surrounds are shown in Figure 3. Note that the shape of snakes surrounded by perpendicular elements ( Figure 3b) is particularly easy to determine while both types of contours, but particularly ladders, are difficult to localize when surrounded by near-parallel elements (Figures 3a and 3c). 

Figure 3

 

Examples of the stimuli used in the experiment. (a, b) Contain “snakes” either surrounded by (a) near-parallel or (b) near-perpendicular distracters. (c, d) Similar containing ladders. All contours are at the same location and have the same shape. Note that the shape of the contour in (b) is particularly easy to discriminate and that stimuli in (b) and (d) are exactly equated for any local orientation differences between contour elements and their surrounds.

Design

The experiment had a within-subjects design. Subjects were shown a test screen displaying a contour embedded in an array of background elements ( Figure 2b). The independent variable was the mean orientation of the elements surrounding the contour relative to the orientation of the contour elements. That is, the orientation offset of the surrounding elements. Eight different levels of orientation offset were investigated: 0° (surrounding elements were parallel to contour elements), 15°, 30°, 45°, 60°, 75°, 90° (surrounding elements were perpendicular to contour elements), and random (surrounding elements were randomly oriented). 

Procedure

Subjects were presented a test stimulus (for a variable exposure duration) containing a single embedded contour (e.g., Figure 2c). This screen was immediately followed by a response screen containing two isolated (non-embedded) contours (e.g., Figure 2d). Subjects then used the computer keyboard to indicate which of the two contours matched the shape of the contour they had seen in the test phase. A feedback tone indicated a correct or incorrect response. Subjects fixated a centrally presented marker during presentation of the test stimulus but were permitted to move their eyes freely during the response phase. The exposure duration of the test screen was controlled by an adaptive staircase procedure (Watson & Pelli, 1983) with correct responses causing a reduction in exposure duration. The procedure converged on the exposure duration that led to 82% correct performance. We refer to this measure as the threshold exposure duration. The final performance measure was the mean threshold exposure duration obtained over at least three runs of 45 trials each. Every subject completed a practice session of at least 256 trials before data collection commenced. 

Subjects

The authors and three naive participants served as observers in the experiments. All had normal or corrected-to-normal vision and performed a series of practice trials before commencing the experiment. SCD and NJB are experienced in participating in psychophysical experiments, but other subjects were not. 

Apparatus

The experiment was run on a Apple iMac computer under the MatLab programming environment (MathWorks) and incorporated elements of the Psychophysics Toolbox (Brainard, 1997). Stimuli were displayed in true 14-bit grayscale (via the BITS++ video processor; Cambridge Research Systems) on a calibrated 22″ LaCie CRT monitor. Subjects made their responses using the computer keyboard. 

Results

Figure 4 shows results from the psychophysical experiment for (a) five and (b) four subjects. Data are threshold exposure durations, so that the smaller the number the less time observers needed to attain the criterion level of performance. Comparing Figure 4a to Figure 4b we find poorer baseline performance with ladders compared to snakes—thresholds are about three times higher in the random-surround condition—replicating several earlier findings (Bex, Simmers, & Dakin, 2001; Field et al., 1993; Ledgeway et al., 2005). Comparing these thresholds to those measured with a random context, it is clear that the manipulation of context has a substantial effect on performance. Figure 4a, which shows thresholds for snakes, indicates that the presence of a near-parallel surround (0° orientation offset) approximately doubles the average time for subjects to determine the contour shape. By contrast near-perpendicular surrounds (90° orientation offset) approximately halves the time require to perform the same task. For ladders ( Figure 4b) the only pattern to emerge is that any manipulation of the orientation structure of the context induces slightly faster performance. The most likely explanation for this is that the presence of any surround will alert the subject to the approximate location of the contour. This would produce a general reduction in threshold. These trends are highlighted in Figure 5, which shows a general elevation in sensitivity (1/threshold, rescaled to units of 1/s) to ladder contours, but a marked reduction and elevation in sensitivity when snakes are presented in near-parallel or perpendicular surrounds, respectively. 

Figure 4

 

Results from the experiment for (a) snake and (b) ladder stimuli. Symbols indicate threshold exposure duration (the minimum exposure time supporting 83% shape identification). The solid lines are the average performance across all observers. (a) Near-parallel surrounds (0° orientation offset) interfere with processing of snakes compared to random backgrounds for the five subjects tested. By contrast, near-perpendicular surrounds (90° orientation offset) elicit faster performance. (b) For ladders, any organization of the surround induces generally slightly better performance.

Figure 5

 

(a) Results from the psychophysical experiment reexpressed as average sensitivity across observers (1/threshold in units of 1/seconds). Note generally higher sensitivity of ladders with any manipulation of surround, and selective reduction and elevation of sensitivity to snakes with near-parallel and near-perpendicular surrounds, respectively. (b) Ratio of sensitivities for organized versus random surrounds with snakes (open circles) and ladders (squares). By taking the ratio of these two sets of relative sensitivities (triangles) one can effectively quantify the extra sensitivity gain/loss conferred by an organized surround for snakes, compared to what would be expected from simple pop-out cues.

It is possible that these results presented so far could originate, at least in part, from subjects being oriented to the location of the contour by the organized structure of the surround. This would predict a general increase in sensitivity compared to random surrounds, which is the pattern of results obtained with the ladders but not the snakes. However, a second possibility is that location cues will be more effective if coupled with local pop-out cues arising from a large orientation difference between contour elements and nearby distracter elements. We would expect this to be manifested as an extra increase in sensitivity with near-perpendicular surrounds (across both snake and ladder conditions). However, we note that there is only a modest extra increase in sensitivity for performance with ladders embedded in near perpendicular compared to near-parallel surrounds. Even so, we can factor this extra cue out of the analysis by taking the ratio of observers' mean sensitivity for snakes (relative to performance with a random background) to their sensitivity for ladders (again, normalized to their performance with random backgrounds). These ratios are plot as the triangles in Figure 5. These data convey the “value-added” to (or subtracted from) the detection of snakes (compared to ladders) by the presence of organized contexts, over and above general consequences of any local changes in orientation statistics. Note that this comparison is between two conditions where the only difference in stimuli is that all Gabors (background and contour) have been rotated by 90°; stimuli are fully equated for orientation contrast. Thus, the robust sensitivity decreases and increases we observe with snakes reflect a genuine difference in the way contours are processed under such conditions. 

Different patterns of sensitivity decrease and increase for snakes compared to ladders suggests that our results with snakes do not arise from false matching of contour with surround elements, since exactly the same arguments would apply to ladders. Strictly speaking there are two ways to interpret our data. The first is that the cue to the location of the contour generated by the surround is selectively more effective at overcoming false matches for (a) ladders and (b) snakes with perpendicular surrounds (i.e., the two conditions where performance is facilitated by surrounds). We consider this explanation unlikely since we can see no reason why only snakes surrounded by like-orientation elements should not show a similar improvement from a reduction in false matching. However, given that our task has two components (localization and shape discrimination) we cannot rule this possibility out entirely. The second interpretation is that our data arise from either a genuine difference in the role of context on a common contour processing mechanism (for snakes and ladders) or that the linking mechanism is completely different for both. 

Implications for mechanism

Our results demonstrate a robust effect of local orientation context on contour integration, a finding not predicted by existing association-field models. We believe that these results indicate a role for surround suppression in the visual processing of contour information. There is some precedent for this idea in the literature. It has long been known that many neurons in V1 are side-stopped (that is their activity is inhibited when structure at the cells' preferred orientation extends too far into side bands of the neurons' receptive field). This led Born and Tootell ( 1991) to propose that this could be a logical mechanism to account for suppression of contour information by texture. Such suppression is illustrated in Figure 6. When many contours are placed close to one another ( Figure 6a) the overarching impression is of visual texture and there is a loss of any but the crudest sense of the shape of the constituent contours. This inability to process contour shape under such conditions persists even when one is given prior knowledge of the shape of a component contour ( Figure 6b). Note that these contours are continuous so there can be no role for uncertainty in matching of local contour structure. 

Figure 6

 

Suppression of global form perception within texture. It is not possible to ascertain the global shape of the embedded outlines comprising (a). Even when a constituent contour is isolated, as in (b), it is impossible to localize and determine the shape of that contour within (a).

In the remainder of this section we sketch out a model of contour processing that can qualitatively explain our findings for the influence of context. Note that the model described here is not intended to be a full account of contour detection but merely to serve as a “back-pocket” framework to illustrate how contextual influences could be accounted for using opponent orientation processing. The approach is illustrated in Figure 7. First, we filter images with two banks of orientation-tuned Gabor filters: one with a narrow and one with a broad orientation bandwidth (in this example shown, filters are matched to the spatial frequency of the Gabor patches and have aspect ratios of 1.3 and 0.7, respectively). The local energy is computed for each orientation by taking the sum of squares of the output of a filter pair in quadrature phase. The local orientation energy for the narrowly tuned filter bank is illustrated in Figure 7b. We next compute the local opponent-orientation energy at orientation θ by taking the response of the narrowly tuned mechanism at θ and subtracting the response of the broadly tuned mechanism at θ + 90°. This stage is illustrated in Figure 7c. Note the high contrast between the response at the contour location compared to the response to the surround. We introduce the difference in bandwidth to reflect our intuition that the primary purpose of inhibition is to reduce response of the contour mechanism to extended texture and that it is likely that the texture surrounding a contour will be relatively broadly distributed in orientation. Contour linkage within the local opponent-orientation energy representation is then computed using a second stage of Gabor filtering with a relatively narrow orientation bandwidth ( Figure 7d; again an aspect ratio of 1.3 tuned to a spatial frequency 2 octaves lower than the first-stage filters). Finally the output of all the second-stage filters are combined together simply by selecting the filter response at each location that has the correspondingly highest energy (sum of quadrature squares) across all filters (Dakin, 1997). This gives a representation ( Figure 7e) that shows clear regions of higher activity at locations in the original image containing snakes of near co-aligned Gabor elements. Note that within this representation, low/dark regions of activity correspond to the likely locations of ladders, a point we expand in the Discussion section. Note that these regions of activity are intended only to drive contour linkage; regions of low energy are unrelated to, e.g., the detectability of single elements per se. 

Figure 7

 

Incorporating opponent orientation filtering (OOF) into a simple model of contour linkage. (a) A contour, surrounded by near-perpendicular structure, embedded in otherwise randomly oriented Gabors. (b) The response (energy) of matched Gabor filters (with orientations shown in each panel inset). Contour elements are highlighted by the horizontal filter, while elements immediately surrounding the contour are highlighted by the vertical filter. (c) The opponent orientation energy calculated by taking that energy of a narrowly tuned filter and subtracting the response of a more broadly tuned Gabor differing in orientation by 90°. Note the positive center and negative flanks in the output of the leftmost (vertical minus horizontal) filter combination at the contour location. (d) This structure can be grouped using a further stage of oriented filtering; this contour is highlighted by a horizontal second-stage filter receiving input from the horizontal-minus-vertical first-stage filters. (e) These individual second-stage filter outputs can be combined (by selecting pixels with the highest energy across filter orientation) and (f) reduced to minimally connected pixels (using the MatLab morphological operator bwmorph (image, ‘thin’, Inf) to derive minimally connected pixels) to indicate linkage consistent with snakes (red).

This model can straightforwardly account for the effects of local context on contour integration. Figure 8 shows the model output for contours embedded in backgrounds that are rotated (a) 90°, (b) 45°, and (c) 0° from the local contour orientation. Note that the strength of output at the contour location falls steadily from (a)–(c), mirroring the increasing difficulty of detecting the contour. 

Figure 8

 

Model output under manipulation of local orientation context; note that the contour is always at the same location within the background. (a–c) Surrounds oriented (a) at 90°, (b) randomly, and (c) at 0° compared to the mean contour orientation. Under each image is the response of the model described above. (d) The Yen and Finkel ( 1998) model. Orientation detectors consistent with “snake” or “ladder” structure are mutually excitatory but there is a second competitive stage between configurations.

We note that there are several limitations of the simple approach adopted here. For example, the use of simple Gabors as second-stage filters cannot capture relevant third-order orientation relationships between elements, e.g., the effects of path smoothness (Dakin & Hess, 1998; Pettet, 1999; Pettet et al., 1998). These findings could straightforwardly be accommodated the use of curved second-stage filters. We reiterate that this is not intended to be a full model as presented but merely a means of illustrating how opponent inputs could account for contextual effects on contour detection. 

Discussion

To summarize we have measured subjects' ability to perform a combined contour-localization and shape-discrimination task and demonstrated robust facilitation and suppression by near-perpendicular and near-parallel surrounds, respectively. We have also proposed that such effects, along with the established suppressive influence of texture on contour perception, can be understood in terms of a two-stage filtering model of contour integration that incorporates an orientation-opponent stage after first-stage filtering. 

Relationship to previous work

There are parallels between the work presented here and psychophysical studies of contrast facilitation (Polat & Sagi, 1993, 1994). This work has for example shown that flanking Gabors can facilitate detection of a target when the triplet is consistent with underlying contour structure (“snakes”) but that the degree of facilitation is reduced for ladder-consistent structure. It may be presumptive to draw too close a relationship between these areas; we know for example that contour integration is relatively immune to the effects of contrast randomization of elements (Hess et al., 1998), which precludes the visual system using any simple form of local contrast enhancement to signal the presence of contours. That said, while contrast enhancement may not be the mechanism it is quite possible that the two effects could be explained by a common cortical circuit. Given this it is intriguing to note that standard flank facilitation effects are trumped by surround inhibition; the addition of similarly oriented flanks in the side regions of a target cancel the facilitation being gained by the co-linear flanks (Solomon & Morgan, 2000). 

In terms of computational modeling, Yen and Finkel ( 1998) used a co-circularity constraint to determine patterns of connectivity between orientation detectors, grouped contour components using synchronized oscillation of activity, and then pooled these responses to signal contour salience. They also incorporated an additional set of facilitatory connections (running “trans-axial,” i.e., perpendicular to the local contour direction) to support the detection of ladders. The model then worked by comparing co-axial and trans-axial patterns of activity around a given point in space, and then switching off elements in the less active configuration. This effectively allows competition between locally parallel and perpendicular configurations. This architecture could explain the disruptive effect of parallel surrounds on snakes (which would maximize the probability of the co-axial/incorrect configurations winning) and the facilitatory effect of perpendicular surrounds (which would minimize the probability of interference from the surround). Figure 8c shows that even though the element comprising the surrounds used in our experiment are randomly positioned there do contain chance ladder alignments. Essentially having competition between ladder and snake configurations at the same location is a form of contextual suppression, the difference being that the Yen and Finkel ( 1998) model requires that the surround contain elements whose positions are ladder-consistent. It remains to be seen if performance on this task shows tuning for the location of elements in the surround. 

Previously Zhaoping and colleagues (Li, 1998, 2000; Zhaoping & May, 2007) have presented evidence for a model of saliency in V1, which incorporates both surround orientation inhibition and contour linking and can account for a range of pop-out effects. Our findings would be consistent with the architecture of such a system. Our results also fit nicely with recent work in machine vision from Grigorescu, Petkov, and Westenberg ( 2003). These authors have shown that augmenting a standard Gabor-energy contour-detection algorithm with iso-orientation surround suppression produces an operator that extracts isolated edges and lines but does not respond well to such structure within textures. These results, which are much sparser than the output of standard edge (e.g., Canny) operators, accord more closely with “ground-truth” images derived using manual selection of salient contour structure by human observers. 

Effect of element rotation

Although the model presented uses regions of high activity in the second-stage filter to detect co-aligned (contour-consistent) orientation structure it is clear from Figure 7e that this filter response also contains dark regions of particularly low activity corresponding to ladders. For example, the region curving away from the upper flank of the contour nicely highlights a ladder that has arisen by chance in the background. In this section, we briefly consider how the model described could be extended to account for the known effects of rotating contour elements away from the local contour orientation. 

The manifest difficulty subjects have in detecting ladders compared to snakes is, as mentioned in the Introduction section, the cornerstone on which the contour integration paradigm is built, since (from a statistical standpoint) ladders are essentially identical to snakes (i.e., containing similar levels of redundancy, cues from local mean orientation, orientation variance, and so forth). Thus, subjects' difficulty with ladders confirms that the task is tapping in to a visual process that is related to processing of real contours. By contrast, in terms of the model described above, ladder detection is the logical inverse of snake detection where ladders simply lead to consistently lower activity in the second-stage filter. We suggest four ways in which the model could be extended to fully account for poorer ladder detection: 

1.  
   An identical mechanism is used for snakes and ladders but ladder detectors receive more broadly orientation-tuned input from the first-stage filters.
2.  
   The second-stage filters for processing ladders are bigger/have broader orientation bandwidth.
3.  
   Overlap in the output of local first-stage filter contributes confers a selective advantage on snakes (Watt, Ledgeway, & Dakin, 2008).
4.  
   Noise considerations. Ladders are detected as a decrement in local contour energy, and this confers a disadvantage on their detection. In the same way that detecting a moving target among static distracters is much easier than detecting a static target among moving distracters (Royden, Wolfe, & Klempen, 2001), this will lead to search asymmetries of the kind reported previously (Bex et al., 2001; Field et al., 1993; Ledgeway et al., 2005).

Presently, the model described incorporates only (1) but the results may still be informative. Figure 9 illustrates how this asymmetry makes ladder detection more prone to the presence of nearby snakes than vice versa. This account has the advantage that it both explains poor performance with ladders and why elements rotated 45° are impossible to see. 45° is essentially the “d.c.” component of an opponent-orientation mechanism. Further psychophysical work is required to elaborate specific differences in the processing of snakes and ladders (May & Hess, 2008). 

Figure 9

 

Predicted linkage for contours in random backgrounds when contour elements are rotated by (a) 0°, (b) 45°, and (c) 90°. Red lines indicate snakes, green lines indicate ladders. Ladders are more prone to disruption by nearby snake-like structure than vice versa. This asymmetry is a consequence of the broader orientation bandwidth of input to the inhibitory component of second-stage filters. Note that contours composed of elements rotated by 45° cannot be signaled by this scheme at all.

Spatial extent of contextual interaction

In the experiments described we fixed the spatial extent over which context was manipulated to a Gaussian weighted region around the target, with a standard deviation of 60 arc min (i.e., around 1.5× the mean element separation). In pilot experiments we manipulated this value (for the near-perpendicular surround) and the results of this experiment for one observer are presented in Figure 10. We note a steady decrease in thresholds as the size of the context increased, with thresholds plateauing at around twice the element separation. It is not yet clear what aspects of the stimulus determine the spatial scale of such contextual modulation. Many aspects of contour integration appear to be scale invariant (Hess & Dakin, 1997) so one might expect that the contextual modulation field will scale with element separation. However if contextual modulation arises from low-level surround-suppression mechanisms, we would expect, e.g., the spatial frequency or even contrast of Gabor elements to be a primary determinant of contextual field size since these parameters have been shown to be crucial in determining facilitatory or inhibitory interactions in primary visual cortex (e.g., Angelucci & Bressloff, 2006). 

Figure 10

 

Threshold exposure duration plotted as a function of the spatial extent of a near-perpendicular surround. Spatial extent is measured as the standard deviation of a two-dimensional Gaussian weighting function centered on the contour location (whose inverse magnitude determines the degree of orientation variability of elements in the contour vicinity). Data have been fit with a Gaussian with standard deviation of 53 arc min.

Other implications

Revealing that contextual suppression is required for effective contour integration could provide a novel explanation why some clinical groups have difficulties with certain grouping tasks. For example, people with schizophrenia have been shown to have particular problems with contour integration (Silverstein, Kovács, Corry, & Valone, 2000; Uhlhaas, Silverstein, Phillips, & Lovell, 2004) but are also known to exhibit weaker contextual suppression (Dakin, Carlin, & Hemsley, 2005). We suggest that what has been classed as “poor integration” in various clinical groups may have more to do with those groups' failure to appropriately suppress irrelevant distracter information rather than with any inability to effectively group information across space. Based on this argument one would predict that integration problems should only be manifested under conditions of high noise (when failure to suppress will have the most devastating effects on behavior). It is interesting to note that many tasks that probe integration deficits (e.g., in autism, Dakin & Frith, 2005) rely on stimuli containing high levels of stimulus noise to elicit reliable group differences. 

Conclusions

The presence of near-perpendicular orientation structure promotes out ability to effectively process visual contours while near-parallel structure interferes with this ability. These results are consistent with a role for surround suppression in contour integration. Surround suppression is a natural component of a model of contour integration operating on opponent-orientation mechanisms. 

Acknowledgments

This research was supported by the Wellcome Trust EY-12345 (S.C.D.). We are grateful to Peter Bex, John Cass, and Beau Lotto for constructive discussion of this work. 

Commercial relationships: none. 

Corresponding author: Steven C. Dakin. 

Email: [e-mail]. 

Address: Department of Visual Neuroscience, UCL Institute of Ophthalmology, University College London, Bath Street, London EC1V 9EL, UK.