Ventral extra-striate cortical areas are required for human visual texture segmentation

Abstract

A patient (HJA) with bilateral occipital lobe damage to ventral cortical areas V2, V3 and V4 was tested on a texture segmentation task involving texture bar detection in an array of oriented lines. Performance detecting a target shape was assessed as the orientations of the background lines had increasing orientation noise. Control participants found the task easier when the background lines had the same orientation or only slightly shifted in orientation. HJA was poor with all backgrounds but particularly so when the background lines had the same or almost the same orientations. The results suggest that V1 alone is not sufficient to perform easy texture segmentation, even when the background of the display is a homogeneous texture. Ventral extra-striate cortical areas are needed in order to detect texture boundaries. We suggest that extra-striate visual areas enhance the borders between the target and background, while also playing a role in reducing the signal from homogeneous texture backgrounds.

Introduction

Texture segmentation is thought to involve the detection of boundaries based on discontinuities along feature dimensions, rather than grouping similar feature elements. A texture bar ‘pops out’ due to sufficient orientation contrast between neighboring lines at the region border, for example a horizontal line in a background of upright lines. The orientation contrast necessary to generate the perception of a texture border depends on the average orientation contrast between background elements, or the “noise” in the pattern (Nothdurft, 1993). With noisier backgrounds a greater difference between the average background orientation and the target's orientation is required before observers can detect the target. 

The origin of this ability is debated. Several studies have suggested that area V4 plays a crucial role in texture segmentation. For example, monkeys with lesioned area V4 have been found to be severely impaired in the perception of texture defined contours, suggesting that V4 is a critical site within a hierarchy of visual areas mediating texture segmentation and figure-ground separation (Merigan, 2000). Furthermore, brain activity (as measured by functional MRI) in areas V4 and TEO, but not lower visual areas (such as V2) significantly increases during the presentation of texture stimuli containing several region borders, compared with homogeneous textures (Kastner, De Weerd, & Ungerleider, 2000). There is also greater stimulus-specific adaptation in these mid-level and higher visual areas than in lower visual areas when participants view patterns containing a border defined by the offset of two sets of oriented lines (Montaser-Kouhsari, Landy, Heeger, & Larsson, 2007). This adaptation effect indicates that cells in these areas are selective to texture-defined borders (Grill-Spector & Malach, 2001) so is consistent with these areas playing a dominant role in texture segmentation. Thielscher, Kolle, Neumann, Spitzer, and Gron (2008) similarly found that brain activity in the mid-level visual areas (e.g. V4 and LOC) was higher for more salient texture borders than for lower salient texture borders whereas this was not the case for lower visual areas (e.g. V1 or V2). 

There are a number of models of texture segmentation. Typically these involve at least two stages of filters, with an intermediate rectification stage (e.g. Graham, Beck, & Sutter, 1992; Malik & Perona, 1990; Sperling, Chubb, Solomon, & Lu, 1994). Recent computational models propose that these mid-level stages modulate texture segmentation by feedback to processing in lower visual areas. For example in the model of Thielscher and Neumann (2003), (2005), and (2007) feedback from model V4 modulates activity, via a gain control type mechanism, in earlier stages to enhance weak contours and texture edges. In this model, local orientations are initially estimated by model V1 cells, the outputs of which are pooled over space and across areas of similar orientation by model V2 cells. These V2 cells provide input into model V4 cells which are sensitive to changes in input orientation and do not respond to areas of iso-oriented texture. V4 sends excitatory feedback at these regions of orientation change. Since this feedback is weakly modulatory, it particularly enhances weak bottom up signals (having a lesser effect when the bottom up signal is already strong). Removing feedback from this model reduced it's ability to detect salient texture patterns at putative region boundaries, especially when there was a high level of orientation noise (Thielscher & Neumann, 2003, 2005, 2007). 

Both Thielscher and Neumann (2003), (2005), and (2007) and Bhatt, Carpenter, and Grossberg (2007) propose accounts in which feedback from higher to lower visual areas is essential to detect texture discontinuities in noisy patterns. Furthermore, it is also proposed that even when there is no orientation noise in the background, some feedback is necessary for texture discontinuities to be detected. For example, a texture of homogeneously oriented lines can provide strong activation, due to enhancement of V2 cell activation along aligned contours. Feedback from higher areas is required to suppress this signal. For example, following the removal of feedback pathways from the Thielscher and Neumann (2003) model, texture boundaries became more difficult to detect against an aligned texture. However, the magnitude of the effect of removing feedback from the model was greater when there was a greater amount of background noise (i.e. with heterogeneous rather than homogeneous background texture). In contrast, lesioning the model of Bhatt et al. (2007) led to decreases in texture segmentation performance for both high noise patterns (as in Thielscher & Neumann, 2003) and low noise patterns. In this case feedback connections amplify boundaries even then there is little background noise, so withdrawal of this feedback reduces saliency. 

In contrast to these accounts, there is evidence that texture segmentation can be achieved in V1 alone. Neurophysiological recordings from macaque show that V1 cell activity to an oriented line or texture is modulated by the presence of an orthogonal surround (e.g. Knierim & Van Essen, 1992). Activity is particularly enhanced when the texture within the receptive field is perceived as a ‘figure’ rather than ‘ground’ (Zipser, Lamme, & Schiller, 1996). Consistent with it being sufficient to identify texture borders, Hupé, James, Girard, and Bullier (2001) reported that modulation of responses in macaque V1 by the surrounding texture was not affected by the inactivation of V2, suggesting that any modulation comes solely from lateral interactions in V1. Furthermore, V1 activity in the anesthetized macaque monkey is enhanced at texture borders while there is also suppression of activity by homogeneous texture backgrounds (Nothdurft, Gallant, & Van Essen, 2000). Models of V1 cells that incorporate both the classic center-surround receptive field and areas of larger non-classical inhibitory areas are able to identify orientation defined contours or texture boundaries (Grigorescu, Petkov, & Westenberg, 2003; Huang, Jiao, & Jia, 2008; Petkov & Westenberg, 2003). Similarly, Li (2000) and Mesrobian and Skrzypek (1995) have also illustrated, that given the properties of V1, it is theoretically possible for texture segmentation to occur there, without the need for top-down feedback, using low level visual processes at least for high salience texture borders. On the other hand, however, Lamme, Zipser, and Spekreijse (1998) found that in some cases anesthesia does reduce the effects of surrounding backgrounds on activity of V1 cells. Similarly, lesions of extra-striate cortex in macaques have been shown to remove responses to orientation defined figures (Lamme, Supèr, & Spekreijse, 1998). 

In the present paper we report a neuropsychological test of the necessary role in texture perception of mid-level to early visual areas. The patient we examined, HJA, had a well documented bilateral lesion involving V2 (partial), V3, and V4 along with sparing of V1 (Allen, Humphreys, & Bridge, 2007). He had a range of problems in early and intermediate visual processing, especially involving grouping and perceptual organization. The site of the lesion and the nature of the perceptual problems experienced means that HJA provides a critical test of the role of mid-level visual areas in human texture perception. We used texture segmentation stimuli inspired by Nothdurft (1985) and similar to those used by Theilscher and Neumann (2003) to develop their model of texture segmentation. In these textures a target shape (here, a bar) can be detected if it differs from the background orientation by sufficient orientation contrast. The amount of orientation contrast required increases with increasing orientation gradient applied to the background items (i.e. a form of orientation ‘noise,’ see Figure 1). The investigation of the sensitivity to structure gradients in texture discrimination is important since previous psychophysical experiments, with healthy subjects, have already demonstrated that pattern segregation is based on orientation differences rather than the (dis-) similarity of orientation features (Nothdurft, 1985, 1991, 1992). When orientation gradients are applied to the background elements, they can group to form ‘flow’ patterns which can interfere with texture segmentation. Alternately grouping of texture elements within a texture can facilitate segmentation and thus suppression of texture segmentation (Harrison & Keeble, 2008) which is a key aspect of some models of texture segmentation (e.g. Theilscher & Neumann, 2003). 

Figure 1

 

Examples of displays for orientation contrast value of 30° with (a) background shift = 0°, (b) background shift = 2°, (c) background shift = 10°, and (d) background shift = 30°. Note that the background lines in this illustration do not always appear to increment by a constant amount, this is merely a limitation of the reproduction and not true of the actual stimuli. It is evident that as the background shift increases, the orientation-defined texture bar becomes more difficult to detect.

Experiment 1 reports an examination of basic texture perception in HJA's case. Subsequent experiments explore the nature of the evident deficit. 

Experiment 1: Texture segmentation

Participants

Patient HJA

HJA was aged 86 years at the time of the present study. He suffered a posterial cerebral artery stroke peri-operatively in 1981, when aged 61, resulting in bilateral lesions of the occipital lobe, extending anteriorly to the temporal lobes. After his stroke, HJA experienced a dense visual agnosia, prosopagnosia, alexia without agraphia, achromatopsia and topographical impairments. He showed no indication of a general intellectual deficit (Riddoch & Humphreys, 1987). In a recent structural MRI study (Allen et al., 2007) it was shown that HJA's lesion is likely to involve a variety of extra-striate areas, including V2 (partial), V3 and V4, however the lesion does not extend dorsally. Despite his severe deficit, HJA was able to perform well in several tests of visual function such as detecting sine gratings or identifying hierarchical forms (Riddoch et al., 2008). 

This study measured HJA's ability to detect an orientation defined bar. This task critically depends on participants ability to discriminate orientation. HJA's basic orientation discrimination showed no indication of being impaired. Previously he has performed as well as (or slightly better than) unimpaired participants searching for oriented lines (Humphreys, Riddoch, Quinlan, Price, & Donnelly, 1992). More recently we measured HJA's ability to discriminate the orientation of a single Gabor in an unpredictable location within a 6° window (Allen et al., 2007). His threshold for this was 2.5°, which was similar to normal observers (see, for example Beaudot & Mullen, 2006). Lindblom and Westheimer (1992) found orientation discrimination threshold of 1–2° for isolated lines, increasing to between 1–4° when the lines are presented with a grid of lines and to 2.5–11° when the position of the tilted line is uncertain. Given that spatial bandwidth does not greatly affect thresholds in normal observers (Beaudot & Mullen, 2006) HJA's orientation discrimination performance was similar to controls and thus suitable for this task. 

Control participants

5 older controls were tested (age range = 69–77). These control participants were slightly younger than HJA at the time of testing, this reflects the difficulty in matching to HJA in terms of general cognitive facilities in an age-matched population. There were 4 males and 1 female, 4 of whom were right handed and one left handed. 4 younger controls (age range = 24–31) were also tested, 1 male and 3 females, all of whom were right handed. All had normal or corrected-to-normal vision. 

Equipment

Stimuli were presented on a Mitsubishi Diamond Scan 50n monitor driven by an ATI Rage 128y graphics card. The mean luminance of the screen was 26 cd/m ². The experimental program was written on an Apple Macintosh G3 computer using the Matlab environment and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). The monitor had a resolution of 1024 by 768 and a frame refresh rate of 85 Hz. One pixel on the screen was equal to 0.27 mm². The screen was viewed binocularly at approximately 100 cm, although no restraints were used. 

Stimuli

The stimuli consisted of arrays of black lines on a white screen (see Figure 1). The array consisted of 12 by 12 initially vertical lines presented at 95% contrast. The array was 546 pixels/147.42 mm square (10.32 degrees ²). Each line element was 5 pixels/1.4 mm wide, 18 pixels/4.86 mm high (0.048 degrees ²), and evenly spaced throughout the array. The orientations of the lines was perturbed by, first, a background orientation shift (0°–30°). An additional orientation shift, or contrast, was then applied to one subregion to create the orientation-defined texture bar. 

The background shift was the value by which the orientation of neighboring lines (either across a line or down/up a column) in the background array differed. When the background shift was zero, all the lines in the background texture area were identical. When the background shift was not zero, the patterns had a continuous feature gradient over the whole pattern, in which the sign (and direction of the gradient) could change at random positions in the array, in order to minimize effects due to apparent ‘flow patterns.’ 

The shape defined by the orientation contrast was a bar of 7 lines by 2 lines. It was positioned from the third row down and offset from the center by a random offset of up to 4 grid positions (i.e. 8 columns). The magnitude of the orientation contrast between the background and texture bar varied between 0° and 90° to the left ( p = 0.5) or right. 

Procedure

The aim was to measure texture segregation as a function of orientation contrast. Participants detected an orientation-defined texture bar within a larger texture field (see Figure 1). Stimuli were presented on the screen for 500 msec. Participants were required to answer yes if the texture bar was present, and no if not present, and responses were recorded after each display by the experimenter by pressing allocated keys on a keyboard. Young participants pressed the keys themselves. The next display was presented only when a response had been recorded for the previous one. Full training was given prior to the start of formal data collection. Participants were informed of the task and practice trials were performed until they reported that they understood the task. Participants were tested with arrays of lines with background shift = 0°, 2°, 10° and 30° in separate testing blocks. The orientation contrast at the borders of the texture bars was varied, with ten different levels of orientation contrast for each background shift value. The orientation contrast values were chosen to span the psychometric function of the observer such that their performance ranged from chance to perfect, where possible. Stimuli were presented in blocks of 40 displays in testing sessions of 2 hours. The room in which testing took place was dimly lit, at the same level for all participants. Performance was measured as the proportion of correct answers for each orientation contrast within each background shift condition. 

Results

Each participants' data were fitted with separate cumulative Gaussian functions using psignifit ( http://www.bootstrap-software.org/psignifit/) and fmins in Matlab. Thresholds were defined as the line orientation contrast at which the texture was detected on 75% of trials for each background shift. 10000 boot strap replications of the fit were carried out to estimate both 95% confidence intervals of the threshold and goodness of fit measures (Wichmann & Hill, 2001a, 2001b). To find differences between groups and participants, threshold performance was compared using ANOVAs. At the highest level of background shift the proportion of correct responses rarely reached over 75%, even at the highest orientations tested. For this condition we report the comparisons between raw percent correct scores. To compare HJA's performance in different conditions we compared 95% bootstrapped confidence intervals of his thresholds (equivalent to p < 0.05). 

To first establish if there was any age related decline in the ability to do the task, we compared older and younger control participants. Figure 2 shows the data from the two groups. Figure 2a illustrates that the mean thresholds were similar across the two groups for background shifts of 0, 2 and 10 degrees. A mixed ANOVA found that there was no significant effect of group ( F(1,7) = 0.48 p = 0.5 partial η ² = 0.06) and that group did not interact with the effect of background shift ( F(1,14) = 0.96 p = 0.41 partial η ² = 0.1). As expected though, there was a significant effect of background shift ( F(2,14) = 139.4 p < 0.0005 partial η ² = 0.95). For the largest background shift (background shift = 30°), no thresholds could be estimated so an analysis was performed on the accuracy data ( Figure 2b). A mixed ANOVA on the raw scores found that there was no effect of group ( F(1,7) = 1.5 p = 0.26 partial η ² = 0.17), no effect of increasing the orientation contrast ( F(4,28) = 1.75, p = 0.167 partial η ² = 0.2) and no significant interaction between group and orientation contrast ( F(4,28) = 0.7 p = 0.6 partial η ² = 0.09). 

Figure 2

 

Comparison of Old and Young Controls. a) Mean threshold estimates for old and young participants for each of the three levels of background shift that could be estimated. Error bars (where visible) are 95% confidence intervals for the group mean. b) Mean percent correct responses for old and young participants at the largest background shift = 30°, error bars represent the standard deviation of the participants' scores.

Because there were no significant differences between the two groups of controls (and therefore no effect of aging on the controls' ability to do the tasks), for all further analysis the old and young control subjects were combined to form one group. 

To compare HJA's data to the data of the combined control group we carried out a two stage analysis to ensure that we avoided false positives. First we carried out a mixed ANOVA on the threshold data comparing HJA's mean scores to the mean scores of the control group. Controls were entered as one group and HJA was entered as a second group with only one member. There was a significant effect of the magnitude of the background shift ( F(2,16) = 58.192 p < 0.0005 partial η ² = 0.88) and a significant effect of the group (control or HJA) ( F(1,8) = 53.2 p < 0.0005 partial η ² = 0.87) as well as a significant interaction between group and background shift ( F(2,16) = 5.11 p = 0.019 partial η ² = 0.39). These effects and interactions can be seen in Figure 3a, which shows the mean thresholds for each background shift for HJA and the controls. Independent samples t-tests confirmed that there was a significant difference between HJA and the controls when background shift = 0 ( t(8) = −14.97 p < 0.0005) and also when the background shift = 2 ( t(8) = −3.1 p = 0.015) and also when the background shift = 10 ( t(8) = −3 p = 0.018). The interaction arose because the deficit was most apparent when the background shift was 0. 

Figure 3

 

Comparison of HJA and controls in Experiment 1. a) Mean threshold estimates for HJA and averaged control participants for each of the three levels of background shift that could be estimated. Error bars for the controls are 95% confidence intervals for the group mean, error bars for HJA (some smaller than symbols) are those estimated from the bootstrap procedure described in the text. b) Mean percent correct responses for HJA and control participants at background shift = 30°, error bars are the standard deviation of performance between runs or over the group.

To further assess whether there was a greater deficit for HJA when background shift was 0 we used a modified F test (Hulleman & Humphreys, 2007) for comparing a single case against a group of participants. This test adjusts the confidence levels and F values to take account of differences in variance and to minimize the likelihood of false positives. We calculated the difference between each participant's threshold when the background shift was 2 and when the background shift was 0. The change in scores between the two levels of background shift was significantly different between HJA and controls (F(1,8) adjusted = 20.71 p = 0.0018, two tailed) with HJA showing a decrease in threshold and the controls showing an increase. 

Raw data from the condition with the highest background shift are shown in Figure 3b. Since thresholds could not be estimated from these data, an ANOVA was performed on the raw percent correct scores. There was no significant effect of the magnitude of orientation contrast ( F(4, 32) = 1.03 p = 0.41 partial η ² = 0.11), or group (controls vs. HJA), ( F(1,8) = 6.14 p = 0.038 partial η ² = 0.434), nor was there a significant interaction ( F(4,32) = 1.75 p = 0.16 partial η ² = 0.18). At this level of background shift, the control group appeared to reach a floor in performance. HJA's performance dips well below 50% correct when the orientation contrast is 30 and 90 degrees. His performance could reflect an inversion in performance, indicating that he is able to reliably indicate a difference between the present and absent trials, even if they are incorrectly labeled. This might suggest that he may be able to extract some information from these patterns, which might reflect a still functioning V1. His performance, however, never reaches beyond 25% and is highly variable so it is difficult to draw conclusions from these data. 

Thus, it seems that HJA was consistently worse on the task than the control participants. Surprisingly perhaps, his deficit was proportionally much worse when the background lines were all vertical. 

Discussion

HJA's ability to detect the orientation defined bar was significantly worse than control participants (both young and age-matched). Interestingly, HJA was proportionately worse when the background lines were all the same orientation (i.e. background shift = 0) than when the background shift was greater. This is particularly surprising since our displays were not completely anti-aliased such that, even at our viewing distance, the texture bar could have been particularly salient due to the aliasing on the tilted lines. It is possible that some of the difference in performance between controls and HJA is attributable to slightly worse orientation discrimination for HJA. Given any deficit in orientation discrimination is mild (see Methods) this is unlikely to explain either the magnitude or pattern of results (especially when the background items are all the same orientation). Since HJA lacks mid-level visual areas, it is tempting to conclude from these data that mid-level visual areas are required for texture segmentation. Given that he had difficulty with the case where there was no background noise, it is possible that HJA found it difficult to discount this signal when making judgments about the target texture. Consistent with this HJA has previously been found to be able to group oriented elements, but he found it difficult to disregard potential groupings when asked to make judgments about the surface properties of shapes (Giersch, Humphreys, Boucart, & Kovacs, 2000). Bhatt et al. has suggested that there are two complementary feedback processes (Bhatt et al., 2007); boundary-enhancement feedback from V2 that enhances texture boundaries and surface-based feedback from V4 and parietal cortex that acts to suppress or enhance entire objects or surfaces. Noise, or absence of this latter suppressive feedback process, triggered by earlier grouping of perceptual boundaries may disrupt HJA's judgments. To test the role of the strength of the signals from the background lines Experiment 2 repeated Experiment 1 but manipulated the background so that the lines were less likely to group together. 

Experiment 2: Jittered or oblique backgrounds

Participants performed the detection task as in Experiment 1, except that the background lines were either jittered in their orientation or had oblique (on the diagonal) orientations. These two manipulations will decrease the strength of signal from the background elements. First, jittering the background orientations (as illustrated in Figure 4a) will reduce the number of line ends that connect, reducing grouping between them. Second, the visual system is less responsive to oblique orientations (Pointer, 1996). Furthermore, if the pattern elements are tilted on to the oblique, the gap between the ends of the lines will also be increased, further decreasing the possibility of grouping. If the local signals or strength of grouping between the signals is reduced, then the overall signal from the background may be less. For control participants, both these changes should make the background harder to exclude. If HJA is unable to detect the texture bar because of an overly strong signal from the background, then reducing the strength of the grouping may, on the other hand, improve his performance. 

Figure 4

 

Examples of stimuli in Experiment 2. a) Orientation jitter is ±5 degrees. b) Oblique line elements (no orientation jitter).

Method

Methods were the same as Experiment 1, except as stated below. 

Participants

HJA and five control participants were tested. The control participants did not take part in the previous experiments and were naïve to the purpose of the study. Since there was no difference between age-matched and non-age-matched controls in Experiment 1, we used non-age-matched controls here. 

Stimuli

The stimuli were constructed as in Experiment 1, except that in one condition a 5° orientation change (jitter) was added or subtracted (randomly) from the orientations of the background lines. This meant that the background lines were less likely to group or form coherent swirling patterns (see Figure 4a). In the other condition the starting orientation was tilted 45 degrees anticlockwise of vertical (see Figure 4b). This experiment only tested performance with low background shift. For jittered backgrounds we tested background shifts of 0°, 2° and 10°. For oblique backgrounds we tested only background shifts of 0° and 2°. These are the conditions where HJA's results differed most clearly controls and where performance may have been more influenced by the background. 

Results and discussion

We compared performance in Experiment 1 with performance in Experiment 2, separately for HJA and controls and separately for jittered and oblique backgrounds. When the background orientation was jittered, performance of the control participants got worse, compared to when the background was not jittered (data from Experiment 1), but not significantly so ( F(1,12) = 3.511 p = 0.086 partial η ² = 0.226, Figure 5b). Although there was a significant effect of background shift ( F(2,24) = 65.37 p < 0.0005 partial η ² = 0.85) this did not interact with the experiment ( F(2,24) = 1.8 p = 0.187 partial η ² = 0.13). Comparison of the participant specific 95% confidence intervals for HJA showed that while jittering the elements' orientation appeared to improve HJA's performance in comparison to Experiment 1, this was also not significant ( Figure 5a). 

Figure 5

 

Comparison of HJA and controls in Experiment 2 when the orientations of the background lines were jittered (a, b) and lines were oriented on the oblique (c, d). Data from HJA (top row, a, c) and controls (bottom row, b, d) was compared to data from Experiment 1. Error bars are 95% confidence intervals of the group or of HJA's threshold.

When the background lines were angled on the oblique ( Figure 5d), as expected control participants' performance decreased reliably ( F(1,12) = 8.97 p = 0.01 partial η ² = 0.43). There was also a significant effect of background shift ( F(1,12) = 44.68 p < 0.0005 partial η ² = 0.79) but no interaction between the background shift and experiment ( F(1,12) = 0.25 p = 0.63 partial η ² = 0.02). On the other hand, comparison of the confidence intervals for HJA show that his performance improved significantly when pattern elements were oblique compared to when they were upright ( Figure 5c). 

The pattern of results was clearly different for controls and HJA. The controls found the texture discrimination task more difficult when the background lines were oblique. This is consistent with the strength of signal from the background lines decreasing with oblique stimuli. Control participants, thus, appear able to exploit background grouping to help segment the differently oriented texture bar. HJA, on the other hand, performed better when the background elements were oblique. This suggests that the strength of the signal from the background lines (in Experiment 1) disrupted his ability to segment the texture bar. 

Experiment 3: Single target and density control

In Experiment 3 we tested whether density affected HJA's performance. Given that HJA seems to be impaired by grouping of background lines, it is possible that he is abnormally sensitive to local lateral interactions between visual elements. Nothdurft (2000) reported that the salience of an oriented line is greatest in line arrays with medium density and less in arrays with lower and higher densities. This is replicated in the model of Thielscher and Neumann (2007) where at high densities the target's strength is reduced by lateral interactions within model V1 and at low densities adjacent texture elements fall outside the receptive field of the model's V4 neurons so their activity cannot be suppressed when necessary. 

Experiment 3 also allowed us to examine performance when the target was a single oriented oriented line, compared to when it was a bar defined by multiple lines at the same orientation ( Experiment 1). In Experiment 1, it was possible to perform the texture bar detection task by detecting the presence of any single discontinuous line. This might be adopted as a strategy by any participant, but might be particularly relevant in HJA's case where there may be problems integrating multiple lines within the target. 

Method

Nine additional participants and HJA were asked to find a single tilted line amongst the background lines. The method was the same as above except that there was one line with additional orientation contrast, compared to the background lines. This target line could appear in any of the positions that could contain the texture bar in the previous experiments. An example stimulus (at the medium density) is shown in Figure 6. As well as the original density, participants were tested with arrays with half and double the number of lines covering the same spatial array to manipulate density. 

Figure 6

 

Example of stimulus using in Experiment 3, where participants detected 1 tilted line, shown here with a background shift of 2°.

Results and discussion

Performance here (with a single oriented target) was compared with data in Experiment 1 (multiple elements at medium density). 

To assess whether HJA or controls were better, or worse, at detecting a single tilted line than the orientation defined texture bar, we compared data from Experiment 1 with the medium density condition here. The control participants' a showed a significant effect of background shift level ( F(1,16) = 57, p < 0.0005, partial η ² = 0.78), but no significant difference between the two experiments ( F(1,16) = 0.2, p = 0.66, partial η ² = 0.1) and no interaction ( F(1,16) = 0.02, p = 0.89, partial η ² = 0.001), see Figure 7b. The control participants in this experiment tended to have increased discrimination thresholds when there was a one tilted line target, relative to when the target was constructed from multiple lines but this was not reliable. 

Figure 7

 

Comparison of HJA and controls in Experiment 3 where participants detected 1 tilted line. a) Data from HJA with a single target line compared to his data from Experiment 1. b) Same as a, but for control participants. c) Data from HJA tested with arrays of half and double density, compared to his data from Experiment 1. d) Same as c, but for control participants. Error bars are 95% confidence intervals of the group or estimated from HJA individual data.

For HJA, however, when there was no background orientation shift, performance improved reliably when there was one, rather than multiple target elements (see the 95% confidence intervals in Figure 7a). This suggests that the presence of additional items in the texture bar may have impeded HJA's performance. 

We next investigated the effect of density. Thresholds for detecting the single target at different densities are shown in Figure 7 for both control participants ( Figure 7d) and HJA ( Figure 7c). For control participants, as before, thresholds increased when background shift increased ( F(1,8) = 25.5 p = 0.001, partial η ² = 0.8), however, performance did not change significantly with density ( F(1,8) = 0.11 p = 0.89 partial η ² = 0.01), nor was there a significant interaction between background shift and density ( F(2,16) = 0.9 p = 0.42 partial η ² = 0.1). 

The data again differed for HJA ( Figure 7c). When texture density reduced, he required less orientation contrast to detect the target line. This was the case at both levels of background shift. This improvement in performance when the density of the background reduced, especially without background orientation shift, suggests that the presence of proximal background elements disrupted HJA's ability to detect the target line. When density increased, however, HJA improved when there was no background orientation shift but performance decreased when the background shift was present. Worse performance at the higher density, as found when the background shift = 2°, is consistent with the background interfering with detection of the target. Improved performance with high density and no background shift, however, appears to contradict this. One possible explanation for this result is that with the high density and aligned pattern, for HJA, the background groups into a surface and the task therefore changes. Detecting a surface discontinuity can be easier than searching for a target in noise. An alternate explanation comes from the difference in length of the texture borders in this experiment and Experiment 1. If normal vision benefits from enhancement at texture boundaries, then there will be greater enhancement in Experiment 1 (with a 7 × 2 shape) than in this experiment (with only 1 tilted item). This feedback may not be present in HJA, so compared to controls he performs badly. In fact, in his case, the signal from the texture-defined border may even be suppressed due to local suppressive interactions between non-aligned elements (Field, Hayes, & Hess, 1993). Where there is just one target element, performance will be determined solely by the orientation contrast between the two adjacent elements and this may allow HJA to perform better, particularly when the neighboring items are proximal and a single canonical orientation. 

General discussion

If texture segmentation can be achieved in V1, at least for easy segmentations, then HJA should not be impaired at this task. In contrast, if HJA's lesioned cortex is required for texture segmentation, then HJA may be impaired at texture segmentation. The data are consistent with the latter. HJA was poor at detecting a bar defined by the orientation of its elements. The pattern of the deficit was unexpected. In Experiment 1 we found that HJA was poor at detecting the orientation defined bar and proportionately worse when the background texture lines were all had the same orientation. In Experiment 2 we manipulated the background so as to reduce the grouping of the background elements. We propose that the ‘over-grouped’ background lines may have acted as a mask for HJA for the orientation defined bar. Control participants, on the other hand, should be able to integrate and separate the grouped background lines from the target, to facilitate performance. Consistent with this, when the array elements were all tilted to an oblique angle HJA's performance improved, whereas control participants' performance worsened. A similar, but weaker effect was found when we jittered the position of the pattern lines. Finally, in Experiment 3, we measured the ability to spot one single tilted bar and tested performance with different array densities. HJA was better at detecting one tilted line than the 14 element bar tested in Experiments 1 and 2, whereas control participants were, if anything, worse with single line targets. The disruptive effect of multiple elements in the target for HJA points to HJA being impaired at integrating the oriented elements into one target bar when there is competition from a strong background group. Reducing the density of the background ( Experiment 3) improved HJA's performance, suggesting again, that he may have had difficulty suppressing the background elements. There was one exception to this, when there was no background noise, performance improved when density increased. In this case we suggest that grouping enabled HJA to detect a single target as a surface discontinuity, or because he detected single targets efficiently based on a local discontinuity to proximal elements. Increasing background noise then decreased performance, either because the strength of the surface texture of the background decreased or the local discontinuity defining the target element decreased. 

These results add to the body of data on visual grouping and perceptual organization in HJA. Prior data have demonstrated that he was able to group collinear elements (Giersch et al., 2000) and he could average across line orientations (Allen et al., 2007). On the other hand, HJA was impaired at integrating grouped elements in coherent surface representations of shapes based on 2D edge cues (see Chainay & Humphreys, 2001 for evidence of the use of other cues to depth and surface coding). Here we found that HJA was poor at texture coding, but not completely impaired. It seems that some operations required for texture grouping and segmentation are performed in early visual cortex, even V1. This is consistent with results from previous macaque studies where V2 is lesioned and some texture segmentation responses remain (Hupé et al., 2001). Similarly, the model of Bhatt et al. (2007) suggests that many of the initial orientation grouping processes can occur at the stage of V1 (compare also Li, 2000). These early signals appear insufficient, however, to perform reliable texture segmentation in many cases. FMRI evidence from humans indicates that higher visual areas respond strongly and selectively to segmentable textures (Kastner et al., 2000; Montaser-Kouhsari et al., 2007), and these are the areas lesioned in HJA, suggesting that these regions are essential for normal texture segmentation. 

It is interesting to interpret our results in the context of recent models of texture segmentation. In models that propose a predominantly feedforward mechanism (e.g. Malik & Perona, 1990) our results imply that the texture segmentation stage must be located in the mid-level visual areas. Texture segmentation is proposed to come about by combining the output of orientation selective first stage filters at a second stage. Reducing the orientation noise in the stimulus should increase the signal in these first stage filters and improve segmentation at the second stage. While this is consistent with the behavior of our control participants, it is difficult to reconcile with HJA's performance (see also Thielscher & Neumann, 2003 for a discussion of these models behavior with the type of patterns used here). The model of Thielscher and Neumann (2003), (2005), and (2007) proposes that feedback from mid-level visual areas plays a role in texture coding, particularly when patterns are noisy. In the model, V4 acts to modulate the activity in lower visual areas to enhance orientation-defined borders. When the background elements are noisy, feedback from V4 is important since the feedback uses evidence pooled across regions to lessen the effects of noise. Feedback signals increase the responses of cells at region borders. Center-surround inhibition then normalizes activity in the surrounding areas. Thus, the enhancement of activity of certain cells in a pool in turn leads to a reduction in activation for those cells which did not receive any top-down bias. In effect this will ‘turn-down’ the signal from background elements. When texture elements are perfectly aligned (as in our stimuli), recurrent feedback from V4 still enhances the borders between regions, but the effect is less pronounced (see Figure 7, Thielscher and Neumann, 2003 for one such case). 

Thielscher and Neumann (2003) investigated the relative contribution of the feedback effects in an intact system and found that loss of feedback substantially reduced the models performance. Although this provides a slightly better account of HJAS's performance, this is also insufficient to explain HJA's performance, especially when there was no background shift. On the other hand Bhatt et al. (2007) proposed separate surface and border related feedback pathways and modeled the effects of removing the border enhancing feedback. As in Thielscher and Neumann (2003)'s model, texture segmentation performance decreases without feedback. Intriguingly, however, when the background shift was low (5 degrees), increasing orientation contrast did not improve performance, in fact performance worsens. This illustrates one possible mechanism whereby a more homogeneous background, without feedback, can lead to much worse performance. However, there is little evidence for two segregated feedback pathways in the visual system. 

We suggest another similar possible mechanism to explain HJA's poor performance. Recent findings by Roelfsema, Tolboom, and Khayat (2007) have provided evidence that primary feature detection and figure-ground segregation are signaled in different temporal phases of the response pattern of neurons. They demonstrated that the motion onset in a random dot pattern is signaled by early responses while figure-ground separation has a temporally delayed signal. Similarly, Lamme (1995) demonstrated that a central figure surrounded by a background region is tagged by a neuronal signal that yields a delayed response amplification. We suggest that different sweeps of feedback activity exist that define different epochs in the neural segmentation process involved in figure-ground segregation. It was suggested by Roelfsema, Lamme, Spekreijse, and Bosch (2002) that texture boundaries (as signaled by orientation discontinuities) are detected locally and mainly during the feedforward sweep of neural processing. The binding of elements to form the interior of the figural surface patch is mainly achieved by the reverse sweep during feedback processing (cf. Lamme, Supèr et al., 1998). Thus, it seems plausible that an early signal is generated that spreads inwardly (or fills-in) to cover the texture and links the elements into an apparent surface patch. This signal modulates, via feedback, the neural activations for the individual items inside the figure. The modulation strength is weaker than the one that is generated by the boundary signals. While in the intact network only the smaller target pattern generates enough activation to be segmented, in the impaired network, without the enhancement of the texture boundary, two interfering groupings might be created, one from the inner segmented region and one from the edges of the entire pattern. Both signals might start to interfere since they cover overlapping regions. Taken together, in HJA the responses from strong groupings remain and are not reduced by competitive interactions. The figural parts (corresponding to the texture bar) cannot be reliably enhanced, i.e. neither the boundaries nor the figural interior are enhanced in order to generate a significant effect. This leads to the “overshadowing” of the texture boundaries by strong collinear grouping of the background items. We suggest that the neural boundary response is increased by larger orientation contrast as shown in the fMRI experiments in Thielscher et al. (2008). This forms a testable prediction and we plan to advance the modeling along these lines. 

Acknowledgments

This project was funded by grants from the Wellcome Trust and the ESRC. H.N.'s research has been supported in part by a grant from the EU to STREP project no. 027198-Decisions-in-Motion and to ICT-project no. 215866-SEARISE as well as the Transregional Collaborative Research Center SFB/TRR62 “Companion Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG). We also thank HJA for his patient participation in all our studies over the years. We are saddened to report that HJA died in 2008. We also thank Axel Thielscher for helpful comments on an earlier version of this paper. 

Commercial relationships: none. 

Corresponding author: Harriet A. Allen. 

Email: [e-mail]. 

Address: Brain and Behavioural Sciences, School of Psychology, University of Birmingham, Birmingham, B15 2TT, UK.