Segregation by onset asynchrony

Abstract

We describe a simple psychophysical paradigm for studying figure-ground segregation by onset asynchrony. Two pseudorandom arrays of Gabor patches are displayed, to left and right of fixation. Within one array, a subset of elements form a figure, such as a randomly curving path, that can only be reliably detected when their onset is not synchronized with that of the background elements. Several findings are reported. First, for most participants, segregation required an onset asynchrony of 20–40 ms. Second, detection was no better when the figure was presented first, and thus by itself, than when the background elements were presented first, even though in the latter case the figure could not be detected in either of the two successive displays alone. Third, asynchrony segregated subsets of randomly oriented elements equally well. Fourth, asynchronous onsets aligned with the path could be discriminated from those lying on the path but not aligned with it. Fifth, both transient and sustained neural activity contribute to detection. We argue that these findings are compatible with neural signaling by synchronized rate codes. Finally, schizophrenic disorganization is associated with reduced sensitivity. Thus, in addition to bearing upon basic theoretical issues, this paradigm may have clinical utility.

Introduction

Temporal cues to figure-ground segregation have been much studied and much debated, but several basic issues remain unresolved. Most paradigms provide results showing that temporal cues have a role in perceptual organization, but some do not, and interpretation of those that do is often controversial. For reviews of psychophysical evidence, see Blake and Lee (2005), Farid (2002), and Watt and Phillips (2000). For multiple perspectives on the neurophysiological evidence, see the September 1999 issue of Neuron. 

The original Gestalt notion of “common fate” implied a role for temporal cues in perceptual organization, and this has since been generalized to various patterns of change over time (e.g., Guttman, Gilroy, & Blake, 2007; Palmer, 1999; Sekuler & Bennett, 2001). Several neurophysiological theories propose that internally determined temporal relations are used to signal fundamental aspects of perceptual organization. For example, some propose that synchrony signals grouping (e.g., Singer, 1999), and others propose that latency signals salience (e.g., Gawne, Kjaer, & Richmond, 1996; Opara & Wörgötter, 1996; VanRullen & Thorpe, 2002). Our focus here is on sensitivity to externally determined temporal cues, and in particular on segregation produced by the asynchronous onset of figure and ground elements. Our paradigm was made as simple as possible by using single discrete onsets on each trial, not sequences of events as in flicker or rapid stochastic sequences of stimulus change. One advantage of this is that much is known about the brief transient bursts of neural activity that are produced by abrupt visual onsets. With short latency, they rise well above the sustained response to any sustained stimulus input, and they last for less than 100 ms. Although there is variability in the response of single cells, the transient local population response to abrupt onsets is tightly time-locked to stimulation, so studies of asynchronies in stimulus onset can be used to make plausible inferences concerning sensitivity to asynchronies in transient neural activity. 

Five issues are of particular relevance here. First, if synchronous inputs are perceived as a single group, what asynchrony is required for them to be segregated into distinct groups? This issue is important because it relates to debates concerning the roles of both rate and synchrony in neural coding. Some studies suggest that asynchronies of as little as 16 ms (Usher & Donnelly, 1998), 5 ms (Fahle, 1993), or even less than 1 ms (Greene, 2007) produce segregation. Others suggest that 20–40 ms is required (e.g., Hancock & Phillips, 2004). Measures of the “perceptual moment” within which events are seen as simultaneous often range from 50 to 100 ms (Elliott, Shi, & Kelly, 2006; VanRullen & Koch, 2003). Although there have been many previous psychophysical studies of the effects of such temporal cues on perceptual organization, their interpretation often raises difficult issues. Many use patterns of stimulation in which figure and ground change repeatedly or flicker at high rates and are distinguished by the exact timing, or phase, of those changes. Although such paradigms are widely used, several computational and experimental studies show that their interpretation is difficult because the temporal cues may covary with other cues (e.g., Beaudot, 2002; Dakin & Bex, 2002; Farid, 2002; Morgan & Castet, 2002; Waliis, 2005). Furthermore, when onsets and offsets occur in close temporal and spatial proximity the transient responses to onset and offset interact in complex ways. This was first shown using square-wave flicker where transient responses to onsets and offsets were shown to be affected by reciprocal inhibition between on- and off-center cells (Phillips & Singer, 1974; Singer & Phillips, 1974). As such interactions affect both latency and strength of the transient neural response, there is no simple equivalence between the timing of external events and the timing of internal neural events. Furthermore, it has been shown that sensitivity to asynchronies between a single pair of events can be greater than that to asynchronies within rapidly and repeatedly changing sequences of events (Guttman et al., 2007), so responses to the latter are no simple combination of responses to the former. We therefore study the much simpler case in which figure and ground are presented only once on each trial, but with slightly different onset times. 

Second, does the order of figure and ground presentation matter, i.e., is the effect of asynchrony asymmetric? One simple prediction can be made on both intuitive and empirical grounds. The figure should be easier to see when it is displayed first because it is then initially displayed by itself in a blank field, whereas when it is displayed second it is displayed against a background within which it is hidden. Empirical support for this prediction comes from studies of the effects of presentation order in rapidly cycling displays of figure and ground. In that case, the path is more likely to be seen when it is displayed in the first frame of the cycle than when the ground is displayed first (Beaudot, 2002). Similarly, studies of cycling displays in which the first frame is either masked or are gradually ramped on suggest that sensitivity to phase differences between figure and ground is predominantly due to presentation of the figure in the first frame of the display sequence (e.g., Dakin & Bex, 2002). In contrast to this, the hypothesis that grouping and segregation are signaled by synchrony and asynchrony respectively predicts no asymmetry, as it implies that it is simply the duration of the asynchrony that matters, not its polarity. 

Third, is the salience of stimuli within a scene signaled by response latency? Various considerations suggest that it is, and a strong case has been made for this hypothesis (e.g., VanRullen & Thorpe, 2002). The basic idea is that neural responses to salient stimuli are facilitated so as to speed their transmission through the visual pathway. Salient stimuli could thus be distinguished from a cluttered background and could be processed more rapidly simply by selecting only the first few spikes and suppressing later spikes. The speed with which some figures can be detected and recognized supports this hypothesis, as does the finding that spike latency predicts spike frequency and can be computed much more rapidly. Several neurocomputational models show this idea to be feasible (e.g., Thorpe, Delorme, & Van Rullen, 2001). Our study of the effects of the polarity of onset asynchrony provides a simple test of this hypothesis. If salient stimuli are segregated from the background by having shorter latency, then displaying figures a little before the background should make them more salient, and showing them a little later should make them less salient. 

Fourth, does segregation by onset asynchrony interact with spatial cues? Various considerations suggest that it may. The segregation of a continuous contour, or path, from a background of random elements by collinearity of the contour elements has been much studied, and there is much evidence that this is mediated by “association fields” that link neighboring collinear or near collinear feature detectors (Hess, Hayes, & Field, 2003). It is therefore possible that the efficacy of temporal cues to grouping depends upon modulating such spatial interactions, and there are various sources of empirical support for this prediction, as reviewed by Blake and Lee (2005). 

Fifth, are there differences between individuals and mental states in the asynchrony required for segregation? Several considerations suggest that they may be important. First, measures of other forms of temporal sensitivity are commonly used to index arousal or alertness, particularly in psychopharmacological studies. These include critical flicker-fusion frequency (CFF), and dual-pulse resolution (DPR). Second, schizophrenia and schizotypy involve various perceptual deficits. These are particularly clear in tests thought to reflect processing in the magnocellular visual pathways (Butler & Javitt, 2005; Slaghuis, 2004), and, as these pathways signal rapid change, schizophrenia spectrum disorders may also involve impaired use of temporal cues for perceptual organization. Steady-state visual and auditory-evoked potentials are also reduced at higher frequencies in these disorders (e.g., Krishnan et al., 2005; Light et al., 2006), which suggests that they involve a reduced ability to track and segregate rapidly changing events. This also fits with the increased backward masking that has been observed in these disorders (e.g., Green, Nuechterlein, Breitmeyer, & Minz, 1999; Slaghuis, 2004). Furthermore, many other aspects of dynamic Gestalt organization are also impaired (Uhlhaas & Silverstein, 2005), and it is possible that reduced temporal sensitivity could contribute to several of these impairments. Finally, various other cognitive processes require the precise temporal synchronization and segregation of neural activities, such as communication between brain regions (Fries, 2005), selective attention (Serences & Yantis, 2006), and working memory (Tallon-Baudry, Kreiter, & Bertrand, 1999). All these processes are impaired in disorganized forms of schizophrenia (Phillips & Silverstein, 2003; Uhlhaas & Singer, 2006), so some have suggested that many of the cognitive deficits in schizophrenia may be due to a core impairment in the precise temporal organization of cortical activities (e.g., Green & Nuechterlein, 1999; Light et al., 2006; Spencer et al., 2004), which is in good agreement with the idea of a common cortical algorithm (Phillips & Singer, 1997) and its impairment in schizophrenia (Phillips & Silverstein, 2003). 

The goal of the current studies was to develop a simple, sensitive, and specific paradigm that can be used to study these issues. The paradigm that we have developed measures the onset asynchrony required for pre-attentive figure-ground segregation. It includes a short version of the paradigm that provides an estimate of the asynchrony required for segregation within a few minutes. This is needed so that we can screen large numbers of participants in the study of individual differences, and test participants, such as patients or young children, who cannot sustain attention over a long sequence of trials. 

In the paradigm introduced here, figure and ground are displayed at various stimulus onset asynchronies (SOAs) as shown in Figure 1. When the figure appears first, these are called positive SOAs. When the background appears first, they are called negative SOAs. The background is a large array of randomly oriented Gabor patches, and, in most experiments, the figure is a subset that forms a randomly curving path randomly located within the background. We use this task because it has already been much used to study figure-ground segregation by spatial cues (e.g., Field, Hayes, & Hess, 1993; Hess et al., 2003). Those studies are usually described as being concerned with “contour” integration. The notion of a “path” is more general in that it includes figures in which the individual elements are not aligned with the path or are not oriented elements. Our paradigm differs from those previous studies in that in order to study the role of temporal cues we set the parameters defining the spatial relations between figure and ground elements such that the spatial cues alone are too weak to support reliable detection. This can be seen by looking at Figures 1A and 1B where the path is hard to see in the final frame alone, which is all that is displayed at zero SOA. The figure is made visible by displaying it either a little earlier or a little later. This use of the path detection task provides precise control of the strength of both temporal and non-temporal cues. In further experiments, we also study detection of figures formed by randomly locating elements within regions of various sizes. 

Figure 1

 

Examples of the stimuli and display sequences used. In the left sequence, the target path is displayed first (positive SOAs of 20, 40, or 100 ms) and remains present in the final display in which background elements are added. In the right sequence, the background elements are shown first (negative SOAs of 20, 40 or 100 ms), and then the path elements are added. The path onset was always at 1000 ms after fixation point onset, and it was always presented for 300 ms.

The paradigm developed here is based upon earlier studies of change detection (Phillips & Singer, 1974; Singer & Phillips, 1974; Wilson, 1981) and of pop-out from abrupt visual onsets (Hancock & Phillips, 2004). It has some similarities to masking paradigms and indeed some may consider it to be a variety of Type A masking. Breitmeyer and Ogmen (2007) offer the following definition of visual masking: “the reduction or elimination of the visibility of one brief (≤50 ms) stimulus, called the “target,” by the presentation of a second brief stimulus, called the “mask.” By that definition this paradigm is not masking, as our target is always on for 300 ms, and the background for between 200 and 400 ms. In addition, background elements do not reduce the visibility of target elements, although they can affect their salience. Given that the underlying mechanisms of both masking and our paradigm are not fully understood, we remain open to the possibility that they are, indeed, related. We discuss this relationship further in the Relations between visual masking and segregation by onset asynchrony section. 

Our paradigm also has some similarities to studies of displays where a rapid sequence of changes occurs either deterministically or stochastically and with various phase relations between figure and ground, but it differs from them in that there is not a rapidly repeated sequence of events on each trial but simply a single presentation of figure and ground, and with onsets that may be either synchronous or asynchronous. These differences require a modified terminology. Here the stimulus onset asynchrony (SOA) is simply the onset asynchrony of target and background elements, not the display time prior to the replacement of one stimulus by another. Furthermore, as the offsets of target and background are always synchronous, SOA here is not confounded with stimulus offset asynchrony, as it is in most previous paradigms. 

General methods

The paradigm used is shown in Figure 1. The figure to be detected was a continuous path of elements embedded within a large array of elements. The task was a spatial two-alternative forced-choice (2AFC), in which observers indicated whether the figure was in the array to the left or the right of fixation. This stimulus design was developed from the contour integration paradigm that has been much used to study pre-attentive figure-ground segregation by spatial cues (see review by Hess et al., 2003). We adapted this paradigm to the study of temporal cues by making the spatial cues too weak to reliably support detection when there was no onset asynchrony. Varying the strength of the spatial cues then enabled us to study their interactions with the temporal cues. The form of the path and its location within the background array varied randomly from trial to trial. 

Each of the two arrays was 256 pixels (9°) square. The display elements consisted of center symmetric Gabor patches, with a sine wave frequency of 7 pixels (0.25°) and a Gaussian envelope standard deviation of 3.5 pixels (0.125°). The possible locations for display elements averaged 20 pixels (0.7°) apart on each axis, with a position jitter of up to 20 pixels, subject to not overlapping. For most experiments, only 50% of all possible distracter elements in each array were displayed, as in Figure 1. The two arrays were separated by 25 pixels (0.88°), with a fixation cross at the center. A path of seven elements was placed in each array, but the orientation of the Gabor patches for one of them was randomized. The path angle between each element was usually 20 degrees, and the spacing between path elements varied from 25 to 45 pixels (0.88° to 1.58°) across experiments. In most experiments, only the aligned path could onset asynchronously from the distracter elements, but in one condition both the aligned and the unaligned paths could onset asynchronously. Which array contained the aligned path and its location within that array was selected at random. 

On each trial, the total display time was 1300 ms. At time zero the central fixation cross was presented, with two rectangular arrays being at a uniform gray level of 127 (32.5 cd/m²), the remainder of the screen at gray level 100 (25.5 cd/m²). On all trials, path onset was at 1000 ms, so, on synchronous trials, the distracter and path elements appeared simultaneously after 1000 ms and remained present for 300 ms. On positive SOA trials, the distracter elements appeared after the path, usually by 20, 40, or 100 ms. On negative SOA trials, they appeared before the path, again usually by 20, 40, or 100 ms. Distracter elements were therefore displayed for between 200 and 400 ms in total. After the end of the display sequence, the whole screen returned to a uniform gray level of 100. As RTs were not measured, there were no time limitations on response. The next trial began approximately 1 second after response. 

The monitor used was a Panasonic S110 21 inch CRT, set to 800 × 600 pixels at 100 Hz refresh rate. The gray scale was linearized from maximum brightness (gray level 255, 65 cd/m²) down to gray level 8 (2 cd/m²), and the test room was lit by diffuse lighting at approximately 10 Lux. Michelson contrast for the Gabor patches was measured at 0.73. Distance to display was 80 cm, giving a visual angle for the two arrays of 19 degrees wide by 9 degrees high, with each Gabor patch approximately 0.56 degrees. Programs were written in Matlab: data reported in Figures 2, 7, 8 and 10 were obtained using a Cambridge Control VSG2/3 card and software, those in Figures 4, 5 and 11 used the Psychophysics Toolbox (Brainard, 1997). 

Figure 2

 

Mean proportion of correct responses of 63 students. Stimulus onset asynchronies (SOAs) are shown as negative when the distracters are shown first, and as positive when the path is shown first. Standard errors are shown by the bars around each mean.

The asynchrony required to detect a continuous path hidden within a random array

This experiment used a short form of the paradigm that is being developed to facilitate study of individual differences and pathologies in the use of onset asynchrony for perceptual grouping. It therefore uses only a few trials per condition and was made as simple as possible for observers to understand, details in the procedure section below. The results given here are those from a large population of psychophysically naïve students. 

Methods

Observers

Sixty-three (25 male) students at the University of Stirling took part in return for course credit. Age ranged from 17 to 48, with a median of 19 years. 

Stimuli

The stimuli were as described in the General methods section except that path angle was 30 degrees and there was ±10 degree random variation in the orientation of the individual Gabor patches. Inter-element distance along the path was 35 pixels (1.25°). 

Procedure

Each observer completed a practice block of 28 trials with a reduced path element spacing of 25 pixels (0.88°), which makes the path easier to detect using spatial cues alone. They then completed a total of 84 experimental trials, 12 in each of the seven SOA conditions, which occurred at random across trials. They then answered a Schizotypal Personality Questionnaire and an Autism Quotient questionnaire. Data from the questionnaires will be given in the Individual differences section. 

Results

The mean psychophysical data from the 63 students at seven different SOAs is shown in Figure 2. Each point is the average of 756 responses. A repeated-measures within-subject analysis of variance was performed to examine the effect of stimulus onset asynchrony. Maunchly's test of sphericity was found to be significant, so the Greenhouse–Geisser epsilon correction was used. The main effect of timing was highly significant, F(6,372) = 75.8, p < 0.001, partial eta squared = 0.55. Post hoc Bonferroni-adjusted pairwise comparisons revealed that asynchronies of 20 ms facilitate detection, asynchronies of 40 ms improve detection even more, and a further increase to 100 ms results in a marginally significant improvement over detection at 40 ms. There were no significant differences between positive and negative SOAs of the same magnitude. 

At synchrony, performance did not differ significantly from chance (resampling, p > 0.05). At 40 ms, it was almost as good as at 100 ms, which is near to 90% correct detection in the data averaged across participants as here. The analysis reported in the Individual differences section shows that this asymptotic performance is at 100% for many of the observers studied. Performance at 20 ms is clearly better than chance, but well short of that at 40 ms. The most striking finding here is that performance is symmetric around zero SOA, i.e., in contrast to expectations, it makes no difference whether figure or ground is shown first. 

Detection and discrimination of paths formed by aligned and unaligned elements

The above results seem to suggest that events with an asynchrony of less than 20–40 ms are perceived as a group. This is not necessarily so, however, as it is also possible that they are simply perceived as individual onsets that are processed independently of each other. One way to show that they are perceived as a group would be to show that the asynchrony required for segregation depends upon the spatial relationships between the asynchronous elements. There may be no such dependence, however, even if the asynchronous elements are processed as a group. One of the possible computational advantages of using temporal signals is that they can in principle group or segregate any set of features whatever, without restrictions such as those set by the “association fields” through which spatial cues may operate. If the visual system makes use of this freedom then grouping and segregation by synchrony and asynchrony will not be constrained to apply only to paths formed by aligned elements. 

One task used here was therefore as above, but in one condition the elements lay along the underlying path, and were aligned with it, whereas in another condition they lay along the path, but were not aligned with it. Examples of the stimuli used are shown in Figure 3. When figure-ground segregation is based on spatial cues alone, the path in the unaligned condition is either more difficult or impossible to detect, depending upon the relative density of path and background elements. 

Figure 3

 

Examples of elements that are and are not aligned with the underlying path.

A more direct way to show that the synchronous target elements are perceived as a group is to show that some property defined on the group as a whole can be perceived. In a third condition, observers were therefore asked to discriminate between aligned and unaligned paths when both were presented asynchronously from the distracter elements, and with one being in the array on the left of fixation and the other being on the right. This is a subtle discrimination and requires the computation of alignment relations between the path elements. Thus, in that sense, this discrimination reflects a grouping process. Given sufficient spacing between the path elements, detection and discrimination are at or near chance when there is no asynchrony, so successful detection and discrimination must be based on the asynchrony between the transient neural signals produced at path and distracter onset. 

Methods

Observers

Six observers took part: three with knowledge of the aims of the study and three naïve to them. 

Stimuli

Stimuli for the aligned path detection task were as above, but with a spacing of 45 pixels (1.58°), rather than 35 pixels, between path elements. This was also so for the unaligned path detection task, except that paths were generated for both arrays as in the previous study, and the orientation of the Gabor patches for both was then randomized. At zero SOA in this condition, therefore, there was no cue as to which side the path was on. For the discrimination task, the orientation of the elements along the path in only one of the arrays was randomized, but both aligned and unaligned paths appeared with the same asynchrony from the distracter elements. 

Procedure

Seven SOAs were used, as above, randomized within blocks. Each observer completed four blocks of 84 trials (12 per SOA) for each of the three conditions (aligned detection, unaligned detection, and discrimination), which were interleaved in the order 123, 321, 123, 321 to reduce any order effects. A gap ranging from a couple of hours to a couple of days was left between sets of three blocks, to prevent fatigue. 

Results

The performance of all six observers is shown in Figure 4. For all, detection of aligned and unaligned paths was very similar, except that at zero SOA there was a tendency for detection to be slightly above chance for the aligned path. This is as predicted because, even at a path element spacing of 45 pixels, it is occasionally possible to detect the aligned path in the unchanging 300 ms display of the two arrays, whereas this is not possible for the unaligned path. Only one observer, i.e., that shown bottom right in Figure 4, showed better detection of aligned over unaligned paths at SOAs of 20 ms or more. However, the average results, shown in Figure 5A, also suggest a difference between aligned and unaligned at 20 ms offset. This difference is explored further in the Effects of asynchronies between 8.5 and 50 ms section. 

Figure 4

 

Detection and discrimination of paths formed by aligned and unaligned elements by three informed and highly practiced observers (A–C ) and by three naïve observers (D–F). Red squares show aligned path detection; green circles show unaligned path detection; and blue Xs show discrimination between aligned and unaligned paths.

Figure 5

 

(A) Average of all six observers combined for aligned path detection (red squares), unaligned path detection (green circles), and discrimination between aligned and unaligned paths (blue crosses). (B) Average of three observers: aligned paths (red squares) and unaligned paths (green circles).

Although discrimination was clearly less accurate than detection, onset asynchrony did enable all observers to discriminate between paths formed by aligned and unaligned elements at substantially better than chance levels. Thus, the synchronously displayed target elements that were segregated from the background by onset asynchrony were grouped in that the alignment of the individual elements with the underlying path could be computed. To the observer in this experiment, the target elements are clearly seen as forming a continuous path, whether with or without aligned elements, so it is likely that discrimination of less subtle holistic properties would be even more accurate than that found for the discrimination used here. 

When SOA was +20 ms, i.e., when the paths in each array were presented 20 ms before the distracter elements, discrimination was close to that for zero SOA for all but one observer. Overall, there was a slight asymmetry around zero SOA, with positive SOAs giving slightly lower performance than negative SOAs, as shown in Figure 5. Performance at +20 does not differ from that at zero, while −20 does; +40 is worse than −40 (both p < 0.01 by resampling). A possible explanation for this is that the transient activity produced by the distracter onset interferes more with the processing required to discriminate between the two paths when it occurs after rather than before the transient activity produced by the paths. 

Effects of asynchronies between 8.5 and 50 ms

Detection of aligned paths is better at 0 ms SOA in Figure 5A because of residual spatial cues to the target. Detection is significantly better also at 20 ms offset, which raises the question of whether there is any synergy between the timing and the spatial cues (i.e., whether a combination of temporal and spatial cues is more detectable than the sum of the two alone). To study this more carefully, three highly practiced observers (PH, WP, and YP) ran the task at asynchronies of 0, 8.5, 17, 25, 33, 42, and 50 ms (monitor refresh rate 120 Hz) for aligned and unaligned path detection. Other parameters were as above, except that only the case where the distracters are presented first was used. The conditions were run in counter-balanced order. 

Detection at 8.5 ms was not significantly different from 0 ms (p > 0.5, resampling), while close to 100% for SOAs of 33 ms or more, as shown in Figure 5B. In the aligned condition, performance is again significantly better than chance at synchrony (proportion correct 0.604, p < 0.01 by resampling). Importantly, however, there is no evidence for any interaction with SOA. If asynchrony and spatial cues combined synergistically, then the detectability of aligned contours should rise relatively more rapidly with SOA. Instead, the performance on the aligned condition is consistent with a simple additive probability model, using performance on aligned at zero SOA to give the probability of detection by spatial cues alone, and performance on the unaligned condition for the contribution of timing alone (resampling, p > 0.2 at each time point). 

The effect of a steady-state background on the detection of asynchronous paths

The above results show that the transient responses to the abrupt onset of stimuli are crucial to detection of the asynchronous path because detection required an onset asynchrony. If the figure-ground segregation observed in those conditions depends upon those transient responses to onset being processed in a way that is independent of the sustained responses to a steady-state background, as is sometimes suggested (e.g., Dakin & Bex, 2002), then the presence of such a background should have no effect. To test this prediction, aligned path detection was studied, as above, except that a background of random Gabor elements was displayed together with the fixation point, as shown in Figure 6. Any transient responses produced by the onset of this steady-state background would have decayed long before the transient activity produced by the much later path and distracter onset, so should have no effect on detection if that depends only on the processing of transient neural activity. 

Figure 6

 

Examples of displays with a prior steady-state background with a contrast that is either low (left) or equal to that of the path and distracters (right).

Methods

Observers

Three highly practiced observers, PH, BP, and YP. 

Stimuli

The 50% of possible Gabor elements that were not displayed in the previous studies, as described in the General methods section, were used to provide a steady-state background, with an onset at time zero. There were two conditions: zero contrast (i.e., no background, as in the previous studies) and high background contrast (100%), equal to all other display elements. Examples are shown in Figure 6. There were 7 elements in each path, spaced at 45 pixels (1.58°). Display onset timing was zero and ±20, 30, and 40 ms. 

Procedure

Each observer completed 96 trials for each level of timing and background. Background condition was blocked, eight blocks each, alternating order. 

Results

Results for each observer and the average of all three are shown in Figure 7. Using resampling statistics, the background and no background performance differ (p < 0.01) for only three pairs: −30 ms for PH and ±30 for YP. The average results differ only at +30 ms, a consequence of the anomalous-looking result for YP. Note that the difference is in the direction of being easier with a background present. 

Figure 7

 

The effects of a steady-state background on the detection of paths distinguished by asynchrony. Red squares show no background; green circles show background.

As predicted, the addition of a steady state background has minimal effect on this task. This is additional evidence against a masking interpretation of the paradigm, as it would be expected that more non-target elements in the display would make detection harder. 

Individual differences in the asynchrony required for segregation within and between groups of students and schizophrenia patients

The above results suggest that the asynchrony required for segregation is much the same in a wide variety of stimulus conditions, and lies between 20 and 40 ms, as in our studies of pop-out from onsets (Hancock & Phillips, 2004). This supports the view that this asynchrony reflects a constraint having wide generality. It might therefore be supposed that it is much the same across individuals and mental states. The introduction gave reasons for doubting this, however, and here we show that there are wide variations across individuals. The results reported here for schizophrenia patients should be viewed as only preliminary because sample size is small and we do not yet have extensive background information on those patients. Nevertheless, the data are reported here because they greatly encourage a more in-depth use of this paradigm with patients and people who may be at risk of psychosis. 

Methods

Here we report further data on the 63 students whose psychophysical performance is reported in Section 3, and also on 9 chronic schizophrenia in-patients, who were tested using the same conditions as the students. Both groups also completed the Schizotypal Personality Questionnaire (SPQ) and the Autistic Quotient questionnaire (AQ). 

Materials

The Autism Quotient (AQ) is a self-report questionnaire consisting of 50 items, with four responses from definitely agree to definitely disagree, designed to assess communication, imagination, social skills, attention to detail, and attention switching (Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, 2001). A score of 32 or more is regarded as clinically significant. 

The Schizotypal Personality Questionnaire (Raine, 1991) contains 74 items designed to measure nine traits associated with a schizotypal personality. A score greater than 49 is more than 2 standard deviations above the mean for the original population used to develop the instrument. 

Observers

The patients were nine (8 male) chronic in-patients at Stratheden Hospital in Cupar, Fife. Age ranged from 25 to 48, median 37. Inclusion criterion was a DSM diagnosis of schizophrenia or schizo-affective disorder, which was verified by a consultant psychiatrist. Sampling method was opportunistic; potential observers were identified by hospital staff and, following approval from the consultant psychiatrist, were approached and invited to participate in the study. Ethics approval was obtained from the local NHS ethics committee, as well as from that at the University of Stirling. 

An independent samples t test showed that there was a significant difference in age between the student group and the schizophrenic group, t(70) = 8.13, p < 0.001. However, a lack of significant correlation between path detection accuracy and age within the student group indicated that group differences in task performance did not result from age. Further analysis of covariance, examining both groups, showed no interaction between age and path detection; F(6,414) = 0.62, p = 0.68. Therefore, age was not included as a covariate in the reported analyses. 

Procedure

The patients' ability to detect aligned paths as a function of SOA was tested using the same stimulus conditions, equipment, and procedures as for the 63 students whose mean results are reported above, except that the patients were tested in the hospital. Each patient completed a practice block of 28 trials and then a total of 84 experimental trials, in all taking less than 10 minutes. They then completed the SPQ and the AQ as did the students. 

Results and discussion

Data from the Schizotypal Personality Questionnaire (SPQ) and the Autism Quotient (AQ) are given in Table 1. Both scales clearly distinguish the two groups. The absence of a significant group difference on the attention and attention-to-detail subscales of the AQ reflects earlier data casting doubt on their validity. It is important to emphasize that the patient group scored highly on the disorganization subscale, as previous results suggest that it is such patients in particular who may have perceptual impairments, as noted above. 

Table 1

 

Means and standard deviations (SD) on the Autistic Quotient (AQ) and Schizotypal Personality Questionnaire (SPQ) for the 63 students (control) and 9 chronic schizophrenia in-patients.

		Control		Clinical		Mann–Whitney U	Exact Sig (2 tailed)
Mean	SD	Mean	SD
AQ	Total	15.1	5.5	24.9	3.5	29.0	<0.001
	Social skill	1.7	1.7	4.0	2.2	88.0	0.006
	Attention	4.1	2.1	5.6	2.4	142.0	0.124
	Attention to detail	5.0	2.1	5.9	2.0	178.5	0.411
	Communication	2.2	1.9	4.9	1.3	59.0	0.001
Imagination	2.1	1.7	4.6	1.0	47.0	<0.001
SPQ	Total	19.7	12.1	48.3	6.1	13.5	<0.001
	Cognitive	8.2	7.0	21.1	2.7	34.5	<0.001
Interpersonal	8.0	6.0	23.0	4.2	10.0	<0.001
	Disorganization	5.5	3.8	10.6	1.3	60.0	0.001

Mean path detection for the 9 patients is compared with that of the student group in Figure 8. Performance of the patient group is clearly impaired. A mixed analysis of variance was performed on path detection accuracy with a within subject factor of timing and a between subject factor of group. Maunchly's test of sphericity was found to be significant, so the Greenhouse–Geisser epsilon correction was used. The main effect of timing was highly significant, F(6,42) = 13.45, p < 0.001, partial eta squared = 0.16; the difference between the clinical group and the student group was highly significant, F(1,70) = 27.4, p < 0.001, partial eta squared = 0.28, showing a large effect size; and there was a significant interaction between group and timing, F(6,420) = 7.40, p < 0.001, partial eta squared = 0.096. 

Figure 8

 

Aligned path detection as a function of SOA for 9 chronic schizophrenia in-patients (green circles) and 63 students (red squares).

These means mask individual differences, however. Performance of each of the 63 students and 9 patients when SOA was ±100 ms is shown in Figure 9. As there were 12 trials at each of these two SOAs, this shows the total number of correct detections out of the 24 trials. Most students performed much better than chance, as do two of the patients. The other seven patients performed at chance, as did seven of the 63 students. 

Figure 9

 

Aligned path detection at SOAs of ±100 ms for each of 63 students (shown as an o) and each of the 9 patients (shown as an x). There were 24 trials for each participant, so 17 or more correct is significantly better than chance.

The reasons for these differences within and between groups remain to be determined, but they clearly raise important questions. Do they indicate important subtypes within the group of patients? Does the subset of students with poor ability to segregate by asynchrony reflect an underlying liability to psychosis? This we shall not know until more data is collected concerning these individual differences within the student group, but these results clearly raise that possibility. It is therefore important to note that four of the seven students, highlighted in Figure 9, had clinically significant self-report SPQ and/or AQ scores that were comparable to those of the patients. 

There may be other explanations for the poor performance of the patients, for example, they may not have understood the task, or they may not have been able to maintain attention for the few minutes required. Neither of these is likely, since the practice trials showed that they did understand and attend to the task. Nevertheless, more formal demonstration is required. We therefore tested four of the nine patients again, using a wider range of SOAs. Results are shown in Figure 10. Path detection by the patient shown on top right is clearly constrained by SOA, but with an SOA of 160 ms or more being required for segregation. The two patients shown top left and bottom right are the two whose performance was close to the student mean. Performance of the patient shown on bottom left is close to chance at all SOAs used. We have just obtained ethical permission to study these differences further. 

Figure 10

 

Segregation by asynchrony measured over a wider range of SOAs for each of four chronic schizophrenia in-patients (green circles) compared with student controls (red squares).

Other findings

We present here brief reports of three of the many variations on our paradigm that we have investigated. The aim is to demonstrate the generality of the principal results, that the critical asynchrony required for segregation is in the region of 20–40 ms and is generally symmetrical, despite changes such as not using paths or substituting simple dots for Gabor patches. There are hints of some more subtle interactions in the data, which we are continuing to explore. 

Detection of figures formed by randomly locating elements within regions of different size

In this study, the target elements were not constrained to form a path but were randomly located within regions of various sizes. Two observers, PH and WP, detected targets formed by randomly placing seven Gabor patch elements within randomly located regions of diameter 160, 200, 240, or 280 pixels, of one array. These sizes correspond to approximately 5.6, 7.0, 8.4, and 10 degrees at the viewing distance of 80 cm. Nothing was added to the non-target array, so it would be possible in principle to distinguish path and non-path arrays by counting the total number of elements in each array. 

For all four region sizes, detection was close to chance at zero SOA, as shown in Figure 11A, so differences in the total number of elements within each array were not a sufficient spatial cue to support detection. There were no systematic differences between the different region sizes, and performance was comparable to that for the path detection task, requiring asynchronies of about 20 ms or more. 

Figure 11

 

(A) Average of two observers detecting elements within a patch, size 160 (red squares), 200 (yellow circles), 240 (green X), and 280 (blue diamonds). (B) Average of two observers on detecting one (red squares), three (yellow circles), five (green X), and seven elements (blue diamonds). (C) Average of two observers detecting paths of dots: spacing 10 pixels (0.35°, red squares) and 40 pixels (1.4°, green circles).

Effects of the number of asynchronous elements

Observers PH and WP detected targets formed by randomly locating 1, 3, 5, or 7 elements within a randomly located patch of radius 140 pixels (4.9°) of one array. We used SOAs of −100, −40, −20, 0, 20, 40, and 100 ms as above. 

Performance was at chance at zero SOA, again showing that differences between the total numbers of elements in the two arrays was not a sufficient cue for target detection. Detection when there were 7 elements in the target varied with SOA as previously found above; being near 100% when SOA was 40 ms, as shown in Figure 11B. Detection was clearly worse when there was only a single asynchronous element; being well below 100% at 40 ms. Detection was at intermediate levels when there were 3 or 5 figural elements. 

Effects of using dots rather than Gabor patches

PH and WP detected figures using Gabor element parameters set to produce an unmodulated 8 pixel diameter filled circle (a “dot”, 0.28°). Inter-element spacing on the path was set to 10, 20, 30, or 40 pixels (0.35° to 1.4°). As in the standard method, there was a path on both sides of the display, so with dots for elements, there was nothing to distinguish the two arrays at zero SOA. Results for 10 and 40 pixel spacing are shown in Figure 11C. Performance is necessarily at chance at zero SOA and was near 100% by 40 ms. Performance at 20 ms was better for a spacing of 10 pixels. 

General discussion

We stress four principal findings from these results. First, the asynchrony required for segregation is between 20 and 40 ms. Second, it is symmetric about zero. Third, it is largely independent of spatial cues to segregation. Fourth, there is evidence of reduced sensitivity to asynchrony in at least some forms of schizophrenia and a suggestion of an association with schizotypy. The empirical findings are clear and robust. For all of the various targets used, the asynchrony required for segregation was 20–40 ms for most participants. Detection was no better when the figure was presented first, and thus by itself, than when the distracter elements were presented first. This finding is much the same as in Hancock and Phillips (2004), but the symmetry of the effect of SOA is more surprising here because when the path was presented first it was shown in a blank field, thus requiring no figure-ground segregation, whereas when it was presented second it was present only in the final frame and could not be detected in that frame alone, as shown by near chance performance at zero SOA. Thus, segregation in this paradigm depends upon the extent of the asynchrony, but, contrary to predictions given in the introduction, not upon the temporal order of figure and ground onsets. For the cues used here, there was little evidence of any substantial interaction between the effects of spatial and temporal cues. As sensitivity to asynchrony was much the same for unaligned paths, or for paths of dots, as for aligned contours, its effect cannot be due to modulation of mechanisms, such as association fields, that may implement the effects of spatial cues. Thus, sensitivity of grouping and segregation to onset asynchrony is not secondary to the effects of spatial cues. The paradigm developed here therefore seems to be successful in providing a highly specific measure of the asynchrony required for segregation. We have also shown that it can be used in a short standardized form that is sensitive to individual differences and, perhaps, to pathological conditions. We can thus obtain these measures from a wide variety of participants and conditions. Whether it is worth doing so or not depends upon what significance they have, so the following discussion examines various issues in depth. 

The asynchrony required for segregation and its relation to theories of rate coding

The asynchrony required for segregation was constant across a variety of experimental conditions. The 20- to 40-ms range found for most observers is similar to the widths of the cross-correlogram peaks found in many neurophysiological studies (Engel & Singer, 2001). There is a simple functional interpretation for this result. Different cortical regions respond to stimulus events with latency differences of up to about 20 ms (Lamme & Roelfsema, 2000). If activities with asynchronies of less than that were segregated, then activities arising from the same distal object would be segregated rather than being processed as a whole. Onset asynchronies of less than 20 ms are therefore not used to segregate activity into discrete subsets. 

Some earlier findings have been interpreted as showing sensitivity of grouping processes to temporal asynchronies of a few ms (e.g., Alais, Blake, & Lee, 1998; Fahle, 1993; Usher & Donnelly, 1998), We find no evidence for sensitivity to asynchronies of 10 ms or less in our paradigm, however. One possible explanation for this contrast is that the findings suggesting sensitivity to asynchronies of a few milliseconds may instead be due to the effects of fine involuntary eye movements (Dakin & Bex, 2002; Waliis, 2005) or to specialized luminance, change, or motion detection filters (e.g., Farid, 2002; Morgan & Castet, 2002). If so, the absence of any such fine temporal sensitivity in our paradigm suggests that it avoids those particular difficulties. Furthermore, small movements of the array coincident with the change do not enhance change detection in paradigms related to that used here but impair it (Phillips, 1974). This also indicates that the effects of SOA observed above are not due to the translation of temporal into spatial differences by eye movements. 

Evidence for very fine temporal sensitivity has also been obtained by Greene (2006, 2007); however, he finds that shape recognition is impaired by sub-millisecond asynchronies. This clearly contrasts with our failure to find sensitivity to asynchronies on that time scale. There are several differences between his paradigm and ours, however. For example, he used a large array of LEDs rather than a CRT and studied segregation within figures rather than segregation of figures from ground. Which of these differences are crucial to the contrasting findings remains to be determined. 

The asynchrony found here to be required for segregation is compatible with recent theories concerning the role of rate coding in neural information transmission. Rolls, Franco, Aggelopoulos, and Jerez (2006) measured information transmission by various possible neuronal codes and conclude that the most efficient and plausible measure of input rate at particular synapses is the number of spikes within a time period of about 20 ms. They argue that this is compatible with the time over which neurons integrate their inputs as determined by the combined operation of synaptic and membrane time constants and with the speed of feedforward transmission through the hierarchy of visual areas. Transient responses to onsets that occur within less than 20 ms at any stage would therefore produce a single integrated output and that agrees with the asynchrony that we find to be necessary for reliable segregation. Evidence for rate coding is not evidence against a role for synchrony, however, so we now discuss that possibility in more depth. 

Relevance to the synchronization hypothesis

If the number of spikes received within 25 ms is a major component of neural coding, then this strongly suggests that synchrony of separate inputs to within 25 ms is also crucial. Cortical neurons rarely transmit 5 or more spikes within 25 ms and usually only 1 or 2 spikes (Rolls et al., 2006). By themselves, 1 or 2 spikes from a single input site are unlikely to produce post-synaptic transmission, so synchronization to within about 25 ms with other inputs will usually be required. Our results indicate that an asynchrony of around 25 ms is required for segregation. If it had been much shorter (∼ a few ms), this would leave little time for rate codes to be useful. If it had been longer (∼100 ms), there is less need for synchronization, as rate codes might be sufficient. This suggests that synchronized rate codes play a major role in neuronal communication. Although our results suggest an integration time of about 25 ms, this timescale may not be rigidly fixed and could be under dynamic control from various sources, thus producing some of the individual variations in sensitivity to onset asynchrony found above. 

The synchronization hypothesis that is most debated goes further than this, however. It proposes that internally determined temporal relations between spike trains signal grouping and segregation. Studies of externally determined temporal relations cannot show that any such internal use occurs, but they can test some aspects of that hypothesis. Here we note some ways in which our results support the hypothesis but then go on to emphasize some reservations. First, the marked symmetry of the SOA effect is consistent with the hypothesis because it supports the implication that it is the amount rather than the direction of asynchrony that matters. Second, a set of elements lying along a path but not aligned with it could be detected and discriminated from those lying along a path. This implies that synchronous onsets are processed as a group because subtle holistic relationships within them can be computed. Third, an argument frequently made in support of temporal cues and codes is that they are free to signal indefinitely many novel groupings. This contrasts with grouping by spatial cues, which can group only those inputs meeting strict criteria, such as those embodied within “association fields”. This greater freedom of temporal cues is clear in the above results. We found that alignment of elements with an underlying path had little or no effect on the asynchrony required for segregation, and that, although detection improves with the number of asynchronous elements, they need not lie along a path but can be randomly located. Finally, a phenomenological observation providing some support for the synchronization hypothesis is that, at SOAs of 20 ms, figure and ground were often clearly segregated but without their temporal order being at all clear. This agrees with other findings indicating that, in the absence of perceived motion, an asynchrony of 40 ms or more is usually required for reliable perception of the temporal order of distinct events. If this is confirmed by more formal studies within our paradigm then that would support the synchronization hypothesis because that implies a timescale at which asynchronies are used to signal perceptual organization but not temporal order. 

One major reservation concerning the role of synchrony in grouping that arises from our results is that we find no evidence for sensitivity to asynchronies at or below 10 ms. Any version of the synchronization hypothesis proposing that timing on such a fine time scale is used as an internally computed signal will have to explain why the visual system cannot use such time differences when they are externally generated. Although other findings have been interpreted as evidence for sensitivity of visual grouping to such small time differences, those findings are subject to various ambiguities as noted above. Another reservation is that we have found no evidence for oscillatory responses that are phase-locked to the stimulus. If such oscillatory responses were necessary for perceptual organization, then the effect of SOA would be non-monotonic. We have no evidence for any such non-monotonicity, but it must be noted that we have not yet applied structural equation modelling techniques that might reveal evidence for more subtle underlying oscillations. It must also be noted that our results provide no evidence against oscillatory responses that are not phase-locked to the stimulus. 

One possible response to our reservations concerning the timescale on which segregation occurs is to note that our findings relate only to the segregation produced by stimulus generated transients. Perhaps internally generated segregation by asynchrony of the sustained activity, where inter-spike intervals are much longer, operates on shorter timescales, and modelling studies suggest that this may be possible (Chapman, Wright, & Bourke, 2002). The final reservation is that the greater freedom of grouping by temporal cues emphasizes a major problem that is too easily neglected when grouping seems more constrained, as by association fields, for example. The creation of novel groupings that are signaled by synchrony requires mechanisms by which such groupings can be used. As they are endlessly variable, this is no trivial requirement and cannot be solved by convergence onto a higher level neuron that selectively responds to that set. This is essentially the problem that theories such as that of dynamic link architectures (von der Malsburg, 1999) are designed to solve, but whether they solve it in the same way as visual cortex remains a major outstanding issue. 

The contrast with previous studies of figure-ground presentation order

The predominant effect of onset asynchrony found here is striking in its independence of the presentation order of figure and ground. This contrasts with previous evidence showing that the transient responses to the first and final frames play an important role in the effects of phase relations between figure and ground in rapidly cycling displays (e.g., Beaudot, 2002; Dakin & Bex, 2002). The paradigm used by Beaudot (2002) is most relevant to our findings, so will be discussed in depth here. He found that figures were segregated more effectively when displayed first in rapidly cycling displays. This contrasts with our finding that temporal order has little or no effect on segregation by asynchrony, so what are the relevant differences between the paradigms used? Beaudot studied the effect of onset asynchrony on the ability to discriminate between elements lying along a path and aligned with it and elements lying along a path but not aligned with it. Figure and ground were alternated, whereas in our paradigm whatever came second was simply added to whatever was displayed first. Using rapidly cycling displays he found that discrimination was much better when the path rather than the ground was displayed first. This was only so when 4 or more cycles were presented, however. When only one or two cycles were displayed, presentation order had little or no effect, as in our paradigm. The use of cyclic displays, and even more complex temporal structures, therefore introduces rich non-linear interactions (Guttman et al., 2007). Although the explanation for the differences between flicker and single asynchronies was not clear, they led Beaudot (2002) to conclude that flickering stimuli may be inadequate to reveal the neural code for figure-ground segregation unless they control for the effects of stimulus onset. Our results now show that much can be learned from the study of single discrete onset asynchronies. This also has the advantage that the timings of the transient neural responses to the onsets are then closely locked to the stimulus timings. A further advantage is that it emphasizes a point, which, although obvious, is too often ignored. Segregation by asynchrony does not necessarily require oscillations, either in the stimuli or in the brain. 

Beaudot (2002) also studied the effects of presentation order in displays with only a single discrete onset asynchrony, as in our paradigm, and again found a large effect of display order with much better performance when the paths to be discriminated were presented before the background. This effect was only present at very short total display durations, however, and may have been due in whole or in part to the covariation of the display durations of the paths with onset asynchrony. Paths displayed first were also displayed longer. This will matter less when total display durations are already long, and for those Beaudot found little or no effect of presentation order. This latter result was not clear, however, because the strength of the spatial cues was such that at the longer display durations performance was high, even at synchrony, thus leaving little room to detect any segregation by asynchrony. Our results now clearly show that when brief asynchronies determine segregation there is little or no effect of temporal order. 

Relevance to the hypothesis proposing that salience is signaled by latency

Although similar to the synchronization hypothesis in emphasizing temporal codes, the latency hypothesis contrasts with it in proposing that the temporal order of spiking activity is important, with salient subsets being segregated from the background by having shorter latency (e.g., Rousselet, Thorpe, & Fabre-Thorpe, 2004; VanRullen & Thorpe, 2002). Much evidence and some strong arguments support this hypothesis, as noted in the introduction, but, prima facie, the above results are evidence against it. If subsets of elements that form a figure are made more salient by producing responses with a shorter latency, then displaying the figure shortly before the ground should make it more salient, and displaying it a little later should make it less salient. This is clearly not what happens. Figures displayed a little later than the ground are just as salient as those shown a little earlier. This seems at odds with the latency hypothesis and Simon Thorpe (personal communication) agrees that the absence of asymmetry is not what he would have initially expected. He observed that our stimuli are very sparse compared to natural images and suggested that with richer images, we might find an asymmetry. We are currently considering ways to test this. He also suggests that processing may occur in waves of processing, separated by 20–40 ms. 

Relations between transient and steady-state signals

It is tempting to suppose that grouping processes operate upon transient neural responses to abrupt onsets in a way that is wholly or largely independent of the sustained neural responses to pre-existing stimuli. Several findings have been interpreted as suggesting that this is so (e.g., Beaudot, 2002; Dakin & Bex, 2002). A different view is proposed by Hess, Dakin, and Field (1998), however. They report evidence that contour integration is not impaired by randomizing element contrast and hypothesize that contrast is signaled by the transient responses and grouping by the sustained responses. Hess et al. (2003) then make this the main conclusion of their wide-ranging review of contour integration. That hypothesis faces at least two major difficulties, however. First, it does not explain how the contrast of gradually ramped-on stimuli is signaled. Second, it implies that Gestalt organization does not apply to brief displays, but many studies of tachistoscopic perception imply that it does. It has now been specifically shown that contour integration can be achieved with display times prior to masking of as little as 60 ms (Mandon & Kreiter, 2005). This is strong evidence against the view that grouping is only signaled within the sustained activity that follows the transient response to onset. 

Our results indicate that both transient and sustained signals are important for perceptual organization. Small onset asynchronies are conveyed only by the transient signals, but are effective in segregating figure from ground. Results reported in The effect of a steady-state background on the detection of asynchronous paths section show that, in the absence of spatial cues, there is little effect of a steady-state background on detection by onset asynchrony. When there are spatial cues, however, then their contribution to detection is greatly weakened by the presence of a steady-state background. Thus, figure-ground segregation by onset asynchrony is based upon the processing of the transient onset signals in a way that is little affected by any sustained activity, but that can be combined with segregation by spatial cues, which is much affected by sustained activity. The way in which transient and sustained signals are combined in the formation of visual groupings is not yet clear but may involve interactions between magnocellular and parvocellular pathways. 

Finally, the above effects of a pre-existing steady state background show that the “visual marking” concept, which was developed to account for the effects of prior displays of background items in visual search paradigms (e.g., Olivers & Humphreys, 2003; Watson & Humphreys, 1997), does not apply to segregation by onset asynchrony as studied here, just as it does not apply to pop-out from abrupt visual onsets (Hancock & Phillips, 2004). 

Relations between visual masking and segregation by onset asynchrony

In previous studies of figure-ground segregation, the backgrounds do not mask the figures, but contrast with them, if sufficiently different (Nothdurft, 1992). Similarly here, the backgrounds do not mask the figures, but contrast with them, if sufficiently asynchronous. Nevertheless, prima facie, our paradigm may seem similar to visual masking in that performance worsens as the onset asynchrony between figure and ground is reduced. Furthermore, we find that the asynchrony required for segregation is increased in schizophrenia, which has already been shown to be associated with impaired performance in backward masking paradigms (e.g., Green et al., 1999). Our paradigm for studying segregation by onset asynchrony differs from masking paradigms in important ways, however. Masking techniques have been defined as follows “In visual masking, the visibility of a briefly presented target is reduced by a mask that is presented very shortly before or after the target” (Green, Mintz, et al., 2003, p. 1113). Our paradigm does not study the visibility of briefly presented targets. Both target and background were presented for several hundred milliseconds. Longer presentation would make little difference to the findings, because, at steady-state, the target cannot be reliably segregated from the background, as inspection of Figures 1 and 6 will confirm. The background elements in figure-ground segregation tasks do not make the target elements invisible by masking them. If they did, then the detailed properties of the target elements would not be expected to determine detection, but they do, as shown here and by many prior studies of figure-ground segregation such as those of contour integration and pop-out. Another difference is that in most masking paradigms targets are replaced by masks or vice versa. Neither form of replacement occurs in our paradigm. Yet another difference is that, in forward masking, prior presentation of the mask impairs performance. In our paradigm prior presentation of the background does not impair performance but, if anything, enhances it. 

As these differences show our paradigm to be primarily concerned with figure-ground segregation, rather than with masking, we would not expect the effects of SOA, as defined here, to be the same as the effects of stimulus onset asynchrony as found in masking paradigms. To test this, we surveyed the masking literature to find results similar to those reported above. Prima facie, those that seem the most similar are reported by Green, Nuechterlein, Breitmeyer, Tsuang, and Mintz (2003). They used a standard “masking by structure” paradigm, and, somewhat atypically, found forward and backward masking to have approximately the same effects. Dependence on SOA therefore appears similar to that found in our studies. The temptation to equate the two sets of results must be resisted, however. All the differences between our paradigm and theirs, as noted above, remain. For example, forward masking in their paradigm involved first briefly displaying the mask before replacing it, after a delay, by the target. When the background is displayed first in our paradigm, it is not removed but remains present throughout the remainder of the display. Thus, reduction of masking due to decay in the persistence of the mask after it is removed can play a major role in the dependence of forward masking on the asynchrony between mask and target, but it can have no role in our paradigm. Closer inspection of the dependence of performance on asynchrony as reported above and in that of Green, Nuechterlein, et al. (2003) shows them to be far less similar than may appear at first glance. Segregation by onset asynchrony depends on asynchronies between about 20 and 40 ms. Masking operates over much longer time intervals, up to at least 120 ms. Furthermore, Green, Nuechterlein, et al. (2003) found lower performance for schizophrenic patients in their masking paradigm, but the impairment found in their study was far less profound than that found here. 

Individual differences in sensitivity to asynchrony

In the Introduction section, we outlined grounds for predicting important individual differences in the asynchrony required for segregation. In particular, it was suggested that difficulties in distinguishing small temporal differences may reflect a basic underlying impairment in schizophrenia spectrum disorders. The aim of this paper was not to examine that hypothesis in detail but to establish an appropriate psychophysical paradigm for doing so. The preliminary findings reported above encourage this effort. Seven of the nine schizophrenia patients tested were no better than chance even at 100 ms asynchrony. Evidence that insensitivity to small asynchronies contributes to their impaired performance is provided by the much improved performance at longer SOAs seen in one patient. Not all of the patients were impaired, however, and a major goal for future work will be to relate differences between patients on this measure of segregation by asynchrony to differences in diagnoses and symptom profiles. Prior work studying spatio-temporal contrast sensitivity and backward masking suggests that temporal summation and visual persistence may be prolonged in some schizophrenia patients but only those with the negative or cognitive deficit syndrome (Slaghuis, 2004). An obvious prediction to be tested is that this will also apply to the asynchrony required for segregation. 

There is evidence that dysfunctions of visual perception, and in particular of the magnocellular system, may be endophenotypes (a heritable trait associated with the condition) for schizophrenia, in that siblings of schizophrenic patients show signs of such dysfunctions (Kéri, Kelemen, Benedek, & Janka, 2004). Studies of visual masking suggest a similar conclusion (Green, Nuechterlein, Breitmeyer, & Minz, 2006). These deficits in magnocellular pathway functions and perceptual organization are positively correlated with reports of anomalous visual experiences in schizophrenia, and are not due to pharmacotherapy as they occur in unmedicated patients (Kéri, Kiss, Kelemen, Benedek, & Janka, 2005). In our paradigm, a few of the student observers failed to reliably detect the figures, even with SOAs of 100 ms. This is a surprising result, because, for most people, asynchronies of 100 ms are highly salient. Four of the seven students with such low accuracy also had high scores on the schizotypy or autistic self-rating scales. A major goal for future studies is therefore to determine whether or not the asynchrony required for segregation is an endophenotype for psychotic disorganization. 

Many other questions also deserve future study. Evidence obtained using other paradigms clearly indicates that the impairments of perceptual organization seen in schizophrenia spectrum disorders are not due to either a general performance deficit or to medication (Uhlhaas & Silverstein, 2005). For example, the visual dysfunctions observed by Slaghuis (2004) occurred only in patients with the negative syndrome, although both they and those with the positive syndrome were medicated. Medication, psychoactive substance use, clinical signs and symptoms, chronicity, and sex now need to be related to segregation by asynchrony. To explore the possibility that this is an endophenotype for psychotic disorganization, it will be necessary to test unaffected relatives of patients. Furthermore, the effects of various neuropharmacological agents need to be studied to test the role of particular neurotransmitter and receptor systems. Finally, there is no evidence here that atypical sensitivity to asynchrony as a cue to segregation is specific to schizophrenia, and it may also be relevant to other disorders. For example, we now need to determine whether it also occurs in autistic spectrum disorders, which have already been shown to be associated with a bias to local rather than global visual processing (Dakin & Frith, 2005), and, if so, whether in those conditions sensitivity to small asynchronies is atypically high or atypically low. Although these are all questions for the future, the results reported above support the view that the paradigm that we have developed for studying segregation by asynchrony provides a simple and efficient means for addressing them. 

Acknowledgments

The concept of synchronized rate codes was developed through many discussions between Phillips and Wolf Singer in Frankfurt. Roger Watt also gave valuable advice at various stages of this work. Ernest Greene, Szabolcs Kéri, Edmund Rolls, David Simmons, Walter Slaghuis, Simon Thorpe, Peter Uhlhaas, Guy Wallis, and Jim Wright provided expert commentary on an earlier draft of the paper. Some of these results were presented by Peter Hancock at the European Conference on Visual Perception 2007; others appeared in an unpublished MSc thesis by Laura Walton. George Lovell wrote the code to generate the contours. 

Commercial relationships: none. 

Corresponding author: Peter Hancock. 

Email: [e-mail]. 

Address: Department of Psychology, University of Stirling, FK9 4LA, UK.