The time course of feature integration in plaid patterns revealed by meta- and paracontrast masking

Abstract

Abstract

Abstract:

Abstract  When a plaid object is presented, the visual system decomposes it into its constituting orientation primitives and integrates them at later processing stages. The present study reveals the time course of this process by applying meta- and paracontrast masking to both simple oriented and plaid gratings. With various stimulus onset asynchronies (SOA) between target gratings and surrounding mask annuli, subjects were asked to identify whether targets were simple gratings collinear to the masks, orthogonal to the masks, plaid, or whether no target was presented. The resulting time courses for each type of stimulus confusion showed that metacontrast peaked when orientation primitives had already begun to be integrated into one object, indicated by a dominance of “no target” responses given to plaid stimuli at SOAs around 70 ms. At SOAs around 10 to 30 ms masking also had a significant impact but acted on separable components, indicated by a dominance of “orthogonal” responses given plaid stimuli. Probability summation of “no target” responses given simple gratings revealed that only at shorter SOAs performance for plaid stimuli could be predicted assuming independent features but not at SOAs of at 50–70 ms. We discuss in how far these results could also be explained by the dynamics of cross-orientation suppression (COS) and how they might relate to the process of feature integration in plaids.

Introduction

The “when”-question in dynamic systems

The brain in general, and the visual system in particular, are certainly no serial input processors. At any time in awake humans, a myriad of processes occur in parallel. The current view is that these processes are at least in part dynamic, i.e., processes change other processes or themselves. It is therefore a great challenge to describe the temporal properties of these processes. Electrophysiological recordings may give precise information about when two different stimulations cause different signals, but in order to interpret the meaning of these differences for complex visual processes, for example perceptual grouping, object categorization, or feature integration, strong assumptions have to be made about the cortical sites generating these effects and their functional role in a complex dynamic system. Psychophysics, on the other hand, lacks this direct access to neural activity. As a consequence psychophysical data can never yield the absolute time stamp of a process or an effect on a process. However, as exemplified below, psychophysicists have repeatedly shown the ability to put these processes in temporal order—based on assumptions which processes are manipulated in an experiment—thus yielding the relative time stamp of these processes. 

Feature integration in plaid stimuli

Adelson and Movshon (1982) provided a well-known example for answering a “when” question in a relative way by superimposing two drifting gratings. If both gratings were of similar spatial frequency, the superimposed stimulus was perceived as drifting in a direction not corresponding to any of the two drifting directions of the single components. The authors concluded that during a first stage, the visual system extracted the motion of each component and at a second stage, the individual component motions were integrated to yield one coherent pattern motion. The result thus implied a temporal order of two processes: Process A extracted component motion information of the two stimulus components, and Process B established the perceived motion direction by integrating this information. In other words, feature integration (B) occurred after the calculation of component motion directions (A). 

In our study, we briefly presented stationary plaids, not drifting gratings as in Adelson and Movshon's study. Stationary plaids, although apparently being simple stimuli constructed by superimposing two oblique sine waves, are subject to complex processes. Current models on plaid processing agree that at least one stage involves oriented spatial filters signaling each constituting component of a plaid (Georgeson & Meese, 1997; Li, Thompson, Duong, Peterson, & Freeman, 2006; Meese, 2010; Priebe & Ferster, 2006). But on the phenomenological as well as the neural level, a plaid does not simply behave like the sum of its constituting sine waves. Obviously, more stages than oriented filtering and subsequent linear combination are needed. 

First of all, plaids typically do not look like superimposed sine waves; depending on the angle and luminance contrast between its components they can be perceived as blurred checkerboards, wavy gratings, or elongated squares (Georgeson & Meese, 1997). Second, on the neural level a plaid also differs from the sum of its components: Several studies on cross-orientation suppression (COS) showed that the responses of simple cells in primary visual cortex (V1) to a plaid are weaker than the responses to the sum of its isolated components. Correspondingly, the perceived contrast of a plaid is higher than the contrast of each single component, but lower than the sum of its components (Georgeson & Shackleton, 1994). 

Single cell recordings of cat V1 suggest that COS is a very early process, in part already accounted for by the behavior of cells in the lateral geniculate nucleus (LGN; Li et al., 2006; Priebe & Ferster, 2006). In addition to these subcortical processes, COS is mainly caused by intracortical mechanisms within V1 (Kimura & Ohzawa, 2009; Smith, Bair, & Movshon, 2006). In contrast to surround suppression, COS is assumed not to involve feedback signals from higher cortical areas (Smith et al., 2006). Thus, the simple cortical cells found in V1 can be viewed as the final destination of COS. At the same time, these simple cortical cells define the first stage in visual processing whose spatial filtering properties are sufficient for decomposing a plaid into two oriented components. Given this functional coincidence, it will be necessary to discuss how COS relates to the extraction and integration of orientation components in plaid. 

The perceived structure of a plaid on the other hand, was shown to be predictable from zero crossings in the pooled output of oriented filters. It is thus determined by processes subsequent to plaid decomposition (Georgeson & Meese, 1997; Georgeson & Shackleton, 1994). 

With the present study we seek to probe this processing sequence with metacontrast masking. Specifically, we ask how far the process of feature integration has evolved at the moment where metacontrast masking interferes with it. 

Visual masking: A noninvasive probe for the timing of visual processes

As noted above, electrophysiological recordings can reveal the absolute timing of neural events but are often limited when it comes to linking these events to phenomenological outcomes. Visual backward masking, especially metacontrast masking, is a psychophysical technique measuring phenomenological outcomes while precisely varying spatiotemporal stimulus relationships. In general, a target stimulus is presented briefly, followed, after various stimulus onset asynchronies (SOA) in the range of 0 to about 200 ms, by a mask stimulus that closely fits around the target. The phenomenological outcome measures are usually target detection, identification measures, or subjective visibility ratings. These paradigms are frequently employed to study the spatiotemporal properties of conscious and nonconscious visual processing (for recent reviews see e.g., Breitmeyer, Koç, Öğmen, & Ziegler, 2008; Breitmeyer & Öğmen, 2006; Enns & Di Lollo, 2000). 

What is masked in visual masking?

The typical result of a metacontrast experiment is a U-shaped relationship between the outcome measure (e.g., subjective visibility) and SOA, a so called type-B masking function: high target visibility with synchronous or near synchronous target-mask presentation, low visibility around 40–80 ms, and usually fully restored visibility around 100–150 ms. In most metacontrast studies only a single outcome measure is obtained, suggesting that masking affects this and only this measure. This assumption, however, has to be questioned for at least two reasons: First, it has been shown that the shape of the masking function is influenced by the subject's criterion content (Kahneman, 1968; Ventura, 1980), i.e., the specific target feature he or she is planning to respond to on the next trial. For example, with black disks on a bright background as targets the SOA of optimal masking (SOA_max) is lower when subjects are asked to report whether the target's contour contained a small deletion as opposed to being asked to rate the brightness of its surface (Breitmeyer et al., 2006). Individual differences have also been shown to have an effect on the shape of the masking function. With a square/diamond discrimination task, Albrecht, Klapötke, and Mattler (2010) identified one group producing typical type-B masking functions and another group producing type-A functions (i.e., a monotonous increase of performance with SOA) and linked this finding to differences in criterion content. Secondly, Kouider et al. (Kouider, de Gardelle, Sackur, & Dupoux, 2010; Kouider & Dehaene, 2007; Kouider & Dupoux, 2004) introduced the concept of partial awareness. Typically, a subject's response in a metacontrast experiment is assumed to reflect the content of his or her conscious perception. However, according to Kouider and colleagues, conscious access to a stimulus occurs in a hierarchical fashion, starting with low-level properties, such as stimulus energy, continuing with stimulus features of increasing complexity until an object can be categorized and its semantic information is available. Consequently, subjects may not be able to report the meaning of e.g., a masked word, but to report that something was presented, that this something contained oriented line elements, and maybe that these line elements form letters. 

In the present study, we asked whether metacontrast masking can be used to decompose this hierarchical process in plaid stimuli. We assume that during an initial stage a plaid object is decomposed into orientation primitives, which are recombined at a later stage. Thus, before feature integration, the visual system signals the presence of two independent components with orthogonal orientation. In addition to the plaid target, we present a mask that is collinear to one of those components. In a previous study we showed that a grating is masked stronger by a collinear mask compared to an orthogonal mask and that SOA_max is shorter in the former as compared to the latter case (Bruchmann, Breitmeyer, & Pantev, 2010). Consequently, we assume that if metacontrast suppression occurs before feature integration, subjects would report the presence of an orthogonal grating rather than a collinear one. If, however, metacontrast suppression occurred after feature integration, masking would determine the perceptual fate of the plaid pattern as a whole, causing subjects to report the absence of a target rather than the presence of a collinear or orthogonal target. 

Since feature integration and metacontrast masking are likely to engage processes that extend over a certain amount of time, it is also possible that these processes overlap. To gain insights into these temporal relationships we use a fine sampling of SOAs, using also negative SOAs, which refer to forward, or paracontrast masking, where the mask occurs before the target. 

Methods

Subjects

Ten subjects (five female) took part in the experiment. All had normal or corrected to normal vision and no history of neurological or psychiatric diseases. Age ranged from 23 to 30 years, mean = 26.50 years, SD = 2.88 years. The subjects gave their informed consent and were paid 9€ per hour. All procedures were carried out according to the declaration of Helsinki and were approved by the ethical committee of the medical faculty of the University of Münster. 

Apparatus and stimuli

The experiments were run using MATLAB and the Psychophysics toolbox (Version 3.0.8; Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). Stimuli were presented on a calibrated ViewSonic G90fB CRT monitor at 100 Hz and a resolution of 1024 × 768 pixels at a viewing distance of 80 cm. The mean brightness of the monitor was set to approximately 50 cd/m² (I_min = 0.434 cd/m², I_max = 100.002 cd/m²). Participants gave their responses by pressing one of four buttons on an external response box. 

The maximum Michelson contrast was (I_max – I_min) / (I_max + I_min) = 0.991, henceforth defined as 100% luminance contrast. As targets we used Gabor patches with a diameter of 2° of visual angle (degree; measured from −2.5 to 2.5 SD of the Gaussian envelope). As masks we used a grating annulus with a Gaussian profile envelope. The diameter of the annulus (at maximum height of the Gaussian) was 3°; the width of the Gaussian envelope was 2° (also measured from −2.5 to 2.5 SD of the Gaussian profile envelope). Targets and masks were randomly presented 6° to the left or right of a central fixation mark. 

The mask was always a simple grating with 4 c/° and 90 % luminance contrast. Its orientation was randomly chosen on each trial. Targets consisted of 4 c/° gratings that could either be collinear to the mask, orthogonal to the mask, or a superposition of a collinear and an orthogonal grating (referred to as plaid pattern). The target's luminance contrast was 50% per component, i.e., 50% for the single collinear and orthogonal gratings and 100% of the maximum contrast for the plaid pattern. 

Procedure

The participants were instructed to focus on a central fixation mark. After a randomized interval of between 400 and 600 ms a target-mask sequence appeared randomly to the left and right of the fixation mark. Target and mask duration was always 30 ms. The stimulus onset asynchrony (SOA) between target and mask was −200, −140, −100, −70, −50, −30, −10, 0, 10, 30, 50, 70, 100, 140, or 200 ms, negative values referring to paracontrast masking, i.e., with masks preceding targets, and positive values referring to metacontrast masking, i.e., with masks following targets. With equal probabilities a trial contained a collinear, an orthogonal, a plaid, or no target. 

The subjects' task was to identify the target with four keys, each assigned to one of the four target conditions (including no target). The assignment of conditions to keys was randomly chosen for each subject. After each trial, subjects were reminded of the assignment by the presentation of four (always vertical) masks with the four target types (exemplified in Figure 1a). The four patterns remained on screen until the subject identified the target by a key press. After the identification the subjects were asked to rate their confidence in their identification by using one of four keys using their left hand. An illustration on the screen reminded the subjects after each trial that the keys from right to left were defined as “guessing,” “maybe,” “quite certain,” and “very certain.” After each trial subjects received tone feedback about the correctness of their identification irrespective of the confidence rating. 

Figure 1

 

(a) Stimulus examples representing the four stimulus types used in the experiment. The overall orientation of the stimuli was randomized on each trial. (b) Schematic stimulus sequence. The fixation period had a randomized duration between 400 and 600 ms. Targets and masks had a duration of 30 ms each and were presented with a stimulus onset asynchrony (SOA) between −200 (i.e., mask first) and 200 ms (i.e., target first). After the stimulus sequence the participant was presented with a reminder for the stimulus-response-key assignment. In the actual experiment, the four reminder stimuli were horizontally arranged. The square arrangement in (b) was chosen only to enlarge the stimuli for illustrative purpose. The reminder stayed on the screen until the participant made the identification response. After identification, the subject was instructed to rate his/her confidence in the identification decision by pressing one of four keys.

In total there were 60 experimental conditions (4 Target Types × 15 SOAs). Each condition was repeated 60 times. The resulting 3,600 trials were distributed over two sessions of 1.5 hours. Before starting each session the participants completed 50 practice trials to get familiar with the task. Subjects were told to rest as often as needed by withholding responses. Additionally, every 100 trials subjects were forced to take a pause of 30 seconds, in which zero to five golden stars were presented, each representing 20% correct responses in the past 100 trials to increase motivation. 

Results

Two types of analysis for two different hypotheses

To analyze the responses we took two different approaches: The first was based on signal detection theory (SDT) and yielded a bias free and interval scaled measure of the strength of masking. By taking the confidence ratings on each trial into account we estimated receiver operant characteristic (ROC) curves and calculated the area A_Z under the ROC as described below. This approach allowed for statistical comparisons of the masking effect for different target types. 

One advantage of analyzing ROCs is that the resulting sensitivity is not affected by the individual response criterion of a given participant. The disadvantage, however, is that a single value is obtained for each combination of two stimuli. For example, a low A_Z score for the pair orthogonal/plaid does not tell whether plaids have been mistaken for orthogonal targets or vice versa. 

Contrary, our second approach, the analysis of relative response frequencies, has the advantage that we obtain one value per response for each stimulus. The disadvantage is that higher values of e.g., an erroneous “no target” response to collinear targets, do not tell whether they stem from decreased sensitivity or from a more conservative criterion (i.e., the general tendency to respond with “no target”). In the present case we will analyze averaged response frequencies with the restriction that we do not focus on the absolute value of the relative frequencies but instead how they vary with SOA. This rationale rests on the assumption that the participants' response criteria on which the decision to choose one of the four responses is based do not vary systematically with SOA. Our assumption then simply states that a participant will not change any of his or her criteria just because of the time passing between the onsets of target and mask. Under this assumption we obtain valid information about when (i.e., at which SOA) a specific response to a given stimulus is maximal. The values do not tell whether e.g., 30% “no target” responses to a collinear stimulus imply poorer performance than 20% “no target” responses to a plaid stimulus. 

ROC fitting

We defined the strength of masking as the discrimination performance between each possible pair of the four stimuli. Since the order of stimuli in a pair is unimportant in this case, there are six unique stimulus pairs (no target/collinear, no target/orthogonal, no target/plaid, collinear/plaid, collinear/orthogonal, and orthogonal/plaid). For each pair we calculated response frequencies as illustrated in Table 1. We then fitted a receiver operation characteristic (ROC) curve to the cumulative probabilities (i.e., h_ij|k/h_k) using the algorithm described by Dorfman and Berbaum (1986). For the ROCs we assumed a normal distribution for noise (i.e., internal activation in trials without a target) with μ₀ = 0 and σ₀ = 1 and a normal distribution for signal + noise (i.e., internal activation in trial with targets) with μ₁ and σ₁ as free parameters. The analysis is based on the measure A_Z, i.e., the area under the ROC curve (see e.g., Wickens, 2001) which ranges from A_Z = 0.5 for performance at chance level to A_Z = 1 for perfect detection/discrimination. 

Table 1

 

Response categorization for a given stimulus pair (e.g., Stimulus A = orthogonal, Stimulus B = collinear). ROCs were fitted to the relative response frequencies h, where h_Ri|S refers to the relative frequency of choosing response category R with the ith certainty level given stimulus S.

Identification	Response: Stimulus A				Response: Stimulus B
Confidence:	4	3	2	1	1	2	3	4
Stimulus
A	h A4|A	h A3|A	h A2|A	h A1|A	h B1|A	h B2|A	h B3|A	h B4|A
B	h A4|B	h A3|B	h A2|B	h A1|B	h B1|B	h B2|B	h B3|B	h B4|B

The task can be viewed as a simultaneous detection and identification paradigm (Macmillan & Creelman, 2005; p. 255 ff), i.e., if one stimulus category in a pair (e.g., Stimulus B) is no target, the resulting performance measure will be interpreted as a detection measure for Stimulus A. Otherwise, it is an identification measure of Stimulus A vs. Stimulus B. 

Figure 2 shows the averaged detection and identification performance measured as A_Z. Error bars represent 95% confidence intervals of the within-subject effect SOA × Stimulus Type (Loftus & Masson, 1994). Figure 2a shows that the detectability of the three stimulus types exhibits the typical U-shaped masking function for metacontrast. No obvious paracontrast effects appear to be present. Overall, detectability is best for plaid patterns, followed by orthogonal patterns, followed by collinear patterns. The data also indicate that the SOA of maximum masking (SOA_max; henceforth defined as the SOA where the strongest masking effect, i.e., least amount of hits or largest amount of errors, is observed) occurs later for orthogonal targets (70 ms) than for collinear targets (50 ms), replicating previous results (Bruchmann et al., 2010; Ishikawa, Shimegi, & Sato, 2006). 

Figure 2

 

Results from the four AFC identification experiment. The data were analyzed separately for each possible stimulus pair. The small stimulus pictograms symbolize the stimulus types collinear (green, square symbols), orthogonal (red, triangle symbols), and plaid (blue, star symbols). The discrimination performance for each pair is plotted as the area under the ROC (A_Z). Figure 2a shows the three discrimination pairs that included the no target stimulus. Confusions between Stimulus X and no target are referred to as detection performance for Stimulus X. Figure 2b contains the results for each possible discrimination among collinear, orthogonal, and plaid stimuli. The error bars represent 95% confidence interval of the within-subject effect SOA × Stimulus Type (Loftus & Masson, 1994).

The identification performance for the pairs collinear/plaid, orthogonal/plaid, and collinear/orthogonal is depicted in Figure 2b. While the first two also yield typical U-shaped functions, discriminating between collinear and orthogonal targets appeared on average to be more difficult than between the other two pairs and also showed a different relation to SOA: The farther target and mask were temporally separated (in positive as well as in negative direction) the lower the observers' ability to discriminate between collinear and orthogonal targets. 

Average response frequencies

The averaged response frequencies are discussed separately for each stimulus type. Figure 3a shows the relative frequency of each response category to trials without a target. The data clearly show the most frequent answer was the correct rejection (M = 69.20%, SD = 9.96%), followed by the response “collinear” (M = 20.61%, SD = 9.54%), followed by “orthogonal” (M = 7.62%, SD = 2.66%), followed by “plaid” (M = 2.57%, SD = 2.44%). 

Figure 3

 

Results from the four AFC identification experiment. Each plot contains the relative response frequencies of no target, collinear, orthogonal, and plaid responses (in yellow, green, red, and blue, respectively) for each of the four stimuli. Since there is no SOA in trials without targets (Figure 2a) only Figures b–d show the response frequencies as a function of SOA, where negative values refer to forward and positive to backward masking. For illustrative purposes, the response frequencies for absent targets were subjected to a one-way ANOVA with repeated measurements, and in case of collinear, orthogonal, and plaid targets to three 4 Response Category × 15 SOA ANOVAs with repeated measurements. Based on these ANOVAs, Figure (a) contains error bars reflecting the 95% confidence interval of the main effect of response category, (Loftus & Masson, 1994). Figures b–d show confidence intervals of the interaction effect Response Category × SOA. Figure 4d also contains the prediction for giving a “no target” response to a plaid stimulus (purple line), based on the probabilities of giving “no target” responses to collinear and orthogonal stimuli. Error bars for the probability summation estimates again reflect 95% confidence intervals of the main effect response category, this time based on a 2 Response Category × 15 SOA ANOVAs with repeated measurements where only the two relevant categories no target and probability summation were used (see Supplementary Figure for individual results of the 10 observers).

Figure 4

 

Averaged temporal positions of SOA_max.for each response category given plaid targets. Error bars indicate the 95% confidence interval of the mean.

For trials with collinear target-mask-sequences (Figure 3b), we observe that the correct responses and the “no target” responses show the typical type-B masking pattern with a peak of metacontrast masking at SOA = 50 ms, as it is also evident from the A_Z-analysis. The response category “orthogonal” exhibits the same temporal characteristics as the “collinear”-response, indicating a base rate error for confusing collinear and orthogonal targets. The “plaid” response occurred least often for all SOAs. 

Orthogonal target-mask-sequences (Figure 3c) show that for metacontrast, most errors were made at SOA = 70 ms, which again replicated our earlier findings (Bruchmann et al., 2010). For paracontrast we observed an unexpected pattern in the form of a systematic misperception of orthogonal stimuli as plaid stimuli, peaking at SOA = −70 ms. 

The results for plaid stimuli (Figure 3d) reveal that “no target” responses peaked at SOA = 50 ms, whereas “orthogonal” responses peaked around 10–30 ms. This indicates that at short SOAs the mask interrupted target processing at a stage where the two stimulus components existed as isolated features. 

Probability summation as an index for feature independence

At SOAs 50–70 ms the mask suppresses the visibility of the plaid pattern as a whole. The question arises whether the “no target” responses to plaid patterns reflect the consecutive masking of isolated collinear and orthogonal components or whether these responses reflect a masking effect that acts on the plaid stimulus after its components have been integrated into one object. To answer this question we calculated the predicted probability of a “no target” response to a plaid target, based on the relative frequencies of a “no target” response to a collinear and an orthogonal target, respectively. Probability summation is a standard model for detection, assuming that subjects detect a target if any one of its features is detected and that those features are detected independently (Brindley, 1954; Graham, 1977; Pelli, Burns, Farell, & Moore-Page, 2006; Quick, 1974; Robson & Graham, 1981; Watson & Ahumada, 2005). Thus, under the assumption of independent features, the predicted probability of giving a “no target” response to a plaid stimulus is defined as: 

 

The purple line in Figure 3d shows that the “no target” responses to a plaid target were perfectly predictable from the “no target” responses to collinear and orthogonal stimuli except at SOAs = 50–100 ms. Two-sided Bonferroni-corrected binomial tests revealed significant differences between predicted and observed frequencies at SOA = −200 ms (p = 0.034), SOA = 50 ms (p = 0.002), SOA = 70 ms (p = 0.002), and SOA = 100 ms (p = 0.038). We conclude that the probability to miss the plaid pattern was higher than expected under the assumption of independent components, whereas at SOAs = 10–30 ms, the predicted probability exactly matched the observed performance. 

Temporal comparison of SOA_max of erroneous responses

We then evaluated the temporal position of the minima of the measured metacontrast functions for “no target,” “orthogonal,” and “plaid” responses per participant with a one-way ANOVA for repeated measures. The averaged temporal positions of the masking functions' extrema are depicted in Figure 4b. The ANOVA revealed a highly significant difference in SOA_max depending on the responses, F(2, 18) = 6.536, p < 0.01. Planned comparisons revealed that the peak of “orthogonal” responses (SOA_max = 33 ms, SD = 31.29 ms) preceded the peak of “no target” responses (SOA_max = 61 ms, SD = 16.63 ms), t(9) = −3.231, p < 0.01, and that “orthogonal” responses preceded “plaid” responses with marginal significance (SOA_max = 48 ms, SD = 11.35 ms), t(9) = −1.8, p = 0.0525. 

Discussion

Metacontrast is orientation sensitive

The analysis of detection performance (A_Z) confirmed our previous finding that collinear gratings are masked stronger and at a shorter SOA than orthogonal gratings (Bruchmann et al., 2010). Plaid patterns in general were easier to detect than single gratings, as expected from the fact that their contrast was twice as high. 

Metacontrast acts on independent target features

Two observations support the interpretation that metacontrast masking interferes with target processing at a stage where the visual system encodes the target's features in independent modules: First, the most frequent erroneous perception of plaid stimuli as orthogonal stimuli at SOAs between −20 ms and 50 ms shows that the mask selectively removed the collinear target component, leaving the orthogonal component almost unaffected. Second, in this range of SOAs the probability of completely missing a plaid target (i.e., giving a “no target” response) can be precisely predicted from the probabilities of missing the collinear and orthogonal components under the assumption of independent features. This indicates that any effect of masking on the collinear component did not alter its effect on the orthogonal component and vice versa. 

The question may arise why masking of collinear targets peaked at SOA = 50 ms, not at SOAs = 10–30 ms, where the collinear component of the target was most effectively reduced. As we recently showed, metacontrast masking functions can be understood as the result of suppressive effects within and between spatial frequency-selective channels (Bruchmann et al., 2010). Within-channel masking peaks at short SOAs and is orientation selective, whereas between-channel masking peaks at longer SOAs and is not orientation selective. The masking effect of collinear gratings is comprised of both of these suppressive effects. In other words, the suppressive effects we observed for orthogonal gratings, peaking at SOA = 60 ms, also acted on collinear gratings, additionally to the early peaking orientation selective effect. As a result, the sum of these suppressive effects peaked at SOA = 50 ms. 

Metacontrast peaks after the start feature integration

The frequency of “no target” responses peaked at an SOA of about 60 ms. At this SOA the frequency of “orthogonal” responses was about as high. However, the peak of “orthogonal” responses occurred around 10–30 ms. Since the frequency of “orthogonal” responses from thereon declines continuously, we take this as a first evidence that the two orientation components were no longer independent at SOA = 60 ms. The second evidence stems once more from the comparison of observed responses to predicted responses, assuming independent features: The observed probability of completely missing the plaid target was higher than the probability of missing its collinear and its orthogonal component. Thus, some effects of masking on the collinear component did alter their effect on the orthogonal component and vice versa. We conclude that at the peak of metacontrast masking, orientation primitives are no longer fully separable. We cannot conclude whether—at this SOA—the integration of orientation primitives is completed, nor can we say whether other target features such as color, shape, or motion are independent or not, but we can conclude that the observers' responses reflect more the properties of a coherent plaid object rather than the combined sum of properties of its two constituting orientation primitives. 

Feature integration versus cross-orientation suppression

It has to be noted that a deviation of observed plaid detection from a prediction based on probability summation can be explained without feature integration. There is ample evidence from mostly electrophysiological (Allison, Smith, & Bonds, 2001; Brouwer & Heeger, 2011; DeAngelis, Robson, Ohzawa, & Freeman, 1992; Geisler & Albrecht, 1992; Heeger, 1992; Kimura & Ohzawa, 2009; Li et al., 2006; Smith et al., 2006) but also psychophysical (Georgeson & Meese, 1997; Georgeson & Shackleton, 1994; Meese & Holmes, 2007), and neuroimaging (McDonald, Mannion, & Clifford, 2012) studies that the response to a plaid is weaker than the response to the sum of its isolated components. This finding is known as cross-orientation suppression (COS). 

Our interpretation regarding how metacontrast changed the percept of the plaid target if it occurred before or after feature integration was again as follows: Metacontrast before feature integration removed mostly the collinear component, causing the plaid to be perceived as orthogonal. Metacontrast after feature integration acted on the plaid as a whole, thus the “no target” errors got more frequent with increasing SOA. In fact, if we assume that it was not feature integration but COS which evolved with SOA, we would predict the same result: At short SOAs the mask would have suppressed two isolated and strong signals, corresponding to the collinear and the orthogonal components. Since orthogonal suppression is weaker (see Figure 2a), the orthogonal component would have survived masking. Assuming that COS would have weakened both signals at SOAs 50–60 ms, “no target” errors would arise because metacontrast was strong enough to remove both the collinear and the orthogonal signals completely. Thus, it appears that both interpretations—feature integration and COS—are supported by the present data. 

To clarify whether COS may be an additional or even alternative explanation for the presented data we have to look at what is known about the time course of COS, feature integration, and metacontrast, but also surround suppression, both on a relative and an absolute time scale. As noted in the Introduction, COS occurs very early in the visual processing hierarchy and orientation selective V1 cells can be viewed as the final destination of COS. At the same time, these cells' orientation sensitivity makes them the first candidate for the extraction of orientation components from a plaid. This already indicates that the time course of feature extraction and integration, as proposed to be revealed by our study, may be interwoven with COS, could occur in parallel or even be in part identical to COS. 

How does the timing of metacontrast relate to the COS? COS does not appear to involve the contribution of feedback from higher areas (Smith et al., 2006). Contrarily, the inhibitory effects of metacontrast masking are assumed to be the result of disrupted feedback loops from higher areas to low level visual areas (Fahrenfort, Scholte, & Lamme, 2007; Ro, Breitmeyer, Burton, Singhal, & Lane, 2003). A COS interpretation would then imply that the feedback-mediated target inhibition of metacontrast increases from SOA = 0 to 60 ms in parallel to the unfolding of COS within V1. Electro- (EEG; Fahrenfort et al., 2007) and magneto-encephalographic (MEG; Haynes, Roth, Stadler, & Heinze, 2003) measures of visual masking with oriented stimuli showed that early visual areas show effects around 120 ms after target onset. However, single cell recordings in monkey V1 reported COS to be peaking around 40 to 60 ms (Kimura & Ohzawa, 2009) after the onset of the plaid. Given the differences in stimulus setup and recording method, one may argue that these results are difficult to compare, especially because single cell recordings may not capture all neural mechanisms causing COS and thus might not reflect its time course completely and EEG/MEG measures might simply miss possible earlier correlates of masking. In our view, this is nevertheless one argument against a COS interpretation. On the other hand, if COS was finished before any target-mask-interaction occurred, the orientation components could hardly be called independent and it would be difficult to explain why probability summation exactly predicted plaid masking. It appears that a discussion of the present behavioral results on the grounds of electrophysiological findings does not lead to unequivocal conclusions. 

On a behavioral level Petrov, Carandini, and McKee (2005) provided clear evidence that COS precedes surround suppression, which is also supported by single cell recordings in monkey V1 (Smith et al., 2006) and more recent behavioral studies (Meese, Challinor, Summers, & Baker, 2009; Saarela & Herzog, 2008, 2009). As discussed below in more detail, surround suppression experiments are in many ways comparable to metacontrast with SOA = 0 ms. Thus, if COS precedes the suppressive effects of the surround already at SOA = 0 ms this can be viewed as one further argument against the interpretation that COS unfolds with SOAs between 0 and 60 ms. 

At present, we conclude that the time courses of COS, metacontrast, and surround suppression imply that COS is not necessarily completed before the mask interacts with the target but that COS is unlikely to increase and peak in parallel to metacontrast. However, claiming that COS is simply a process that (partly or completely) precedes all observed effects contradicts our interpretation that metacontrast with SOAs around 30 ms acts on independent features because COS is essentially a process that creates (or is created by) the dependence of orientation components in a plaid. Based on the present data, we can therefore not explain why probability summation exactly predicts plaid masking at all SOAs except at 50 and 60 ms. The exact relationship of COS to feature integration in plaids has to be addressed in future research. 

Implications for surround suppression studies

Surround suppression and metacontrast masking paradigms are highly comparable in spatial setup, especially when the latter involves grating stimuli (Bruchmann et al., 2010; Bruchmann, Hintze, & Mota, 2011; Ishikawa et al., 2006; Rogowitz, 1983): Both typically use a peripherally presented center and surround arrangement for targets and masks. The temporal setups and stimulus characteristics differ but remain comparable. In metacontrast, spatially abutting stimuli are presented briefly (ca. 10–40 ms; for reviews see Breitmeyer, 1984; Breitmeyer & Öğmen, 2000, 2006; Enns & Di Lollo, 2000), with variable SOAs between ±300 ms. The stimulus contrast of target and mask is typically matched. In surround suppression experiments, stimuli are usually presented longer (150–2000 ms; Cannon & Fullenkamp, 1991; Nurminen, Peromaa, & Laurinen, 2010; Petrov et al., 2005; Petrov & McKee, 2006, 2009) and simultaneously (and thereby comparable to SOA = 0 in metacontrast masking; for an exception see Petrov & McKee, 2009) but using low contrast targets in combination with higher contrast masks and not necessarily abutting. 

Given the similarity of the two paradigms, our results provide important implications for the investigation of surround suppression. As shown in Figure 4a, suppression of the collinear component (indexed by the frequency of “orthogonal” responses; red line) undergoes dramatic changes when the SOA is changed only slightly from SOA = −20 ms to +20 ms. Our results suggest the strongest suppressive effects of the mask on the collinear target component to be found around 25 ms. However, this SOA is most probably not representative for all stimulus setups. For example, in a previous study we showed that at SOA = 0 ms the suppressive effect on collinear and orthogonal targets is about equally weak (Bruchmann et al., 2010). On the other hand, Ishikawa et al. (2006) observed large differences between orthogonal and collinear targets at SOA = 0 ms. Petrov and McKee (2009), who varied the SOA in a standard surround suppression setup, found maximal suppression between −1 and 5 ms. These comparisons indicate that the power of surround suppression varies considerably with small changes in SOA. We assume that factors such as stimulus contrast, contrast ratio, size, and eccentricity determine at which SOA surround suppression is maximal. In this case, the limitation to simultaneous presentation of target and mask in surround suppression paradigms appears to be an unwarranted narrowing of experimental focus. A variation of SOA allows access to the temporal dynamics of surround suppression and helps reconcile the findings in both paradigms. 

Filling-in or feature attribution in paracontrast

An unexpected finding was that orthogonal targets are systematically perceived as a plaid target around SOA = −70 ms (see Figure 3c). This result may be explained by a filling-in process from the outer border of the mask into its center. We speculate that the filled-in orientation may have been added onto the orthogonal target, causing it to appear plaid. In line with this interpretation is the observation that the most common error in trials without targets was the “collinear” response. This error was about twice as likely as giving a “plaid” or “orthogonal” response. 

However, filling-in is assumed to be initiated at the edges of a stimulus and to proceed inward from there (Arrington, 1994; Grossberg, 1997; Grossberg & Hong, 2006; Grossberg & Kelly, 1999; Grossberg, Kuhlmann, & Mingolla, 2007; Grossberg, Mingolla, & Todorović, 1989; Paradiso & Nakayama, 1991). For the annulus mask this would imply that actually two filling-in processes should be initiated; one at the outer stimulus border, filling-in the grating texture, and one at the inner border, filling-in the uniform gray center. The filling-in interpretation implies (a) that the texture flow can override the uniform gray flow in order to fill-in the empty center of the mask and (b) that this filled-in texture is added onto the orthogonal orientation of the following target. 

One reason why the centripetal flow did not stop at the inner annulus border could be that this border was itself masked by the target. This argument is supported by the observation that the “plaid” responses to the orthogonal target yield a typical U-shaped metacontrast function if we let targets and mask exchange their roles, i.e., if we treat the disk shaped stimulus as a mask for the annulus. 

A different explanation for the phenomenon is feature attribution, mediated by apparent motion: Breitmeyer, Herzog, and Öğmen (2008) showed that a later stimulus gained an attribute (in this case Vernier offset) from an earlier stimulus if the stimuli produced apparent motion, irrespective of whether the later stimulus masked the earlier one. Interestingly, feature attribution and apparent motion were strongest when the later stimulus was presented 80 ms after the earlier one (see Experiment 1; no SOAs closer to 70 ms were used, but 0, 40, 80, 120, 160, and 320 ms). A difficult question is how well target and mask have to match in order to produce apparent motion. Breitmeyer et al. (2008) showed that the first and second stimulus can differ in Vernier offset but still produce apparent motion. In fact, a diverging feature was necessary to show that actually this feature was inherited by the second stimulus. In our case, the target and the mask differed in shape (disk vs. annulus) and in orientation. It is known from metacontrast paradigms that a disk followed by an annulus can produce the perception of a radial dispersion (explosion) or a rapid visual expansion (looming; Breitmeyer & Öğmen, 2006 p. 102). We assume that in paracontrast the opposite can occur, i.e., that a mask followed by a target can produce the percept of an imploding or receding stimulus. The question is how the visual system will integrate the diverging orientation information. In our view, this could be achieved by rotating the target by 90° or by inheriting the mask orientation, comparable to the Vernier offset in Breitmeyer et al.'s (2008) study. We expect that in the first case, the percept should include a receding, rotating single grating. The apparent rotation would then be indicative that the target was orthogonal, not collinear. In the second case, observers should perceive a receding but not rotating grating that inherits the mask orientation. 

In the light of the present findings these interpretations have to remain speculative since neither did we challenge a filling-in model (e.g., by manipulating the mask's radii) nor ask our participants to indicate perceived motion or rotation. Since these questions aim beyond the scope of the present article, we subject them to future research. 

Conclusion

The reported study is further evidence for the usefulness of metacontrast masking as a paradigm to uncover the time course of visual processing in an indirect and noninvasive yet temporally precise fashion. We showed that asking subjects to map their percept on more than a simple target absent versus present dimension or a holistic subjective visibility dimension reveals multiple, distinct consequences of metacontrast masking, each having its own temporal characteristic. 

Acknowledgments

This work was funded by the German Research Foundation (DFG) BR3832/1-2. We thank Dirk Vorberg for valuable discussions and Kathrin Thaler and Simon Mota for their assistance during data collection and manuscript preparation. 

Commercial relationships: none. 

Corresponding author: Maximilian Bruchmann. 

Email: [e-mail]. 

Address: Institute for Biomagnetism and Biosignalanalysis, University of Muenster, Muenster, Germany.