The aftereffect of a spatial offset between Gabor patches depends on carrier orientations

Abstract

This study explored the orientation connectivity in a contrast modulation processing mechanism, modeled as two filtering stages with nonlinear processing in between, by investigating how a negative aftereffect of a contrast-defined spatial offset is influenced by carrier orientations in the adapting stimulus. After adaptation to multiple, globally presented pairs of Gabor patches with a specific horizontal offset, subjects perceived a vertically aligned test pair of patches as offset in the orientation opposite to that of the adaptor. Although the orientations of the carrier gratings in the adaptor pairs were irrelevant to the task, the aftereffect magnitude depended on them. A large aftereffect was observed when the carrier orientations were parallel and/or perpendicular to the contrast-defined orientation, supporting the notion that second-stage filters receive strong inputs from first-stage filters with parallel and perpendicular orientation preferences. Furthermore, the aftereffect was also large when the carrier for only one patch was parallel or perpendicular, and no significant difference in the aftereffect magnitude was observed whether the adaptor pair contained one or two such patches. These results suggest that connectivity is not strictly selective to parallel and perpendicular relationships. Spatially heterogeneous connectivity might explain the observed effect.

Introduction

Typical visual stimuli used in psychophysical experiments, such as bars and edges, are characterized by spatiotemporal variations in luminance and color that are thought to be detectable by simple linear filters (Blakemore & Campbell, 1969; Campbell & Robson, 1968). However, some visual attributes, such as spatial modulations of contrast, spatial frequency, and local orientation, are not defined solely by variations in luminance. Such information cannot be detected by linear filters but is nonetheless important in processing the visual stimuli contained in natural scenes (Johnson & Baker, 2004; Schofield, 2000). 

Previous studies have reported psychophysical evidence supporting the efficacy of a family of models referred to as filter–rectify–filter (FRF) models (e.g., Graham, Beck, & Sutter, 1992; Landy & Bergen, 1991; Malik & Perona, 1990; Wilson, Ferrera, & Yo, 1992) that identify three steps in the perceptual process. The first step involves a bank of linear filters that extract luminance variations, often called carriers. In the second step, the outputs of the first-stage filters undergo nonlinear processing (e.g., full-wave rectification). In the last step, a second-stage filter detects contrast modulations, often called envelopes. 

One of the important questions about FRF mechanisms is how first-stage filters are connected to second-stage filters in terms of their orientation preferences. It has been widely accepted that first-stage filters are selective for orientation and spatial frequency of luminance modulation; the detection threshold for contrast modulation on a sinusoidal grating is elevated by either adaptation or masking using a luminance grating with the same frequency and orientation as the carrier of the tested grating (Dakin & Mareschal, 2000; Langley, Fleet, & Hibbard, 1996). Most studies assume that second-stage filters are also selectively oriented, enabling them to contribute to the processing of contrast-defined orientations. However, several different lines of evidence point to different orientation connectivity between first-stage and second-stage filters (Figure 1). 

Figure 1

 

Models of orientation connectivity between the first-stage and second-stage filters in the FRF mechanism. Bold arrows represent strong inputs from first-stage filters to a second-stage filter, and dotted arrows represent weak inputs. (a) Mechanism with parallel connectivity. The second-stage filter receives strong inputs from the first-stage filter with a parallel orientation preference. (b) Mechanism with perpendicular connectivity. (c) Isotropic mechanism. The second-stage filter receives inputs from all first-stage filters irrespective of their orientation preference. (d) Mechanism with both parallel and perpendicular connectivities.

The perceived orientation of a region defined by contrast modulation is affected by carrier orientation (Fraser illusion and Zöllner illusion; Dakin, William, & Hess, 1999; Morgan & Baldassi, 1997; Morgan, Mason, & Baldassi, 2000). This clearly shows that connectivity between first-stage and second-stage filters is dependent on their orientation preferences. More specifically, Morgan and Baldassi (1997) and Morgan et al. (2000) have argued that this phenomenon can be explained by postulating that each second-stage filter is connected only to first-stage filters with the identical or “parallel” orientation preference (Figure 1a), whereas Dakin and Mareschal (2000) and Dakin et al. (1999) proposed the existence of second-stage filters that are connected to first-stage filters with the orthogonal or “perpendicular” orientation preference (Figure 1b). Thus, these studies favored orientation-specific connectivity. 

However, more recent studies have argued that the outputs of first-stage filters with different orientation preferences are integrated at the level of second-stage filters. For example, Mussap (2001) reported that the detection and discrimination thresholds for contrast-modulated patterns are independent of carrier orientations and bandwidth. Given that first-stage filters are oriented (Dakin & Mareschal, 2000; Langley et al., 1996), the simplest interpretation of Mussap's results is that connectivity between first-stage and second-stage filters is not orientation-selective but “isotropic” (Figure 1c). Further support for this integration model has come from the texture segregation studies conducted by Motoyoshi and Nishida (2004) and Prins (2008). However, the limited orientation conditions tested in these studies allow for the alternative interpretation that each second-stage filter integrates the outputs of first-stage filters with multiple orientation preferences (as in the “isotropic” mechanism) but only when they are in a “parallel” or “perpendicular” relationship (Figure 1d). 

Positive support for this type of connectivity with anisotropic integration across carrier orientations has come from studies using multiple spatially distinct Gabor patches. In an experiment conducted by Keeble and Hess (1998), subjects reported whether a central Gabor patch was located to the right or left of upper and lower Gabor patches aligned vertically to each other. This three-Gabor alignment task was easier when all carriers were vertical (i.e., parallel), when all were horizontal (i.e., perpendicular), and when one was vertical and the others were horizontal but more difficult when all were tilted at 45° and when they were randomly oriented (see also Keeble & Hess, 2002; Popple, Polat, & Bonneh, 2001; Popple & Levi, 2002). Keeble and Nishida (2001) observed the same performance pattern in a two-Gabor alignment task. Although these studies did not specify the exact orientation connectivity, their results can be explained by the presence of a mechanism in which each second-stage filter is strongly connected with first-stage filters characterized by both parallel and perpendicular preferences (Figure 1d; also see schematic Figure 7 in Keeble & Nishida, 2001). 

Detection of a contour formed by multiple Gabor patch elements in an array of randomly oriented distractor patches is a closely related task (Field, Hess, & Hayes, 1993; Hess, Hayes, & Field, 2003). This contour detection is thought to be mediated by a kind of FRF mechanism (Dakin et al., 1999; May & Hess, 2008). Contour detection is least difficult when the elements are parallel to the contour, most difficult when elements are oriented at 45° (Ledgeway, Hess, & Geisler, 2005), and moderately difficult when the elements are oriented orthogonally (Bex, Simmers, & Dakin, 2001; Ledgeway et al., 2005). May and Hess (2008) suggested that two concurrent FRF mechanisms could explain the effect of carrier orientations, one with parallel connectivity and a stronger contribution (Figure 1a) and another with perpendicular connectivity and a weaker contribution (Figure 1b). 

To summarize, there are four primary candidates for connectivity between first-stage and second-stage filters: “parallel,” “perpendicular,” “isotropic,” and “parallel and perpendicular.” No sound consensus has yet been reached as to which best fits the accumulated evidence. These apparently conflicting perspectives might be explained by different natures of various tasks that require different roles for carrier orientations. If the task is to detect contrast modulation, carrier orientation is an irrelevant feature, and thus, it would be advantageous for the observer to use mechanisms in which second-stage filters integrate the outputs of first-stage filters with any orientation preference, particularly given the existence of intrinsic noise in early detectors. In contrast, carrier orientation might play a role when the task is to report the orientation of contrast modulation, given that luminance-defined and contrast-defined orientations correlate in natural scenes (Johnson & Baker, 2004). In this case, it might be reasonable to have orientation-specific connectivity between first-stage and second-stage filters. In an orientation judgment task using a pair or triplet of Gabor patches, both parallel and perpendicular carriers might serve as a frame of reference and only the outputs of first-stage filters with parallel and perpendicular preferences might be fed into second-stage filters. When the task is to detect a contour formed by multiple patches, the adaptive strategy might be to use an orientation-specific mechanism because observers have to reject false bindings with randomly oriented distractor patches. Because different types of mechanisms might be recruited to solve different tasks, the relationship between tasks or stimuli and the roles of carrier orientations should be carefully examined to reveal the principle behind contrast modulation processing. 

The present study aims to reveal the orientation preference of contrast modulation processing, and thus, the orientation connectivity between first-stage and second-stage filters in a situation wherein carrier orientation is irrelevant to the task, which was to judge the orientation of a spatial offset between two contrast-modulated patches. This goal was accomplished by investigating the aftereffect of spatial offset between two Gabor patches (Kobayashi & Murakami, 2009, 2011) and by manipulating the carrier orientations of the adapting stimulus. After prolonged viewing of pairs of Gabor patches that are spatially offset, a new pair of physically aligned Gabor patches appears to be offset in the opposite direction. This paradigm has three advantages. First, a pair of spatially offset positions is the minimal condition for defining orientation; thus, it is an ideal tool to investigate the effect of carrier orientation on contrast modulation processing. Previous investigations of carrier orientation influence on the perceived orientation of contrast modulation have used stimuli with clear contrast-defined edges (Dakin et al., 1999; Morgan & Baldassi, 1997; Morgan et al., 2000) that make it difficult to judge whether contrast-defined orientation per se or the existence of edges is the critical factor. Second, we can independently manipulate the carrier orientations of the upper and lower patches to investigate orientation connectivity between first-stage and second-stage filters and, in particular, the integration of second-stage filter processing. Third, and most importantly, we can manipulate the carrier orientations of the adapting stimulus while keeping those of the test stimulus constant. 

Experiment 1: Effects of carrier orientation

In Experiment 1, the effect of the carrier orientations on the magnitude of the aftereffect was examined. To this end, we manipulated the carrier orientations of the adapting stimulus while the carrier of the test stimulus remained horizontal. 

Methods

Subjects

Seven subjects (one female) with normal or corrected-to-normal visual acuity participated. All but the first author (KK) were naive to the purpose of the experiment. 

Apparatus

Stimuli were generated using the Matlab programming environment (Mathworks) and Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) operating on a computer (Apple PowerMac G5) and were displayed on a CRT monitor (Mitsubishi Electric RDF223H, 1200 × 800 pixels, refresh rate of 90 Hz, gamma-corrected) in a dark room. The viewing distance, constrained by a chin rest, was 86 cm. Viewing was monocular with the right eye. 

Stimulus

A bull's-eye-shaped fixation point (radius of 10′) was presented at the center of the display, and subjects were instructed to fixate on that point throughout each session. 

The adapting and test stimuli consisted of pairs of Gabor patches. The spatial frequency of each patch was 4 cpd, and the SD of its Gaussian window was 10.6′. The nominal maximum contrast of the adapting stimulus was 100%. The maximum contrast of the test stimulus should be decreased to yield a large aftereffect; we adjusted this variable in the range of 20–25% for each subject in a pilot experiment to enable reliable judgments about the orientation of the offset. The background and the mean luminance of the Gabor patches were 45 cd/m². 

The adapting stimulus consisted of 36 pairs of Gabor patches distributed within a region subtending 24° × 18° centered at the fixation point (Figure 2). The positions of the pairs were determined randomly within the constraint that the nearest two patches belonged to the same pair. To avoid position-specific contrast adaptation (McGraw, Levi, & Whitaker, 1999; Whitaker, McGraw, & Levi, 1997), positions were changed every 100 ms. The phases of the carriers were set randomly and independently across patches and were changed simultaneously with the position change. The vertical distance between the two patches in each pair was fixed at 75′, and the horizontal distance, or spatial offset, was 25′. The lower patches were either right or left in all the pairs throughout each session. We hereafter use the term “contrast-defined orientation” to refer to the orientation of the theoretical line connecting the centers of the upper and lower patches to distinguish it from luminance-defined carrier orientations. Orientation is denoted as the signed angular difference from the vertical, with positive (negative) angles meaning counterclockwise (clockwise) rotations. The horizontal offset between patches is denoted as a signed value: When the lower patch is right (left) relative to the upper one, the offset is positive (negative). Thus, when the spatial offset was +25′ (the lower patches were right), the contrast-defined orientation was +18.4°, and when the spatial offset was −25′ (the lower patches were left), the contrast-defined orientation was −18.4°. 

Figure 2

 

The adapting stimulus. The positions and carrier phases of all patches changed simultaneously every 100 ms. The fixation point was always located at the center. The carrier orientation was varied systematically.

The carrier orientations of the adapting stimulus were systematically manipulated under seven conditions (Figures 3a–3f). In the following, all orientation values refer to the configurations with the contrast-defined orientation of +18.4° (the carrier orientations with the contrast-defined orientation of −18.4° were the mirror reversals of these configurations). Under five conditions (a–e), the carriers of the upper and lower patches had the same orientation. The carriers were oriented at ±18.4° (b and d), ±71.6° (a and e), or 0°, i.e., vertically (c). Therefore, configuration (b) enabled the carrier orientation to be parallel to the contrast-defined orientation, and configuration (e) enabled the carrier and contrast-defined orientations to be perpendicular to each other. Under the two remaining conditions (f and g), the carriers of the upper and lower patches were perpendicular to each other; the carrier of the upper patch was oriented at ±18.4° and that of the lower patch was oriented at ∓71.6°, respectively. Configuration (f) resulted in the upper patch's carrier being oriented parallel to, and the lower patch's carrier being oriented perpendicular to, the contrast-defined orientation. 

Figure 3

 

Results of Experiment 1. The magnitude of the aftereffect is shown as a function of the carrier orientation(s) in the adapting stimulus (averaged across seven subjects). Error bars represent 95% confidence intervals obtained as 1.96 SEM across subjects. Each screenshot along the abscissa illustrates the stimulus configuration under each condition. The table below the screenshots shows the carrier orientation of the upper and lower patches with respect to vertical (0°) and the contrast-defined orientation (+18.4°), respectively. The rightmost screenshot illustrates the test stimulus. (a–e) Conditions under which the carriers of the upper and lower patches had the same orientation. (f, g) Conditions under which the carriers of the upper and lower patches were perpendicular to each other.

The test stimulus consisted of one pair of Gabor patches with horizontally oriented carriers. We used this carrier orientation, which differed from that of the adapting stimulus, to minimize the effect of carrier phase and orientation on the judgment of spatial offset between the patches (Keeble & Hess, 1998; Keeble & Nishida, 2001; Keeble & Hess, 2002; Popple et al., 2001; Popple & Levi, 2002; Whitaker, McGraw, Keeble, & Skillen, 2004). The phases of the carriers were set randomly and independently across trials. The test stimulus was presented 5° to the left of the fixation point to maximize the aftereffect magnitude and the detectability of its orientation dependency. Horizontal jitters in the range of ±8′ were introduced in each trial to reduce the availability of external cues (e.g., the frame of the display). In the test stimulus, the vertical distance between the two patches in each pair was 75′, and the horizontal distance between the test Gabor patches was varied, as described in the following section. 

Procedure

Each trial consisted of the presentation of the adapting stimulus, the presentation of the test stimulus, and a response phase (Movie 1). The adaptation phase lasted for 90 s in the first trial of each session and for 6 s in subsequent trials (top-up adaptation). The adaptation phase was followed by a 50-ms blank period. The test stimulus was then presented for 200 ms, and this was followed by the response phase in which only the fixation point was displayed. The subject was asked to judge whether the lower patch of the test stimulus was to the right or left relative to the upper patch. The subject's response, provided via a keyboard, triggered the adaptation phase of the next trial following the 50-ms blank period. To prevent decay in adaptation, subjects were instructed to respond within 1 s. The next trial began automatically if no response was produced within that time limit, and trials without responses were omitted from the data analysis. The first five trials in each session were discarded from the data analysis. 

 

Movie 1

 

The stimuli and procedures used inExperiment 1. This movie consists of 11 consecutive trials. The first trial contained the 90-s initial adaptation period, and the top-up adaptation at the beginning of each of the remaining trials was 6 s in duration. The fixation point was placed at the center. The adapting stimulus consisted of 36 pairs of Gabor patches, and the position of each pair and the phase of each patch were set randomly and refreshed every 100 ms. The test stimulus, a pair of Gabor patches, was shown to the left of the fixation point for 200 ms. For purposes of demonstration, the test stimulus presented in this movie has no horizontal offset. If adaptation occurs successfully, the observer would perceive the physically aligned pair of test patches as offset in the direction opposite to that of the adapting stimulus. The upper patches are always right relative to the lower patches in the adapting stimulus in this movie, but there were also conditions in which the upper patches were always left in the actual experiments.

The horizontal offset in the test stimulus was manipulated to determine the physical offset under which the two patches were seen as vertically aligned (point of subjective alignment: PSA). The PEST method was used (Findlay, 1978). The difference between PSAs obtained under the two mirror-image conditions divided by two was calculated as the aftereffect magnitude to eliminate the bias of orientation judgment within each subject. 

Predictions and results

In the Introduction section, we described four candidates for orientation connectivity between first-stage and second-stage filters (Figure 1). If only the “isotropic” hypothesis were correct, the magnitude of the aftereffect would not change across the conditions, whereas the other hypotheses predict that the aftereffect magnitude would change depending on the carrier orientations relative to the contrast-defined orientation. If the “parallel” hypothesis were correct, then the configuration (b) in Figure 3 would elicit the largest aftereffect, whereas configurations (e) and (f) would elicit no aftereffect because these stimuli contain carrier(s) orthogonal to the second-stage filter. If the “perpendicular” hypothesis were correct, then configuration (e) would elicit the largest aftereffect, whereas configurations (b) and (f) would elicit no aftereffect. The hypothesis of coexistence or a combination of these two orientation-selective mechanisms predicts the largest aftereffect for configurations (b) and (e) but no aftereffect for configuration (f). On the other hand, if the “parallel and perpendicular” hypothesis were correct, configuration (f) would elicit a large aftereffect comparable to those emerging for configurations (b) and (e). 

In Figure 3, the observed magnitude of the aftereffect is plotted against the seven conditions, each differing in the carrier orientations of the adapting stimulus. We compared this pattern of results with the aforementioned hypotheses. 

First, the aftereffect clearly depended on the carrier orientations (ANOVA, F(6, 36) = 10.90, p < 0.001; Friedman test, χ ²(6) = 26.37, p < 0.001), which is inconsistent with the “isotropic” hypothesis. Second, configurations (b) and (e) elicited an aftereffect larger than zero (t(6) = 4.87, p < 0.01; t(6) = 3.72, p < 0.01). Third, configuration (f) also elicited an aftereffect significantly larger than zero (t(6) = 7.41, p < 0.001), and the post hoc pairwise comparison revealed that the magnitude of the illusion was comparable to those for configurations (b) and (e) (parametric test using Tukey's WSD, p > 0.05; nonparametric test using Scheffe's method, p > 0.05). This pattern of results is explicable only in terms of the “parallel and perpendicular” hypothesis and not in terms of either of the two orientation-selective hypotheses, i.e., “parallel,” “perpendicular,” or a combination thereof. 

Discussion

The carriers that were parallel and/or perpendicular to the contrast-defined orientation yielded a stronger aftereffect. A large aftereffect was obtained even when one patch was parallel and the other patch was perpendicular, providing evidence that the carriers of the two Gabor patches need not be identical to elicit a strong aftereffect. 

Our results are apparently in accordance with the “parallel and perpendicular” hypothesis, that is, that each second-stage filter is exclusively connected to first-stage filters tuned to the parallel orientation as well as to first-stage filters tuned to the perpendicular orientation but not to first-stage filters with substantially different orientation preferences (Figure 1d). The connectivity between first-stage and second-stage filters is also presumed to have a broad orientation tuning at some level because the carrier orientations differing 18.4° from the parallel orientation (Figure 3c) and 36.8° from the perpendicular orientation (Figure 3a) elicited a substantial aftereffect. In contrast, the other hypotheses described in the Introduction section, or combinations thereof, cannot explain our results and can be successfully rejected. However, the actual nature of connectivity remains uncertain; the possibility exists that the activation of first-stage filters with irrelevant orientation preferences can trigger the activation of the second-stage filters when accompanied by the activation of first-stage filters with parallel or perpendicular preferences, and this would not be consistent with the simple “parallel and perpendicular” connectivity. To test this possibility, we conducted Experiment 2. 

Experiment 2: Integration across carrier orientations

The carrier orientations of the adapting stimulus were manipulated as in Experiment 1, but we added conditions in which the carrier of one patch was either parallel or perpendicular to the contrast-defined orientation, whereas the carrier of the other patch was not. 

Methods

Seven subjects (all males) participated. All but the first author (KK) were naive to the purpose of the experiment. Among them, five subjects also participated in Experiment 1, either before or after Experiment 2. The adapting stimulus was the same as that used in Experiment 1 (Figure 2) with the exception of the carrier orientations. We used six conditions for the carrier orientations (Figures 4a–4f), and among these configurations, (a), (b), and (c) were the same as the configurations shown in Figures 3b, 3e, and 3f, respectively, in Experiment 1. 

Figure 4

 

Results of Experiment 2. The magnitude of the aftereffect is plotted as a function of the carrier orientations in the adapting stimulus (averaged across seven subjects). Error bars represent 95% confidence intervals obtained as 1.96 SEM across subjects. Each screenshot along the abscissa illustrates the stimulus configuration under each condition. The table below the screenshots shows the carrier orientation of the upper and lower patches with respect to vertical (0°) and the contrast-defined orientation (+18.4°), respectively. The rightmost screenshot illustrates the test stimulus. (a–c) Conditions under which the carriers of both upper and lower patches were either parallel or perpendicular to the contrast-defined orientation. (d, e) Conditions under which the carrier of only one patch was either parallel or perpendicular to the contrast-defined orientation. (f) Condition under which the carriers of the two patches were neither parallel nor perpendicular to the contrast-defined orientation.

In configurations (d) and (e), the angular difference between the carriers of the upper and lower patches was 45°. In (d), the orientation of the carrier of the upper patch was +18.4° (parallel to the contrast-defined orientation), whereas that of the lower patch was −26.6° (rotated by −45° relative to the contrast-defined orientation). In (e), the orientation of the upper patch was +63.4° (rotated by +45° relative to the contrast-defined orientation), whereas that of the lower patch was −71.6° (perpendicular to the contrast-defined orientation). The carrier oriented at −26.6° or +63.4° would give rise to the minimum activation of first-stage filters, which are parallel or perpendicular to a second-stage filter, because these orientations differ by 45° compared with the preferred orientations for these first-stage filters. In (f), the carriers of the upper and lower patches were oriented at +63.4° and −26.6°, respectively, and thus both of these Gabor patches were poor stimuli to activate these first-stage filters. 

The test stimulus was the same as that used in Experiment 1, namely, a pair of Gabor patches with horizontal carriers. The method of constant stimuli was used, and two psychometric functions for each configuration were obtained from the two mirror-image conditions. Each psychometric function was fitted with the cumulative Gaussian function by the maximum-likelihood procedure and the PSA was determined to be the 50% point. The magnitude of the aftereffect was defined as the distance between the two PSAs divided by two. Thus, the obtained aftereffect magnitudes were free of within-subject orientation judgment bias and consistent with those in Experiment 1. The confidence intervals of the magnitude of the aftereffect were calculated using the bootstrap method (Foster & Bischof, 1997). 

Results

In Figure 4, the magnitude of the aftereffect is plotted against the six conditions differing in the carrier orientations of the adapting stimulus. The aftereffect clearly depended on the carrier orientations (ANOVA, F(5, 30) = 15.24, p < 0.001; Friedman test, χ ²(5) = 24.96, p < 0.001), and only the aftereffect magnitude for configuration (f), in which both carrier orientations were 45° away from the contrast-defined orientation, was not statistically significant (t(6) = 0.13, p > 0.10). Therefore, the present results for (a), (b), and (c) confirmed the orientation dependency found in Experiment 1. As in Experiment 1, the aftereffect magnitude was substantial for all of these configurations, and no statistical differences emerged among them. We also found that the aftereffect magnitude for (d) and (e), in which the carrier of one patch was parallel or perpendicular to the contrast-defined orientation while the other was 45° away, was significantly larger than zero (t(6) = 7.72, p < 0.001; t(6) = 4.97, p < 0.01). 

Post hoc pairwise comparisons (parametric: Tukey WSD method, nonparametric: Scheffe's method) revealed that the aftereffect magnitudes for (d) and (e) were not statistically distinguishable from those for (b) and (c), in which the carrier orientations of both patches were either parallel or perpendicular (p > 0.10). A parametric test revealed that configuration (a), which consisted of two parallel patches, elicited a larger aftereffect than configurations (d) and (e) (p < 0.05), but the differences were rejected by the nonparametric test. 

We conducted an additional experiment to confirm that the observed results of Experiment 2 were not specific to the particular stimulus size and shape used. First, we repeated Experiment 2 using an inter-patch distance twice as large as that in the original experiment. Thus, the vertical distance between the two patches in each pair was 150′ in the adapting stimulus as well as in the test stimulus, and the horizontal distance in the adapting stimulus was changed to 50′. The stimuli were rescaled by setting the viewing distance at 43 cm. The spatial frequency and SD of each Gabor patch was left unchanged from those in the original experiment, i.e., 4 cpd and 10.6′, respectively. The 36 pairs of Gabor patches in the adapting stimulus were distributed within a region subtending 48° × 36° centered at the fixation point. 

The average of the aftereffect magnitude across four naive subjects is shown in Figure 5. The results from the original experiment were clearly replicated; the aftereffect magnitude depended on carrier orientations (ANOVA, F(5, 15) = 9.12, p < 0.001; Friedman test, χ ²(5) = 15.71, p < 0.01), and equally robust magnitudes were obtained whether one or two patches in each adapting pair was parallel or perpendicular (Figures 5a–5e; p > 0.05). 

Figure 5

 

Results of an additional experiment with the doubled inter-patch distance. The magnitude of the aftereffect is plotted as a function of the carrier orientations in the adapting stimulus (averaged across four subjects). Error bars represent 95% confidence intervals obtained as 1.96 SEM across four subjects. The rightmost screenshot illustrates the test stimulus.

Second, we tested whether the results were sensitive to carrier isotropy in the test stimulus. We replaced the horizontal Gabor patch in the test stimulus with a concentric luminance grating modulated by a Gaussian window. The test patches had the same spatial frequency, RMS contrast, and Gaussian SD as the original horizontal test stimulus. To eliminate possible intrusion of luminance cues, we chose two mutually anti-phase modulations, both of which yielded total DC components of zero, and randomly assigned one to either the upper or lower patch and the other to the remaining patch. The results from a naive subject and two authors are shown separately in Figure 6. Again, the pattern found in the original experiment was clearly replicated; the aftereffect magnitudes were significantly larger than zero when at least one patch in each adapting pair was parallel or perpendicular (Figures 6a–6e) but not in configuration Figure 6f, in which neither patch was parallel or perpendicular. 

Figure 6

 

Results of an additional experiment in which a pair of concentric Gabor patches was used as the test stimulus (shown on the right). The magnitude of the aftereffect is plotted as a function of the carrier orientations in the adapting stimulus for each subject. Error bars represent 95% confidence intervals obtained by the bootstrap procedure. The configurations of the adapting stimulus were the same as those in the original Experiment 2.

Third, we used a random-dot pattern as the test stimulus carrier. A randomly generated pattern of 50% white and 50% black dots was spatially filtered by a band-pass filter with the center frequency of 4 cpd (the same as that used in the original experiment) and a bandwidth of 1 octave. Different patterns were chosen for the upper and lower patches individually and across trials. The randomly generated luminance peaks of this stimulus perturbed perception of the position, causing shallow psychometric function slopes. Nonetheless, one naive subject was able to accomplish the task with this random noise (Figure 7). She exhibited the same pattern of results as the original experiment; the aftereffect magnitudes were significantly larger than zero when at least one patch in each adapting pair was parallel or perpendicular (Figures 7a–7e) but not in Figure 7f. 

Figure 7

 

Results of an additional experiment in which a pair of random-dot patches was used as the test stimulus (shown on the right). The magnitude of the aftereffect for one subject is plotted as a function of the carrier orientations in the adapting stimulus. Error bars represent 95% confidence intervals obtained by the bootstrap procedure. The configurations of the adapting stimulus were the same as those in the original Experiment 2.

Discussion

The configurations in which the carrier orientation of at least one patch was either parallel or perpendicular to the contrast-defined orientation yielded a significant aftereffect (Figures 4a–4e). The aftereffect magnitude was not significantly different across these conditions irrespective of whether the configuration contained one or two patches with parallel or perpendicular carriers. 

Schematic account

As discussed in the previous section, the simplest explanation for the results of Experiment 1 is that a second-stage filter is connected only with first-stage filters having parallel or perpendicular preferences. However, this theory predicts no activation of the second-stage filter when one of the two patches defining contrast-defined orientation has a diagonal carrier, and therefore, it cannot explain the results of Experiment 2. The results can be explained by considering that a second-stage filter is equally activated whenever a subpopulation of first-stage filters tuned to the parallel or perpendicular orientation is activated. Although several FRF mechanisms may behave in this manner, one of the simplest examples is spatially heterogeneous connectivity: The input to a certain region of a second-stage filter comes from parallel or perpendicular first-stage filters, whereas the input to other regions comes from all first-stage filters irrespective of their orientation preference. An example of such connectivity is shown in Figure 8, in which a second-stage filter covers a triplet of Gabor patches in line. The central region of the second-stage filter has anisotropic connectivity, maintaining stronger connections with first-stage filters having parallel and perpendicular preferences, whereas other regions have isotropic connectivity, accepting inputs from all first-stage filters. 

Figure 8

 

Schema of spatially heterogeneous connectivity between first-stage filters and a second-stage filter in an FRF mechanism that explains the present results. The second-stage filter covers a triplet of Gabor patches. The central region of the second-stage filter is connected to first-stage filters with parallel or perpendicular orientation preferences. The surround region is connected with first-stage filters having all orientation preferences.

The output of this second-stage filter is determined by the strength of the input to the central region as well as the input to the neighboring regions. When subjects are exposed to our adapting stimulus, this second-stage filter produces an output that is large enough to render the aftereffect only if a significant amount of the input to the central region is coupled with a significant amount of the input to one of the neighboring regions. Because the contrast of the Gabor patches is constant, the input to these isotropically receiving regions would be constant across all of the stimulus configurations examined in our experiments. Therefore, the only determinant of the output of the second-stage filter is the input to the central region. As a result, the output is large when one patch is parallel or perpendicular and is paired with another patch defined by contrast modulation, whatever the carrier of that modulation might be. The output gradually decreases as the stimulation to the central region deviates from either parallel or perpendicular orientation. 

Our results support the previously proposed preference for a perpendicular as well as parallel relationship in contrast modulation processing (Dakin & Mareschal, 2000; Dakin et al., 1999; Keeble & Hess, 1998; Keeble & Nishida, 2001; Keeble & Hess, 2002). However, little evidence exists for neural mechanisms specifically tuned to a perpendicular relationship. Connections between neurons with similar orientation preferences are abundant in the horizontal connections of V1 (Bosking, Zhang, Schofield, & Fitzpatrick, 1997; Stettler, Das, Bennett, & Gilbert, 2002) and among the feedback connections from V2 to V1 (Shmuel et al., 2005). Responses of V1 neurons to an edge are facilitated by collinear flankers (Kapadia, Ito, Gilbert, & Westheimer, 1995; Mizobe, Polat, Pettet, & Kasamatsu, 2001; Polat, Mizobe, Pettet, Kasamatsu, & Norcia, 1998). In contrast, supports for perpendicular connectivity are rare (Merlin et al., 2011). The connectivity observed in the present and previous psychophysical studies might be neurally implemented in the higher cortex, e.g., V4. 

The biological significance of having simultaneous preferences for parallel and perpendicular relationships remains an intriguing issue. An ecological advantage may exist; however, previous studies on natural images have suggested the importance of processing solely relying on parallel relationships; only collinear and parallel structures are abundant in natural settings (e.g., Geisler, Perry, Super, & Gallogly, 2001), and luminance-defined and contrast-defined modulations tend to exist in the same, not perpendicular, orientations (Johnson & Baker, 2004). This issue should be addressed in future research. 

Our adaptation and aftereffect of horizontal offset may be mediated by a mechanism similar to that proposed to account for the conventional tilt aftereffect (Gibson, 1937; Gibson & Radner, 1937; cf. Georgeson, 2004); a bank of second-stage filters are tuned to various orientations of contrast modulation, and the distribution of activity across a population of such filters determines the perceived orientation of the spatial offset. This distribution may be altered by adaptation to a specific orientation of spatial offset. 

Task specificity

Is the observed connectivity specific to the task used? Investigations on perceived orientation of contrast modulation (Dakin & Mareschal, 2000; Dakin et al., 1999; Morgan & Baldassi, 1997; Morgan et al., 2000) and contour detection (Field et al., 1993; May & Hess, 2008) tend to favor orientation-specific connectivity (Figures 1a and 1b), while those on contrast modulation sensitivity (Motoyoshi & Nishida, 2004; Mussap, 2001; Prins, 2008) and orientation discrimination sensitivity (Keeble & Hess, 1998; Keeble & Nishida, 2001; Mussap, 2001) favor orientation integration mechanisms (Figures 1c and 1d). As discussed in the Introduction section, task differences might underlie this apparent discrepancy; carrier orientation might be an important cue in the perceived orientation of contrast modulation and contour detection studies but not in contrast modulation and orientation discrimination sensitivity studies. In our experiments, the manipulated carrier orientations were completely irrelevant to the task, but the results showed anisotropy, that is, both parallel and perpendicular relationships were equally favored compared to other orientation relationships (Experiment 1), and at the same time, the results supported integration of parallel or perpendicular orientation and other orientations at the level of second-stage filters (Experiment 2). Based on the lack of task relevancy of manipulated carrier orientations, this pattern of connectivity is thought to be intrinsic to FRF mechanism processing of contrast modulation. 

The possibility exists that other FRF mechanisms with more orientation-specific connectivity exist and play a major role when advantageous or necessary. For example, when the situation requires detection of orientation modulation across space rather than contrast modulation, as in discrimination between parallel and perpendicular carriers (Graham & Wolfson, 2004; Motoyoshi & Nishida, 2004; Prins, 2008), FRF mechanisms with sharp orientation tunings would be most useful. Such mechanisms might also be useful when explicit reporting of perceived orientation is required (Dakin & Mareschal, 2000; Dakin et al., 1999; Morgan & Baldassi, 1997; Morgan et al., 2000). In general, different FRF mechanisms could be dedicated to processing different features of contrast modulation (e.g., Arsenault, Wilkinson, & Kingdom, 1999; Kingdom, Prins, & Hayes, 2003; Motoyoshi & Kingdom, 2003). Yet, the similarity between our data and those of Keeble and Nishida (2001), who used a similar two-Gabor alignment task that did not involve adaptation and found parallel and perpendicular preferences, suggests that the results from the adaptation paradigm can be generalized to similar task situations. 

Stimulus specificity

We used a pair of horizontal Gabor patches as the test stimulus. However, this might have introduced unwanted side effects. First, prolonged viewing of the adapting stimulus might have induced contrast adaptation (Blakemore & Campbell, 1969; Langley et al., 1996) depending on the carrier orientations of the adapting stimulus. Second, afterimages of the adapting stimulus might have caused cross-orientation suppression to the subsequently viewed test stimulus (Polat & Sagi, 1993). Third, the conclusions of the present study regarding orientation connectivity in FRF mechanisms might only be applicable to the test stimulus consisting of horizontal Gabor patches. However, these possibilities were refuted by additional experiments in which isotropic patches (concentric patches and random-dot patches) used as the test stimulus produced the same pattern of results (Figures 6 and 7). 

Nonetheless, different stimuli may tap different mechanisms. Several previous investigations of carrier orientation influence on perceived orientation of contrast modulation have used stimuli physically containing clear tilted edges defined by contrast (Dakin & Mareschal, 2000; Dakin et al., 1999; Morgan & Baldassi, 1997; Morgan et al., 2000). These edges could be detected, for example, by first rectifying and low-pass filtering the image contrast and then identifying zero-crossings in the second spatial derivative. On the contrary, a pair of spatially offset contrast-defined patches is the minimum condition for defining contrast-defined orientation. It might be possible that the existence of tilted edges recruit additional mechanisms with higher orientation specificity, a mechanism dedicated, for example, to cue-invariant orientation processing. A similar aftereffect as ours was reported by Georgeson and Schofield (2002); however, they saw a tilt aftereffect for a contrast-defined sinusoidal grating over a random-dot carrier (see also Cruickshank & Schofield, 2005). It is difficult to determine whether their aftereffect is based on the same mechanism as ours. 

Our adapting stimulus consisted of randomly placed pairs of patches widely spread in the visual field and was a variation of Glass patterns (Figure 2). The present results might have been specific to this spatiotemporal global structure. A remote aftereffect would be an indication of dependence on spatial global structure; Roach, Webb, and McGraw (2008) showed that adaptation to a luminance-defined global structure elicited a remote tilt aftereffect, and they argued that this aftereffect resulted from adaptation at a higher stage where global form is detected. Our preliminary study suggested the presence of a remote aftereffect (Kobayashi & Murakami, 2009). A future study using an adapting stimulus defined by local spatial correlations in contrast but devoid of global structure or vice versa must be conducted to resolve this issue. 

Furthermore, the observed results might have been specific to the spatial scale of the stimuli. Skillen, Whitaker, Popple, and McGraw (2002) suggested that orientation connectivity between first-stage and second-stage filters is sensitive to spatial scales of the carrier and envelope. Although extended investigation of the effect of inter-patch distance on our aftereffect is beyond the scope of this article, our additional experiment using doubled inter-patch distance showed the same pattern of results (Figure 5), suggesting that the orientation connectivity in question is not strictly spatial scale-specific. Moreover, this observation ruled out the possibility that our results can be explained by luminance cues in the adapting stimulus. The perceptual alignment of a triplet of Gabor patches is highly sensitive to the phase (Whitaker et al., 2004) and orientation (Keeble & Hess, 1998; Keeble & Nishida, 2001; Keeble & Hess, 2002; Popple et al., 2001; Popple & Levi, 2002) of the carriers. Although we randomized the carrier phases of the adapting stimulus to avoid the intrusion of luminance cues, we might have failed to eliminate all of them. However, these effects of luminance cue diminish as the inter-patch distance increases (Akutsu, McGraw, & Levi, 1999; Popple et al., 2001; Whitaker et al., 2004). In clear contrast, our additional experiment showed a robust effect against the distance doubling (Figure 5). 

Relationship with contour formation

Our stimuli resembled those used in contour detection studies (Field et al., 1993; Hess et al., 2003). FRF mechanisms may be responsible for this contour detection task (May & Hess, 2008), although whether the same mechanism is responsible in our study is not clear; however, several differences exist between our study and contour detection situations. 

First, we observed a significant aftereffect with the adapting stimulus that consisted of perpendicular patches and parallel patches, whereas contour detection is poor in the perpendicular case (Field et al., 1993; May & Hess, 2007, 2008). The difference in detectability between parallel and perpendicular contours depends on inter-patch separation (May & Hess, 2008), but we observed a similar pattern of results when the separation was doubled (Figure 5). Second, detection performance deteriorates with large deviations in element orientation along the contour (e.g., Field et al., 1993; May & Hess, 2008), but we observed a significant aftereffect following adaptation to pairs in which one patch was oriented orthogonally (Figures 3f and 4c) or at 45° (Figures 4d and 4e) to the other patch. Third, the configurations of our adapting stimulus in which one patch was parallel or perpendicular to the contrast-defined orientation and the other patch was not (Figures 4d and 4e) could have signaled a curved rather than a straight contour and, thus, should have been less effective in eliciting the aftereffect if a contour linking system were solely responsible. However, we obtained a vigorous aftereffect in either case. 

Of course, the involvement of a contour detection system cannot be completely ruled out. Keeble and Hess (1998) investigated performance on a 3-Gabor alignment task with various carrier orientations and found that thresholds were higher in the configurations in which the central patch had a parallel orientation and the upper and lower patches were 45° away, in the opposite direction from each other, forming a curved contour. The authors suggested that intrusion of contour perception established by some other mechanisms may be responsible for the deterioration in performance, based on the observation that the points of subjective alignment with these configurations were greatly shifted in the direction consistent with the contour formed by the carriers, as well as the perceptual saliency of the contour structure reported by the subjects. Our experimental paradigm may show a similar effect if the magnitude of aftereffect after adaptation to triplets of Gabor patches is measured as a function of the carrier orientations. 

Relationship with contrast facilitation

One may argue that the change in the aftereffect magnitude reflects the outputs of first-stage filters rather than the connectivity between first-stage and second-stage filters. A related phenomenon is contrast facilitation, in which the detection threshold for one Gabor patch decreases when it is accompanied by other Gabor patches with the same carrier orientation (Polat & Sagi, 1993, 1994). This modulation might occur before second-stage filters are involved; some studies have proposed that a possible physiological basis for this facilitation exists as early as in V1 (e.g., Bosking et al., 1997; Kapadia et al., 1995). Thus, differences in the configuration of the adapting stimulus might have affected the signal strength prior to input to second-stage filters. 

However, this scenario is unlikely for three reasons. First, our results showed that the upper and lower carriers need not be parallel to elicit a large aftereffect (Experiment 1, Figure 3f; Experiment 2, Figures 4c–4e). A number of psychophysical studies have reported the absence of facilitation by flankers with a perpendicular orientation (e.g., Chen & Tyler, 2002; Polat & Sagi, 1993). Facilitation in a cross-orientation configuration has been reported in limited cases, but the orientation tuning of such facilitation is too broad to account for our results (Yu, Klein, & Levi, 2002). Second, our aftereffect was robust across the various phases of the adapting stimulus carriers, whereas contrast facilitation strongly depends on phase (Williams & Hess, 1998). Third, contrast facilitation was found at perithreshold levels rather than at suprathreshold levels (Williams & Hess, 1998), whereas our adapting stimulus was characterized by a high contrast. 

Conclusion

The aftereffect of spatial offset between two Gabor patches was measured as a function of the carrier orientations of the adapting stimulus. The aftereffect was large when at least one patch in the adaptor pair had a carrier either parallel or perpendicular to the contrast-defined orientation, and the aftereffect magnitude was constant regardless of whether the adaptor pair contained one or two such patches. These results suggest that connectivity between first-stage and second-stage filters depends on their preferred orientations, but the outputs of first-stage filters are integrated in the second-stage filters in a complex manner. One possibility is that the connectivity is spatially heterogeneous and is characterized by anisotropy in one region and by isotropy in other regions. 

Acknowledgments

This study was supported by the Nissan Science Foundation, JSPS Funding Program for Next Generation World-Leading Researchers (LZ004), JSPS Grant-in-Aid for Scientific Research (2310037), and JSPS Grant-in-Aid for JSPS Fellows. 

Commercial relationships: none. 

Corresponding author: Kenji Kobayashi. 

Email: [email protected]. 

Address: Department of Life Sciences, University of Tokyo, 3-8-1 Komaba, Meguro-ku, Tokyo 153-8902, Japan.