Two contrast adaptation processes: Contrast normalization and shifting, rectifying contrast comparison

Abstract

We present psychophysical results demonstrating the interaction of two contrast adaptation processes in human vision: (1) A contrast‐gain‐control process of the normalization type and (2) a recently‐discovered shifting, rectifying contrast‐comparison process. Observers adapted (for 1 s) to a grid of Gabor patches at one contrast, then a brief (94 ms) test pattern was shown, and then the adapt pattern was shown again (1 s). The test pattern was the same as the adapt pattern except that the Gabor patches had two different contrasts arranged to create vertical or horizontal contrast-defined stripes. Observers identified the orientation of the test pattern's stripes. Performance is a complicated (“butterfly shaped”) function of the average test contrast, centered at the adapt contrast. This shape is a consequence of the interaction of the two contrast adaptation processes. At the ends of the function are “Weber zones” in which the contrast‐gain‐control process dominates, and at the center of the function is a “Buffy zone” in which the recently‐discovered contrast‐comparison process dominates.

Introduction

Spatial contrast (variation in luminance over space) is very important. When we look at an image, we tend to look at the parts with high contrast (e.g., Reinagel & Zador, 1999). This makes intuitive sense since we tend to look at objects, and objects tend to have contrast. That is, we do not spend a huge amount of our time, in general, looking at uniform areas such as the blue sky. 

This paper presents psychophysical results demonstrating the interaction of two processes in human vision both of which depend heavily on the contrast of the preceding and current visual stimuli. One process is a contrast‐gain‐control process of the normalization type that has been discussed a great deal before and thus is relatively “old” although extended to further conditions here. The other process is a shifting, rectifying contrast‐comparison process that has recently been suggested to explain a recently‐discovered psychophysical effect and thus is relatively “new”. 

The benefits of adaptation processes are usually thought to fall into two categories (see, e.g., Clifford, 2005; Graham & Wolfson, 2007; Kohn, 2007):

•  
  Adaptation should improve performance near the adapting level by moving a limited dynamic range to be centered near the adapting level (as seen in light adaptation).
•  
  Adaptation should highlight changes or novel information because differences are important (as seen in the center–surround organization of retinal ganglion cells); a corollary to this is that adaptation should make neural coding more efficient (by reducing redundancy or improving representational efficiency).

We will discuss below how the two contrast adaptation processes might relate to these categories, but it is not straightforward.

The terms “adaptation” and “masking” have overlapping uses. The experiments we present in this paper could be called either masking or adaptation experiments, and they might well be called mixed‐adaptation‐and‐masking (e.g., Graham, 1989, p. 27). The same terms “masking” and “adaptation” are also used to apply to processes. Here again they have usages that overlap. To us at least, the term “adaptation” is more likely to mean processes in which the buildup of effects from past events influences current events. And the term “masking” is more likely to mean processes in which, of two stimuli simultaneously present, one interferes with the perception of the other (as in the face of a person wearing the item of clothing called a mask). Our explanations for the effects seen in the experiments below depend primarily on the buildup of past influences, so here we are going to generally use the term “adaptation” rather than “masking.” The period over which this buildup occurs is less than 1 s in the experiments reported here, so others might find “masking” a more congenial description for the experimental procedure. When using any term like “adaptation” or “masking,” we will try to make it clear whether we are using the term to refer to a paradigm, effect, or process. 

Contrast‐gain‐control process of the normalization type (“old”)

The first of the two processes is a contrast‐gain‐control process of the normalization type, which we will hereafter call simply contrast normalization. (The early literature includes Bonds, 1989; Foley, 1994; Heeger, 1992; Robson, 1988; Wilson & Humanski, 1993. A comprehensive early article is Carandini, Heeger, & Movshon, 1997. An in-depth article on our earlier work and normalization is Graham & Sutter, 2000.) While the underlying neural mechanism suggested to produce contrast normalization has changed over time (see Carandini, Heeger, & Senn, 2002), the basic divisive nature of normalization has remained the same: The response from one unit (the “target unit,” which can be thought of as a cell) is normalized by (divided by) the collective response of a bunch of units (the normalization pool). Thus, the response of the target unit will change if the normalization pool's response changes, even if the target unit itself continues to receive the same stimulation. So the target unit's response does not just reflect its stimulation but also reflects the context in which it resides. A process like normalization is very useful since it can move the limited dynamic range to be centered near the ambient contrast level while preserving selectivity along dimensions like orientation and spatial frequency (see discussions and references in, e.g., Carandini, 2004; Lennie, 1998; Victor, Conte, & Purpura, 1997). Further, such a process has the right properties to help encode natural images efficiently (Malo, Epifanio, Navarro, & Simoncelli, 2006; Schwartz & Simoncelli, 2001). Many investigators besides ourselves have invoked inhibition among channels, frequently in a normalization network, to account for behavioral results from texture segregation and other perceptual tasks using patterns both in experiments that are explicitly adaptation or masking experiments and in experiments that are framed in other ways (e.g., Foley, 1994; Foley & Chen, 1997, 1999; Itti, Koch, & Braun, 2000; Meese, 2004; Meese & Holmes, 2002; Olzak & Thomas, 2003; Teo & Heeger, 1994; Watson & Solomon, 1997; Wilson & Humanski, 1993; Wilson & Kim, 1998). 

We (e.g., Graham & Sutter, 2000; Graham & Wolfson, 2004; Wolfson & Graham, 2005) have seen contrast normalization many times in results collected with a 0% adapting contrast. Adapting to 0% contrast means adapting to a blank gray field. (Sometimes this is considered to be “no adaptation”, but this is a misleading way of thinking about contrast adaptation as we will show.) Suppose an observer adapts to 0% contrast for 1 s and then is shown a test pattern like that in Figure 1 for 100 ms. If the contrasts of the Gabor patches in the test pattern are something like 5% and 15%, it is easy for the observer to identify the orientation of the contrast-defined stripes (horizontal in this example). On the other hand, it is very hard if the Gabor patch contrasts are something like 55% and 65%. The difference between the contrasts in both of these imagined test patterns is the same (10%), but the activity in the normalization pool will be much greater with the higher contrast Gabor patches than with the lower contrast ones. In summary, when the test contrasts are far above the 0% adapt contrast, performance declines. We will show results below and also explain how this can be thought of as Weber-law-like behavior. 

Figure 1

 

Example of a 15 × 15 test pattern created by alternating rows of two different Gabor patch contrasts. The orientation of the Gabor patches (the first-order orientation) is vertical in this example. The orientation of the contrast-defined stripes (the second-order orientation) is horizontal in this example. The orientation of the contrast-defined stripes in a test pattern is hard to identify after adapting to some contrasts and easy to identify after adapting to others.

Shifting, rectifying contrast‐comparison process (“new”)

The second of the two processes is a shifting, rectifying contrast‐comparison process that we recently proposed (Graham & Wolfson, 2007; Wolfson & Graham, 2007a) and nicknamed Buffy adaptation for lack of a better short name. In brief, the contrast‐comparison process shifts a rectification function along a contrast axis, based on the recently experienced contrast, so the output of this process at each location in the visual field is the unsigned difference between the current contrast and the recent average contrast (Figure 2). Of course, suggesting a rectification process itself is not new; however, such a process is rarely if ever suggested for the contrast dimension rather than the luminance dimension. Nor is the shifting of a function new; shifting monotonic functions along a contrast axis has often been suggested for contrast adaptation. It is the shifting of a rectification function along a contrast axis—the contrast‐comparison process—that is new. (Further references to previous literature are presented in the General discussion section.) 

Figure 2

 

Shifting, rectifying contrast‐comparison process. The input to this process reflects the local contrast at each position in the visual field. We do not yet know how local, but something like the size of a Gabor patch in our patterns. The input is compared to an adaptable contrast‐comparison level. The contrast‐comparison level adapts to equal the recently experienced contrast (which is the adapt contrast in our experiments). The output from this process is an unsigned measure of the difference between the current contrast and the recent average contrast. Thus, the output's magnitude gives the magnitude of the change from one moment (the adapt pattern in our case) to the next (the test pattern).

The contrast‐comparison process is easily seen with nonzero adapt contrasts (Wolfson & Graham, 2007a). Suppose an observer adapts to a grid of 50% contrast Gabor patches (so just like Figure 1, but all of the Gabors have the same contrast). If the test pattern is composed of Gabor patches with contrasts a bit below the adapt contrast (e.g., 35% and 45%), the observer can easily identify the orientation of the contrast-defined stripes. Similarly, if the Gabor patch contrasts are a bit above the adapt contrast (e.g., 55% and 65%) it is also easy. On the other hand, it is hard to identify the orientation of the contrast-defined stripes in the test pattern if it is composed of two contrasts that straddle the adapt contrast (e.g., 45% and 55%). We call this result the “straddle effect”. 

As stated above, it is often thought that the function of adaptation is to improve performance near the adaptation level. The straddle effect is the opposite of that: performance is worse on contrasts near the adaptation level and improves on contrasts a bit away from the adaptation level. Maybe this effect falls into the second category, that of “highlighting changes”: performance is enhanced for novel stimuli, that is, performance is better on test patterns composed of contrasts different than the adaptation contrast level. However, as we will show (and as can be inferred from the 0% adapt contrast case mentioned in the prior section), this is not always the case. 

Organization

In the first part of this paper (Experiments 1 and 2), we further explore the newly discovered contrast‐comparison process at middle ranges of adapt and test contrasts. We show that the straddle effect occurs both for large patterns as previously reported (Wolfson & Graham, 2007a) and also for other conditions. We present subsets of these results in the form of constant‐transient trios, a form that is useful in rejecting alternative explanations of the straddle effect. 

In the second part of the paper (Experiments 3–6), we present results from the full possible range of adapt and test contrasts. This allows us to show and explore the interaction of the contrast‐comparison process and contrast normalization. 

Methods

Observers

All observers were Columbia University undergraduates with normal (or corrected-to-normal) visual acuity. All observers gave informed consent and were paid for their participation. Observer SYP had extensive knowledge of the experiment but the others did not. 

Patterns

Each pattern was a grid of Gabor patches. Two grid sizes were used: 15 × 15 ( Figure 1) and 2 × 2 (multiple examples in Figure 3). The grid of Gabors was centered within a 16 × 16 deg (1024 × 1024 pixel) gray square at the same mean luminance as the Gabor patches. When no Gabor patches were present, the 16 × 16 degree area was always gray at the same mean luminance. 

Figure 3

 

The time‐course of a trial is shown at top. One adapt pattern is shown here with 3 possible test patterns. The adapt pattern is composed of Gabor patches all at the same contrast: the adapt contrast A. The test pattern is composed of Gabor patches with two different contrasts: test contrast C1 and test contrast C2. The post-test pattern is identical to the adapt pattern. Example contrast values are shown. Between trials the screen remained gray. Contrast differences in the gray-level images are exaggerated to increase their salience. The test pattern names (“Above”, “Straddle”, and “Below”) refer to the test contrasts relative to the adapt contrast. The lower left box shows the 4 different test‐pattern configurations and the task. The lower right box defines some terms in symbols. In words, the average test contrast is the average of the two Gabor patch contrasts in the test pattern. The test contrast difference is the (positive) difference between the two Gabor patch contrasts in the test pattern.

There are 4 different test‐pattern configurations as shown in the lower left box (labeled “Task”) in Figure 3: i. vertical contrast-defined stripes composed of horizontal Gabor patches (upper left image), ii. vertical stripes composed of vertical Gabors (lower left image), iii. horizontal stripes of horizontal Gabors (upper right image), and iv. horizontal stripes of vertical Gabors (lower right image). 

An adapt pattern was a uniform grid of Gabor patches (that is, the contrast of all the Gabor patches was identical in any given adapt pattern). During a trial, the orientation of the Gabor patch elements did not change (that is, if the adapt pattern was composed of vertical Gabor patch elements, then the test pattern was too). However, the orientation of the Gabor patches varied randomly from trial to trial. Throughout an experimental session, the grid size of the adapt and test patterns was the same (2 × 2 or 15 × 15). 

Details of the Gabor patches. Each Gabor patch was truncated at 1 × 1 deg (64 × 64 pixels) at the viewing distance of 90 cm. (Distances are approximate as observers' heads were not constrained.) A Gabor patch is a sinusoidal grating windowed by a two-dimensional Gaussian function. The sinusoidal grating in our Gabor patches had a period of 0.5 deg (32 pixels), which corresponds to a spatial frequency of 2 c/deg. The positive zero‐crossing of the sinusoid was always at the center of each patch (so the “dark bar” of the sinusoid was to the left of the center for vertical patches and on top for horizontal patches). The Gaussian function had a full‐width‐at‐half-height of 0.5 deg (32 pixels). The contrast of a Gabor patch is computed by taking the difference between the luminance at the peak of the Gaussian and the mean luminance of the pattern, and then dividing that difference by the mean luminance. 

Observer's task

The observer's task (illustrated in the lower left box of Figure 3) was to identify the orientation of the contrast-defined stripes in the test pattern using the computer's keyboard. Feedback was provided to the observer. 

This task, in which the observer identifies the spatial arrangement of two different contrast levels in a test pattern, depends on a comparison between two different values of contrast that are simultaneously present. This is not the kind of task used in most studies of contrast adaptation or masking although there have been some (presented in the General discussion section). 

The patterns were always shown foveally with one exception. The one exception—in which patterns are shown in the near-periphery to the left and right of fixation—is briefly mentioned in the Some generalizations of the straddle effect section. 

Procedure

The time‐course of each trial was as follows: the observer pressed the “0” key to start the trial, the screen remained gray (0% contrast) for 500 ms, then the adapt pattern was shown for 1 s, then the test pattern was shown for 94 ms, then the adapt pattern was shown again for 1 s, then the screen returned to gray for at least 100 ms, then the observer responded. The screen remained gray between trials. The mean luminance was constant throughout the experiment. The time‐course is sketched at the top of Figure 3. 

Figure 3 shows three example trials, all of which have the same adapt contrast (A = 50%) but different test pattern contrasts. The test patterns are named “Above”, “Straddle,” and “Below.” The contrasts of the Gabor patches in the test pattern (C1 and C2) are written in parentheses just below the name. The test contrasts are different in each of the three examples and thus the average test contrast (the average of C1 and C2) is different for each test pattern (and is written below the values of C1 and C2). However, the test contrast difference (the difference between C1 and C2) is the same in each of the examples (10%). 

The adapt pattern shown before the test pattern was identical (in contrast, duration, size, etc.) to the pattern shown after the test pattern (the post-test pattern) in all experiments except for one. The one exception—in which the screen returned to gray (0% contrast) immediately after the test pattern—is brief mentioned in the Some generalizations of the straddle effect section. 

An aside. There is an alternate, formally equivalent way to describe our sequence of visual patterns. In Figure 3 and the paragraphs above (and in the rest of this paper), we describe it as: a 1-s exposure of the adapt pattern, followed by a 94-ms exposure of the test pattern (during which the adapt pattern is not present), followed by a further 1-s exposure of the adapt pattern. One could instead describe it as: an adapt or background pattern that was exposed for the full period of 2+ s, with a 94-ms exposure of a probe pattern superimposed on the background pattern in the middle of the full time period. The probe pattern in this second description is just equal to the difference between the test pattern in the first description and the background pattern. For example, consider the “Straddle” test pattern in Figure 3, which contains contrast levels of 45% and 55% presented briefly immediately after an adapt pattern of 50% contrast and then followed immediately by a post-test pattern of 50% contrast. (The post-test pattern is the same as the adapt pattern.) Using the alternate probe-background description, one would say there was a probe pattern containing contrast levels of −5% (a contrast decrement) and +5% (a contrast increment) superimposed briefly on the background pattern of 50%. 

The original experiments from which the experiments here developed were described using the probe-background terminology (Graham & Wolfson, 2007). In these original experiments the background pattern was on for at least 2 s and a very brief probe pattern was superimposed on the background pattern in the middle of its presentation time. Some of the background patterns in these original experiments were stationary unchanging patterns, exactly like those here. Most background patterns in these original experiments, however, had contrast that was flickering in time at various frequencies with the probes superimposed at various phases. 

These original experiments using spatial patterns for backgrounds and probes were analogs to still earlier experiments we and others had done (see review in Wolfson & Graham, 2006) that used spatially‐homogeneous disks of light to explore the dynamics of light adaptation. We substituted patterns for the homogeneous disks in order to explore the dynamics of contrast adaptation. (“Light adaptation” and “contrast adaptation” are not used in a totally consistent fashion throughout the literature. Here we mean “light adaptation” to refer to processes that are primarily dependent on luminance levels in the pattern, that is, processes having results that are better predicted by luminance than by contrast. And we mean “contrast adaptation” to refer to processes that are primarily dependent on contrast, that is, processes having results better predicted by contrast than by luminance.) 

Details of the experiments

For each of the 6 experiments presented in this paper, Table 1 lists the experiment number, the adapt contrasts, the lower of the two contrasts in the test pattern, the test contrast difference, the grid size, and the figure where the data is plotted. Within a session of an experiment, all 4 test‐pattern configurations, all of that experiment's adapt contrasts, and all of that experiment's test contrasts were intermixed. For Experiment 1 (2, 3, 4, 5, 6) each session was 320 (320, 370, 408, 416, 416) trials long. The number of trials per point per observer per experiment is as follows:

• Experiment 1:  
  JRC 240, KLM 72, RK1 104, RK2 80, VR 96.
• Experiment 2:  
  JRC 160, KLM 72, MM 72, RK 80, VR 80.
• Experiment 3:  
  RK 60, SYP 60, NA 22.
• Experiment 4:  
  MM 56, RK 72, VR 56.
• Experiment 5:  
  NA 80, RK 96, SYP 80.
• Experiment 6:  
  NA 80, RK 80, SYP 80.

Table 1

 

For reference: Contrasts of the Gabor patches in the experiments. All Gabor patches in an adapt pattern have the same contrast (the adapt contrast). A test pattern has alternating rows (or columns) of two Gabor patch contrasts (test contrast C1 and test contrast C2). In this table test contrast C1 is always the lower of the two test contrasts, so test contrast C2 would be the sum of test contrast C1 and the test contrast difference. In the experiments, which contrasts we call C1 and C2 is arbitrary, that is, if there is a test pattern composed of contrasts (C1, C2), then there is also one composed of (C2, C1). “W&G07” is Wolfson and Graham (2007a). Curly brackets indicate that only a subset of the data is shown in that figure.

Experiment	Adapt contrast (%)	Test contrast C1 (%)	Test contrast difference	Grid size	Plotted
1	35	15, 20, 25, 30, 35, 40, 45	10	15 × 15	Figure 5
	50	30, 35, 40, 45, 50, 55, 60	10	15 × 15	Figures 4 and 5, {W&G07 2}
	65	45, 50, 55, 60, 65, 70, 75	10	15 × 15	Figure 5
	35	15, 25, 35, 45, 55, 65	20	15 × 15	W&G07 1
	50	10, 20, 30, 40, 50, 60, 70	20	15 × 15	W&G07 1, {W&G07 2}
	65	15, 25, 35, 45, 55, 65	20	15 × 15	W&G07 1
2	35	15, 20, 25, 30, 35, 40, 45	10	2 × 2	Figures 6 and { 7}
	50	30, 35, 40, 45, 50, 55, 60	10	2 × 2	Figures 6 and { 7}
	65	45, 50, 55, 60, 65, 70, 75	10	2 × 2	Figures 6 and { 7}
	35	15, 25, 35, 45, 55, 65	20	2 × 2	{ Figure 7}
	50	10, 20, 30, 40, 50, 60, 70	20	2 × 2	{ Figure 7}
	65	15, 25, 35, 45, 55, 65	20	2 × 2	{ Figure 7}
3	0	0, 10, 20, …, 70, 80, 90	10	2 × 2	Figures 8, 11, 12, 13, and A4
	25	0, 10, 20, …, 70, 80, 90	10	2 × 2	Figures 11, 12, 13, and A4
	50	0, 10, 20, …, 70, 80, 90	10	2 × 2	Figures 11, 12, 13, and A4
	75	0, 10, 20, …, 70, 80, 90	10	2 × 2	Figures 11, 12, 13, and A4
	100	0, 10, 20, …, 70, 80, 90	10	2 × 2	Figures 11, 12, 13, and A4
4	0	5, 10, 15, …, 75, 80, 85	10	2 × 2	Figures 8, 12, and A4
	10	5, 10, 15, …, 75, 80, 85	10	2 × 2	Figures 12 and A4
	50	5, 10, 15, …, 75, 80, 85	10	2 × 2	Figures 12 and A4
5	50	5, 7.5, 10, …, 87.5, 90, 92.5	5	2 × 2	Figure 10
	50	5, 10, 15, …, 75, 80, 85	10	2 × 2	Figures 9 and 10
6	50	5, 7.5, 10, …, 87.5, 90, 92.5	5	15 × 15	Not shown
	50	5, 10, 15, …, 75, 80, 85	10	15 × 15	Figure 9

The standard error bars on all the figures were calculated as the standard error of the mean of the performances in different sessions. Thus these error bars incorporate both the within-session variability and the between-session variability due to systematic shifts in performance from one session to the next. 

Observer RK is shown twice in Figures 4 and 5 because she ran Experiment 1 twice. The second time she ran Experiment 1 ( Figure 5, × symbols) she interleaved sessions of Experiment 2 ( Figure 6, square symbols). RK is also listed twice on Figures 8 and 12 (and Figure A4) because those figures show data from two different experiments and RK ran in both experiments (Experiment 3 plotted with square symbols and Experiment 4 plotted with × symbols). 

In Experiments 3 and 4, a small square fixation point was shown (for 1 s) immediately after the observer pressed the “0” key to start each trial, before the screen went gray for 500 ms. We added the fixation point since some trials had a 0% contrast adapt pattern (that is, the screen remained gray at the same mean luminance) and we wanted to ensure that the observer knew that their keypress was successful and the trial had started. Observers were instructed to look at the fixation point in these experiments. In the experiments without a fixation point, observers were simply instructed to look at the patterns (which were centered on the screen). In Experiments 1 and 6, which used the large 15 × 15 grid of Gabor patch elements, the pattern covered nearly the whole screen. Observers were encouraged to blink between trials. 

Equipment

The experiments were run on a Macintosh G4 with an iiyama VisionMaster Pro 451 CRT and an ATI Radeon 8500 Mac edition video card. The resolution was 1280 × 1024 pixels at 85 Hz. The mean luminance was about 50 cd/meter ² and was constant throughout the experiment. The room was dark. The monitor's lookup table was linearized. Stimuli were generated and presented using MathWorks' MATLAB with the Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). 

Results and discussion

Middle adapt and average test contrasts (Experiments 1 and 2)

The straddle effect

The effect that led us to propose the shifting, rectifying contrast‐comparison process is seen in the results plotted in Figure 4. Most of the subsequent results will be plotted in a similar manner. Remember that each test pattern is composed of Gabor patches at two contrasts (C1 and C2) in alternating rows (or columns) resulting in horizontal (or vertical) contrast-defined stripes. On the vertical axis is percent correct identification of these contrast-defined stripes in the test pattern. The top x-axis shows the values of C1 and C2. The test contrast difference (the difference between C1 and C2) is always 10% in this figure. The bottom x-axis shows the average test contrast (the average of C1 and C2). The adapt contrast is indicated by the arrow and is 50% in this figure. Results are shown for 4 observers (one of which ran the experiment twice, see Methods section for details), distinguished by different symbols. The gray-background legend at the top of the figure indicates all of these values and symbols (as well as indicating that the patterns' grids were 15 × 15 Gabor patch elements). 

Figure 4

 

Percent correct identification of the orientation (of the contrast-defined stripes) in the test pattern for multiple observers. Two equivalent x-axes are shown: the axis at the top gives the two test contrasts (C1 and C2) of the Gabor patches in the test pattern; the axis at the bottom gives the average test contrast (the average of C1 and C2). For all points on this plot, the test contrast difference (that is, the positive difference between C1 and C2) is the same (10%), and the adapt contrast (shown by an arrow along the bottom x-axis) is the same (50%). Performance is good on the Below and Above patterns (in the range shown here) and poor on the Straddle pattern. We call this the straddle effect. These data are from Experiment 1. Error bars show ±1 SE of the means across sessions.

Each curve in Figure 4 is shaped roughly like the letter V: performance is poor at the center and good at the ends. The test pattern plotted in the center was composed of 45% and 55% contrast Gabor patches. We call this a Straddle test pattern since the two test contrasts straddle the adapt contrast (of 50%). Performance is poor for Straddle test patterns. Performance is better on, for example, the test pattern composed of 55% and 65% contrast Gabor patches. This pattern has an average test contrast of 60%, which is above the adapt contrast of 50% so we call this an Above test pattern. Performance is good on Above test patterns. Performance is also good on Below test patterns. This “straddle effect” was described verbally in the Introduction section. 

If we change the adapt contrast, the whole curve shifts as is shown in Figure 5. The green curves are from Figure 4. Those were collected with an adapt contrast of 50%. The red curves were collected with an adapt contrast of 35% and the blue curves with 65%. There are subtle differences, but overall, the curves are all the same V shape. Changing the adapt contrast simply shifts the curves along the contrast axis. The poorest performance is always at the point where the adapt contrast (indicated by the arrows) equals the average test contrast. Those test patterns are the Straddle test patterns. Performance is better on the Above and Below test patterns. 

Figure 5

 

Results showing the straddle effect with three different adapt contrasts: 35% in red, 50% in green (the same results shown in Figure 4), and 65% in blue. The adapt contrasts are indicated by the arrows along the bottom x-axis. The results curves look like “V”s that shift with adapt contrast. Performance is poor for Straddle patterns (that is, when the average test contrast equals the adapt contrast) and good for Below and Above patterns (that is, when the average test contrast is less than or greater than the adapt contrast) in the range shown here. Error bars show ±1 SE of the means across sessions. These data are from Experiment 1. These results are like those published in Wolfson and Graham (2007a) with a larger test contrast difference.

This effect is big. Consider the points that are above the green arrow in Figure 5. All of these points represent performance on the exact same test pattern (composed of 45% and 55% contrast Gabor patches) after adapting to different contrasts. After an observer adapts to 50% (green points), performance is generally poor on this test pattern. But after an observer adapts to 65% (blue points), performance is very good on that same test pattern. And after an observer adapts to 35% (red points), performance is also very good on that same test pattern. These three cases correspond to the same test pattern (containing 45% and 55% contrast Gabor patches) being a Straddle test pattern (adapt contrast of 50%), a Below test pattern (adapt contrast of 65%), and an Above test pattern (adapt contrast of 35%). 

Some generalizations of the straddle effect

Does the grid size matter? We are curious about the spatial extent of this adaptation effect. To begin exploring this, we changed the size of the patterns. In the experiment just discussed, the pattern's grid of Gabor patches was 15 × 15 (see Figure 1). We have repeated the experiment using a much smaller grid size: 2 × 2 (see examples in Figure 3; this 2 × 2 grid is the same as a 2 × 2 subsection of the 15 × 15 grid). Results collected with the large grid and the small grid are very similar as can be seen by comparing the results in Figure 5 to those in Figure 6. In both cases, performance is poorest for the Straddle test patterns and better for Below and Above test patterns. There are differences in detail between the results, but at the descriptive level of analysis here, the results are effectively the same. 

Figure 6

 

Same conditions as plotted in Figure 5, but the patterns here were only 2 × 2 grids of Gabor patch elements (Experiment 2) whereas they were 15 × 15 grids in Figure 5 (Experiment 1). Error bars show ±1 SE of the means across sessions.

Does the pattern shown after the test pattern matter? We ran a brief control in which the offset of the test pattern was followed immediately by a gray pattern (that is 0% contrast) rather than showing the 50% adapt pattern again. While there are some differences in detail, performance remains poorest for the Straddle test pattern (see Supplementary Figure 1). 

Does the retinal location of the pattern matter? To check for possible differences in foveal versus peripheral processing, we ran a brief control on two observers. We presented two identical 2 × 2 grids (subtending 2 × 2 deg), one to the left and one to the right of fixation. The inner edges of the two grids were at 1, 3, or 5 degrees from fixation. A letter identification task at fixation was used to check for adequate control by the observer of eye movements. The bad performance on the Straddle test pattern that occurs for foveal fixation also occurs for these near-periphery patterns (see Supplementary Figure 2). 

Explanation based on size of transient does not work for the straddle effect

Consider performance on the three test patterns plotted at the center of Figure 4 (those with an average test contrast of 45%, 50%, and 55%). The test contrast difference is the same for all of these patterns (that is, the positive difference between the two contrasts in the test pattern, ∣C2 − C1∣, is always 10%). However, another contrast difference is not: the change from the adapt pattern contrast to the test pattern contrasts is not the same for all of these patterns. We will call this change the transient contrast difference. 

For the center test pattern (average test contrast of 50%), the absolute value of the difference between the adapt contrast and C1, ∣50% − 45%∣, is 5%. The absolute value of the difference between the adapt contrast and C2, ∣50% − 55%∣, is also 5%. The transient is the larger of these: 5%. 

However, the transient for the test patterns just to the left and right of the center point in Figure 4 is 10%. This is calculated as follows for the just-to-the-right pattern. That pattern has C1 = 50% and C2 = 60%. The difference between the adapt contrast and C1 is 0%. The difference between the adapt contrast and C2 is 10%. The transient is the larger of these two: 10%. 

The transient goes in the same direction as our results. That is, the transient is smallest on the test pattern with the poorest performance (in Figure 4 the test pattern with an average test contrast of 50% has a transient of 5%). The transient is always smallest on the Straddle test pattern. The transient increases to the left and right of the Straddle pattern (for the test patterns with average test contrasts of 45% or 55%, the transient is 10%) and performance also increases. 

While the transient does seem to correlate with the general direction of our results, it cannot account for the full set of results. Figure 7 plots many “constant‐transient trios”. That is, for each panel in Figure 7, the size of the transient is the same for all three patterns (three bars). If the size of the transient accounted for performance, then each bar in a trio should be the same height. The test pattern contrasts (C1, C2) are shown along the horizontal axis (as are the test contrast differences and the transients). 

Figure 7

 

Constant-transient trio results. The transient (that is, the maximum difference between the adapt contrast and the test contrasts) is the same throughout this figure. Each panel has a trio of results (from a particular observer with a particular adapt contrast). The open bars on the left of each trio were collected with a test contrast difference of 10% while the filled bar on the right of each trio was collected with a test contrast difference of 20%. The transient was 10% for all the bars. The size of the grid was 2 × 2. Different observers are shown in different rows and different adapt contrasts in different columns. If the transient determined the observers' ability to identify the orientation of the stripes in the test pattern, then performance within a trio should be equal. Clearly this is not the case. These data are from Experiment 2. Error bars show ±1 SE of the means across sessions.

In each trio (panel in Figure 7), the two open bars are for Below and Above test patterns with a 10% test contrast difference. These correspond to the two points in Figure 6 just to the left and just to the right of the arrow. The filled bar in each trio is for the corresponding Straddle test pattern with a 20% test contrast difference. (These are not the Straddle patterns in Figure 6.) Performance is not the same for each test pattern in a trio, and the poorest performance is actually for the test pattern with the 20% test contrast difference. These data were collected with a 2 × 2 grid, but the same effect is seen with a 15 × 15 grid (Wolfson & Graham, 2007a). 

Explanations based on a shifting monotonic transducer do not work for the straddle effect

The constant-transient trio results in Figure 7 also rule out another possibility. As is widely known, traditional pedestal effects in psychophysical results can be explained by accelerating nonlinearities (e.g., Bex, Mareschal, & Dakin, 2007; Foley & Legge, 1981; Nachmias & Sansbury, 1974; Solomon, Watson, & Morgan, 1999; Stromeyer & Klein, 1974). People often suggest to us that a shifting monotonic transducer that shows acceleration for both increases and decreases from the comparison level—that is, a monotonic function with a flatter portion near the comparison level than on either side (see example in Figure A1)—can explain our results. While such a function can indeed account for the results in Figures 4–6, it is ruled out by the constant-transient trio results in Figure 7. This argument is presented in greater detail in 13. 

We have considered many other possible explanations that did not work. The two just discussed (the size of the transient; shifting monotonic transducer) are simply the ones most often suggested to us. 

An explanation that does work for the straddle effect

A process like that sketched in Figure 2 can account for the straddle effect seen in Figures 4– 7. The input to the process is the local contrast (e.g., the contrast of a Gabor patch). This process shifts so that the contrast‐comparison level is always at the recently experienced contrast (the adapt contrast in our experiments). So, for example, if the adapt contrast is 50% and the contrast of some Gabor patch in the test pattern is 50%, the output of the process is 0. However, if some Gabor patch in the test pattern has 40% contrast, and another Gabor patch in the test pattern has 60% contrast (and the adapt contrast is still 50%), the output from the process will be the same for both of these Gabor patches. In words, the output of the process is an unsigned measure of the difference between the current contrast (e.g., the contrast of a Gabor patch in the test pattern) and the recent average contrast (e.g., the contrast of a Gabor patch in the adapt pattern). 

One way to think of this contrast‐comparison process is as being embedded in complex channels (also called second-order channels or a filter–rectify–filter model). Such a channel consists of a layer of linear filters (often spatial frequency and orientation selective), a rectification, and another layer of linear filters (usually at a lower spatial frequency than the first layer of filters; Sutter, Sperling, & Chubb, 1995). See, e.g., Kingdom, Prins, and Hayes (2003) and Landy and Graham (2003) for discussions of such models. The contrast‐comparison process can be inserted into such channels so that outputs after the contrast‐comparison process are the inputs to the final layer of linear spatial filtering. To be explicit: in this paper, we are not making predictions about a simple contrast increment or decrement in, for example, a sinusoidal grating or a single Gabor patch at all. We are only discussing the identification of second-order (contrast-defined) patterns. (See Baker & Mareschal, 2001, and introduction to Schofield & Georgeson, 1999, for nice reviews of second-order vision.) 

The function shown in Figure 2 is full-wave rectification. This causes the sign of a contrast change to be completely lost. For example, if the recent average contrast is 40%, the output of the process to a test‐pattern Gabor patch of 0% contrast will be the same as the output to a test‐pattern Gabor patch of 80% contrast. If this were literally true, observers would never be able to see Straddle test patterns. This is not in fact the case. What is, in fact, sufficient to explain the observed results is a pair of otherwise‐identical channels with asymmetric rectification functions (one favoring contrast increments and one favoring contrast decrements). The amount of asymmetry in the rectification function depends on the individual observer and can be estimated from the data. However, for the verbal descriptions in this paper, the full-wave approximation is adequate. 

Also, the function shown in Figure 2 is a piecewise-linear rectification. With a straightforward version of this kind of explanation, the dip in performance near the adapt contrast would be expected to include only those test patterns having contrasts that do straddle the adapt contrast. Once a test pattern's contrasts are outside this range, performance should be at a uniformly high level. However, exceptions are regularly found for test patterns that do not quite straddle the adapt contrast. For example, in Figure 6 consider the test pattern composed of contrasts 65% and 75% (average test contrast of 70%) after adaptation to 65% (blue data). Performance on it is not as good as performance on the test pattern composed of 70% and 80% (average test contrast of 75%) except for cases where performance is already effectively at ceiling for the method's ability to measure (up very near 100%). If one makes the rectification an expansive power function (that is, makes the function in Figure 2 U-shaped rather than V-shaped), then the process will predict a broader straddle effect. (This is essentially identical to the demonstration in Graham & Sutter, 2000, for expansive nonlinearities at the rectification after adaptation to 0% contrast.) However for the verbal descriptions in this paper, continuing with a piecewise-linear rectification is adequate. 

Many implicit assumptions occur in this brief description we have given of an explanation that might work. For example, this is a deterministic explanation and thus cannot explain why the observer gives different responses on different trials. Roughly, characteristics of a function that appears in a deterministic model can appear in a probabilistic version of that model either in assumptions about the mean or in assumptions about the variance of random variable distributions. (Under most assumptions about human behavior, it is the ratio of mean to variance that matters). 

There are, of course, bound to be entirely different kinds of explanations that might work for the results here. For example, one could postulate quite complex and higher level processes that are calculating much more intelligently about the probable overall perception of what is in the visual field. For the present, however, we find this a satisfactory level of modeling as it gives us some conceptual grasp of the phenomena and some hints about what might be looked for in neurophysiological experiments. 

Full range of adapt and average test contrasts (Experiments 3–6)

So far we have only presented data from middle adapt and average test contrasts, demonstrating the effect of the shifting, rectifying contrast‐comparison process (the “new” process). We will now move on to the full range of adapt and average test contrasts, starting with results that demonstrate contrast normalization (the “old” process). 

Adaptation to 0% contrast (uniform gray screen)

As stated in the Introduction section, we have previously collected many sets of results with patterns like those in this paper using an adapt contrast of 0% (e.g., Wolfson & Graham, 2005). Some recently collected results are shown in Figure 8. These were collected using the exact same patterns as the experiments described in the previous section (with 2 × 2 grids). These results are typical: after adapting to 0% contrast, performance declines as average test contrast increases. The absolute performance level varies from observer to observer, so we have drawn the average performance with a thick gray line to show the shape. 

Figure 8

 

Percent correct identification of the orientation (of the contrast-defined stripes) in the test pattern after adapting to 0% contrast (a blank gray screen). The thick gray line shows the average performance across observers. These data are from Experiments 3 and 4. Error bars show ±1 SE of the means across sessions. Each observer's data is plotted separately in Supplementary Figures 3–8.

We think of the results in Figure 8 as showing Weber-law-like behavior in the sense that, as average test contrast increases, the observer's performance declines in the same way that the Weber ratio for these test patterns declines. The Weber ratio is  

$w=(\mathrm{C}1+\Delta \mathrm{C})/\mathrm{C}1=\mathrm{C}2/\mathrm{C}1,$

(1)

where C1 is the lower of the two contrasts in the test pattern, C2 is the other test contrast, and ΔC is the test contrast difference. Suppose ΔC is always 10% (as is the case in Figure 8). For an average test contrast of 10%, C1 = 5%, and thus w = (5% + 10%) / 5% = 3. For an average test contrast of 20%, C1 = 15%, and thus w = (15% + 10%) / 15% = 1.7. For an average test contrast of 30%, C1 = 25%, and thus w = (25% + 10%) / 25% = 1.4. And so on. We have previously shown that, after adaptation to 0% contrast, results for test contrasts far away from the adapt contrast depend on this Weber ratio no matter what the actual values of the two test contrasts are (e.g., Graham & Sutter, 2000). We will revisit this idea after presenting some other results. 

Adaptation to 50% contrast with the full range of average test contrasts

We already looked at a restricted range of test contrasts with a 50% adapt contrast (e.g., green points, Figure 6). Performance is poor on Straddle test patterns (that is, when the average test contrast equals the adapt contrast). As the average test contrast decreases or increases, the test pattern becomes a Below or Above test pattern and performance improves. 

What happens to performance on Far Below and Far Above test patterns after adapting to 50% contrast? As shown in Figure 9, performance declines at the far ends of the curves. The generally symmetric shape of the full curve is somewhat reminiscent of the two wings of a butterfly, and we will refer to it as a “butterfly curve”. (For a previous use of this term in a very different perceptual adaptation context, see Hochberg, 1978, p. 240. In his example the observer has adapted to bath water at body temperature. That temperature is neutral. Slightly higher and slightly lower temperatures are pleasant while far higher and far lower temperatures are unpleasant. Plotting temperature versus pleasantness yields a butterfly curve.) 

Figure 9

 

Results from a wide range of test contrasts with a 50% adapt contrast. (Upper panel) These results were collected under the same conditions as those plotted in green in Figure 6, but here we have used many more test contrasts. These curves are shaped like “butterfly curves”. We suspect that (i) the drop at the center of the butterfly curves is due to the shifting, rectifying contrast‐comparison process, and (ii) the drop on the outside of the butterfly wings is due to contrast normalization. Error bars show ±1 SE of the means across sessions. These data are from Experiment 5. (Lower panel) Same as upper panel except that patterns were 15 × 15 grids of Gabor patch elements rather than 2 × 2 grids. These data are from Experiment 6.

The results in the upper panel of Figure 9 were collected using the small patterns (2 × 2 grid of Gabor patch elements). We also ran this same experiment with the big patterns (15 × 15 grid of the same elements), and these data are shown in the lower panel. The curves in the upper and lower panels are a bit different. The differences are worth exploring, and we are doing so, but for the purposes of this paper, the differences are negligible: All of these curves are butterfly shaped. 

The drop in performance on the outside edges of the butterfly wings shows Weber‐law-like behavior (as the drop in performance in Figure 8 did). The points to the right of the green arrow in Figure 9 are equivalent to the points to the right of the gold arrow in Figure 8 if we consider average test contrast with respect to the adapt contrast. That is, the Weber-like behavior seems to move with adapt contrast. The results in Figure 9 are also (fairly) left–right symmetric, indicating that it is the unsigned distance from the adapt contrast that matters. 

In the prior section we calculated a Weber ratio w with respect to actual contrast. That worked fine for results collected with a 0% adapt contrast. To account for nonzero adapt contrasts, we need to calculate the ratio using the unsigned differences between test contrasts and adapt contrast rather than using the test contrasts themselves. Note that we only define this for test patterns in which both the test contrasts are on the same side of the adapt contrast. Then, we define  

$w^{*}=(|\mathrm{C}1-\mathrm{A}|+\Delta \mathrm{C})/|\mathrm{C}1-\mathrm{A}|=|\mathrm{C}2-\mathrm{A}|/|\mathrm{C}1-\mathrm{A}|,$

(2)

where A is the adapt contrast, C1 is the contrast in the test pattern that is closest to the adapt contrast, C2 is the other test contrast, and ΔC is the test contrast difference. In words, we will say that w* equals the “ratio of the two test contrasts (with respect to the adapt contrast)”. In the case of a 0% adapt contrast, w* and w are equivalent. In the case of nonzero adapt contrasts, w* declines as the average test contrast increases or decreases from the adapt contrast, as do the outside edges of the butterfly wings. 

Two test contrast differences compared

The data in Figure 9 (and all the prior line plots) were collected with a test contrast difference of 10%. However, other test contrast differences are possible of course. Figure 10 shows data collected with a 10% test contrast difference (brighter green, same data as shown in Figure 9, upper panel) and data collected with a 5% test contrast difference (darker green). Plotted against average test contrast (left panel), the 5% test contrast difference data look very much like a scaled version of the 10% test contrast difference data. (There are ceiling effects near 100% correct performance flattening out the peaks in the 10% curve, and there are “basement effects” near 50% contrast flattening out the tails of the 5% curve. The overall symmetry of the curve is clear in both cases.) This seems reasonable since the 5% test contrast difference condition is just a harder version of the 10% test contrast difference condition. 

Figure 10

 

Results collected with two test contrast differences (10% in brighter green and 5% in darker green) plotted against two x-axes: at left against average test contrast and at right against contrast‐ratio angle. Error bars show ±1 SE of the means across sessions. (The results for a test contrast difference of 10% were plotted before in Figure 9, upper panel.) The transformation from test contrasts to contrast‐ratio angle is explained in 14. These data are from Experiment 5.

The data in the left panel of Figure 10 are replotted in the right panel with a different x-axis. The x-axis in the right panel is contrast‐ratio angle. Informally, this axis expands the results curves in the center and compresses the curves on the ends. Pictorially, the butterfly wings have become pointed, more like bat wings. We have often found it useful to plot results against contrast‐ratio angle (e.g., Graham & Sutter, 2000). Our prior results plotted against contrast‐ratio angle were collected with an adapt contrast of 0%, so the adapt contrast was effectively ignored. With the current results, adapt contrast is critical and the angle is calculated relative to the adapt contrast. 14 describes how contrast‐ratio angle is formally defined and calculated. 

Plotting multiple test contrast differences against contrast‐ratio angle elucidates some aspects of the data. As seen in the right panel of Figure 10, results collected with the two test contrast differences (brighter green 10%, darker green 5%) fall on top of one another on the outside edges of the butterfly's wings. This means that, for this portion of the curve, performance is determined by the contrast‐ratio angle, which itself is determined by w*, the ratio of the two test contrasts (with respect to the adapt contrast). (If two test patterns have the same contrast‐ratio angle, then they also have the same w*.) This is just another way of saying that this behavior is Weber-like and is consistent with the action of contrast normalization. Clearly, in the center of the plot, contrast‐ratio angle does not make the results coincide. That the points do not coincide shows that this behavior is not Weber-like. 

Full range of adapt contrast levels

Figure 11 shows results (from only one observer, others will be shown in the next figures) collected with the full range of adapt contrasts. Adapt contrast increases from the top panel (0%, that is, a blank gray field) to the bottom panel (100%). As adapt contrast increases, the butterfly curve shifts from left to right. At the center of the butterfly is a dip in performance. This dip occurs at Straddle test patterns, that is, when the average test contrast equals the adapt contrast. Performance improves dramatically just to the left and right of the Straddle test patterns (Below and Above test patterns). These are the inside edges of the butterfly wings. Further to the left and right (Far Below and Far Above test patterns) performance drops again. These are the outer edges of the butterfly wings. 

Figure 11

 

Results, for a single observer, with a full range of adapt contrasts (0%, 25%, 50%, 75%, 100%) and a full range of average test contrasts. The middle curve (50% adapt contrast, shown in green) looks like a butterfly curve. The other curves look (generally) like butterfly curves shifted to be centered at the adapt contrast. (The adapt contrast in each panel is indicated by the small vertical arrow.) Error bars show ±1 SE of the means across sessions. Results from more observers are shown in Figure 12. These data are from Experiment 3.

As mentioned before, the results are not exactly symmetrical around the adapt contrast. There is frequently worse performance to the right of the adapt contrast than to the left. 

One can also see in the results of Figure 11, a second detail: the dip in performance at the adapt contrast gets wider as the adapt contrast increases. 

Such details of the results can be ignored in this paper since the effects of interest here are so massive in comparison. However, these details can be informative going forward and are discussed briefly below in the Explaining details of the results in the full range section. 

Figure 12 shows the same data as Figure 11 plus data from other observers (plus an additional experiment). Individual observers differ from one another in some ways, but they clearly show butterfly curves moving with adapt contrast. (The data in Figure 12 are plotted in Figure A4 against contrast‐ratio angle. Individual curves from Figure 12 are plotted separately in Supplementary Figures 3– 8.) 

Figure 12

 

Results with a full range of adapt contrasts for many observers. Results from Figure 11 are plotted here along with other observers' results in Experiment 3 (adapt contrasts of 0%, 25%, 50%, 75%, 100%). Also plotted are the results from Experiment 4 with a different set of adapt contrasts (0%, 10%, 50%). Error bars show ±1 SE of the means across sessions. These curves are shown plotted against contrast‐ratio angle in Figure A4. Supplementary Figures 3– 8 show each observer's data plotted separately against average test contrast.

We can summarize these results as follows: there is a region of peak performance that moves from left to right (with increasing adapt contrast), and that region of peak performance has a notch in it at the adapt contrast. 

A subset of the results from Figure 12 is replotted in an alternate way in Figure 13. Results are shown for three observers (one per panel) at five adapt contrasts (different colors). On the horizontal axis is the average test contrast minus adapt contrast. This transformation amounts to shifting each curve until its adapt contrast is at zero. For each observer, the results juxtapose well across adapt contrasts. The butterfly curve shape is quite apparent. 

Figure 13

 

Results plotted relative to the adapt contrast. These results (from Experiment 3) were also plotted in Figure 12. Error bars show ±1 SE of the means across sessions. The results look like butterfly curves centered on the adapt contrast. Zero marks the point at which the average test contrast equals the adapt contrast. Performance is best for test patterns with an average test contrast near, but not at, the adapt contrast.

Buffy and Weber zones

Figure 14 shows an idealized representation of the results in Figure 13 (a smoothed, symmetrical average of those results). The horizontal axis shows average test contrast relative to the recently experienced contrast. The vertical axis shows performance. 

Figure 14

 

The curve shows an idealized representation of the results in Figure 13. The curve is centered at the comparison level (that is, at the recently experienced contrast). Two “zones” are shown. Weber zone: The drop in performance at the ends of the curve shows Weber-law-like behavior, an effect that we strongly suspect is the result of contrast normalization (see text for explanation). Buffy zone: The drop in performance in the center—the Straddle effect—we suspect is the result of a shifting, rectifying contrast‐comparison process (we previously nicknamed “Buffy”). At top are italicized labels indicating the test patterns. Performance is poor on Straddle test patterns, improves for Below and Above test patterns, and drops for Far Below and Far Above patterns.

We think the butterfly shape arises from the interaction of two different processes which both operate throughout the full stimulus range but have particularly prominent effects in different zones. The center of the curve—called the Buffy zone in Figure 14—shows the straddle effect that, as was discussed above, can be explained by a contrast‐comparison process (and not by some other ways). The comparison level (indicated by the pink arrow) is set by the recently experienced contrast and therefore is equal to the adapt contrast (at least if the adapt contrast has been on for 1 s as it was in the experiments reported here). 

If there were only the contrast‐comparison process, the tails of the curves would be predicted to be horizontal lines, showing constant high performance. Instead performance declines in both directions away from the center, requiring that the explanation contain a second process, a process that produces Weber-law-like behavior for both tails (without disturbing the straddle effect shown in the center). Due to the combined action of these two processes, performance is best for test patterns with an average test contrast near, but not at, the adapt contrast. 

More details of a process that can explain the behavior in the Weber zones are given below. However, first we mention an explanation that does not work for behavior in the Weber zones. 

Explanations based on a transducer function (e.g., Fechner) do not work for the Weber zones

In previous work we showed that the Weber-law-like behavior after adaptation to 0% contrast (e.g., Figure 8) cannot be explained in a Fechnerian sort of way. For example, if the compressive function of contrast at each spatial position were a logarithmic function, the differential response to two patches would be precisely determined by the ratio of their contrasts, decreasing as that ratio decreased. However, we ruled out this possible explanation in a body of experiments done some years ago (Graham & Sutter, 1998, 2000). Briefly, the argument involved comparing results from test patterns in which all the Gabor patches are of the same size (as in the experiments described here) with results from other test patterns composed of two different sizes of Gabor patches (always after adaptation to 0% contrast). This argument rules out not only monotonic but also nonmonotonic transducer explanations. While we have not repeated these experiment for different adapt contrasts, it seems unlikely that the Weber zones at adapt contrasts other than 0% would be explained by a totally different process than at 0%. 

Explanation that works to explain the results in the Weber zones

We have previously shown that the behavior in the Weber zone after adaptation to 0% contrast (Graham & Sutter, 2000) is consistent with a contrast‐gain‐control process of the normalization type. The normalization pool in this explanation contained both simple (linear) and complex (second-order) spatial-frequency and orientation-selective channels. The intuition is easy for the experiment shown in Figure 8 here, in which the test contrast difference is always the same (and the adapt contrast is 0%). As the contrasts of the Gabor patch elements increase, the strength of the normalization pool increases. It increases because there are many channels in that pool that do respond in proportion to the patch contrasts (although they cannot do the perceptual task and are not responsive to the test contrast difference). On the other hand, the response of the complex (second-order) channels that can do the task (because they are “tuned” to the test pattern) does not increase with patch contrast but is fairly constant through a constant difference series of patterns (when the adapt contrast is 0%). The observer's performance is a function of the response of the channels that can do the task, divided by the response of the full normalization pool. Thus performance is predicted to decline as average test contrast increases (as seen in the results in Figure 8). As mentioned in the Introduction section, a number of investigators in addition to us have successfully used this concept to predict behavior for a wide variety of visual patterns and visual tasks. 

To explain the results here where the adapt contrast is not necessarily 0%, one can include the contrast‐comparison process in the model of Graham and Sutter (2000). Such a modified model predicts, at least qualitatively, the Weber-law-like behavior at both ends of the curves plotting performance versus average test contrast. To see this, compare the predictions in the lower right panel of Figure 14 in Graham and Sutter (2000) to the results plotted here against contrast‐ratio angle in Figures 10 (right panel) and A4. One can legitimately make this comparison for the following reason. The predictions of the modified model in some cases are formally identical to those of the original model once each test contrast value (C1 and C2) in the original model has been replaced by the unsigned difference between that test contrast and adapt contrast (∣C1 − A∣ and ∣C2 − A∣, respectively) in the modified model. The contrast‐ratio angle plot described in 14 incorporate this replacement, which is why one can compare the original model predictions to the experimental results here. 

Explaining details of the results in the full range

Two more subtle details in the experimental results were mentioned above (the widening of the dip with increasing adapt contrast; and the left–right asymmetry of individual curves, see discussion of Figure 11). Preliminary modeling has shown that one way to predict these two effects is to allow the normalization pool to include not only outputs of channels containing a contrast‐comparison process but also some outputs of channels without a contrast‐comparison process (channels that have outputs that are monotonic with contrast from 0% contrast). This modification interacts with the exact form of the rectification function (to what degree it is U-shaped vs. V-shaped), which was discussed previously. (It was discussed at the end of An explanation that does work for the straddle effect section as an explanation for why the dip in the results contains some test patterns that do not, in fact, quite straddle the adapt contrast.) 

General discussion

The curve in Figure 14 summarizes our results well (compare to Figure 13). And we even think we know something about the underlying contrast adaptation processes as indicated by the zones: At the center performance reflects the shifting, rectifying contrast‐comparison process, and at both ends performance reflects contrast normalization. However, we are puzzled about the combined function of these processes. 

If the function of adaptation is to move the system's operating range (along some dimension, in our case contrast) to be in the range of values currently in the visual world, one might expect performance to peak when the test contrasts are very near the adapt contrast. However, the point at which the average test contrast equals the adapt contrast (the Straddle test pattern) has very poor performance. 

On the other hand, if the function of adaptation is to highlight novel or new aspects of the visual world, one might expect performance to improve as the test contrasts increase or decrease from the adapt contrasts. However, the most extreme examples of this, the Far Above and Far Below test patterns, show very poor performance. 

We can think of these results in a slightly different way that suggests that the combined action of the two processes is beneficial. Consider the following. After adapting to 0% contrast, it is hard to identify the orientation of contrast-defined stripes for a large range of contrasts (see Figure 8). However, performance can be dramatically improved by adapting to some other contrast for a second. This can be seen by looking, for example, at Figure 11. Draw a vertical line through the whole figure at 25% average test contrast: After adapting to 0% contrast, the observer's performance is very poor (top panel), and after adapting to 25% contrast performance may be even worse (second to the top panel), but after adapting to 50% contrast, performance is near perfect (middle panel). 

But we are still puzzled. In particular, we wonder why performance is so poor on Straddle test patterns. Could this be desirable in some real-world situation? Or does this not arise in real-world situations? Or is this a side effect of, for example, some “contrast conservation” mechanism (Fiser, Bex, & Makous, 2003)? Or does this somehow increase efficiency? 

Speed of the contrast‐comparison process

The contrast‐comparison process is fairly fast: it shifts in less than 250 ms (Graham & Wolfson, 2007). Some other processes that are primarily dependent upon (can be predicted by) contrast rather than luminance are also fairly fast (e.g., Wilson & Kim, 1998). Analysis of natural images has shown that the local image contrast at one fixation has little correlation with the local image contrast at the next fixation (Frazor & Geisler, 2006; Mante, Frazor, Bonin, Geisler, & Carandini, 2005). Thus having contrast-controlled processes that reset within the duration of a typical eye fixation seems reasonable. 

Why has no one reported the straddle effect before?

At first, it seemed odd to us that nobody had previously reported the poor performance on straddle patterns since this effect of contrast adaptation is neither small nor subtle. There have been hundreds, probably thousands, of adaptation and masking studies in which the observer is first adapted to one pattern and then asked to report some aspect of a test pattern's perceived appearance. Many of these studies (like those of Blakemore & Campbell, 1969; Pantle & Sekular, 1968) are ones in which a simple adapting pattern like a sinusoidal grating or line is followed by a simple test pattern in the same place in the visual field, and the observer has to answer a simple question, for example, “Is the grating visible or not?” or “What orientation is the line?”. This general design is analyzed in Chapter 2 of Graham (1989), with many studies using this design given in the reference list of that chapter. There are also hundreds if not thousands of studies in which a simple test pattern and simple mask pattern are on simultaneously (e.g., Henning, Hertz, & Broadbent, 1975; Legge & Foley, 1980; Nachmias, 1993; Stromeyer & Julesz, 1972). There are also experiments in which both a preceding adapt and a simultaneous mask stimulus were studied in a contrast-discrimination-after-adaptation paradigm (e.g., Foley & Chen, 1997; Ross & Speed, 1991). In addition, there are a very large number of studies in which a simple adapt (or mask) pattern does not appear in the same location as the test pattern but close to it (e.g., Petrov, Carandini, & McKee, 2005; Polat & Sagi, 1993; Rogowitz, 1983). In addition to experiments using relatively simple patterns like gratings or lines, there are at least hundreds of adaptation and masking experiments using overlapping and nonoverlapping spatial patterns with much more complicated patterns and tasks (e.g., Anderson & Wilson, 2005; Bex et al., 2007; Clifford & Rhodes, 2005; Fiser & Fine, 2000; Pelli, Palomares, & Majaj, 2002; Webster & Miyahara, 1997). 

Many of these adaptation and masking experiments produced results showing Weber-law-like behavior, and these results have been explained by divisive contrast-gain-control processes. 

So why have none of these experiments produced results showing the straddle effect? Let us look again at the experiment ( Figure 3) in which the straddle effect is seen. The patterns are relatively simple: gray fields, grids of identical Gabor patches, and grids of two different contrasts of Gabor patches arranged in rows or columns. The task is also relatively simple: identification of the orientation of the contrast-defined stripes in the test patterns. 

We suspect our experiments have several characteristics that have not been studied very often, and that have rarely, if ever, been studied in this combination:

•  
  The observer's task depends on the spatial arrangement of two different contrast levels in the test pattern that can vary independently of the contrast in the adapt pattern;
•  
  The observer's response is based on the contrast values in the test pattern, and performance is studied as a function of both test contrast values and of adapt contrast values;
•  
  And, in particular, the contrasts studied include cases where the contrasts in the test pattern straddle the contrast of the adapt pattern.

Many experiments in the last 25 years or so have the first characteristic, in particular, many of the studies in which the tasks and stimuli are called second-order do. These are the experiments that can be explained by the original complex (second-order) channels; see references near Figures 10 and 11. And, by now, many experiments have studied the effects of adaptation or masking on second-order tasks (e.g., Dosher & Lu, 2006; Kingdom et al., 2003; Larsson, Landy, & Heeger, 2006; Oruç, Landy, & Pelli, 2006). However, we know of no such experiment that has also varied contrast systematically in the way that would be necessary to discover the poor performance on straddle test patterns. Thus perhaps the new effect has never been seen in experimental results simply because the conditions necessary to produce it have never been studied in the laboratory. 

The closest previous literature we know of. In our search to find any such studies, we recently discovered one study (Kachinsky, Smith, & Pokorny, 2003), the results of which may at least partially reflect the process producing the straddle effect in our results. The patterns used by Kachinsky et al. (2003) were composed of regions of different luminances (homogeneous squares set in a homogeneous background) rather than regions of different contrasts. Thus, their task might be done by linear (first-order, simple) orientation- and spatial-frequency-selective channels. However, the contrast at the edge of any of their homogeneous squares can be considered as analogous to the contrast of a Gabor patch. (In other words, their squares can be considered to be very crude Gabor patches containing much broader bands of spatial frequencies and orientations than ordinary Gabor patches.) When their results are viewed from this point of view and plotted in the way we plot our results, their results do show poor performance on the straddle test patterns. Thus it is possible that the new adaptation process proposed here could explain their experimental results, although they discuss their experiment in a seemingly different way as manifestations of inferred separate magnocellular and parvocellular pathways. 

Another study (Zenger-Landolt & Koch, 2001) was pointed out to us by a reviewer. This study is one in which mask and test stimuli were simultaneously present (as opposed to having an adapt stimulus precede a test stimulus). This study can be seen as an analog of the ones reported here, and the two sets of results bear some similarity. The observer is asked to say which of the two Gabor patches (one to the left and one to the right of fixation) has the higher contrast. A “pedestal” contrast is present in both of these patches on every trial and an increment contrast is presented randomly on one patch or the other. Consider these two patches to be the analog to our test pattern. There are further patches flanking each of the two patches in the test pattern. Consider these flanking patches to be the analog to our adapt pattern. Rather than measuring performance for a given size increment contrast, their figures plot the threshold contrast difference so bad performance produces higher points in their figures. Were their results analogous to ours, and were their results plotted as ours were (performance for a given increment contrast versus the average of the two contrasts in the two Gabor patches in the test pattern), their curves should have a butterfly shape. Their results are not plotted in this fashion although in a related fashion (thresholds instead of performance on the vertical axis, and pedestal contrast instead of the average of the contrasts in the two test patches on the horizontal axis). And their results do show an approximate butterfly shape (or inverted‐W shape in the manner they plot them). The details are different: the butterfly curve is not centered at the contrast of the flanking patches (which would be the analog of the adapt contrast); and their butterfly curves are much less symmetric around their center than the ones from our experiments. Their predictions for these results are based on a rather different although potentially related kind of idea involving changing contributions of divisive and subtractive inhibition. 

Neurophysiological substrate of these processes

Some fMRI BOLD responses in hV4 (but not V1, V2, V3) are positive to both increments and decrements in contrast (Gardner et al., 2005). This could produce a “confusion” of increments and decrements like that produced by the contrast‐comparison process. On the other hand, we suspect that the contrast‐comparison process occurs before some contrast normalization process. Yet we have always considered the normalization process demonstrated in this paper to have substrate in V1 or V2 (and neurons in V1 exhibit normalization; Carandini et al., 1997). So now we speculate that there might be two distinct layers of normalization, one in V1 and another at a higher level (reminiscent of the MT model of Simoncelli & Heeger, 1998). 

Summary

We have found evidence for a shifting, rectifying contrast‐comparison process that acts in concert with a contrast gain control of the normalization type. 

The contrast‐comparison process results in poor performance on Straddle test patterns ( Figure 4). That is, after adapting to a grid of identical Gabor patches for 1 s, observers are poor at identifying the orientation of contrast-defined stripes in a test pattern when the two Gabor patch contrasts in the test pattern straddle the adapt contrast. However, performance is good on Above and Below test patterns. The poor performance on Straddle test patterns relative to Above and Below test patterns (the straddle effect) cannot be explained by the size of the transient or by a shifting monotonic transducer. 

The contrast normalization process produces poor performance for Far Above and Far Below test patterns ( Figure 9). We refer to this behavior as Weber-law-like behavior. 

The full results curve is “butterfly shaped” and is centered on the adapt contrast ( Figure 13). This curve results from the interaction of the contrast‐comparison process and the contrast normalization process (labeled as Buffy and Weber zones in Figure 14). The consequence of the combined effect of the two processes is that performance is best for test patterns composed of contrasts near, but not at, the adapt contrast. 

Supplementary Materials

Supplementary Figure 1 - Supplementary Figure 1 

Supplementary Figure 1. Results collected with 0% contrast (gray) immediately following the test pattern (left column) and results collected with 50% contrast Gabor patches immediately following the test pattern (right column). In both conditions the adapt contrast was 50%. Error bars show ±1 SE of the means across sessions. Observers show a clear straddle effect in all cases except for BSG in the left column, where a straddle effect may be hidden by the strong ceiling effect. 

Supplementary Figure 2 - Supplementary Figure 2 

Supplementary Figure 2. Results collected in the near-periphery for two observers show the straddle effect. Adapt and test patterns were 2 × 2 grids of Gabor patches to the left and right of fixation. The inner edge of the grid was 1, 3, or 5 degrees eccentric (top row, middle row, bottom row). A fixation square was presented throughout the trial and a simple letter identification task (“x” or “o”) occurred in the fixation square at the same time the test patterns were shown in the near-periphery. Observers gave both responses on each trial and were provided with feedback on both responses. Plotted here is percent correct identification of the orientation (of the contrast-defined stripes) in the test patterns on trials in which the observer also correctly identified the letter at fixation. Error bars show ±1 SE of the means across sessions. Plots that include all trials look very much the same. Letter identification rates for CG (left column) at 1, 3, and 5 degrees eccentric were 83%, 87%, and 90%; for RK (right column) they were 86%, 86%, and 91%. 

Supplementary Figure 3 - Supplementary Figure 3 

Supplementary Figure 3. Results from observer NA in Experiment 3. Error bars show ±1 SE of the means across sessions. (Subset of the results plotted in Figure 12.) 

Supplementary Figure 4 - Supplementary Figure 4 

Supplementary Figure 4. Results from observer RK in Experiment 3. Error bars show ±1 SE of the means across sessions. (Subset of the results plotted in Figure 12. This figure is the same as Figure 4 of the main text but also provided here for completeness.) 

Supplementary Figure 5 - Supplementary Figure 5 

Supplementary Figure 5. Results from observer SYP in Experiment 3. Error bars show ±1 SE of the means across sessions. (Subset of the results plotted in Figure 12.) 

Supplementary Figure 6 - Supplementary Figure 6 

Supplementary Figure 6. Results from observer MM in Experiment 4. Error bars show ±1 SE of the means across sessions. (Subset of the results plotted in Figure 12.) 

Supplementary Figure 7 - Supplementary Figure 7 

Supplementary Figure 7. Results from observer RK in Experiment 4. Error bars show ±1 SE of the means across sessions. (Subset of the results plotted in Figure 12.) 

Supplementary Figure 8 - Supplementary Figure 8 

Supplementary Figure 8. Results from observer VR in Experiment 4. Error bars show ±1 SE of the means across sessions. (Subset of the results plotted in Figure 12.) 

Appendix

Appendix A1: Shifting monotonic transducer

As mentioned in the main text, people have often suggested to us that a shifting monotonic transducer function like that in Figure A1 could explain our results. This monotonic transducer (solid black curve in Figure A1) shifts to be centered on the contrast value to which the observer has recently adapted (assumed to be 50% here) and shows acceleration for both increases and decreases from the recent average contrast (so it has a flatter portion near the recent average contrast). The horizontal axis in this figure indicates the contrast values of individual Gabor patches in 3 test patterns each having a contrast difference of 10%: the Below pattern has contrasts (35%, 45%), the Straddle has contrasts (45%, 55%), and the Above has contrasts (55%, 65%), where the adapt contrast is 50%. The vertical axis shows the output R in response to each contrast. The difference between the outputs to the two contrasts in a given test pattern is labeled by the symbol dR. The magnitude of dR is represented by vertical arrows on the right side of the figure. If one assumes that the observer's performance is monotonic with dR, this explanation does indeed predict that the observer's performance on the Straddle test pattern (dR = 2) is much lower than that on the Above or Below test patterns (dR = 8). 

Figure A1

 

Diagram of the monotonic transducer explanation's successful prediction that performance should be worse on a Straddle test pattern than on an Above or Below test pattern if the test contrast difference is the same for all three patterns. The adapt contrast is 50% as marked by the green arrow.

However, once one has looked at the experimental results quantitatively, it is easy to demonstrate that any explanation based on a monotonic transducer applied locally (even a weakly monotonic one) cannot explain the results after all. A failed prediction from the transducer explanation is illustrated in Figure A2 and will be explained here. The horizontal axis of Figure A2 shows a trio of test patterns A, B, and C for which two conditions hold:

•  
  C's contrast range spans A's and B's contrast range;
•  
  A's and B's contrast ranges are either nonoverlapping or only overlap at one value.

Figure A2

 

Diagram of failed prediction of the monotonic (even if only weakly monotonic) transducer explanation. As indicated by the contrast values on the horizontal axis, patterns B, C, and A correspond to the Below, the Straddle, and the Above test patterns in the constant-transient trio corresponding to the middle column of Figure 7. The relative differential response values are predicted to be, for any weakly monotonic transducer, dR(C) = dR(A) + dR(B). So the best performance is predicted to occur on the Straddle pattern C, but in fact the observer's performance is worst there (middle column of Figure 7). The adapt contrast is 50% as marked by the green arrow.

First, although Figure A2 shows a particular function and particular values of contrast, it illustrates a general prediction that holds true for any weakly monotonic function and any trio of test patterns obeying conditions (i) and (ii). The predictions is: The differential response to pattern C will be larger than or equal to the sum of the differential responses to A and B. In symbols,  

$\mathrm{d}\mathrm{R}\left(\mathrm{C}\right)>=\mathrm{d}\mathrm{R}\left(\mathrm{A}\right)+\mathrm{d}\mathrm{R}\left(\mathrm{B}\right).$

(A1)

 

If one again assumes that the observer's performance is monotonic with the differential response dR, then the observer's performance on the Straddle pattern C is predicted to be greater than (or equal to) both the performance on the Below pattern B and to the performance on the Above pattern A. In fact, if we had a suitable metric of performance that was linearly proportional to the differential response dR, then by that metric the performance on C is predicted to be greater than or equal to the sum of the performances on B and A. 

The particular contrasts used on the horizontal axis in Figure A2 are the contrasts for the constant-transient trio of patterns in the middle column of Figure 7 showing experimental results. As we just went through, the monotonic transducer explanation predicts that performance on the Straddle test pattern (C) should be the best of all three. However, in the experimental results shown in Figure 7, performance on the Straddle test pattern is generally the worst of all three (except when ceiling effects hide any differences). This prediction from the monotonic transducer explanation thus fails. 

Appendix A2: Contrast‐ratio angle

Informally, transforming from test contrasts to contrast‐ratio angle amounts to:

•  
  shifting the data so that the adapt contrast is at 0,
•  
  expanding the x-axis near the adapt contrast, and
•  
  compressing the x-axis far from the adapt contrast.

This can be seen by comparing the left and right panels in Figure 10.

The underlying “contrast space” is shown in Figure A3. Test contrast 1 (C1) is shown in black on the lower x-axis and test contrast 2 (C2) on the right y-axis. Contrast‐ratio angle is shown in brown on the upper x-axis and left y-axis. The diagram is centered on the adapt contrast (A). Every possible point on the diagram represents a possible test pattern; illustrated with red dots are a particular set of test patterns. The lower right half of the space has been grayed out; in this part of the space, C1 and C2 have been swapped. In experiments we, of course, use this part of the space, but we can ignore it for this discussion. 

Figure A3

 

Relationship between test contrasts (right y-axis and lower x-axis in black) and contrast‐ratio angle (left y-axis and upper x-axis in brown). This “contrast space” is centered at the adapt contrast (A). Contrast‐ratio angle is determined by the ratio of the two contrasts in the test pattern relative to the adapt contrast. Along any brown line through the origin, contrast‐ratio angle is the same. Along any right diagonal (such as the red dots), the test contrast difference is the same. (The numbered dots are discussed in the appendix text.) The lower right half of the space has been grayed out for explanatory purposes.

Suppose the adapt contrast (A) is 50% and the test constant difference (Δ) is 10% in Figure A3. The test pattern marked with a 1 is at (A + 2Δ, A + 3Δ), which is the test pattern with contrasts (70%, 80%). The test pattern marked with a 2 is (50%, 60%). Indeed, all the red dots have the same test contrast difference. The line labeled 90 deg is a “degenerate” case in which the test contrast difference is 0 (so all the Gabor patches have the same contrast). The test pattern marked with a 2 has one test contrast equal to the adapt contrast (in this case C1 = A); all points on the lines labeled 45 deg and −45 deg have this property (and we have previously called them one element only). The test pattern marked with a 3 is a Straddle test pattern; all points on the line labeled 0 deg have this property (and we have previously called them opposite sign of contrast). 

All of the red dots in Figure A3 are “equally spaced” with respect to contrast. That is, the difference between test contrast 1 and test contrast 2 is always Δ = 10% in this example. This is not the case for contrast‐ratio angle. For example, all five points marked 4 are well within 30 deg of one another while the points marked 2 and 3 are 45 deg apart. This is why the results in the right panel of Figure 10 compared to the left panel are spread out at the center and compressed at the edges. 

In summary, any right-diagonal line in the space (such as the red dots) represents a set of patterns in which the test contrast difference is always the same:  

$|\mathrm{C}1-\mathrm{C}2|\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}.$

(A2)

Any line through the origin of the space (any brown line) represents a set of patterns in which the ratio of the test contrasts with respect to the adapt contrast is always the same:  

$(\mathrm{C}1-\mathrm{A})/(\mathrm{C}2-\mathrm{A})\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}.$

(A3)

 

Figure A4 shows the same results as Figure 12 but now plotted against contrast‐ratio angle. This makes some aspects of the data easier to see. Clearly at the outside edges of the butterfly wings, there is a massive drop in performance. There is also a drop in performance in the center at contrast‐ratio angle 0 deg, which is when the adapt contrast equals the average test contrast. We think the drop at the outside edges of wings is dominated by a contrast normalization process and the drop at the center is dominated by the contrast‐comparison process. 

Figure A4

 

The results from Figure 12 replotted against contrast‐ratio angle.

Acknowledgments

This work was supported in part by the National Eye Institute Grant EY08459. Some of the results were presented at the Vision Sciences Society annual meetings (Wolfson & Graham, 2007b; Wolfson, Graham, & Pan, 2008). We thank Stephanie Pan and Gauri Wable for their preliminary modeling work, our observers for their many hours of work, and the two reviewers for their extensive and helpful comments. 

Commercial relationships: none. 

Corresponding authors: S. Sabina Wolfson and Norma Graham. 

Email: [e-mail] and [e-mail]. 

Address: Department of Psychology, Columbia University, New York, NY 10027, USA.