How attention and contrast gain control interact to regulate lightness contrast and assimilation: A computational neural model

Abstract

Recent theories of lightness perception assume that lightness (perceived reflectance) is computed by a process that contrasts the target's luminance with that of one or more regions in its spatial surround. A challenge for any such theory is the phenomenon of lightness assimilation, which occurs when increasing the luminance of a surround region increases the target lightness: the opposite of contrast. Here contrast and assimilation are studied quantitatively in lightness matching experiments utilizing concentric disk-and-ring displays. Whether contrast or assimilation is seen depends on a number of factors including: the luminance relations of the target, surround, and background; surround size; and matching instructions. When assimilation occurs, it is always part of a larger pattern in which assimilation and contrast both occur over different ranges of surround luminance. These findings are quantitatively modeled by a theory that assumes lightness is computed from a weighted sum of responses of edge detector neurons in visual cortex. The magnitude of the neural response to an edge is regulated by a combination of contrast gain control acting between neighboring edge detectors and a top-down attentional gain control that selectively weights the response to stimulus edges according to their task relevance.

Introduction

A common assumption of many recent theories of lightness perception is that lightness (i.e., perceived surface reflectance) is computed by a process that contrasts the amount of light reflected from a given surface in a visual scene with the light reflected from one or more other surfaces in the scene. This spatial comparison is thought to involve neural calculations based on luminance ratios, although the underlying physiological mechanisms are still far from well understood. 

In the very simplest lightness displays, it has been known for a long time that the appearance of a target patch depends on the target's contrast with respect to its local surround. For example, a gray paper viewed against a white background looks darker than an identical paper viewed against a black background (Chevreul, 1839/1967; von Goethe, 1810/1970). In more complex displays, including natural scenes, it has been proposed more recently that the target luminance is compared with the luminances of surfaces that group with the target, either on the basis of Gestalt grouping principles—e.g., contrast polarity, shape, and belongingness—or because the target and comparison surfaces appear to share a common illuminant (Bressan, 2006a, 2006b; Gilchrist, 2006; Gilchrist et al., 1999; Gilchrist & Radonjić, 2010). 

Precisely because lightness computation is thought to depend on luminance contrast, a class of phenomena that challenges all existing lightness models is that of lightness assimilation, which occurs whenever the target lightness covaries in a positive way with the luminance of the local surround (Helson, 1963; Helson & Joy, 1962; Hong & Shevell, 2004a; Jameson & Hurvich, 1961; Shevell, 2003; Shevell & Cao, 2003): the opposite of contrast. Although assimilation has been studied since at least the 19th century (von Bezold, 1876), it has no generally accepted explanation; nor have the properties of assimilation been explored in the same detail as those of contrast. 

Assimilation has recently received renewed interest through studies of White's illusion (Spehar, Gilchrist, & Arend, 1995; White, 1979, 1981), where it been suggested that assimilation depends critically on the existence of T-junctions that produce a perception of figure–ground scission in depth (Anderson, 1997; Ross & Pessoa, 2000; Todorović, 1997). However, this explanation has been cast in some doubt by demonstrations that assimilation effects can also be seen in versions of White's display in which all T-junctions have been removed (Hong & Shevell, 2004a, 2004b; Howe, 2005; Yazdanbakhsh, Arabzadeh, Babadi, & Fazl, 2002). 

Here I show that assimilation can be observed and studied quantitatively in one of the simplest of lightness displays: the classical disk-and-ring (DAR) stimulus (Gilchrist et al., 1999; Heinemann, 1955, 1972; Wallach, 1948, 1963, 1976). When assimilation occurs in such displays, it is always part of a larger pattern of results in which assimilation occurs over some range of surround luminances and contrast occurs over another range of surround luminances. Thus, to understand assimilation in DAR displays—and perhaps in more complex displays such as White's and in natural scenes—it is necessary to understand the overall pattern of results entailing both assimilation and contrast. 

A series of experiments will be presented, which together demonstrate that assimilation and contrast in DAR stimuli are influenced in systematic ways by manipulations of the ring size, the contrast polarities of the inner and outer edges of the ring, and the cognitive assumptions that the observer makes about the nature of the illumination, which can be manipulated through instructions. Changing any of these variables can alter the direction of the surround influence from one of contrast to assimilation or vice versa. To account for the pattern of results seen in these experiments and in other related experimental results from the lightness literature, a neural model will be proposed. At the end of the paper, I discuss how the model might be generalized to account for lightness perception in more complex scenes. 

Experiment 1: Lightness matching with double-decrement DAR displays of three sizes

The experiments reported here are all based on the experimental technique of lightness matching. Figure 1 illustrates the display and procedure. The lightness of the target disk on the right is manipulated by varying the luminance of its surround ring. The observer's task is to adjust the luminance of the match disk on the left to equate two disks in appearance. The rest of the luminances in the display—those of the target disk, match ring, and background field—are all held constant. The observer's match setting served as the operational definition of the target lightness. 

Figure 1

 

Diagram of the disk-and-ring display used in the matching experiments. The observer adjusted the intensity D _M of the match disk on the left to match the appearance of the two disks. The intensity R _T of the ring surrounding the target disk was manipulated to influence the target disk appearance. The luminances R _M, D _T, and B of the match ring, target disk, and background fields were held constant.

In the first experiment, lightness matches were performed with a DAR stimulus in which the background field was white (i.e., the highest luminance in the display), the rings were luminance decrements with respect to the background, and the disks were luminance decrements with respect to the rings. I will use the term double-decrement display to refer to DAR stimuli having these luminance polarity relations, meaning that each luminance is a decrement relative to the luminance of its immediate surround. 

Methods

The experiment was run in a dimly lit room, the walls of which were covered with black matte material. The stimulus was displayed on an Apple 22″ cinema display under the control of an Apple G4 computer running software written in the MATLAB programming language using the Psychophysics Toolbox application (Brainard, 1997). 

The match and target disks each measured 0.35 deg in radius. Three ring widths were used: 0.120, 0.583, and 1.050 deg. The widths of the match and target rings were always identical. The luminances of the target disk D _T, match ring R _M, and background B were fixed at the values D _T = 0, R _M = 0.7, and B = 1.4 log cd/m². The target ring luminance was varied from 0.2 to 1.2 log cd/m² in six steps separated by intervals of equal RGB units. The experiment was run in three blocks of 36 trials. Each block contained six trials each of the six target ring luminances, presented in random order. Altogether, 18 trials were run at each target ring luminance. 

Two observers participated in the experiment: MER, the author, and HK, a psychology graduate student, who was a paid participant. Neither subject was aware of the hypotheses because all of the interesting properties of the data (described below) were discovered only after the results were analyzed. 

Results

Figure 2 plots the observers' match settings against the target ring luminance on a log–log scale. The three plots in each panel correspond to the three ring widths. Two aspects of the data are particularly noteworthy. First, the plot for each observer and ring width has the mathematical form of a parabola, a term that I will use here as shorthand for the more mathematically correct term “second-order polynomial.” In other words, a large percentage of the variance in the matches is accounted for by a regression equation having the form 

$D_{\mathrm{M}}=a+b{R}_{\mathrm{T}}+c{R}_{\mathrm{T}}^2,\mathrm{P}\mathrm{a}\mathrm{r}\mathrm{a}\mathrm{b}\mathrm{o}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{i}\mathrm{n}\mathrm{g},$

(1)

where D _M is match disk luminance, and R _T is the target ring luminance, both specified in log units. It will be convenient to always take log units to be the fundamental units of luminance throughout this paper, unless otherwise noted. The equations for the least-squares parabolic models for each observer and ring size are given in the figure. Second, increasing the ring width has the effect of decreasing both the parabola's curvature (parameter c in Equation 1) and its slope b. 

Figure 2

 

Results of Experiment 1: Matches made with double-decrement displays having ring widths of 0.12, 0.58, and 1.05 deg plotted against the target ring luminance. (a) Observer MER. (b) Observer HK. The solid curves and equations are the least-squares parabolic regression models of the data. Error bars indicate standard errors of the means.

In general, parabolic lightness matching plots like the ones in Figure 2 exhibit two regimes, depending on whether the target ring luminance belongs to the ascending or descending range of the parabolic curve. For the double-decrement matching data in Figure 2, increasing the ring luminance increases the target disk lightness (assimilation) at low target ring luminances but decrease the target disk lightness (contrast) at high target ring luminances. The assimilation regime is more pronounced with narrower surround rings, consistent with earlier reports that assimilation is more likely to be observed with narrow surrounds (Helson, 1963; Helson & Joy, 1962; Jameson & Hurvich, 1961; von Bezold, 1876). 

As stated in the Introduction, the goal of the present paper is to model the results not just of Experiment 1 but of a large number of matching experiments carried out with DAR stimuli. In the following sections, I review some previously published data that the model is designed to explain, as well as some previously proposed lightness computation models that it is intended to supersede. 

The “blockage” model

The parabolic shape of the matching plot was first predicted by the so-called “blockage” model of Rudd and Arrington (Rudd, 2001; Rudd & Arrington, 2001). According to that model, the lightness of the disk in a DAR stimulus is computed from a combination of two separate achromatic color induction signals originating from the inner and outer borders of the surround ring. These induction signals perceptually “fill in” the area inside the generating border with either a lightness or darkness percept, depending on whether the side of the edge that faces the disk is light or dark relative to the side of the edge that faces away from the disk; that is, the edge contrast polarity. The magnitude of each lightness or darkness signal is assumed to be proportional to the logarithm of the luminance ratio of the border that generates it. This logarithm of the luminance ratio is the same thing as the signed luminance step in log units. The overall perceived disk color (that is, its shade of gray) is determined by the quantitative sum of the two achromatic color filling-in signals. Support for the quantitative assumptions of the model can be found in the following papers: Rudd (2001, 2003a, 2003b, 2007), Rudd and Arrington (2001), Rudd and Popa (2007), Rudd and Zemach (2002a, 2002b, 2004, 2005, 2007), Vladusich, Lucassen, and Cornelissen (2006a, 2006b, 2007). 

A key implication of the blockage model is the idea that the disk appearance does not just depend on the luminance ratio at the disk edge but also on the luminance ratio at the outer edge of the surround ring. The signed contrasts at the two edges are perceptually integrated (Arend, Buehler, & Lockhead, 1971; Arrington, 1996; Gilchrist, 1988; Land, 1977, 1983, 1986a, 1986b; Land & McCann, 1971; Reid & Shapley, 1988; Shapley & Reid, 1985; Shevell, Holliday & Whittle, 1992). A large number of psychophysical studies lend support to the edge integration idea. In particular, Shevell et al. showed that a luminance step shown to one eye only in a haploscope is summed perceptually across space with a luminance step presented only to the other eye. The significance of this perceptual outcome deserves emphasis. The lightness percept in their study is not accounted for by the target luminance per se; nor is it accounted for by the average of the luminances in the two eyes. It can only be explained by assuming the existence of a cortical mechanism that sums luminance steps across space. 

A demonstration of perceptual edge integration is shown in Figure 3. A schematic diagram of how edge integration is realized in the blockage model is presented in Figure 4. 

Figure 3

 

Demonstration of perceptual edge integration in lightness. The luminances of the disks and rings are the same on the two sides of the figure, but both appear lighter on the left because the background field is darker on that side. Thus, the disk appearance does not depend solely on contrast at the disk–ring edge but also depends on remote contrast or luminance. The disk lightness can be modeled as a weighted sum of the luminance steps at the inner and outer borders of the surround disks (see text for further details).

Figure 4

 

Schematic diagram of the achromatic color filling-in model. Signed induction signals generated by the inner and outer edges of the surround ring fill in lightness or darkness within the boundaries defined by the generating edges. The signs of the induction signals depend on the contrast polarity of the generating edge. Induction signal magnitudes are proportional to the luminance step at the generating edge in log units, which is the same thing as the log of the luminance ratio at the edge. Disk appearance is computed from a weighted sum of the induction signals originating at the inner and outer edges of the ring. In the double-increment DAR illustrated, both edges have a “light-inside” contrast polarity, so both edges fill in lightness, rather than darkness.

The name “blockage” model derives from a second key assumption of the model: that the contribution to the overall disk color of the induction signal generated by the outer ring border (i.e., its effective strength) is progressively decreased, or blocked, as the luminance ratio at the inner border is increased. To explain how this assumption accounts for the parabolic form of the matching plot, it is necessary to mathematically formalize the model. 

Mathematical formalization of the blockage model

The lightness Φ_D of either the target of match disk in the DAR stimulus is computed from a weighted sum of the luminance steps in log units at the inner and outer borders of the surround ring. This is expressed formally by the following equation: 

${\Phi }_{\mathrm{D}}=w_1(D-R)+w_2(R-B),$

(2)

where D − R is the luminance step traversed in crossing the border from the ring to the disk; R − B is the luminance step traversed in crossing from the background to the ring; and w ₁ and w ₂ are the weights associated with the disk–ring and ring–background borders in the disk lightness computation. Previous results (see quantitative references above) suggest that the weights assigned to the two edges tend to decrease—or at least do not increase—as a function of distance from the disk; that is, w ₂ ≤ w ₁. 

Equation 2 formalizes the edge integration assumption of the model. The other key model assumption, blockage, is that the weight w ₂ assigned to the outer border of the surround ring is attenuated by the contrast of the disk–ring edge in a contrast-dependent manner. The blockage assumption is formalized by 

$w_2=[1-\beta \delta (D-R){]}^{+}{w}_2^{*},$

(3)

where w ₂* is the weight that would be assigned to the outer border in the absence of an inner border (the “a priori” weight); [ ]+ denotes the half-wave rectification operator, which returns either the value of the algebraic expression in brackets or zero, whichever is smaller; the function δ = +1 or −1, according to the sign of the luminance step D − R; and the “blockage” parameter β quantifies the degree to which the a priori weight associated with the outer ring border is attenuated as a function of the edge step at the inner (disk–ring) border. Note that the attenuation term (the terms in brackets) saturates (goes to zero) when δ(D − R) = ∣D − R∣ > β ⁻¹; thus, the blockage parameter is also the inverse of the absolute value of the luminance step that saturates the attenuation. 

Substituting Equation 3 into Equation (3) yields the following expression for the disk lightness: 

${\Phi }_{\mathrm{D}}=w_1(D-R)+[1-\beta \delta (D-R\left){]}^{+}{w}_2^{*}\right(R-B).$

(4)

By multiplying out the second term on the right-hand side of Equation 4, it can be seen that the disk lightness varies as the square of the ring luminance over the range of ring luminances in which the attenuation term is nonsaturating. Over this range, the model predicts that the target disk lightness will exhibit a parabolic dependence on the target ring luminance. 

The elaborated blockage model of Vladusich et al. (2006b)

Although the blockage model correctly predicts the parabolic shape of the matching plot for DAR stimuli, tests of the model performed by Rudd and Zemach (2007) and Vladusich et al. (2006a) have shown that the model must be rejected because it sometimes predicts the wrong direction of curvature for the parabolic matching plots. 

The direction of curvature in double-decrement matching plots shown in Figure 2 is negative: the local slope of the plot tends to decrease as the ring luminance is increased. The tangent slope is positive in the assimilation regime of the plot and negative in the contrast regime. The blockage model predicts this negative curvature because the coefficient multiplying the R ² term in the polynomial expansion of Equation 4—βδw ₂*—must be negative for a double-decrement stimulus. This is because the weight w ₂* is constrained to be positive, the sign δ of the contrast polarity of the disk–ring border is −1 for the double-decrement stimulus, in which D < R, and the blockage parameter β must be positive in order for Equation 4 to represent a process by which the contrast of the inner ring border partially blocks (attenuates) the contribution of the outer ring border to the disk lightness computation. 

If the contrast polarity of the inner ring border is inverted by making the disk a luminance increment with respect to the surround ring, while leaving everything else in the display fixed, the blockage theory predicts that the plot curvature should change from negative to positive. Contrary to this prediction, Rudd and Zemach and Vladusich et al. both found that the curvature is also negative for this disk-contrast-inverted DAR stimulus. 

It was noted in both studies that the results could be accommodated by allowing the parameter β in Equation 4 to be negative for the double-decrement stimulus and positive for the disk polarity-reversed stimulus, but then the process modeled by Equation 4 would not represent blockage in the latter case. Equation 4 would represent either blockage (attenuation) or “anti-blockage” (amplification) of the remote induction signal originating from the outer border, depending on the contrast polarity of the inner border (Rudd & Zemach, 2007). 

Rudd and Zemach and Vladusich et al. also examined lightness matches with DAR stimuli in which the background field was dark, rather than the highest luminance in the display. Different patterns of results were obtained in the two studies. Vladusich et al. found that “blockage” (i.e., attenuation of the remote border induction signal) occurred whenever the contrast polarities of the inner and outer borders were the same, as in either double-decrement or double-increment DAR stimuli, whereas “anti-blockage” (amplification of the remote induction signal) occurred whenever the contrast polarities of the inner and outer borders were different. They proposed that an edge integration model having these auxiliary assumptions could account for the assimilation effects observed in both in their own study with DAR stimuli and in White's illusion. I will refer to this elaborated model that associates blockage with same contrast polarity edges and anti-blockage with opposite contrast polarity edges as the VLC model. 

Experiment evidence against the VLC model

The remainder of the experimental section of this paper is devoted to discussing aspects of lightness matching data that cannot be accounted for by the VLC model. These data also serve to motivate the new neural edge integration model proposed below. 

The first main result that cannot be accounted for by the VLC model is the change in matching plot curvature that occurs when the ring size is changed, as was found with double-decrement displays in Experiment 1 of the present study. To account for this effect, an additional mechanism would have to be added to the model that would alter the strength of blockage or anti-blockage as a function of the inter-border distance (i.e., the ring width). This criticism applies equally to the original blockage model. 

Rudd and Popa (2007) proposed a model that can account for at least some effects of ring size on disk lightness by combining edge integration with contrast gain control, a mechanism that we will revisit below. However, their model cannot account for the effects of instructions on lightness matches to be demonstrated in Experiment 2. The successful properties of the model of Rudd and Popa are retained in the neural edge integration model proposed below. 

A second key problem for the VLC model arises from its assumption that the critical variable determining whether blockage or anti-blockage occurs is the relationship between the contrast polarities of the inner and outer ring borders. If the contrast polarities are the same, blockage occurs; if the border contrast polarities differ, anti-blockage occurs. From Equation 4, it can be seen that the direction of the curvature in the matching plot depends strictly on the contrast polarities of the edges and the sign of the blockage constant. Because the VLC model associates a unique blockage constant sign with each pair of edge polarities, the model predicts that only one direction of curvature should be observed for any given set of edge contrast polarities. 

The matching plots shown in Figure 5 illustrate a situation in which this prediction is violated. The data are taken from an experiment by Rudd and Zemach (2004) in which the disks were luminance decrements with respect to the surround rings and the background field was dark. Since Wallach (1948, 1963, 1976) used DAR stimuli having these polarity relations in his well-known lightness constancy experiments, I will refer them as Wallach stimuli. 

Figure 5

 

Appearance matches made with a Wallach stimulus (decremental disks, dark background). The data from individual observers have been fit with least-squares second-order polynomial regression models (parabolas, solid lines). The plots for JL, LT, and IKZ have been shifted downward successively by 0.2 log unit for clarity of presentation. Error bars indicate standard errors of the means, which are in many cases smaller than the markers. The stimulus dimensions are given in Figure 1; the ring width was 0.35 deg. For other details of the method, see Rudd and Zemach (2004).

The matching plots in Figure 5 have been fit with parabolic regression functions. The least-squares parabolic models account for 99.7–99.8% of the variance in the match disk settings for each of the four observers in the study. For three of the four observers, the improvement in the fit of the parabolic model over a linear regression model is statistically significant (LT: r ²-change = 0.003, t(102) = −1.380, p = 0.019; JL: r ²-change = 0.015; t(105) = 4.174, p < 0.0005; AD: r ²-change = 0.001, t(105) = 0.886, p = 0.378; IKZ: r ²-change = 0.005, t(105) = −3.557, p = 0.001). However, the additional variance accounted for by the parabolic model is quite small, ranging from 0.1 to 1.5% for those three observers. For this reason, Rudd and Zemach fit their matching data with a linear model. For each of the four observers, the slope of the least-squares linear model was about −0.7 (Figure 6). We will return to this finding of −0.7 matching plot slopes below because it also shows up in other studies involving DAR stimuli having the same size as these Wallach stimuli but different combinations to edge contrast polarities. 

Figure 6

 

Linear regression model fits to the Wallach stimulus matching data shown in Figure 5 (solid lines). The slopes of the models are about −0.7 for each observer. Error bars indicate standard deviations of the means.

The conclusion that size is a critical factor in determining the slope of the matching plot is based on the fact that Rudd and Zemach repeated their experiment with ring widths ranging from 0.06 to 2.48 deg and found that the slopes progressively decreased over this range from about −0.6 to −1. They modeled this ring size effect with a simple edge integration theory in which the weights assigned to edges decrease as a function of distance from the disk (Figure 7). The matching data shown in Figure 6 correspond to a stimulus in which the disk and ring radii were both 0.35 deg. For the largest ring sizes used in the study, the slope tended toward the slope of −1 corresponding to a ratio match. Thus, the results of the ring size manipulation suggest that Wallach's well-known claim that appearance matches made with Wallach stimuli conform to ratio matches only holds for sufficiently wide rings. The steepness of the slope provides a measure of the strength of the lightness induction from the ring; wider rings induce more darkness in a decremental disk than do narrow rings. A demo of the fact that induction strength depends on the surround width is presented in Figure 8. 

Figure 7

 

Effect of surround size on matches made with Wallach stimuli. The slopes of the matching plots are plotted against the ring width in log deg for two observers. Errors indicate standard errors of the slope estimates. For other details, see Rudd and Zemach (2004).

Figure 8

 

Demonstration of the effect of surround size on disk lightness. Fixate the red cross and the disk with the larger surround appears darker than the disk with the smaller surround, despite the fact that the two disks are physically identical. The edge integration model accounts for this illusion by postulating that the disk lightness is synthesized in the brain by combining a darkness signal filled in from the disk–ring edge with a lightness signal filled in from the ring–background edge. The disk appearance depends on the difference between the two signals. The magnitude of the lightness-inducing signal from the outer border decreases as a function of distance, so the disk on the left looks darker because less lightness is induced in the left disk than in the right disk.

In the present context, the most important finding of the parabolic regression analysis of the data shown in Figure 5 is that the matching plots curve in different directions for different observers. JL's plot exhibits statistically significant positive curvature, while the plots of LT and IKZ both exhibit significant negative curvature. The finding of opposite plot curvatures for JL and IKZ holds even if the p-values associated with the percent improvement in the fit of the parabolic model over the linear model are subject to a Bonferroni correction in order to take into account the number of statistical tests performed in the study (p _crit = 0.013). The 95% confidence intervals for the curvature (the coefficient that multiplies the R _T ² term in the parabolic regression model) do not include any values whose sign is inconsistent with the sign of the coefficient estimate (confidence intervals = LT: −0.599, −0.054; JL: 0.414, 1.162; IKZ: −0.761, −0.216); so we can be reasonably confident that observer JL's true matching plot curvature is positive, while the curvatures in IKZ's and perhaps LT's matching plots are both negative. 

A similar analysis was performed on the matching data from a study by Rudd and Zemach (2005) in which observers matched double-increment DAR stimuli: that is, DAR stimuli in which the disk luminance exceeded the ring luminance and the background had the lowest luminance in the display. The stimuli used by Rudd and Zemach in their 2005 study were the same size as the Wallach stimuli that produced the matching plots shown in Figures 5 and 6. The matching plots from the study using double-increment stimuli are shown in Figure 9, along with least-squares parabolic models of the data. The parabolic models accounted, on average, for 96.3% of the variance in the match disk settings made by the three observers, compared to 94.3% for the linear model. For two of the three observers, the parabolic model accounted for a very large proportion of the match setting variance: 98.6% and 99.1%. Nevertheless, the improvement in the fit of the parabolic model over the linear model was statistically significant for only one observer (AD: r ²-change = 0.041, t(71) = −4.522, p < 0.0005) and borderline significant for another (JL: r ²-change: = 0.005; t(105) = −1.793, p = 0.076). For the third observer, no statistical improvement was found (LT: r ²-change = 0.001, t(105) = 0.313, p = 0.755). 

Figure 9

 

Appearance matches made with a double-increment DAR stimulus. The data from individual observers have been fit with second-order polynomial regression models (parabolas, solid lines). The data from observers JL and LT have been shifted downward by 0.2 and 0.4 log unit, respectively, for clarity of presentation. The error bars, indicating standard errors of the means, are in many cases smaller than the markers. The ring width was 0.35 deg. Other details of the stimulus were as shown in Figure 1 and reported by Rudd and Zemach (2005).

The two data plots in Figure 7 that exhibit either significant or borderline significant curvature both curve in the negative direction. For the plot having the largest—and statistically significant—negative curvature (AD), the 95% confidence interval for the regression coefficient associated with the degree of curvature includes only negative values (−0.704, −0.273). For the plot having borderline-significant curvature (JL), the majority of the 95% confidence interval for this coefficient (−0.242, 0.012) comprises negative values. Thus, we find no evidence from this study to suggest that the matching plots for double-increment stimuli curve in different directions for different observers. 

Rudd and Zemach fit these data with linear regression models (Figure 10) and noted that the slopes of the least-squares models varied considerably across observers. Experiment 2 was undertaken to explore some possible reasons for this inter-observer variability. 

Figure 10

 

Least-squares linear regression models (solid lines) of the data shown in Figure 9. All three observers exhibited statistically significant contrast induction effects but the strengths of the effects vary between observers, as indicated by the variable slopes of their matching plots. Error bars indicate standard errors of the means.

Overall, the statistical analyses presented in this section suggest that parabolic matching plots from different observers can, at least in some cases, bend in different directions. However, this conclusion depends critically on the data from a single observer (JL) in the study using Wallach stimuli, so it might be prudent to consider it tentative. In Experiment 2, I present further evidence that the direction of curvature can vary between observers using a matching task involving the double-increment stimuli and I show that this effect can be brought under some degree of experimental control by varying the instructions given to the observers. 

Before discussing that experiment, it should perhaps be noted that the statistics reported in this section for the matching data from the two Rudd and Zemach experiments vary slightly from the results reported in the original papers (Rudd & Zemach, 2004, 2005, 2007). The discrepancies are due to the fact that I have here performed post-hoc corrections to the monitor calibration. First, the luminances of the match disk and ring have been adjusted downward by 0.057 log unit to correct for a difference in the LCD screen luminances on the left- and right-hand sides of the display (see Rudd & Zemach, 2005 for details). Second, the data have been corrected to compensate for a one-line error in the lookup table that was discovered after the experiments were run and the data were published. The first correction affects only the intercept of the least-squares model and thus does not affect the curvature results. The second correction has a minor influence on the parameters of the least-squares regression models but does not affect the qualitative conclusions of the hypothesis tests, which agree with the conclusions of the analyses reported in the original papers. 

Experiment 2: Effects of instructions on the shape of the matching plots obtained with double-increment stimuli

Experiment 2 was originally motivated by a desire to better understand the large inter-subject variability in the matching plots obtained in Rudd and Zemach's (2005) study with double-increment stimuli (Figure 10). Several previous studies carried out with double-increment stimuli found that the surround luminance had either little or no effect on the target lightness (Agostini & Bruno, 1996; Diamond, 1953; Economou, Zdravkovic, & Gilchrist, 2007; Gilchrist, 1988; Heinemann, 1955, 1972; Jacobsen & Gilchrist, 1988; Kozaki, 1963, 1965); whereas we found statistically significant contrast effects for all three of our observers. Further complicating the overall picture is the fact that Bressan and Actis-Grosso (2001) noted a significant contrast effect for double-increment stimuli over a range of low surround luminances but assimilation over a range of high surround luminances. Thus, the overall pattern of results for double-increment stimuli is very mixed. To better understand the full set of results of double-increment stimuli is particularly important because the highest luminance anchoring rule—one of the key tenets of Gilchrist's Anchoring Theory of lightness—makes a clear prediction that there should be no effect of surround luminance for double-increment stimuli (Gilchrist, 2006; Gilchrist et al., 1999). 

The wide range of actual results obtained in previous experiments with double-increment stimuli raises the possibility that different observers may adopt different perceptual criteria for matching these stimuli. In Experiment 2, the matching instructions were varied in an attempt to gain some experimental control over the observer's matching criterion. 

Arend and Spehar (1993a, 1993b) showed that appearance matches made with simple stimuli similar to the DAR stimuli used here can be based on any one of at least three different perceptual dimensions of disk appearance: perceived reflectance (lightness); perceived luminance (brightness); or perceived contrast with respect to the surround (referred to by those authors as “brightness contrast”). Furthermore, an observer who is asked to produce a reflectance match can do so by making either of two assumptions regarding the nature of the illumination falling on the match and target disks. The observer could assume that the illumination across the entire display is constant and remains unchanged when the target ring luminance is varied, in which case experimental manipulations of the target ring luminance must signal changes in the target ring reflectance. Otherwise, the observer could assume that a change in the target ring luminance signals a change in the illumination on the target side of the display only—including the target disk and ring—while the target ring reflectance remains constant. 

In the absence of instructions informing the subjects of which of these assumptions they should make, appearance matches could be based on either of the two sets of lightness matching assumptions described, or on some sort of compromise between them, or on either brightness or brightness contrast judgments. This ambiguity likely helps explain the high variability in the matches made with double-increment stimuli in previous naive matching studies where no special instructions were provided to the subjects. 

In Experiment 2, two new observers performed appearance matches under each of four sets of matching instructions described above using the same double-decrement stimuli that were used in Rudd and Zemach's (2005) naive matching experiment. With double-decrement stimuli. In what follows, I will refer to the four sets of instructions as brightness, brightness contrast, one-spotlight lightness, and two-spotlight lightness matching. 

Before presenting the experimental results, it may be helpful to consider what an ideal observer would do when given each of these four sets of instructions. For brightness matching, the observer is instructed to match the two disks on perceived luminance, so an ideal observer would perform a luminance match. For brightness contrast matching, the observer is instructed to match the disk–ring contrast on the two sides of the display, so an ideal observer would perform a ratio match. For one-spotlight lightness matching, variations in the target ring luminance are to be interpreted as changes in the target ring reflectance, which the subjects were told explicitly to think of as a paper surface. Such changes should not influence the perceived reflectance of the target disk, so an ideal observer would perform a luminance match. For two-spotlight lightness matching, variations in the target ring luminance are to be interpreted as changes in the illumination on both the target disk and its surround. Here, ideal observer would match the disk–ring ratios on the two sides of the display. The ideal observer predictions for the four sets of matching instructions are illustrated in Figure 11. 

Figure 11

 

Appearance matches for an ideal observer who performs brightness (perceived luminance), brightness contrast (perceived luminance ratio), one-spotlight lightness, and two-spotlight lightness matches with a double-decrement DAR stimulus. In one-spotlight lightness matching, the observer is instructed to imagine that the disks and rings are papers lit by a single global spotlight and that luminance changes in the test ring signal changes in the reflectance of the paper comprising that ring. In two-spotlight lightness matching, the observer is instructed to imagine that disk-and-ring papers on the two sides of the display are lit by separate spotlights whose intensities may differ and that changes in the luminance of the test ring signal changes in the light illuminating the target disk and ring.

Methods

The disk and ring radii were 0.35 deg. The luminances of the target disk, match ring, and background field were D _T = 0.5, R _M = 0.0, and B = −1.0 log cd/m². The target ring luminance was varied from −0.326 to 0.201 log cd/m² in six steps separated by intervals of equal RGB units. The experiment was run in three blocks of 36 trials. Each block contained six trials each of the six target ring luminances, presented in random order. Eighteen trials were run at each target ring luminance, for a grand total of 108 trials. 

Two observers participated in the experiment. MER was the author of the paper. AH, a psychology undergraduate at the University of Washington, participated for independent research credit. The author was aware of the ideal observer model predictions but not the predictions of the neural model that was ultimately fit to the data from both observers. The parabolic nature of the matching plots was discovered only after the results of the experiment were analyzed. 

There exist small differences in the luminances reported here for D _T, R _T, R _M, and B and the ones reported by Rudd and Zemach (2005) for the same stimuli. Before performing Experiment 2, it was discovered that the lookup table used in the earlier study contained an error that shifted the values in the table by one line, as discussed in the previous section. This problem was corrected before running Experiment 2. At the same time, the monitor was recalibrated and the luminances D _T, R _T, R _M, and B were measured directly with a photometer. The directly measured values are reported here. Rudd and Zemach instead reported the nominal luminance values that were targeted by the MATLAB program used to generate the stimuli. Experience in modeling the data from experiments such as these has underscored the importance of precise luminance measurement. 

Other details of the method are as in Experiment 1. 

Results

The actual matches produced in response to each of the four sets of matching instructions are plotted in Figure 12. For each subject and condition, the plots have been fit with either a linear or a parabolic regression model on the basis of the following statistical procedure. First, a parabolic (i.e., second-order polynomial) model was forced on the data by regressing the matches from each of the four conditions against two predictor variables: the target ring luminance and the square of the target ring luminance (both in log units). A linear model was adopted in place of the parabolic model if the second variable could be removed without producing a statistically significant decrease in the amount of variance in the match disk settings accounted for by the regression model. For one analysis only (AH, brightness matching), neither predictor variable was significant. The results from that condition have been fit with a parabolic model in Figure 12. This procedure resulted, for both observers, in fitting the brightness and one-spotlight lightness matches with a parabolic model and the brightness contrast and two-spotlight lightness matches with a linear model. 

Figure 12

 

Matches made with a double-increment display under the four sets of matching instructions described in Figure 11. (a) Observer MER. (b) Observer AH. The data from each observer and condition have been fit with either a linear or a parabolic regression model based on a statistical procedure described in the text. The similarity in the model fits for MER's brightness and one-spotlight lightness matches, and for his brightness contrast and two-spotlight lightness matches, suggests that a common neural mechanism may underlie these matches. Error bars indicate standard errors of the means.

The parabolic model is parameterized by three regression coefficients: the intercept (b ₀), slope (b ₁), and curvature (b ₂) of the second-order polynomial model. The results of the parabolic regression analyses are as follows: Brightness (AH: b ₀ = 0.505 ± 0.003, p < 0.001; b ₁ = 0.002 ± 0.015, p = 0.916; b ₂ = 0.102 ± 0.080; r ² = 0.457; MER: b ₀ = 0.500 ± 0.011, p < 0.001; b ₁ = 0.265 ± 0.051, p = 0.015; b ₂ = 0.886 ± 0.270, p = 0.046; r ² = 0.901); brightness contrast (AH: b ₀ = 0.422 ± 0.034, p = 0.001; b ₁ = −1.065 ± 0.152, p = 0.006; b ₂ = −0.730 ± 0.805; r ² = 0.960; MER: b ₀ = 0.444 ± 0.010, p < 0.001; b ₁ = −0.790 ± 0.045, p < 0.001; b ₂ = −0.069 ± 0.241, p = 0.794; r ² = 0.994); one-spotlight lightness (AH: b ₀ = 0.497 ± 0.004, p < 0.001; b ₁ = 0.084 ± 0.020, p = 0.025; b ₂ = 0.331 ± 0.107, p = 0.053; r ² = 0.856; MER: b ₀ = 0.491 ± 0.007, p < 0.001; b ₁ = 0.254 ± 0.030, p = 0.003; b ₂ = 0.865 ± 0.158, p = 0.012; r ² = 0.961); two-spotlight lightness (AH: b ₀ = 0.466 ± 0.039, p = 0.001; b ₁ = −0.854 ± 0.175, p = 0.016; b ₂ = −1.023 ± 0.929, p = 0.351; r ² = 0.913; MER: b ₀ = 0.449 ± 0.012, p < 0.001; b ₁ = −0.776 ± 0.052, p = 0.001; b ₂ = 0.033 ± 0.278, p = 0.912; r ² = 0.992). 

The linear (reduced) model is parameterized by two regression coefficients only: the intercept (b ₀) and slope (b ₁). The results of the linear regression analyses, where appropriate, are: brightness contrast (AH: b ₀ = 0.399 ± 0.022, p < 0.001; b ₁ = 0.975 ± 0.112, p = 0.001; r ² = 0.950; MER: b ₀ = 0.442 ± 0.006, p < 0.001; b ₁ = −0.782 ± 0.030, p < 0.001; r ² = 0.994); two-spotlight lightness (AH: b ₀ = 0.434 ± 0.027, p = 0.001; b ₁ = −0.728 ± 0.136, p = 0.006; r ² = 0.877; MER: b ₀ = 0.450 ± 0.007, p < 0.001; b ₁ = −0.780 ± 0.034, p < 0.001; r ² = 0.992). 

In two of the four conditions—brightness and brightness contrast—the matching results from observer AH conformed to the ideal observer models. No statistically significant curvature was found in her matching plots in either of these conditions; and the slopes of her plots were not significantly different from 0 in the brightness condition and −1 in the brightness contrast condition. 

For the other observer in the brightness and brightness contrast conditions, and for both observers in the two lightness conditions, the results conformed to one of two other quantitative patterns. Either (1) the plot was a straight line having a slope of about −0.7 (contrast, two-spotlight lightness), as was also found in Rudd and Zemach's (2004) naive matching study employing Wallach stimuli; or (2) the matching plots were parabolic (brightness, one-spotlight lightness conditions) and exhibited a contrast effect at low target ring luminances and assimilation at high target ring luminances. The latter pattern is the same as that found by Bressan and Actis-Grosso (2001) for lightness matches performed with double-increment stimuli. 

The main take-home message of Experiment 2 is that the matches made with DAR stimuli can depend on the subject's interpretation of the stimulus, as shown earlier by Arend and Spehar. Thus, one should be cautious in interpreting the results of matching experiments as evidence for a theory of lightness perception unless it is clear what perceptual dimensions of the stimulus are being attended to and matched by the observers. Furthermore, lightness matches are not always independent of the surround luminance in the case of double-increment stimuli. Contrary to the predictions of Anchoring Theory, the effect of manipulating the luminance of the lower luminance surround in a double-increment stimulus depends on the assumptions that the observer makes about the nature of the illumination. Overall, ideal observer models provide a better account of the results of Experiment 2 than does anchoring to the highest luminance. 

It is worth noting in this regard that in at least one earlier study that found no effect of surround luminance on the matches made with double-increment stimuli (Jacobsen & Gilchrist, 1988), observers were explicitly told to match the stimuli on brightness when the target was an increment relative to its surround luminance. Despite this fact, the results of Jacobsen and Gilchrist's study have subsequently been cited (erroneously) as evidence in favor of highest luminance anchoring in lightness perception (Gilchrist, 2006; Gilchrist et al., 1999). 

Recall that the naive matching results obtained by Rudd and Zemach (2005) with the same physical stimuli used in Experiment 2 were highly variable (see Figures 9 and 10). Nevertheless, for all three of their observers, the naive matches lay in between the results obtained here with the two different sets of lightness matching instructions (or, alternatively, the brightness and brightness contrast instructions). This suggests that the naive matches in our 2005 study may have been based on a compromise between alternative interpretations of the disk appearance dimension to be matched. 

The regularities seen across observers in terms of their patterns of errors or deviations from the ideal observer predictions are also worth emphasizing. The results from the two lightness matching conditions of Experiment 2 exhibit regularities that mirror findings obtained in our 2004 study with Wallach stimuli and with double-decrement stimuli in Experiment 1 of the present study. First, roughly linear matching plots having slopes of about −0.7 were found in both the two-spotlight matching condition of Experiment 2 and in our earlier study employing Wallach stimuli having surround rings of the same size. The fact that strong contrast effects were obtained both with Wallach stimuli in our 2004 study and with two-spotlight instructions here is consistent with Wallach's interpretation of his own results as an attempt on the part of the observers to achieve lightness constancy under the assumption that the two sides of the display are lit by separate illuminants. Nevertheless, in both our 2004 study and in the two-spotlight condition of Experiment 2, our observers failed to make the exact ratio matches predicted for an ideal observer under two-spotlight lightness matching instructions, as indicated by the fact that the matching plot slopes were about −0.7, rather than −1. It seems likely that that this deviation from exact ratio matching is due to the small size of the 0.35 deg surround used in both experiments. The plots obtained with 0.35 deg Wallach stimuli in our 2004 study were about −0.7, but the slopes tended to decrease toward −1 as the ring size was increased in that study (Rudd & Zemach, 2004). Further work is required to determine whether a perfect ratio match would hold for double-increment and two-spotlight instructions if the surround size were increased. 

Second, parabolic matching functions exhibiting both contrast and assimilation effects over different ranges of the surround luminance ranges were obtained both for naive matches made with double-decrement stimuli in Experiment 1 of the present study and for one-spotlight lightness matches made with double-increment stimuli in Experiment 2. In the next section, a neural model that accounts for this regularity is proposed. The model also accounts for the rest of the results discussed to this point, including both the parabolic and approximately linear matching plots, and their dependence on luminance polarity, surround size, and matching instructions. The model is able to account both for the aspects of the data that approximate ideal observer behavior (i.e., constancy) and for the pattern of errors. By incorporating attentional feedback to adjust the model parameters in a task-specific way, the model is able to fit the results of both lightness matching conditions of Experiment 2 in the context of a single unified neural theory. 

Neural lightness computation model

The model comprises three hierarchical layers of cortical visual neurons. In Layer 1, orientation-tuned neurons encode as spike rates the spatially oriented contrast associated with luminance steps in the input image. I will refer to these neurons as “edge detector” (ED1) neurons, but it should be understood that this is just shorthand for a bank of neurons having odd-symmetric receptive fields that that can be modeled as linear filters followed by a half-wave rectifying threshold. The neurons in Layer 1 filter the image (defined in log units) at all spatial orientations and across a range of spatial frequencies. For concreteness, we might think of them as simple cells in cortical area V1 having odd-symmetric Gabor-like receptive fields (see Rudd & Zemach, 2004, 2005). 

To model the neural response to circularly-symmetric DAR stimuli, we can reduce the problem to one dimension by considering a one-dimensional slice of a single disk-and-ring luminance profile, with the disk center taken to be the origin. This reduces the number of possible edge orientations to two: edges whose light or dark side points to the disk center. This reduced model suffices to illustrate the key principles of the theory. 

In this one-dimensional model, two types of edge detector neurons exist at each location in Layer 1: one detector for each of the two edge contrast polarities. In response to a DAR stimulus, ED1 neurons having receptive fields centered on the positions of the disk–ring and ring–background edges are the only ones that will fire in the Layer 1 array. The edge contrast polarity determines which of the two types of ED1 neurons will fire at each edge location. ED1 firing rates are proportional to the luminance step in log units across the edge to which a neuron responds. 

Note that this edge detector model differs from the standard model of a V1 neuron in that the neuron's receptive field is here assumed to spatially filter the input image defined in log units, rather than raw luminance. Some recent quantitative analyses of single-cell recordings support the idea that the responses of V1 neurons are linear with respect to log luminance, rather than raw luminance (Kinoshita & Komatsu, 2001; Vladusich et al., 2006b). Linear and logarithmic models of the visual response to edges are equivalent when the edge contrast is low (Reid & Shapley, 1988). 

The gain of each ED1 unit is controlled by feedback from an unspecified higher cortical center. This feedback adjusts the strength of each edge response based on the edge's importance for the matching task at hand: for example, either one-spotlight or two-spotlight lightness matching. 

Model Layer 2 comprises a second dual array of edge detector neurons. A pair of neurons at each array location encodes the presence and magnitude of either a light-inside or a dark-inside luminance edge step at the corresponding position in the input image. ED2 neurons differ from ED1 neurons in the manner in which their neural gains are controlled. Unlike ED1 neurons, ED2 units are not subject to top-down gain modulation; instead, their gains are modified and are either increased or decreased relative to an initial (tonic) gain state by the firing of ED1 neurons whose receptive fields lie within a neighborhood of the receptive field center of the ED2 neuron whose gain is modulated. 

The gain change produced in the ED2 neuron by the activities of nearby ED1 neurons is proportional to a weighted sum of the ED1 neuronal firing rates, which in turn are proportional to the luminance steps across the edges that cause the ED1 neurons to fire. Since the gain factor of an ED2 neuron determines the magnitude of the ED2 neuronal response to an edge, this gain control by other nearby edges produces a “contrast–contrast” effect. In what follows, I will refer to the gain control produced in the pathway from Layer 1 to Layer 2 as “contrast gain control” to distinguish it from the top-down “attentional gain control” that is directed from higher visual centers to Layer 1 and that produces task-specific modifications of the gains of the ED1 neurons. 

The direction of curvature in the lightness matching plot depends on whether the gain applied to a particular edge of the surround ring in Layer 2 is either turned up or turned down by a neural firing response to the other edge in Layer 1. Both increases and decreases in gain relative to the tonic state must be included in the model in order to fit psychophysical matching plots having both directions of curvature for a given DAR stimulus. If the gain control directed from the inner ring edge to the outer ring edge is opposite in sign to the gain control directed from the outer edge to the inner edge, then the relative strengths of the two gain controls determine the direction of curvature in the matching plot. 

The magnitudes of the gain modulations in Layer 2 are assumed to fall off linearly with distance between the receptive field centers of the ED2 neuron whose gain is being modulated and an ED1 neuron whose spiking response causes the ED2 gain change. The linear falloff assumption is made strictly for computational convenience, as any monotonic spatial form factor could fit equally well the data from the experiment discussed here since a DAR stimulus has only two edges. 

Layer 3 consists of a third dual array of neurons, termed achromatic color (AC) neurons. The population response of the AC neurons determines the achromatic color assigned to each point in the image array. AC neurons come in two types: lightness neurons and darkness neurons. Each AC neuron performs a weighted spatial summation of the responses of the ED2 neurons located within its receptive field. The receptive field weighting function falls off linearly from the receptive field center to its outer perimeter. AC neuronal receptive fields must be much broader than the receptive fields of the edge detector neurons in Layers 1 and 2 to account for the fact that achromatic color can be influenced by remote edge contrast (Bressan & Kramer, 2008; Shevell et al., 1992). 

While the Layer 1 and 2 neurons function as edge detectors, the AC neurons function as edge integrators. Darkness neurons sum the outputs of ED2 units that fire in response to dark-inside edges. Lightness neurons sum the outputs of ED2 units that fire in response to light-inside edges (Figure 13). The achromatic color assigned to a given image location depends on the relative strengths of the lightness and darkness neuronal responses at that location (i.e., the difference L(i) − D(i), where i is the array index). To keep things simple, it is assumed in what follows that the lightness of either disk in any given DAR display is determined solely by the difference in the responses of the lightness and darkness neurons whose receptive field centers are located at the disk center. A more sophisticated version of the model would assign an achromatic color to each point within the disk interior, according to the computational rules described above. In that case, lightness could vary within the disk, along its radius as a function of distance from the disk edge. 

Figure 13

 

Receptive field organization of the AC neurons in Layer 3. (a) Lightness neurons compute a weighted sum of the outputs of Layer 2 edge detector neurons that fire in response to edges whose light sides point toward the neuron's receptive field center. Outputs of neurons responding to edges whose light sides face right (red dots) are summed over the left side of the receptive field. Outputs of neurons responding to edges whose light sides face left (blue dots) are summed over the right-hand side of the receptive field. (b) Darkness neurons compute a weighted sum of the outputs of Layer 2 edge detector neurons that fire in response to edges whose dark sides point toward the neuron's receptive field center. Outputs of neurons responding to edges whose dark sides face right (blue dots) are summed over the left side of the receptive field; outputs of neurons responding to edges whose dark sides face left (red dots) are summed over the right side of the receptive field. The contribution of a Layer 2 output to an AC unit's firing rate falls off linearly with distance between the receptive field centers of the pre-synaptic (Layer 2) and post-synaptic (Layer 3) neurons.

To model the lightness matches made with DAR stimuli, it suffices to specify the conditions in which the match and target disks will look alike. The model is not required to decide whether disks appear “white,” “black,” “mid-gray,” etc. The model described is able to compute the relative lightnesses of any two regions in the display and thus suffices to model the matching data. In order to assign absolute lightness values to the disks—that is, values like “white,” “mid-gray,” etc.—the model would need to be supplemented with an “anchoring” rule. It might be assumed, for example, that the maximum firing rate across the Layer 3 array always corresponds to the value “white” (that is, a reflectance of 90%). The perceived reflectance corresponding to any other arbitrary point in the image would then be computed from the ratio of the neuronal firing rate associated with the arbitrary point to the firing rate associated with the white point. 

Given this latter scheme for anchoring the relative lightnesses computed by the model, the highest lightness in the image will always be white (Kingdom, in press; Rudd & Zemach, 2005). The highest lightness rule is closely related to, but not identical with, the well-known hypothesis that highest luminance is always seen as white (Gilchrist, 2006; Gilchrist & Radonjić, 2009; Gilchrist et al., 1999; Li & Gilchrist, 1999). In the few cases where these two alternative anchoring rules have been tested against one another by experiment, the psychophysical data have supported the highest lightness rule over the highest luminance rule (Bressan & Actis-Grosso, 2001; Rudd & Zemach, 2005). 

Though it is beyond the scope of the present paper to identify the hypothetical neural mechanisms for lightness computation proposed here with neural activities in actual visual cortical brain regions, an educated guess might locate the AC neurons in area V4, where a nonlocal “Retinex-like” computation supporting color constancy is thought to occur (Bartels & Zeki, 2000; Clarke, Walsh, Schoppig, Assal, & Cowey, 1998; Kennard, Lawden, Morland, & Ruddock, 1995; Kentridge, Heywood, & Cowey, 2004; Smithson, 2005; Walsh, 1999; Zeki, Aglioti, McKeefry, & Berlucchi, 1999; Zeki & Marini, 1998). The Layer 1 and Layer 2 edge detector neurons are likely to be located in area V1 or area V2, where edge-sensitive neurons are known to exist (Hubel & Wiesel, 1959, 1962, 1965, 1968; Peterhans, von der Heydt, & Baumgartner, 1986). 

The basic architecture of the model is illustrated in Figures 14 and 15. The formal equations of the model are presented in 1. 

Figure 14

 

Two-dimensional schematic diagram of the neural lightness computation model looking down from above Layer 3. Polarity-sensitive cortical edge detector neurons (E) encode as spike rates the directed luminance steps at the disk–ring and ring–background borders. Each edge detector response is proportional to the luminance step in log units at the border, which is the same as the logarithm of the local luminance ratio. Nearby edge detector neurons interact to control each other's contrast gains in the mapping from model Layer 1 to Layer 2, with the gain control strength depending on the response of the neuron producing the gain change. Achromatic color neurons (C) in Layer 3 perform a weighted spatial summation of the responses of Layer 2 edge detector neurons across the Layer 3 neuronal receptive field.

Figure 15

 

One-dimensional cross-section of the lightness computation model showing connections between layers. The model is organized into three hierarchical layers, each of which is topographically organized. Layer 1 consists of a dual array of edge detector neurons that respond to stimulus edges of the appropriate contrast polarity. At each array location exist two edge detectors, which respond to light-inside and dark-inside edges, respectively. Layer 2 also contains edge detector neurons. The gains of the Layer 2 neurons are adjusted by feedforward, laterally spreading, contrast gain modulations from Layer 1 edge detectors. The gain adjustment in a Layer 2 neuron is a linear sum of Layer 1 edge detector firing rates. The contribution of each Layer 1 neuron to the gain adjustment in a Layer 2 neuron decreases as a linear function of the spatial distance between receptive field centers of the two neurons. Achromatic color neurons in Layer 3 spatially integrate the outputs of Layer 2 neurons, with weights determined by the receptive field profile of the Layer 3 neuron, which falls off linearly with distance.

Figure 16 illustrates how the various components of the model interact to determine the lightness of the target or matching disk in a DAR stimulus. A change in either the size of the luminance step at an edge, or the magnitude of the top-down attentional gain factor applied to the ED1 neurons responding to that edge, may alter the spatial spread of the contrast gain control generated by the edge in the feedforward pathway from Layer 1 to Layer 2. If the spatial span of the contrast gain control fails to bridge the distance between the inner and outer edges of the surround ring, the matching plot will be a straight line. If the contrast gain control successfully spans the ring width, the matching plot will have the form of a parabola. The contrast gain control will fail to span the distance if the ring is too wide; or the luminance ratio of the edge that generates the contrast gain control is too weak; or if the top-down attentional gain applied to the edge that generates the gain control is damped down sufficiently. If the contrast gain control does span the ring width, the degree of curvature in the matching plot will depend on the strengths of the contrast and attentional gain controls, as well as on the ring width. Larger gain factors and smaller widths produce greater curvature. 

Figure 16

 

How the model accounts for the changes in the shape of lightness matching plots with changes in instructions. (a) The neural model, showing the edge detector (Layer 1), contrast gain control (Layer 2), and edge integration (Layer 3) processing stages. (b) Presentation of a double-increment DAR stimulus generates spiking activity in Layer 1 edge detector neurons whose receptive fields are centered on the disk–ring and ring–background borders. Layer 1 activations, in turn, produce activity at the locations of the stimulus borders in the Layer 2 map. If the Layer 1 activations are sufficiently strong, a contrast gain control spreads from the location of the Layer 1 activations to control the gains of nearby edge detectors in Layer 2. The red and blue lines indicate the strength of this distance-dependent gain control. Points at which gain control interactions occur are indicated by the yellow circles. (c) When the ring luminance increases, the Layer 1 response to the disk–ring edge decreases, while the response to the ring–background edge increases. As a result, the strength of the Layer 2 gain modulation directed from the ring–background edge to the ring–edge is increased and the strength of the gain modulation directed from the disk–ring edge to the ring–background edge in decreased and may be turned off if the span of the gain control is decreased sufficiently. The matching plot slope and curvature change as a function of the Layer 2 gains. (d) These effects can be further influenced by attentional feedback to control the gains of the Layer 1 neurons. If the matching instructions ideally call for discounting of the ring–background edge information—as in the two-spotlight lightness matching condition—the gain applied to that edge will be decreased in Layer 1, in turn decreasing the strength of the inwardly directed contrast gain control in Layer 2 and changing the strength of the matching plot curvature.

Figure 17 illustrates a neurally plausible mechanism that would cause the edge weights to fall off linearly with distance from the receptive field center. 

Figure 17

 

Neurally-plausible account of the linear falloff in edge weights and contrast gain control strengths with distance. The magnitude of either an AC neuron's response or the gain modulation in a Layer 2 neuron depends on the degree of spatial overlap between the dendritic arbor of the neuron and the axonal arbor of the neuron that either causes the AC neuron to fire or modulates the Layer 2 neuron's gain. If the density of axonal and dendritic arbors are uniform over the neurons' receptive and projective fields, and the neurons are embedded in topographically organized spatial maps with metrics equivalent to that of the retinal image, the magnitude of the feedforward activation or gain modulation will decrease as a linear function of the distance between the receptive field centers of the pre- and post-synaptic neurons.

In 2, it is shown that an appropriately parameterized version of this neural lightness model can account in a quantitatively exact way for the results of Experiments 1 and 2. 

Predicted relationship between the slope and curvature of the matching plot

In Figure 18, I have plotted on a single graph the matching plots obtained in all of the matching studies that I have conducted with double-increment stimuli, including the data from Rudd and Zemach's (2005) naive matching study, and the data from the one-spotlight and two-spotlight lightness matching conditions of Experiment 2 of the present study. The data from each observer and matching instruction set have been fit with a parabolic (second-order polynomial) regression model, regardless of whether that model provided a statistically better fit to the data than a linear model did. 

Figure 18

 

Least-squares second-order polynomial regression models of the matching data from the one-spotlight and two-spotlight lightness matching conditions of Experiment 2 (instructional variation) and from naive matches performed with the same double-increment stimuli. Error bars indicate standard errors of the mean.

The plots form a well-ordered series in which the matching plots near the top of the figure (right side) have the most positive slopes and the most positive curvatures, while the plots near the bottom of the figure tend to have the most negative slopes and the most negative curvatures. The two uppermost plots belong to the two observers in the one-spotlight condition of Experiment 2, while the two bottommost plots are from the same two observers in the two-spotlight condition. In between lie the plots from the three observers in the naive matching study. 

The neural lightness computation model described in the previous section can account for the results of all of these experiments by allowing the model parameters to vary across observers and matching instructions. Each matching plot in Figure 18 can be summarized by the three parameters of the regression model used to fit that plot: the intercept, slope (linear trend), and curvature (quadratic trend) of the least-squares second-order polynomial model. The theoretical relationships between these three regression model parameters and the parameters of the neural model are derived in 2. 

To account for the fact that the matching plots near the top of Figure 18 have positive curvature while the matching plots near the bottom of the figure have negative curvature, it is necessary to assume that the theoretical expression associated with the quadratic term of the regression model can be either positive or negative. The sign of this expression is determined by its numerator, which is given by Equation B4c in 2. For the double-increment stimulus, this expression simplifies to 

$c_2=-g{1}_{1}g{1}_2({\delta }_{21}{\omega }_{1\mathrm{T}}{\alpha }_{\mathrm{T}}+{\delta }_{12}{\omega }_{2\mathrm{T}}{\beta }_{\mathrm{T}}),$

(5)

where δ ₂₁ and δ ₁₂ represent the signs of the inwardly and outwardly directed gain controls; g1₁ and g1₂ are the neural gains applied to the inner and outer ring edges at Layer 1 of the model; ω _1T and ω _2T are the AC neuronal receptive field weights applied to the ED2 neuronal responses to the inner and outer ring edges; and α _T and β _T are the strengths of the inwardly and outwardly directed gain controls associated with the target side of the display. 

With the exception of the signs of the inwardly and outwardly directed gain controls, all of the model parameters in Equation 5 are positive, by definition. Since the matching plots from various experiments carried out with the same double-decrement displays have both positive and negative curvatures, it follows that the inwardly and outwardly directed gain controls must have opposite signs in order to account for the data from all observers and matching conditions. The sign of the curvature in any particular matching plot depends on which of the two opposing gain controls is dominant. The relative strengths of the inwardly and outwardly directed gain controls depend, in turn, on the strengths of the top-down attentional and contrast gain controls. By manipulating the strengths of these two gain control processes, and the rate of spatial falloff of the AC neuronal receptive field, it is possible to simulate all of the matching plots with a single unitary neural theory. 

From inspection of the model equations presented in 2, it can be seen that the model makes an interesting and unanticipated prediction about how the slopes of the matching plots should relate to the plot curvatures. When the slopes of the matching plots associated with different observers and instructions are plotted against the curvatures of the same plots, the model predicts that the slope–curvature pairs should fall on a straight line, which itself has a slope equal to −(D _T + B). For the double-increment DAR stimulus used in the experiments, −(D _T + B) = −(0.5 − 1.0) = 0.5 log cd/m². Thus, the model makes a strong parameter-free prediction that depends only on the physical properties of the stimulus. 

This prediction was tested by plotting in Figure 19 the slopes of the seven double-increment matching plots shown in Figure 18 (i.e., the first-order regression parameters) against the curvatures of these same plots (i.e., the second-order regression parameters). A straight line has been fit to the slope-curvature pairs from five of the seven plots (red circles in Figure 19). These five slope-curvature pairs correspond to the matching plots of the two observers in the one-spotlight condition of Experiment 2 and the three observers in the 2005 uninstructed matching study. The linear regression model shown in the figure accounts for 95% of the variance in the slope estimates for these five observer/instruction conditions. The estimated slope of the model (0.504 ± 0.067) verifies that the predicted relationship between the matching plot slope and curvature holds for these five conditions, considered in isolation. 

Figure 19

 

Slopes of the second-order regression models of the double-increment matching data shown in Figure 18 plotted against the curvatures of the same regression models. The data from both the one-spotlight lightness matching condition and the naive matches from Rudd and Zemach's (2005) study (purple circles) have been fit with a single least-squares linear regression model described by the following equation: Slope = 0.5044 * Curvature − 0.1366 (r ² = 0.9502). The least-squares model conforms to the prediction of the neural model that the slope should equal to 0.5 for this DAR stimulus (see text). The slope-curvature pair from at least one of the observers in the two-spotlight matching condition (MER, green squares) clearly falls below the line. Error bars indicate standard errors of the regression parameter estimates.

The predicted relationship is clearly violated for at least one of the two observers in the two-spotlight lightness matching condition (green squares). For the other observer in that condition, the results are ambiguous: they might either fall on the predicted curve or else be consistent with the results of the first observer, whose data clearly fall off the regression line. The two possibilities cannot be distinguished because of the large standard error associated with the curvature estimates for the two-spotlight matching condition (horizontal error bars in Figure 19). 

Since the slope–curvature pair associated with at least one of the two observers in the two-spotlight condition do not conform the model prediction, one might wonder if the results that do conform to the model prediction are somehow a fluke, or perhaps that the model appears to work only because I selected a opportunistic subset of the points in Figure 19 to fit with the linear regression model. To address this potential concern, I reanalyzed the data from the other three contrast polarity experiments analyzed by Rudd and Zemach (2007) to see whether the predicted relationship between the slope and curvature of the matching functions holds for those three additional data sets. The additional data sets include the matches obtained with Wallach stimuli by Rudd and Zemach (2004), and with double-decrement and incremental disks and white background DARs by Rudd and Zemach (2007). As can be seen from the plots shown in Figure 20, the model prediction is verified for each of these three additional data sets. Thus, the data from the two-spotlight matching condition of Experiment 2 is the only data that I have analyzed to date that fails to conform to the model prediction relating the matching plot slope to its curvature. 

Figure 20

 

Slope versus curvature plots for matches made with: (a) Wallach stimuli; (b) double-decrement DAR stimuli; and (c) DAR stimuli comprising incremental disks and a highest luminance background (Rudd & Zemach, 2004, 2007). Solid red lines are unconstrained least-squares linear regression models. Solid black lines are least-squares regression models forced to conform to the model prediction that the slope is proportional to the curvature times −(D _T + B). Error bars indicate standard errors of the regression parameter estimates.

What then should we make of the two-spotlight data? One possible explanation comes from the insight that the matching plot slopes are guaranteed to fall on the predicted line if and only if variations in the neural parameters that explain the variation along this line do not also vary the line's intercept. Any parameter change that shifts the line's intercept for a particular observer or set of matching instructions would result in an apparent violation of the model. The reader can verify from inspection of the systems of Equations B4–B4c in 2 that two and only two model parameters can be varied to shift the data along the regression line in Figure 19 without simultaneously changing the line's intercept. Those are the parameters α _T and β _T, which model the strengths of the inwardly and outwardly directed gain controls associated with the target side of the display. Thus, if the neural lightness computation model is correct, the differences between the observers and matching instructions represented by the purple circles in Figure 19 might result from variations in contrast gain control strengths across observers and instructions. However, the possibility cannot be ruled out that variations between observers and conditions instead reflect differences in combinations of neural parameters that together vary in such a way that the intercept of the slope versus curvature plot is held constant. 

The slope–curvature pair for at least one of the two observers in the two-spotlight condition is definitely shifted downward from the line containing the one-spotlight and naive matching data. If all of the data are to be accounted for by a single theory, then this downward shift must be caused by a variation of one or more of the other model parameters. In the previous section, I proposed that changes in the lightness matching instructions cause the observer to change the attentional gain applied to the outer edge of the surround ring at Layer 1 of the model (see Figure 16d). Since the prescription for ideal matching behavior in the two-spotlight condition is to match the disk–ring ratios on the two sides of the display, all of the weight in the lightness computation should ideally be given to the inner edge. Thus, the observer in the two-spotlight condition ideally ought to completely damp down the weight associated with the outer ring edge. In principle, this might be accomplished either by changing the gain applied to that edge at Layer 1 or by altering the shape of the AC neuronal receptive fields in Layer 3. 

To verify that changing the gain applied to the outer edge in Layer 1 is sufficient to account for the changes in the shape of the matching plots shown in Figure 12 when the matching instructions are changed, I fit the data from each of the two observers in the one-spotlight and two-spotlight lightness matching conditions by assuming that the only model parameter that changes across the two instructional conditions is the attentional gain applied to the outer ring edge in Layer 1. In the one-spotlight lightness matching condition, I assumed that both observers set the neural gains applied to the inner and outer edges of the surround ring to be equal. In the absence of both contrast gain control and the spatial falloff in the AC receptive field weighting function in Layer 3, this choice of Layer 1 gain settings would produce a luminance match, which is ideal behavior in the one-spotlight condition. To model the results of the two-spotlight condition, I assumed that the gain applied to the outer ring edge in Layer 1 was reduced relative to the gain applied to that edge in the one-spotlight condition. This reduction in the attentional gain applied to the outer ring edge reduces the strength of the gain control directed from the outer edge to the inner edge in the pathway from Layer 1 to Layer 2. The resulting gain change in Layer 2 in turn reduces the curvature of the theoretical matching plots associated with the two-spotlight condition relative to those associated with the one-spotlight condition, while simultaneously making the slope of the matching function steeper and more negative. The effect of changing the Layer 1 gain on the slope–curvature pairs for “simulated” two-spotlight matches is illustrated in Figure 21. Details of the procedure used to compute the simulated matches are given in 2. It is not possible to simulate two-spotlight matching plots having negative curvature. However, the 95% confidence intervals of the curvature estimates for the least-squares parabolic models of the actual matching plots from this matching condition include positive values for both observers. Thus, a mechanism based on attentional modulation of the neural gain in Layer 1 is shown to be sufficient to account for all aspects of the instructional effects seen in Experiment 2, down to quantitative details, within measurement error.  

Figure 21

 

Changing the attentional gain applied to the outer ring edge in Layer 1 to model the changes in the shape of the matching plot that occur with changes in the lightness matching instructions. Reducing the gain decreases both the slope and curvature of the theoretical matching plots for two-spotlight matches (orange diamonds) relative to those obtained in the one-spotlight matching conditions (gray arrows). The theoretical slope–curvature pairs are statistically indistinguishable from the actual results obtained for two-spotlight matches. Details of the model fitting procedure are given in 2.

I also explored the idea that the effects of instructions in Experiment 2 might be due to an alternative neural mechanism based on a change in the AC neuronal receptive field size in Layer 3. I was not able to find a set of Layer 3 receptive field weights that could satisfactorily mimic the effects of instructional changes, but there are enough free parameters in the model that I cannot claim to have definitively ruled out such a mechanism. Further work will be required to thoroughly explore this alternative model. 

The fact that the data from the one-spotlight condition falls on the same predicted line in Figure 18 as the data from Rudd and Zemach's (2005) naive matching experiment suggests that the observers in both of these instructional conditions were in some sense “doing the same thing.” Since I was able to fit the one-spotlight data on the assumption that the attentional gain factors applied to the inner and outer edges of the surround were equal and that the spatial falloff in the AC neuronal receptive field was very shallow (i.e., ω ₂ = 0.98ω ₁), it appears that the observers in the two studies set the weights on the two ring edges to be approximately equal in both of the one-spotlight and naive matching experiments. Setting the inner and outer edge weights to be equal is ideal behavior for both one-spotlight lightness matching and brightness matching, so the modeling results suggest that the observers in both of these conditions interpreted disk and rings as papers lit by a single global illuminant or, alternatively, that they matched the stimuli on brightness, rather than lightness. 

The similarity between the matching plots obtained in the two-spotlight conditions and in Rudd and Zemach's (2004) naive matching study using Wallach stimuli suggests that it is more natural to match Wallach stimuli on contrast or—as Wallach himself suggested—to assume that in the case of Wallach stimuli the two sides of the display are interpreted as reflecting surfaces lit by separate illuminants. The difference between the “luminance-like” matches obtained in naive matching studies with double-increment stimuli and “contrast-like” matches obtained in naive matching studies with Wallach stimuli has been interpreted as evidence of highest luminance anchoring (e.g., Gilchrist et al., 1999). However, highest luminance anchoring rule cannot explain other salient and reproducible quantitative aspects of the data, including the parabolic curvature seen in most matching plots, the dependence of the matching plot slope and curvature on the surround size, effects of surround luminance on matches made with double-increment stimuli, and the effects of matching instructions. Thus, there are several reasons to reject the highest luminance anchoring interpretation of the data in favor of the neural edge integration model proposed here. 

According to the model, errors in lightness constancy—that is, deviations from either ideal one-spotlight or ideal two-spotlight matching behavior—result from a combination of factors: namely, the contrast gain control occurring in the feedforward pathway from Layer 1 to Layer 2, the spatial falloff in AC neuronal receptive field weights in Layer 3, and the observers' tendency not to completely damp down the weight given to the outer edge in the two-spotlight condition. In fitting the model to the data from the one- and two-spotlight conditions, the size of the AC neuronal receptive field was assumed to be very large (about 35 deg); therefore, the spatial falloff was quite shallow. Errors in constancy were therefore produced mainly by the combination of contrast gain control and nonoptimal Layer 1 attentional gain settings. The attentional gain settings in Layer 1, through their influence on the strength and spatial spread of the contrast gain control, affect both the matching plot slope and its curvature, which are constrained to covary by the combination of neural mechanisms proposed in the model. It is this common influence of the attentional gain on slope and curvature that produces the empirical law relating the matching plot slope and curvature verified in Figures 19 and 20. 

Although it is possible to account for the pattern of lightness constancy errors with a few simple neural mechanisms, the functional interpretation of the errors remains unclear. Thus the reasons for the existence of both the contrast gain control mechanism and the seemingly nonoptimal attentional gain settings required to fit the data are also not clear and will require further study. 

Relationship to other lightness computation models

Edge integration models

The proposed lightness computation model is a close relative of the Retinex theory of Land and McCann (1971; Land, 1977, 1983, 1986a, 1986b), a theory that was originally proposed as a biologically plausible mechanism for achieving lightness and color constancy under the challenge of variable global illumination. Consider the “Mondrian” stimulus in Figure 22. A Mondrian consists of a random 2D array of patches, each having homogeneous reflectance and separated by sharp edges. To compute the lightness of any given patch, Retinex first computes the local luminance ratios at all of the edges within the image. The luminance ratio associated with any two image patches can then be computed by chain multiplying the local luminance ratios encountered along a path connecting the two patches. Under the assumption that the patches are papers viewed under a single global illuminant, the luminance ratio will be the same thing as the reflectance ratio. 

Figure 22

 

How Retinex works. The input image consists of a Mondrian pattern, comprising an arbitrary arrangement of surfaces, each of which is homogeneous in reflectance, separated by sharp luminance borders and lit by a single global illuminant. To compute the relative lightnesses of any pair of patches within the Mondrian, Retinex computes the local luminance ratio at each luminance border in the image then chain multiplies the luminance ratios encountered along an arbitrary path connecting the two patches. Equivalently, the image could be log-compressed and spatially filtered to encode the local luminance steps at edges then the outputs of the edge detectors (e.g., V1 receptive fields) could be linearly summed across space. To compute the absolute lightness of the patches, an anchoring rule must be applied to the output of the relative lightness (i.e., edge integration) computation. One oft-cited and psychophysically motivated anchoring rule assigns the value “white” (i.e., a perceived reflectance of 90%) to the image region having the greatest lightness. The perceived reflectances of the other patches are then determined by the luminance ratios of those patches with respect to the white point.

To compute the perceived reflectance of any given Mondrian patch, an additional “anchoring” rule must be invoked. The anchoring rule maps the scale of relative reflectances onto a scale of perceived lightness values. Perceptual experiments suggest that the human visual system accomplishes this by assigning the value “white” (e.g., 90% reflectance) to the patch or patches with the maximum reflectance (Bruno, Bernardis, & Schirillo, 1997; Gilchrist, 2006; Gilchrist & Radonjić, 2009; Gilchrist et al., 1999; Li & Gilchrist, 1999; Schirillo & Shevell, 1996). The rest of the perceived gray levels are then uniquely determined by their relative reflectances with respect to the white point. Although the anchoring principle just described is often referred to as highest luminance anchoring, it should be clear from the forgoing discussion that—in the context of Retinex theory at least—it is technically not the highest luminance that is mapped to white but instead the highest reflectance. Under some—but not all—conditions—the highest reflectance and the highest luminance are equivalent (see Rudd & Zemach, 2005 for further details). 

From a mathematical perspective, multiplying ratios is equivalent to adding the logarithms of the ratios; and the logarithm of a luminance ratio at an edge is the same as the luminance step across the edge in log units. Retinex can therefore be instantiated by a lightness computation algorithm that sums steps of log luminance at edges, as in the current model. What differentiates Retinex from other edge integration algorithms—including the model proposed here—is that Retinex assigns equal weights to all edges in the scene; whereas, in subsequently proposed algorithms the weights are assumed to fall off with distance from the patch whose reflectance is being computed (Gilchrist, 1988; Reid & Shapley, 1988; Rudd, 2001, 2003a, 2003b, 2007; Rudd & Arrington, 2001; Rudd & Popa, 2007; Rudd & Zemach, 2002a, 2002b, 2004, 2005, 2007; Shapley & Reid, 1985). Because of its “equal weights for all edges” assumption, the Retinex model exhibits a property that Gilchrist et al. (1999) referred to as Type II lightness constancy: that is, constancy with respect to changes in the reflectances of other surfaces in the target's surround. Altering this assumption leads to an edge integration model in which the surround influences the target lightness, which of course is required in order to model contrast and assimilation. 

The proposed theory goes much farther than previously proposed edge integration models by assuming that edge weights can be adjusted strategically according to the observer's interpretation of the visual scene. This point is perhaps best made with reference to a more natural image that includes illumination gradients in addition to sharp reflectance edges: Adelson's well known checker-shadow illusion (Figure 23). Here, checks A and B have identical luminances, but they differ in perceived lightness. Why? One possibility is that in the process of integrating edge steps along the path from A to B, the luminance “step” associated with the slow luminance gradient corresponding to the shadow edge is discounted (i.e., left out of the edge integration computation). If the gradient is discounted entirely, the algorithm will achieve Type I lightness constancy (constancy with respect to changes in the illuminant; Gilchrist et al., 1999). Alternatively, the shadow edge might be partially integrated in the relative reflectance computation, which would result in a partial failure of Type I constancy. In either case, the illusion would be explained at least qualitatively. von Helmholtz (1866/1924) suggested that discounting the illuminant results from a process of “unconscious inference.” In the case of the checker-shadow illusion, the “unconscious inference” might be achieved mechanistically by a biological strategy that excludes gradient edges from the edge integration computation. 

Figure 23

 

The checker-shadow illusion. Regions A and B have the same luminance but they appear very different in lightness because of the presence of strong pictorial illumination cues in the spatial surround. Copyright 1995, Edward H. Adelson. Used by permission. For more information, see http://persci.mit.edu/gallery/checkershadow.

The results of Experiment 2 demonstrate that observers can judge the lightness of double-increment DAR stimuli in at least two ways: either by assuming that changes in surround ring luminance result from illumination changes on the target side of the display (one-spotlight assumption) or from target ring reflectance changes (two-spotlight assumption). If a luminance change is ascribed to illumination variation, then the luminance step at the outer ring edge should be discounted from the disk reflectance computation for the same reason that the shadow gradient should be left out of the computation of relative reflectance in the checker-shadow illusion: because the luminance ratio at that edge does not reliably represent a relative reflectance ratio. The border between the disk and ring, on the other hand, should be included in the edge integration computation because it does signal a reflectance difference between the disk and its immediate surround. This interpretation of two-spotlight lightness matching is consistent with ”lightness constancy” interpretation given by Wallach (1948, 1963, 1976) to account for the ratio matches that his observers produced with the DAR stimuli referred to here as Wallach stimuli. In the two-spotlight condition of Experiment 2, neither observer was able to entirely discount the luminance step information associated with the outer ring edge, despite the fact that they were (implicitly) instructed to interpret this edge as an illumination edge. The latter result suggests the existence of an inherent bias to interpret sharp edges as reflectance edges and to spatially integrate them with the target edge when they lie in sufficiently close proximity to the target. 

If the DAR display is instead interpreted as papers appearing within a single large spotlight (one-spotlight condition), then all of the edges in the display should be treated as reflectance edges. In that case, the outer edges or the rings should be included in the edge integration computation and all edges should be given equal weights, as in Retinex theory. Consistent with this idea, the results from the one-spotlight condition indicate that observers do tend to give the outer edge more weight when they are instructed to interpret this edge as a reflectance edge. 

In light of these findings, the results of earlier experiments using naive observers but the same stimulus (Rudd & Zemach, 2005) could be interpreted to mean that different naive observers tend to partially integrate the outer edge to different degrees when computing the disk reflectance and that the outer edge is more likely to be discounted when it is farther from the disk (Rudd & Zemach, 2004). In lieu of instructions indicating whether the outer edge represents a spatial discontinuity in illumination or in reflectance, observers tend toward a compromise interpretation, with some observers making matches that are more consistent with the single spotlight interpretation than others. 

In summary, while the results presented here do not prove that the lightness computation mechanism embodied in the human visual cortex involves edge integration per se, they do show that any account of lightness computation based solely on edge integration must necessarily include a mechanism for strategically varying the edge weights. 

Contrast gain control models

Several previously proposed lightness computation models invoke contrast gain control of one type or another. Bindman and Chubb (2004a, 2004b) proposed an account of their Bull's-eye illusion based on suppressive lateral gain control interactions between local edge responses. In the multi-scale filtering Oriented Difference-of-Gaussians (ODOG) model (Blakeslee & McCourt, 1999, 2001, 2003, 2004, 2005; Blakeslee, Pasieka, & McCourt, 2005), contrast normalization serves to equate the root-mean-square filter outputs across six orientation channels. In the multi-filter model of Dakin and Bex (2003), filter outputs are normalized across spatial scales rather than orientations. Barkan, Spitzer, and Einav (2008) proposed a multi-filter edge integration model in which the output of each filter is divided by the sum of the filter outputs across both spatial frequency and space. 

The present model differs from all of these previously proposed models in important ways. First, in all of these previously proposed models, the gain control interactions that occur between channels are either strictly attenuating (Barkan et al., 2008; Bindman & Chubb, 2004a, 2004b) or else strictly amplifying (Blakeslee & McCourt, 1999, 2001, 2003, 2004, 2005; Blakeslee et al., 2005; Dakin & Bex, 2003). A model is strictly attenuating if an increase in the response of one channel has the effect of decreasing the gain applied to the other channels. A model is strictly amplifying if an increase in the response of one channel has the effect of boosting the gain applied to the other channels. Amplification is the rule in the multi-filter contrast normalization models because any increase in the response of any given filter must boost the relative response of the other filters in order to equate the filter responses. 

The gain control in the present model, on the other hand, is both attenuating and amplifying. An increase in the luminance step at a given edge can either increase or decrease the gain applied to the other edge. Attenuating and amplifying gain modulations must both be included in the model in order to account for the fact that the matching plot curvature can be either positive or negative, depending on the observer and instructions, for the same physical stimulus. Furthermore, recent quantitative modeling of single-cell recording data in cortical areas V1 and V2 supports the idea that both attenuating and amplifying gain control interactions exist in vivo (Vladusich et al., 2006b). 

Second, the filters in all previous multi-filter models encode Michelson contrast, whereas the filters in the present model encode luminance steps in log units. There is some controversy in the literature regarding which of these two quantitative assumptions best describes human vision. As discussed above in the context of Retinex theory, summing steps in log units is the correct thing to do if the goal is to compute relative reflectance. Reid and Shapley (1988; Shapley & Reid, 1985) substituted Michelson contrast for steps in log luminance because of neurophysiological evidence indicating that the spike rates of early visual neurons in the retina and cortex are proportional to the former measure (Dean, 1981; Shapley & Enroth-Cugell, 1984; Tolhurst, Movshon, & Dean, 1983). The two measures are equivalent near threshold (Reid & Shapley, 1988) but they can differ substantially far from threshold. 

Rudd and Zemach (2004) showed that an edge integration model based on a weighted sum of log luminance ratios produced a better fit to their lightness matching data than did the model based on Michelson contrast. More recently, Pereverzeva and Murray (2009) published an analysis of lightness matches performed with time-varying DAR stimuli that appeared to support the Michelson contrast model over a model based on log luminance ratios. However, that analysis failed to take into account the effects of spatial context on the match disk used to measure lightness and therefore must be considered inconclusive. No comparison has been made between edge integration models that include a contrast gain control mechanism, although in the case of Rudd and Zemach's data the proportion of variance accounted for by contrast gain control is a only fraction of a percent and therefore is probably negligible. 

The very large proportion of the variance accounted for by the present model would seem unlikely to be captured by the model if Michelson contrast were the relevant measure. For this reason—as well as the fact that summing steps in log luminance across space makes more sense from a functional point of view—I think the assumption that edge responses in human vision are proportional to the log luminance ratios at the edges is more likely to be correct. Moreover, recent modeling of single-unit responses in V1 and V2 has demonstrated that an edge integration model based on log luminance ratio also provides a good fit to spike data (Kinoshita & Komatsu, 2001; Vladusich et al., 2006b). 

A third and final difference between the present model and previously proposed lightness models incorporating contrast gain control is that none of the previous models includes an attentional component. Reynolds and Heeger (2009) proposed a model incorporating both contrast gain control and attention to account for psychophysical and neural data on contrast detection and discrimination. However, their model does not include an edge integration mechanism and it was not intended as a model of surface appearance. Furthermore, like many of the other gain control models discussed above, the gain control in their model is strictly suppressive. The neural responses in the Reynolds and Heeger model also depend on Michelson contrast rather than steps in log luminance. One important similarity between their model and the model proposed here is that both models assume that attentional gain control occurs prior to the neural processing stage at which contrast gain control occurs. 

Lightness filling-in models

A number of visual phenomena—collectively known as lightness and color “filling-in”—have been interpreted as indicating the existence of an underlying neural mechanism that first encodes information about contrast at borders, then uses this border information to reconstruct, or color in, the appearance of regions lying interior to the borders (Cohen & Grossberg, 1984; Craik, 1940, 1966; Davidson & Whiteside, 1971; Fry, 1948; Gerrits & Vendrik, 1970; Gerrits, de Haan, & Vendrik, 1966; Gerrits & Timmermann, 1969; Grossberg & Mingolla, 1985a; Grossberg & Todorović, 1988; Horn, 1974, 1986; Pessoa, Thompson, & Noe, 1998; Walls, 1954). Some well-known filling-in phenomena include: the Craik–O'Brien–Cornsweet effect (Cornsweet, 1970; Craik, 1940, 1966; O'Brien, 1958); perceptual filling-in across the blind spot and retina scotomas (Gerrits & Timmermann, 1969); the perceived filling-in of the surround when a target is made to disappear through retinal stabilization (Gerrits et al., 1966; Krauskopf, 1963; Yarbus, 1967); the Watercolor illusion (Pinna, Brelstaff, & Spillmann, 2001); and neon color spreading (Redies & Spillmann, 1981; Van Tuijl, 1975; Varin, 1971). 

A number of neural models have been proposed to account for these filling-in phenomena. Probably the best known of these is the FACADE theory of Grossberg and his colleagues (Cohen & Grossberg, 1984; Grossberg & Mingolla, 1985a, 1985b; Grossberg, Mingolla, & Todorović, 1989; Grossberg & Todorović, 1988). FACADE theory accounts for filling-in by proposing that color signals diffuse dynamically from the locations of contrast borders in a cortical map of the visual environment to fill in the regions of the map lying interior to neurons that encode and delineate surface boundaries. The spread of the diffusing color signals is blocked by the activities of the boundary-encoding neurons (Fang & Grossberg, 2009; Grossberg & Mingolla, 1985a, 1985b; Grossberg et al., 1989; Grossberg & Todorović, 1988). 

Because FACADE theory assumes that the diffusing color “filling-in” signals are stopped at the locations of borders, the theory has no mechanism by which spatially oriented luminance steps can be perceptually integrated across space. In the edge integration model proposed here, filling-in of achromatic color occurs as a result of the spatial integration of polarity-specific edge information by the receptive fields of the AC neurons in Layer 3. Thus, the proposed model is able to explain the perceptual spread of achromatic colors from borders while simultaneously accounting for the fact that luminance steps from multiple borders are spatially integrated. The edge integration model predicts that achromatic color will radiate perceptually from borders simply because AC neurons whose receptive field centers lie within some neighborhood of a border will be activated by the border presence. The radiating color will either decay spatially as a function distance from the border, or else fill in homogeneously between borders, depending on the contrast polarities and proximities of other borders within the input image. 

Figure 24 illustrates the mechanisms responsible for these model properties. When an AC neuron is activated by a single border of appropriate contrast polarity lying to one side of the neuron's receptive field center, the degree to which that border will activate the neuron will depend both on the distance between the receptive field center and the border, and on the rate of falloff in the neuron's receptive field profile. The amount of lightness or darkness induced by the border at nearby locations will decay as a function of distance from the border. 

Figure 24

 

Lightness and darkness fill in uniformly within the interiors of sufficiently small target patches. If edges with opposite contrast polarities are presented to the receptive field of a single AC neuron (illustrated here as a lightness neuron), the neuron's firing rate will be independent of the patch placement within the neuron's receptive field. This is a consequence of the linear spatial falloff of the receptive field weighting function. If the left edge of the patch shown in (a) is moved closer to the receptive field center, as in (b), the contribution of the edge to the firing rate will increase. However, this increase will be exactly compensated for by the concomitant effect of moving the other patch edge away from the receptive field center on the other side of the receptive field. An identical effect can be achieved by keeping the location of the stimulus patch fixed while translating the receptive field relative to the patch. Thus, AC neurons whose receptive field centers lie between the locations of the patch edges will fire at equal rates to the patch. For patches whose width is larger than the width of the AC neuron's receptive field, only one edge will excite a given AC neuron and the distance between the edge and the receptive field center will determine the neuronal firing rate. Neurons whose receptive field centers are near the edge will fire at a greater rate than neurons whose receptive field centers are farther from the edge. Thus, when an image region is larger than the characteristic size of the AC neuronal receptive field, lightness and darkness induction strengths will decay with distance from the borders both inside and outside the bounded region.

When two borders that lie on opposite sides of an AC neuron's receptive field center simultaneously stimulate the neuron, their relative contributions to the neuron's firing rate will depend on their relative distances from the neuron's receptive field center. Since the AC neuronal receptive field profile falls off linearly with distance, any common translation of the two borders that keeps both borders within the receptive field will preserve the neuron's firing rate. Alternatively, the firing rate will be preserved if the borders are fixed and the receptive field moves. It follows that the firing rates of the color-encoding neurons will be identical for all receptive field positions lying between the borders, as long as the characteristic receptive field size of the color neurons is large enough to contain both borders. Thus, the model predicts that lightness will fill in homogeneously between two borders having opposite contrast polarities that lie sufficiently close together in the image. 

However, the edge integration model, as described, cannot account for all of the phenomenal properties of lightness filling-in. Consider the Kanizsa triangle figure shown in Figure 25 (Kanizsa, 1955, 1979). Most observers, when shown this figure, report seeing a bright illusory triangle having sharply defined edges that appears to stand out in front of three partially occluded disks and a partially occluded black triangular outline. Many observers also report that the illusory square appears whiter or brighter than background, despite the fact that most of the illusory edge is not supported by a real edge having signed contrast in the image. 

Figure 25

 

Kanizsa triangle figure. The three spatially aligned Pac-Man inducers generate a percept of a white occluding triangle seen in front of three black disks. Most observers perceive the edges of the illusory occluding triangle even in regions of the image that lie between the Pac-Men, where no actual contrastive edges exist in the physical image.

Grossberg and his colleagues have modeled a variant of this illusion, the Kanizsa square, on the hypothesis that a neural “Boundary Contour System” (BCS) synthesizes the illusory edges of the occluding figure and a separate “Feature Contour System” (FCS) fills in the surface color seen within the figure's borders (Cohen & Grossberg, 1984; Grossberg & Mingolla, 1985a). The FCS/BCS distinction is retained in the current version of FACADE theory (Fang & Grossberg, 2009). A key claim of the theory is that the activity of the BCS is itself invisible and its effects are only made visible by the fact that they delimit the visible colors filled in by the FCS. To synthesize the illusory contours of the Kanizsa figure, contrast-polarity-insensitive BCS neurons having receptive fields lying along the path of the illusory contours—and aligned with them—are assumed to be activated by a process of recruitment within a cortical neural network. This recruitment happens only when the real physical edges belonging to Pac-Man inducers are present and in proper spatial alignment with one another. 

In the edge integration model proposed here, there are no separate boundary contour and feature contour systems. Furthermore, no contrast-polarity-insensitive edge encoding neurons are assumed. To account for the perceptual emergence of the occluding Kanizsa triangle or square, some additional mechanism beyond the edge integration model presented above is needed to generate illusory borders. One possibility is that properly phase-aligned Pac-Man inducers produce a cooperative activation along the path of the illusory contour of edge detector neurons whose receptive fields are tuned to the same contrast polarity as the inducer edges. This edge detector activation would not only create the seen border but also produce light and dark filling-in percepts on either side of the border, which would explain why the square looks lighter than the background. Although further work would be required to flesh out this model in detail, this description should suffice to make the basic point that filling-in does not necessarily have to be produced by a combination of separate BCS and FCS activations nor by a diffusive filling-in process that stops at figural borders. 

Another line of evidence that has sometimes been interpreted as supporting diffusive filling-in theories is evidence that filling-in takes time (Arrington, 1994; Huang & Paradiso, 2008; Hung, Ramsden, & Roe, 2007; Paradiso & Hahn, 1996; Paradiso & Nakayama, 1991; Rossi & Paradiso, 1996). However, this claim has recently been challenged in the literature by contrary evidence that filling-in occurs either instantaneously on near-instantaneously (Blakeslee & McCourt, 2008; Robinson & de Sa, 2008). The latter results would appear to be more consistent with the edge integration model proposed here. But even if filling-in were clearly demonstrated to take time to fill in from borders, the time-dependent filling-in might potentially be accounted for by neural delays in axonal and dendritic conduction times in the feedforward projections from edge detector neurons to color integrator neurons, and thus be consistent with the model. For this reason, it may prove difficult to distinguish the two types of models—edge integration versus diffusive filling-in—on the basis of the temporal properties of filling-in alone. 

Summary and conclusions

It has been shown here that a neural model that includes edge integration, contrast gain control, and top-down attentional gain control can account for the results of several appearance matching experiments performed with disk-and-ring stimuli. The model accounts for the psychophysical data with a high degree of quantitative accuracy. It does a better accounting for the overall pattern of results than do several previously proposed quantitative models, including: Wallach's ratio rule; Retinex theory; Reid and Shapley's edge integration model based on a weighted sum of Michelson contrasts; and a simple lightness anchoring theory in which the target lightness is determined by the ratio of its luminance with respect to the highest luminance in the scene. Unlike these earlier models, the proposed model accounts for both contrast and assimilation, as well as for the effects of instructions on the overall pattern of spatial induction effects. 

An important question for future research is whether the proposed model can be fruitfully extended to deal with more complex images: that is, to images not comprising homogeneous patches demarcated by sharp luminance borders and lying in a single depth plane. In previous papers, I have discussed ways in which edge integration models could be generalized to deal with images having blurred edges at arbitrary spatial scales: in other words, to account for lightness percepts in arbitrary two-dimensional grayscale images (Rudd & Popa, 2007; Rudd & Zemach, 2004, 2005, 2007; Zemach & Rudd, 2007). Spatially oriented steps in log luminance could be computed at a range of spatial scales by a collection of two-dimensional oriented band-pass spatial filters, such as the Gabor functions that have been used extensively to model simple cell receptive fields in cortical area V1 (e.g., Daugman, 1985; Jones & Palmer, 1987a, 1987b; Marcelja, 1980; Ringach, 2002) operating on a logarithmically compressed neural image of the stimulus luminance profile (Cornsweet, 1970). To compute lightness in three-dimensional scenes, depth cues such as stereopsis could be used as control signals for determining whether or not edges will be spatially integrated. 

The issue of when and by how much one should weight an edge in any particular lightness computation—or a generalized edge, in a multi-filter version of the model—is a key issue for future modeling. In discussing Adelson's checker-shadow illusion, I have suggested that low-frequency edges (i.e., luminance gradients) might be given low weights or even excluded from edge integration altogether because they are unlikely to represent reflectance edges in the real world. However, the brain apparently does not determine edge weights by applying edge classification rules alone. As has been shown here, even to model the results of experiments conducted with simple stimuli such as DAR patterns, it is necessary to assume that the edge weighting process has a top-down component: that edge integration is both flexible and task-dependent. In other words, the process of deciding when and when not to treat luminance edges as reflectance edges, as opposed to illumination edges, has a strategic component that is at least to some degree under attentional control. 

Perhaps the most important property distinguishing the proposed model from most other models of visual appearance is that it does not compute a fixed neural image of the world that can be further processed by attention. Instead, the neural image itself depends on the visual measurement that the observer intends to make. In other words, the visual system can “morph” into a range of different visual measuring devices. An implication of this idea is that there is no such thing as “low-level” vision. 

From previous research, it is known that lightness can be influenced by many configural image properties that have not been considered here. These include, but are not limited to, gestalt properties such as figural similarity (Bressan, 2001, 2006a, 2006b; Bressan & Kramer, 2008), coplanarity (Gilchrist & Radonjić, 2006, 2010), belongingness (Gilchrist, 2006; Gilchrist et al., 1999; Gilchrist & Radonjić, 2010), border ownership (Craft, Schutze, Niebur, & von der Heydt, 2007; von der Heydt, Zhou, & Friedman, 2003; Zhou, Friedman, & von der Heydt, 2000), and cues to scission in depth (Anderson & Winawer, 2008). To account for the dependence of lightness on such configural properties, additional mechanisms would obviously need to be added to the model. These would likely take the form of mechanisms to control the edge weights on the basis of a more sophisticated analysis of the spatial context than the one proposed in the current version of the model. Such mechanisms could be expected to exploit a combination of hardwired rules—for example, AC neuronal receptive fields might preferentially combine the outputs of edge detectors tuned to the same orientation and disparity—and more complex network interactions, possibly involving feedback from higher visual centers capable of analyzing the visual scene for meaning. 

It should be clear from these remarks that the edge integration model presented here is not intended to be a complete theory of lightness computation. In fact, the mechanisms included in the current version of the model are likely to comprise only a small piece of the cortical machinery required to compute lightness. The model presented here should thus be viewed, more properly, as a skeletal framework on which to hang a more complete neural theory of lightness perception. 

An important model property—and one that would generalize to the sorts of possible model extensions just outlined—is that lightness computations must be carried out in a certain sequence in order for the model to be able to account for the matching data. The influence of attention on the edge weights must come first, followed by further adjustments of the weights in light of the wider spatial image context. Spatial edge integration to compute color must come last, after the edge weights have been adjusted on the basis of the task and the spatial context. Any additional mechanisms required to implement computations related to figural organization, such as grouping effects, would exert their influence at a processing stage that is intermediate between the “early” attentional control of edge weights and the “late” edge integration stage. If the effects of grouping were realized by feedback control of edge weights by higher order visual areas subserving visual pattern recognition, this feedback would likely occur “offline”: that is, off of the direct pathway leading from V1 to the visual area or areas that constitute the direct neural correlate of visual appearance. An example of a neural mechanism that could work in this way is the one proposed by Craft et al. (2007) to account for the sensitivity of V2 neurons to border ownership. 

This inferred order of the lightness computations suggests that visual cortex is performing something akin to a Bayesian analysis, in which edge information is first weighted on the basis of a priori assumptions about which local image contrasts are relevant to the lightness computation task at hand. The a priori weights are then further adjusted in light of the sensory evidence contained in the larger image array. Neurophysiological evidence suggests that the neurons responding to local edge contrast are likely to be located early in the cortical processing stream: in areas V1 and/or V2 (Hubel & Wiesel, 1959, 1962, 1965, 1968; Peterhans et al., 1986). A likely cortical locus of the edge integration process is area V4, an area that has been implicated in the maintenance of color constancy (Bartels & Zeki, 2000; Clarke et al., 1998; Kennard et al., 1995; Kentridge et al., 2004; Smithson, 2005; Walsh, 1999; Zeki et al., 1999; Zeki & Marini, 1998). If edge integration took place in V4, contrast gain control and any additional spatial context effects such as grouping would necessarily occur along the pathway from area V1 to V4. 

Appendix A

Formal specification of the neural lightness model

The lightness of either the match or target disk in the DAR stimulus is modeled as 

${\Phi }_k=\mathrm{A}{\mathrm{C}}_k^{+}\left(0\right)-\mathrm{A}{\mathrm{C}}_k^{-}\left(0\right),$

(A1)

where AC _k ⁺(0) and AC _k ⁻(0) denote the activities of the lightness and darkness neurons in Layer 3, respectively, whose receptive fields are located at origin, which is taken to be the center of the disk on the kth side of the display, k = M, T. 

The lightness and darkness neurons each perform a weighted summation of the outputs of edge detector neurons in Layer 2. Lightness neurons sum only the responses of Layer 2 neurons whose receptive field structure and half-wave rectifying properties cause them to fire in response to stimulus edges whose light sides point inward toward the origin. This fact is denoted by the following equation: 

$\mathrm{A}{\mathrm{C}}_k^{+}\left(0\right)={\omega }_{1k}^{+}E{2}_{1k}^{+}+{\omega }_{2k}^{+}E{2}_{2k}^{+},$

(A2a)

where E2_1k ⁺ and E2_2k ⁺ represent the activities of the Layer 2 edge detectors located at the positions of the inner and outer edges—edges 1 and 2—of the surround ring; and ω _1k ⁺ and ω _2k ⁺ represent the weights given to those activities by the lightness neuron centered at the origin. Similarly, darkness neurons sum only the responses of Layer 2 neurons that fire in response to edges whose dark sides point inward: 

$\mathrm{A}{\mathrm{C}}^{-}\left(0\right)={\omega }_{1k}^{-}E{2}_{1k}^{-}+{\omega }_{2k}^{-}E{2}_{2k}^{-}.$

(A2b)

 

The AC weights applied to E2 neural responses to edges of contrast polarity ρ = +, − are assumed to fall off linearly with distance from the AC neuron's receptive field center. Thus 

${\omega }_{1k}^{\rho }=A^{\rho }-B^{\rho }{d}_k,$

(A3a)

and 

${\omega }_{2k}^{\rho }=A^{\rho }-B^{\rho }(d_k+r_k),$

(A3b)

where d _k and r _k are the disk and ring radii on the k ^th side of the display. 

The response of a Layer 2 neuron having contrast polarity sensitivity ρ and located at the position n = 1, 2 is given by the products of the Layer 1 neuron having the same polarity sensitivity and located at the same position, and the Layer 2 neuronal gain factor: 

$E{2}_{nk}^{\rho }=g{2}_{nk}^{\rho }E{1}_{nk}^{\rho }.$

(A4)

 

The gain factor of the Layer 2 neuron that responds to an edge n of contrast polarity ρ is initially set to the value

$g{2}_{nk}^{*{\rho }_n}$

. This initial gain factor is adjusted either upward or downward by the presence of Layer 1 activity in the vicinity of edge n. The magnitude of the adjustment is assumed to fall off as a linear function of distance such that 

$g{2}_{1k}^{{\rho }_1}=g{2}_{1k}^{{\rho }_1^{*}}{[1+{\delta }_{21}^{{\rho }_2{\rho }_1}{[{\alpha }_k^{{\rho }_2{\rho }_1}E{1}_{2k}^{{\rho }_2}-{\nu }^{{\rho }_2{\rho }_1}{r}_k]}^{+}]}^{+},$

(A5a)

and 

$g{2}_{2k}^{{\rho }_2}=g{2}_{2k}^{{\rho }_2^{*}}{[1+{\delta }_{12}^{{\rho }_1{\rho }_2}{[{\beta }_k^{{\rho }_1{\rho }_2}E{1}_{1k}^{{\rho }_1}-{\lambda }^{{\rho }_1{\rho }_2}{r}_k]}^{+}]}^{+},$

(A5b)

where

${\delta }_{21}^{{\rho }_2{\rho }_1}$

,

${\alpha }^{{\rho }_2{\rho }_1}$

, and

${\nu }^{{\rho }_2{\rho }_1}$

denote the sign (+ or −), magnitude, and drop-off rate of the gain control communicated through the feedforward connection from the Layer 1 neuron responding to an edge 2 of contrast polarity ρ ₂ to the Layer 2 neuron responding on an edge 1 having contrast polarity ρ ₁; and similarly,

${\delta }_{12}^{{\rho }_1{\rho }_2}$

,

${\beta }^{{\rho }_1{\rho }_2}$

, and

${\lambda }^{{\rho }_1{\rho }_2}$

denote the sign, magnitude, and spatial drop-off rate of the gain control communicated through the feedforward connection from the Layer 1 neuron responding to an edge 1 of contrast polarity ρ ₁ to the Layer 2 neuron responding on an edge 2 of contrast polarity ρ ₂. 

The responses of the Layer 1 neurons at the inner and outer ring edges are proportional to the luminance steps in log units at those edges: 

$E{1}_{1k}^{+}=g{1}_{1k}^{+}[D-R{]}^{+},$

(A6a)

 

$E{1}_{1k}^{-}=g{1}_{1k}^{-}[R-D{]}^{+},$

(A6b)

 

$E{1}_{2k}^{+}=g{1}_{2k}^{+}[R-B{]}^{+},$

(A6c)

 

$E{1}_{2k}^{-}=g{1}_{2k}^{-}[B-R{]}^{+}.$

(A6d)

Because of the presence of the half-wave rectification operators in Equations A6a–A6d, only either Equation A6a or A6b can be nonzero for a given inner edge contrast polarity. Similarly, only either Equation A6c or A6d can be nonzero. It follows that the Equations A6a–A6d can be expressed in following more compact notation: 

$E{1}_{1k}^{{\rho }_1}=g{1}_{1k}^{{\rho }_1}{\rho }_1(D-R),$

(A7a)

 

$E{1}_{2k}^{{\rho }_2}=g{1}_{2k}^{{\rho }_2}{\rho }_2(R-B).$

(A7b)

 

By combining the above formulas, we arrive at the following expression for the neural response to the disk located on the kth side of the display: 

$\begin{array}{ccc}{\Phi }_k& =& (a-b{d}_k)g{1}_{1k}{[1+{\delta }_{21}{[{\alpha }_{k}g{1}_{2k}{\rho }_2(R_k-B)-\nu r_k]}^{+}]}^{+}{\rho }_1(D_k-R_k)\hfill \\ \\ & & +(a-b(d_k+r_k))g{1}_{2k}{[1+{\delta }_{12}{[{\beta }_{k}g{1}_{1k}{\rho }_1(D_k-R_k)-\lambda r_k]}^{+}]}^{+}{\rho }_2(R_k-B)\end{array},$

(A8)

where it is understood that the model parameters are allowed to vary as a function of the edge contrast polarities; (and so the polarity indices have been dropped); and the a priori gain factors g2_1k * and g2_2k * of the Layer 2 edge detectors have been absorbed into the expressions for the weights ω _1k and ω _2k . 

Appendix B

In this appendix, I show how the lightness computation model defined by the system of equations given in 1 can account for the results of Experiments 1 (naive double-decrement matches) and 2 (double-increment matches, lightness conditions only). In particular, I show how the model can be parameterized so that it produces the parabolic matching functions observed in the two experiments. The model is able to reproduce the fits of the least-squares regression models of the data, including the slopes and curvatures of those models. 

How the solutions were obtained

For a lightness match, the neural responses to the match and target disks should be equal: 

${\Phi }_{\mathrm{M}}={\Phi }_{\mathrm{T}}.$

(B1)

Our goal is to obtain an expression for the model observer's match disk setting D _M in terms of the target ring luminance R _T that satisfies Equations A8 and B1. For present purposes, we can write Equation A8 in the following simplified form: 

$\begin{array}{ccc}{\Phi }_k& =& (a-bd)g{1}_1{[1+{\delta }_{21}{[{\alpha }_{k}g{1}_2{\rho }_2(R_k-B)-\nu r]}^{+}]}^{+}{\rho }_1(D_k-R_k)\hfill \\ \\ & & +(a-b(d+r))g{1}_2{[1+{\delta }_{12}{[{\beta }_{k}g{1}_1{\rho }_1(D_k-R_k)-\lambda r]}^{+}]}^{+}{\rho }_2(R_k-B)\end{array},$

(B2)

where the subscripts on the symbols d _k and r _k denoting the disk and ring radii have been dropped because the disks and rings on the two sides of the displays were the same sizes in Experiments 1 and 2; and it has been assumed that identical Layer 1 gains are applied to target and match sides of the display. 

In order for the contrast gain control in the feedforward connection from Layer 1 to Layer 2 to be effective (i.e., span the distance between the inner and outer ring edges), the ring width must be sufficiently small. For the inwardly directed gain control to be effective, the following inequality must hold: 

$r\le {\nu }^{-1}{\alpha }_{k}g{1}_2{\rho }_2(R_k-B),$

(B3a)

and for the outwardly directed gain control to be effective, the following inequality must hold: 

$r\le {\lambda }^{-1}{\beta }_{k}g{1}_1{\rho }_1(D_k-R_k).$

(B3b)

 

The matching Equation B1 yields the solution 

$D_{\mathrm{M}}=C^{-1}(c_0+c_1{R}_{\mathrm{T}}+c_2{R}_{\mathrm{T}}^2),$

(B4)

where 

$\begin{array}{ccc}{c}_0& =& g{1}_1(1-{\delta }_{21}\nu r)D_{\mathrm{T}}+(g{1}_1(1-{\delta }_{21}\nu r)-g{1}_2\omega (1-{\delta }_{12}\lambda r))R_{\mathrm{M}}\hfill \\ \\ & & -g{1}_{1}g{1}_2(({\rho }_2{\delta }_{21}{\alpha }_{\mathrm{T}}+{\rho }_1{\delta }_{12}\omega {\beta }_{\mathrm{T}})D_{\mathrm{T}}B+({\rho }_2{\delta }_{21}{\alpha }_{\mathrm{M}}+{\rho }_1{\delta }_{12}\omega {\beta }_{\mathrm{M}})R_{\mathrm{M}}(R_{\mathrm{M}}-B))\end{array},$

(B4a)

 

$c_1=-g{1}_1(1-{\delta }_{21}\nu r)+g{1}_2\omega (1-{\delta }_{12}\lambda r)+g{1}_{1}g{1}_2({\rho }_2{\delta }_{21}{\alpha }_{\mathrm{T}}+{\rho }_1{\delta }_{12}\omega {\beta }_{\mathrm{T}})(D_{\mathrm{T}}+B),$

(B4b)

 

$c_2=-g{1}_{1}g{1}_2({\rho }_2{\delta }_{21}{\alpha }_{\mathrm{T}}+{\rho }_1{\delta }_{12}\omega {\beta }_{\mathrm{T}}),$

(B4c)

 

$C=g{1}_1(1-{\delta }_{21}\nu r)+g{1}_{1}g{1}_2({\rho }_2{\delta }_{21}{\alpha }_{\mathrm{M}}+{\rho }_1{\delta }_{12}\omega {\beta }_{\mathrm{M}})(R_{\mathrm{M}}-B),$

(B4d)

 

$\omega =\frac{{\omega }_2}{{\omega }_1}=\frac{1-2(d+r)/\theta }{1-2d/\theta }\mathrm{a}\mathrm{n}\mathrm{d}\theta =\frac{2a}{b}.$

(B4e)

The constant θ is the size of the Layer 3 AC neuronal receptive field. 

Model solutions were obtained through the use of the software package Mathematica 5.2 (Wolfram Research, Cambridge, MA). 

Experiment 1: Double-decrement matches

Stimulus parameters: ρ ₁ = ρ ₂ = −1; D _T = 0; R _M = 0.7; B = 1.4; d = 0.35; r = 0.12 (small ring), 0.583 (medium ring), 1.05 (large ring). Model parameters (both subjects): g1₁ = 1.0; g1₂ = 1.2; δ ₂₁ = δ ₁₂ = −1. Model constraints (to ensure contrast gain control spans the ring width): α _M = α _T = 2.698 × 10⁻² ν/g1₂; β _M = β _T = 2.698 × 10⁻² λ. Subject-specific parameters: HK, θ _M = θ _T = 5.667, α _M = α _T = 0.452116; β _M = β _T = 0.157049; ν = 1.017 × 10⁻¹; λ = 2.945 × 10⁻²; MER, θ _M = θ _T = 4.1333, α _M = α _T = 0.263923; β _M = β _T = 0.332915; ν = 5.278 × 10⁻²; λ = 5.549 × 10⁻². Regression coefficients for model fits (actual coefficients, where different, in parentheses): HK (small ring) c ₀ = −0.6832 (−0.684 ± 0.092), c ₁ = 1.6967 (1.814 ± 0.295), c ₂ = −1.0296 (−1.283 ± 0.207); (medium ring) c ₀ = −0.3161 (−0.307 ± 0.077), c ₁ = 1.0008, c ₂ = −0.7847; (large ring) c ₀ = −0.0095 (−0.120 ± 0.034), c ₁ = 0.4257 (0.520 ± 0.108), c ₂ = −0.5888 (−0.563 ± 0.076); MER (small ring) c ₀ = −0.6832 (−0.865 ± 0.060), c ₁ = 2.0575 (2.181 ± 0.192), c ₂ = −1.3114 (−1.529 ± 0.135); (medium ring) c ₀ = −0.1922 (−0.033 ± 0.032), c ₁ = 0.9152, c ₂ = −0.7847; (large ring) c ₀ = 0.2584 (0.308 ± 0.124), c ₁ = 0.0726 (0.155 ± 0.398), c ₂ = −0.6311 (−0.820 ± 0.280). Notes: Model gets c ₁ and c ₂ exactly for medium ring (by design); g1₂ had to be >1 and values in the range of 1.1–1.2 gave acceptable fits; θ _M = θ _T had to be >2.5 (deg) for HK and >2.0 (deg) for MER. 

Experiment 2: Double-increment matches

Stimulus parameters: ρ ₁ = ρ ₂ = 1; D _T = 0.5; R _M = −0.112; B = −1; d = r = 0.35. Model parameters (both subjects): δ ₂₁ = −1; δ ₁₂ = 1; g1₁ = 1.0. Subject-specific parameters: AH, θ _M = θ _T = 35.7; g1₂ = 1.0 (one-spotlight), 0.166181 (two-spotlight); α _M = 0.266348; β _M = 0.0122513; α _T = 0.278583; β _T = 0.0122513; ν = 0.0892039; λ = 0.0420291; MER, θ _M = θ _T = 35.7; g1₂ = 1.0 (one-spotlight), 0.272765 (two-spotlight); α _M = 0.522321; β _M = 0.0141519; α _T = 0.552814; β _T = 0.0141519; ν = 0.213039; λ = 0.048549. Regression coefficients produced by model fits (actual regression coefficients, where different, in parentheses): AH (one-spotlight) c ₀ = 0.497, c ₁ = 0.084, c ₂ = 0.331; (two-spotlight) c ₀ = 0.402681 (0.466 ± 0.039), c ₁ = −0.854, c ₂ = 0.0445812 (−1.023 ± 0.929); MER (one-spotlight) c ₀ = 0.491, c ₁ = 0.254, c ₂ = 0.865; (two-spotlight) c ₀ = 0.406399 (0.449 ± 0.012), c ₁ = −0.776, c ₂ = 0.154514 (0.033 ± 0.278). 

Acknowledgments

This work was partially supported by a grant from the University of Washington Royalty Research Fund. 

Dr. Davida Teller provided space in her laboratory to perform Experiment 1. Erin Harley, Iris Zemach, and Dorin Popa programmed the experiments. Jonathan An, Amanda Heredia-Montesinos, and Heather (Knapp) Patterson performed data analysis and contributed many hours as psychophysical observers. 

Commercial relationships: none. 

Corresponding author: Michael E. Rudd. 

Email: [e-mail]. 

Address: Howard Hughes Medical Institute, Department of Physiology and Biophysics, Box 357370, University of Washington, Seattle, WA 98195-7370, USA.